SoftmaxLayer
represents a softmax net layer.
SoftmaxLayer[n]
represents a softmax net layer that uses level n as the normalization dimension.
Details and Options
- SoftmaxLayer[…][input] explicitly computes the output for input.
- SoftmaxLayer[…][{input1,input2,…}] explicitly computes outputs for each of the inputi.
- When given a NumericArray as input, the output will be a NumericArray.
- SoftmaxLayer is typically used inside NetChain, NetGraph, etc. to normalize the output of other layers in order to use them as class probabilities for classification tasks.
- SoftmaxLayer can operate on arrays that contain "Varying" dimensions.
- SoftmaxLayer exposes the following ports for use in NetGraph etc.:
-
"Input" a numerical array of dimensions d1×d2×…×dn "Output" a numerical array of dimensions d1×d2×…×dn - When it cannot be inferred from other layers in a larger net, the option "Input"->n can be used to fix the input dimensions of SoftmaxLayer.
- SoftmaxLayer[] is equivalent to SoftmaxLayer[-1].
- SoftmaxLayer effectively normalizes the exponential of the input array, producing vectors that sum to 1. For the default level of -1, the innermost dimension is used as the normalization dimension.
- When SoftmaxLayer[-1] is applied to a vector v, it produces the vector Normalize[Exp[v],Total]. When applied to an array of higher dimension, it is mapped onto level -1.
- When SoftmaxLayer[n] is applied to a k-dimensional input array
, it produces the array 
, where n is the summed-over index of x. - SoftmaxLayer[…,"Input"shape] allows the shape of the input to be specified. Possible forms for shape are:
-
n a vector of size n {d1,d2,…} an array of dimensions d1×d2×… {"Varying",d2,d3,…} an array whose first dimension is variable and remaining dimensions are d2×d3×… - Options[SoftmaxLayer] gives the list of default options to construct the layer. Options[SoftmaxLayer[…]] gives the list of default options to evaluate the layer on some data.
- Information[SoftmaxLayer[…]] gives a report about the layer.
- Information[SoftmaxLayer[…],prop] gives the value of the property prop of SoftmaxLayer[…]. Possible properties are the same as for NetGraph.
Examples
open all close allBasic Examples (2)
Create a SoftmaxLayer:
SoftmaxLayer[]Create a SoftmaxLayer that takes a vector of length 5 as input:
softmax = SoftmaxLayer["Input" -> {5}]Apply the layer to an input vector:
softmax[{0.1, 4.5, -0.2, 3.3, 5.4}]The elements of the result sum to 1:
Total[%]Scope (5)
Create a SoftmaxLayer that takes a matrix of dimensions 3×2 as input:
softmax = SoftmaxLayer["Input" -> {3, 2}]softmax[{{1, 2}, {3, 0}, {4, -1}}]//MatrixFormEach row of the matrix is normalized:
Map[Total, %]Create a SoftmaxLayer that uses the first dimension as the normalization dimension:
softmax = SoftmaxLayer[1, "Input" -> {3, 2}]softmax[{{1, 0}, {1, 0}, {0, 1}}]Create a SoftmaxLayer that normalizes over a variable-length dimension:
softmax = SoftmaxLayer["Input" -> "Varying"]Apply it to sequences of different lengths:
softmax[{1, 1, 1, 2}]softmax[{1, 1, 2}]softmax[{1, 2}]SoftmaxLayer threads over a batch of inputs:
softmax = SoftmaxLayer["Input" -> {2}];
softmax[{{1, 1.4}, {1.6, 3.2}}]Create a SoftmaxLayer that uses a NetDecoder to interpret the output as class probabilities:
dec = NetDecoder[{"Class", {"class1", "class2"}}];
softmax = SoftmaxLayer["Output" -> dec]softmax[{0.4, 0.6}]Interpret the outputs of the softmax layer as probabilities:
softmax[{0.4, 0.6}, "Probabilities"]Properties & Relations (3)
SoftmaxLayer[-1] computes the following:
softmax[x_] := N@Exp[x] / Total[Exp[x], {-1}];data = {-1.2, 0.3, 1.42};
softmax@data
SoftmaxLayer[]@datadata = {{1, 2}, {5.4, 3}};
softmax@data
SoftmaxLayer[]@dataThe dimension used as the normalization dimension cannot be 1, as this always normalizes to a constant array:
SoftmaxLayer[-1, "Input" -> {5, 1}]
SoftmaxLayer cannot accept symbolic inputs:
soft = SoftmaxLayer[];
soft[{1, 2}]soft[{1, z}]
Neat Examples (1)
Create a set of three instances of SoftmaxLayer that take and return an RGB image, but that normalize on the color channel dimension, the height and the width dimension, respectively:
softmaxes = Table[SoftmaxLayer[n, "Input" -> NetEncoder["Image"], "Output" -> NetDecoder["Image"]], {n, 3}]Apply the three layers to a single test image:
ImageAdjust /@ Through[softmaxes[[image]]]Text
Wolfram Research (2016), SoftmaxLayer, Wolfram Language function, https://reference.wolfram.com/language/ref/SoftmaxLayer.html (updated 2018).
CMS
Wolfram Language. 2016. "SoftmaxLayer." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2018. https://reference.wolfram.com/language/ref/SoftmaxLayer.html.
APA
Wolfram Language. (2016). SoftmaxLayer. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/SoftmaxLayer.html