LinearLayer[n]
represents a trainable, fully connected net layer that computes 
with output vector of size n.
LinearLayer
LinearLayer[n]
represents a trainable, fully connected net layer that computes 
with output vector of size n.
LinearLayer[{n1,n2,…}]
represents a layer that outputs an array of dimensions n1×n2×….
leaves the dimensions of the output array to be inferred from context.
LinearLayer[n,opts]
includes options for initial weights and other parameters.
Details and Options
- The following optional parameters can be included:
-
"Biases" Automatic initial vector of biases (b in w.x+b) "Weights" Automatic initial matrix of weights (w in w.x+b) LearningRateMultipliers Automatic learning rate multiplier(s) to apply to weights and/or biases - When weights and biases are not explicitly specified or are given as Automatic, they are added automatically when NetInitialize or NetTrain is used.
- The setting "Biases"->None specifies that no biases should be used.
- If weights and biases have been added, LinearLayer[…][input] explicitly computes the output from applying the layer.
- LinearLayer[…][{input1,input2,…}] explicitly computes outputs for each of the inputi.
- When given a NumericArray as input, the output will be a NumericArray.
- NetExtract can be used to extract weights and biases from a LinearLayer object.
- LinearLayer is typically used inside NetChain, NetGraph, etc.
- LinearLayer exposes the following ports for use in NetGraph etc.:
-
"Input" an array "Output" an array of size n1×n2×… - LinearLayer[{}] specifies that the LinearLayer should produce a single real number.
- LinearLayer[n,"Input"->m] is the most common usage of LinearLayer and represents a LinearLayer that takes a vector of length m and produces a vector of length n.
- When it cannot be inferred from previous layers in a larger net, the option "Input"shape can be used to fix the input of LinearLayer. Possible forms for shape include:
-
"Real" a single real number m a vector of length m {m1,m2,…} an array of dimensions m1×m2×… NetEncoder[…] an encoder - Options[LinearLayer] gives the list of default options to construct the layer. Options[LinearLayer[…]] gives the list of default options to evaluate the layer on some data.
- Information[LinearLayer[…]] gives a report about the layer.
- Information[LinearLayer[…],prop] gives the value of the property prop of LinearLayer[…]. Possible properties are the same as for NetGraph.
Examples
open all close allBasic Examples (2)
Create a LinearLayer whose output is a length-5 vector:
LinearLayer[5]Create a randomly initialized LinearLayer:
linear = NetInitialize@LinearLayer[5, "Input" -> 3]Apply the layer to an input vector to produce an output vector:
linear[{0., 0.2, .1}]Scope (11)
Arguments (2)
Create a LinearLayer that produces a 3×2 matrix:
LinearLayer[{3, 2}]Specify an arbitrary depth output array:
LinearLayer[{3, 2, 1, 2}]An empty list corresponds to a scalar output:
LinearLayer[{}]Ports (6)
Specify that the "Input" port of the layer takes a vector of length 3:
LinearLayer[{3, 2}, "Input" -> 3]Specify the "Input" and "Output" ports explicitly:
LinearLayer["Input" -> 3, "Output" -> {3, 2}]Define a layer that takes and returns real numbers:
linear = NetInitialize@LinearLayer["Input" -> "Real", "Output" -> "Real"]Apply the initialized layer to an input:
linear[1]A LinearLayer with fully specified "Input" and "Output" ports can be initialized:
linear = NetInitialize@LinearLayer["Input" -> 3, "Output" -> {3, 2}]Apply the initialized layer to an input:
linear[{1, 2, 1}]//MatrixFormDefine a NetEncoder that takes a class and produces its one-hot encoding vector:
enc = NetEncoder[{"Class", {"class1", "class2", "class3"}, "UnitVector"}]Attach the encoder to the layer "Input" port:
linear = NetInitialize@LinearLayer[3, "Input" -> enc]Apply the layer to a member of the class:
linear["class2"]Define a NetEncoder that takes an image and produces a 28×28 matrix:
enc = NetEncoder[{"Image", {28, 28}, "Grayscale"}]Attach the encoder to the layer "Input" port:
linear = NetInitialize@LinearLayer[3, "Input" -> enc]Apply the layer to a member of the class:
linear[[image]]Parameters (3)
"Biases" (1)
Define and initialize a LinearLayer without biases:
linear = NetInitialize@LinearLayer[3, "Biases" -> None, "Input" -> 2]This is equivalent to Dot:
linear[{2.3, 4.5}]Dot[NetExtract[linear, "Weights"]//Normal, {2.3, 4.5}]"Weights" (2)
Specify a weight matrix, and use no biases:
linear = LinearLayer[3, "Weights" -> {{0.8}, {0.6}, {5.6}}, "Biases" -> None]linear[{1.0}]NetExtract[linear, "Weights"]Use a specific weight matrix and bias vector:
linear = LinearLayer[2, "Weights" -> {{0.1, 0.2}, {0.2, 0.4}}, "Biases" -> {0.1, 0.2}]This layer is fully specified and does not need to be initialized:
linear[{0., 0.}]Options (2)
LearningRateMultipliers (2)
Create a LinearLayer with frozen weights and biases:
linear = NetInitialize@LinearLayer[2, "Input" -> "Real", LearningRateMultipliers -> 0]Train a net with this layer inside:
{trained, lr} = NetTrain[NetChain[{linear, Tanh, LinearLayer[{}], LogisticSigmoid}], {-1 -> -0.4, -.5 -> 0, .5 -> 2, 1 -> 2.4}, {"TrainedNet", "ArraysLearningRateMultipliers"}]The weights and biases of the layer have been unchanged:
NetExtract[trained, {1, "Weights"}] == NetExtract[linear, "Weights"]NetExtract[trained, {1, "Biases"}] == NetExtract[linear, "Biases"]Create a LinearLayer with frozen weights, but free biases:
linear = NetInitialize@LinearLayer[2, "Input" -> "Real", LearningRateMultipliers -> {"Weights" -> 0}]Train a net with this layer inside:
{trained, lr} = NetTrain[NetChain[{linear, Tanh, LinearLayer[{}], LogisticSigmoid}], {-1 -> -0.4, -.5 -> 0, .5 -> 2, 1 -> 2.4}, {"TrainedNet", "ArraysLearningRateMultipliers"}]The weights have been unchanged, but the biases changed during training:
NetExtract[trained, {1, "Weights"}] == NetExtract[linear, "Weights"]NetExtract[trained, {1, "Biases"}] == NetExtract[linear, "Biases"]Applications (1)
Create a two-layer perceptron by stacking linear layers in a NetChain:
mlp = NetChain[{LinearLayer[100], Ramp, LinearLayer[], SoftmaxLayer[]}]Train the perceptron on the "MNIST" dataset of handwritten digits:
data = ResourceData["MNIST", "TrainingData"];
trained = NetTrain[mlp, data, MaxTrainingRounds -> 10]trained[{[image], [image], [image]}]Properties & Relations (2)
LinearLayer[n] can be specified as simply n in a NetChain:
NetChain[{5, 2}]For inputs and outputs that are vectors, LinearLayer computes:
linear[data_, weight_, bias_] := Dot[weight, data] + biasEvaluate a LinearLayer on data:
data = {2, 10, 3};
layer = NetInitialize@LinearLayer[2, "Input" -> 3]
layer[data]Manually compute the same result:
linear[data, NetExtract[layer, "Weights"]//Normal, NetExtract[layer, "Biases"]//Normal]Possible Issues (3)
LinearLayer cannot be initialized until all its input and output dimensions are known:
NetInitialize@LinearLayer[3]
NetInitialize@LinearLayer[3, "Input" -> 2]LinearLayer cannot accept symbolic inputs:
linear = NetInitialize@LinearLayer[3, "Input" -> 2];
linear[{1, 2}]linear[{1, z}]
A LinearLayer with output size n and input size m has a weight matrix of dimensions n×m:
linear = NetInitialize@LinearLayer[5, "Input" -> 3];
Dimensions@NetExtract[linear, "Weights"]If n and m are too large, there might not be enough system or GPU memory to initialize or train a net containing such a LinearLayer:
NetInitialize@LinearLayer[100000, "Input" -> 100000]
Text
Wolfram Research (2016), LinearLayer, Wolfram Language function, https://reference.wolfram.com/language/ref/LinearLayer.html (updated 2020).
CMS
Wolfram Language. 2016. "LinearLayer." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2020. https://reference.wolfram.com/language/ref/LinearLayer.html.
APA
Wolfram Language. (2016). LinearLayer. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/LinearLayer.html