NetInsertSharedArrays
converts all ordinary arrays in net into NetSharedArray objects.
NetInsertSharedArrays[net,"prefix"]
uses a prefix for the names of all newly shared arrays.
Examples
open all close allBasic Examples (4)
Convert arrays in a layer into shared arrays:
NetInsertSharedArrays[LinearLayer[2, "Input" -> 2]]Convert arrays in a layer into shared arrays, using a specific prefix:
NetInsertSharedArrays[LinearLayer[2, "Input" -> 2], "Linear/"]Convert arrays in layers within a net into shared arrays:
NetInsertSharedArrays[NetChain[{2, Ramp, 3}]]Create an LSTM layer whose arrays will be shared every time it is used in a larger net:
lstm = NetInsertSharedArrays[LongShortTermMemoryLayer[5, "Input" -> {"Varying", 3}]]Use the LSTM layer on two separate inputs in a graph and combine the resulting states with a CatenateLayer:
NetGraph[<|"lstm1" -> lstm, "lstm2" -> lstm, "last1" -> SequenceLastLayer[], "last2" -> SequenceLastLayer[], "cat" -> CatenateLayer[]|>, {NetPort["A"] -> "lstm1" -> "last1" -> "cat",
NetPort["B"] -> "lstm2" -> "last2" -> "cat"}]Properties & Relations (1)
Insert shared arrays into a layer:
layer = NetInsertSharedArrays[LinearLayer[2, "Input" -> 2]]When used multiple times in a larger network, only one set of weights will be created and stored:
net = NetInitialize[NetChain[{layer, Ramp, layer}]]When extracted, the weights for layers 1 and 3 are identical:
NetExtract[net, {{1, "Weights"}, {3, "Weights"}}]//Map[Normal]The underlying shared array can also be extracted using NetExtract:
NetExtract[net, NetSharedArray["Weights"]]//NormalThe amount of storage required for arrays in the entire net is the same as the storage required for a single layer:
Information[{net, layer}, "ArraysTotalSize"]Possible Issues (1)
The "MovingMean" and "MovingVariance" arrays of BatchNormalizationLayer cannot be shared.
Create a BatchNormalizationLayer with shared arrays:
sharedBatchNorm = NetInsertSharedArrays[BatchNormalizationLayer[]]net = NetTrain[NetChain[{2, sharedBatchNorm, 2, sharedBatchNorm, 2}], {{0, 1} -> {1, 0}, {1, 0} -> {0, 1}}]Extract the trained batch normalization layers:
{batchnorm1, batchnorm2} = NetExtract[net, {{2}, {4}}]The "Scaling" and "Biases" arrays were shared, but not "MovingMean" or "MovingVariance":
Normal /@ Information[batchnorm1, "Arrays"]Normal /@ Information[batchnorm2, "Arrays"]Text
Wolfram Research (2018), NetInsertSharedArrays, Wolfram Language function, https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html (updated 2019).
CMS
Wolfram Language. 2018. "NetInsertSharedArrays." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2019. https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html.
APA
Wolfram Language. (2018). NetInsertSharedArrays. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html