NetPairEmbeddingOperator
represents a net that takes a pair of arrays, embeds them into a vector space using net, and outputs the distance under the embedding.
NetPairEmbeddingOperator[net,opts]
includes options for distance function to use and other parameters.
Details and Options
- In NetPairEmbeddingOperator[net], net should take exactly one input and produce exactly one output.
- NetExtract can be used to extract net from a NetPairEmbeddingOperator[net] object.
- The following optional parameter can be included:
-
DistanceFunction EuclideanDistance distance function to use LearningRateMultipliers Automatic learning rate multipliers for trainable arrays in the net - The following values of DistanceFunction are supported:
-
EuclideanDistance Euclidean distance between inputs CosineDistance cosine distance between inputs - The input and output ports of the net represented by NetPairEmbeddingOperator are:
-
"Input" a pair of arrays "Output" a real number - When it cannot be inferred from other layers in a larger net, the option "Input"->{n1,n2,…} can be used to fix the input dimensions of each of the pair of input arrays of NetPairEmbeddingOperator.
- When a NetPairEmbeddingOperator is trained, NetTrain will automatically attach a ContrastiveLossLayer if a loss is not specified.
- NetExtract allows access to the forward and reverse nets via "Net".
- Options[NetPairEmbeddingOperator] gives the list of default options to construct the operator. Options[NetPairEmbeddingOperator[…]] gives the list of default options to evaluate the operator on some data.
- Information[NetPairEmbeddingOperator[…]] gives a report about the operator.
- Information[NetPairEmbeddingOperator[…],prop] gives the value of the property prop of NetPairEmbeddingOperator[…]. Possible properties are the same as for NetGraph.
Examples
open all close allBasic Examples (2)
Create a NetPairEmbeddingOperator that embeds inputs using a NetChain:
net = NetPairEmbeddingOperator[NetChain[{LinearLayer[]}]]Create a NetPairEmbeddingOperator that embeds a pair of length-2 vectors using a randomly initialized NetChain:
net = NetInitialize@NetPairEmbeddingOperator[NetChain[{8, Ramp, 2}, "Input" -> {2}]]Apply the net to a pair of input vectors:
net[{{1, 2}, {3, 4}}]Thread the layer across a batch of input vectors:
net[{{{1, 2}, {3, 4}}, {{4, 2}, {7, 8}}}]Scope (1)
Create a NetGraph with one input and output:
graph = NetGraph[{LinearLayer[], Ramp, Tanh, CatenateLayer[]}, {1 -> 2 -> 4, 1 -> 3 -> 4}]Create a NetPairEmbeddingOperator that embeds vectors using a NetGraph:
net = NetPairEmbeddingOperator[graph]The embedding network in NetPairEmbeddingOperator[net] can be extracted using NetExtract:
NetExtract[net, "Net"]Options (1)
DistanceFunction (1)
Create a NetPairEmbeddingOperator that uses the CosineDistance function to calculate the distance between the two embedded arrays:
net = NetPairEmbeddingOperator[ElementwiseLayer[Tanh], DistanceFunction -> CosineDistance]Apply the layer to a pair of input vectors:
net[{{1, 2}, {4, 5}}]Applications (2)
Train a multilayer perceptron to embed a synthetic dataset based only on its topology. First, create the training data on a spiral-like manifold that is dense in the plane:
points = Table[r -> AngleVector[{r + RandomReal[5.5], r + RandomReal[0.1]}] / 15, {r, 0, 6Pi, .002}];Graphics[Point[Values@points], Frame -> True]mlp = NetChain[{200, Ramp, 100, Ramp, 20, Tanh, 1}, "Input" -> 2];Use NetPairEmbeddingOperator to define a loss network that measures the performance of the embedding:
lossNet = NetPairEmbeddingOperator[mlp]Create a generator that will sample pairs of points and associate them with True if their parameterization on the manifold differs by more than Pi:
sampler[n_] := Block[{a, b},
a = RandomSample[points, n];
b = RandomSample[points, n];
Thread[Transpose[Values /@ {a, b}] -> Thread[Abs[Keys[a] - Keys[b]] > Pi]]];sampler[4]Train the network, using a generator to sample pairs of points, and classify them as the same if their original parameterization was close:
trained = NetTrain[lossNet, sampler[#BatchSize]&, LossFunction -> ContrastiveLossLayer[], BatchSize -> 512, TimeGoal -> 60]Extract the embedding from the net:
trainedMLP = trained[["Net"]];Plot the 1D embedding learned by the net as a color map:
ArrayPlot[Flatten /@ trainedMLP /@ CoordinateBoundsArray[1.3 * {{-1, 1}, {-1, 1}}, .05], ColorFunction -> Hue
]Learn an embedding of the digits in the MNIST dataset. First, import the data and take only those examples with labels between 0 and 4:
mnist = Select[ResourceData["MNIST"], Last[#] < 5&];Create a training set by sampling pairs of images and associating them with True if their labels are different and False if their labels are the same:
toRule[{img1_ -> l1_, img2_ -> l2_}] := {img1, img2} -> l1 ≠ l2;trainingData = Table[toRule@RandomSample[mnist, 2], 10000];Take[trainingData, 4]Define a convolutional network to use as an embedding network:
lenet = NetChain[{
ConvolutionLayer[10, 4], Ramp, PoolingLayer[2, 2],
ConvolutionLayer[20, 4], Ramp, PoolingLayer[2, 2],
FlattenLayer[], 2},
"Input" -> NetEncoder[{"Image", {28, 28}, "Grayscale"}]]Construct the pair-embedding net:
net = NetPairEmbeddingOperator[lenet]trained = NetTrain[net, trainingData]Apply the network to a list of pairs of digits to compute their distances under the embedding. Digits with the same label have small distances:
trained[{{[image], [image]}, {[image], [image]}, {[image], [image]}}]Extract the embedding network:
embedding = NetExtract[trained, "Net"]Compute the embedding of a digit:
embedding[[image]]Sample 500 digits and group them by their labels:
groups = KeySort@GroupBy[RandomSample[mnist, 2000], Last -> First];Compute their embeddings and plot them. Digits with the same label are clustered under the learned embedding:
points = Map[embedding, groups];ListPlot[Table[points[[i]], {i, 1, 5}], PlotLegends -> Range[0, 4], Axes -> False, FrameTicks -> False, Frame -> True,
PlotStyle -> PointSize[0.012], AspectRatio -> 1]Properties & Relations (1)
NetMapOperator can be used to implement NetPairEmbeddingOperator. Create an initialized LinearLayer:
linear = NetInitialize@LinearLayer[2, "Input" -> {2}]Create NetPairEmbeddingOperator using the linear layer as its embedding net, and evaluate it on an input:
pairNet = NetPairEmbeddingOperator[linear, DistanceFunction -> EuclideanDistance]pairNet[{{1, 2}, {3, 4}}]Define a NetGraph with a NetMapOperator that is equivalent to the previous NetPairEmbeddingOperator, and evaluate it on the same input:
mapNet = NetGraph[{NetMapOperator[linear, "Input" -> {2, 2}], PartLayer[1], PartLayer[2], ThreadingLayer[(#1 - #2) ^ 2&], SummationLayer[]}, {1 -> 2, 1 -> 3, {2, 3} -> 4 -> 5}, "Input" -> {2, 2}]mapNet[{{1, 2}, {3, 4}}]Possible Issues (1)
Nets with multiple inputs or outputs cannot be used as the embedding network in NetPairEmbeddingOperator[net]:
NetPairEmbeddingOperator[MeanSquaredLossLayer[]]
Text
Wolfram Research (2017), NetPairEmbeddingOperator, Wolfram Language function, https://reference.wolfram.com/language/ref/NetPairEmbeddingOperator.html (updated 2020).
CMS
Wolfram Language. 2017. "NetPairEmbeddingOperator." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2020. https://reference.wolfram.com/language/ref/NetPairEmbeddingOperator.html.
APA
Wolfram Language. (2017). NetPairEmbeddingOperator. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetPairEmbeddingOperator.html