Wolfram Language & System Documentation Center

SingularValueDecomposition

gives the singular value decomposition for a numerical matrix m as a list of matrices {u,σ,v}, where σ is a diagonal matrix and m can be written as u.σ.ConjugateTranspose[v].

SingularValueDecomposition[{m,a}]

gives the generalized singular value decomposition of m with respect to a.

SingularValueDecomposition[m,spec]

gives the singular value decomposition associated with the largest singular values specified by spec.

SingularValueDecomposition

SingularValueDecomposition[m]

gives the singular value decomposition for a numerical matrix m as a list of matrices {u,σ,v}, where σ is a diagonal matrix and m can be written as u.σ.ConjugateTranspose[v].

SingularValueDecomposition[{m,a}]

gives the generalized singular value decomposition of m with respect to a.

SingularValueDecomposition[m,spec]

gives the singular value decomposition associated with the largest singular values specified by spec.

Details and Options

The matrix m may be rectangular.
The diagonal elements of σ are the singular values of m.
SingularValueDecomposition sets to zero any singular values that would be dropped by SingularValueList.
The option Tolerance can be used as in SingularValueList to determine which singular values will be considered to be zero. »
u and v are column orthonormal matrices, whose transposes can be considered as lists of orthonormal vectors.
SingularValueDecomposition[{m,a}] gives a list of matrices {{u,ua},{w,wa},v} such that m can be written as u.w.ConjugateTranspose[v] and a can be written as ua.wa.ConjugateTranspose[v]. »
The second argument spec can take the following settings:

	k	return the decomposition that goes with the k largest singular values
	UpTo[k]	return a decomposition for as many of the first k largest singular values as are available
	"Thin"	return a thin singular value decomposition using the Min[Dimensions[m]] largest singular values

With the setting TargetStructure->"Structured", SingularValueDecomposition[m] returns the matrices {u,σ,v} as structured matrices.

Examples

open all close all

Basic Examples (3)

Compute a singular value decomposition:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[(⁠|   |   |
| - | - |
| 1 | 2 |
| 1 | 2 |⁠)]

Reconstruct the input matrix:

Wolfram Language code: u.σ.Transpose[v]//MatrixForm

Compute a singular value decomposition for an invertible matrix:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[(⁠|     |     |
| --- | --- |
| 0.5 | 1   |
| 2   | 2.5 |⁠)]

The matrix of singular values is also invertible:

Wolfram Language code: σ//MatrixForm

Reconstruct the input matrix:

Wolfram Language code: u.σ.Transpose[v]

Compute a singular value decomposition for a non-square matrix:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[(⁠|     |     |
| --- | --- |
| 1.2 | 3.4 |
| 5.6 | 7.8 |
| 9.0 | 1.2 |⁠)]

Format the results:

Wolfram Language code: MatrixForm /@ {u, σ, v}

Reconstruct the input matrix:

Wolfram Language code: u.σ.Transpose[v]//MatrixForm

Scope (18)

Basic Uses (7)

Find the singular value decomposition of a machine-precision matrix:

Wolfram Language code:

SingularValueDecomposition[(⁠|     |     |     |
| --- | --- | --- |
| 1.1 | 1.3 | 2.1 |
| 2.2 | 2.2 | 3.3 |
| 3.3 | -3  | 4.7 |⁠)]

Format the result:

Wolfram Language code: MatrixForm /@ %

Singular value decomposition of a complex matrix:

Wolfram Language code:

MatrixForm /@ SingularValueDecomposition[(⁠|             |             |
| ----------- | ----------- |
| 0.5 + 0.5 I | 1.1         |
| -I          | 3.2 - 4.5 I |⁠)]

Singular value decomposition for an exact matrix:

Wolfram Language code: SingularValueDecomposition[{{2, 3}, {0, 2}}]

Singular value decomposition for an arbitrary-precision matrix:

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[MatrixExp[RandomReal[1, {2, 2}, WorkingPrecision -> 12]]]

Singular value decomposition of a symbolic matrix:

Wolfram Language code:

Simplify[SingularValueDecomposition[(⁠|           |   |
| --------- | - |
| Sqrt[2] x | 1 |
| 0         | 1 |⁠)], x∈ℝ]

The singular value decomposition of a large numerical matrix is computed efficiently:

Wolfram Language code: mat = RandomReal[{0, 9}, {1000, 1000}];

Wolfram Language code: AbsoluteTiming[SingularValueDecomposition[mat];]

Singular value decomposition of a non-square matrix:

Wolfram Language code:

MatrixForm /@ SingularValueDecomposition[(⁠|     |     |     |
| --- | --- | --- |
| 1.1 | 2   | 5   |
| 3   | -11 | 4.2 |⁠)]

Subsets of Singular Values (5)

Find the singular value decomposition associated with the three largest singular values of a matrix:

Wolfram Language code:

m = (⁠|     |     |      |       |        |
| --- | --- | ---- | ----- | ------ |
| 0.5 | 0.5 | 0.5  | 0     | 0      |
| 1.  | 2.  | 4.   | 8.    | 0      |
| 1.5 | 4.5 | 13.5 | 40.5  | 121.5  |
| 0   | 8.  | 32.  | 128.  | 512.   |
| 0   | 0   | 62.5 | 312.5 | 1562.5 |⁠);

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m, 3];
Map[MatrixForm, {u, σ, v}]//Chop

Unlike the full decomposition, these matrices do not recreate any part of the matrix exactly:

Wolfram Language code: m - u.σ.ConjugateTranspose[v]//MatrixForm

Find singular value decomposition associated with the three smallest singular values:

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[m, -3]

Find the "compact" decomposition associated with the nonzero singular values:

Wolfram Language code: m = {{0.5, 1., 1.5}, {2., 2.5, 3.}, {3.5, 4., 4.5}};

Wolfram Language code: {ut, σt, vt} = SingularValueDecomposition[m, MatrixRank[m]]

This decomposition still has sufficient information to reconstruct the matrix:

Wolfram Language code: m == ut.σt.Transpose[vt]

The full singular value decomposition contains a row of zeros:

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[m]

Find the thin decomposition of a rectangular matrix:

Wolfram Language code: m = {{1.2, 3.4}, {5.6, 7.8}, {9.0, 1.2}};

Wolfram Language code: MatrixForm /@ ({ut, σt, vt} = SingularValueDecomposition[m, "Thin"])

The matrix is square:

Wolfram Language code: SquareMatrixQ[σt]

The number of singular values used equals the smaller of the dimensions of the input matrix:

Wolfram Language code: Length[σt] == Min[Dimensions[m]]

This decomposition still has sufficient information to reconstruct the matrix:

Wolfram Language code: m == ut.σt.Transpose[vt]

The full singular value decomposition contains rows or columns of zeros in a rectangular :

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[m]

Find the decomposition associated with the three largest singular values, or as many as there are if fewer:

Wolfram Language code:

MatrixForm /@ SingularValueDecomposition[(⁠|     |     |     |     |     |
| --- | --- | --- | --- | --- |
| 0.5 | 1.  | 1.5 | 2.  | 2.5 |
| 2.  | 2.5 | 3.  | 3.5 | 4.  |
| 3.5 | 4.  | 4.5 | 5.  | 5.5 |
| 4.  | 4.5 | 5.  | 5.5 | 6.  |⁠), UpTo[3]]

Compute a truncated singular value decomposition for a matrix with repeated singular values:

Wolfram Language code: mat = {{(7/2), 0, (1/2), 0}, {0, 3, 0, 1}, {(1/2), 0, (7/2), 0}, {0, 1, 0, 3}};

Wolfram Language code: SingularValueList[mat]

Repeated singular values are counted separately when doing a partial decomposition:

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[mat, 3]

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[mat, -3]

Generalized Singular Value Decomposition (2)

Find the generalized singular value decomposition of a machine-precision real matrix:

Wolfram Language code: m = {{1.5, 1.3}, {3.1, 2.2}};

Wolfram Language code: a = {{1.1, 5.1}, {4.2, 3.4}};

Wolfram Language code:

{{u, ua}, {σ, σa}, v} = SingularValueDecomposition[{m, a}];
MatrixForm /@ {u, ua, σ, σa, v}

Verify the result:

Wolfram Language code: {m == u.σ.ConjugateTranspose[v], a == ua.σa.ConjugateTranspose[v]}

Find the generalized singular value decomposition of a machine-precision complex matrix:

Wolfram Language code:

m = (⁠|                                          |                                          |
| ---------------------------------------- | ---------------------------------------- |
| 4.641175490606916 + 7.207263282382337 I  | 2.522671922520866 + 8.989217121399136 I  |
| 4.114478069008367 + 7.010706036954508 I  | 8.95961276370625 + 3.569738897382109 I   |
| 4.405169936758572 + 4.4509297530197465 I | 9.145107926643146 + 6.3767022076415465 I |⁠);

Wolfram Language code:

a = (⁠|                                         |                                          |
| --------------------------------------- | ---------------------------------------- |
| 7.341517476299638 + 9.810565178638917 I | 8.24181899398409 + 3.4570598473462972 I  |
| 7.775955216608212 + 1.68269060515731 I  | 3.944961113408187 + 1.4889990223210265 I |
| 6.431377195364613 + 6.245150387114993 I | 7.026661544586792 + 1.0790173484547747 I |⁠);

Wolfram Language code:

{{u, ua}, {σ, σa}, v} = SingularValueDecomposition[{m, a}];
MatrixForm /@ {u, ua, σ, σa, v}//Chop

Verify the result:

Wolfram Language code: {m == u.σ.ConjugateTranspose[v], a == ua.σa.ConjugateTranspose[v]}

Special Matrices (4)

Singular value decomposition of sparse matrices:

Wolfram Language code: SparseArray[{{1, 3} -> 2, {2, 2} -> 3, {3, 1} -> 1, {4, 2} -> 5}, {4, 4}]

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[%]

Find the decomposition associated to the three largest singular values:

Wolfram Language code:

Block[{n = 200}, SparseArray[{{i_, i_} -> -2., 
     {i_, j_} /; Abs[i - j] == 1 -> 1.}, n]]

Wolfram Language code: {u, σ, v} = SingularValueDecomposition[%, 3, Method -> {"Arnoldi", MaxIterations -> 3000}];

Visualize the three right-singular vectors:

Wolfram Language code: ListPlot[Transpose[v]]

Singular value decomposition of structured matrices:

Wolfram Language code: SymmetrizedArray[{{1, 1} -> 2, {1, 2} -> 1}, {2, 2}, Symmetric[All]]

Wolfram Language code: SingularValueDecomposition[%]

Use a different structure:

Wolfram Language code: QuantityArray[{{1, 2}, {3, 4}}, {"Meters", "Meters"}]

The units go with the singular values:

Wolfram Language code: MatrixForm /@ SingularValueDecomposition[%]//N

Singular value decomposition of an identity matrix:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[IdentityMatrix[3]];
Map[MatrixForm, {u, σ, v}]

Verify the decomposition:

Wolfram Language code: u.σ.Transpose[v] == IdentityMatrix[3]

and could have been chosen to be identity matrices—the decomposition is not unique:

Wolfram Language code:

ui = vi = IdentityMatrix[3];
ui.σ.Transpose[vi] == IdentityMatrix[3]

Singular value decomposition of HilbertMatrix:

Wolfram Language code:

svd = SingularValueDecomposition[HilbertMatrix[3]];
MatrixForm /@ N[svd]

Options (3)

TargetStructure (2)

A real rectangular matrix:

Wolfram Language code: ma = RandomReal[1, {6, 4}]

With TargetStructure->"Dense", the result of SingularValueDecomposition is a list of three dense matrices:

Wolfram Language code: SingularValueDecomposition[ma, TargetStructure -> "Dense"]

With TargetStructure->"Structured", the result of SingularValueDecomposition is a list containing two OrthogonalMatrix objects and a DiagonalMatrix:

Wolfram Language code: SingularValueDecomposition[ma, TargetStructure -> "Structured"]

A complex rectangular matrix:

Wolfram Language code: ma = RandomComplex[1 + I, {6, 4}]

With TargetStructure->"Dense", the result of SingularValueDecomposition is a list of three dense matrices:

Wolfram Language code: SingularValueDecomposition[ma, TargetStructure -> "Dense"]

With TargetStructure->"Structured", the result of SingularValueDecomposition is a list containing two UnitaryMatrix objects and a DiagonalMatrix:

Wolfram Language code: SingularValueDecomposition[ma, TargetStructure -> "Structured"]

Tolerance (1)

m is a nearly singular matrix:

Wolfram Language code: m = {{1, 0}, {1, 10^-14}};

To machine precision, the matrix is effectively singular:

Wolfram Language code: SingularValueDecomposition[N[m]]

With a smaller tolerance, the nonzero singular value is detected:

Wolfram Language code: SingularValueDecomposition[N[m], Tolerance -> 10^-15]

The default tolerance is based on precision, so the small value is detected with precision 20:

Wolfram Language code: SingularValueDecomposition[N[m, 20]]

Applications (11)

Geometry of SVD (5)

Compute the singular value decomposition of the 2×2 matrix :

Wolfram Language code:

m = (⁠|      |       |
| ---- | ----- |
| 0.76 | -0.32 |
| 0.26 | -0.75 |⁠);
MatrixForm /@ ({u, σ, v} = SingularValueDecomposition[m])

The action of is a rotation and possibly—as happens for the axis in this case—a reflection:

Wolfram Language code: ParametricPlot[{Transpose[v].{Cos[θ], Sin[θ]}, {Cos[θ], Sin[θ]}}, {θ, 0, 2π}, IconizedObject[«Plot Options»]]

The action of is a scaling—either a dilation or compression—along each axis:

Wolfram Language code:

ParametricPlot[{σ.Transpose[v].{Cos[θ], Sin[θ]}, Transpose[v].{Cos[θ], Sin[θ]}}, {θ, 0, 2π}, IconizedObject[«Plot Options»]]

The action of is a rotation and possibly—though not in this case—a reflection in the target space:

Wolfram Language code:

ParametricPlot[{u.σ.Transpose[v].{Cos[θ], Sin[θ]}, σ.Transpose[v].{Cos[θ], Sin[θ]}}, {θ, 0, 2π}, IconizedObject[«Plot Options»]]

Compute the singular value decomposition of the 3×2 matrix :

Wolfram Language code:

m = (⁠|      |      |
| ---- | ---- |
| 0.96 | 0.49 |
| 0.3  | 0.83 |
| 0.72 | 0.39 |⁠);
MatrixForm /@ ({u, σ, v} = SingularValueDecomposition[m])

After the rotation in the plane by the matrix, the matrix embeds the unit circle as an ellipse in 3D:

Wolfram Language code:

addArrows[inMat_, outMat_] := Graphics3D[...]
Show[ParametricPlot3D[{σ.Transpose[v].{Cos[θ], Sin[θ]}, Append[Transpose[v].{Cos[θ], Sin[θ]}, 0]}, {θ, 0, 2π}, IconizedObject[«Plot Options»]], addArrows[σ.Transpose[v], Append[Transpose[v], {0, 0}]]]

The matrix rotates the ellipse in three dimensions:

Wolfram Language code:

Show[ParametricPlot3D[{u.σ.Transpose[v].{Cos[θ], Sin[θ]}, σ.Transpose[v].{Cos[θ], Sin[θ]}}, {θ, 0, 2π}, IconizedObject[«Plot Options»]], addArrows[u.σ.Transpose[v], σ.Transpose[v]]]

Compute the singular value decomposition of the 2×2 matrix :

Wolfram Language code:

m = {{1.1, -1.3}, {1.2, 2.5}};
MatrixForm /@ ({u, σ, v} = SingularValueDecomposition[m])

Let and denote the columns, respectively, of and :

Wolfram Language code: {{u1, u2}, {v1, v2}} = {Transpose[u], Transpose[v]};

is the direction in which is maximized, and the maximum value is :

Wolfram Language code: Maximize[Norm[m.x], x∈Circle[]]

Similarly, is the direction in which is minimized, and the minimum value is :

Wolfram Language code: Minimize[Norm[m.x], x∈Circle[]]

Visualize , and the unit circle along with their image under the multiplication on the left by :

Wolfram Language code:

ParametricPlot[m.{Cos[t], Sin[t]}, {t, 0, 2Pi}, Epilog -> {Thick, {RGBColor[0.880722, 0.611041, 0.142051], Line[{{0, 0}, m.v1}], Dashed, Line[{{0, 0}, v1}]}, {RGBColor[0.560181, 0.691569, 0.194885], Line[{{0, 0}, m.v2}], Dashed, Line[{{0, 0}, v2}]}, {RGBColor[0.368417, 0.506779, 0.709798], Dashed, Circle[]}}, Ticks -> {{-1.5, 1.5}, Automatic}]

is the direction in which is maximized, and again the maximum value is :

Wolfram Language code: Maximize[Norm[x.m], x∈Circle[]]

Similarly, is the direction in which is minimized, and again the minimum value is :

Wolfram Language code: Minimize[Norm[x.m], x∈Circle[]]

Visualize , and the unit circle along with their image under the multiplication on the right by :

Wolfram Language code:

ParametricPlot[{Cos[t], Sin[t]}.m, {t, 0, 2Pi}, Epilog -> {Thick, {RGBColor[0.880722, 0.611041, 0.142051], Line[{{0, 0}, u1.m}], Dashed, Line[{{0, 0}, u1}]}, {RGBColor[0.560181, 0.691569, 0.194885], Line[{{0, 0}, u2.m}], Dashed, Line[{{0, 0}, u2}]}, {RGBColor[0.368417, 0.506779, 0.709798], Dashed, Circle[]}}, Ticks -> {{-1.5, 1.5}, Automatic}]

Compute the singular value decomposition of the 3×2 matrix :

Wolfram Language code:

m = (⁠|           |           |
| --------- | --------- |
| 3         | 3         |
| 4         | -4        |
| 3 Sqrt[3] | 3 Sqrt[3] |⁠);
MatrixForm /@ ({u, σ, v} = SingularValueDecomposition[m])

Let and denote the columns, respectively, of and :

Wolfram Language code: {{u1, u2, u3}, {v1, v2}} = {Transpose[u], Transpose[v]};

is the direction in which is maximized, and the maximum value is :

Wolfram Language code: Maximize[Norm[m.x], x∈Circle[]]

Similarly, is the direction in which is minimized, and the minimum value is :

Wolfram Language code: Minimize[Norm[m.x], x∈Circle[]]

Visualize , and the image of the unit circle in the plane under left-multiplication by :

Wolfram Language code:

Show[ParametricPlot3D[m.{Cos[t], Sin[t]}, {t, 0, 2Pi}], Graphics3D[{RGBColor[0.880722, 0.611041, 0.142051], Tube[{{0, 0, 0}, m.v1}], RGBColor[0.560181, 0.691569, 0.194885], Tube[{{0, 0, 0}, m.v2}]}]]

is the direction in which is maximized, and again the maximum value is :

Wolfram Language code: Maximize[Norm[x.m], x∈Sphere[]]

minimizes —the minimum is zero, as the sphere is compressed into an ellipse in the plane:

Wolfram Language code: Minimize[Norm[x.m], x∈Sphere[]]

maximizes subject to the constraint , and the maximum value is :

Wolfram Language code: Maximize[{Norm[x.m], u1.x == 0}, x∈Sphere[]]

Visualize , , and the image of the unit sphere in the plane under right-multiplication by :

Wolfram Language code:

ParametricPlot[{Cos[p]Sin[t], Sin[p]Sin[t], Cos[t]}.m, {p, 0, 2Pi}, {t, 0, π}, Epilog -> {Thick, {RGBColor[0.880722, 0.611041, 0.142051], Arrow[{{0, 0}, u1.m}]}, {RGBColor[0.560181, 0.691569, 0.194885], Arrow[{{0, 0}, u2.m}]}, {RGBColor[0.922526, 0.385626, 0.209179], AbsolutePointSize[5], Point[u3.m]}}]

Compute the singular value decomposition of the 3×3 matrix :

Wolfram Language code:

m = (⁠|        |        |      |
| ------ | ------ | ---- |
| -0.069 | 0.35   | 0.98 |
| 0.50   | -0.072 | 0.64 |
| 0.92   | 0.93   | 0.15 |⁠);
MatrixForm /@ ({u, σ, v} = SingularValueDecomposition[m])

Let and denote the columns, respectively, of and :

Wolfram Language code: {{u1, u2, u3}, {v1, v2, v3}} = {Transpose[u], Transpose[v]};

is the direction in which is maximized, and the maximum value is :

Wolfram Language code: Maximize[Norm[m.x], x∈Sphere[]]

is the direction in which is maximized if , and the maximum value is :

Wolfram Language code: Maximize[{Norm[m.x], x.v1 == 0}, x∈Sphere[]]

Similarly, is the direction in which is minimized, and the minimum value is :

Wolfram Language code: Minimize[Norm[m.x], x∈Sphere[]]

Visualize , , and the unit sphere along with their image under the multiplication on the left by :

Wolfram Language code:

{Show[ParametricPlot3D[{Cos[p] Sin[t], Sin[p] Sin[t], Cos[t]}, {p, 0, 2 Pi}, {t, 0, π}, IconizedObject[«Plot Options»]], Graphics3D[Riffle[{RGBColor[0.5, 0, 0.5], RGBColor[0.560181, 0.691569, 0.194885], RGBColor[0.922526, 0.385626, 0.209179]}, Arrow[Tube[{{0, 0, 0}, #}]]& /@ Transpose[v]]], ImageSize -> 250], Show[ParametricPlot3D[m.{Cos[p] Sin[t], Sin[p] Sin[t], Cos[t]}, {p, 0, 2 Pi}, {t, 0, π}, IconizedObject[«Plot Options»]], Graphics3D[Riffle[{RGBColor[0.5, 0, 0.5], RGBColor[0.560181, 0.691569, 0.194885], RGBColor[0.922526, 0.385626, 0.209179]}, Arrow[Tube[{{0, 0, 0}, m.#}]]& /@ Transpose[v]]], ImageSize -> 250]}

is the direction in which is maximized, and again the maximum value is :

Wolfram Language code: Maximize[Norm[x.m], x∈Sphere[]]

is the direction in which is maximized if , and again the maximum value is :

Wolfram Language code: Maximize[{Norm[x.m], x.u1 == 0}, x∈Sphere[]]

Similarly, is the direction in which is minimized, and again the minimum value is :

Wolfram Language code: Minimize[Norm[x.m], x∈Sphere[]]

Visualize , , and the unit sphere along with their image under the multiplication on the right by :

Wolfram Language code:

{Show[ParametricPlot3D[{Cos[p] Sin[t], Sin[p] Sin[t], Cos[t]}, {p, 0, 2 Pi}, {t, 0, π}, IconizedObject[«Plot Options»]], Graphics3D[Riffle[{RGBColor[0.5, 0, 0.5], RGBColor[0.560181, 0.691569, 0.194885], RGBColor[0.922526, 0.385626, 0.209179]}, Arrow[Tube[{{0, 0, 0}, #}]]& /@ Transpose[u]]], ImageSize -> 250], Show[ParametricPlot3D[{Cos[p] Sin[t], Sin[p] Sin[t], Cos[t]}.m, {p, 0, 2 Pi}, {t, 0, π}, IconizedObject[«Plot Options»]], Graphics3D[Riffle[{RGBColor[0.5, 0, 0.5], RGBColor[0.560181, 0.691569, 0.194885], RGBColor[0.922526, 0.385626, 0.209179]}, Arrow[Tube[{{0, 0, 0}, #.m}]]& /@ Transpose[u]]], ImageSize -> 250]}

Least Squares and Curve Fitting (6)

If the linear system has no solution, the best approximate solution is the least-squares solution. That is the solution to , where is the orthogonal projection of onto the column space of , which can be computed using the singular value decomposition. Consider the following and :

Wolfram Language code:

m = (⁠|   |    |
| - | -- |
| 1 | 2  |
| 3 | -4 |
| 1 | -1 |⁠);
b = {0, -2, 1};

The linear system is inconsistent:

Wolfram Language code: LinearSolve[m, b]

Find the matrix of the compact singular value decomposition of . Its columns are orthonormal and span :

Wolfram Language code:

u = First[SingularValueDecomposition[m, MatrixRank[m]]];
u//N//MatrixForm

Compute the orthogonal projection of onto the space spanned by the columns of :

Wolfram Language code: bPerp = u.Transpose[u].b//FullSimplify

Visualize , its projections onto the columns of and :

Wolfram Language code:

{b1, b2} = Transpose[u](Transpose[u].b);
Graphics3D[...]

Solve :

Wolfram Language code: LinearSolve[m, bPerp]

Confirm the result using LeastSquares:

Wolfram Language code: LeastSquares[m, b]

Solve the least-squares problem for the following and using only the singular value decomposition:

Wolfram Language code:

m = (⁠|              |             |              |
| ------------ | ----------- | ------------ |
| 4.  + 0.5 I  | 3.  + 0.5 I | 4.5  + 0.5 I |
| 3.5  + 0.5 I | 4.  + 0.5 I | 3.5  + 0.5 I |
| -3. - 0.5 I  | -5. - 0.5 I | -2.5 - 0.5 I |⁠);
b = {0, -2, 1 + I};

Compute the compact singular value decomposition where only the nonzero singular values are kept:

Wolfram Language code: MatrixForm /@ ({u, σ, v} = SingularValueDecomposition[m, MatrixRank[m]])

Let :

Wolfram Language code: x = v.Inverse[σ].ConjugateTranspose[u].b

By definition, , so , the orthogonal projection of onto :

Wolfram Language code: m.x == u.ConjugateTranspose[u].b

Thus, is the solution to the least-squares problem, as confirmed by LeastSquares:

Wolfram Language code: x == LeastSquares[m, b]

Solve the least-squares problem for the following and two different ways: by projecting onto the column space of using just the matrix of the singular value decomposition, and the direct solution using the full decomposition. Compare and explain the results:

Wolfram Language code:

m = (|   |   |   |
| - | - | - |
| 1 | 5 | 4 |
| 2 | 6 | 4 |
| 3 | 7 | 4 |
| 4 | 8 | 4 |);
b = {0, -2, 1, 2};

Compute the compact singular value decomposition of:

Wolfram Language code: {u, σ, v} = SingularValueDecomposition[m, MatrixRank[m]]//FullSimplify;

Compute the orthogonal projection of onto :

Wolfram Language code: bPerp = u.Transpose[u].b//FullSimplify

Solve :

Wolfram Language code: xPerp = LinearSolve[m, bPerp]

The direct solution can be found using , as both and are real-valued:

Wolfram Language code: x = v.Inverse[σ].Transpose[u].b//FullSimplify

While x and xPerp are different, both solve the least-squares problem because m.x==m.xPerp:

Wolfram Language code: m.xPerp == m.x

The two solutions differ by an element of NullSpace[m]:

Wolfram Language code: x - xPerp

Wolfram Language code: NullSpace[m]

Note that LeastSquares[m,b] gives the result using the direct method:

Wolfram Language code: x == LeastSquares[m, b]

For the matrices and that follow, find a matrix that minimizes $TemplateBox[{{{m, ., x}, -, b}}, Norm]_F$ :

Wolfram Language code:

m = {{1, 1}, {1, 2}, {1, 3}};
b = {{7, 1}, {7, 2}, {8, 3}};

One solution, in this case unique, is given by :

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m, MatrixRank[m]];
v.Inverse[σ].Transpose[u].b//FullSimplify

This result could also have been obtained using LeastSquares[m,b]:

Wolfram Language code: LeastSquares[m, b]

Confirm the answer using Minimize:

Wolfram Language code: Minimize[Norm[m.{{w, x}, {y, z}} - b, "Frobenius"], {w, x, y, z}]

SingularValueDecomposition can be used to find a best-fit curve to data. Consider the following data:

Wolfram Language code:

data = IconizedObject[«Data»];
ListPlot[data]

Extract the and coordinates from the data:

Wolfram Language code: {x, y} = Extract[data, {{All, 1}, {All, 2}}];

Construct a design matrix, whose columns are and , for fitting to a line :

Wolfram Language code: m = Map[x |-> {1, x}, x];

Get the coefficients and for a linear least‐squares fit using a thin singular value decomposition:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m, 2];
{a, b} = v.Inverse[σ].Transpose[u].y

Verify the coefficients using Fit:

Wolfram Language code: Fit[data, {1, s}, s]

Plot the best-fit curve along with the data:

Wolfram Language code: Show[Plot[a + b x, {x, 0, 1}], ListPlot[data]]

Find the best-fit parabola to the following data:

Wolfram Language code:

data = IconizedObject[«Data»];
ListPlot[data]

Extract the and coordinates from the data:

Wolfram Language code: {x, y} = Extract[data, {{All, 1}, {All, 2}}];

Construct a design matrix, whose columns are , and , for fitting to a line :

Wolfram Language code: m = DesignMatrix[data, {1, ,  ^ 2}, ];

Get the coefficients , and for a least‐squares fit:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m, 3];
{a, b, c} = v.Inverse[σ].Transpose[u].y

Verify the coefficients using Fit:

Wolfram Language code: Fit[data, {1, s, s ^ 2}, s]

Plot the best-fit curve along with the data:

Wolfram Language code: Show[Plot[a + b x + c x ^ 2, {x, 0, 3}], ListPlot[data, IconizedObject[«plot options»]]]

Properties & Relations (13)

The singular value decomposition {u,σ,v} of m decomposes m as u.σ.ConjugateTranspose[v]:

Wolfram Language code:

m = RandomComplex[1 + I, {3, 4}];
{u, σ, v} = SingularValueDecomposition[m];
Chop[m - u.σ.ConjugateTranspose[v]]

If a is an n×m matrix with decomposition {u,σ,v}, then u is an n×n matrix:

Wolfram Language code:

n = 5;m = 3;a = RandomReal[1, {n, m}];
{u, σ, v} = SingularValueDecomposition[a];
Dimensions[u] == {n, n}

v is an m×m matrix:

Wolfram Language code: Dimensions[v] == {m, m}

And σ is an n×m matrix:

Wolfram Language code: Dimensions[σ] == {n, m}

SingularValueDecomposition[m] is built from the eigenvalue decompositions of and :

Wolfram Language code:

m = (⁠|      |     |      |
| ---- | --- | ---- |
| -5.2 | 0   | -5   |
| 1.7  | 5.4 | -4.3 |⁠);

Wolfram Language code: {u, Σ, v} = SingularValueDecomposition[m];

Compute EigenvalueDecomposition[m.ConjugateTranspose[m]]:

Wolfram Language code: {Subscript[t, L], Subscript[d, L]} = EigenvalueDecomposition[m . ConjugateTranspose[m]];

Then is up to phase in each column:

Wolfram Language code: {u//MatrixForm, Subscript[t, L]//MatrixForm}

Compute the eigenvalue decomposition of :

Wolfram Language code: {Subscript[t, R], Subscript[d, R]} = EigenvalueDecomposition[ConjugateTranspose[m].m];

Then is up to phase in each column:

Wolfram Language code: {v//MatrixForm, Subscript[t, R]//MatrixForm}

Since has fewer rows than columns, the is (as opposed to ):

Wolfram Language code: MatrixForm /@ {Σ, Sqrt[Subscript[d, L]]}

The first right singular vector can be found by maximizing over all unit vectors:

Wolfram Language code: m = {{0.74, 0.43, 0.28}, {0.63, 0.74, 0.4}, {0.47000000000000003, 0.9400000000000001, 0.55}};

Wolfram Language code: v1 = ArgMax[Norm[m.x], x∈Sphere[{0, 0, 0}]]

Each subsequent vector is a maximizer with the constraint that it is perpendicular to all previous vectors:

Wolfram Language code: v2 = ArgMax[{Norm[m.x], v1.x == 0}, x∈Sphere[{0, 0, 0}]]

Wolfram Language code: v3 = ArgMax[{Norm[m.x], v1.x == 0 && v2.x == 0}, x∈Sphere[{0, 0, 0}]]

Compare the with the found by SingularValueDecomposition; they are the same up to sign:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m];
MatrixForm /@ {Transpose[{v1, v2, v3}], v}

The analogous statement holds for the left singular vectors with :

Wolfram Language code:

u1 = ArgMax[Norm[x.m], x∈Sphere[{0, 0, 0}]];
u2 = ArgMax[{Norm[x.m], u1.x == 0}, x∈Sphere[{0, 0, 0}]];
u3 = ArgMax[{Norm[x.m], u1.x == 0 && u2.x == 0}, x∈Sphere[{0, 0, 0}]];
MatrixForm /@ {Transpose[{u1, u2, u3}], u}

The diagonal entries of are the respective maximum values:

Wolfram Language code: {Diagonal[σ], Norm /@ {m.v1, m.v2, m.v3}, Norm /@ {u1.m, u2.m, u3.m}}//MatrixForm

If is the smaller of the dimensions of , the first columns of and are related by :

Wolfram Language code:

m = RandomInteger[1, {3, 4}];
{u, σ, v} = SingularValueDecomposition[m];
Table[σ[[i, i]]u[[All, i]] == m.v[[All, i]], {i, 3}]//FullSimplify

The first columns of and are also related by :

Wolfram Language code: Table[σ[[i, i]]v[[All, i]] == u[[All, i]].m, {i, 3}]//FullSimplify

If m is a square matrix, the product of the diagonal elements of equals Abs[Det[m]]:

Wolfram Language code:

m = RandomReal[1, {5, 5}];
Times@@Diagonal[SingularValueDecomposition[m][[2]]] == Abs[Det[m]]

If is a normal matrix, both and are composed of the same vectors:

Wolfram Language code:

m = (⁠|                     |                     |                      |
| ------------------- | ------------------- | -------------------- |
| 0                   | -0.2785693493596917 | -0.12624545596692704 |
| 0.2785693493596917  | 0                   | -0.08798023011262301 |
| 0.12624545596692704 | 0.08798023011262301 | 0                    |⁠);
NormalMatrixQ[m]

The vectors will appear in a different order unless is positive semidefinite and Hermitian:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m]//Chop;
{u//MatrixForm, v//MatrixForm}

The diagonal entries of equal Abs[Eigenvalues[m]]:

Wolfram Language code: Diagonal[σ] == Chop[Abs[Eigenvalues[m]]]

For positive definite and Hermitian , SingularValueDecomposition and EigenvalueDecomposition coincide:

Wolfram Language code:

m = (⁠|                                              |                                              |                                              |
| -------------------------------------------- | -------------------------------------------- | -------------------------------------------- |
| 3.830367325120047 + 0. I                     | 0.07350793635153963 - 0.12698026356721126 I  | -0.7939695217640885 - 0.002520762074857519 I |
| 0.07350793635153963 + 0.12698026356721126 I  | 2.4790235245220367 + 0. I                    | -0.046544276125373374 - 0.5373197314346221 I |
| -0.7939695217640885 + 0.002520762074857519 I | -0.046544276125373374 + 0.5373197314346221 I | 1.042188083119043 + 0. I                     |⁠);
{u, σ, v} = SingularValueDecomposition[m];
{t, d} = EigenvalueDecomposition[m];
PositiveDefiniteMatrixQ[m] && HermitianMatrixQ[m]

and are equal:

Wolfram Language code: u - v//Chop

Their columns are unit eigenvectors of , so they are the same as up to phase:

Wolfram Language code: Abs[t / u]

The nonzero elements of are the eigenvalues of , so :

Wolfram Language code: σ == d

MatrixRank[m] equals the number of nonzero singular values:

Wolfram Language code:

m  = RandomInteger[1, {3, 3}];
{u, σ, v} = SingularValueDecomposition[m];
MatrixRank[m] == Length[Select[Diagonal[σ], x |-> x > 0]]

The compact decomposition that only keeps nonzero singular values can compute PseudoInverse[m]:

Wolfram Language code: m = {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}};

Wolfram Language code: {u, σ, v} = SingularValueDecomposition[m, MatrixRank[m]];

If decomposes as , then $m^((-1))=v.TemplateBox[{sigma}, Inverse].TemplateBox[{u}, ConjugateTranspose]$ :

Wolfram Language code: PseudoInverse[m] == v.Inverse[σ].ConjugateTranspose[u]//FullSimplify

A matrix m that is an outer product of two vectors has MatrixRank[m]==1:

Wolfram Language code:

x1 = {1, 2, 3};x2 = {4, 5};
m = Outer[Times, x1, x2]

Wolfram Language code: MatrixRank[m]

The nonzero singular value of m is the product of the norms of the vectors:

Wolfram Language code: {u, σ, v} = SingularValueDecomposition[m, 1]

Wolfram Language code: σ[[1, 1]] == Norm[x1]Norm[x2]

The corresponding left and right singular vectors are the input vectors, normalized:

Wolfram Language code: {u[[All, 1]] == (x1/Norm[x1]), v[[All, 1]] == (x2/Norm[x2])}

SingularValueDecomposition[{m,a}] decomposes m as u.w.ConjugateTranspose[v]:

Wolfram Language code:

m = RandomComplex[1 + I, {3, 4}];
a = RandomComplex[1 + I, {2, 4}];
{{u, ua}, {w, wa}, v} = SingularValueDecomposition[{m, a}];
Chop[m - u.w.ConjugateTranspose[v]]

It decomposes a as ua.wa.ConjugateTranspose[v]:

Wolfram Language code: Chop[a - ua.wa.ConjugateTranspose[v]]

SingularValueDecomposition[{m,a}] can be related to Eigensystem[{m.m,a.a}]:

Wolfram Language code:

m = (⁠|                                            |                                             |                                            |
| ------------------------------------------ | ------------------------------------------- | ------------------------------------------ |
| 0.9753346860069796 + 0.32225791006850857 I | 0.8493715230848708 + 0.39224803196754077 I  | 0.5839367017062629 + 0.11364379814173331 I |
| 0.8359555932254314 + 0.7712743289239168 I  | 0.48928975439484756 + 0.48056805668910774 I | 0.3042001847125009 + 0.11991764017567741 I |
| 0.4727880193915137 + 0.9358524506075876 I  | 0.9540225347992926 + 0.545522455352986 I    | 0.1542301183700603 + 0.02063002461235164 I |⁠);
a = (⁠|                                               |                                            |                                             |
| --------------------------------------------- | ------------------------------------------ | ------------------------------------------- |
| 0.0002671678119234855 + 0.17886262704796918 I | 0.23822746561953023 + 0.6255181037100543 I | 0.8983010971084864 + 0.06465535916883036 I  |
| 0.4125835170325072 + 0.6294871356437639 I     | 0.17227933769920623 + 0.5053818767600351 I | 0.20608708391474617 + 0.7815903406108518 I  |
| 0.19927550189059895 + 0.16958348510576626 I   | 0.6395479165043441 + 0.7231362087260429 I  | 0.030839103381344435 + 0.5213431075486419 I |⁠);
{{u, ua}, {w, wa}, v} = SingularValueDecomposition[{m, a}];
{λ, vλ} = Eigensystem[{m.m, a.a}];

The diagonal elements of w are /:

Wolfram Language code: {Diagonal[w], (Sqrt[λ]/Sqrt[1 + λ])}

The diagonal elements of wa are 1/:

Wolfram Language code: {Diagonal[wa], (1/Sqrt[1 + λ])}

The columns of v are scaled multiples of the columns of Conjugate[Inverse[vλ]]:

Wolfram Language code: r = (v/Conjugate[Inverse[vλ]])

The magnitude of the scaling is the ratio of the corresponding diagonal elements of w and vλ.m.m.vλ:

Wolfram Language code: Abs[First[r] / (Diagonal[Sqrt[vλ.m.m.vλ]]/Diagonal[w])]

Equivalently, it is the ratio of the corresponding diagonal elements of wa and vλ.ma.ma.vλ:

Wolfram Language code: Abs[First[r] / (Diagonal[Sqrt[vλ.a.a.vλ]]/Diagonal[wa])]

Possible Issues (1)

m is a 2×1000 random matrix:

Wolfram Language code: m = RandomReal[1, {2, 1000}];

The full singular value decomposition is very large because u is a 1000×1000 matrix:

Wolfram Language code: Map[Dimensions, SingularValueDecomposition[m]]

The condensed singular value decomposition is much smaller:

Wolfram Language code:

{u, σ, v} = SingularValueDecomposition[m, Min[Dimensions[m]]];
Map[Dimensions, {u, σ, v}]

It still contains sufficient information to reconstruct m:

Wolfram Language code: Max[Abs[m - u.σ.ConjugateTranspose[v]]]

Top

More Learning

Tech Support

Wolfram Solutions

Wolfram Solutions For Education

Get Started

Grow Your Skills

Work with Us

Educational Programs for Adults

Educational Programs for Youth

Read

SingularValueDecomposition

Details and Options

Examples

Basic Examples (3)

Scope (18)

Basic Uses (7)

Subsets of Singular Values (5)

Generalized Singular Value Decomposition (2)

Special Matrices (4)

Options (3)

TargetStructure (2)

Tolerance (1)

Applications (11)

Geometry of SVD (5)

Least Squares and Curve Fitting (6)

Properties & Relations (13)

Possible Issues (1)

Text

CMS

APA

BibTeX

BibLaTeX

SingularValueDecomposition

Details and Options

Examples

Basic Examples (3)

Scope (18)

Basic Uses (7)

Subsets of Singular Values (5)

Generalized Singular Value Decomposition (2)

Special Matrices (4)

Options (3)

TargetStructure (2)

Tolerance (1)

Applications (11)

Geometry of SVD (5)

Least Squares and Curve Fitting (6)

Properties & Relations (13)

Possible Issues (1)

See Also

Tech Notes

Related Guides

History

Text

CMS

APA

BibTeX

BibLaTeX