LogLikelihood
LogLikelihood[dist,{x1,x2,…}]
gives the log‐likelihood function for observations x1, x2, … from the distribution dist.
LogLikelihood[proc,{{t1,x1},{t2,x2},…}]
gives the log-likelihood function for the observations xi at time ti from the process proc.
LogLikelihood[proc,{path1,path2,…}]
gives the log-likelihood function for the observations from path1, path2, … from the process proc.
Details
- The log‐likelihood function LogLikelihood[dist,{x1,x2,…}] is given by
, where 
is the probability density function at xi, PDF[dist,xi]. - For a scalar‐valued process proc, the log-likelihood function LogLikelihood[proc,{{t1,x1},{t2,x2},…}] is given by LogLikelihood[SliceDistribution[proc,{t1,t2,…}],{{x1,x2,…}}].
- For a vector‐valued process proc, the log-likelihood function LogLikelihood[proc,{{t1,{x1,…,z1}},{t2,{x2,…,z2}},…}] is given by LogLikelihood[SliceDistribution[proc,{t1,t2,…}],{{x1,…,z1,x2,…,z2,…}}].
- The log-likelihood function for a collection of paths LogLikelihood[proc,{path1,path2,…}] is given by
LogLikelihood[proc,pathi].
Examples
open all close allBasic Examples (4)
Get the log‐likelihood function for a normal distribution:
LogLikelihood[NormalDistribution[μ, σ], {Subscript[x, 1], Subscript[x, 2], Subscript[x, 3], Subscript[x, 4], Subscript[x, 5]}]Compute a log‐likelihood for numeric data:
data = RandomVariate[GammaDistribution[2, 3], 100];LogLikelihood[GammaDistribution[2, 3], data]Plot log‐likelihood contours as a function of 
and 
:
ContourPlot[LogLikelihood[GammaDistribution[α, β], data], {α, 1, 3}, {β, 1, 5}]Compute the log‐likelihood for multivariate data:
data = RandomVariate[BinormalDistribution[.8], 100];LogLikelihood[BinormalDistribution[ρ], data]Compute the log-likelihood for a process:
proc = WienerProcess[];
data = RandomFunction[proc, {1, 2, .1}];LogLikelihood[proc, data]Scope (12)
Univariate Parametric Distributions (2)
Compute the log‐likelihood for a continuous distribution:
data = RandomVariate[GumbelDistribution[3, 6], 100];LogLikelihood[GumbelDistribution[3, 6], data]Compute the log‐likelihood for a discrete distribution:
LogLikelihood[BinomialDistribution[20, 1 / 4], Range[5, 10]]Plot the log‐likelihood, assuming 
is unknown:
Plot[LogLikelihood[BinomialDistribution[20, p], Range[5, 10]], {p, 0, 1}]Multivariate Parametric Distributions (2)
Obtain the log‐likelihood for a continuous multivariate distribution with unknown parameters:
data = RandomVariate[DirichletDistribution[{2, 5, 10}], 100];ll = LogLikelihood[DirichletDistribution[{Subscript[γ, 1], Subscript[γ, 2], Subscript[γ, 3]}], data]//PowerExpandVisualize the log‐likelihood surface, assuming 
:
Plot3D[ll /. Subscript[γ, 3] -> 10, {Subscript[γ, 1], 1, 3}, {Subscript[γ, 2], 3, 7}]For a multivariate discrete distribution with known parameters:
𝒟 = MultinomialDistribution[100, {.3, .2, .5}];
data = RandomVariate[𝒟, 100];LogLikelihood[𝒟, data]Derived Distributions (5)
Compute the log‐likelihood for a truncated standard normal:
𝒟 = TruncatedDistribution[{-1, 3 / 2}, NormalDistribution[]];
data = RandomVariate[𝒟, 100];LogLikelihood[𝒟, data]Plot the log‐likelihood contours as a function of the truncation points:
ContourPlot[LogLikelihood[TruncatedDistribution[{min, max}, NormalDistribution[]], data], {min, -2, Min[data]}, {max, Max[data], 3}]Compute the log‐likelihood for a constructed distribution:
dist = ProbabilityDistribution[λ Exp[λ(1 - x)], {x, 1, Infinity}, Assumptions -> λ > 0]LogLikelihood[dist, {1.4, 5.1, 1.7, 1.6, 1.1, 3.9, 2.2, 1.3, 2., 1.5}]//PowerExpandCompute the log‐likelihood for a product distribution:
𝒟 = ProductDistribution[BetaDistribution[2, 3], LaplaceDistribution[10, 1]];data = RandomVariate[𝒟, 100];LogLikelihood[𝒟, data]Obtain the result as a sum of the independent componentwise log‐likelihoods:
LogLikelihood[BetaDistribution[2, 3], data[[All, 1]]] + LogLikelihood[LaplaceDistribution[10, 1], data[[All, 2]]]Compute the log‐likelihood for a copula distribution:
𝒟 = CopulaDistribution[{"Frank", α}, {GammaDistribution[2, 3], ChiSquareDistribution[10]}];data = RandomVariate[𝒟 /. α -> 1 / 2, 100];Plot the log‐likelihood as a function of the kernel parameter:
Plot[LogLikelihood[𝒟, data], {α, 1 / 10, 9 / 10}]Compute the log‐likelihood for a component mixture:
𝒟 = MixtureDistribution[{1 / 3, 2 / 3}, {GammaDistribution[2, 3], NormalDistribution[1, 1 / 4]}];data = RandomVariate[𝒟, 100];LogLikelihood[𝒟, data]Random Processes (3)
Compute the log-likelihood of a continuous parametric process:
proc = GeometricBrownianMotionProcess[1, 2, 1];
data = RandomFunction[proc, {1, 2, .1}];LogLikelihood[proc, data]Compute the log-likelihood of a scalar-valued discrete parametric process:
proc = BinomialProcess[.3];
data = RandomFunction[proc, {10}];LogLikelihood[proc, data]Plot the log‐likelihood as a function of the process parameter:
Plot[LogLikelihood[BinomialProcess[p], data], {p, 0, 1}]Compute the log-likelihood of a scalar-valued time series process:
proc = ARMAProcess[{.3, .2}, {-.8}, 1];
data = RandomFunction[proc, {100}];LogLikelihood[proc, data]Compute the log-likelihood of a vector-valued time series process:
phi = {{{.2, .1}, {.3, .1}}};
sigma = {{1, .3}, {.3, .5}};
proc = ARProcess[phi, sigma];
data = RandomFunction[proc, {100}];LogLikelihood[proc, data]Applications (4)
Visualize the log‐likelihood surface for a distribution of two parameters:
data = RandomVariate[WeibullDistribution[4, 5], 100];Plot3D[LogLikelihood[WeibullDistribution[α, β], data], {α, 3, 5}, {β, 4, 6}]Visualize as contours of equal log‐likelihood:
ContourPlot[LogLikelihood[WeibullDistribution[α, β], data], {α, 3, 5}, {β, 4, 6}]Show log-likelihood functions with mixed continuous and discrete parameters:
𝒟 = BinomialDistribution[n, p];
data = RandomVariate[𝒟 /. {n -> 10, p -> 0.1}, 100];Plot[Table[LogLikelihood[𝒟, data], {n, r = {5, 10, 15}}]//Evaluate, {p, 0, 1}, PlotLegends -> (Row[{"n = ", #}]& /@ r)]DiscretePlot[Table[LogLikelihood[𝒟, data], {p, r = {0.05, 0.1, 0.15}}]//Evaluate, {n, 1, 15}, ExtentSize -> 1 / 2, AxesOrigin -> {0, 0}, PlotLegends -> (Row[{"p = ", #}]& /@ r)]Solve for the Poisson maximum log-likelihood estimate in closed form:
loglike = LogLikelihood[PoissonDistribution[μ], {x, y, z}]Solve[Refine[D[loglike, μ] == 0, x > 0 && y > 0 && z > 0], μ]Compute a maximum log-likelihood estimate directly:
data = RandomVariate[GeometricDistribution[1 / 2], 500];loglfun[p_] = LogLikelihood[GeometricDistribution[p], data];{maxVal, maxPar} = FindMaximum[{loglfun[p], 0 < p < 1}, p]Label the optimal point on a plot of the log-likelihood function:
Plot[loglfun[p], {p, .01, .99}, Epilog -> {Directive[Red, PointSize[.03]], Point[{p /. maxPar, maxVal}]}]Estimate the variance of the MLE estimator as the reciprocal of the expectation of second derivative of the log-likelihood function with respect to parameters:
Expectation[-D[Refine[LogLikelihood[GeometricDistribution[p], {x}], x ≥ 0], {p, 2}] /. maxPar, xdata]^-1Properties & Relations (5)
LogLikelihood is the sum of logs of PDF values for data:
𝒟 = ChiSquareDistribution[10];
data = RandomVariate[𝒟, 100];LogLikelihood[𝒟, data]Total[Log[PDF[𝒟, data]]]LogLikelihood is the log of Likelihood:
𝒟 = BetaDistribution[2, 6];
data = RandomVariate[𝒟, 100];llhd = Likelihood[𝒟, data]Log[llhd]LogLikelihood[𝒟, data]EstimatedDistribution estimates parameters by maximizing the log‐likelihood:
𝒟 = PoissonDistribution[μ];
data = RandomVariate[𝒟 /. μ -> 10, 100];EstimatedDistribution[data, 𝒟]FindDistributionParameters gives the parameter estimates as rules:
muhat = FindDistributionParameters[data, 𝒟]Visualize the log‐likelihood function near the optimal value:
Show[Plot[LogLikelihood[𝒟, data], {μ, 8, 12}], Graphics[{PointSize[Medium], Red, Point[{μ, LogLikelihood[𝒟, data]} /. muhat]}]]Log-likelihood of a process can be computed using its slice distribution:
proc = ARMAProcess[{.3, -.2}, {.8}, .5];
data = RandomFunction[proc, {100}];LogLikelihood[proc, data]dist = proc[data["Times"]];LogLikelihood[dist, data["ValueList"]]phi = {{{.2, .1}, {.3, .1}}};
sigma = {{1, .3}, {.3, .5}};
proc = ARProcess[phi, sigma];
data = RandomFunction[proc, {100}, 2];LogLikelihood[proc, data]dist = proc[data["Times"]];Vectorize the path values for use in the LogLikelihood of the time slice distribution:
LogLikelihood[dist, Flatten /@ data["ValueList"]]Given a sample from a distribution , the difference between values of LogLikelihood for and the MLE estimated distribution doubled is random and follows ChiSquareDistribution with degrees of freedom equal to the number of distribution parameters:
ℓ[data_, s𝒟_, 𝒟_] := 2(LogLikelihood[EstimatedDistribution[data, s𝒟], data] - LogLikelihood[𝒟, data])Draw samples from a 5-parameter binormal distribution in 1000 batches:
𝒟 = BinormalDistribution[{1, 1}, {2, 2}, 2 / 3];
batches = RandomVariate[𝒟, {1000, 1000}];Compute the difference of log-likelihoods for each batch:
gaps = ℓ[#, BinormalDistribution[{m1, m2}, {s1, s2}, rho], 𝒟]& /@ batches;Check that the log-likelihood differences are consistent with 
‐distribution:
QuantilePlot[gaps, ChiSquareDistribution[5]]DistributionFitTest[gaps, ChiSquareDistribution[5], "TestConclusion"]Possible Issues (1)
Log-likelihood of a continuous parametric process may be undefined:
proc = GeometricBrownianMotionProcess[1, 3, 1 / 4];
data = RandomFunction[proc, {0, 0.2, .1}];LogLikelihood[proc, data]
This is due to degenerate slice distribution at time 0:
Plot[Table[PDF[proc[t], x], {t, r = {10^-4, 10^-3, 10^-2, 10^-1}}]//Evaluate, {x, 0, 0.6}, PlotRange -> All, PlotLegends -> (Row[{"t = ", #}]& /@ N@r)]data = RandomFunction[proc, {0.1, 1, .1}];
LogLikelihood[proc, data]Neat Examples (2)
Visualize isosurfaces for an exponential power log‐likelihood:
data = {2.03, 2.92, 3.32, 1.51, 2.28, 2.13, 2.09, 1.12, 0.73, 1.06};ContourPlot3D[LogLikelihood[ExponentialPowerDistribution[k, μ, σ], data]//Evaluate, {k, 0.5, 2}, {μ, 2, 3.5}, {σ, 1.5, 3}]Visualize isosurfaces for a bivariate normal log‐likelihood:
data = {{2.03, 2.92}, {3.32, 1.51}, {2.28, 2.13}, {2.09, 1.12}, {0.73, 1.06}};ContourPlot3D[Evaluate@LogLikelihood[BinormalDistribution[{μ, μ}, {σ, σ}, ρ], data], {μ, 1.5, 3.5}, {σ, 2.5, 4.5}, {ρ, -.75, .75}]Text
Wolfram Research (2010), LogLikelihood, Wolfram Language function, https://reference.wolfram.com/language/ref/LogLikelihood.html (updated 2014).
CMS
Wolfram Language. 2010. "LogLikelihood." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2014. https://reference.wolfram.com/language/ref/LogLikelihood.html.
APA
Wolfram Language. (2010). LogLikelihood. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/LogLikelihood.html