WOLFRAM

Wolfram Language & System Documentation Center

"GaussianProcess" (Machine Learning Method)

  • Method for Predict.
  • Infers values by conditioning a Gaussian process on the training data.

Details & Suboptions

  • The "GaussianProcess" method assumes that the function to be modeled has been generated from a Gaussian process. The Gaussian process is defined by its covariance function (also called kernel). In the training phase, the method will estimate the parameters of this covariance function. The Gaussian process is then conditioned on the training data and used to infer the value of a new example using a Bayesian inference.
  • The following options can be given:
  • AssumeDeterministic Falsewhether or not the function should be assumed to be deterministic
    "CovarianceType" Automaticthe covariance type to use
    "EstimationMethod""MaximumPosterior"the method to infer the values
    "OptimizationMethod"Automaticthe optimization method to estimate parameters
  • Possible settings for "CovarianceType" include:
  • "SquaredExponential"exponential kernel
    "HammingDistance"exponential kernel for nominal variables
    "Periodic"periodic kernel
    "RationalQuadratic"rational quadratic kernel
    "Linear"linear kernel
    "Matern5/2"Matérn kernel with exponent 5/2
    "Matern3/2"Matérn kernel with exponent 3/2
    "Composite"a composition of the previous kernels
    assocspecify a different kernel for each feature type
  • In Method{"GaussianProcess", "CovarianceType"assoc}, assoc needs to be of the form <|"Numerical" kernel1,"Nominal"kernel2|>.
  • Possible settings for "EstimationMethod" include:
  • "MaximumPosterior"maximize the posterior distribution
    "MaximumLikelihood"maximize the likelihood
    "MeanPosterior"mean of the posterior distribution
  • Possible settings for "OptimizationMethod" include:
  • "SimulatedAnnealing"uses simulated annealing to find the minimum
    "FindMinimum"uses FindMinimum to find the minimum

Examples

open all close all

Basic Examples  (2)

Train a classifier function on labeled examples:

Wolfram Language code: p = Predict[{1, 2, 3, 4} -> {.3, .4, .6, 9}, Method -> "GaussianProcess"]

Obtain information about the predictor:

Wolfram Language code: Information[p]

Predict a new example:

Wolfram Language code: p[1.3]

Train a predictor on labeled examples:

Wolfram Language code: data = {-1.2 -> 1.2, 1.4 -> 1.4, 3.1 -> 1.8, 4.5 -> 1.6};p = Predict[data, Method -> "GaussianProcess"]

Compare the data with the predicted values and look at the standard deviation:

Wolfram Language code: Show[Plot[{p[x], p[x] + StandardDeviation[p[x, "Distribution"]], p[x] - StandardDeviation[p[x, "Distribution"]]}, {x, -2, 6}, PlotStyle -> {Blue, Gray, Gray}, Filling -> {2 -> {3}}, Exclusions -> False, PerformanceGoal -> "Speed", PlotLegends -> {"Prediction", "Confidence Interval"}], ListPlot[List@@@data, PlotStyle -> Red, PlotLegends -> {"Data"}]]

Options  (4)

AssumeDeterministic  (2)

Assume deterministic data and train a predictor on it:

Wolfram Language code: c = Predict[{1, 2, 3, 4} -> {.3, .4, .6, 9}, Method -> {"GaussianProcess", AssumeDeterministic -> True}]

Generate some labeled data normally distributed around a polynomial function:

Wolfram Language code: trainingset = Table[x -> x ^ 2 + RandomVariate[NormalDistribution[0, 3]], {x, -3, 3, .5}]; ListPlot[List@@@trainingset]

Train a predictor by assuming the data is not deterministic:

Wolfram Language code: p1 = Predict[trainingset, Method -> {"GaussianProcess", AssumeDeterministic -> False, "OptimizationMethod" -> "SimulatedAnnealing"}]

Train a predictor by assuming the data is deterministic:

Wolfram Language code: p2 = Predict[trainingset, Method -> {"GaussianProcess", AssumeDeterministic -> True, "OptimizationMethod" -> "SimulatedAnnealing"}]

Compare the results:

Wolfram Language code: Show[ListPlot[List@@@trainingset], Plot[{p1[x], p2[x]}, {x, -3, 3}]]

"CovarianceType"  (2)

Use a specific covariance type to train a predictor:

Wolfram Language code: c = Predict[{1, 2, 3, 4} -> {.3, .4, .6, 9}, Method -> {"GaussianProcess", "CovarianceType" -> "Periodic" + "SquaredExponential"}]

Generate a labeled training set and visualize it:

Wolfram Language code: trainingset = Table[x -> Sin[x] + RandomReal[1], {x, -10, 10, .2}]; ListPlot[List@@@trainingset]

Train two predictors using different covariance types:

Wolfram Language code: {p1, p2} = Predict[trainingset, Method -> {"GaussianProcess", "CovarianceType" -> #}]& /@ {"SquaredExponential", "Periodic"};

Train a third predictor using the "Composite" covariance type:

Wolfram Language code: p3 = Predict[trainingset, Method -> {"GaussianProcess", "CovarianceType" -> "Composite"}]

Look at the kernel type that has been found:

Wolfram Language code: p3[[1, "Model", "TrainedGPModel", "CovarianceFunction", Key[ Method]]]

Get its internal parameters:

Wolfram Language code: Information[p3, "InternalParameters"]

Compare the three results:

Wolfram Language code: Show[ListPlot[List@@@trainingset], Plot[{p1[x], p2[x], p3[x]}, {x, -10, 10}]]