WOLFRAM

Wolfram Language & System Documentation Center

"NaiveBayes" (Machine Learning Method)

  • Method for Classify.
  • Determines the class using Bayes's theorem and assuming that features are independent given the class.

Details & Suboptions

  • Naive Bayes is a classification technique based on Bayes's theorem which assumes that the features are independent given the class. The class probabilities for a given example are then: , where is the probability distribution of feature given the class, and is the prior probability of the class. Both distributions are estimated from the training data. In the current implementation, distributions are modeled using a piecewise-constant function (i.e a variable-width histogram).
  • The following suboption can be given
  • "SmoothingParameter" .2regularization parameter

Examples

open all close all

Basic Examples  (2)

Train a classifier function on labeled examples:

Wolfram Language code: c = Classify[{1, 2, 3, 4} -> {1, 1, 2, 2}, Method -> "NaiveBayes"]

Obtain information about the classifier:

Wolfram Language code: Information[c]

Classify a new example:

Wolfram Language code: c[1.3]

Generate some normally distributed data:

Wolfram Language code: sampledata[center_] := RandomVariate[MultinormalDistribution[center, IdentityMatrix[2]], 200];
Wolfram Language code: clusters = sampledata /@ {{1.5, 1}, {-1.5, 1}, {0, -3}};

Visualize it:

Wolfram Language code: ListPlot[clusters, PlotStyle -> Darker@{Yellow, Blue, Green}]

Train a classifier on this dataset:

Wolfram Language code: c = Classify[<|Yellow -> clusters[[1]], Blue -> clusters[[2]], Green -> clusters[[3]]|>, Method -> "NaiveBayes"]

Plot the training set and the probability distribution of each class as a function of the features:

Wolfram Language code: Show[ Plot3D[{ c[{x, y}, "Probability" -> Yellow], c[{x, y}, "Probability" -> Blue], c[{x, y}, "Probability" -> Green] }, {x, -4, 4}, {y, -5, 4}, Exclusions -> None], ListPointPlot3D[Map[Append[#, 1]&, clusters, {2}], PlotStyle -> {Yellow, Blue, Green}]]

Options  (2)

"SmoothingParameter"  (2)

Train a classifier using the "SmoothingParameter" suboption:

Wolfram Language code: Classify[{1, 2, 3.4, -5, -6} -> {"positive", "positive", "positive", "negative", "negative"}, Method -> {"NaiveBayes", "SmoothingParameter" -> 2}]

Train several classifiers on the "FisherIris" dataset by using different settings of the "SmoothingParameter" option:

Wolfram Language code: trainingset = ExampleData[{"MachineLearning", "FisherIris"}, "TrainingData"];
Wolfram Language code: {c1, c2, c3, c4} = Classify[trainingset, Method -> {"NaiveBayes", "SmoothingParameter" -> #}]& /@ {.5, 20, 40, 90};

Evaluate these classifiers on a data point that is unlike points from the training set and compare the class probability for class "setosa":

Wolfram Language code: minmaxs = MinMax /@ (Transpose[trainingset[[All, 1]]])
Wolfram Language code: #[{4, 5, 9, -2}, "Probability" -> "setosa"]& /@ {c1, c2, c3, c4}