TrainTextContentDetector
TrainTextContentDetector[{text1{span1class1,…},…}]
trains a ContentDetectorFunction[…] based on the examples given.
Details and Options
- TrainTextContentDetector attempts to train an entity detector using the provided training data.
- TrainTextContentDetector is typically used to create a custom tagging function for text, with only limited training data.
- TrainTextContentDetector returns a ContentDetectorFunction[…] that can then be applied to new input texts.
- The spans spani should be given as lists with starting and ending positions.
- The classes classi can be arbitrary expressions.
- TrainTextContentDetector[…][text] can be used to detect objects in text and returns the result as a list of associations of the form <"String"str,"Class"class>.
- TrainTextContentDetector supports the following options:
-
PerformanceGoal Automatic favor settings with specific advantages ProgressReporting Automatic how to report progress during training RandomSeeding 1234 how to seed pseudorandom generators internally TargetDevice "CPU" the target device on which to perform training TimeGoal Automatic number of seconds to train for ValidationSet None the set of data on which to evaluate the model during training
Examples
open all close allBasic Examples (1)
Train a simple entity detector:
detector = TrainTextContentDetector[{"The big table was red" -> {{9, 13} -> "Furniture"},
"Four chairs were surrounding the table" -> {{6, 11} -> "Furniture", {34, 38} -> "Furniture"}
, "My chair is very comfortable" -> {{4, 8} -> "Furniture"},
"I'm sitting on a chair" -> {{18, 22} -> "Furniture"},
"The cat sat on the table" -> {{20, 24} -> "Furniture"}, "I don't think we can fit a table in that room" -> {{28, 32} -> "Furniture"}}]Apply the detector on a new text:
detector["I want two tables and two chairs"]Scope (1)
Train a detector to identify multiple classes:
detector = TrainTextContentDetector[{
"I like banana" -> {{8, 13} -> "Fruit"},
"He eat apples watching TV" -> {{8, 13} -> "Fruit"},
"She is enjoying raspberries" -> {{17, 27} -> "Fruit"},
"I play soccer" -> {{8, 13} -> "Sport"},
"The basketball game is tomorrow" -> {{5, 14} -> "Sport"},
"My daughter is very talented at football" -> {{33, 40} -> "Sport"},
"I watch TV" -> {},
"Let's have a beer" -> {{14, 17} -> "Drink"},
"My children like soda" -> {{18, 21} -> "Drink"},
"They bought wine" -> {{13, 16} -> "Drink"}
}]Apply the detector on new texts:
detector[{"She likes eating ananas", "I enjoy playing handball", "I will drink a cider"}]Ask for particular properties about detected snippets of text:
detector[{"She likes eating ananas", "I enjoy playing handball", "I will drink a cider"}, {"HighlightedSnippet", "Class", "Probability"}]Options (4)
PerformanceGoal (1)
Use PerformanceGoal"Quality" to emphasize the quality of the result:
training = {"The big table was red" -> {{9, 13} -> "Furniture"}, "Four chairs were surrounding the table" -> {{6, 11} -> "Furniture", {34, 38} -> "Furniture"}, "My chair is very comfortable" -> {{4, 8} -> "Furniture"}, "I'm sitting on a chair" -> {{18, 22} -> "Furniture"}, "The cat sat on the table" -> {{20, 24} -> "Furniture"}, "I don't think we can fit a table in that room" -> {{28, 32} -> "Furniture"}};AbsoluteTiming[TrainTextContentDetector[training, PerformanceGoal -> "Quality"]]Use PerformanceGoal"Speed" to emphasize the speed of computation:
TrainTextContentDetector[training, PerformanceGoal -> "Speed"]//AbsoluteTimingProgressReporting (1)
By default, progress is reported in a panel:
TrainTextContentDetector[{"The big table was red" -> {{9, 13} -> "Furniture"}}]
| |
Use ProgressReportingFalse to avoid displaying the progress panel:
TrainTextContentDetector[{"The big table was red" -> {{9, 13} -> "Furniture"}}, ProgressReporting -> False]TimeGoal (1)
The training time can be influenced by several factors, such as the number of examples and classes:
TrainTextContentDetector[{"The big table was red" -> {{9, 13} -> "Furniture"}}]//AbsoluteTimingUse TimeGoal to specify a target time for the training:
TrainTextContentDetector[{"The big table was red" -> {{9, 13} -> "Furniture"}}, TimeGoal -> 6]//AbsoluteTimingValidationSet (1)
By default, no validation is performed on the detector. Use ValidationSet to provide validation data:
TrainTextContentDetector[{"The big table was red" -> {{9, 13} -> "Furniture"},
"Four chairs were surrounding the table" -> {{6, 11} -> "Furniture", {34, 38} -> "Furniture"}
, "My chair is very comfortable" -> {{4, 8} -> "Furniture"}}, ValidationSet -> {"I'm sitting on a chair" -> {{18, 22} -> "Furniture"},
"The cat sat on the table" -> {{20, 24} -> "Furniture"}, "I don't think we can fit a table in that room" -> {{28, 32} -> "Furniture"}}]Text
Wolfram Research (2021), TrainTextContentDetector, Wolfram Language function, https://reference.wolfram.com/language/ref/TrainTextContentDetector.html.
CMS
Wolfram Language. 2021. "TrainTextContentDetector." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/TrainTextContentDetector.html.
APA
Wolfram Language. (2021). TrainTextContentDetector. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/TrainTextContentDetector.html