TextPosition
Listing of Text Content Types »TextPosition[text,form]
gives a list of the starting and ending positions at which instances of form occur in text.
TextPosition[text,{form1,form2,…}]
gives an association of results for all the types formi.
TextPosition[text,formspec,n]
gives the positions of the first n cases found.
Details and Options
- In TextPosition[text,form], text can be a string, a file with plain text, a ContentObject expression or a list of these text objects.
- TextPosition[{text1,text2,…},…] gives cases for each texti.
- Identification type form can be:
-
"type" any text content type (e.g. "Noun", "City") Entity[…,…] a specific entity of a text content type form1form2… form matching any of the formi Containing[outer,inner] forms of type outer containing type inner Verbatim["string"] a specific string to be matched exactly pattern a string pattern to be matched - Possible choices for the property prop are:
-
"String" string of the identified text (default) "Position" start and end position of the string in text "Probability" estimated probability that the identification is correct "Interpretation" standard interpretation of the identified string "Snippet" a snippet around the identified string "HighlightedSnippet" a snippet with the identified string highlighted f apply f to the association containing all properties {prop1,prop2,…} a list of property specifications - The following options can be given:
-
AcceptanceThreshold Automatic minimum probability to accept identification PerformanceGoal Automatic favor algorithms with specific advantages TargetDevice "CPU" whether CPU or GPU computation should be used for entity detection VerifyInterpretation False whether interpretability should be verified
Examples
open all close allBasic Examples (6)
Find mentions of cities in a piece of text:
TextPosition["NYC, Los Angeles, and Chicago are the largest cities in the United States of America in 2018.", "City"]TextPosition["The quick brown fox jumps over the lazy dog.", "Noun"]TextPosition["The shirt cost $50 in America, but only 5€ in Italy.", "CurrencyAmount"]Find positions of cities, countries and dates in text:
TextPosition["NYC, Los Angeles, and Chicago are the largest cities in the United States of America in 2018.", {"City", "Country", "Date"}]Function[text, Map[StringTake[text, #]&, TextPosition[text, {"City", "Country", "Date"}]]]["NYC, Los Angeles, and Chicago are the largest cities in the United States of America in 2018."]Find all the locations and get their positions:
TextPosition["NYC, Los Angeles, and Chicago are the largest cities in the USA in 2018.", "Location"]Function[text, StringTake[text, TextPosition[text, "Location"]]]["NYC, Los Angeles, and Chicago are the largest cities in the USA in 2018."]Find all references to New York City in a text:
TextPosition["I love New York - I ❤ NYC", Entity["City", {"NewYork", "NewYork", "UnitedStates"}]]Scope (4)
ContentObject and Files (2)
Find instances of colors in a ContentObject:
doc = TextSearch["ExampleData/Text", "dog"][1]TextPosition[doc, "Color"]StringTake[doc["Plaintext"], TextPosition[doc, "Color"]]Find quantities in a File:
file = TextSearch["ExampleData/Text", "dog"][1, "Location"]TextPosition[file, "Color"]StringTake[Import[file, "Plaintext"], %]Alternatives and Containing (2)
Use Alternatives to match multiple types:
TextPosition["John and Mary went to the store.", "Noun" | "Verb"]TextPosition["John and Mary went to the store.", "Noun" | "ProperNoun" | "Verb"]Find all sentences in a string that contain currency amounts:
TextPosition["I have a fairly clear idea of what I will buy at the store. I want shoes, a computer, and a jacket. The computer will be the most expensive, and will cost over $1000.", Containing["Sentence", "CurrencyAmount"]]Find all sentences in a string that contain countries:
TextPosition["On vacation, I first went to France, then I went to Belgium. The food was amazing in both countries.", Containing["Sentence", "Country"]]Combine Alternatives and Containing to form highly structured queries:
TextPosition["I have a fairly clear idea of what I will buy at the store. I want shoes, a computer, and a jacket. The computer will be the most expensive, and will cost over $1000. John will like my computer.", Containing["Sentence", "CurrencyAmount" | "ProperNoun"]]Options (3)
AcceptanceThreshold (1)
By default, all the detected entities have an estimated probability higher than 0.5:
TextPosition[ExampleData[{"Text", "JFKInaugural"}], {"Country", "Date", "Person"}]Map[StringTake[ExampleData[{"Text", "JFKInaugural"}], #]&, TextPosition[ExampleData[{"Text", "JFKInaugural"}], {"Country", "Date", "Person"}]]Get only the entities that are highly probable to be correct by setting a high AcceptanceThreshold:
TextPosition[ExampleData[{"Text", "JFKInaugural"}], {"Country", "Date", "Person"}, "AcceptanceThreshold" -> 0.9]Map[StringTake[ExampleData[{"Text", "JFKInaugural"}], #]&, TextPosition[ExampleData[{"Text", "JFKInaugural"}], {"Country", "Date", "Person"}, "AcceptanceThreshold" -> 0.9]]PerformanceGoal (1)
Using PerformanceGoal->"Speed" can help to have faster detection, at the cost of lower accuracy:
AbsoluteTiming@TextPosition["My favourite cities are New York and Foix", "City"]AbsoluteTiming@TextPosition["My favourite cities are New York and Foix", "City", PerformanceGoal -> "Speed"]VerifyInterpretation (1)
By default, some entities cannot be interpreted, either because they are not correct or because they are not yet in the knowledgebase. In these cases, a string is returned instead of an interpretation:
TextPosition["We visited Toulouse and Foix in France.", "City"]AssociationMap[Interpreter["City"], StringTake["We visited Toulouse and Foix in France.", {{12, 19}, {25, 28}}]]Use VerifyInterpretation to filter out the entities that cannot be interpreted:
TextPosition["We visited Toulouse and Foix in Midi-Pyrénées in France.", "City", VerifyInterpretation -> True]Applications (6)
Word and Sentence Segmentation (2)
Word segmentation preserves syntactic elements such as email addresses, URLs and Twitter handles:
wordPositions = TextPosition["His email address is user@domain.com and Twitter handle is @username. http://www.wolfram.com is a useful resource for Wolfram Language programmers.", "Word"]TextElement@StringTake["His email address is user@domain.com and Twitter handle is @username. http://www.wolfram.com is a useful resource for Wolfram Language programmers.", wordPositions]All the non-whitespace characters are grabbed with forms "Word" and "Punctuation":
tokenPositions = TextPosition["Washington D.C. is the capital of the United States. Mr. Anthony A. Williams was the mayor.", "Word" | "Punctuation"]TextElement@StringTake["Washington D.C. is the capital of the United States. Mr. Anthony A. Williams was the mayor.", tokenPositions]Sentence segmentation intelligently ignores acronyms and other misleading boundaries:
text = "Washington D.C. is the capital of the United States. Mr. Fox is the CEO of the company. She co-founded the company with Mrs. Smith.";
sentencePositions = TextPosition[text, "Sentence"]TextElement@StringTake[text, sentencePositions]Parts of Speech (2)
Return all words of a given part of speech:
TextPosition["John and Mary went to the new store.", "Noun"]TextPosition["John and Mary went to the new store.", "Verb"]TextPosition["John and Mary went to the new store.", "Preposition"]Make a table of word clouds from parts of speech:
alice = ExampleData[{"Text", "AliceInWonderland"}];partsOfSpeech = TextPosition[alice, {"Noun", "Verb", "Adjective", "Adverb"}]Grid @Partition[KeyValueMap[Labeled[WordCloud[#2, ImageSize -> 200], #1]&, Map[StringTake[alice, #]&, partsOfSpeech]], 2]Entities and Interpretable Objects (2)
text = "On vacation, I first went to France, then I went to Belgium.";
TextPosition[text, "Country"]StringTake[text, TextPosition[text, "Country"]]Return interpreted strings as Entity objects:
AssociationMap[Interpreter["Country"], {"France", "Belgium"}]Find currency amounts in a Wikipedia article:
dollarstore = WikipediaData@First@WikipediaSearch["Variety Store"];currencies = TextPosition[dollarstore, "CurrencyAmount"]Interpreter["CurrencyAmount"][StringTake[dollarstore, currencies]]Properties & Relations (1)
TextPosition handles the same types as TextCases and TextContents, and always identify the same substrings as these functions for a given type:
TextContents["Boston, Worcester, and Springfield are the largest cities in Massachusetts.", "City"]TextCases["Boston, Worcester, and Springfield are the largest cities in Massachusetts.", "City"]TextPosition["Boston, Worcester, and Springfield are the largest cities in Massachusetts.", "City"]TextCases is a generalization of TextPosition:
TextCases["Boston, Worcester, and Springfield are the largest cities in Massachusetts.", "City" -> "Position"]TextPosition["Boston, Worcester, and Springfield are the largest cities in Massachusetts.", {"City", "AdministrativeDivision"}]TextCases["Boston, Worcester, and Springfield are the largest cities in Massachusetts.", {"City", "AdministrativeDivision"} -> "Position"]Text
Wolfram Research (2015), TextPosition, Wolfram Language function, https://reference.wolfram.com/language/ref/TextPosition.html (updated 2019).
CMS
Wolfram Language. 2015. "TextPosition." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2019. https://reference.wolfram.com/language/ref/TextPosition.html.
APA
Wolfram Language. (2015). TextPosition. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/TextPosition.html