WOLFRAM

Wolfram Language & System Documentation Center
"ClickLocations" (Question Interface Type)

"ClickLocations" (Question Interface Type)

"ClickLocations"

provides an interface for selecting locations on a background.

Properties

  • In QuestionInterface["ClickLocations",props], props is an Association that supports the following keys:
  • "Prompt"Nonetext asking the user a question
    Backgroundrequiredimage to place the items on
    "Count"Automaticnumber of items
    "Labels"Nonelist with corresponding labels for each item

Corresponding AssessmentFunction

  • "ClickLocations" is not an automatically assigned interface type for an AssessmentFunction. It must be specified in QuestionInterface.
  • When the Background is an image, values in the AssessmentFunction correspond to pixel locations.
  • By default, the "Count" matches the length of the correct answer in the AssessmentFunction.
  • Clicking exact locations is not practical, therefore, a nonzero tolerance and a "ComparisonMethod" such as "Point" should be set in the AssessmentFunction.
  • Locations in the answer key should be given as lists of coordinate pairs: AssessmentFunction[{{{chx1, chy1}, {chx2, chy2},…} 1},…]
  • When no "Labels" are provided, "ListAssessment" must be set to "AllElementsOrderless": AssessmentFunction[…, <|"ComparisonMethod"->"Vector","ListAssessment" "AllElementsOrderless"|>, Tolerance tol]
  • If "Labels" are provided, "ListAssessment" must be set to "AllElementsOrdered" and the "Labels" should match the order of locations in the AssessmentFunction: AssessmentFunction[…, <|"ComparisonMethod"->"Vector","ListAssessment" "AllElementsOrdered"|>, Tolerance tol]

Examples

open all close all

Basic Examples  (3)

Create a click-locations question without assessment:

Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "How are you feeling today?", "Background" -> [image]|>]]

Create a click-locations question with assessment:

Wolfram Language code: stoplights = [image]; pts = ImageCases[stoplights, Entity["Concept", "TrafficLight::b4966"] -> "Position"]
Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the stoplights", "Background" -> stoplights|>], AssessmentFunction[{pts}, <|"ComparisonMethod" -> "Vector", "ListAssessment" -> "AllElementsOrderless"|>, Tolerance -> 80]]

Create a click-locations question with labels. Use the coordinate tool to locate the missing labels in the image:

Wolfram Language code: brain = [image];
Wolfram Language code: labellocations = <|"Brain Stem" -> {340, 106}, "Temporal" -> {228, 202}, "Frontal" -> {154, 700}, "Parietal" -> {852, 696}, "Occipital" -> {900, 548}, "Cerebellum" -> {846, 154}|>
Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the labels in the indicated brain lobes", "Background" -> Image[brain, ImageSize -> Large], "Labels" -> Keys[labellocations]|>], AssessmentFunction[{Values[labellocations]}, <|"ComparisonMethod" -> "Vector", "ListAssessment" -> "AllElementsOrdered"|>, Tolerance -> 100]]

Scope  (2)

Use the Coordinates Tool to extract some pixel positions from an image:

Wolfram Language code: img = Image[ExampleData[{"TestImage", "House2"}], ImageSize -> Medium]
Wolfram Language code: locations = {{304, 110}, {112, 118}}

Create a question object with assessment using the image and chosen locations:

Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the wheels of the car", "Background" -> img|>], AssessmentFunction[{locations}, <|"ComparisonMethod" -> "Vector", "ListAssessment" -> "AllElementsOrderless"|>, Tolerance -> 20]]

Create an image for a geometry problem:

Wolfram Language code: triangle = Image[Graphics[{FaceForm[White], EdgeForm[Black], Triangle[{{0, 0}, {1, 2}, {2, 0.5}}], Dashed, Blue, TriangleConstruct[{{0, 0}, {1, 2}, {2, 0.5}}, "AngleBisectingCevian"], Red, TriangleConstruct[{{0, 0}, {2, 0.5}, {1, 2}}, "Altitude"], Green, TriangleConstruct[{{1, 2}, {0, 0}, {2, 0.5}}, "Median"]}], ImageSize -> Medium]

Use the Coordinates Tool to find positions, then create a question with labels, awarding 20 points for a correct answer:

Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the non vertex intersection of each line with the triangle", "Background" -> triangle, "Labels" -> {"Angle Bisector", "Altitude", "Median"}|>], AssessmentFunction[{{{262, 75}, {147, 286}, {353, 296}} -> 20}, <|"ComparisonMethod" -> "Vector", "ListAssessment" -> "AllElementsOrdered"|>, Tolerance -> 50]]

Applications  (1)

Create an image containing many flags:

Wolfram Language code: flags = ImageCollage[EntityValue[#, "Flag"]& /@ EntityList@EntityClass["Country", "EuropeanUnion"], ImageSize -> Medium, ImagePadding -> 2]

Find specific flags using the Coordinates Tool and create a question:

Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the tags on the corresponding flags", "Background" -> flags, "Labels" -> {"Italy", "Portugal", "Spain"}|>], AssessmentFunction[{{{391, 790}, {925, 804}, {914, 431}} -> 20}, <|"ComparisonMethod" -> "Vector", "ListAssessment" -> "AllElementsOrdered"|>, Tolerance -> 50]]

Possible Issues  (1)

The "Background" must be an Image:

Wolfram Language code: gr = Graphics[{Thick, Green, Rectangle[{0, -1}, {2, 1}], Red, Disk[], Blue, Circle[{2, 0}], Yellow, Polygon[{{2, 0}, {4, 1}, {4, -1}}], Purple, Arrowheads[Large], Arrow[{{4, 3 / 2}, {0, 3 / 2}, {0, 0}}], Black, Dashed, Line[{{-1, 0}, {4, 0}}]}]
Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the center of the red circle", "Background" -> gr|>]]

Use Image or Rasterize to convert Graphics into an Image:

Wolfram Language code: QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> "Locate the center of the red circle", "Background" -> Image[gr, ImageSize -> Medium]|>]]

Interactive Examples  (1)

Find locations of several types of objects in an image:

Wolfram Language code: transportations = [image];
Wolfram Language code: spots = ImageCases[transportations, {Entity["Concept", "Aeroplane::239dv"], Entity["Concept", "Motorcycle::4gd85"], Entity["Concept", "Boat::fx4n4"], Entity["Concept", "Motortruck::9c7br"]} -> "Position", AcceptanceThreshold -> .3]

Create a question generator with a click-locations question object using the location data:

Wolfram Language code: transportationTest = QuestionGenerator[{"transport" :> RandomChoice[Normal@spots]}, QuestionObject[QuestionInterface["ClickLocations", <|"Prompt" -> StringForm["Find all `` in the image", Pluralize@CommonName@Keys@#transport], "Background" -> transportations|>], AssessmentFunction[{Values@#transport -> 1}, <|"ComparisonMethod" -> "Vector", "ListAssessment" -> "AllElementsOrderless"|>, Tolerance -> 200]]&]
Wolfram Language code: transportationTest[]