Wolfram Language & System Documentation Center

ImageKeypoints

ImageKeypoints[image]

finds key features in image and returns their coordinates.

ImageKeypoints[image,prop]

gives the specified property prop for each keypoint.

ImageKeypoints[video,…]

finds keypoints in frames of video.

Details and Options

Image keypoints are distinct positions on an image highlighting specific image features such as shape, contrast, orientation. Corresponding keypoints are typically used for aligning images.
ImageKeypoints[image] finds image keypoints and returns their positions as a list of {{x₁,y₁},{x₂,y₂},…}.
The following properties can be specified:

	"Confidence"	blob response, given as a positive number
	"ContrastSign"	if the keypoint is lighter than its surroundings, otherwise
	"Descriptor"	keypoint descriptor
	"Orientation"	orientation angle, given in radians
	"OrientedDescriptor"	keypoint-oriented descriptor
	"PixelPosition"	pixel coordinates {x,y} in the range ,
	"Position"	image coordinates {x,y} in the range , (default)
	"Scale"	keypoint scale

ImageKeypoints sorts the results based on the "Confidence" property of the keypoints.
ImageKeypoints[image,{prop₁,prop₂,…}] returns multiple properties.
The feature descriptors returned by ImageKeypoints are numerically robust against translation, rotation, and scale changes.
The following options can be specified:

KeypointStrength	0	minimum strength of the keypoints
Masking	All	region of interest
MaxFeatures	All	maximum number of keypoints
Method	"SURF"	type of keypoint to return

With a setting MaxFeatures->n, at most n keypoints with largest "Confidence" are returned.
Possible method settings include:

	"AGAST"	Adaptive and Generic Accelerated Segment Test
	"AKAZE"	Accelerated KAZE and binary descriptors
	"BRISK"	Binary Robust Invariant Scalable Keypoints
	"FAST"	Features from Accelerated Segment Test
	"KAZE"	nonlinear scale-space detector and descriptor
	"ORB"	FAST detector and Binary Robust Independent Elementary Features (BRIEF) descriptor
	"SIFT"	Scale-Invariant Feature Transform detector and descriptor
	"RootSIFT"	SIFT keypoints with an improved descriptor
	"SURF"	Speeded-Up Robust Features

When a property is not available with a specified method, the corresponding element in the result is set to Missing["NotAvailable"]. »

Examples

open all close all

Basic Examples (2)

Find keypoints in an image:

Wolfram Language code: i = [image];

Wolfram Language code: ImageKeypoints[i]//Short

Highlight keypoints on the image:

Wolfram Language code: HighlightImage[i, %]

Count the number of points of interest in an image:

Wolfram Language code: ImageKeypoints[[image]]//Length

Scope (10)

Keypoints of a grayscale image:

Wolfram Language code: ImageKeypoints[[image]]//Short

Keypoints of a color image:

Wolfram Language code: ImageKeypoints[[image]]//Short

Find keypoints in an image:

Wolfram Language code: ImageKeypoints[[image]]

Return the keypoint descriptor:

Wolfram Language code: ImageKeypoints[[image], "Descriptor"]

Compute multiple properties:

Wolfram Language code: ImageKeypoints[[image], {"Scale", "Orientation", "ContrastSign"}]//MatrixForm

Get properties "Position" and "PixelPosition":

Wolfram Language code: pos = ImageKeypoints[[image], {"Position", "PixelPosition"}]

The two corresponding coordinate systems are offset by half a pixel:

Wolfram Language code: pos[[All, 1]] + 1 / 2 == pos[[All, 2]]

Get the "Scale" in pixels:

Wolfram Language code: img = [image];

Wolfram Language code: scales = ImageKeypoints[img, "Scale"]

The scale corresponds to the size of an intrinsic region around the keypoint. Visualize the disc region
around a "SURF" keypoint:

Wolfram Language code: HighlightImage[img, {Thick, MapThread[Circle, {ImageKeypoints[img, "Position"], 2.5 * scales}]}, ImageSize -> 100]

Get the "Orientation" in radians:

Wolfram Language code: img = [image];

Wolfram Language code: orient = ImageKeypoints[img, "Orientation"]

Show keypoints with a pattern rotated based on orientation:

Wolfram Language code:

HighlightImage[img, 
	MapThread[
	{Circle[#2, 5], Line[{#2, #2 + 5 * {Cos[#1], Sin[#1]}}]}&, 
	{orient, ImageKeypoints[img]}], 
	ImageSize -> 200]

Get the "Strength" score:

Wolfram Language code:

img = [image];
strength = ImageKeypoints[img, "Confidence"];
strength//Short

Display the distribution of the strength of keypoints in an image:

Wolfram Language code: Histogram@strength

Get the "Descriptor" representing the distribution of pixel values inside a keypoint's neighborhood:

Wolfram Language code: img = [image];

Wolfram Language code: desc1 = ImageKeypoints[img, "Descriptor", MaxFeatures -> 1];

The descriptor is similar to the one of the rotated image:

Wolfram Language code: desc2 = ImageKeypoints[ImageRotate@img, "Descriptor", MaxFeatures -> 1];

Wolfram Language code: EuclideanDistance[desc1, desc2]

The "ConstrastSign" is 1 for keypoints lighter than their surroundings:

Wolfram Language code: ImageKeypoints[[image], "ContrastSign"]

The sign is -1 for keypoints darker than their surroundings:

Wolfram Language code: ImageKeypoints[[image], "ContrastSign"]

Options (11)

KeypointStrength (1)

Use a specific threshold for keypoint strength:

Wolfram Language code: ImageKeypoints[[image], KeypointStrength -> .005]

By default, all detected keypoints are returned:

Wolfram Language code: ImageKeypoints[[image]]//Length

Masking (1)

By default, all keypoints are returned:

Wolfram Language code: i = [image];

Wolfram Language code: HighlightImage[i, ImageKeypoints[i]]

Exclude keypoints in the foreground using a mask region:

Wolfram Language code: HighlightImage[i, ImageKeypoints[i, Masking -> [image]]]

MaxFeatures (1)

By default, all detected keypoints are returned:

Wolfram Language code: img = [image];

Wolfram Language code:

pos = ImageKeypoints[img];
HighlightImage[img, pos]

Get the 50 strongest keypoints:

Wolfram Language code:

pos = ImageKeypoints[img, MaxFeatures -> 50];
HighlightImage[img, pos]

Method (8)

By default, "SURF" keypoints are computed:

Wolfram Language code: i = [image];

Wolfram Language code: HighlightImage[i, ImageKeypoints[i]]

Compute and visualize a different keypoint:

Wolfram Language code: HighlightImage[i, ImageKeypoints[i, Method -> "AKAZE", MaxFeatures -> 80]]

"FAST" and "AGAST" keypoints are defined by their location and strength at scale 3.5:

Wolfram Language code: i = [image];

Wolfram Language code: ImageKeypoints[i, {"Position", "Confidence", "Scale"}, Method -> "FAST", MaxFeatures -> 50]//Short

Compute and visualize FAST keypoints:

Wolfram Language code: HighlightImage[i, ImageKeypoints[i, Method -> "FAST", MaxFeatures -> 50]]

Compute and visualize AGAST keypoints:

Wolfram Language code: HighlightImage[i, ImageKeypoints[i, Method -> "AGAST", MaxFeatures -> 50]]

"BRISK" and "ORB" keypoints are defined by their location, scale, orientation and strength:

Wolfram Language code: img = [image];

Wolfram Language code:

{brisk, orb} = ImageKeypoints[img, {"Position", "Scale", "Orientation", "Confidence"}, Method -> #]& /@ {"BRISK", "ORB"};

Wolfram Language code: RandomSample[brisk, 2]

Wolfram Language code: RandomSample[orb, 2]

Descriptors are vectors of 0s and 1s of length 512 for "BRISK" and 256 for "ORB":

Wolfram Language code: {brisk, orb} = ImageKeypoints[img, "Descriptor", Method -> #]& /@ {"BRISK", "ORB"};

Wolfram Language code: brisk[[1]]//Short

Wolfram Language code: orb[[1]]//Short

Wolfram Language code: Dimensions[#]& /@ {brisk, orb}

"AKAZE" and "KAZE" keypoints are defined by their location, scale, orientation and strength:

Wolfram Language code: img = [image];

Wolfram Language code:

{akaze, kaze} = ImageKeypoints[img, {"Position", "Scale", "Orientation", "Confidence"}, Method -> #]& /@ {"AKAZE", "KAZE"};

Wolfram Language code: RandomSample[akaze, 2]

Wolfram Language code: RandomSample[kaze, 2]

AKAZE descriptors are vectors of 0s and 1s of length 480:

Wolfram Language code: ImageKeypoints[img, "Descriptor", Method -> "AKAZE"]//Shallow

Wolfram Language code: Dimensions[%]

KAZE descriptors are vectors of 128 real numbers with unit norm:

Wolfram Language code: ImageKeypoints[img, "Descriptor", Method -> "KAZE"]//Short

Wolfram Language code: {Dimensions[%], Norm /@ RandomSample[%, 5]}

AKAZE oriented descriptors are computed without correcting for keypoints' intrinsic orientation:

Wolfram Language code: img = [image];

Wolfram Language code: {kod, orientations} = Transpose@ImageKeypoints[img, {"OrientedDescriptor", "Orientation"}, Method -> "AKAZE"];

Oriented descriptors match descriptors for keypoints with orientation close to 0:

Wolfram Language code: δ = MapThread[HammingDistance, {kod, ImageKeypoints[img, "Descriptor", Method -> "AKAZE"]}];

Wolfram Language code: Pick[orientations, δ, x_ /; x < 20]

With the "SURF" method, a keypoint is defined by its location, scale, orientation, contrast sign and strength:

Wolfram Language code: img = [image];

Wolfram Language code: surf = ImageKeypoints[img, {"Position", "Scale", "Orientation", "ContrastSign", "Confidence"}];

Wolfram Language code: RandomSample[surf, 2]

SURF descriptors consist of vectors of 64 real numbers with unit norm:

Wolfram Language code: ImageKeypoints[img, "Descriptor", Method -> "SURF"]//Short

Wolfram Language code: {Dimensions[%], Norm /@ RandomSample[%, 5]}

Oriented descriptors match descriptors for keypoints with orientation close to 0:

Wolfram Language code: {kod, orientations} = Transpose@ImageKeypoints[img, {"OrientedDescriptor", "Orientation"}];

Wolfram Language code: δ = MapThread[EuclideanDistance, {kod, ImageKeypoints[img, "Descriptor"]}];

Wolfram Language code: Pick[orientations, δ, x_ /; x < 0.15]

The "SIFT" method uses Scale-Invariant Feature Transform to find the location of image keypoints:

Wolfram Language code: img = [image];

Wolfram Language code:

ImageKeypoints[img, {"Position", "Scale", "Orientation", "Confidence"}, Method -> "SIFT"];
RandomSample[%, 2]

The descriptors use histograms of orientations to construct vectors of 128 numbers:

Wolfram Language code: desc = ImageKeypoints[img, "Descriptor", Method -> "SIFT"];

Wolfram Language code: Short /@ RandomSample[desc, 2]

Note that the norm of the descriptor is not equal to one:

Wolfram Language code: Norm /@ RandomSample[desc, 5]

The "RootSIFT" method uses the same keypoints locations as "SIFT", but improved descriptors:

Wolfram Language code: ImageKeypoints[[image], "Descriptor", Method -> "RootSIFT"]//Short

Norm of RootSIFT descriptors is equal to one:

Wolfram Language code: Norm /@ %//Short

Applications (5)

Visualize SURF keypoints using their scale, orientation and contrast:

Wolfram Language code:

img = [image];
points = ImageKeypoints[img, {"Position", "Scale", "Orientation", "ContrastSign"}, MaxFeatures -> 100];
scalefactor = 2.5;
HighlightImage[img, Table[{{If[p[[4]] == 1, Yellow, Red], Circle[p[[1]], p[[2]] * scalefactor], Line[{p[[1]], p[[1]] + p[[2]] * scalefactor * {Cos[p[[3]]], Sin[p[[3]]]}}]}}, {p, points}]]

Visualize BRISK keypoints:

Wolfram Language code: img = [image];

Wolfram Language code:

points = ImageKeypoints[img, {"Position", "Scale", "Orientation"}, Method -> "BRISK", MaxFeatures -> 50];
HighlightImage[img, {FaceForm[], Table[Rotate[{Line[{p[[1]], p[[1]] + p[[2]] * {1, 0}}], Rectangle[p[[1]] + p[[2]] * {-1, -1}, p[[1]] + p[[2]] * {1, 1}]}, p[[3]]], {p, points}]}]

Extract local patches of fixed size around detected keypoints:

Wolfram Language code:

img = [image];
ImageTrim[img, {#}, 12]& /@ ImageKeypoints[img, Method -> "KAZE", MaxFeatures -> 100]

Extract patches of size proportional to the scale of keypoints:

Wolfram Language code:

{pos, size} = Transpose@ImageKeypoints[img, {"Position", "Scale"}, Method -> "KAZE", MaxFeatures -> 100];
MapThread[ImageTrim[img, {#1}, 2 * #2]&, {pos, size}]

Use keypoints to crop an image to keep the main features:

Wolfram Language code:

imgs = {[image], [image], [image]};
ImageTrim[#, ImageKeypoints[#, KeypointStrength -> .0005], 50]& /@ imgs

Create thumbnails of uniform size:

Wolfram Language code: ImageCrop[ImageResize[#, {{240}, {180}}], {240, 180}]& /@ %

Object recognition using "bag of words" on a dataset of 5,000 images 32×32 each, belonging to 10 categories:

Wolfram Language code:

obj = ResourceObject["CIFAR-10"];
training = ResourceData[obj, "TrainingData"];
testing = Flatten[Table[training[[i + 1000 ;; i + 1099]], {i, 1, 50000, 5000}]];training = Flatten[Table[training[[i ;; i + 499]], {i, 1, 50000, 5000}]];

Compute keypoint descriptors on 256×256 images and create the codebook of visual words:

Wolfram Language code:

AbsoluteTiming[
	kp = ParallelTable[
	ImageKeypoints[ImageResize[i, 256], "Descriptor"], {i, training[[All, 1]]}
	];]

The codebook contains all image descriptors of length 64:

Wolfram Language code:

codebook = Flatten[kp, 1];
Dimensions[codebook]

Find 100 visual codewords using -means clustering:

Wolfram Language code:

AbsoluteTiming[
	clust = ClusteringComponents[codebook, 100, 1, Method -> "KMeans", PerformanceGoal -> "Speed"];]

Wolfram Language code:

codewords = Table[
	p = Flatten@Position[clust, i];
	Mean[codebook[[p]]], 
	{i, Max@clust}];

Image features are defined as the normalized counts of all the codewords:

Wolfram Language code: nf = Nearest[codewords -> Automatic];

Wolfram Language code:

imageCodewordsFeatureExtract[image_ ? ImageQ] := imageCodewordsFeatureExtract[ImageKeypoints[ImageResize[image, {256, 256}], "Descriptor"]];
imageCodewordsFeatureExtract[keypoints_List] := Module[
	{h}, 
	h = N@BinCounts[First@nf[#, 1] & /@ keypoints, {1, Length@codewords + 1}];
	h / Total[h]
	];

Wolfram Language code: imageCodewordsFeatureExtract[RandomSample[training][[1, 1]]]//Histogram

Construct a classifier trained on extracted image features:

Wolfram Language code:

t = Table[imageCodewordsFeatureExtract[kp[[i]]] -> training[[i, 2]], 
	{i, Length@training}];

Wolfram Language code: classifier = Classify[t];

Evaluate the classifier on test data:

Wolfram Language code:

cm = ClassifierMeasurements[classifier, Table[imageCodewordsFeatureExtract[First@sample] -> Last@sample, {sample, testing}]];

Wolfram Language code: cm["ConfusionMatrixPlot"]

Properties & Relations (7)

Not all properties are supported for all methods:

Missing["NotAvailable"] is returned when a property is not available with the specified method:

Wolfram Language code: ImageKeypoints[[image], {"Position", "Descriptor"}, Method -> "FAST", MaxFeatures -> 5]

"FAST" method does not find contrast sign:

Wolfram Language code: ImageKeypoints[[image], {"Position", "ContrastSign"}, Method -> "AGAST", MaxFeatures -> 4]

"BRISK" method does not compute oriented descriptors:

Wolfram Language code: ImageKeypoints[[image], {"Position", "OrientedDescriptor"}, Method -> "BRISK", MaxFeatures -> 4]

"SURF" and "KAZE" descriptors are typically compared using the Euclidean distance:

Wolfram Language code: img = [image];

Wolfram Language code: kp = ImageKeypoints[img, {"Position", "Descriptor"}, MaxFeatures -> 11, Method -> "KAZE"];

Distances between the strongest keypoint and each of the next 10 strongest:

Wolfram Language code:

best = kp[[1, 2]];
distances = Table[EuclideanDistance[best, d], {d, kp[[2 ;; , 2]]}]

"AKAZE", "BRISK" and "ORB" descriptors are typically compared using the Hamming distance:

Wolfram Language code: kp = ImageKeypoints[img, {"Position", "Descriptor"}, MaxFeatures -> 11, Method -> "BRISK"];

Distances between the strongest keypoint and each of the next 10 strongest:

Wolfram Language code:

best = kp[[1, 2]];
distances = Table[HammingDistance[best, d], {d, kp[[2 ;; , 2]]}]

Cluster the keypoints based on their descriptors:

Wolfram Language code:

img = [image];
k = ImageKeypoints[img, {"Position", "Descriptor"}, MaxFeatures -> 80];
clust = FindClusters[k[[All, 2]] -> k[[All, 1]], 3];
colors = ColorData[3] /@ Range[Length[clust]];
HighlightImage[img, Transpose[{colors, clust}]]

ImageCorners may be used as keypoints:

Wolfram Language code: img = [image];

Computer corners using radius 3.5 for similarity to the scale of some keypoint detectors:

Wolfram Language code: corners = ImageCorners[img, 3.5, MaxFeatureDisplacement -> 1, MaxFeatures -> 10]

Wolfram Language code: HighlightImage[img, {White, corners}]

Use CornerFilter to get the strength of detected corners:

Wolfram Language code:

cf = CornerFilter[img, 3.5];
strengths = ImageValue[cf, corners];
Transpose[{corners, strengths}]

Top 10 FAST keypoints:

Wolfram Language code: ImageKeypoints[img, {"Position", "Confidence"}, Method -> "FAST", MaxFeatures -> 10]

Top 10 AGAST keypoints:

Wolfram Language code: ImageKeypoints[img, {"Position", "Confidence"}, Method -> "AGAST", MaxFeatures -> 10]

ImageCorrespondingPoints gives the locations for keypoints that have matching descriptors:

Wolfram Language code: {img1, img2} = {[image], [image]};

Wolfram Language code: {pos1, pos2} = ImageCorrespondingPoints[img1, img2];

Wolfram Language code: MapThread[HighlightImage, {{img1, img2}, {pos1, pos2}}]

Compute keypoints on both images:

Wolfram Language code: {k1, k2} = ImageKeypoints[#, {"Position", "Descriptor"}, MaxFeatures -> 100]& /@ {img1, img2};

Take two keypoints with and without a corresponding point in the second image:

Wolfram Language code: {m, n} = {k1[[23]], k1[[24]]};

Compute all distances between descriptors for these keypoint to all keypoints in the second image:

Wolfram Language code:

δm = EuclideanDistance[m[[2]], #]& /@ k2[[All, 2]];
δn = EuclideanDistance[n[[2]], #]& /@ k2[[All, 2]];

Wolfram Language code: Histogram[#, ImageSize -> 200]& /@ {δm, δn}

The second-to-nearest ratio is typically used to decide whether a keypoint has a corresponding point:

Wolfram Language code:

{bestm, secondbestm} = TakeSmallestBy[k2, EuclideanDistance[m[[2]], #[[2]]]&, 2];
{bestn, secondbestn} = TakeSmallestBy[k2, EuclideanDistance[n[[2]], #[[2]]]&, 2];

Wolfram Language code: EuclideanDistance[m[[2]], secondbestm[[2]]] / EuclideanDistance[m[[2]], bestm[[2]]]

Wolfram Language code: EuclideanDistance[n[[2]], secondbestn[[2]]] / EuclideanDistance[n[[2]], bestn[[2]]]

The keypoint with a correspondence is red; the other one is yellow:

Wolfram Language code:

{HighlightImage[img1, {Red, m[[1]], Yellow, n[[1]]}, ImageSize -> 250], 
	HighlightImage[img2, {White, k2[[All, 1]], Yellow, bestn[[1]], Red, bestm[[1]]}, ImageSize -> 250]}

Top

More Learning

Tech Support

Wolfram Solutions

Wolfram Solutions For Education

Get Started

Grow Your Skills

Work with Us

Educational Programs for Adults

Educational Programs for Youth

Read

ImageKeypoints

Details and Options

Examples

Basic Examples (2)

Scope (10)

Options (11)

KeypointStrength (1)

Masking (1)

MaxFeatures (1)

Method (8)

Applications (5)

Properties & Relations (7)

Text

CMS

APA

BibTeX

BibLaTeX

ImageKeypoints

Details and Options

Examples

Basic Examples (2)

Scope (10)

Options (11)

KeypointStrength (1)

Masking (1)

MaxFeatures (1)

Method (8)

Applications (5)

Properties & Relations (7)

See Also

Related Guides

History

Text

CMS

APA

BibTeX

BibLaTeX