WOLFRAM

Wolfram Language & System Documentation Center

DeleteDuplicates[data]

deletes all duplicates from data.

DeleteDuplicates[data,test]

applies test to pairs of elements to determine whether they should be considered duplicates.

Details
Details and Options Details and Options
Examples  
Basic Examples  
Scope  
Properties & Relations  
Possible Issues  
See Also
Related Guides
Related Links
History
Cite this Page

DeleteDuplicates

DeleteDuplicates[data]

deletes all duplicates from data.

DeleteDuplicates[data,test]

applies test to pairs of elements to determine whether they should be considered duplicates.

Details

  • DeleteDuplicates[data] deletes all but the first occurrence of each distinct element that appears in data.
  • DeleteDuplicates also works on associations, removing entries whose values duplicate those earlier in the association.
  • The data can have the following forms:
  • {e1,e2,…}list of values, including numbers, quantities, dates, ...
    Association[…]association of values
    QuantityArray[…]quantity array or other structured array
    Tabular[…]type-consistent tabular data
    TabularColumn[…]type-consistent column data
    Dataset[…]general hierarchical data
  • DeleteDuplicates never reorders elements, but only removes them.
  • DeleteDuplicates[data] is equivalent to DeleteDuplicates[data,SameQ]. »

Examples

open all close all

Basic Examples  (2)

Delete duplicated elements:

Wolfram Language code: DeleteDuplicates[{1, 7, 8, 4, 3, 4, 1, 9, 9, 2}]

Delete elements whose values duplicate those earlier in the association:

Wolfram Language code: DeleteDuplicates[<|a -> 1, b -> 2, c -> 1, d -> 3, e -> 2|>]

Scope  (6)

The data need not have head List:

Wolfram Language code: DeleteDuplicates[Hold[1 + 2, 4 - 1, 1 + 2, 2 + 1]]

Delete elements that are preceded by at least one larger element:

Wolfram Language code: DeleteDuplicates[{1, 7, 8, 4, 3, 4, 1, 9, 8, 9, 2}, Greater]

Treat numbers as equal if they differ by 0.01 or less:

Wolfram Language code: DeleteDuplicates[{3.0, 3.1, 3.11, 3.5, 3.51}, Abs[#1 - #2] <= 0.01&]

Delete duplicate rows in a Tabular object:

Wolfram Language code: Tabular[{{1, 2}, {4, 5}, {1, 2}}, {"col1", "col2"}]
Wolfram Language code: DeleteDuplicates[%]

Delete rows whose first column value is larger than a previous value:

Wolfram Language code: Tabular[{{2, 2}, {3, 3}, {1, 5}}, {"col1", "col2"}]
Wolfram Language code: DeleteDuplicates[%, #1["col1"] < #2["col1"]&]

Delete duplicate rows in a Dataset object:

Wolfram Language code: Dataset[<|"row1" -> <|"col1" -> 1, "col2" -> 3|>, "row2" -> <|"col1" -> 1, "col2" -> 3|>, "row3" -> <|"col1" -> 4, "col2" -> 3|>|>]
Wolfram Language code: DeleteDuplicates[%]

Properties & Relations  (4)

DeleteDuplicates[list] is equivalent to DeleteDuplicates[list,SameQ]:

Wolfram Language code: list = Join[RandomInteger[100, {50}], RandomReal[1, {50}], {x, y, z, π, E}];
Wolfram Language code: DeleteDuplicates[list] == DeleteDuplicates[list, SameQ]

DeleteDuplicates is similar to Union without sorting:

Wolfram Language code: list = {9, 0, 0, 3, 2, 3, 6, 2, 9, 8, 4, 9, 0, 2, 6, 5, 7, 4, 9, 8};
Wolfram Language code: Union[list]
Wolfram Language code: DeleteDuplicates[list]
Wolfram Language code: Sort[%]

Avoiding the sort improves the speed substantially:

Wolfram Language code: list = RandomInteger[10 ^ 5, 10 ^ 7];
Wolfram Language code: {First[Timing[DeleteDuplicates[list]]], First[Timing[Union[list]]]}

DeleteDuplicates[list,test] applies test to two elements to determine if they are duplicates:

Wolfram Language code: list = {{0, 0, 0, 1, 0}, {1, 0, 1, 0, 1}, {1, 1, 1, 0, 0}, {0, 0, 0, 0, 1}, {1, 1, 1, 0, 1}};
Wolfram Language code: DeleteDuplicates[list, Total[#1] == Total[#2]&]

DeleteDuplicatesBy[list,f] transforms each element using f, then compares the results:

Wolfram Language code: DeleteDuplicatesBy[list, Total]

CountDistinct[list] is equivalent to Length[DeleteDuplicates[list]]:

Wolfram Language code: list = RandomInteger[100, {33}];
Wolfram Language code: CountDistinct[list] == Length[DeleteDuplicates[list]]

CountDistinct[list,test] is equivalent to Length[DeleteDuplicates[list,test]]:

Wolfram Language code: lastDigitSameQ[x_, y_] := Mod[x, 10] === Mod[y, 10]
Wolfram Language code: CountDistinct[list, lastDigitSameQ] == Length[DeleteDuplicates[list, lastDigitSameQ]]

Possible Issues  (1)

If the comparison function returns neither True nor False, elements are assumed to be distinct:

Wolfram Language code: DeleteDuplicates[{3, 3, 3}, f]

Compare with the function that always reports elements as distinct:

Wolfram Language code: DeleteDuplicates[{3, 3, 3}, False&]
Wolfram Research (2008), DeleteDuplicates, Wolfram Language function, https://reference.wolfram.com/language/ref/DeleteDuplicates.html (updated 2025).

Text

Wolfram Research (2008), DeleteDuplicates, Wolfram Language function, https://reference.wolfram.com/language/ref/DeleteDuplicates.html (updated 2025).

CMS

Wolfram Language. 2008. "DeleteDuplicates." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2025. https://reference.wolfram.com/language/ref/DeleteDuplicates.html.

APA

Wolfram Language. (2008). DeleteDuplicates. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/DeleteDuplicates.html

BibTeX

@misc{reference.wolfram_2026_deleteduplicates, author="Wolfram Research", title="{DeleteDuplicates}", year="2025", howpublished="\url{https://reference.wolfram.com/language/ref/DeleteDuplicates.html}", note=[Accessed: 13-June-2026]}

BibLaTeX

@online{reference.wolfram_2026_deleteduplicates, organization={Wolfram Research}, title={DeleteDuplicates}, year={2025}, url={https://reference.wolfram.com/language/ref/DeleteDuplicates.html}, note=[Accessed: 13-June-2026]}