Remove Duplicate Lines & Sort

Paste any list with one item per line: names, emails, product codes. Repeated lines are dropped, then you can sort, shuffle or number the list.

Steps

They run in order from 1 to 8. The result above updates straight away.

  1. Spaces
  2. Old numbers
  3. Remove text
  4. Empty lines
  5. Duplicates
    Keep
  6. Add text
  7. New numbers
Lines in the result
0
Lines in the original
0
Lines removed
0

Your list is processed on this device and is not uploaded to any server.

How to use it

  1. Paste your list into Original text, one item per line, or open a .txt or .csv file.
  2. Pick the steps you need. Trim spaces at the start and end of lines and Remove duplicate lines are on from the start.
  3. The Result box changes every time you edit the text or an option. The figures below it show the line count before and after.
  4. Press Copy result, or Download .txt to save it as a file.

The order of the steps

The steps always run from top to bottom in their numbered order, whatever order you tick them in: tidy spaces, remove old numbers, remove text at the start or end, remove empty lines, remove duplicates, sort, then add text and new numbers. Cleaning comes first so that duplicates are found reliably. Without it, "Ben Carter" and "Ben Carter " (with a trailing space) would count as two different names. New numbers come last, so they still run 1, 2, 3 after sorting.

Example

The list behind Try an example has 10 lines: 8 names and 2 empty lines. With the options below, 5 names are left.

StepEffectLines left
1. Trim spaces at the start and end"  Tom Baker" and "Ben Carter " are tidied10
4. Remove empty lines2 empty lines dropped8
5. Remove duplicates, ignoring caseThe second Ben Carter, Tom Baker and Anna Smith are dropped5
6. Sort A-Zanna smith, Ben Carter, Chloe Davis, Emma Wilson, Tom Baker5

"anna smith" is kept rather than "Anna Smith" because it comes first. Choose The last if the spelling you want is further down.

Sort options

For the list file10, File2, file1, Äpfel, apel, zebra:

OptionResult
A-Zapel, Äpfel, file1, file10, File2, zebra
Natural number orderapel, Äpfel, file1, File2, file10, zebra
Character codeFile2, apel, file1, file10, zebra, Äpfel
Shortest firstapel, File2, file1, Äpfel, zebra, file10

Natural number order reads runs of digits as numbers, so file2 comes before file10. Minus signs and decimals are not taken into account: -10 is placed after -3, and 1.25 after 1.5. A-Z uses the alphabet rules of the page language; on the Spanish page, for example, ñ comes after n. Lines that are equally long or compare as equal keep their original order.

Removing old numbers

Only numbering at the start of a line is removed: "1. ", "2) ", "3: ", "(4) ", or a number followed by a tab. The mark must be followed by a space, so "3.14", "10:30" and "2024 was" are left alone.

Questions

Is my list sent to a server?
No. Every step runs in your browser, on your device, so a list of names, phone numbers or emails never leaves it.
Why are some lines that look the same not removed?
Usually there is a hidden space at the start or end of the line, or a difference in capitals. Turn on Trim spaces at the start and end of lines and Ignore upper/lower case. Spaces in the middle ("Ben  Carter" with two spaces) only count as the same after Collapse double spaces is ticked.
Which line is kept when there are duplicates?
Choose The first to keep the occurrence nearest the top, or The last to keep the one nearest the bottom. The kept line stays where it was. When case is ignored, the spelling of the kept line is the one you get.
What is the difference between A-Z and Character code order?
A-Z follows the alphabet rules of the page language: capitals and small letters sit together, and Ä or é are placed next to A and E. Character code order sorts by Unicode number, like the sort command in a terminal: all capitals before all small letters, and accented letters after z.
How random is Shuffle?
The order is shuffled with the Fisher-Yates algorithm using random numbers from the browser's crypto.getRandomValues, so every order is equally likely. It is fine for drawing a speaking order or picking winners. Press Shuffle again for a new order.
Can I use it with data from Excel?
Yes, for one column: copy the column from Excel or Google Sheets, paste it here, then copy the result back. If you paste several columns at once, each line holds the columns separated by tabs, and a line only counts as a duplicate when every column matches.