Remove Duplicate Lines & Sort
Paste any list with one item per line: names, emails, product codes. Repeated lines are dropped, then you can sort, shuffle or number the list.
- Lines in the result
- 0
- Lines in the original
- 0
- Lines removed
- 0
Your list is processed on this device and is not uploaded to any server.
How to use it
- Paste your list into Original text, one item per line, or open a .txt or .csv file.
- Pick the steps you need. Trim spaces at the start and end of lines and Remove duplicate lines are on from the start.
- The Result box changes every time you edit the text or an option. The figures below it show the line count before and after.
- Press Copy result, or Download .txt to save it as a file.
The order of the steps
The steps always run from top to bottom in their numbered order, whatever order you tick them in: tidy spaces, remove old numbers, remove text at the start or end, remove empty lines, remove duplicates, sort, then add text and new numbers. Cleaning comes first so that duplicates are found reliably. Without it, "Ben Carter" and "Ben Carter " (with a trailing space) would count as two different names. New numbers come last, so they still run 1, 2, 3 after sorting.
Example
The list behind Try an example has 10 lines: 8 names and 2 empty lines. With the options below, 5 names are left.
| Step | Effect | Lines left |
|---|---|---|
| 1. Trim spaces at the start and end | " Tom Baker" and "Ben Carter " are tidied | 10 |
| 4. Remove empty lines | 2 empty lines dropped | 8 |
| 5. Remove duplicates, ignoring case | The second Ben Carter, Tom Baker and Anna Smith are dropped | 5 |
| 6. Sort A-Z | anna smith, Ben Carter, Chloe Davis, Emma Wilson, Tom Baker | 5 |
"anna smith" is kept rather than "Anna Smith" because it comes first. Choose The last if the spelling you want is further down.
Sort options
For the list file10, File2, file1, Äpfel, apel, zebra:
| Option | Result |
|---|---|
| A-Z | apel, Äpfel, file1, file10, File2, zebra |
| Natural number order | apel, Äpfel, file1, File2, file10, zebra |
| Character code | File2, apel, file1, file10, zebra, Äpfel |
| Shortest first | apel, File2, file1, Äpfel, zebra, file10 |
Natural number order reads runs of digits as numbers, so file2 comes before file10. Minus signs and decimals are not taken into account: -10 is placed after -3, and 1.25 after 1.5. A-Z uses the alphabet rules of the page language; on the Spanish page, for example, ñ comes after n. Lines that are equally long or compare as equal keep their original order.
Removing old numbers
Only numbering at the start of a line is removed: "1. ", "2) ", "3: ", "(4) ", or a number followed by a tab. The mark must be followed by a space, so "3.14", "10:30" and "2024 was" are left alone.
Questions
Is my list sent to a server?
Why are some lines that look the same not removed?
Which line is kept when there are duplicates?
What is the difference between A-Z and Character code order?
How random is Shuffle?
crypto.getRandomValues, so every order is equally likely. It is fine for drawing a speaking order or picking winners. Press Shuffle again for a new order.