Text tools

Remove duplicate lines and sort

Remove repeated and empty lines from a list, and sort it alphabetically or by number if you like.

Options

This tool runs entirely inside your browser. No file or text is sent to any server.

How to use

  1. Type or paste your list into the “Text or list” box, one item per line. If you would like to see the tool at work first, press “Insert example”.
  2. Turn the options on or off and, if you need it, choose an order from “Sort”. The result updates by itself with every change.
  3. Read the result in the “Result” box. Under it you are told how many lines you gave, how many are left and what was removed.
  4. Press “Copy” to take the result. To start again, press “Reset”.

Notes on this tool

Of several identical lines, the first one stays and the later copies are removed. The order of the lines does not change either, unless you choose a sort.

An empty line is never counted as a repeat. Unless “Remove empty lines” is on, every empty line, such as the gap between paragraphs, stays where it is.

Two lines count as the same only when they are the same character for character. The tool does not guess: an Arabic “ي” and a Persian “ی”, or a half-space and a space, are different things. If your list was put together from several places, first tidy it with the Persian text cleaner.

Alphabetical sorting follows the letter order of the page’s language and reads a number inside a line as a number, so “file 2” comes before “file 10”. In a numeric sort the whole line must be one number; lines that are not numbers come after the numbers, in alphabetical order.

Your text is processed only in your own browser and is never sent or saved anywhere.

Frequently asked questions

Why were some lines that look alike not removed?
There is probably a difference you cannot see, such as an extra space at the end of a line, or Latin letters where one is a capital and the other is not. Turn on “Remove spaces at the start and end of each line” and “Treat capital and small Latin letters as the same”. If they are still kept, the difference is in the characters themselves; see the note above about the Arabic “ي” and the half-space.
Excel can remove duplicates too, so what is this tool for?
Excel has an option called Remove Duplicates, and it works on the columns of a file. This tool is for when there is no file: you have text from a messenger, an email or a website and just want to clean up the list. Sorting is right here too, and there is nothing to install or open.
Does it work with very big lists?
Lists of tens of thousands of lines are done in a moment. For far bigger ones, a few hundred thousand lines for example, it can take a few seconds, because showing very long text is slow in the browser itself; more so on a phone. If the page slows down, give the list in several parts.
In an alphabetical sort, how do Persian and English lines sit together?
On the Persian page, the Persian lines come first and the Latin lines after; on the English page it is the other way round. Within each group the order is that language’s alphabet order.

Related tools