When to use uniq
Use uniq to compare adjacent lines. Sort first when equal records can occur in different parts of a file, but preserve original ordering through another method if sequence matters.
Syntax
uniq [PARAMETERS] [INPUT [OUTPUT]]Square brackets mark optional arguments; an ellipsis means repetition. Replace example file names, user names and server addresses with your own values.
Common options and arguments
Usage examples
Remove adjacent duplicates
uniq ./sorted.txtOnly consecutive repeated lines are collapsed.
Count identical lines
sort ./names.txt | uniq -cSorting groups equal names so occurrence counts cover the entire input.
Show duplicate values
sort ./names.txt | uniq -dOnly names that occur repeatedly are printed after sorting.
Notes and limitations
- Only adjacent rows are compared: for a shared list, first do sort.
- The order and concept of equality may depend on the locale and the selected options.
Common questions and troubleshooting
Why are duplicate lines still present?
Only adjacent identical lines are compared. Separate occurrences remain unless you group them first or use a tool with a full-input key set.
Will sort followed by uniq preserve input order?
No. Sorting changes order. If original order matters, use an order-preserving deduplication method and consider the memory needed for its key set.
Further help
Options vary by version and distribution. Check the installed documentation with this command: man uniq.