Remove Duplicate Words
Keep only the first occurrence of each word in your text. Optional case-insensitive comparison.
0 duplicates removed
About Remove Duplicate Words
Word-level deduplication keeps the first occurrence of each word and drops subsequent identical ones. Words are tokenized on whitespace, so punctuation attached to a word is preserved (e.g. 'cat.' and 'cat' are different unless you strip punctuation first).
When to use it
- Producing a list of unique tags from a comma-separated string
- Cleaning a thesaurus-style list of synonyms
- Removing accidental repetitions from a draft
- Generating a unique word list for vocabulary study
How it works
The input is split on whitespace. A Set tracks which words have been seen; duplicates are skipped. The first occurrence of each word keeps its position and casing.
Examples
cat dog cat bird dog cat
cat dog bird
Frequently asked questions
- Are word boundaries based on whitespace or punctuation?
- Whitespace. 'cat,' and 'cat' are different words because the comma is attached. Strip punctuation first if needed.
- What separator does the output use?
- A single space. To get a different separator (newline, comma, etc.), post-process the output.
- Does it preserve original casing?
- Yes — only the first occurrence is kept, and its casing is preserved. Subsequent occurrences (regardless of casing if case-insensitive is on) are dropped.