Introduction
Removing duplicate words while preserving order is useful for text cleaning and data normalization.
Method 1: Using LinkedHashSet
import java.util.LinkedHashSet;
import java.util.Set;
public class RemoveDuplicateWords {
public static String removeDuplicateWords(String sentence) {
if (sentence == null || sentence.isEmpty()) {
return sentence;
}
String[] words = sentence.split(" ");
Set<String> seen = new LinkedHashSet<>();
for (String word : words) {
seen.add(word.toLowerCase());
}
return String.join(" ", seen);
}
public static void main(String[] args) {
System.out.println(removeDuplicateWords("hello world hello java world"));
// Output: hello world java
}
}Output:
hello world javaMethod 2: Using Streams
import java.util.Arrays;
import java.util.stream.Collectors;
public class RemoveDuplicateWordsStream {
public static String removeDuplicateWords(String sentence) {
return Arrays.stream(sentence.split(" "))
.distinct()
.collect(Collectors.joining(" "));
}
}Frequently Asked Questions
Q1. Should order be preserved?
Answer: Yes, use LinkedHashSet.
Advertisement
Q2. Case sensitivity?
Answer: Convert to lowercase for comparison.
Q3. Punctuation handling?
Answer: Remove or normalize before processing.
Q4. Performance?
Answer: Both O(n), similar speed.
Q5. Keep multiple occurrences?
Answer: Don't use distinct() or set.
Q6. Large text?
Answer: Streams handle memory better.
Q7. Filter specific words?
Answer: Add condition in loop/stream.
Q8. Count duplicates?
Answer: Track removed count separately.
Q9. Consecutive duplicates only?
Answer: Different logic with previous word tracking.
Q10. Multiple spaces between?
Answer: Use regex split with " +".
Conclusion
LinkedHashSet preserves order. Streams more modern. Choose based on Java version and requirements.
