Deduplicate command history

[W. D. Sadeep]

Hello,

My command history contains duplicates, eating into HISTSIZE budget. How
do I deduplicate it?

Thinking of adding something like this to .kshrc:

clean_history() {
    if [ -f "$HISTFILE" ]; then
        awk '!x[$0]++' "$HISTFILE" > "$HISTFILE.tmp" && mv "$HISTFILE.tmp" 
"$HISTFILE"
    fi
}
trap clean_history EXIT

Does that make sense? Or is there a better way?

-- 
Sadeep