The script is also fast, which I was worried about given my files were very large.

Notice that the shell is slow. It is a reason to substitute loops in shell scripts (like "while read line; do ... ; done < file") with calls to commands processing the whole input line by line. Essentially all text-processing commands do that, AWK programs included. If possible, those commands had better communicate through pipes, so that they run in parallel.

Reply via email to