Hi Lloyd, I'm happy to report your code compiled without error on a modern Linux system, although I did run into some very minor non-breaking bugs in operation. Currently it ignores the "output file" and emits the tokenization to stdout. After that, it produces output that matches your test output perfectly.
Thanks for providing source that made it so easy to run. This can be very useful for program analysis.
