I've been rewriting a bit of the lexer in DScanner. https://github.com/Hackerpilot/Dscanner/blob/NewLexer/stdx/lexer.d(Ignore the "invariant" block. I've been trying to hunt down some unreelated memory corruption issue)
One thing that I've found to be very useful is the ability to increment column or index counters inside of the lexer range's popFront method. I think that I'd end up using my own range type for arrays, but construct a lexer range on top of your buffer range for anything else.
