On 2/9/13 10:05 AM, Jacob Carlborg wrote:
On 2013-02-09 09:05, Jonathan M Davis wrote:
The lexer that I'm writing doesn't even support pure input ranges,
because it
has to be able to save to do what it does, and the only reason that
decodeFront exists is because I wrote it to support that use case. And
needing
to operate on ranges of code units is likely to be quite rare, and the
function is quite new, so I don't expect that much code is affected by
any bugs
in decodeFront or that much would be broken were it to be changed.
Following this discussion it seems it's complicated to support ranges in
a lexer and still maintain the speed. Is it really worth the trouble to
support ranges?
Requiring a random-access range of ubyte with a terminating zero may be
the most general approach to a fast lexer - and that's fine.
Andrei
