On Wednesday, 6 February 2013 at 03:51:33 UTC, Andrei
Alexandrescu wrote:
I think it would be reasonable for a lexer to require a range
of ubyte as input, and carry its own decoding. In the first
approximation it may even require a random-access range of
ubyte.
Playing around that, I discovered a bug in std.utf : slice and
other range are not threated the same way by decodeFront, which
is rather problematic. Jonathan also hit that bug :
http://d.puremagic.com/issues/show_bug.cgi?id=9456
That make the lexer hard to write with a consistent behavior for
InputRanges.
The bug probably exists in everything that rely on decodeFront at
some point.
