08-Feb-2013 12:01, Jonathan M Davis пишет:
On Tuesday, February 05, 2013 22:51:32 Andrei Alexandrescu wrote:
I think it would be reasonable for a lexer to require a range of ubyte
as input, and carry its own decoding. In the first approximation it may
even require a random-access range of ubyte.
Another big issue is the fact that in some ways, using a pointer like dmd's
lexer does is actually superior to using a range. In particular, it's trivial
to determine where in the text a token is, because you can simply subtract the
pointer in the token from the initial pointer. Strings would be okay too,
because you can subtract their ptr properties. But the closest that you'll get
with ranges is to subtract their lengths, and the only ranges that are likely
to define length are random-access ranges.
Not true, certain ranges know length but can't be random access as
indexing is O(lgN) or worse. Including a stripe of chunks as taken from
file.
--
Dmitry Olshansky
