On 2/9/2013 6:37 AM, Andrei Alexandrescu wrote:
On 2/9/13 3:07 AM, Jonathan M Davis wrote:
On Friday, February 08, 2013 23:48:50 Walter Bright wrote:
On 2/8/2013 12:01 AM, Jonathan M Davis wrote:
It's not quite a use case where
ranges shine - especially when efficiency is a top priority.
A more problematic case is dmd's lexer relies on a 0 byte at the end to be a
"sentinel" for the end of file. Without such a sentinel, every consumption
of a source character requires two operations rather than one.
I didn't know that. That's a cute trick. But yeah, without controlling the
input, it's not possible, and that won't work with a general implementation in
a library. It would be trivial enough to put a wrapper around the input to add
the 0 byte at the end, but the wrapper would still have to keep checking for
empty, so you wouldn't gain anything.
That's not a problem. You may require that the input has a terminating zero.
Perhaps we can formalize this a bit. Define a SentinelInputRange, which has an
additional manifest constant with the name 'sentinel'. empty becomes defined as:
empty = front == sentinel;
popFront() does not advance if empty. Advancing over a SentinelInputRange can be
done the usual:
for (r = range; !r.empty; r.popFront()) { auto e = r.front; }
or can be done as:
for (r = range; r.front != sentinel; r.popFront()) { auto e = r.front; }
This should work well with 0 terminated strings. The output of the lexer should
also be a SentinelInputRange, with TOKeof as the sentinel.
