28-Feb-2013 22:38, Andrei Alexandrescu пишет:
On 2/28/13 12:20 PM, Dmitry Olshansky wrote:
28-Feb-2013 19:31, Andrei Alexandrescu пишет:
On 2/28/13 5:54 AM, Walter Bright wrote:
On 2/27/2013 11:55 PM, Jonathan M Davis wrote:
Again, please see how lexer.c works. I assure you, there is no double
copying going on, nor is there a double test for the terminating 0.
I know what the lexer does, and remember that it _doesn't_ operate on
ranges,
and there are subtle differences between being able to just use char*
and
trying to handle generic ranges.
Hence the need to invent SentinelInputRange.
I don't think the sentinel input range is a blocker for redoing the
parser (with ranges) in D. This discussion has probably run its course.
The right thing to do at this point is port the lexer and figure what
primitives are necessary from its input.
No need to port - look at std.d.lexer again. It was revamped and is
ready for the new round of review I should say. Let's not use the old
source for the new module and go the long path of :
split off --> port --> patch up --> D-ify & re-write to ranges
Instead we can just tweak the current std.d.lexer a little bit more and
we have good clean-room lexer written in the idiomatic D. Well, it's
getting there w.r.t. idiomaticness but it supports ranges including both
random-access and forward ones (by transparently specializing for each
one).
I think that's a good idea but I took a look at
https://github.com/bhelyer/std.d.lexer/blob/master/std/d/lexer.d and I
will destroy it. In a good sense :o).
That's the wrong one. This is the one:
https://github.com/Hackerpilot/Dscanner/tree/range-based-lexer
Though feel free to destroy the other one too ;)
But I need your full powers with Dscanner first :o)
Andrei
--
Dmitry Olshansky
