28-Jan-2013 00:18, Philippe Sigaud пишет:
I concur. One of the biggest reason* there is a separate lexer step is
because it could be made to do this stage very-very fast. Then the rest of
the parser will greatly benefit from this underlying speed.
*Otherwise we could have just as well add the lexer stage as simple rules to
the grammar that treats all of codepoints as terminals.
... which is exactly what parsing expression grammars (and other
scannerless parsers) do.
That is only a happenstance and is highly overrated. What does it buy
you is the right question to ask. After all every LL-parser can be
written avoiding notion of lexer and strangely I see no fuss about it.
So is it just hype? I dunno.
AFAICT, one interesting consequence is the ability to have composition
of grammars, which I sure have with Pegged. But maybe it's not related
that much, that's not something I stopped to think about.
In any case, grammar composition is something I've learnt to like quite a lot.
You can still have composability of grammars. In fact I'd define a
couple of a less powerful but better optimized variations if I were you.
PEG-style memoization is still possible as long as underlying grammars
are context-free all the useful usually are.
LR hardly composable ( I haven't seen a single one that does).
But all top-down one always are unless there is some explicit work spent
to undermine this ability ;)
--
Dmitry Olshansky
