On Wednesday, 9 October 2013 at 04:38:02 UTC, Andrei Alexandrescu
wrote:
On 10/8/13 9:32 PM, deadalnix wrote:
Overall, I think this is going into the right direction.
However, there
is one thing I don't like with that design.
When you go throw the big switch of death, you match the
beginning of
the string and then you go back to a function that will test
where does
it come from and act accordingly. That is kind of wasteful.
What SDC does is that it calls a function-template with the
part matched
by the big switch of death passed as template argument. The
nice thing
about it is that it is easy to trnsform this compile time
argument into
a runtime one by simply forwarding it (what is done to parse
identifier
that begins by a keyword for instance).
I think a bit of code would make all that much clearer.
Andrei
Sure.
So here is the lexer generation infos (this can be simplified by
using the tok!"foobar" thing) : http://dpaste.dzfl.pl/7ec225ee
Using theses infos, a huge switch based boilerplate is generated.
Each "leaf" of the huge switch tree call a function template as
follow, by passing as template argument what has been matched so
far. You can then proceed as follow :
http://dpaste.dzfl.pl/f2f0d22c
You may wonder about the "?lexComment". The boilerplate generator
understand ? as an indication that lexComment may or may not
return a token (depending on lexer configuration) and generate
what is needed to handle that (by testing if the function return
a token, via some static ifs).
You obviously ends up with a log of instance of
lexIdentifier(string s)(), but this simply forward to
lexIdentifier()(string s) and the forwarding function is removed
trivially by the inliner.
