On 2012/09/29 13:37:28, janek wrote:
On Sat, Sep 29, 2012 at 3:10 PM, <mailto:d...@gnu.org> wrote: > Embarrassingly, not really. The point of this change is to move
forward
> in removing the special-casing of various *_IDENTIFIER token types.
The
> target is to arrive at only a single SCM_IDENTIFIER type. Once that
is
> achieved, it is no longer necessary to evaluate identifiers in the > lexer, removing a source for awkward timing problems like
So the idea is to make the grammar simpler (use less different types) and have things happen on higher level? Sounds reasonable, i guess...
Actually, it is not really "simplifying" the grammar. At the current point of time, the lexer has to make decisions about the type of identifiers (and functions) at a point of time where those decisions are actually premature. As a result of those decisions, different tokens are generated for identifiers and functions. I am moving towards generating only one kind of token. The same kind of decisions as before still have to be made, and since the lexer no longer makes them, the decisions are instead taken in the actions of the parser. The results of those decisions are only apparent in the parser rules when those decisions lead to "synthetic" tokens (similar to the previous "real" ones) being generated in the parser actions. So the distinctions that were previously made are still there, they are just done at a later point of time, and result in different actions through different mechanisms. Of course, once the lexer only delivers identifiers in a single way, there are some simplifications possible, but the overall architecture is still a worse fit to the Bison/Flex pairing of parser/scanner generator than the state from two years ago, as that pairing is based on lookahead principles with rigid tokenization rules that are not really a perfect fit for LilyPond's sort of dynamic typing. So overall, the parser is not getting simpler. I can straighten out some stuff when passing certain milestones, "boiling down" complexity again, but since I am moving functionality from where it is "intended to be" from the view of the basic Bison/Flex architecture, the best it can be is a consistent subsystem not looking more complex than the task it is taking over. http://codereview.appspot.com/6561053/ _______________________________________________ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
