Furthermore, once you combine both lexical and hierarchical syntax, i.e., rely on scannerless parsing, you can do strictly more. Martin Hirzel and I have combined the complete Java and C languages into a new language (called Jeannie) to substantially simplify Jave Native Interface programming. The syntactic composition of the existing grammars for Java and C was trivial thanks in part to scannerless parsing. Several papers by Eelco Visser make a similar case for other language extensions, only they are building on a scannerless GLR parser generator.
Impressive!
Thank you! :)
In principle, you could use ANTLR's lexer as a parser for a big language. Would work same way. It would use LL(*) as optimization unless it had to backtrack. Should have speed and composability per the groovy Rats! creature! :)
One key feature of the combined Jeannie = Java + C language is that Java keywords are only keywords in a Java context and C keywords are only keywords in a C context. Similarly, legal identifiers for Java and C are pretty close but different. Yet, in Jeannie, Java identifiers are recognized in a Java context and C identifiers in a C context. Similarly for symbols...
If you have a separate lexer and parser, then you need to have different lexer states to capture the different contexts. But the original lexer specifications are not written with states in mind and you have to backfit them. In contrast, with scannerless parsing everything works as expected. On a much smaller scale, I ran into this problem when bootstrapping Rats! with an ANTLR grammar. I mentioned to you before that I gave up on the ANTLR grammar when trying to add character classes and couldn't get lexer states to work in ANTLR. The not getting it to work part is likely my short-coming, but having to go through this extra step makes grammar extension/ composition so much harder for everyone.
Robert _______________________________________________ PEG mailing list PEG@lists.csail.mit.edu https://lists.csail.mit.edu/mailman/listinfo/peg
