Hello Ter!
Anyway, just wanted to point out that I find this case rare and like having
> a separate lexer, but of course I want it all in one specification as if it
> were scannerless.
Just a note on why I reluctantly had to opt out of ANTLR and seek PEG for
my ongoing projects:
- Software AG Natural allows relevant language words to be used as
identifiers. In fact, words are keywords or not depending on context.
- Natural also makes many of the relevant language words optional. The
predictive ANTLR *v3* grammar was inundated with lookaheads to resolve
the ambiguity caused by that.
- Natural has exactly one statement that is not led by a keyword:
assignment. And most statements lack terminators, so ANTLR would consume
the left hand side of an assignment unless a negative lookahead was added
to most rules.
- COBOL programs embed subprograms in CICS and SQL, and those must be
parsed to have a complete translation.
It was time-consuming and difficult to deal with the above with a mandated
tokenizer, and with v3's allergy to apparent ambiguity (I couldn't try v4
because the Python runtime wasn't ready).
I know. They are both stupid programming languages... but they are real,
and there are millions of lines of source code written in them and still
alive. PEG deals with both easily. You may be right in that cases like
these are rare, though.
Just to be clear, some lexical analysis is good to have in PEG too, to deal
with comments, to ease the burden over whitespace, and to help with words
that are prefixes of other words ("ID" and "IDENTIFICATION", f.i.). Without
the filter, "ws" and "comment" would by far be the most used non-terminals
in most grammars, and much more thought would have to go into rule ordering.
Cheers,
--
Juancarlo *Añez*
_______________________________________________
PEG mailing list
PEG@lists.csail.mit.edu
https://lists.csail.mit.edu/mailman/listinfo/peg
