Hi all! Just a quick response about scannerless parsers. These are most
useful for modular grammars and for languages where there is no clear
lexical sentinel that differentiates between islands.
This is not hard to do in antlr v4. I have an example in the extras
directory of the book where I do a mini C + SQL example. Here's some sample
input:
void f(int i) {
resultset r = select * from users where ID='1999' and state<>'CA';
int select = 'a';
int from = 'b';
int x = select * from;
int nextID = select ID from users where name='parrt' + 1;
}
PEG folks will recognize it as very PEG like. It's inefficient in v4
because each character must be a token.
Anyway, just wanted to point out that I find this case rare and like having
a separate lexer, but of course I want it all in one specification as if it
were scannerless.
Ter
On Sun, May 5, 2013 at 9:40 AM, Juancarlo Añez <apal...@gmail.com> wrote:
> Todd,
>
> The tokenizer was a component that already existed in ParseKit. So when I
>> began work on the new PEG-based static packrat generator a few weeks ago, I
>> took a shortcut. I used the existing tokenizer for token recognition. I may
>> unify parser/tokenizer recognition in the future for the PEG-based
>> generator (like ANTLR). But either way, the existing tokenizer will remain
>> as a useful stand-alone component for other applications.
>
>
> ANTLR is not PEG! It is ALLSTAR (aLL[*]).
>
> The reason I migrated away from ANTLR is because of the tokenizer, which
> cannot be parser-driven (not without a huge amount of "glue" code). Quite
> unlike PEG.
>
> My way out of it was this:
>
> https://bitbucket.org/apalala/grako
>
>
> I can explain further, if you care.
>
> Cheers,
>
> --
> Juancarlo *Añez*
>
> _______________________________________________
> PEG mailing list
> PEG@lists.csail.mit.edu
> https://lists.csail.mit.edu/mailman/listinfo/peg
>
>
--
Dictation in use. Please excuse homophones, malapropisms, and nonsense.
_______________________________________________
PEG mailing list
PEG@lists.csail.mit.edu
https://lists.csail.mit.edu/mailman/listinfo/peg
