I have attached a reworked version, which I have tested. A few comments: 1.) In your version you had a long JUNK lexeme which slurped up the entire line. Marpa uses LATM lexing (Longest Acceptable Token Matching). JUNK would usually be longest, meaning the parser would usually see nothing else and report almost every line as junk. I changed it so that "junk" characters are lexed one at a time, meaning that they will at best tie other lexemes.
2.) Your JUNK lexeme would accept spaces, which you were also discarding. Having spaces both be discarded and be part of other lexemes is possible, but a very tricky technique -- best to go with one or the other. I set things up so that spaces are not junk characters and discard handles them. For your full example, you'll have to deal with end-of-lines and should think out carefully what gets handled where. 3.) Very useful, even when the problem is not with the lexing, is the trace_terminals recognizer option <https://metacpan.org/dist/Marpa-R2/view/pod/Scanless/R.pod#trace_terminals>. You may want to try it with both your original and my reworked script, and note what is going on. A value of 99 turns on everything and for a small case like this is not too verbose. I had to reread my own doc for this example to refresh myself on how ranking worked. My description of it a couple of emails ago was (to be honest) somewhat confusing. On the good side, it's a much more useful technique than I remembered, and I enjoyed working this up. Thanks, jeffrey -- You received this message because you are subscribed to the Google Groups "marpa parser" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/marpa-parser/CABWWkhOiCjZp3Xkp0sUjJe%2B4_YtNSkKMBBZA60d%2BGs4QUa74pw%40mail.gmail.com.
fli.t
Description: Perl program
