[il-antlr-interest: 23010] Re: [antlr-interest] parsing ugly grammars

Jim Idle Sat, 04 Apr 2009 10:17:52 -0700

Tomasz Jastrzebski wrote:

Hello all,

Writing a parser for some ugly grammar I came across a problem I do not know how to approach. Here is a sample grammar illustrating the problem:

grammar test;

program : (statement)*;

statement
   : RawData
   | Identifier ';'
   ;

RawData: 'data;' ((options {greedy=false;} : .)* ';;')? ;

Identifier : ('a'..'z')+;

WhiteSpace : (' ' | '\t' | '\r\n' | '\r')+ { $channel=HIDDEN; };

The RawData can contain data ended with ‘;;’ or can be empty. Two sample valid inputs:
data; some raw data here;; identifier;
data; identifier;

The parser does not correctly recognize the second input (mismatched character '<EOF>' expecting ';') .
Parser does not realize that what follows ‘data’ keyword is not followed by ‘;;’ so it is not "raw data" and should be interpreted as Identifier.

I am clueless. Could anyone help?

Thomas

Is this the entire structure? If would use a program for this - ANTLR is overkill. In a program you can just accumulate after the data; then when you see if you get ; or ;; you can assign the text you gathered to one or the other. However, you can do this with ANTLR too:

grammar test;

program
   : dataLine* EOF;

dataLine
@init
{
    String tbd = "";
}
    : DATA
       (
          ID { tbd += $ID.text; }
       )+
       SEMI
          (
              SEMI // Second semi, tbd holds the raw data
                   id=ID SEMI
                   {System.out.println("ID is " + $id.text + ", data is:\n" + tbd); }
              | // NO second semi, tbd is just the identifier
                { System.out.println("ID is " + tbd);
          )
      ;

DATA : 'data';
SEMI: ';';
ID : ('a'..'z')+;

WhiteSpace : (' ' | '\t' | '\r\n' | '\r')+ { $channel=HIDDEN; }; // ARE YOU SURE YOU WANT TO HIDE spaces from the RAW DATA?

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [email protected]
For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

[il-antlr-interest: 23010] Re: [antlr-interest] parsing ugly grammars

Reply via email to