[il-antlr-interest: 25156] Re: [antlr-interest] Extracting a string whose value clashes with token value

Wed, 12 Aug 2009 07:00:26 -0700

David-Sarah Hopwood wrote: 
Benoit Fouletier wrote:
  
I think special_string should be a lexer rule, not a parser rule: rename it
to SPECIAL_STRING. Also, the lexer depends on the order with which you define
tokens, so make sure you put ANTLRTOKEN  above SPECIAL_STRING.
    

This will work, but it will generate ambiguity warnings (because "ANTLR"
can be lexed either as an ANTLRTOKEN or a SPECIAL_STRING). To eliminate
the warning and make the intent clear, put ANTLRTOKEN in a 'tokens' block,
i.e.:

  grammar sample_parser;

  options {
    language=C;
  }

  tokens {
    ANTLRTOKEN = 'ANTLR';
  }

  requestline : ANTLRTOKEN WHITESPACE special_string ;

  SPECIAL_STRING : (CHAR | '=' | '.' | '-' | '@' )+ ;
  WHITESPACE  : ( '\t' | ' ' | '\u000C' )+;
  NEWLINE: ('\r')? '\n';
  fragment CHAR: (('a'..'z')|('A'..'Z'));


Note that it is necessary to change CHAR to a fragment rule if
SPECIAL_STRING is a lexer rule (which I agree it probably should be).
The grammar above is not correct. Here is one that works as you want. Note that the keyword token is defined as a lexer rule (avoid the tokens { } section as it will confuse you) and is listed first in the lexer rule order. You can conflate CHAR into the SPECIAL_STRING lexer rule and avoid a method/function call at runtime:

grammar t;

requestline     : ANTLRTOKEN WHITESPACE SPECIAL_STRING EOF;

ANTLRTOKEN      : 'ANTLR';
SPECIAL_STRING  : (CHAR | '=' | '.' | '-' | '@' )+ ;
WHITESPACE      : ( '\t' | ' ' | '\u000C' )+;
NEWLINE         : ('\r')? '\n';
fragment CHAR   : (('a'..'z')|('A'..'Z'));

Jim



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [email protected]
For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

Reply via email to