At 16:45 23/10/2009, Hieu Phung wrote: >Alpha = %x41-5A; >Numeric = %x30-39; >Decimal = %x30-39 / "."; >Mixed = Alpha / Numeric; >Text = %x41-5A / %x30-39 / "." / "-" / " "; <--- this is my >MIX token > >This format can be written in ABNF easily... but in Antlr, once I >introduce the MIX token, everything which is mixed of numeric and >alpha is returned as a MIX. Currently I have to use Java code in >action to split the MIX string. I wonder if there's a better way >to define tokens because my grammar now is full of Java code :(!
If you don't want to continue down that path, then I think the only other options are: 1. eliminate the MIX token and live with multiple sub-tokens generated in contexts where mixed values are expected; at the parser level examine sequences of other tokens and determine which ones are really part of one value. 2. eliminate all other tokens and just produce MIXes (ie. the lexer is purely consolidating whitespace vs. non-whitespace vs. SLANTs), then in the parser figure out which MIXes consist entirely of numbers or letters and accordingly whether they're valid in the position you find them. Also note that if you're generating a tree for later use by a tree parser, you can have the parser convert the tokens once it figures out what type they should really be from their context. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en -~----------~----~----~----~------~----~------~--~---
