The example on the Wiki already does all of this in the lexer, but without any code.
Jim > -----Original Message----- > From: [email protected] [mailto:antlr-interest- > [email protected]] On Behalf Of Bart Kiers > Sent: Friday, November 04, 2011 7:12 AM > To: Fabien Hermenier > Cc: [email protected] > Subject: Re: [antlr-interest] about range float and stuff > > You're welcome Fabien, but note that it most likely looks a lot like > something I found on the ANTLR Wiki: so I can't claim credit for it > (perhaps a small part! :)). > I'll have a look later on and see if I can dig up the Wiki page. > > Regards, > > Bart. > > > On Fri, Nov 4, 2011 at 3:04 PM, Fabien Hermenier > <[email protected]>wrote: > > > Thanks Bart, I think I have understand your approach and indeed, it > > seems beautiful and simple. > > I will try your solution during the week-end. > > > > Fabien. > > > > Le 04/11/11 02:48, Bart Kiers a écrit : > > > > Hi Fabien, > > > > Handling this in the parser will make your life much harder than it > > has to. Doing it in the lexer, you will need a bit of custom code, > but > > I'd go for something similar to this (something like it is on the > WIki > > somewhere, but can't find it...): > > > > grammar RangeDemo; > > > > @lexer::members { > > > > java.util.Queue<Token> tokens = new java.util.LinkedList<Token>(); > > > > public void offer(int ttype, String ttext) { > > emit(new CommonToken(ttype, ttext)); > > } > > > > @Override > > public void emit(Token t) { > > state.token = t; > > tokens.offer(t); > > } > > > > @Override > > public Token nextToken() { > > super.nextToken(); > > return tokens.isEmpty() ? Token.EOF_TOKEN : tokens.poll(); > > } > > } > > > > parse > > : (t=. {System.out.printf("\%-10s \%s\n", tokenNames[$t.type], > > $t.text);})* EOF > > ; > > > > FLOAT > > : INT '..' {offer(INT, $INT.text); offer(RANGE, "..");} > > | OCTAL '..' {offer(OCTAL, $OCTAL.text); offer(RANGE, "..");} > > | '.' DIGITS > > | DIGITS '.' DIGITS? > > ; > > > > RANGE > > : '..' > > ; > > > > INT > > : '1'..'9' DIGIT* > > | '0' > > ; > > > > OCTAL > > : '0' ('0'..'7')+ > > ; > > > > fragment DIGITS : DIGIT+; > > fragment DIGIT : '0'..'9'; > > > > SPACE > > : (' ' | '\t' | '\r' | '\n') {skip();} > > ; > > > > And if you run the class: > > > > import org.antlr.runtime.*; > > > > public class Main { > > public static void main(String[] args) throws Exception { > > String src = "..07..8.5 1.9..02 1..3.4"; > > RangeDemoLexer lexer = new RangeDemoLexer(new > ANTLRStringStream(src)); > > RangeDemoParser parser = new RangeDemoParser(new > > CommonTokenStream(lexer)); > > System.out.println("Parsing: '" + src + "'"); > > parser.parse(); > > } > > } > > > > You'll see the following being printed to the console: > > > > Parsing: '..07..8.5 1.9..02 1..3.4' > > RANGE .. > > OCTAL 07 > > RANGE .. > > FLOAT 8.5 > > FLOAT 1.9 > > RANGE .. > > OCTAL 02 > > INT 1 > > RANGE .. > > FLOAT 3.4 > > > > > > Regards, > > > > Bart. > > > > > > > > On Fri, Nov 4, 2011 at 7:28 AM, Fabien Hermenier < > > [email protected]> wrote: > > > >> Hi > >> > >> In an earlier version of my language, I had to parse range of > >> integers in various base. Now I want to include float. I have read > >> > >> > http://www.antlr.org/wiki/display/ANTLR3/Lexer+grammar+for+floating+p > >> oint,+dot,+range,+time+specs > >> but I've still got some questions. > >> > >> All the work seems to be done at the lexer level so the type of the > >> following tokens will be as example: > >> 5 : DECIMAL_LITTERAL > >> 07 : OCTAL_LITTERAL > >> 7.5: FLOATING_POINT_LITTERAL > >> 5..7 : DOTDOT > >> > >> In the last example, the result is not very convenient because I > will > >> still have to extract the bounds and compute their type by myself > >> which seems quite redundant with the job performed by the lexer. > >> May be I am missing something ? > >> > >> I would rather be able to express the range at the parser level > which > >> seems much more convenient to me: > >> range: FLOATING_POINT_LITTERAL DOTDOT FLOATING_POINT_LITTERAL. > >> In this way, I will also be able to manage the possible spaces > >> between the bounds and the DOTDOT. > >> > >> So, am I right to try to parse range at the parser level ? Or is > >> there a solution to extract easily the bounds with their type if I > am > >> doing the job at the lexer level ? > >> > >> Thanks in advance, > >> Fabien. > >> > >> > >> > >> > >> List: http://www.antlr.org/mailman/listinfo/antlr-interest > >> Unsubscribe: > >> http://www.antlr.org/mailman/options/antlr-interest/your-email- > addres > >> s > >> > > > > > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- > email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
