It's funny that you say that backtracking will not work in lexers. I don't know if it would help in the particular case the OP asked about, but I've found that it not only works, it's the only way to get Antlr lexers to work the way lexers are generally expected to work based on experience with FSA-based lexers. With backtracking turned on, rules such as for floating point and integer tokens can be kept separate (as long as they're in the right order). Without backtracking the first possible match is the only one that will ever be tried (if the float rule is first, only floats can be lexed, never integers; if the integer rule is first, floats can never be lexed), so it becomes necessary to merge all such rules into one big ugly rule with lots of predicates and set type calls.
Ron Jim Idle wrote: > 1) Backtracking will not work in lexers > 2) Don't advise people to use backtrack mode > 3) That rule will give warnings anyway > > > BTW Guys, don't forget antlr.markmail.org !!! > > Jim > > >> -----Original Message----- >> From: [email protected] [mailto:antlr-interest- >> [email protected]] On Behalf Of Bart Kiers >> Sent: Thursday, March 25, 2010 6:51 AM >> To: Anders Sollander >> Cc: [email protected] >> Subject: Re: [antlr-interest] Lexer rule for strings with quoted >> strings within >> >> On Thu, Mar 25, 2010 at 2:01 PM, Anders Sollander < >> [email protected]> wrote: >> >> >>> Hi, >>> >>> I've been trying to write a lexer rule for strings with strings in >>> >> them, >> >>> like >>> >>> "This has a ""quoted string"" within" >>> >>> Is there a simple lexer rule for this, or do I need some kind of look >>> ahead? >>> >>> >>> >> You could enable the `backtrack` option: >> >> String >> options { backtrack = true; } >> : '"' ('""' | ~'"')+ '"' >> ; >> >> Regards, >> >> Bart. >> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest >> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- >> email-address >> > > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: > http://www.antlr.org/mailman/options/antlr-interest/your-email-address > > -- Ron Hunter-Duvar | Software Developer V | 403-272-6580 Oracle Service Engineering Gulf Canada Square 401 - 9th Avenue S.W., Calgary, AB, Canada T2P 3C5 All opinions expressed here are mine, and do not necessarily represent those of my employer. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
