This just means that you haven't covered the predicated cases correctly. In general it means that you needed a gated predicate , not a simple ? predicate. Post the code for further help.
Basically your lexer should not throw any exceptions or reach unrecognizable input. Jim > -----Original Message----- > From: [email protected] [mailto:antlr-interest- > [email protected]] On Behalf Of Ron Hunter-Duvar > Sent: Wednesday, February 10, 2010 11:25 AM > To: [email protected] > Subject: [antlr-interest] Antlr Bug: Failed semantic predicate in lexer > triggers endless loop > > Hi, > > I've run into something that is definitely a bug in Antlr's lexer code: > if a semantic predicate fails within a lexer rule, it triggers an > endless loop. The problem is in the Lexer.nextToken() method: > > public Token nextToken() { > while (true) { > state.token = null; > state.channel = Token.DEFAULT_CHANNEL; > state.tokenStartCharIndex = input.index(); > state.tokenStartCharPositionInLine = > input.getCharPositionInLine(); > state.tokenStartLine = input.getLine(); > state.text = null; > if ( input.LA(1)==CharStream.EOF ) { > return Token.EOF_TOKEN; > } > try { > mTokens(); > if ( state.token==null ) { > emit(); > } > else if ( state.token==Token.SKIP_TOKEN ) { > continue; > } > return state.token; > } > catch (NoViableAltException nva) { > reportError(nva); > recover(nva); // throw out current char and try again > } > catch (RecognitionException re) { > reportError(re); > // match() routine has already called recover() > } > } > } > > If a NoViableAltException is thrown, the recover method is called, > which > consumes one character and continues. But when a semantic predicate > fails, it throws a FailedPredicateException, which is a subclass of > RecognitionException. As you can see in the code above, the exception > is > caught and reported, but it then loops around and tries matching again > at the same point in the input, resulting in the same exception. Here's > a sample of Antlr's output messages: > > line 1:21 rule FLOAT failed predicate: { notIntFollowedByRangeOp() }? > line 1:21 rule FLOAT failed predicate: { notIntFollowedByRangeOp() }? > line 1:21 rule FLOAT failed predicate: { notIntFollowedByRangeOp() }? > line 1:21 rule FLOAT failed predicate: { notIntFollowedByRangeOp() }? > line 1:21 rule FLOAT failed predicate: { notIntFollowedByRangeOp() }? > ... > > I was able to work around this easily because I already had a custom > lexer superclass, so I just pasted in that nextToken() code and added a > "recover(re);" call to the second catch. > > Ron > > -- > Ron Hunter-Duvar | Software Developer V | 403-272-6580 > Oracle Service Engineering > Gulf Canada Square 401 - 9th Avenue S.W., Calgary, AB, Canada T2P 3C5 > > All opinions expressed here are mine, and do not necessarily represent > those of my employer. > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- > email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
