I have been trying to work through an issue with an infinite loop caused
when no tokens can be matched because a predicate has failed its test. The
problem appears to be in the Lexer.NextToken() (looking at CSharp2 sources)
method, which I have copied here for reference:
/// <summary>
/// Return a token from this source; i.e., Match a token on the
char stream.
/// </summary>
public virtual IToken NextToken()
{
while (true)
{
state.token = null;
state.channel = Token.DEFAULT_CHANNEL;
state.tokenStartCharIndex = input.Index;
state.tokenStartCharPositionInLine =
input.CharPositionInLine;
state.tokenStartLine = input.Line;
state.text = null;
if (input.LA(1) == (int)CharStreamConstants.EOF)
{
return Token.EOF_TOKEN;
}
try
{
mTokens();
if (state.token == null)
{
Emit();
}
else if (state.token ==
Token.SKIP_TOKEN)
{
continue;
}
return state.token;
}
catch (NoViableAltException nva) {
ReportError(nva);
Recover(nva); // throw out current char
and try again
}
catch (RecognitionException re) {
ReportError(re);
// Match() routine has already called
Recover()
}
}
}
Note the RecognitionException clause. This is the clause which will
catch the FailedPredicateException(). Unfortunately, because the
FailedPredicateException gets thrown just before Match() occurs in the
rule, Recover will *not* have been called by the rule or its callees,
and therefore the DFA will continue to try processing the same token.
It would appear that there should instead be a specific
FailedPredicateException clause which does the same thing as the
NoViableAltException clause.
I have seen two other people ask questions about this error, and in
neither case was a suitable response given. Has this bug been fixed
in non-released builds? Can someone give me an up-or-down on whether
this is a correct and appropriate fix?
Thanks.
- Cliff
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
