Firstly, thank you very much for sharing the case, Rui!
I have run a test with the SQL you provided and also run into the same
exception (under a global LOOKAHEAD 3). After debugging the generated
parser code, I think the problem is probably in the generated LOOKAHEAD
method SqlParserImpl#jj_3R_42():
> final private boolean jj_3R_42() {
> if (!jj_rescan) trace_call("SqlSelect(LOOKING AHEAD...)");
> if (jj_scan_token(SELECT)) { if (!jj_rescan)
trace_return("SqlSelect(LOOKAHEAD FAILED)"); return true; }
> if (jj_3R_190()) { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD
FAILED)"); return true; }
> { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD SUCCEEDED)");
return false; }
> }
The LOOKAHEAD method checks only a single token <SELECT>. This is
definitely not enough since we have already set the number to 3.
Unfortunately I didn't find a root cause so far, but after uncommenting
a line[1] in production "SqlSelect()" then everything goes back to
normal. I'm inclined to believe JavaCC has some unexpected behavior when
dealing with LOOKAHEAD on a production with the shape like
"SqlSelectKeywords()"[2].
Please feel free to log a JIRA ticket with which we can track further
information of the issue.
Best,
Hongze
[1]
https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L1030
[2]
https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L288
------ Original Message ------
From: "Rui Li" <[email protected]>
To: [email protected]
Sent: 2019/3/26 16:53:44
Subject: Calcite doesn't work with LOOKAHEAD(3)
Hi,
I'm trying to extend Calcite grammar to support some custom statements. And
I need to increase LOOKAHEAD to 3 to resolve some ambiguity. But when I did
that, the parser fails to parse queries like:
* select t.key from (select key from src) t*
With exception:
*Caused by: org.apache.calcite.sql.parser.impl.ParseException:*
*Encountered "( select key" at line 1, column 19.*
*Was expecting one of:*
* <IDENTIFIER> ...*
* <QUOTED_IDENTIFIER> ...*
* <BACK_QUOTED_IDENTIFIER> ...*
* <BRACKET_QUOTED_IDENTIFIER> ...*
* <UNICODE_QUOTED_IDENTIFIER> ...*
* "LATERAL" ...*
* "(" "WITH" ...*
*...*
So I'm wondering whether there's some limitation on the LOOKAHEAD we can
use?
--
Best regards!
Rui Li
