Thanks, Yuzhao. Since the more generic problem is that the production "E()"[1] causes the parent production's looking ahead returns too early, I tried to find a bad case of the same reason under current default setting LOOKAHEAD=2 but it seems that under this number we didn't have a chance to meet the issue yet.
So after that I suggest to not to treat this as a Calcite's issue currently. Best, Hongze [1] https://github.com/apache/calcite/blob/11c067f9992d9c8bc29e2326dd8b299ad1e9dbdc/core/src/main/codegen/templates/Parser.jj#L335 > On Mar 26, 2019, at 20:42, Yuzhao Chen <[email protected]> wrote: > > Maybe we should fire a jira if it is a bug. > > Best, > Danny Chan > 在 2019年3月26日 +0800 PM8:33，Hongze Zhang <[email protected]>，写道： >> Ops, correct a typo: >> >> "... after uncommenting a line ..." -> "... after commenting a line >> ...". >> >> Best, >> Hongze >> >> ------ Original Message ------ >> From: "Hongze Zhang" <[email protected]> >> To: [email protected] >> Sent: 2019/3/26 19:28:08 >> Subject: Re: Calcite doesn't work with LOOKAHEAD(3) >> >>> Firstly, thank you very much for sharing the case, Rui! >>> >>> I have run a test with the SQL you provided and also run into the same >>> exception (under a global LOOKAHEAD 3). After debugging the generated >>> parser code, I think the problem is probably in the generated LOOKAHEAD >>> method SqlParserImpl#jj_3R_42(): >>> >>> >>>> final private boolean jj_3R_42() { >>>> if (!jj_rescan) trace_call("SqlSelect(LOOKING AHEAD...)"); >>>> if (jj_scan_token(SELECT)) { if (!jj_rescan) >>>> trace_return("SqlSelect(LOOKAHEAD FAILED)"); return true; } >>>> if (jj_3R_190()) { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD >>>> FAILED)"); return true; } >>>> { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD SUCCEEDED)"); return >>>> false; } >>>> } >>> >>> The LOOKAHEAD method checks only a single token <SELECT>. This is >>> definitely not enough since we have already set the number to 3. >>> >>> Unfortunately I didn't find a root cause so far, but after uncommenting a >>> line[1] in production "SqlSelect()" then everything goes back to normal. >>> I'm inclined to believe JavaCC has some unexpected behavior when dealing >>> with LOOKAHEAD on a production with the shape like "SqlSelectKeywords()"[2]. >>> >>> Please feel free to log a JIRA ticket with which we can track further >>> information of the issue. >>> >>> Best, >>> Hongze >>> >>> >>> [1] >>> https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L1030 >>> [2] >>> https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L288 >>> >>> ------ Original Message ------ >>> From: "Rui Li" <[email protected]> >>> To: [email protected] >>> Sent: 2019/3/26 16:53:44 >>> Subject: Calcite doesn't work with LOOKAHEAD(3) >>> >>>> Hi, >>>> >>>> I'm trying to extend Calcite grammar to support some custom statements. And >>>> I need to increase LOOKAHEAD to 3 to resolve some ambiguity. But when I did >>>> that, the parser fails to parse queries like: >>>> * select t.key from (select key from src) t* >>>> >>>> With exception: >>>> *Caused by: org.apache.calcite.sql.parser.impl.ParseException:* >>>> *Encountered "( select key" at line 1, column 19.* >>>> *Was expecting one of:* >>>> * <IDENTIFIER> ...* >>>> * <QUOTED_IDENTIFIER> ...* >>>> * <BACK_QUOTED_IDENTIFIER> ...* >>>> * <BRACKET_QUOTED_IDENTIFIER> ...* >>>> * <UNICODE_QUOTED_IDENTIFIER> ...* >>>> * "LATERAL" ...* >>>> * "(" "WITH" ...* >>>> *...* >>>> >>>> So I'm wondering whether there's some limitation on the LOOKAHEAD we can >>>> use? >>>> >>>> -- >>>> Best regards! >>>> Rui Li
