Thanks Hongze for looking into the issue! Are you suggesting this is more likely to be a JavaCC bug? I filed a ticket anyway in case we want to further track it: https://issues.apache.org/jira/browse/CALCITE-2957 Besides, if I enable forceLaCheck, JavaCC suggests to use a lookahead of 3 or more. I guess we'd better get rid of these warnings if we want to stick to lookahead(2).
On Wed, Mar 27, 2019 at 8:54 AM Hongze Zhang <notify...@126.com> wrote: > Thanks, Yuzhao. > > Since the more generic problem is that the production "E()"[1] causes the > parent production's looking ahead returns too early, I tried to find a bad > case of the same reason under current default setting LOOKAHEAD=2 but it > seems that under this number we didn't have a chance to meet the issue yet. > > So after that I suggest to not to treat this as a Calcite's issue > currently. > > Best, > Hongze > > [1] > https://github.com/apache/calcite/blob/11c067f9992d9c8bc29e2326dd8b299ad1e9dbdc/core/src/main/codegen/templates/Parser.jj#L335 > > > On Mar 26, 2019, at 20:42, Yuzhao Chen <yuzhao....@gmail.com> wrote: > > > > Maybe we should fire a jira if it is a bug. > > > > Best, > > Danny Chan > > 在 2019年3月26日 +0800 PM8:33，Hongze Zhang <notify...@126.com>，写道： > >> Ops, correct a typo: > >> > >> "... after uncommenting a line ..." -> "... after commenting a line > >> ...". > >> > >> Best, > >> Hongze > >> > >> ------ Original Message ------ > >> From: "Hongze Zhang" <notify...@126.com> > >> To: dev@calcite.apache.org > >> Sent: 2019/3/26 19:28:08 > >> Subject: Re: Calcite doesn't work with LOOKAHEAD(3) > >> > >>> Firstly, thank you very much for sharing the case, Rui! > >>> > >>> I have run a test with the SQL you provided and also run into the same > exception (under a global LOOKAHEAD 3). After debugging the generated > parser code, I think the problem is probably in the generated LOOKAHEAD > method SqlParserImpl#jj_3R_42(): > >>> > >>> > >>>> final private boolean jj_3R_42() { > >>>> if (!jj_rescan) trace_call("SqlSelect(LOOKING AHEAD...)"); > >>>> if (jj_scan_token(SELECT)) { if (!jj_rescan) > trace_return("SqlSelect(LOOKAHEAD FAILED)"); return true; } > >>>> if (jj_3R_190()) { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD > FAILED)"); return true; } > >>>> { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD SUCCEEDED)"); > return false; } > >>>> } > >>> > >>> The LOOKAHEAD method checks only a single token <SELECT>. This is > definitely not enough since we have already set the number to 3. > >>> > >>> Unfortunately I didn't find a root cause so far, but after > uncommenting a line[1] in production "SqlSelect()" then everything goes > back to normal. I'm inclined to believe JavaCC has some unexpected behavior > when dealing with LOOKAHEAD on a production with the shape like > "SqlSelectKeywords()"[2]. > >>> > >>> Please feel free to log a JIRA ticket with which we can track further > information of the issue. > >>> > >>> Best, > >>> Hongze > >>> > >>> > >>> [1] > https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L1030 > >>> [2] > https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L288 > >>> > >>> ------ Original Message ------ > >>> From: "Rui Li" <lirui.fu...@gmail.com> > >>> To: dev@calcite.apache.org > >>> Sent: 2019/3/26 16:53:44 > >>> Subject: Calcite doesn't work with LOOKAHEAD(3) > >>> > >>>> Hi, > >>>> > >>>> I'm trying to extend Calcite grammar to support some custom > statements. And > >>>> I need to increase LOOKAHEAD to 3 to resolve some ambiguity. But when > I did > >>>> that, the parser fails to parse queries like: > >>>> * select t.key from (select key from src) t* > >>>> > >>>> With exception: > >>>> *Caused by: org.apache.calcite.sql.parser.impl.ParseException:* > >>>> *Encountered "( select key" at line 1, column 19.* > >>>> *Was expecting one of:* > >>>> * <IDENTIFIER> ...* > >>>> * <QUOTED_IDENTIFIER> ...* > >>>> * <BACK_QUOTED_IDENTIFIER> ...* > >>>> * <BRACKET_QUOTED_IDENTIFIER> ...* > >>>> * <UNICODE_QUOTED_IDENTIFIER> ...* > >>>> * "LATERAL" ...* > >>>> * "(" "WITH" ...* > >>>> *...* > >>>> > >>>> So I'm wondering whether there's some limitation on the LOOKAHEAD we > can > >>>> use? > >>>> > >>>> -- > >>>> Best regards! > >>>> Rui Li > > -- Best regards! Rui Li
