> Besides, if I enable forceLaCheck, JavaCC suggests to use a lookahead of 3 > or more. I guess we'd better get rid of these warnings if we want to stick > to lookahead(2).
That makes sense. Actually we had a discussion[1] on moving to "LOOKAHEAD=1", and seems we are close to finish it. By doing this we have extra benefits that we don't need to turn forceLaCheck on and JavaCC should give suggestions during maven build. Hongze [1] https://issues.apache.org/jira/browse/CALCITE-2847 > On Mar 27, 2019, at 10:40, Rui Li <lirui.fu...@gmail.com> wrote: > > Thanks Hongze for looking into the issue! Are you suggesting this is more > likely to be a JavaCC bug? > I filed a ticket anyway in case we want to further track it: > https://issues.apache.org/jira/browse/CALCITE-2957 > Besides, if I enable forceLaCheck, JavaCC suggests to use a lookahead of 3 > or more. I guess we'd better get rid of these warnings if we want to stick > to lookahead(2). > > On Wed, Mar 27, 2019 at 8:54 AM Hongze Zhang <notify...@126.com> wrote: > >> Thanks, Yuzhao. >> >> Since the more generic problem is that the production "E()"[1] causes the >> parent production's looking ahead returns too early, I tried to find a bad >> case of the same reason under current default setting LOOKAHEAD=2 but it >> seems that under this number we didn't have a chance to meet the issue yet. >> >> So after that I suggest to not to treat this as a Calcite's issue >> currently. >> >> Best, >> Hongze >> >> [1] >> https://github.com/apache/calcite/blob/11c067f9992d9c8bc29e2326dd8b299ad1e9dbdc/core/src/main/codegen/templates/Parser.jj#L335 >> >>> On Mar 26, 2019, at 20:42, Yuzhao Chen <yuzhao....@gmail.com> wrote: >>> >>> Maybe we should fire a jira if it is a bug. >>> >>> Best, >>> Danny Chan >>> 在 2019年3月26日 +0800 PM8:33，Hongze Zhang <notify...@126.com>，写道： >>>> Ops, correct a typo: >>>> >>>> "... after uncommenting a line ..." -> "... after commenting a line >>>> ...". >>>> >>>> Best, >>>> Hongze >>>> >>>> ------ Original Message ------ >>>> From: "Hongze Zhang" <notify...@126.com> >>>> To: dev@calcite.apache.org >>>> Sent: 2019/3/26 19:28:08 >>>> Subject: Re: Calcite doesn't work with LOOKAHEAD(3) >>>> >>>>> Firstly, thank you very much for sharing the case, Rui! >>>>> >>>>> I have run a test with the SQL you provided and also run into the same >> exception (under a global LOOKAHEAD 3). After debugging the generated >> parser code, I think the problem is probably in the generated LOOKAHEAD >> method SqlParserImpl#jj_3R_42(): >>>>> >>>>> >>>>>> final private boolean jj_3R_42() { >>>>>> if (!jj_rescan) trace_call("SqlSelect(LOOKING AHEAD...)"); >>>>>> if (jj_scan_token(SELECT)) { if (!jj_rescan) >> trace_return("SqlSelect(LOOKAHEAD FAILED)"); return true; } >>>>>> if (jj_3R_190()) { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD >> FAILED)"); return true; } >>>>>> { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD SUCCEEDED)"); >> return false; } >>>>>> } >>>>> >>>>> The LOOKAHEAD method checks only a single token <SELECT>. This is >> definitely not enough since we have already set the number to 3. >>>>> >>>>> Unfortunately I didn't find a root cause so far, but after >> uncommenting a line[1] in production "SqlSelect()" then everything goes >> back to normal. I'm inclined to believe JavaCC has some unexpected behavior >> when dealing with LOOKAHEAD on a production with the shape like >> "SqlSelectKeywords()"[2]. >>>>> >>>>> Please feel free to log a JIRA ticket with which we can track further >> information of the issue. >>>>> >>>>> Best, >>>>> Hongze >>>>> >>>>> >>>>> [1] >> https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L1030 >>>>> [2] >> https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L288 >>>>> >>>>> ------ Original Message ------ >>>>> From: "Rui Li" <lirui.fu...@gmail.com> >>>>> To: dev@calcite.apache.org >>>>> Sent: 2019/3/26 16:53:44 >>>>> Subject: Calcite doesn't work with LOOKAHEAD(3) >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm trying to extend Calcite grammar to support some custom >> statements. And >>>>>> I need to increase LOOKAHEAD to 3 to resolve some ambiguity. But when >> I did >>>>>> that, the parser fails to parse queries like: >>>>>> * select t.key from (select key from src) t* >>>>>> >>>>>> With exception: >>>>>> *Caused by: org.apache.calcite.sql.parser.impl.ParseException:* >>>>>> *Encountered "( select key" at line 1, column 19.* >>>>>> *Was expecting one of:* >>>>>> * <IDENTIFIER> ...* >>>>>> * <QUOTED_IDENTIFIER> ...* >>>>>> * <BACK_QUOTED_IDENTIFIER> ...* >>>>>> * <BRACKET_QUOTED_IDENTIFIER> ...* >>>>>> * <UNICODE_QUOTED_IDENTIFIER> ...* >>>>>> * "LATERAL" ...* >>>>>> * "(" "WITH" ...* >>>>>> *...* >>>>>> >>>>>> So I'm wondering whether there's some limitation on the LOOKAHEAD we >> can >>>>>> use? >>>>>> >>>>>> -- >>>>>> Best regards! >>>>>> Rui Li >> >> > > -- > Best regards! > Rui Li
