Thanks Hongze, that's good to know. On Thu, Mar 28, 2019 at 8:43 AM Hongze Zhang <notify...@126.com> wrote:
> > Besides, if I enable forceLaCheck, JavaCC suggests to use a lookahead of > 3 > > or more. I guess we'd better get rid of these warnings if we want to > stick > > to lookahead(2). > > That makes sense. Actually we had a discussion[1] on moving to > "LOOKAHEAD=1", and seems we are close to finish it. By doing this we have > extra benefits that we don't need to turn forceLaCheck on and JavaCC should > give suggestions during maven build. > > Hongze > > > [1] https://issues.apache.org/jira/browse/CALCITE-2847 > > > On Mar 27, 2019, at 10:40, Rui Li <lirui.fu...@gmail.com> wrote: > > > > Thanks Hongze for looking into the issue! Are you suggesting this is more > > likely to be a JavaCC bug? > > I filed a ticket anyway in case we want to further track it: > > https://issues.apache.org/jira/browse/CALCITE-2957 > > Besides, if I enable forceLaCheck, JavaCC suggests to use a lookahead of > 3 > > or more. I guess we'd better get rid of these warnings if we want to > stick > > to lookahead(2). > > > > On Wed, Mar 27, 2019 at 8:54 AM Hongze Zhang <notify...@126.com> wrote: > > > >> Thanks, Yuzhao. > >> > >> Since the more generic problem is that the production "E()"[1] causes > the > >> parent production's looking ahead returns too early, I tried to find a > bad > >> case of the same reason under current default setting LOOKAHEAD=2 but it > >> seems that under this number we didn't have a chance to meet the issue > yet. > >> > >> So after that I suggest to not to treat this as a Calcite's issue > >> currently. > >> > >> Best, > >> Hongze > >> > >> [1] > >> > https://github.com/apache/calcite/blob/11c067f9992d9c8bc29e2326dd8b299ad1e9dbdc/core/src/main/codegen/templates/Parser.jj#L335 > >> > >>> On Mar 26, 2019, at 20:42, Yuzhao Chen <yuzhao....@gmail.com> wrote: > >>> > >>> Maybe we should fire a jira if it is a bug. > >>> > >>> Best, > >>> Danny Chan > >>> 在 2019年3月26日 +0800 PM8:33，Hongze Zhang <notify...@126.com>，写道： > >>>> Ops, correct a typo: > >>>> > >>>> "... after uncommenting a line ..." -> "... after commenting a line > >>>> ...". > >>>> > >>>> Best, > >>>> Hongze > >>>> > >>>> ------ Original Message ------ > >>>> From: "Hongze Zhang" <notify...@126.com> > >>>> To: dev@calcite.apache.org > >>>> Sent: 2019/3/26 19:28:08 > >>>> Subject: Re: Calcite doesn't work with LOOKAHEAD(3) > >>>> > >>>>> Firstly, thank you very much for sharing the case, Rui! > >>>>> > >>>>> I have run a test with the SQL you provided and also run into the > same > >> exception (under a global LOOKAHEAD 3). After debugging the generated > >> parser code, I think the problem is probably in the generated LOOKAHEAD > >> method SqlParserImpl#jj_3R_42(): > >>>>> > >>>>> > >>>>>> final private boolean jj_3R_42() { > >>>>>> if (!jj_rescan) trace_call("SqlSelect(LOOKING AHEAD...)"); > >>>>>> if (jj_scan_token(SELECT)) { if (!jj_rescan) > >> trace_return("SqlSelect(LOOKAHEAD FAILED)"); return true; } > >>>>>> if (jj_3R_190()) { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD > >> FAILED)"); return true; } > >>>>>> { if (!jj_rescan) trace_return("SqlSelect(LOOKAHEAD SUCCEEDED)"); > >> return false; } > >>>>>> } > >>>>> > >>>>> The LOOKAHEAD method checks only a single token <SELECT>. This is > >> definitely not enough since we have already set the number to 3. > >>>>> > >>>>> Unfortunately I didn't find a root cause so far, but after > >> uncommenting a line[1] in production "SqlSelect()" then everything goes > >> back to normal. I'm inclined to believe JavaCC has some unexpected > behavior > >> when dealing with LOOKAHEAD on a production with the shape like > >> "SqlSelectKeywords()"[2]. > >>>>> > >>>>> Please feel free to log a JIRA ticket with which we can track further > >> information of the issue. > >>>>> > >>>>> Best, > >>>>> Hongze > >>>>> > >>>>> > >>>>> [1] > >> > https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L1030 > >>>>> [2] > >> > https://github.com/apache/calcite/blob/1b430721c0d9e22b2252ffcd893b42959cb7966c/core/src/main/codegen/templates/Parser.jj#L288 > >>>>> > >>>>> ------ Original Message ------ > >>>>> From: "Rui Li" <lirui.fu...@gmail.com> > >>>>> To: dev@calcite.apache.org > >>>>> Sent: 2019/3/26 16:53:44 > >>>>> Subject: Calcite doesn't work with LOOKAHEAD(3) > >>>>> > >>>>>> Hi, > >>>>>> > >>>>>> I'm trying to extend Calcite grammar to support some custom > >> statements. And > >>>>>> I need to increase LOOKAHEAD to 3 to resolve some ambiguity. But > when > >> I did > >>>>>> that, the parser fails to parse queries like: > >>>>>> * select t.key from (select key from src) t* > >>>>>> > >>>>>> With exception: > >>>>>> *Caused by: org.apache.calcite.sql.parser.impl.ParseException:* > >>>>>> *Encountered "( select key" at line 1, column 19.* > >>>>>> *Was expecting one of:* > >>>>>> * <IDENTIFIER> ...* > >>>>>> * <QUOTED_IDENTIFIER> ...* > >>>>>> * <BACK_QUOTED_IDENTIFIER> ...* > >>>>>> * <BRACKET_QUOTED_IDENTIFIER> ...* > >>>>>> * <UNICODE_QUOTED_IDENTIFIER> ...* > >>>>>> * "LATERAL" ...* > >>>>>> * "(" "WITH" ...* > >>>>>> *...* > >>>>>> > >>>>>> So I'm wondering whether there's some limitation on the LOOKAHEAD we > >> can > >>>>>> use? > >>>>>> > >>>>>> -- > >>>>>> Best regards! > >>>>>> Rui Li > >> > >> > > > > -- > > Best regards! > > Rui Li > > -- Best regards! Rui Li
