Sure, we can make sure something gets into 1.22. There is consensus about the parser extensions, whereas the extensions to RelNode and the planner engine are a little more experimental. So let’s go forward with that, stating which parts we think are likely to change.
Julian > On Oct 29, 2019, at 2:09 AM, Seliverstov Igor <gvvinbl...@gmail.com> wrote: > > Colleagues, > > Not only Hazelcast and Apache Flink are interested in SQL hints. Apache > Ignite community is working on Calcite integration too, it’s important for us > to have appropriate API at current development stage. This case we’ll be able > to adapt our solution for SQL hints usage, probably determining additional > approach weaknesses or inconveniences. > > Regards, > Igor > >> 29 окт. 2019 г., в 11:51, Danny Chan <yuzhao....@gmail.com> написал(а): >> >> Julian, can we make some effort to push this feature into release 1.22, >> there are users like Vladimir Ozerov from Hazelcast that are interesting on >> this feature, also the Apache Flink. >> >> I agree that this internal design is not that perfect, at this moment, we >> may hardly to conclude a perfect solution, but at least, the syntax would >> remain unchanged in the future. >> >> So can we mark this feature as experimental and we can promote the internal >> design when accept more feedbacks from the Calcite uses (from Apache Flink >> or from users like Vladimir). >> >> Best, >> Danny Chan >> 在 2019年10月18日 +0800 AM4:55，Julian Hyde <jh...@apache.org>，写道： >>> I wonder whether it is possible to add some kind of “action handler” to the >>> planner engine, called, for example, when a rule has fired and is >>> registering the RelNode created by the rule. People can write their own >>> action handlers to copy hints around. Since the action handlers are the >>> user’s code, they can iterate faster to find a hint-propagation strategy >>> that works in practice. >>> >>> Another idea is to use VolcanoPlanner.Provenance[1]. A RelNode can find its >>> ancestor RelNodes, and the rules that fired to create it. So it can grab >>> hints from those ancestors. It does not need to copy those hints onto >>> itself. >>> >>> Julian >>> >>> [1] >>> https://calcite.apache.org/apidocs/org/apache/calcite/plan/volcano/VolcanoPlanner.Provenance.html >>> >>> <https://calcite.apache.org/apidocs/org/apache/calcite/plan/volcano/VolcanoPlanner.Provenance.html> >>> >>>> On Oct 16, 2019, at 8:38 PM, Haisheng Yuan <h.y...@alibaba-inc.com> wrote: >>>> >>>> Julian, >>>> Your concern is very valid, and that is also our main concern. >>>> I was thinking whether we can put hint into the MEMO group, so that both >>>> logical and physical expression in the same group can share the same hint, >>>> without copying the hint explicitly. But for newly generated expression >>>> that doesn't belong to the original group, we still need to copy hints. >>>> What's worse, in HepPlanner, there is no such concept, we may still need >>>> to copy hints explicity in planner rules, if we want to keep the hint, >>>> which is burdensome. >>>> >>>> - Haisheng >>>> >>>> ------------------------------------------------------------------ >>>> 发件人：Danny Chan<yuzhao....@gmail.com> >>>> 日 期：2019年10月16日 14:54:46 >>>> 收件人：<dev@calcite.apache.org> >>>> 主 题：Re: [DISCUSS] Support Sql Hint for Calcite >>>> >>>> Thanks for the clarification. >>>> >>>> I understand you worried. Yes, the effort/memory would be wasted or >>>> meaningless if hints are not used. This is just what a hint does, it is a >>>> “hint” and non-mandatory, but we should give the chance to let user see >>>> them, it is the use that decide if to use the hints and how to use them. >>>> For big queries I have no confidence to cover the corner cases. So can we >>>> mark this feature as experimental and used for simple queries(no >>>> decorrelation) first ? >>>> >>>> For “reversible”, during the implementation, I try to make the >>>> modifications non-invasive with the current codes. That is why I made all >>>> the interfaces about the hint into one class named RelWithHInt. Different >>>> with trait, I didn’t force users to pass in the hints in the RelNode >>>> constructor. I think if is not a bigwork if we want to remove the API. >>>> >>>> Best, >>>> Danny Chan >>>> 在 2019年10月16日 +0800 AM11:14，Julian Hyde <jh...@apache.org>，写道： >>>>> By “skeptical” I mean that I think we can come up with a mechanism to >>>>> copy hints when applying planner rules, but even when we have implemented >>>>> that mechanism there will be many cases where people want a hint and that >>>>> hint is not copied to the RelNode where it is needed, and many other >>>>> cases where we spend the effort/memory of copying the hint to a RelNode >>>>> and the hint is not used. >>>>> >>>>> By “reversible” I mean if we come up with an API that does not work, how >>>>> do we change or remove that API without people complaining? >>>>> >>>>> Julian >>>>> >>>>> >>>>>> On Oct 15, 2019, at 7:11 PM, Danny Chan <yuzhao....@gmail.com> wrote: >>>>>> >>>>>> Thanks Julian >>>>>> >>>>>>> I am skeptical that RelWithHint will work for large queries. >>>>>> >>>>>> For “skeptical” do you mean how to transfer the hints during rule >>>>>> planning ? I’m also not that confident yet. >>>>>> >>>>>>> How do we introduce it in a reversible way >>>>>> Do you mean transform the RelWithHint back into the SqlHint ? I didn’t >>>>>> implement it in current patch, but I think we have the ability to do >>>>>> that because we have a inheritPath for each RelWithHint, we can collect >>>>>> all the hints together and merge them into the SqlHints, then propagate >>>>>> these SqlHints to the SqlNodes. >>>>>> >>>>>>> What are the other options? >>>>>> Do you mean the way to transfer hints during planning ? I have no other >>>>>> options yet. >>>>>> >>>>>> Best, >>>>>> Danny Chan >>>>>> 在 2019年10月16日 +0800 AM8:03，dev@calcite.apache.org，写道： >>>>>>> >>>>>>> I am skeptical that RelWithHint will work for large queries. >>>>> >>>> >>> >
