Hi, fellows, I’m planning to merge the hints PR in the following week, I’m very appreciated if you have other more review comment address. [1]
Or if you have other thoughts, please address it here :) [1] https://github.com/apache/calcite/pull/1354 Best, Danny Chan 在 2019年10月30日 +0800 AM1:42，Julian Hyde <[email protected]>，写道： > Sure, we can make sure something gets into 1.22. There is consensus about the > parser extensions, whereas the extensions to RelNode and the planner engine > are a little more experimental. So let’s go forward with that, stating which > parts we think are likely to change. > > Julian > > > > On Oct 29, 2019, at 2:09 AM, Seliverstov Igor <[email protected]> wrote: > > > > Colleagues, > > > > Not only Hazelcast and Apache Flink are interested in SQL hints. Apache > > Ignite community is working on Calcite integration too, it’s important for > > us to have appropriate API at current development stage. This case we’ll be > > able to adapt our solution for SQL hints usage, probably determining > > additional approach weaknesses or inconveniences. > > > > Regards, > > Igor > > > > > 29 окт. 2019 г., в 11:51, Danny Chan <[email protected]> написал(а): > > > > > > Julian, can we make some effort to push this feature into release 1.22, > > > there are users like Vladimir Ozerov from Hazelcast that are interesting > > > on this feature, also the Apache Flink. > > > > > > I agree that this internal design is not that perfect, at this moment, we > > > may hardly to conclude a perfect solution, but at least, the syntax would > > > remain unchanged in the future. > > > > > > So can we mark this feature as experimental and we can promote the > > > internal design when accept more feedbacks from the Calcite uses (from > > > Apache Flink or from users like Vladimir). > > > > > > Best, > > > Danny Chan > > > 在 2019年10月18日 +0800 AM4:55，Julian Hyde <[email protected]>，写道： > > > > I wonder whether it is possible to add some kind of “action handler” to > > > > the planner engine, called, for example, when a rule has fired and is > > > > registering the RelNode created by the rule. People can write their own > > > > action handlers to copy hints around. Since the action handlers are the > > > > user’s code, they can iterate faster to find a hint-propagation > > > > strategy that works in practice. > > > > > > > > Another idea is to use VolcanoPlanner.Provenance[1]. A RelNode can find > > > > its ancestor RelNodes, and the rules that fired to create it. So it can > > > > grab hints from those ancestors. It does not need to copy those hints > > > > onto itself. > > > > > > > > Julian > > > > > > > > [1] > > > > https://calcite.apache.org/apidocs/org/apache/calcite/plan/volcano/VolcanoPlanner.Provenance.html > > > > > > > > <https://calcite.apache.org/apidocs/org/apache/calcite/plan/volcano/VolcanoPlanner.Provenance.html> > > > > > > > > > On Oct 16, 2019, at 8:38 PM, Haisheng Yuan <[email protected]> > > > > > wrote: > > > > > > > > > > Julian, > > > > > Your concern is very valid, and that is also our main concern. > > > > > I was thinking whether we can put hint into the MEMO group, so that > > > > > both logical and physical expression in the same group can share the > > > > > same hint, without copying the hint explicitly. But for newly > > > > > generated expression that doesn't belong to the original group, we > > > > > still need to copy hints. What's worse, in HepPlanner, there is no > > > > > such concept, we may still need to copy hints explicity in planner > > > > > rules, if we want to keep the hint, which is burdensome. > > > > > > > > > > - Haisheng > > > > > > > > > > ------------------------------------------------------------------ > > > > > 发件人：Danny Chan<[email protected]> > > > > > 日 期：2019年10月16日 14:54:46 > > > > > 收件人：<[email protected]> > > > > > 主 题：Re: [DISCUSS] Support Sql Hint for Calcite > > > > > > > > > > Thanks for the clarification. > > > > > > > > > > I understand you worried. Yes, the effort/memory would be wasted or > > > > > meaningless if hints are not used. This is just what a hint does, it > > > > > is a “hint” and non-mandatory, but we should give the chance to let > > > > > user see them, it is the use that decide if to use the hints and how > > > > > to use them. For big queries I have no confidence to cover the corner > > > > > cases. So can we mark this feature as experimental and used for > > > > > simple queries(no decorrelation) first ? > > > > > > > > > > For “reversible”, during the implementation, I try to make the > > > > > modifications non-invasive with the current codes. That is why I made > > > > > all the interfaces about the hint into one class named RelWithHInt. > > > > > Different with trait, I didn’t force users to pass in the hints in > > > > > the RelNode constructor. I think if is not a bigwork if we want to > > > > > remove the API. > > > > > > > > > > Best, > > > > > Danny Chan > > > > > 在 2019年10月16日 +0800 AM11:14，Julian Hyde <[email protected]>，写道： > > > > > > By “skeptical” I mean that I think we can come up with a mechanism > > > > > > to copy hints when applying planner rules, but even when we have > > > > > > implemented that mechanism there will be many cases where people > > > > > > want a hint and that hint is not copied to the RelNode where it is > > > > > > needed, and many other cases where we spend the effort/memory of > > > > > > copying the hint to a RelNode and the hint is not used. > > > > > > > > > > > > By “reversible” I mean if we come up with an API that does not > > > > > > work, how do we change or remove that API without people > > > > > > complaining? > > > > > > > > > > > > Julian > > > > > > > > > > > > > > > > > > > On Oct 15, 2019, at 7:11 PM, Danny Chan <[email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > Thanks Julian > > > > > > > > > > > > > > > I am skeptical that RelWithHint will work for large queries. > > > > > > > > > > > > > > For “skeptical” do you mean how to transfer the hints during rule > > > > > > > planning ? I’m also not that confident yet. > > > > > > > > > > > > > > > How do we introduce it in a reversible way > > > > > > > Do you mean transform the RelWithHint back into the SqlHint ? I > > > > > > > didn’t implement it in current patch, but I think we have the > > > > > > > ability to do that because we have a inheritPath for each > > > > > > > RelWithHint, we can collect all the hints together and merge them > > > > > > > into the SqlHints, then propagate these SqlHints to the SqlNodes. > > > > > > > > > > > > > > > What are the other options? > > > > > > > Do you mean the way to transfer hints during planning ? I have no > > > > > > > other options yet. > > > > > > > > > > > > > > Best, > > > > > > > Danny Chan > > > > > > > 在 2019年10月16日 +0800 AM8:03，[email protected]，写道： > > > > > > > > > > > > > > > > I am skeptical that RelWithHint will work for large queries. > > > > > > > > > > > > > > > > > >
