I considered manual rules calling too, for now we use abstract converters + ExpandConversionRule for exchanges producing.
You may create such converters manually (checking appropriate subset) this case you may reduce created converters count, also, a converter is a quite special node, that does almost nothing (without corresponding rule) it may be used just as a rule trigger. Regards, Igor ср, 30 окт. 2019 г., 17:31 Vladimir Ozerov <[email protected]>: > One funny hack which helped me is manual registration of a fake RelNode > with desired traits through VolcanoPlanner.register() method. But again, > this leads to trashing. What could really help is a call to > VolcanoPlanner.fireRules() with desired rel. But this doesn't work out of > the box since some internals of the rule queue needs to be adjusted. > > What does the community think about adding a method which will re-add rules > applicable to the specific RelNode to the rule queue? > > ср, 30 окт. 2019 г. в 17:00, Vladimir Ozerov <[email protected]>: > > > Hi Igor, > > > > Yes, I came to the same conclusion, thank you. This is how it basically > > happens when converters are disabled: > > 1) We start with initial tree: [LogicalProject] <- [LogicalScan] > > 2) Then we convert LogicalScan to PhysicalScan, so it is added to the > > set: [LogicalProject] <- [LogicalScan, PhysicalScan] > > 3) Finally, when it is time to fire a rule for PhysicalScan, we try to > get > > parents of that scan set with traits of the PhysicalScan. Since there are > > no such parents (we skipped it intentionally), the rule is not queued. > > > > But when converters are enabled, a converter rel is created: > [LogicalProject] > > <- [LogicalScan, PhysicalScan, ConverterFromPhysicalToLogical]. No rules > > are fired for PhysicalScan again, but they are fired for converter since > > it has the necessary LOGICAL trait. > > > > It makes sense, that converters essentially allow forcing rule invocation > > on parents, even if the child was created with different traits. But it > > seems that this mechanism may be too heavy for complex queries because it > > literally creates hundreds of new converter rels and triggers rules over > > and over again. > > > > We need some fine-grained alternative. Basically, what would be enough > for > > me is to let the planner know somehow: "I created that rel, and I want > you > > to execute parent rules not only using its trait but also on this and > those > > traits." > > Is there any API in Calcite which allows doing this without creating a > new > > rel node? > > > > Regards, > > Vladimir. > > > > > > ср, 30 окт. 2019 г. в 09:25, Seliverstov Igor <[email protected]>: > > > >> Vladimir, > >> > >> Probably it'll help you: > >> > >> Seems the cause of issue in RelSubset.getParentRels() The check used > when > >> the planner schedules newly matched rules after successful > transformation > >> (see VolcanoRuleCall.matchRecurse), it prevents the parent rule be > applied > >> once again (here your logical project with an input having ANY > >> distribution > >> doesn't satisfy a transformed input traits). > >> > >> In our case we use another workaround, so there are also much more > >> transformations than we wanted, so the desired rule is triggered. > >> > >> > >> вт, 29 окт. 2019 г., 14:46 Vladimir Ozerov <[email protected]>: > >> > >> > Hi Vladimir, > >> > > >> > I am sorry. Pushed, it works now. > >> > > >> > вт, 29 окт. 2019 г. в 14:41, Vladimir Sitnikov < > >> > [email protected] > >> > >: > >> > > >> > > > mvn clean test > >> > > > >> > > [ERROR] The goal you specified requires a project to execute but > >> there is > >> > > no POM in this directory > >> > > > >> > > Vladimir, please push missing files > >> > > > >> > > Vladimir > >> > > > >> > > >> > > >
