zml1206 commented on PR #5750: URL: https://github.com/apache/incubator-gluten/pull/5750#issuecomment-2149023181
> > > > > If the custom strategy can be removed by moving the code to ColumnarOverrides (without more workarounds), Personally I will be inclined to do that since it: > > > > > > > > > > 1. Simplifies code > > > > > 2. Creates "more vanilla" plan when the join operators are falling back > > > > > > > > > > > > Do it in this PR or next? @zhztheplayer > > > > > > > > > Thank you for willing to take this. Of course it can be done in a separate PR. > > > > > > I looked it up and it seems like can't move custom strategy to ColumnarOverrides. Custom strategy is used to transform a LogicalPlan into a SparkPlan. Columnar rule implements Spark plan into Gluten plan. Custom strategy runs before Columnar rule. @zhztheplayer > > I think SHJ and SMJ in Spark share the same `requiredChildDistribution` code. Which means we can just convert one to another in columnar rule then adjust its children to add/remove sorts on top of them. (Ideally, you could double check the relevant code) > > In the simplest case the topic could be very similar to the one about hash/sort agg, see [this code](https://github.com/apache/incubator-gluten/blob/00dda1b67ec81dc841643e1c02323f32df5525d9/gluten-core/src/main/scala/org/apache/gluten/extension/columnar/OffloadSingleNode.scala#L289-L297) This is a good way for me, thank you, I'll do it later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
