advancedxy commented on issue #838: URL: https://github.com/apache/incubator-uniffle/issues/838#issuecomment-1519069871
> https://github.com/apache/kyuubi/blob/master/extensions/spark/kyuubi-extension-spark-3-3/src/main/scala/org/apache/kyuubi/sql/InsertShuffleNodeBeforeJoin.scala#L38 Thanks for pointing to this. It's interesting and clever way to add new rules and syntax using Spark extension mechanism.. > Oh, it has been proposed but still reviewing. https://github.com/apache/spark/pull/40312 In my opinion, this is just another optimize SkewJoin rule, which should be integrated into your internal Spark code if it's not merged into Spark upstream yet. Optimizing skew joins is out of the scope of Uniffle project. However, if these optimization rules is well maintained elsewhere out of Uniffle project(maybe kyubi?), I believe the community is more than happy to add it to the tuning document in details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
