JkSelf commented on code in PR #8931:
URL: https://github.com/apache/incubator-gluten/pull/8931#discussion_r2904397977
##########
gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala:
##########
@@ -998,6 +1001,13 @@ object GlutenConfig extends ConfigRegistry {
.intConf
.createWithDefault(12)
+ val COLUMNAR_PHYSICAL_JOIN_OPTIMIZATION_OUTPUT_SIZE =
+ buildConf("spark.gluten.sql.columnar.physicalJoinOptimizationOutputSize")
Review Comment:
We introduced the FallbackMultiCodegens rule specifically for Q72 to fall
back stages containing multiple continuous joins to Vanilla Spark. However,
enabling BHJ optimization caused Q64 to convert Shuffle Hash Joins into BHJs.
This resulted in multiple joins appearing in a single stage, triggering the
fallback rule for Q64. We have added a parameter to FallbackMultiCodegens to
ensure it is scoped only to Q72, preventing unnecessary fallbacks in Q64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
