zhztheplayer commented on code in PR #7857:
URL: https://github.com/apache/incubator-gluten/pull/7857#discussion_r1833700481
##########
gluten-core/src/main/scala/org/apache/gluten/GlutenPlugin.scala:
##########
@@ -134,7 +134,11 @@ private[gluten] class GlutenDriverPlugin extends
DriverPlugin with Logging {
conf.set(SPARK_SESSION_EXTENSIONS.key, extensions)
// adaptive custom cost evaluator class
- if (GlutenConfig.getConf.enableGluten &&
GlutenConfig.getConf.enableGlutenCostEvaluator) {
+ val glutenEnabled = conf.getBoolean(GlutenConfig.GLUTEN_ENABLED_KEY,
GLUTEN_ENABLED_BY_DEFAULT)
+ lazy val enableGlutenCostEvaluator = conf.getBoolean(
+ GlutenConfig.GLUTEN_COST_EVALUATOR_ENABLED,
+ GLUTEN_COST_EVALUATOR_DEFAULT_VALUE)
+ if (glutenEnabled && enableGlutenCostEvaluator) {
val costEvaluator =
"org.apache.spark.sql.execution.adaptive.GlutenCostEvaluator"
conf.set(SQLConf.ADAPTIVE_CUSTOM_COST_EVALUATOR_CLASS.key, costEvaluator)
}
Review Comment:
It turns out that this part of code is problematical.
The entrypoint `GlutenPlugin` is called only once in the same Spark session.
So say user could set `spark.gluten.enabled=false` then set it to true using
`SET` prompt after a while, `GlutenCostEvaluator` may not load in the case.
Will there be any issue if we just make `GlutenCostEvaluator` always load?
Which means removing the conditions, namely `if (glutenEnabled &&
enableGlutenCostEvaluator)` here.
cc @zml1206
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
