guixiaowen commented on code in PR #1438:
URL: https://github.com/apache/auron/pull/1438#discussion_r2431556420
##########
spark-extension/src/main/scala/org/apache/spark/sql/auron/AuronConverters.scala:
##########
@@ -135,21 +135,20 @@ object AuronConverters extends Logging {
getBooleanConf("spark.auron.enable.scan.orc", defaultValue = true)
def enableBroadcastExchange: Boolean =
getBooleanConf("spark.auron.enable.broadcastExchange", defaultValue = true)
- def enableShuffleExechange: Boolean =
- getBooleanConf("spark.auron.enable.shuffleExchange", defaultValue = true)
+ def enableShuffleExchange: Boolean = {
+ val shuffleManager = SQLConf.get.getConfString(config.SHUFFLE_MANAGER.key)
+ getBooleanConf(
+ "spark.auron.enable.shuffleExchange",
+ defaultValue = true) && !shuffleManager.isEmpty &&
(shuffleManager.contains(
+ "AuronShuffleManager") || shuffleManager.contains(
Review Comment:
> i suggest not to change this logic, just leaving it as what user has set.
for incorrect `spark.shuffle.manager` configuration, we can fast fail in the
initialization code.
@richox Thank you very much for reviewing the code. I completely agree with
your comments. However, if we don’t modify this part, the check for *convert to
native shuffle manager* will be incorrect, and the parameter won’t take effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
