mridulm commented on code in PR #40307:
URL: https://github.com/apache/spark/pull/40307#discussion_r1127076610
##########
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala:
##########
@@ -203,7 +205,8 @@ private[spark] class ExecutorAllocationManager(
throw new SparkException(
s"s${DYN_ALLOCATION_SUSTAINED_SCHEDULER_BACKLOG_TIMEOUT.key} must be >
0!")
}
- if (!conf.get(config.SHUFFLE_SERVICE_ENABLED)) {
+ if (!conf.get(config.SHUFFLE_SERVICE_ENABLED) &&
+ !shuffleDriverComponents.supportsReliableStorage()) {
Review Comment:
`SHUFFLE_SERVICE_ENABLED` will continue to exist for backward compatibility,
but imo would be specific for the default implementation for SPARK-25299.
That should be better handled as part of a follow up to SPARK-25299 imo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
