[GitHub] [spark] mridulm commented on a diff in pull request #40307: [DRAFT][SPARK-42689][CORE][SHUFFLE]: Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

mridulm commented on code in PR #40307: URL: https://github.com/apache/spark/pull/40307#discussion_r1129142718 ## core/src/main/scala/org/apache/spark/SparkContext.scala: ## @@ -596,6 +591,13 @@ class SparkContext(config: SparkConf) extends Logging { _conf.set(APP_ATTEMPT

[GitHub] [spark] mridulm commented on a diff in pull request #40307: [DRAFT][SPARK-42689][CORE][SHUFFLE]: Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

mridulm commented on code in PR #40307: URL: https://github.com/apache/spark/pull/40307#discussion_r1128807716 ## core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala: ## @@ -456,7 +456,9 @@ class SparkListenerSuite extends SparkFunSuite with LocalSparkContex

[GitHub] [spark] mridulm commented on a diff in pull request #40307: [DRAFT][SPARK-42689][CORE][SHUFFLE]: Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

mridulm commented on code in PR #40307: URL: https://github.com/apache/spark/pull/40307#discussion_r1127076610 ## core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala: ## @@ -203,7 +205,8 @@ private[spark] class ExecutorAllocationManager( throw new SparkEx