[GitHub] [spark] mridulm edited a comment on pull request #30876: [SPARK-33870][CORE] Enable spark.storage.replication.proactive by default

mridulm edited a comment on pull request #30876: URL: https://github.com/apache/spark/pull/30876#issuecomment-751979314 Before answering specific queries below, I want to set the context. a) Enabling proactive replication could result in reduced recomputation cost when executors fail.

[GitHub] [spark] mridulm edited a comment on pull request #30876: [SPARK-33870][CORE] Enable spark.storage.replication.proactive by default

mridulm edited a comment on pull request #30876: URL: https://github.com/apache/spark/pull/30876#issuecomment-751979314 Before answering specific queries below, I want to set the context. a) Enabling proactive replication could result in reduced recomputation cost when executors fail.

[GitHub] [spark] mridulm edited a comment on pull request #30876: [SPARK-33870][CORE] Enable spark.storage.replication.proactive by default

mridulm edited a comment on pull request #30876: URL: https://github.com/apache/spark/pull/30876#issuecomment-751934621 Specifically for this [usecase](https://github.com/apache/spark/pull/30876#issuecomment-750471287), we dont need to make it a spark default right ? If I understood

[GitHub] [spark] mridulm edited a comment on pull request #30876: [SPARK-33870][CORE] Enable spark.storage.replication.proactive by default

mridulm edited a comment on pull request #30876: URL: https://github.com/apache/spark/pull/30876#issuecomment-751186157 @dongjoon-hyun proactive replication only applies to persisted RDD blocks, not shuffle blocks - not sure if I am missing something here. Even for persisted RDD