mridulm commented on code in PR #43280:
URL: https://github.com/apache/spark/pull/43280#discussion_r1350810420
##########
core/src/main/java/org/apache/spark/shuffle/api/ShuffleDriverComponents.java:
##########
@@ -66,8 +66,19 @@ default void removeShuffle(int shuffleId, boolean blocking)
{}
* Does this shuffle component support reliable storage - external to the
lifecycle of the
* executor host ? For example, writing shuffle data to a distributed
filesystem or
* persisting it in a remote shuffle service.
+ *
+ * Note: This method is for compatibility with older implementations,
+ * the newer implementation should not use this method.
*/
default boolean supportsReliableStorage() {
return false;
}
+
+ /**
+ * Does this executor support reliable storage for all shuffle data.
+ * @param execId The executor id, null for validation use only.
Review Comment:
executor id should be valid - and spark will pass a valid id when using this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
