Ngone51 commented on a change in pull request #28911:
URL: https://github.com/apache/spark/pull/28911#discussion_r480095206
##########
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/BlockStoreClient.java
##########
@@ -61,4 +78,56 @@ public MetricSet shuffleMetrics() {
// Return an empty MetricSet by default.
return () -> Collections.emptyMap();
}
+
+ /**
+ * Request the local disk directories for executors which are located at the
same host with
+ * the current BlockStoreClient(it can be ExternalBlockStoreClient or
NettyBlockTransferService).
+ *
+ * @param host the host of BlockManager or ExternalShuffleService. It's the
same with current
+ * BlockStoreClient.
+ * @param port the port of BlockManager or ExternalShuffleService.
+ * @param execIds a collection of executor Ids, which specifies the target
executors that we
+ * want to get their local directories. There could be
multiple executor Ids if
+ * BlockStoreClient is implemented by
ExternalBlockStoreClient since the request
+ * handler, ExternalShuffleService, can serve multiple
executors on the same node.
+ * Or, only one executor Id if BlockStoreClient is
implemented by
+ * NettyBlockTransferService.
+ * @param hostLocalDirsCompletable a CompletableFuture which contains a map
from executor Id
+ * to its local directories if the request
handler replies
+ * successfully. Otherwise, it contains a
specific error.
+ */
+ public void getHostLocalDirs(
+ String host,
+ int port,
+ String[] execIds,
+ CompletableFuture<Map<String, String[]>> hostLocalDirsCompletable) {
+ assert appId != null : "Called before init()";
Review comment:
Sounds reasonable!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
