mridulm commented on code in PR #37922:
URL: https://github.com/apache/spark/pull/37922#discussion_r1069857650
##########
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockStoreClient.java:
##########
@@ -256,6 +256,23 @@ public void onFailure(Throwable e) {
}
}
+ @Override
+ public boolean removeShuffleMerge(String host, int port, int shuffleId, int
shuffleMergeId) {
+ checkInit();
+ try {
+ TransportClient client = clientFactory.createClient(host, port);
+ client.send(
+ new RemoveShuffleMerge(appId, comparableAppAttemptId, shuffleId,
shuffleMergeId)
+ .toByteBuffer());
+ // TODO(SPARK-42025): Add some error logs for RemoveShuffleMerge RPC.
+ } catch (Exception e) {
+ logger.error("Exception while sending RemoveShuffleMerge request to
{}:{}",
Review Comment:
We can revisit that in
[SPARK-42025](https://issues.apache.org/jira/browse/SPARK-42025).
Here, let us simply make the `logger.error` in the `catch` to `logger.debug`
for now - since it is this is a best case effort to remove files from an
external service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
