otterc commented on code in PR #39725:
URL: https://github.com/apache/spark/pull/39725#discussion_r1088159016
##########
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##########
@@ -1554,6 +1572,10 @@ public void onFailure(String streamId, Throwable
throwable) throws IOException {
}
}
isWriting = false;
+ // The failed bytes (and the subsequent bytes) landed at ESS, hence we
considered them
+ // also as the bytes being ignored. Including them as ignoredBlockBytes
would also help
+ // measure any server side improvement.
+ updateIgnoredBlockBytes();
Review Comment:
Nit: Can we move this block to line 1559.
Also reword: `The block was received by ESS but didn't get merged so it is
considered as "ignored"`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
