otterc commented on code in PR #37638:
URL: https://github.com/apache/spark/pull/37638#discussion_r1003631087
##########
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##########
@@ -593,6 +607,9 @@ public void onData(String streamId, ByteBuffer buf) {
@Override
public void onComplete(String streamId) {
+ if (isTooLate) {
+ pushMergeMetrics.tooLateResponses.mark();
+ }
Review Comment:
This is explained in the comments above that why we don't want throw
exception and close the channel but rather respond with stream callback and
only when the complete block is received, respond with tooLate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
