otterc commented on a change in pull request #33613:
URL: https://github.com/apache/spark/pull/33613#discussion_r681927734
##########
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ErrorHandler.java
##########
@@ -105,16 +105,16 @@ public boolean shouldRetryError(Throwable t) {
return false;
}
- String errorStackTrace = Throwables.getStackTraceAsString(t);
// If the block is too late or stale block push, there is no need to
retry it
- return
!errorStackTrace.contains(TOO_LATE_OR_STALE_BLOCK_PUSH_MESSAGE_SUFFIX);
+ String msg = t.getMessage();
+ return !(msg != null &&
msg.contains(TOO_LATE_OR_STALE_BLOCK_PUSH_MESSAGE_SUFFIX));
Review comment:
Had some discussions related to making this a proper RPC response with
@Victsm in the past and I think as well that it is a better approach. One
thing to note though is that if we do this, we will have to make changes on the
client to invoke `listener.onFailure()` callback when the RPCResponse
represents these exception types because on the client we leverage
`RetryingBlockTransfer`/ `BlockTransferListener` to retry block push which
relies on the `onFailure` callbacks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
