cloud-fan commented on code in PR #58440:
URL: https://github.com/apache/spark/pull/58440#discussion_r3958039774
##########
common/network-common/src/main/java/org/apache/spark/network/client/TransportResponseHandler.java:
##########
@@ -284,6 +290,50 @@ public void handle(ResponseMessage message) throws
Exception {
}
}
+ /**
+ * Asserts the FIFO ordering invariant for stream responses: the streamId
carried by this
+ * response/failure must equal the streamId of the callback at the head of
the
+ * {@link #streamCallbacks} queue. Responses to {@code StreamRequest}s are
matched to callbacks by
+ * {@code poll()} order (see SPARK-11265), which is correct only if the
server answers requests in
+ * the order the client sent them; this verifies that invariant before the
polled callback is
+ * used. The check cannot false-fire: the client registers each callback
under the exact streamId
+ * it requested, so a matching response always finds its callback at the
head of the queue.
+ *
+ * <p>If the invariant does not hold the callback queue is desynced --
delivering this response
+ * would route the wrong block's bytes to the callback. It fails the polled
callback under its own
+ * streamId (so its caller does not hang waiting for a response it will
never correctly receive;
+ * {@code poll()} has already removed it from the queue) and throws {@link
IllegalStateException},
+ * which propagates to Netty's {@code exceptionCaught} so the connection is
torn down and its
Review Comment:
**Non-blocking (P2):** Only the polled stream callback receives an
`IOException`. The subsequent uncaused `IllegalStateException` is forwarded to
the remaining callbacks, but `RetryingBlockTransferor` retries only an
`IOException` (or one as the cause), and generic RPCs have no automatic
re-fetch path. Please narrow this Javadoc and the corresponding test prose to
say that closing the connection fails the remaining outstanding requests; the
existing callers can then apply their normal retry behavior where supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
