otterc commented on a change in pull request #33034:
URL: https://github.com/apache/spark/pull/33034#discussion_r661852708
##########
File path:
common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
##########
@@ -222,7 +223,7 @@ public void sendMergedBlockMetaReq(
handler.addRpcRequest(requestId, callback);
RpcChannelListener listener = new RpcChannelListener(requestId, callback);
channel.writeAndFlush(
- new MergedBlockMetaRequest(requestId, appId, shuffleId,
reduceId)).addListener(listener);
+ new MergedBlockMetaRequest(requestId, appId, shuffleId,
shuffleSequenceId, reduceId)).addListener(listener);
Review comment:
It does seem that not having `shuffleSequenceId` on fetch-side is
causing confusion and requires a lot of explanation. So, I am fine with adding
this new field in just the merged fetch protocols.
Also discussed with @Victsm offline about his comments and we have a way to
address some of the open issues with the implementation. @Victsm or @venkata91
will elaborate about the approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
