venkata91 commented on a change in pull request #33034:
URL: https://github.com/apache/spark/pull/33034#discussion_r658131269
##########
File path:
common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
##########
@@ -222,7 +223,7 @@ public void sendMergedBlockMetaReq(
handler.addRpcRequest(requestId, callback);
RpcChannelListener listener = new RpcChannelListener(requestId, callback);
channel.writeAndFlush(
- new MergedBlockMetaRequest(requestId, appId, shuffleId,
reduceId)).addListener(listener);
+ new MergedBlockMetaRequest(requestId, appId, shuffleId,
shuffleSequenceId, reduceId)).addListener(listener);
Review comment:
> If the fetcher doesn't include a mapId in the request, how will the
server know which shuffle block to serve?
Agree, it is not exactly same.
As mentioned in other comment, in terms of correctness both the approaches
should work fine. Here it is just a choice of implementation. We can agree to
disagree. I still prefer passing `shuffleSequenceId` in the fetch protocol as
it feels simpler and straightforward to me. If there are other strong reasons
for not doing so other than the ones mentioned earlier, I would definitely
consider them and change the implementation accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
