yabola commented on PR #38560: URL: https://github.com/apache/spark/pull/38560#issuecomment-1310162375
> I am wondering whether the driver needs to pass the merged reduceId to the external shuffle service (but now the driver cannot fully record merged info), or the shuffle service records the merged reduceIds, and driver only need to pass the shuffleId and other information I decided to change the driver to not send reduceId, because only the shuffle service finally understands which shuffle data is stored, no matter how the driver processes or processes the message -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
