waitinfuture commented on PR #2524: URL: https://github.com/apache/celeborn/pull/2524#issuecomment-2122817377
> Why do we want to limit the size at half of shuffleChunkSize and not use full shuffleChunkSize. The reason is to avoid that the next batch exceeds half the shuffleChunkSize so the whole length exceeds shuffleChunkSize. But I think this is trivial, it's ok to use full shuffleChunkSize. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
