xumingming commented on PR #3672:
URL: https://github.com/apache/celeborn/pull/3672#issuecomment-4645238195
```
iterator.next() → serialize to buffer → dataPusher async push →
shuffle service worker
↑
PR checks here records could be
lost anywhere after
```
@RexXiong **Once the data enters the "serialize to buffer", the correctness
is handled by CRC check**, if any data is lost, it will be discovered by the
CRC check mechanism, right? Fully-consumed validation is a compliment to the
existing CRC check, to make sure all data is consumed before enter the "CRC
covered zone".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
