zaynt4606 commented on PR #3873: URL: https://github.com/apache/celeborn/pull/3873#issuecomment-6282765823
Thanks @ForVic! One non-blocking suggestion for the tests: could we extend the coverage beyond the batch row counts to include: - The row limit being reached before the byte limit while byte-based batching is enabled. - Interleaved rows for multiple partitions, verifying that each partition tracks and resets its byte count independently. - Deserializing the pushed batches and checking the actual row contents, including with codegen or dictionary encoding enabled. I ran the existing 14 writer tests on Spark 3.5 and seven additional local boundary/round-trip checks, including these scenarios; all passed. This is a regression-test coverage suggestion and does not block merging; it can be a follow-up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
