JingsongLi commented on PR #10128: URL: https://github.com/apache/paimon/pull/10128#issuecomment-5805856143
This fixes a data-loss-on-read bug: a fully filtered Avro batch must not be reported as EOF while later batches may contain matches. The loop returns `None` only after the fastavro generator is exhausted, and it avoids recursion across long runs of rejected batches. The regression now skips two complete 1,024-row batches before checking the final 500 rows, as requested in the prior review. Local verification on the PR patch: all 3 Avro append-only reader tests passed, including the new end-to-end filtered table read. The current PR checks are still red in Python 3.10, 3.11, 3.12, and 3.13 (plus the aggregate result). Please determine the cause and get the required matrix green before merge; the focused Avro result alone does not establish production readiness. The fix has clear end-to-end value, and I found no blocking issue in the changed reader loop. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
