adriangb opened a new pull request, #11485: URL: https://github.com/apache/arrow-rs/pull/11485
# Which issue does this PR close? - Part of https://github.com/apache/arrow-rs/issues/11234. # Rationale for this change `ParquetPushDecoder` has no test that compares it with the sync reader across many scan configurations. This PR adds one. It has value on `main`, and https://github.com/apache/arrow-rs/pull/11238 will use the same harness to check its new mode. This PR is split out of that PR to make it smaller. # What changes are included in this PR? Tests only, in one new file. ```text Scan { batch_size, projection, row_groups, selection, limit, offset, predicates, policy, .. } ├── push_decoder() ──▶ drive_file ──▶ rows ─┐ └── sync_batches() ─────────────────▶ rows ─┴─ assert_same_rows ``` - Two test files with list and struct columns: one with the same page boundaries in all columns, one with different page boundaries per column. - Fixed cases for batch sizes, projections, row selections, offset and limit, and chains of predicates. - Two fuzz tests (`fuzz_equivalence`, `fuzz_equivalence_heterogeneous_pages`) that combine all options at random, 500 seeds in total. The tests compare rows, not batch boundaries: the sync reader lets a batch span two row groups, and the push decoder does not. # Are these changes tested? The 10 new tests pass on `main` in about 5 seconds (debug build). To check that they can fail, I dropped one row per batch in `drive_file`: 8 of 10 tests failed. The other 2 decode no rows. # Are there any user-facing changes? No. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
