hhhizzz opened a new issue, #10747: URL: https://github.com/apache/arrow-rs/issues/10747
### Is your feature request related to a problem or challenge? #10733 (fixed by #10735) needed four things to be true at the same time: predicate pushdown, page pruning, a Mask-backed `RowSelection`, and the predicate cache. Each of those is covered by tests today. No test covered any two of them together, which is why it shipped in 59.2.0 and survived about two weeks before anyone hit it. Concretely, the two relevant test areas are disjoint: | test area | `RowSelectionPolicy` | `with_row_selection` | `PageIndexPolicy` | predicate cache | |---|---|---|---|---| | `parquet/tests/arrow_reader/predicate_cache.rs` | 0 | 0 | 0 | yes | | mask/sparse tests (`read_plan.rs`, `push_decoder/reader_builder/mod.rs`, `row_filter/async.rs`) | yes | yes | yes | **0** | This is a gap in the cross product rather than a gap in depth: #10288 added a fair number of unit tests for the mask/sparse-page interaction, and they all pass on the buggy code. ### Describe the solution you'd like `test_fuzz_async_reader_selection` in `parquet/src/arrow/async_reader/mod.rs` is the natural home, but it cannot reach this class of bug as written. Today it randomizes only the selection pattern and one projected column, and it has **no `RowFilter` at all** — so predicate pushdown, the predicate cache, and filter-driven sparse fetch are all unreachable from it. Axes worth adding: - [ ] a `RowFilter` (this is the entry condition for the whole path) - [ ] predicate cache size: disabled / small enough to evict / large - [ ] `RowSelectionPolicy`: `Selectors` / `Mask` / `Auto` - [ ] `batch_size` (batch alignment is central to the cached reader) - [ ] projections that overlap the predicate columns, so a column is both cached and part of the output - [ ] page size / row group size, so page boundaries and batch boundaries interleave differently Two other changes would raise the yield: 1. **Stronger oracle.** The current assertion is `assert_eq!(actual_rows, expected_rows)` — row counts only. A read that returns the wrong *values* passes. A differential check against a reference configuration (same selection under `Selectors`, or with the cache disabled) would catch that and needs no golden data. 2. **Better selection generator.** The current generator alternates select/skip with `row_count` drawn from `1..100`, so it essentially never produces the shape that broke here: a single selected row followed by a long skip (`select(1), skip(39), select(1)`). Worth explicitly generating sparse single-row islands, runs that straddle a page boundary, and selections whose gaps are larger than one page. ### Describe alternatives you've considered End-to-end coverage against a real engine (filed separately as #10746). That catches a different failure mode — a configuration nobody thought to test — and is complementary rather than an alternative. ### Additional context The same error class was fixed once before in #9301 (January 2026), which suggests this interaction is worth systematic coverage rather than one more targeted test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
