mohitgurav20 commented on PR #25811: URL: https://github.com/apache/datafusion/pull/25811#issuecomment-6504977995
@alamb That makes total sense — having the baseline test merged to `main` first will make the I/O reduction completely transparent in the test diff. I've already built the tracking reader test harness (using a wrapper around `AsyncFileReader` that records all requested byte ranges, similar to `parquet/tests/arrow_reader/io/async_reader.rs`). I will open a separate PR against `main` containing just the test harness capturing the baseline behavior (where both column filters are fetched upfront). Once that lands, I'll rebase this PR so the diff directly demonstrates the skipped fetch. Also, on this PR I've already: 1. Eliminated the duplication with `prune_by_bloom_filters` by encapsulating guarantee preparation and per-column evaluation in `RowGroupAccessPlanFilter`. 2. Removed `access_plan_mut()`. 3. Dropped the Criterion benchmark and its entry in `core/Cargo.toml`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
