alamb opened a new pull request, #8754: URL: https://github.com/apache/arrow-rs/pull/8754
# Which issue does this PR close? - Related to https://github.com/apache/arrow-rs/issues/8677 - part of https://github.com/apache/arrow-rs/pull/8159 # Rationale for this change I am reworking how the parquet decoder's state machine works in https://github.com/apache/arrow-rs/pull/8159 One of the unit tests, `test_cache_projection_excludes_nested_columns` uses non-public APIs that I am changing Rather than rewrite them into other non public APIs I think it would be better if this test is in terms of public APIs # What changes are included in this PR? 1. refactor `test_cache_projection_excludes_nested_columns` to use high level APIs # Are these changes tested? They are run in CI I also verified this test covers the intended functionality by commenting it out: ```diff --- a/parquet/src/arrow/async_reader/mod.rs +++ b/parquet/src/arrow/async_reader/mod.rs @@ -724,7 +724,9 @@ where cache_projection.union(predicate.projection()); } cache_projection.intersect(projection); - self.exclude_nested_columns_from_cache(&cache_projection) + // TEMP don't exclude nested columns + //self.exclude_nested_columns_from_cache(&cache_projection) + Some(cache_projection) } /// Exclude leaves belonging to roots that span multiple parquet leaves (i.e. nested columns) ``` And then running the test: ```shell cargo test --all-features --test arrow_reader ``` And the test fails (as expected) ``` ---- predicate_cache::test_cache_projection_excludes_nested_columns stdout ---- thread 'predicate_cache::test_cache_projection_excludes_nested_columns' panicked at parquet/tests/arrow_reader/predicate_cache.rs:244:9: assertion `left == right` failed: Expected 0 records read from cache, but got 100 left: 100 right: 0 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace failures: predicate_cache::test_cache_projection_excludes_nested_columns test result: FAILED. 88 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.20s ``` # Are there any user-facing changes? No, this is only test changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
