etseidl opened a new issue, #11974:
URL: https://github.com/apache/arrow-rs/issues/11974
### Is your feature request related to a problem or challenge?
This is a follow up to #9067/#11953. There are currently three locations
that Bloom filters are parsed, with a lot of duplicated code.
-
https://github.com/apache/arrow-rs/blob/14c2fef335c4396552d60d69502950f2d0424c70/parquet/src/arrow/arrow_reader/mod.rs#L1348
-
https://github.com/apache/arrow-rs/blob/14c2fef335c4396552d60d69502950f2d0424c70/parquet/src/arrow/async_reader/mod.rs#L588
-
https://github.com/apache/arrow-rs/blob/14c2fef335c4396552d60d69502950f2d0424c70/parquet/src/bloom_filter/mod.rs#L474
It would be nice to consolidate this some to reduce repetitive code. There
are also some deficiencies in input validation that should be addressed (unsafe
conversions to `usize`, checking for length mismatches, empty filters can be
created in certain circumstances, etc). Cosolidation will allow addressing
these issues once.
### Describe the solution you'd like
Consolidate the above.
### Describe alternatives you've considered
Leave the duplicate code in place.
### Additional context
This is part of a broader campaign to make Bloom filters more accessible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
