bhaskargurram-ai commented on issue #5550: URL: https://github.com/apache/arrow-rs/issues/5550#issuecomment-6088057045
I'd like to work on this if nobody else is. Plan, following the options alamb said sound reasonable: 1. A helper that converts an Arrow value to the Parquet physical value the `ArrowWriter` actually hashed (the same coercion as the writer: Int8/Int16/UInt8/UInt16 → INT32, and the other coerced types mr-brobot listed). 2. An Arrow-aware bloom filter check that applies that conversion before `Sbbf::check`. With tests for each affected type that fail today, using the round trip from the original report. Once it lands, DataFusion could re-enable the bloom filter pruning it disabled for these types (apache/datafusion#9779). Does that approach still work for you, @alamb @Jefffrey? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
