Rich-T-kid opened a new pull request, #12127: URL: https://github.com/apache/arrow-rs/pull/12127
# Which issue does this PR close? - N/A # Rationale for this change Scalar comparisons against dictionary arrays evaluate the predicate once per dictionary value, then expand the Boolean results through the dictionary keys. The expansion currently uses the checked `take` path even when the dictionary has no null keys and all keys were already validated when the array was constructed. This shows up in DataFusion when Parquet string columns are preserved as dictionaries and filtered by scalar comparisons or `LIKE` predicates. # What changes are included in this PR? Use the unchecked take path to expand Boolean results for dictionaries without null keys. Dictionaries with null keys retain the checked path because a null slot can contain an otherwise invalid physical key. The same optimization is applied to scalar comparison and scalar string predicate evaluation. # Are these changes tested? The existing dictionary comparison and string predicate tests cover the optimized path. This PR adds coverage for dictionaries with both null keys and null values, which continue to use the checked fallback. Existing low-cardinality comparison benchmarks improved as follows: | Case | Before | After | Change | |---|---:|---:|---:| | scalar equality | 23.90 us | 17.89 us | -25.2% | | scalar greater-than-or-equal | 23.99 us | 17.95 us | -25.2% | | scalar LIKE | 24.18 us | 17.95 us | -25.8% | | scalar ILIKE | 24.12 us | 18.13 us | -24.8% | The dictionary-to-dictionary equality control was unchanged within noise (+0.4%). # Are there any significant user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
