mhk197 opened a new issue, #12055: URL: https://github.com/apache/arrow-rs/issues/12055
### Is your feature request related to a problem or challenge? While optimizing the export of [Vortex](https://github.com/vortex-data/vortex) primitive arrays to Arrow dictionaries, I noticed that a significant amount of time was spent hashing values. Arrow currently uses `std::collections::HashMap` in `PrimitiveDictionaryBuilder` with the standard `SipHash`, which is significantly slower than alternatives such as `ahash` and `fxhash`. Moreover, `PrimitiveDictionaryBuilder::get_or_insert_key` seems to hash new values **twice**, unnecessarily. This problem was addressed in #1861 for `StringDictionaryBuilder`. ### Describe the solution you'd like Switch to `ahash` for `PrimitiveDictionaryBuilder`. My microbenchmarks show that this can improve performance by > 2x. This is a simple change, and does not add any new dependencies. ### Describe alternatives you've considered None. ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
