Dandandan commented on issue #7095: URL: https://github.com/apache/datafusion/issues/7095#issuecomment-2539778985
> > I don't think there is a 'trivial' way to outperform HashBrown. I suspect that any performance improvements they achieved are due to factors other than having a better hashing mechanism than HashBrown > > Vectorizing the hash calculation is the big one, and DataFusion already do that > > There are more "exotic" hash table strategies that supposedly are able to vectorize the lookup / collision comparisons too (e.g. look up multiple keys in one instruction). The umbra papers talk about this, but that codebase is not open source and I am not sure how much it really improves performance in practice Yeah - my hope is we can find a way to integrate this in some way to the hashbrown APIs (so we can still benefit from the well optimized implementation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
