mapleFU commented on issue #34536: URL: https://github.com/apache/arrow/issues/34536#issuecomment-1465441935
As arrow-rs says, it use DeltaBinaryPacked as default encoding as default encoder for integers. And it found out that, for uniform distributed numbers, wider bitwidth should be much better. I'll benchmark delta binary packed for different size and different input distribution in x86_64 and neon machine, and find out if we should make it larger. To be honest, the best way should be adaptive encoding, but I'm not so familiar with encoding algorithms By the way, tustvoid mentions that, http://arxiv.org/pdf/1209.2137v5.pdf declares why shouldn't we encoding delta in this way... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
