pitrou commented on issue #55448: URL: https://github.com/apache/arrow/issues/55448#issuecomment-6764871505
We could also improve decoding speed with explicit vectorization. I made a quick-and-dirty attempt using xsimd here: https://gist.github.com/pitrou/b52c3af4aa7b8d8b4dd77f4afbf59e70 Benchmark numbers (AVX2): * before: ``` BM_AlpDecodingFloat/1024 824 ns 824 ns 506872 bytes_per_second=4.62949Gi/s items_per_second=1.24272G/s BM_AlpDecodingFloat/4096 2742 ns 2741 ns 152919 bytes_per_second=5.56733Gi/s items_per_second=1.49447G/s BM_AlpDecodingFloat/32768 20420 ns 20417 ns 20591 bytes_per_second=5.97878Gi/s items_per_second=1.60492G/s BM_AlpDecodingFloat/65536 40767 ns 40712 ns 10150 bytes_per_second=5.99674Gi/s items_per_second=1.60974G/s BM_AlpDecodingDouble/1024 892 ns 891 ns 468092 bytes_per_second=8.56015Gi/s items_per_second=1.14892G/s BM_AlpDecodingDouble/4096 2930 ns 2929 ns 142959 bytes_per_second=10.4178Gi/s items_per_second=1.39825G/s BM_AlpDecodingDouble/32768 21736 ns 21725 ns 19332 bytes_per_second=11.2378Gi/s items_per_second=1.50832G/s BM_AlpDecodingDouble/65536 43531 ns 43526 ns 9269 bytes_per_second=11.2181Gi/s items_per_second=1.50566G/s ``` * after: ``` BM_AlpDecodingFloat/1024 378 ns 378 ns 1106542 bytes_per_second=10.0874Gi/s items_per_second=2.70781G/s BM_AlpDecodingFloat/4096 977 ns 976 ns 426826 bytes_per_second=15.6339Gi/s items_per_second=4.1967G/s BM_AlpDecodingFloat/32768 6499 ns 6498 ns 64461 bytes_per_second=18.786Gi/s items_per_second=5.04284G/s BM_AlpDecodingFloat/65536 12610 ns 12606 ns 33576 bytes_per_second=19.3666Gi/s items_per_second=5.19869G/s BM_AlpDecodingDouble/1024 637 ns 637 ns 660226 bytes_per_second=11.9714Gi/s items_per_second=1.60678G/s BM_AlpDecodingDouble/4096 1929 ns 1928 ns 218526 bytes_per_second=15.8279Gi/s items_per_second=2.12438G/s BM_AlpDecodingDouble/32768 13825 ns 13822 ns 30312 bytes_per_second=17.6638Gi/s items_per_second=2.37079G/s BM_AlpDecodingDouble/65536 27631 ns 27630 ns 15174 bytes_per_second=17.672Gi/s items_per_second=2.3719G/s ``` cc @mapleFU -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
