Hi All, Currently we are working on SIMD chance to improves speed of DecodeSpaced on Jira<https://issues.apache.org/jira/browse/PARQUET-1841>, the path for AVX512 is identified with the helper of vpcompressd/vpexpandd instructions. I has uploaded the PR<https://github.com/apache/arrow/pull/7029> for this path and other SIMD unittest, benchmark things.
For the SSE chance, thanks to help from Micah Kornfield, the path can be achieved by a lookup table, method referenced from https://github.com/lemire/simdprune. I has a draft implementation locally which demonstrate double speed on epi32 path and 50% improvements on epi64. The issue is how to approach the reference, one way is vendor the whole simdprune library while another choice is just introducing the lookup table/script we really used. I personally like the later one as we need expand the table to support both encode/decode, simdprune only has encode table, also we need a new epi64 table to support the double type. Thoughts are welcome😊 Thanks, Frank
