Hi Frank,
Vendoring just the required lookup table sounds fine to me. Regards Antoine. Le 29/04/2020 à 08:25, Du, Frank a écrit : > Hi All, > > Currently we are working on SIMD chance to improves speed of DecodeSpaced on > Jira<https://issues.apache.org/jira/browse/PARQUET-1841>, the path for AVX512 > is identified with the helper of vpcompressd/vpexpandd instructions. I has > uploaded the PR<https://github.com/apache/arrow/pull/7029> for this path and > other SIMD unittest, benchmark things. > > For the SSE chance, thanks to help from Micah Kornfield, the path can be > achieved by a lookup table, method referenced from > https://github.com/lemire/simdprune. I has a draft implementation locally > which demonstrate double speed on epi32 path and 50% improvements on epi64. > > The issue is how to approach the reference, one way is vendor the whole > simdprune library while another choice is just introducing the lookup > table/script we really used. I personally like the later one as we need > expand the table to support both encode/decode, simdprune only has encode > table, also we need a new epi64 table to support the double type. Thoughts > are welcome😊 > > Thanks, > Frank >
