Hi Jim, I remember chatting with Jignesh Patel about it at the time. Since his company locomatix was acquired by twitter we had him as an adviser for some time. That sounds like an interesting possibility. It's not that fresh in my mind but I'd say from the storage perspective it's a variation of bit packing. right? We would need an implementation of a runtime for this to make sense, so I suppose that the impala team is looking into implementing this? It would be interesting to have this type of "compressed" vector in Arrow too. But I don't know if you're looking into Arrow on your end Cheers, Julien
On Mon, Oct 8, 2018 at 2:53 PM Jim Apple <jbap...@apache.org> wrote: > The BitWeaving paper from a few years ago demonstrates some large > performance wins in predicate evaluation based partially on reconfiguring > the storage layout: > > http://pages.cs.wisc.edu/~jignesh/publ/BitWeaving.pdf > > Is it technically possible for Parquet to support "Vertical Bit-Parallel" > layout as an option? >
