Hi, Gary is this a joined project with Arrow? or you plan to build own columnar collection?
Thanks yanping On Wed, Aug 17, 2016 at 11:02 AM, Gary <ga...@apache.org> wrote: > Hi, > > Basically, we have completed the durable native computing infra. since the > last release, so it might be a good time to think about how to introduce > the Columnar layout of data into Apache Mnemonic, the reason are > > 1) to take advantage of the SIMD > > 2) to optimize the use of CPU caches > > 3) to compact tensor data > > I figure out the following approaches for your consideration. > > a) to build a set of durable columnar collections for data (using > MemBuffer or MemChunk as underlying storage) > > b) to let durable native computing service layout columnar data as needed > (possible to gather scattered data using AVX instructions without moving > data) > > Please offer advice, Thanks. > > Cheers > +Gary > > > >
