One idea is whether the Plasma object store could be extended to support devices other than POSIX shared memory, like GPU device memory (or multiple GPUs on a single host).
Philipp or Robert or any of the people who know the Plasma code best, any idea how this might be approached? It would have to be developed as an optional extension so that users without e.g. a CUDA installation don't have to bother with nvcc (which is proprietary) or the CUDA runtime libraries. - Wes On Mon, Aug 7, 2017 at 2:15 PM, Wes McKinney <[email protected]> wrote: > hi all, > > A group of companies have created a project called the GPU Open > Analytics Initiative (GOAI), with the purpose of creating open source > software and specifications for analytics on GPU. > > So far, they have focused on building a "GPU Data Frame", which is > effectively putting Arrow data on the GPU: > > https://github.com/gpuopenanalytics/libgdf/wiki/Technical-Overview > http://gpuopenanalytics.com/ > > Shared memory IPC and analytics on Arrow data beyond the CPU are > definitely in scope for the Arrow project, so we should look for ways > to collaborate and help each other. I am sure this will not be the > last time that someone needs to use Arrow memory with GPUs, so it > would be useful for the community to develop memory management and > utility code to assist with using Arrow in a mixed-device setting. > > I am not sure how to best proceed but wanted to make everyone aware of > GOAI and look for opportunities to grow the Arrow community. > > Thanks, > Wes
