Could simply saving the arrow file in streaming mode to shared memory and then mmap-ing the result in each language solve your problem ? Plasma seems to me to be a layer on top of basic mmap operations; as long as you have shared memory and mmap then you can have multiple processes talking to the same logical block of memory.
On Mon, Jan 4, 2021 at 8:27 AM Thomas Browne <[email protected]> wrote: > I am hoping to use the Apache Arrow project for cross-language numerical > computation, and for that the shared-memory idea is very powerful. Am I > correct that the Plasma Store is the enabling technology for this, > especially for soft real-time computation (ie not moving to parquet or > any file-based sharing system)? > > Is that the case? And if so, then I'm wondering which client libraries, > other than Python (and I assume C[++]), implement the Plasma Store. This > table doesn't feature a row for Plasma: > > https://arrow.apache.org/docs/status.html > > and I can't seem to find any reference to the Plasma store in the Julia, > R, or Javascript libraries. > > https://arrow.apache.org/docs/r/ > > https://arrow.apache.org/docs/js/ > > https://arrow.juliadata.org/stable/ > > > Thank you, > > Thomas > > >
