My understanding is that you can use java.nio.MappedByteBuffer to work with memory-mapped files as one way to share memory pages between Java (and non-Java) processes without copying.
I am hoping that we can reach a POC of zero-copy Arrow memory sharing Java-to-Java and Java-to-C++ in the near future. Indeed this will have huge implications once we get it working end to end (for example, receiving memory from a Java process in Python without a heavy ser-de step -- it's what we've always dreamed of) and with the metadata and shared memory control flow standardized. - Wes On Wed, Mar 9, 2016 at 9:25 PM, Corey J Nolet <cjno...@gmail.com> wrote: > If I understand correctly, Arrow is using Netty underneath which is using > Sun's Unsafe API in order to allocate direct byte buffers off heap. It is > using Netty to communicate between "client" and "server", information about > memory addresses for data that is being requested. > > I've never attempted to use the Unsafe API to access off heap memory that has > been allocated in one JVM from another JVM but I'm assuming this must be the > case in order to claim that the memory is being accessed directly without > being copied, correct? > > The implication here is huge. If the memory is being directly shared across > processes by them being allowed to directly reach into the direct byte > buffers, that's true shared memory. Otherwise, if there's copies going on, > it's less appealing. > > > Thanks. > > Sent from my iPad
