hi Yurui, You can also share data structures through JNI without using the IPC tools at all, which could require memory copying to produce the IPC messages.
What you can do is obtain the memory addresses for the component buffers of an array (or vector, as called in Java) and construct the data structure from the memory addresses on the other side. We are doing exactly this already in Python using JPype (which is JNI-based): https://github.com/apache/arrow/blob/master/python/pyarrow/jvm.py The Gandiva project uses JNI to pass Java Netty buffer memory addresses to C++, you can see the code for creating the arrays from the memory addresses and then constructing a RecordBatch: https://github.com/dremio/gandiva/blob/master/cpp/src/jni/native_builder.cc#L602 I believe as time goes on we will have better and more standardized APIs to deal with JNI<->C++ zero-copy passing, these implementations have only been done relatively recently. Your contributions to the Arrow project around this would be most welcomed! Thanks, Wes On Mon, Jul 16, 2018 at 2:00 PM, Philipp Moritz <pcmor...@gmail.com> wrote: > Hey Yuri, > > you can use the Arrow IPC mechanism to do this: > > - https://github.com/apache/arrow/blob/master/format/IPC.md > - Python: https://arrow.apache.org/docs/python/ipc.html > - C++: https://arrow.apache.org/docs/cpp/namespacearrow_1_1ipc.html > - For Java, see the org.apache.arrow.vector.ipc namespace > > On the C++ side, you can for example use a RecordBatchStreamWriter to write > the IPC message, and then on the Java side you could use the > ArrowStreamReader to read it. > > There are some tests here: > https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/ipc-read-write-test.cc > https://github.com/apache/arrow/tree/master/java/vector/src/test/java/org/apache/arrow/vector/ipc > > There is also integration tests here, although I'm not really familiar with > them: > > https://github.com/apache/arrow/tree/master/integration > > If you could write a little tutorial/into on how to do this (maybe using > Plasma for exchanging the data) and contribute it to the documentation, > that would be amazing! > > Best, > Philipp. > > On Mon, Jul 16, 2018 at 4:14 AM, 周宇睿(闻拙) <yurui....@alibaba-inc.com> wrote: > >> Hi guys: >> >> I might miss something quite obviously. But how does Arrow passing objects >> across language? Let’s say I have a java program that invoke a c++ function >> via JNI, how does the c++ function pass an Arrow RecordBack object back to >> Java runtime without memory copy? >> >> Any advise would be appreciated. >> Thanks >> Yurui >> >> from Alimail macOS
