Re: Re: Sharing Java Arrow Buffer with C++ in same process

2019-07-17 Thread Ravindra Pindikura
*gandiva* does the sharing in one direction i.e buffers allocated in java, and passed along to c++ in jni calls. The java side code of extracting of the address and passing it down is here

Aw: Re: Sharing Java Arrow Buffer with C++ in same process

2019-07-17 Thread hans-joachim . bothe
Thanks to Uwe! The Python step from C++ would have been next. You hit it! Perfect shortcut. Jacques, i am new here but i am gladly willing to help and learn. if you could give me a hint for an outline and content of the blog post and where/how to file it. Will read #3191 next. greetings &

Re: Sharing Java Arrow Buffer with C++ in same process

2019-07-17 Thread Wes McKinney
I agree that a blog post would be helpful (of course, blogs are a lot of work to write!) -- it would be good to let people know that the machinery is available to do this kind of bidirectional zero-copy passing. I was glad to see ARROW-3191 and I would like to see the Java folks "tooting their own

Re: Sharing Java Arrow Buffer with C++ in same process

2019-07-17 Thread Jacques Nadeau
Java > C++ has been available for a long time, I believe. With ARROW-3191, we now have a good story as well for C++ > Java. Reference management is the key thing you have to think about. Now that we have the ReferenceManager interface in Java, you can decide how this works between the two layers.

Re: Sharing Java Arrow Buffer with C++ in same process

2019-07-17 Thread Uwe L. Korn
Hello Hans, we sadly have no code for the C++<->Java interaction but a good example is the Python<->Java interaction code in https://github.com/apache/arrow/blob/master/python/pyarrow/jvm.py . This call Java from Python using the jpype1 module and then uses the memory pointers in the Java

Sharing Java Arrow Buffer with C++ in same process

2019-07-17 Thread hans-joachim . bothe
In my application I need to share Arrow buffers allocated in Java with C++ in the same process. Is there already some code in Arrow to pass the native address from Java to C++ or do I have to do my own JNI call? I do not want to go via the Plasma sockets and did not find any hint in docs and