roee88 opened a new pull request #11067:
URL: https://github.com/apache/arrow/pull/11067
Adds experimental support for C Data Interface in Arrow Java.
Roundtrip example with dictionaries:
```java
void roundtrip(FieldVector vector, DictionaryProvider provider) {
// Consumer allocates empty structures
try (ArrowSchema consumerArrowSchema =
ArrowSchema.allocateNew(allocator);
ArrowArray consumerArrowArray = ArrowArray.allocateNew(allocator)) {
// Producer creates structures from existing memory pointers
try (ArrowSchema arrowSchema =
ArrowSchema.wrap(consumerArrowSchema.memoryAddress());
ArrowArray arrowArray =
ArrowArray.wrap(consumerArrowArray.memoryAddress())) {
// Producer exports vector into the FFI structures
FFI.exportVector(allocator, vector, provider, arrowArray,
arrowSchema);
}
// Consumer imports vector (and dictionaries)
try (FFIDictionaryProvider dictionaries = new FFIDictionaryProvider();
FieldVector imported = FFI.importVector(allocator,
consumerArrowArray, consumerArrowSchema, dictionaries)) {
// Do something
}
}
}
```
Some notes and questions:
1. Only enabled when arrow-ffi profile is enabled (currently disabled by
default). Should it be enabled by default? How best to allow enabling it
(+integrate in CI)?
1. Requires building some JNI code (used just for private data and release
callback). This is currently placed in the java/ffi module itself because this
code does not require Arrow C++. Is that the right place for it?
1. The API is marked EXPERIMENTAL and is as close to the C++ API as possible
1. Conversion between StructVector and RecordBatch is not available in the
Java implementation. We added it here by mimicking the
VectorLoader/VectorUnloader approach. Is that the right approach? Where should
this code reside?
1. As a design choice all allocations are done with a BufferAllocator. This
is required for use cases that need a custom memory allocator (we are working
on a Java-Rust demonstration using WebAssembly).
1. A ZeroVector is exported/imported as a NullVector (there is no C Data
interface format for it).
1. Integration tests will be added in a future PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
