andygrove opened a new pull request, #1419: URL: https://github.com/apache/arrow-java/pull/1419
## What's Changed The C Data Interface importer ignored the `offset` of an `ArrowArray` struct, and since #1263 rejected it outright. Exporting a sliced `RecordBatch` from C++ puts the slice offset on every child array, so such batches could not be imported into Java. `ArrayImporter` now supports a non-zero offset. Java vectors cannot represent an offset, so the importer loads the leading `offset + length` values into a temporary vector of the same field and transfers the requested window into the target vector with `splitAndTransfer`. This reuses the per-type slicing that already exists in the vector module: fixed-width data buffers and byte-aligned validity bitmaps are shared with the C memory, while offset buffers, unaligned validity bitmaps and view data are copied. Children with an offset of their own are handled by their own recursive import, so struct, list, union, run-end encoded, extension and dictionary encoded arrays all work without type-specific code. The null count reported by a producer only covers the slice, so the temporary import passes an unknown null count (-1) when a validity bitmap is present and 0 when the producer omitted it. Tests cover int (aligned and unaligned offsets), bit, varchar, string view, list, large list, fixed-size list, map, struct, sparse and dense union, run-end encoded, extension type (top level and nested), dictionary encoded arrays, and the original report's scenario of a `VectorSchemaRoot` whose children carry the slice offset. The documentation now describes offset support and its copying caveat instead of stating that offsets are unsupported. Closes #88. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
