lidavidm commented on code in PR #41967:
URL: https://github.com/apache/arrow/pull/41967#discussion_r1635865070
##########
java/vector/src/main/java/org/apache/arrow/vector/FieldVector.java:
##########
@@ -56,6 +56,13 @@ public interface FieldVector extends ValueVector {
*/
List<ArrowBuf> getFieldBuffers();
+ /**
+ * Retrieves the export buffer count for the C Data Interface.
+ *
+ * @return the number of variadic buffers
+ */
+ int getExportedCDataBuffers();
Review Comment:
nit: let's name this with Count so it's clear
##########
java/c/src/main/java/org/apache/arrow/c/BufferImportTypeVisitor.java:
##########
@@ -227,10 +227,39 @@ public List<ArrowBuf> visit(ArrowType.Utf8 type) {
}
}
+ private List<ArrowBuf> visitVariableWidthView(ArrowType type) {
+ final int viewBufferIndex = 1;
+ final int variadicSizeBufferIndex = this.buffers.length - 1;
+ final long numOfVariadicBuffers = this.buffers.length - 3;
+ final long variadicSizeBufferCapacity = numOfVariadicBuffers * Long.BYTES;
+ List<ArrowBuf> buffers = new ArrayList<>();
+
+ try (ArrowBuf variadicSizeBuffer =
+ importBuffer(type, variadicSizeBufferIndex,
variadicSizeBufferCapacity)) {
+ ArrowBuf maybeValidityBuffer = maybeImportBitmap(type);
+ try (ArrowBuf view =
+ importFixedBytes(type, viewBufferIndex,
BaseVariableWidthViewVector.ELEMENT_SIZE)) {
+ view.getReferenceManager().retain();
+ buffers.add(maybeValidityBuffer);
+ buffers.add(view);
+ }
+ // 0th buffer is validity buffer
+ // 1st buffer is view buffer
+ // 2nd buffer onwards are variadic buffer
+ // N-1 (this.buffers.length - 1) buffer is variadic size buffer
+ final int variadicBufferReadOffset = 2;
+ variadicSizeBuffer.getReferenceManager().retain();
Review Comment:
Why do we need to retain this buffer?
##########
java/c/src/main/java/org/apache/arrow/c/BufferImportTypeVisitor.java:
##########
@@ -227,10 +227,39 @@ public List<ArrowBuf> visit(ArrowType.Utf8 type) {
}
}
+ private List<ArrowBuf> visitVariableWidthView(ArrowType type) {
+ final int viewBufferIndex = 1;
+ final int variadicSizeBufferIndex = this.buffers.length - 1;
+ final long numOfVariadicBuffers = this.buffers.length - 3;
+ final long variadicSizeBufferCapacity = numOfVariadicBuffers * Long.BYTES;
+ List<ArrowBuf> buffers = new ArrayList<>();
+
+ try (ArrowBuf variadicSizeBuffer =
+ importBuffer(type, variadicSizeBufferIndex,
variadicSizeBufferCapacity)) {
+ ArrowBuf maybeValidityBuffer = maybeImportBitmap(type);
+ try (ArrowBuf view =
+ importFixedBytes(type, viewBufferIndex,
BaseVariableWidthViewVector.ELEMENT_SIZE)) {
+ view.getReferenceManager().retain();
+ buffers.add(maybeValidityBuffer);
+ buffers.add(view);
+ }
Review Comment:
Why does the view buffer need a refcount but not the validity buffer?
##########
java/c/src/main/java/org/apache/arrow/c/ArrayExporter.java:
##########
@@ -91,7 +91,8 @@ void export(ArrowArray array, FieldVector vector,
DictionaryProvider dictionaryP
if (buffers != null) {
data.buffers = new ArrayList<>(buffers.size());
Review Comment:
Shouldn't this also use the new buffer count? And we shouldn't call
getFieldBuffers above at all anymore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
