jbonofre commented on code in PR #1465:
URL: https://github.com/apache/arrow-java/pull/1465#discussion_r4559937056
##########
vector/src/main/java/org/apache/arrow/vector/BaseFixedWidthVector.java:
##########
@@ -503,8 +503,11 @@ public void loadFieldBuffers(ArrowFieldNode fieldNode,
List<ArrowBuf> ownBuffers
ArrowBuf bitBuffer = ownBuffers.get(0);
ArrowBuf dataBuffer = ownBuffers.get(1);
+ // Load the new validity buffer first, so that a failure leaves the vector
unchanged
+ ArrowBuf newValidityBuffer =
+ BitVectorHelper.loadValidityBuffer(fieldNode, bitBuffer, allocator);
validityBuffer.getReferenceManager().release();
- validityBuffer = BitVectorHelper.loadValidityBuffer(fieldNode, bitBuffer,
allocator);
+ validityBuffer = newValidityBuffer;
valueBuffer.getReferenceManager().release();
valueBuffer = dataBuffer.getReferenceManager().retain(dataBuffer,
allocator);
Review Comment:
Good points, thanks. You're right: the offset, size and value buffers had
the same problem (`retain` can still throw, for example for a buffer from
another root allocator), and there's no reason to keep the old contents of a
vector we're loading into.
I changed `loadFieldBuffers` to release the vector's own buffers and reset
its value count before loading anything. A failed load now leaves the vector
empty but closeable, with no extra allocation. The leaf vectors just call
`clear()`, as `BaseVariableWidthViewVector` already did. The list and struct
vectors only release their own buffers, so their child vectors are left alone.
I also added a test where retaining a buffer after the validity buffer
fails. On the previous commit it hit `RefCnt has gone negative` on close for
every vector except `BaseVariableWidthViewVector`, which instead leaked the new
validity buffer, so I reverted that one to `main`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]