niekbouman-rl opened a new issue, #1386:
URL: https://github.com/apache/arrow-java/issues/1386
### Describe the bug, including details regarding any error messages,
version, and platform.
Disclaimer: Claude Opus 5 found this bug.
### Describe the bug
`BaseVariableWidthViewVector.splitAndTransferViewBufferAndDataBuffer`
allocates the target's view
buffer without zeroing it:
```java
target.viewBuffer = target.allocator.buffer(length * ELEMENT_SIZE); // line
888 at v19.0.0
```
For an inlined value (12 bytes or fewer) it then writes only the 4-byte
length and the value's
bytes, so the rest of the 16-byte view keeps whatever the allocation held.
The Arrow columnar
format requires those padding bytes to be zero, and `setBytes` already does
this (it zeroes the
view before writing an inlined value), but this path does not.
So any `splitAndTransfer` of a `ViewVarCharVector` / `ViewVarBinaryVector`
can produce invalid
views. That includes `TransferPair.splitAndTransfer` and
`VectorSchemaRoot.slice()`. Consumers
that validate are then right to reject the data. For example, after export
through the C Data
Interface, arrow-rs's `ArrayData::validate_full` fails with:
```
View at index 0 contained non-zero padding for string of length 0
```
### To reproduce
```java
import java.nio.charset.StandardCharsets;
import org.apache.arrow.memory.ArrowBuf;
import org.apache.arrow.memory.BufferAllocator;
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.ViewVarCharVector;
import org.apache.arrow.vector.util.TransferPair;
public class PaddingRepro {
public static void main(String[] args) {
try (BufferAllocator allocator = new RootAllocator();
ViewVarCharVector v = new ViewVarCharVector("s", allocator)) {
v.allocateNew();
v.setSafe(0, "a longer value".getBytes(StandardCharsets.UTF_8));
v.setSafe(1, "".getBytes(StandardCharsets.UTF_8));
v.setValueCount(2);
TransferPair tp = v.getTransferPair(allocator);
tp.splitAndTransfer(1, 1);
ArrowBuf views = ((ViewVarCharVector)
tp.getTo()).getDataBuffer();
for (int i = 4; i < 16; i++) {
System.out.print(views.getByte(i) + " "); // expected all 0
}
System.out.println();
tp.getTo().close();
}
}
}
```
With arrow-vector 19.0.0 on HotSpot 21 this prints (the exact bytes vary):
```
-112 127 0 0 -112 0 0 8 -112 127 0 0
```
### Expected behavior
All 12 bytes after the length of an inlined view are zero after
`splitAndTransfer`, for example
by zeroing the target's view buffer after allocating it (`setZero`), as
`setBytes` does for a
single view. Present at v19.0.0 and unchanged on `main`.
### Component(s)
Java (arrow-vector 19.0.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
