Wes McKinney created ARROW-399:

             Summary: [Java] ListVector.loadFieldBuffers ignores the 
ArrowFieldNode length metadata
                 Key: ARROW-399
                 URL: https://issues.apache.org/jira/browse/ARROW-399
             Project: Apache Arrow
          Issue Type: Bug
          Components: Java - Vectors
            Reporter: Wes McKinney
            Priority: Blocker
         Attachments: list_error.json

Discovered this during integration testing. Because Arrow-C++ writes buffers 
padded to 64 bytes, they may appear larger to the Java library than they need 
to be. In ListVector.loadFieldBuffers, the ArrowFieldNode is never used:

  public void loadFieldBuffers(ArrowFieldNode fieldNode, List<ArrowBuf> 
ownBuffers) {
    BaseDataValueVector.load(getFieldInnerVectors(), ownBuffers);

The value count of the resulting ListVector is thus inferred from the size of 
the offsets buffer. In the case of a length-7 vector in C++, the size of the 
offsets buffer is exactly 64 bytes (padding for SIMD) -- Java infers from 64 
bytes that the value count is 15 (64 / 4 - 1), and the integration test fails.

This message was sent by Atlassian JIRA

Reply via email to