[
https://issues.apache.org/jira/browse/ARROW-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736089#comment-15736089
]
Julien Le Dem commented on ARROW-414:
-------------------------------------
So it's complaining that the padding is more than 64 bytes.
based on fieldNode.getLength() the buffer should be of length 44 ((10 values +
1) x 4 bytes)
I added a check that padding should be no more than 64 bytes so 128 is too big.
(and I guess I should verify that it is on 64 bytes boundaries instead of < 64)
Is the padding rule correct?
How strict do we want to be?
In general I prefer to be stricter to catch bugs.
> [Java] "Buffer too large to resize to ..." error
> ------------------------------------------------
>
> Key: ARROW-414
> URL: https://issues.apache.org/jira/browse/ARROW-414
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java - Vectors
> Reporter: Wes McKinney
> Attachments: struct_example.arrow, struct_example.json
>
>
> On the latest branch for ARROW-394 (after rebasing on all the fixes so far),
> I have the error
> {code}
> Incompatible files
> Could not load buffers for field f2: Utf8. error message: Buffer too large to
> resize to 44: 128
> 13:41:19.356 [main] ERROR org.apache.arrow.tools.Integration - Incompatible
> files
> java.lang.IllegalArgumentException: Could not load buffers for field f2:
> Utf8. error message: Buffer too large to resize to 44: 128
> at
> org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:84)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at
> org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:94)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:63)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at
> org.apache.arrow.tools.Integration$Command$3.execute(Integration.java:156)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at org.apache.arrow.tools.Integration.run(Integration.java:212)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at org.apache.arrow.tools.Integration.main(Integration.java:61)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> Caused by: java.lang.IllegalArgumentException: Buffer too large to resize to
> 44: 128
> at
> org.apache.arrow.vector.BaseDataValueVector.truncateBufferBasedOnSize(BaseDataValueVector.java:55)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at
> org.apache.arrow.vector.NullableVarCharVector.loadFieldBuffers(NullableVarCharVector.java:142)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> at
> org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:82)
> ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> ... 5 common frames omitted
> {code}
> See attached JSON and C++-generated Arrow file
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
