lidavidm commented on a change in pull request #12692:
URL: https://github.com/apache/arrow/pull/12692#discussion_r835556364
##########
File path:
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/consumer/ArrayConsumer.java
##########
@@ -90,13 +97,12 @@ public void consume(ResultSet resultSet) throws
SQLException, IOException {
int count = 0;
try (ResultSet rs = array.getResultSet()) {
while (rs.next()) {
- ensureInnerVectorCapacity(innerVectorIndex + count + 1);
Review comment:
It's because of how a ListVector is laid out in memory. The list `[[1,
2], [], [3, 4, 5]]` is represented as the child vector `[1, 2, 3, 4, 5]` and
the offsets `[0, 2, 2, 5]`. `ensureInnerVectorCapacity` is resizing the child
vector, so when we call `consume` for the last element, we want to ensure the
child vector has enough capacity for the current elements, along with all the
previous elements, and it looks like that's what `innerVectorIndex` is
tracking.
In other words when we call `consume` for what will be `[3, 4, 5]` we need
to ensure the child vector has space for at least 3, 4, 5, … elements not 1, 2,
3… elements.
##########
File path:
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java
##########
@@ -134,11 +122,26 @@ private VectorSchemaRoot createVectorSchemaRoot() {
}
throw new RuntimeException("Error occurred while creating schema root.",
e);
}
+
+ // ensure consumers have been initialized
+ ensureInitialized(root);
return root;
}
+ private void ensureInitialized(VectorSchemaRoot root) throws SQLException {
+ if (!initialized) {
Review comment:
It seems this isn't right if `!config.isReuseVectorSchemaRoot()` because
we'll have recreated the root. Since `ensureInitialized` is only ever called
when creating a new root, I don't think we need to guard this with
`initialized`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
