Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19308#discussion_r140320919
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java
---
@@ -517,6 +517,7 @@ public void loadBytes(ColumnVector.Array array) {
protected void reserveInternal(int newCapacity) {
int oldCapacity = (this.data == 0L) ? 0 : capacity;
if (this.resultArray != null) {
+ oldCapacity = (this.lengthData == 0L) ? 0 : capacity;
--- End diff --
Structs have a similar problem, only `nulls` is used and `data == 0`.
Should we also fix these here?
A related question, maybe we should use `nulls` instead of `data` or
`length` to detect if we are resizing the column or creating a new one.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
