platinumhamburg commented on code in PR #2165:
URL: https://github.com/apache/fluss/pull/2165#discussion_r2654308063
##########
fluss-common/src/main/java/org/apache/fluss/row/arrow/writers/ArrowArrayWriter.java:
##########
@@ -41,7 +41,11 @@ public void doWrite(int rowIndex, DataGetters row, int
ordinal, boolean handleSa
listVector.startNewValue(rowIndex);
for (int arrIndex = 0; arrIndex < array.size(); arrIndex++) {
int fieldIndex = offset + arrIndex;
- elementWriter.write(fieldIndex, array, arrIndex, handleSafe);
+ // Always use safe writes for array elements because the element
index (offset +
+ // arrIndex) can exceed INITIAL_CAPACITY even when the row count
doesn't. The parent's
+ // handleSafe is based on row count, but array element indices
grow based on the total
+ // number of elements across all arrays, which can be much larger.
+ elementWriter.write(fieldIndex, array, arrIndex, true);
Review Comment:
```suggestion
boolean elementHandleSafe = fieldIndex >=
ArrowWriter.INITIAL_CAPACITY;
elementWriter.write(fieldIndex, array, arrIndex,
elementHandleSafe);
```
##########
fluss-common/src/main/java/org/apache/fluss/row/arrow/writers/ArrowArrayWriter.java:
##########
@@ -41,7 +41,11 @@ public void doWrite(int rowIndex, DataGetters row, int
ordinal, boolean handleSa
listVector.startNewValue(rowIndex);
for (int arrIndex = 0; arrIndex < array.size(); arrIndex++) {
int fieldIndex = offset + arrIndex;
- elementWriter.write(fieldIndex, array, arrIndex, handleSafe);
+ // Always use safe writes for array elements because the element
index (offset +
+ // arrIndex) can exceed INITIAL_CAPACITY even when the row count
doesn't. The parent's
+ // handleSafe is based on row count, but array element indices
grow based on the total
+ // number of elements across all arrays, which can be much larger.
+ elementWriter.write(fieldIndex, array, arrIndex, true);
Review Comment:
Perhaps we could apply a clear conditional judgment here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
