pvary commented on code in PR #16791:
URL: https://github.com/apache/iceberg/pull/16791#discussion_r4023472979
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetWriters.java:
##########
@@ -574,32 +578,32 @@ private MapDataWriter(
LogicalType keyType,
LogicalType valueType) {
super(definitionLevel, repetitionLevel, keyWriter, valueWriter);
- this.keyType = keyType;
- this.valueType = valueType;
+ this.keyGetter = ArrayData.createElementGetter(keyType);
+ this.valueGetter = ArrayData.createElementGetter(valueType);
+ this.entryIterator = new EntryIterator();
}
@Override
protected Iterator<Map.Entry<K, V>> pairs(MapData map) {
- return new EntryIterator<>(map);
- }
-
- private class EntryIterator<K, V> implements Iterator<Map.Entry<K, V>> {
- private final int size;
- private final ArrayData keys;
- private final ArrayData values;
- private final ParquetValueReaders.ReusableEntry<K, V> entry;
- private final ArrayData.ElementGetter keyGetter;
- private final ArrayData.ElementGetter valueGetter;
+ // The parent writer fully consumes the iterator inside a single write()
call, so a single
+ // reusable instance (and its reusable entry) avoids allocating per row.
Review Comment:
I have checked and we already expect the reusability, so let me take back my
previous comment.
In this case we should create a separate PR to add javadoc, and test to
ensure that the iterators are reusable.
Then we can use them here without the extra comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
