wombatu-kun commented on code in PR #16791:
URL: https://github.com/apache/iceberg/pull/16791#discussion_r4023905123
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetWriters.java:
##########
@@ -574,32 +578,32 @@ private MapDataWriter(
LogicalType keyType,
LogicalType valueType) {
super(definitionLevel, repetitionLevel, keyWriter, valueWriter);
- this.keyType = keyType;
- this.valueType = valueType;
+ this.keyGetter = ArrayData.createElementGetter(keyType);
+ this.valueGetter = ArrayData.createElementGetter(valueType);
+ this.entryIterator = new EntryIterator();
}
@Override
protected Iterator<Map.Entry<K, V>> pairs(MapData map) {
- return new EntryIterator<>(map);
- }
-
- private class EntryIterator<K, V> implements Iterator<Map.Entry<K, V>> {
- private final int size;
- private final ArrayData keys;
- private final ArrayData values;
- private final ParquetValueReaders.ReusableEntry<K, V> entry;
- private final ArrayData.ElementGetter keyGetter;
- private final ArrayData.ElementGetter valueGetter;
+ // The parent writer fully consumes the iterator inside a single write()
call, so a single
+ // reusable instance (and its reusable entry) avoids allocating per row.
Review Comment:
Done e810a51730. #18133 adds the javadoc on elements() / pairs() and tests
that a reused iterator instance works across rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
