Piotr Nowojski created FLINK-40352:
--------------------------------------
Summary: Add CloseableIterator support to MapState
Key: FLINK-40352
URL: https://issues.apache.org/jira/browse/FLINK-40352
Project: Flink
Issue Type: Improvement
Components: Runtime / State Backends
Reporter: Piotr Nowojski
Assignee: Piotr Nowojski
MapState#entries()/#keys()/#values()/#iterator() only expose a plain
Iterator/Iterable. For state backends built on a native resource (e.g.
RocksDB's native iterator), this forces implementations to either eagerly
materialize entries into a Java collection up front, or repeatedly open/close a
native iterator on every #next() call — unnecessary overhead when a scan only
needs part of the map.
We can add a MapState#closeableIterator() default method returning a
CloseableIterator<Map.Entry<UK,UV>> that the caller must close after use,
giving explicit control over the native iterator's lifetime. RocksDBMapState
would it via a new AbstractRocksDBPrefixIterator that keeps a single native
RocksDB iterator open across the whole scan and lazily deserializes the key
only when accessed, rather than the caching strategy #iterator() uses today.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
