[
https://issues.apache.org/jira/browse/FLINK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299827#comment-15299827
]
Stephan Ewen commented on FLINK-3947:
-------------------------------------
Concerning iterating over an ordered {{MapState}}: Right now we explicitly
limit the access to keys other than the element's current key.
The reason is that keys may be redistributed as part of the dynamic scaling
feature:
https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit
Can you explain a bit what you would need the ordered map state for? Maybe we
can find a different way to incorporate that behavior, while staying compatible
with the key group and rescaling design.
> Provide low level access to RocksDB state backend
> -------------------------------------------------
>
> Key: FLINK-3947
> URL: https://issues.apache.org/jira/browse/FLINK-3947
> Project: Flink
> Issue Type: Improvement
> Components: state backends
> Affects Versions: 1.0.3
> Reporter: Elias Levy
>
> The current state API is limiting and some implementations are not as
> efficient as they could be, particularly when working with large states. For
> instance, a ListState is append only. You cannot remove values from the
> list. And the RocksDBListState get() implementation reads all list values
> from RocksDB instead of returning an Iterable that only reads values as
> needed.
> Furthermore, RocksDB is an ordered KV store, yet there is no ordered map
> state API with an ability to iterate over the stored values in order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
