[
https://issues.apache.org/jira/browse/FLINK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294810#comment-15294810
]
Aljoscha Krettek commented on FLINK-3947:
-----------------------------------------
There is already this: FLINK-3582 which is addressed in this PR:
https://github.com/apache/flink/pull/1957
For the {{ListState}} and reading as an {{Iterable}} the only optimization that
we could do is to deserialize the individual list elements as we read them from
the {{Iterable}}. We still get the whole list state as a BLOB from RocksDB
because it knows nothing except BLOBs for keys and values.
> Provide low level access to RocksDB state backend
> -------------------------------------------------
>
> Key: FLINK-3947
> URL: https://issues.apache.org/jira/browse/FLINK-3947
> Project: Flink
> Issue Type: Improvement
> Components: state backends
> Affects Versions: 1.0.3
> Reporter: Elias Levy
>
> The current state API is limiting and some implementations are not as
> efficient as they could be, particularly when working with large states. For
> instance, a ListState is append only. You cannot remove values from the
> list. And the RocksDBListState get() implementation reads all list values
> from RocksDB instead of returning an Iterable that only reads values as
> needed.
> Furthermore, RocksDB is an ordered KV store, yet there is no ordered map
> state API with an ability to iterate over the stored values in order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
