[
https://issues.apache.org/jira/browse/FLINK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-8297:
----------------------------------
Labels: auto-deprioritized-major pull-request-available stale-minor (was:
auto-deprioritized-major pull-request-available)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> RocksDBListState stores whole list in single byte[]
> ---------------------------------------------------
>
> Key: FLINK-8297
> URL: https://issues.apache.org/jira/browse/FLINK-8297
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.3.2, 1.4.0
> Reporter: Jan Lukavský
> Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available,
> stale-minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> RocksDBListState currently keeps whole list of data in single RocksDB
> key-value pair, which implies that the list actually must fit into memory.
> Larger lists are not supported and end up with OOME or other error. The
> RocksDBListState could be modified so that individual items in list are
> stored in separate keys in RocksDB and can then be iterated over. A simple
> implementation could reuse existing RocksDBMapState, with key as index to the
> list and a single RocksDBValueState keeping track of how many items has
> already been added to the list. Because this implementation might be less
> efficient in come cases, it would be good to make it opt-in by a construct
> like
> {{new RocksDBStateBackend().enableLargeListsPerKey()}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
