[
https://issues.apache.org/jira/browse/FLINK-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616675#comment-14616675
]
Paris Carbone commented on FLINK-2324:
--------------------------------------
We cannot avoid a key-value store to make this efficient imho. There are
several levelDB backed distributed key value stores we can use to persist and
retrieve partitioned state. There is also HBase though if we want to stick to
HDFS for this...
Any other opinions?
> Rework partitioned state storage
> --------------------------------
>
> Key: FLINK-2324
> URL: https://issues.apache.org/jira/browse/FLINK-2324
> Project: Flink
> Issue Type: Improvement
> Reporter: Gyula Fora
> Assignee: Gyula Fora
>
> Partitioned states are currently stored per-key in statehandles. This is
> alright for in-memory storage but is very inefficient for HDFS.
> The logic behind the current mechanism is that this approach provides a way
> to repartition a state without fetching the data from the external storage
> and only manipulating handles.
> We should come up with a solution that can achieve both.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
