[
https://issues.apache.org/jira/browse/FLINK-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605357#comment-14605357
]
Gyula Fora commented on FLINK-2283:
-----------------------------------
I would use stateful java maps using PartitionedState for now.
I see several reasons to do this instead of more complex out-of-core
implementations:
-State will be properly checkpointed with no additional implementation
-We can use the state backend (if necessary) to handle out-of-core state (this
is probably a perfect candidate for lazy state fetching)
-This implementation will scale easily if we implement it for the partitioned
state
-It's a trivial implementation, while managed memory will probably be a lot of
overhead
> Make grouped reduce/fold/aggregations stateful using Partitioned state
> ----------------------------------------------------------------------
>
> Key: FLINK-2283
> URL: https://issues.apache.org/jira/browse/FLINK-2283
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 0.10
> Reporter: Gyula Fora
> Priority: Minor
>
> Currently the inner state of the grouped aggregations are not persisted as an
> operator state.
> These operators should be reimplemented to use the newly introduced
> partitioned state abstractions which will make them fault tolerant and
> scalable for the future.
> A suggested implementation would be to use a stateful mapper to implement the
> desired behaviour.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
