[
https://issues.apache.org/jira/browse/FLINK-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743787#comment-14743787
]
Stephan Ewen commented on FLINK-2283:
-------------------------------------
No state backend can handle infinite state, how should it do that?
This is going to be an inherently super dangerous feature, it runs the risk of
failing badly a few months into operation. It is too easy to misuse.
Given that, we need to decide between the following points:
- Do we want to make the API such that it offers only safe operations where
you do not shoot yourself into the foot (without realizing), or do we want such
unsafe operations in there.
- Is this an important operation in the infinite window sense, or would a
finite window version be an acceptable replacement (given that it wins us
safety of operations)?
> Make grouped reduce/fold/aggregations stateful using Partitioned state
> ----------------------------------------------------------------------
>
> Key: FLINK-2283
> URL: https://issues.apache.org/jira/browse/FLINK-2283
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 0.10
> Reporter: Gyula Fora
> Assignee: Márton Balassi
> Priority: Minor
>
> Currently the inner state of the grouped aggregations are not persisted as an
> operator state.
> These operators should be reimplemented to use the newly introduced
> partitioned state abstractions which will make them fault tolerant and
> scalable for the future.
> A suggested implementation would be to use a stateful mapper to implement the
> desired behaviour.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
