Aljoscha Krettek created FLINK-3200:
---------------------------------------
Summary: Use Partitioned State Abstraction in WindowOperator
Key: FLINK-3200
URL: https://issues.apache.org/jira/browse/FLINK-3200
Project: Flink
Issue Type: Improvement
Components: Streaming
Affects Versions: 1.0.0
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Right now, the WindowOperator uses snapshotState and restoreState to do custom
serialization/deserialization of the windowing state. Moving this to
partitioned state would allow us to re-scale in the future. Also, once the
partitioned state has support to run on ManagedMemory/Out-of-core state the
WindowOperator would also automatically benefit from this, allowing very large
windows and large window state.
The necessary steps are these:
- Enhance state interface with new primitive states: ReducableState, ListState,
etc.. Also make state scoped to a namespace. Namespaces are necessary because
we need state that is scoped by key and by window (window would be the
namespace in this case)
- Enhance timer interface to also make timers scoped to a key/namespace
- Change WindowOperator to use these new interfaces
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
