Stephan Ewen created FLINK-5590:
-----------------------------------
Summary: Create a proper internal state hierarchy
Key: FLINK-5590
URL: https://issues.apache.org/jira/browse/FLINK-5590
Project: Flink
Issue Type: Improvement
Components: State Backends, Checkpointing
Affects Versions: 1.2.0
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 1.3.0
Currently, the state interfaces (like {{ListState}}, {{ValueState}},
{{ReducingState}}) are very sparse and contain only methods exposed to the
users. That makes sense to keep the public stable API minimal
At the same time, the runtime needs more methods for its internal interaction
with state, such as:
- setting namespaces
- accessing raw values
- merging namespaces
These are currently realized by re-creating or re-obtaining the state objects
from the KeyedStateBackend. That method causes quite an overhead for each
access to the state
The KeyedStateBackend tries to do some tricks to reduce that overhead, but does
it only partially and induces other overhead in the course.
The root cause of all these issues is a problem in the design: There is no
proper "internal state abstraction" in a similar way as there is an external
state abstraction (the public state API).
We should add a similar hierarchy of states for the internal methods. It would
look like in the example below:
{code}
* State
* |
* +-------------------InternalKvState
* | |
* MergingState |
* | |
* +-----------------InternalMergingState
* | |
* +--------+------+ |
* | | |
* ReducingState ListState +-----+-----------------+
* | | | |
* +-----------+ +----------- -----------------InternalListState
* | |
* +---------InternalReducingState
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
