Stephan Ewen created FLINK-9428:
-----------------------------------
Summary: Allow operators to flush data on checkpoint pre-barrier
Key: FLINK-9428
URL: https://issues.apache.org/jira/browse/FLINK-9428
Project: Flink
Issue Type: New Feature
Components: State Backends, Checkpointing
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 1.6.0
Some operators maintain some small transient state that may be inefficient to
checkpoint, especially when it would need to be checkpointed also in a
re-scalable way.
An example are opportunistic pre-aggregation operators, which have small the
pre-aggregation state that is frequently flushed downstream.
Rather that persisting that state in a checkpoint, it can make sense to flush
the data downstream upon a checkpoint, to let it be part of the downstream
operator's state.
This feature is sensitive, because flushing state has a clean implication on
the downstream operator's checkpoint alignment. However, used with care, and
with the new back-pressure-based checkpoint alignment, this feature can be very
useful.
Because it is sensitive, I suggest to make this only an internal feature
(accessible to operators) and NOT expose it in the public API at this point.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
