[
https://issues.apache.org/jira/browse/FLINK-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487750#comment-16487750
]
ASF GitHub Bot commented on FLINK-9428:
---------------------------------------
Github user bowenli86 commented on a diff in the pull request:
https://github.com/apache/flink/pull/6066#discussion_r190342404
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamOperator.java
---
@@ -93,6 +93,20 @@
// state snapshots
//
------------------------------------------------------------------------
+ /**
--- End diff --
This description feels like it states well what it's not intended for, but
doesn't clearly describe what it's intended for.
It would be great if you can add what you wrote in the ticket description
here, as of "Some operators maintain some small transient state ... Rather
that persisting that state in a checkpoint, it can make sense to flush the data
downstream ...."
> Allow operators to flush data on checkpoint pre-barrier
> -------------------------------------------------------
>
> Key: FLINK-9428
> URL: https://issues.apache.org/jira/browse/FLINK-9428
> Project: Flink
> Issue Type: New Feature
> Components: State Backends, Checkpointing
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Major
> Fix For: 1.6.0
>
>
> Some operators maintain some small transient state that may be inefficient to
> checkpoint, especially when it would need to be checkpointed also in a
> re-scalable way.
> An example are opportunistic pre-aggregation operators, which have small the
> pre-aggregation state that is frequently flushed downstream.
> Rather that persisting that state in a checkpoint, it can make sense to flush
> the data downstream upon a checkpoint, to let it be part of the downstream
> operator's state.
> This feature is sensitive, because flushing state has a clean implication on
> the downstream operator's checkpoint alignment. However, used with care, and
> with the new back-pressure-based checkpoint alignment, this feature can be
> very useful.
> Because it is sensitive, I suggest to make this only an internal feature
> (accessible to operators) and NOT expose it in the public API at this point.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
