[
https://issues.apache.org/jira/browse/FLINK-21467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17306227#comment-17306227
]
Kezhu Wang commented on FLINK-21467:
------------------------------------
Hi [~pnowojski], here are my imaginative paths:
1. An operator instance enters its finishing steps. {{MAX_WATERMARK}} is
sending out in this phase.
2. A checkpoint completes and {{MAX_WATERMARK}} was overtaken and persisted in
that checkpoint. {{MAX_WATERMARK}} could be persisted in either that operator
instance's result partition or downstream operator instance's input channel.
3. Job restores to checkpoint taken from step#2 due to failure/rescale before
next checkpoint. The restoring job has {{MAX_WATERMARK}} in channel now.
4. If previous finishing operator instance back to work and produces new
records due to "non deterministic", then these records will be late in
downstream windowing operators.
> Document possible recommended usage of Bounded{One/Multi}Input.endInput and
> emphasize that they could be called multiple times
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-21467
> URL: https://issues.apache.org/jira/browse/FLINK-21467
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Affects Versions: 1.13.0
> Reporter: Kezhu Wang
> Priority: Major
>
> It is too tempting to use these api, especially {{BoundedOneInput.endInput}},
> to commit final result before FLIP-147 delivered. And this will cause
> re-commit after failover as [~gaoyunhaii] has pointed out in FLINK-21132.
> I have
> [pointed|https://github.com/apache/iceberg/issues/2033#issuecomment-784153620]
> this out in
> [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033], please
> correct me if I was wrong.
> cc [~aljoscha] [~pnowojski] [~roman_khachatryan]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
