[
https://issues.apache.org/jira/browse/FLINK-21467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17290781#comment-17290781
]
Kezhu Wang commented on FLINK-21467:
------------------------------------
Hi [~pnowojski], I see the possibility. But I think there is little work Flink
can do to cope with this kind of issues. The checkpoint could be a savepoint
triggered from user side and the "non deterministic logic" could be a change
from user(eg. changing of stoppingOffsets in KafkaSource). In this case, after
resuming from latest checkpoint/savepoint, {{endOfInput}} was run once but it
is not belong to current run.
I think, maybe, the documentation should focus more on "no guarantee" for these
methods to commit side effects to external systems.
> Document possible recommended usage of Bounded{One/Multi}Input.endInput and
> emphasize that they could be called multiple times
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-21467
> URL: https://issues.apache.org/jira/browse/FLINK-21467
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Affects Versions: 1.13.0
> Reporter: Kezhu Wang
> Priority: Major
>
> It is too tempting to use these api, especially {{BoundedOneInput.endInput}},
> to commit final result before FLIP-147 delivered. And this will cause
> re-commit after failover as [~gaoyunhaii] has pointed out in FLINK-21132.
> I have
> [pointed|https://github.com/apache/iceberg/issues/2033#issuecomment-784153620]
> this out in
> [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033], please
> correct me if I was wrong.
> cc [~aljoscha] [~pnowojski] [~roman_khachatryan]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
