[
https://issues.apache.org/jira/browse/FLINK-21467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289946#comment-17289946
]
Piotr Nowojski commented on FLINK-21467:
----------------------------------------
Yes, I agree {{Bounded{One/Multi}Input}} interfaces should be documented. Note,
that it's not only that {{endOfInput}} can be called multiple times. There is a
very remote possibility that the following scenario will happen:
1. checkpoint is taken (successfully)
2. sources are finishing
3. endOfInput is issued
4. job fails
5. job restarts to checkpoint 1.
6. after failover, because of some non deterministic logic in the source,
sources are not finishing
So the {{endOfInput()}} from 3., will not be repeated, and it should have been
discarded. Two phase commit support from FLIP-147 should provide a way how to
deal with this kind of issues.
However there is a disagreement whether the operators should be our official
API or not in the first place. (I think they should and currently operator API
is `@PublicEvolving`)
> Document possible recommended usage of Bounded{One/Multi}Input.endInput and
> emphasize that they could be called multiple times
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-21467
> URL: https://issues.apache.org/jira/browse/FLINK-21467
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Affects Versions: 1.13.0
> Reporter: Kezhu Wang
> Priority: Major
>
> It is too tempting to use these api, especially {{BoundedOneInput.endInput}},
> to commit final result before FLIP-147 delivered. And this will cause
> re-commit after failover as [~gaoyunhaii] has pointed out in FLINK-21132.
> I have
> [pointed|https://github.com/apache/iceberg/issues/2033#issuecomment-784153620]
> this out in
> [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033], please
> correct me if I was wrong.
> cc [~aljoscha] [~pnowojski] [~roman_khachatryan]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
