[
https://issues.apache.org/jira/browse/FLINK-21467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307603#comment-17307603
]
Piotr Nowojski edited comment on FLINK-21467 at 3/24/21, 7:15 AM:
------------------------------------------------------------------
{quote}
As a side note, I think there are gaps between operator(or its instance)
"finished" and checkpoint "completed". A finishing operator (or its instance)
could not be counted as "finished" in state as checkpoint could "completed" but
failed before notifyCheckpointComplete materialized.
{quote}
Yes, you are right indeed. It's not as simple just counting it as "finished".
If there will be a finishing state, for example operator has emitted
endOfInput/MAX_WATERMARK, but it's stil waiting for final checkpoint to
complete, if we ever recover back to such state, we must pay attention that
such operator doesn't start emitting new records again.
was (Author: pnowojski):
{quote}
As a side note, I think there are gaps between operator(or its instance)
"finished" and checkpoint "completed". A finishing operator (or its instance)
could not be counted as "finished" in state as checkpoint could "completed" but
failed before notifyCheckpointComplete materialized.
{quote}
Yes, you are right indeed. It's not as simple just counting it as "finished".
> Document possible recommended usage of Bounded{One/Multi}Input.endInput and
> emphasize that they could be called multiple times
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-21467
> URL: https://issues.apache.org/jira/browse/FLINK-21467
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Affects Versions: 1.13.0
> Reporter: Kezhu Wang
> Priority: Major
>
> It is too tempting to use these api, especially {{BoundedOneInput.endInput}},
> to commit final result before FLIP-147 delivered. And this will cause
> re-commit after failover as [~gaoyunhaii] has pointed out in FLINK-21132.
> I have
> [pointed|https://github.com/apache/iceberg/issues/2033#issuecomment-784153620]
> this out in
> [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033], please
> correct me if I was wrong.
> cc [~aljoscha] [~pnowojski] [~roman_khachatryan]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
