[
https://issues.apache.org/jira/browse/FLINK-25920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17871304#comment-17871304
]
Arvid Heise commented on FLINK-25920:
-------------------------------------
I'm still unable to make any sense of the provided logs. Sorry for the
inconvenience.
My current hypothesis that this can only happen with at least once and it's not
appearing in exactly once.
The design of the current sink is a bit suboptimal for the Kafka sink:
depending on the delivery guarantee configuration, we either need a committer
(EOS) or not. So in AT LEAST ONCE, we still emit an empty set of committables.
This may or may not be the source of the issue. However, it's absolutely
unnecessary to have a committer at all for these cases.
So I recommend just switching to exactly once for the time being and I will try
to improve the design of the sink, which takes another Flink release.
> Allow receiving updates of CommittableSummary
> ---------------------------------------------
>
> Key: FLINK-25920
> URL: https://issues.apache.org/jira/browse/FLINK-25920
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream, Connectors / Common
> Affects Versions: 1.15.0, 1.16.0
> Reporter: Fabian Paul
> Assignee: Arvid Heise
> Priority: Major
> Attachments: trace.zip, trace_again.zip
>
>
> In the case of unaligned checkpoints, it might happen that the checkpoint
> barrier overtakes the records and an empty committable summary is emitted
> that needs to be correct at a later point when the records arrive.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
