[
https://issues.apache.org/jira/browse/FLINK-33224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser closed FLINK-33224.
----------------------------------
Resolution: Cannot Reproduce
The missing final watermark is fixed by FLINK-40499 in 2.2.2, 2.3.1 and 2.4.0.
The bounded Kafka reader now removes empty splits up front, so a source with
only empty partitions finishes. Closing; please reopen with a reproduction on a
supported Flink and connector version if the job still doesn't complete.
> Flink Batch Job does not complete when using Kafka topics with empty
> partitions
> --------------------------------------------------------------------------------
>
> Key: FLINK-33224
> URL: https://issues.apache.org/jira/browse/FLINK-33224
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.14.6
> Reporter: Kanthi Vaidya
> Priority: Major
>
> Kafka Source Connector no longer throws an exception when running a flink job
> in batch mode when using Kafka topics with empty partitions. However, flink
> does not emit the final watermark of Long.MAX_VALUE even if all the messages
> have been read. Note that we are using bounded sources, since that is a
> requirement for batch mode. It appears that the Kafka consumer continues to
> run even though the source is bounded.
> Same is the case with an empty topic. We do have optional sources, where data
> need not be present all the time. However if we mark the source as bounded,
> the Kafka connector should recognize empty partitions and mark it as
> completed.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
