[
https://issues.apache.org/jira/browse/FLINK-33224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18134252#comment-18134252
]
Martijn Visser commented on FLINK-33224:
----------------------------------------
FLINK-40499 covers the final MAX_WATERMARK not being emitted when the source
was idle at the end. It doesn't explain the job staying up: a bounded job
finishes without that watermark.
> Flink Batch Job does not complete when using Kafka topics with empty
> partitions
> --------------------------------------------------------------------------------
>
> Key: FLINK-33224
> URL: https://issues.apache.org/jira/browse/FLINK-33224
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.14.6
> Reporter: Kanthi Vaidya
> Priority: Major
>
> Kafka Source Connector no longer throws an exception when running a flink job
> in batch mode when using Kafka topics with empty partitions. However, flink
> does not emit the final watermark of Long.MAX_VALUE even if all the messages
> have been read. Note that we are using bounded sources, since that is a
> requirement for batch mode. It appears that the Kafka consumer continues to
> run even though the source is bounded.
> Same is the case with an empty topic. We do have optional sources, where data
> need not be present all the time. However if we mark the source as bounded,
> the Kafka connector should recognize empty partitions and mark it as
> completed.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
