[
https://issues.apache.org/jira/browse/FLINK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen resolved FLINK-24347.
----------------------------------
Fix Version/s: 1.13.3
1.14.0
Resolution: Fixed
Fixed in
- master via d753cf20826c66f20bfbd3772c58709e00d9bcb7
- 1.14.0 (release-1.14) via a595fbbfe0087181f1b92d2d991a885911f290a2
- 1.13.3 (release-1.13) via 28211c0dec1dfb706c5d9583fd757cdd4de38ce8
> KafkaSource cannot checkpoint if the parallelism is higher than the partition
> number
> ------------------------------------------------------------------------------------
>
> Key: FLINK-24347
> URL: https://issues.apache.org/jira/browse/FLINK-24347
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.14.0, 1.13.2
> Reporter: Fabian Paul
> Assignee: Fabian Paul
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.3
>
>
> The KafkaSourceEnumerator signals all the readers if there are no more splits
> available and the readers shut down. In case the parallelism is higher than
> the partitions of the consumed topic there are not enough partitions to
> distribute and reader subtasks go immediately into FINISHED state.
> Currently, it is not possible to checkpoint if parts of the job are finished.
> (this is lifted once FLIP-147 is by default enabled Flink 1.15+)
> We should only signal to the readers to go into finished if the job is in
> bounded execution mode and keep them idling otherwise.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
