Bartosz Mikulski created FLINK-29094:
----------------------------------------
Summary: Cannot stop Python job with savepoint when using Kafka
Consumer
Key: FLINK-29094
URL: https://issues.apache.org/jira/browse/FLINK-29094
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.15.1
Environment: Local Environment
* OS: Ubuntu 20.04
* Java: openjdk version "11.0.12" 2021-07-20
* Flink: 1.15.1 (clean instalation)
* Kafka: kafka_2.13-2.8.1
Reporter: Bartosz Mikulski
We want to gracefully stop a Python job reading from Kafka. However, running
`flink stop --savepointPath` causes an exception to be thrown and the job
fails. Find the exception in the
[Gist|https://gist.github.com/BartMiki/6eab470a49d8a935ed3e7e544c3f278f#file-exception-txt].
The type of savepoint also doesn't matter both canonical and native fail. This
Stack Overflow [answer|https://stackoverflow.com/a/68280329] also does not
work.
However, checkpoints work fine. This leads to weird scenarios when you want to
stop a Python job, stopping fails with an exception, and the job is restarted
from the latest checkpoint.
The setup was tested in fresh default Flink 1.15.1 installation (however we
also tried 1.15.0 with the same error). The error occurred regardless of the
amount of data processed from Kafka (could be even none). [Gist with minimal
code required to trigger the
exception|https://gist.github.com/BartMiki/6eab470a49d8a935ed3e7e544c3f278f].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
