Mason Chen created FLINK-24622:
----------------------------------
Summary: Unified sources do not close scheduled threads from
SplitEnumeratorContext#callAsync()
Key: FLINK-24622
URL: https://issues.apache.org/jira/browse/FLINK-24622
Project: Flink
Issue Type: Bug
Components: Connectors / Common
Affects Versions: 1.13.2
Reporter: Mason Chen
>From user mailing list:
I was wondering how to cancel a task that is enqueued by the callAsync()
method, the one that takes in a time interval. For example, the KafkaSource
uses this for topic partition discovery. It would be straightforward if the API
returned the underlying future so that a process can cancel it.
For Kafka, the enumerator shutdown seems to be unclean since it only closes the
admin client and kafka consumer but not the topic partition discovery task.
Furthermore, exceptions from that task will cause job failure and can
potentially happen if the task is still running with the admin client closed.
How can we address this?
This seems to be a bug with the current KafkaSource and also the unified
Sources in general. Can you open a bug ticket in jira? I think the enumerator
should take of first joining all the async threads before closing the
enumerator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
