[
https://issues.apache.org/jira/browse/KAFKA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Egerton updated KAFKA-14725:
----------------------------------
Summary: Improve cancellation semantics for connector tasks (was: Improve
cancellation semantics for source tasks)
> Improve cancellation semantics for connector tasks
> --------------------------------------------------
>
> Key: KAFKA-14725
> URL: https://issues.apache.org/jira/browse/KAFKA-14725
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Chris Egerton
> Priority: Major
>
> This came about during discussion on
> [https://github.com/apache/kafka/pull/13208/,] which addressed KAFKA-5756.
>
> Right now, we make some effort to disable and shut down tasks that have been
> scheduled for shutdown but taken longer than the [graceful shutdown timeout
> period|https://kafka.apache.org/documentation.html#connectconfigs_task.shutdown.graceful.timeout.ms].
> The logic for performing this disablement is contained in the {{cancel}}
> method for the
> [WorkerTask|https://github.com/apache/kafka/blob/b9754747d6eaa029c4bb69b073d749ff8df15908/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerTask.java#L129-L136]
> and its subclasses (at the time of writing, that would be the
> [AbstractWorkerSourceTask|https://github.com/apache/kafka/blob/b9754747d6eaa029c4bb69b073d749ff8df15908/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java],
>
> [WorkerSourceTask|https://github.com/apache/kafka/blob/b9754747d6eaa029c4bb69b073d749ff8df15908/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java],
>
> [ExactlyOnceWorkerSourceTask|https://github.com/apache/kafka/blob/b9754747d6eaa029c4bb69b073d749ff8df15908/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTask.java],
> and
> [WorkerSinkTask|https://github.com/apache/kafka/blob/b9754747d6eaa029c4bb69b073d749ff8df15908/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java]
> classes). Right now we don't do much to interrupt in-progress operations,
> which may lead to zombie tasks lying around on a worker that have not yet
> relinquished resources like Kafka clients, file descriptors, or database
> connections despite being scheduled for shutdown.
> We can and should make the cancellation logic for tasks more stringent,
> including but not limited to:
> * Interrupting the work thread for the task
> * Interrupting any in-progress offset commits
> * Preemptively shutting down any Kafka clients created for use by the task
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
