vamossagar12 commented on PR #14101: URL: https://github.com/apache/kafka/pull/14101#issuecomment-1651871848
hey @akitoshka thanks for the PR. One thing that you have rightly pointed out is that `stopAndAwaitTasks` doesn't directly trigger a task stoppage. The restart request is handled by `DistributedHerder` in it's tick thread [here](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L472). This will trigger the task stop [here](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L1475) and this will ask the task threads a signal to shutdown by setting the stopping [flag to true](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerTask.java#L114). Note that the herder thread is a different one from the task thread thereby making the process asynchronous. And all the tasks need to shutdown within 10s in your case or else we will see messages like `Gra ceful Shutdown failed` Now the source tasks will , in the start of execute loop figure out that they have been asked to [stop](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L340C26-L340C26) and it exits from the loop. This will force a close to be triggered [here](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerTask.java#L216) which eventually leads to the closure of task [here](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L314) You might already know all this haha but just thought i will re-iterate the flow here. At this point I am not totally sure if we should add an explicit task stop here because the framework does handle invoking the task stop. Maybe in your case, increasing with the timeouts might help? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
