akitoshka opened a new pull request, #14101: URL: https://github.com/apache/kafka/pull/14101
We face a problem with restarting of the tasks, sometimes it leads to resource leak. We used the jdbc source connector and noticed an increasing of count of opened sessions on Vertica side. But this problem is applicable for all databases and possibly for all source connectors. Our case is the next: 1) Run jdbc source connector (io.confluent.connect.jdbc.JdbcSourceConnector) and set poll.interval.ms (86400000) > task.shutdown.graceful.timeout.ms (it's the property on Kafka-connect side, we set 10000) 2) Send POST /connectors/<connector_name>/tasks/<task_number>/restart ER: count of session is the same as before restart AR: count of session increases The main problem is when org.apache.kafka.connect.runtime.Worker#stopAndAwaitTasks(java.util.Collection<org.apache.kafka.connect.util.ConnectorTaskId>) method is called it doesn't stop a source task itself. The source task stops only if polling process stops on source task side. But if polling is continuing for a long time it leads to resource occupation as a result it can lead to the fail on DB side. We suggest to stop source task explicitly ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
