Per Steffensen created KAFKA-5505:
-------------------------------------
Summary: Connect: Do not restart connector and existing tasks on
task-set change
Key: KAFKA-5505
URL: https://issues.apache.org/jira/browse/KAFKA-5505
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Affects Versions: 0.10.2.1
Reporter: Per Steffensen
I am writing a connector with a frequently changing task-set. It is really not
working very well, because the connector and all existing tasks are restarted
when the set of tasks changes. E.g. if the connector is running with 10 tasks,
and an additional task is needed, the connector itself and all 10 existing
tasks are restarted, just to make the 11th task run also. My tasks have a
fairly heavy initialization, making it extra annoying. I would like to see a
change, introducing a "mode", where only new/deleted tasks are started/stopped
when notifying the system that the set of tasks changed (calling
context.requestTaskReconfiguration() - or something similar).
Discussed this issue a little on dev@kafka.apache.org in the thread "Kafka
Connect: To much restarting with a SourceConnector with dynamic set of tasks"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
