Randall Hauch created KAFKA-3815:
------------------------------------
Summary: Support command line arguments in Kafka Connect
distribute worker
Key: KAFKA-3815
URL: https://issues.apache.org/jira/browse/KAFKA-3815
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Affects Versions: 0.9.0.1
Reporter: Randall Hauch
Assignee: Ewen Cheslack-Postava
Change the Kafka Connect distribute worker so that one connector could be
configured via the command line. This would make it much easier to define
immutable containers (e.g., Docker, Kubernetes), where each container runs a
single distributed worker with a single configured connector. A "force" flag
might specify whether any existing configuration could be overwritten by the
configuration passed via the command line.
In fact, distributed environments that run immutable containers, especially
Kubernetes and OpenShift, would benefit greatly from being able to run each
Kafka Connect connector in one or more containers that are configured exactly
the same way and running as a single Kafka Connect group. Because the Kafka
Connect group has only a single configured connector, the group experiences no
unnecessary rebalances that would normally occur in other topologies with
multiple connectors deployed to one Kafka Connect group.
Ideally, the distributed worker could also be run in read-only mode so that the
connector configuration cannot be changed via the REST API. This would only
help to reinforce the connector as being immutable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
