[
https://issues.apache.org/jira/browse/KAFKA-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178085#comment-16178085
]
ASF GitHub Bot commented on KAFKA-5061:
---------------------------------------
GitHub user Satyajitv opened a pull request:
https://github.com/apache/kafka/pull/3952
KAFKA-5061:clientid should be set for Connect Producers and Consumers
Have added the new config in ConnectorConfig with priority "LOW", providing
an option to override client.id on a per-connector basis.
And set client.id , using (worker group ID + task ID) by default, when
client.id is not provided in connector configuration, for distributed mode.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Satyajitv/kafka
KAFKA-5061_clientid_should_be_set_for_Connect_Producers_and_Consumers
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3952.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3952
----
commit d9e200fed258fb2f6b9bcb0a2b2c876490a223cc
Author: satyajit <[email protected]>
Date: 2017-09-24T02:54:01Z
KAFKA-5061:clientid should be set for Connect Producers and Consumers
----
> client.id should be set for Connect producers/consumers
> -------------------------------------------------------
>
> Key: KAFKA-5061
> URL: https://issues.apache.org/jira/browse/KAFKA-5061
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.10.2.1
> Reporter: Ewen Cheslack-Postava
> Labels: needs-kip, newbie++
>
> In order to properly monitor individual tasks using the producer and consumer
> metrics, we need to have the framework disambiguate them. Currently when we
> create producers
> (https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L362)
> and create consumers
> (https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L371-L394)
> the client ID is not being set. You can override it for the entire worker
> via worker-level producer/consumer overrides, but you can't get per-task
> metrics.
> There are a couple of things we might want to consider doing here:
> 1. Provide default client IDs based on the worker group ID + task ID
> (providing uniqueness for multiple connect clusters up to the scope of the
> Kafka cluster they are operating on). This seems ideal since it's a good
> default; however it is a public-facing change and may need a KIP. Normally I
> would be less worried about this, but some folks may be relying on picking up
> metrics without this being set, in which case such a change would break their
> monitoring.
> 2. Allow overriding client.id on a per-connector basis. I'm not sure if this
> will really be useful or not -- it lets you differentiate between metrics for
> different connectors' tasks, but within a connector, all metrics would go to
> a single client.id. On the other hand, this makes the tasks act as a single
> group from the perspective of broker handling of client IDs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
