[
https://issues.apache.org/jira/browse/KAFKA-14838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Urban updated KAFKA-14838:
---------------------------------
Description:
MM2 code creates a lot of Kafka clients internally. These clients generate a
lot of logs, but since the client.id is not properly specified, connecting the
dots between a specific Connector/Task and its internal client is close to
impossible. This is even more complex when MM2 is running in distributed mode,
in which multiple Connect workers are running inside the same process.
For Connector/Task created clients, the client.id clients should specify the
flow, the Connector name/Task ID and the role of the client. E.g.
MirrorSourceConnector uses multiple admin clients, and their client.id should
reflect the difference between them.
For Worker created clients, the client.id should refer to the flow.
This will help log analysis significantly, especially in MM2 mode.
was:
MM2 code creates a lot of Kafka clients internally. These clients generate a
lot of logs, but since the client.id is not properly specified, connecting the
dots between a specific Connector/Task and its internal client is close to
impossible.
The client.id of such clients should specify the Connector name/Task ID, and
should also specify the role of the client. E.g. MirrorSourceConnector uses
multiple admin clients, and their client.id should reflect the difference
between them. This will help log analysis significantly, especially in MM2 mode.
> MM2 Worker/Connector/Task clients should specify client ID based on flow and
> role
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-14838
> URL: https://issues.apache.org/jira/browse/KAFKA-14838
> Project: Kafka
> Issue Type: Improvement
> Reporter: Daniel Urban
> Assignee: Daniel Urban
> Priority: Major
>
> MM2 code creates a lot of Kafka clients internally. These clients generate a
> lot of logs, but since the client.id is not properly specified, connecting
> the dots between a specific Connector/Task and its internal client is close
> to impossible. This is even more complex when MM2 is running in distributed
> mode, in which multiple Connect workers are running inside the same process.
> For Connector/Task created clients, the client.id clients should specify the
> flow, the Connector name/Task ID and the role of the client. E.g.
> MirrorSourceConnector uses multiple admin clients, and their client.id should
> reflect the difference between them.
> For Worker created clients, the client.id should refer to the flow.
> This will help log analysis significantly, especially in MM2 mode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
