[
https://issues.apache.org/jira/browse/KAFKA-17601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884620#comment-17884620
]
Tamas Kornai edited comment on KAFKA-17601 at 9/25/24 1:52 PM:
---------------------------------------------------------------
I think there are two cases where the client name is not properly populated:
1. Kafka brokers fetching from each other:
{noformat}
request:{"isForwarded":false,"requestHeader":{"requestApiKey":1,"requestApiVersion":13,"correlationId":801,"clientId":"broker-1001-fetcher-0","requestApiKeyName":"FETCH"},"request":{"replicaId":1001,"maxWaitMs":500,"minBytes":1,"maxBytes":10485760,"isolationLevel":0,"sessionId":1346390899,"sessionEpoch":801,"topics":[{"topicId":"vF7yK6kvTMCk5OLzWANVCA","partitions":[{"partition":0,"currentLeaderEpoch":80,"fetchOffset":1818677,"lastFetchedEpoch":80,"logStartOffset":1327483,"partitionMaxBytes":134217728}]}],"forgottenTopicsData":[],"rackId":""},"response":{"throttleTimeMs":0,"errorCode":0,"sessionId":1346390899,"responses":[{"topicId":"2eCLABptRKq-x2x3gi9b5g","partitions":[{"partitionIndex":30,"errorCode":0,"highWatermark":105094597,"lastStableOffset":105094597,"logStartOffset":0,"abortedTransactions":null,"preferredReadReplica":-1,"recordsSizeInBytes":171}]},{"topicId":"vF7yK6kvTMCk5OLzWANVCA","partitions":[{"partitionIndex":0,"errorCode":0,"highWatermark":1818677,"lastStableOffset":1818677,"logStartOffset":1327483,"abortedTransactions":null,"preferredReadReplica":-1,"recordsSizeInBytes":0}]}]},"connection":"172.31.11.8:9095-172.31.12.30:41732-0","totalTimeMs":57.838,"requestQueueTimeMs":0.031,"localTimeMs":5.918,"remoteTimeMs":51.809,"throttleTimeMs":0,"responseQueueTimeMs":0.02,"sendTimeMs":0.058,"securityProtocol":"SSL","principal":"REDACTED","listener":"BROKER_SSL","clientInformation":{"softwareName":"unknown","softwareVersion":"unknown"}}
(kafka.request.logger){noformat}
2. Clients querying API version:
{noformat}
{"isForwarded":false,"requestHeader":{"requestApiKey":18,"requestApiVersion":3,"correlationId":6,"clientId":"adminclient-1","requestApiKeyName":"API_VERSIONS"},"request":{"clientSoftwareName":"apache-kafka-java","clientSoftwareVersion":"3.6.1"},"response":{"errorCode":0,"apiKeys":[{"apiKey":0,"minVersion":0,"maxVersion":9},{"apiKey":1,"minVersion":0,"maxVersion":15},{"apiKey":2,"minVersion":0,"maxVersion":8},{"apiKey":3,"minVersion":0,"maxVersion":12},{"apiKey":4,"minVersion":0,"maxVersion":7},{"apiKey":5,"minVersion":0,"maxVersion":4},{"apiKey":6,"minVersion":0,"maxVersion":8},{"apiKey":7,"minVersion":0,"maxVersion":3},{"apiKey":8,"minVersion":0,"maxVersion":8},{"apiKey":9,"minVersion":0,"maxVersion":8},{"apiKey":10,"minVersion":0,"maxVersion":4},{"apiKey":11,"minVersion":0,"maxVersion":9},{"apiKey":12,"minVersion":0,"maxVersion":4},{"apiKey":13,"minVersion":0,"maxVersion":5},{"apiKey":14,"minVersion":0,"maxVersion":5},{"apiKey":15,"minVersion":0,"maxVersion":5},{"apiKey":16,"minVersion":0,"maxVersion":4},{"apiKey":17,"minVersion":0,"maxVersion":1},{"apiKey":18,"minVersion":0,"maxVersion":3},{"apiKey":19,"minVersion":0,"maxVersion":7},{"apiKey":20,"minVersion":0,"maxVersion":6},{"apiKey":21,"minVersion":0,"maxVersion":2},{"apiKey":22,"minVersion":0,"maxVersion":4},{"apiKey":23,"minVersion":0,"maxVersion":4},{"apiKey":24,"minVersion":0,"maxVersion":4},{"apiKey":25,"minVersion":0,"maxVersion":3},{"apiKey":26,"minVersion":0,"maxVersion":3},{"apiKey":27,"minVersion":0,"maxVersion":1},{"apiKey":28,"minVersion":0,"maxVersion":3},{"apiKey":29,"minVersion":0,"maxVersion":3},{"apiKey":30,"minVersion":0,"maxVersion":3},{"apiKey":31,"minVersion":0,"maxVersion":3},{"apiKey":32,"minVersion":0,"maxVersion":4},{"apiKey":33,"minVersion":0,"maxVersion":2},{"apiKey":34,"minVersion":0,"maxVersion":2},{"apiKey":35,"minVersion":0,"maxVersion":4},{"apiKey":36,"minVersion":0,"maxVersion":2},{"apiKey":37,"minVersion":0,"maxVersion":3},{"apiKey":38,"minVersion":0,"maxVersion":3},{"apiKey":39,"minVersion":0,"maxVersion":2},{"apiKey":40,"minVersion":0,"maxVersion":2},{"apiKey":41,"minVersion":0,"maxVersion":3},{"apiKey":42,"minVersion":0,"maxVersion":2},{"apiKey":43,"minVersion":0,"maxVersion":2},{"apiKey":44,"minVersion":0,"maxVersion":1},{"apiKey":45,"minVersion":0,"maxVersion":0},{"apiKey":46,"minVersion":0,"maxVersion":0},{"apiKey":47,"minVersion":0,"maxVersion":0},{"apiKey":48,"minVersion":0,"maxVersion":1},{"apiKey":49,"minVersion":0,"maxVersion":1},{"apiKey":50,"minVersion":0,"maxVersion":0},{"apiKey":51,"minVersion":0,"maxVersion":0},{"apiKey":56,"minVersion":0,"maxVersion":3},{"apiKey":57,"minVersion":0,"maxVersion":1},{"apiKey":58,"minVersion":0,"maxVersion":0},{"apiKey":60,"minVersion":0,"maxVersion":0},{"apiKey":61,"minVersion":0,"maxVersion":0},{"apiKey":65,"minVersion":0,"maxVersion":0},{"apiKey":66,"minVersion":0,"maxVersion":0},{"apiKey":67,"minVersion":0,"maxVersion":0}],"throttleTimeMs":0,"finalizedFeaturesEpoch":1},"connection":"172.31.12.30:9093-172.31.12.30:48832-0","totalTimeMs":2.107,"requestQueueTimeMs":0.3,"localTimeMs":0.043,"remoteTimeMs":1.467,"throttleTimeMs":0,"responseQueueTimeMs":0.065,"sendTimeMs":0.23,"securityProtocol":"SSL","principal":"REDACTED","listener":"SSL","clientInformation":{"softwareName":"unknown","softwareVersion":"unknown"}}
(kafka.request.logger){noformat}
was (Author: tkornai):
I think there are two cases where the client name is not properly populated:
1. Kafka brokers fetching from each other:
{noformat}
request:{"isForwarded":false,"requestHeader":{"requestApiKey":1,"requestApiVersion":13,"correlationId":801,"clientId":"broker-1001-fetcher-0","requestApiKeyName":"FETCH"},"request":{"replicaId":1001,"maxWaitMs":500,"minBytes":1,"maxBytes":10485760,"isolationLevel":0,"sessionId":1346390899,"sessionEpoch":801,"topics":[{"topicId":"vF7yK6kvTMCk5OLzWANVCA","partitions":[{"partition":0,"currentLeaderEpoch":80,"fetchOffset":1818677,"lastFetchedEpoch":80,"logStartOffset":1327483,"partitionMaxBytes":134217728}]}],"forgottenTopicsData":[],"rackId":""},"response":{"throttleTimeMs":0,"errorCode":0,"sessionId":1346390899,"responses":[{"topicId":"2eCLABptRKq-x2x3gi9b5g","partitions":[{"partitionIndex":30,"errorCode":0,"highWatermark":105094597,"lastStableOffset":105094597,"logStartOffset":0,"abortedTransactions":null,"preferredReadReplica":-1,"recordsSizeInBytes":171}]},{"topicId":"vF7yK6kvTMCk5OLzWANVCA","partitions":[{"partitionIndex":0,"errorCode":0,"highWatermark":1818677,"lastStableOffset":1818677,"logStartOffset":1327483,"abortedTransactions":null,"preferredReadReplica":-1,"recordsSizeInBytes":0}]}]},"connection":"172.31.11.8:9095-172.31.12.30:41732-0","totalTimeMs":57.838,"requestQueueTimeMs":0.031,"localTimeMs":5.918,"remoteTimeMs":51.809,"throttleTimeMs":0,"responseQueueTimeMs":0.02,"sendTimeMs":0.058,"securityProtocol":"SSL","principal":"REDACTED","listener":"BROKER_SSL","clientInformation":{"softwareName":"unknown","softwareVersion":"unknown"}}
(kafka.request.logger){noformat}
2. Clients querying API version:
{noformat}
[2024-09-24 14:52:19,547] DEBUG Completed
request:{"isForwarded":false,"requestHeader":{"requestApiKey":18,"requestApiVersion":3,"correlationId":6,"clientId":"adminclient-1","requestApiKeyName":"API_VERSIONS"},"request":{"clientSoftwareName":"apache-kafka-java","clientSoftwareVersion":"3.6.1"},"response":{"errorCode":0,"apiKeys":[{"apiKey":0,"minVersion":0,"maxVersion":9},{"apiKey":1,"minVersion":0,"maxVersion":15},{"apiKey":2,"minVersion":0,"maxVersion":8},{"apiKey":3,"minVersion":0,"maxVersion":12},{"apiKey":4,"minVersion":0,"maxVersion":7},{"apiKey":5,"minVersion":0,"maxVersion":4},{"apiKey":6,"minVersion":0,"maxVersion":8},{"apiKey":7,"minVersion":0,"maxVersion":3},{"apiKey":8,"minVersion":0,"maxVersion":8},{"apiKey":9,"minVersion":0,"maxVersion":8},{"apiKey":10,"minVersion":0,"maxVersion":4},{"apiKey":11,"minVersion":0,"maxVersion":9},{"apiKey":12,"minVersion":0,"maxVersion":4},{"apiKey":13,"minVersion":0,"maxVersion":5},{"apiKey":14,"minVersion":0,"maxVersion":5},{"apiKey":15,"minVersion":0,"maxVersion":5},{"apiKey":16,"minVersion":0,"maxVersion":4},{"apiKey":17,"minVersion":0,"maxVersion":1},{"apiKey":18,"minVersion":0,"maxVersion":3},{"apiKey":19,"minVersion":0,"maxVersion":7},{"apiKey":20,"minVersion":0,"maxVersion":6},{"apiKey":21,"minVersion":0,"maxVersion":2},{"apiKey":22,"minVersion":0,"maxVersion":4},{"apiKey":23,"minVersion":0,"maxVersion":4},{"apiKey":24,"minVersion":0,"maxVersion":4},{"apiKey":25,"minVersion":0,"maxVersion":3},{"apiKey":26,"minVersion":0,"maxVersion":3},{"apiKey":27,"minVersion":0,"maxVersion":1},{"apiKey":28,"minVersion":0,"maxVersion":3},{"apiKey":29,"minVersion":0,"maxVersion":3},{"apiKey":30,"minVersion":0,"maxVersion":3},{"apiKey":31,"minVersion":0,"maxVersion":3},{"apiKey":32,"minVersion":0,"maxVersion":4},{"apiKey":33,"minVersion":0,"maxVersion":2},{"apiKey":34,"minVersion":0,"maxVersion":2},{"apiKey":35,"minVersion":0,"maxVersion":4},{"apiKey":36,"minVersion":0,"maxVersion":2},{"apiKey":37,"minVersion":0,"maxVersion":3},{"apiKey":38,"minVersion":0,"maxVersion":3},{"apiKey":39,"minVersion":0,"maxVersion":2},{"apiKey":40,"minVersion":0,"maxVersion":2},{"apiKey":41,"minVersion":0,"maxVersion":3},{"apiKey":42,"minVersion":0,"maxVersion":2},{"apiKey":43,"minVersion":0,"maxVersion":2},{"apiKey":44,"minVersion":0,"maxVersion":1},{"apiKey":45,"minVersion":0,"maxVersion":0},{"apiKey":46,"minVersion":0,"maxVersion":0},{"apiKey":47,"minVersion":0,"maxVersion":0},{"apiKey":48,"minVersion":0,"maxVersion":1},{"apiKey":49,"minVersion":0,"maxVersion":1},{"apiKey":50,"minVersion":0,"maxVersion":0},{"apiKey":51,"minVersion":0,"maxVersion":0},{"apiKey":56,"minVersion":0,"maxVersion":3},{"apiKey":57,"minVersion":0,"maxVersion":1},{"apiKey":58,"minVersion":0,"maxVersion":0},{"apiKey":60,"minVersion":0,"maxVersion":0},{"apiKey":61,"minVersion":0,"maxVersion":0},{"apiKey":65,"minVersion":0,"maxVersion":0},{"apiKey":66,"minVersion":0,"maxVersion":0},{"apiKey":67,"minVersion":0,"maxVersion":0}],"throttleTimeMs":0,"finalizedFeaturesEpoch":1},"connection":"172.31.12.30:9093-172.31.12.30:48832-0","totalTimeMs":2.107,"requestQueueTimeMs":0.3,"localTimeMs":0.043,"remoteTimeMs":1.467,"throttleTimeMs":0,"responseQueueTimeMs":0.065,"sendTimeMs":0.23,"securityProtocol":"SSL","principal":"REDACTED","listener":"SSL","clientInformation":{"softwareName":"unknown","softwareVersion":"unknown"}}
(kafka.request.logger){noformat}
> Inter-broker connections do not expose their clientSoftwareName and
> clientSoftwareVersion tags
> ----------------------------------------------------------------------------------------------
>
> Key: KAFKA-17601
> URL: https://issues.apache.org/jira/browse/KAFKA-17601
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 3.6.1
> Reporter: Tamas Kornai
> Priority: Minor
>
> [KIP-511|[https://cwiki.apache.org/confluence/display/KAFKA/KIP-511%3A+Collect+and+Expose+Client%27s+Name+and+Version+in+the+Brokers]]
> made it possible to see what library versions are used by the Kafka clients.
> When Kafka brokers are connecting to other brokers this information is not
> properly populated, we see the "unknown" value instead for both
> `ClientSoftwareName` and `ClientSoftwareVersion`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
