Hi Aditya, Thanks for your comments.
AK1: I also see that users stuff all manner of things into the client.id config. Also, some implementations overload the client.id to tunnel configurations into their clusters, such as availability zone information and so on. Transmitting this stuff on every single RPC seems unnecessary to me. I think that a future KIP ought to enable key-value pairs separate from client.id, and I'd rather not move to overloading client instance ID too. Also, KIP-714 introduced client instance ID as a UUID and the client telemetry infrastructure depends upon this. AK2: Yes, precisely. AK3: You are correct that the various clientInstanceId(Duration) methods no longer need the duration. Thanks for pointing this out. I'll update the KIP accordingly. It would also be possible to update the GetTelemetrySubscriptions and PushTelemetry RPCs to reflect the new way that client instance ID is handled. Thanks, Andrew On 2026/04/21 06:21:42 Aditya Kousik wrote: > Hi Andrew, Thanks for the KIP. I feel it will be very beneficial to all > users. While the crux of the KIP concerns adding clientInstanceId to headers > (and where the uuid is generated), I'd like to propose a slightly larger > scope as part of this plan. AK1: To the point you’ve raised in the KIP, it’s > fairly common for teams to stuff part of their infra into the client.id : > I’ve seen a hyphenated concat of > producer-applicationName-instanceId-podSHA-IP-hostname-UUID. On the flip > side, client.id being optional means people do not set it and end up with > unhelpful qualifiers like producer-1 or consumer-2. AK1_1: To that end, I > propose a format that moves away from a plain uuid. clientInstanceId can be a > single string field of the format prefix@uuid. The prefix would be client > supplied with a client.instance.id.prefix commonconfig. To be honest the > prefix alone would be considered client.id by another name but really the key > difference is that clientInstanceId will always contain the unique id across > each client/application instance making it genuinely useful on each RPC > request whereas a plain uuid will always require more lookup to breakdown > where the root of the issue lies. It will be self-contained in a log line > with both helpful metatadata as well as the client uniqueness. AK1_2: There's > also the documentation intent. Users have been stuffing infra metadata into > the client.id for simple lack of a better place, in my opinion. > client.instance.id.prefix gives them a blessed approach (and a migration > plan) from the client to place it which offering a cleanup process of the > client.id string. AK2: One of the critical changes you've proposed is that > the broker is no longer the source of truth for the uuid, it's the client. I > think it's the right direction for essentially what's 128bits generated over > the wire before any RPC requests can actually be made. AK3: There are some > API implications because of this: clients currently provide a Uuid > clientInstanceId(Duration timeout); I think the timeout will no longer be > required since it's generated client-side. So a new no-arg clientInstanceId() > would suffice here. AK3_1: With a String clientInstanceId() method, we can > move away from Uuid and deprecate that method. We can still support the uuid > by picking the last bytes of the client.instance.id format until AK5.0 > Thanks, Aditya On 20 Apr 2026, at 13:22, Andrew Schofield > <[email protected]> wrote: Hi Kirk, Thanks for your feedback. 1. The > clientInstanceId is just a UUID. If client ID is non-null, it's a string and > could be quite chunky. I do accept that the change to client ID in this KIP > is probably the most contentious part because it's not necessary for the > principal intent of this KIP. 2. In Kafka Streams, each of the embedded > clients (producer, consumer, admin) will have its own clientInstanceId. There > is no separate clientInstanceId for the "logical" client which pulls them > together. There could be, but that's not currently part of the KIP. 3. > Because clientInstanceId is a tagged field in v2 of the request header, it > applies to the latest versions of almost all of the RPCs. If you look at > o.a.k.common.message.ApiMessageType.requestHeaderVersion(short), you'll see > the mapping. The alternative of defining v3 of the RequestHeader would have > necessitated a version bump of every RPC we wanted to have the new header > version. Thanks, Andrew On 2026/04/17 23:50:50 Kirk True wrote: Thanks for > the KIP Andrew! Some questions: 1. Why is it preferable to send the client ID > only on the initial request but the client instance ID on every request? 2. > Are "logical" clients (e.g. Kafka Streams) also assigned a client instance > ID? If so, how does its client instance ID relate to the embedded clients’ > instance IDs? 3. Is there any advice on "adoption" of the new client instance > ID on the broker side? I could see a situation where some RPCs end up using > the client instance ID, but not others. Thanks, Kirk On Fri, Apr 3, 2026, at > 9:17 AM, Andrew Schofield wrote: > Hi, > I would like to start the discussion > on KIP-1313. This adds a unique client instance ID to the request header of > all Kafka protocol requests to give a unique identifier which can be used to > correlate the requests from each client for the purposes of problem > determination. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313%3A+Client+instance+ID+in+all+request+headers > > > Thanks, > Andrew >
