Thanks Andrew for AK2,3. Re: AK1, I’d like to implement it as the follow-up KIP after your KIP-1313 has landed. I have a rough sketch in mind that might address the issue cleanly.
Thanks, Aditya > On Apr 21, 2026, at 00:47, Andrew Schofield <[email protected]> wrote: > > > Hi Aditya, > Thanks for your comments. > > AK1: I also see that users stuff all manner of things into the client.id > config. Also, some implementations overload the client.id to tunnel > configurations into their clusters, such as availability zone information and > so on. Transmitting this stuff on every single RPC seems unnecessary to me. I > think that a future KIP ought to enable key-value pairs separate from > client.id, and I'd rather not move to overloading client instance ID too. > > Also, KIP-714 introduced client instance ID as a UUID and the client > telemetry infrastructure depends upon this. > > AK2: Yes, precisely. > > AK3: You are correct that the various clientInstanceId(Duration) methods no > longer need the duration. Thanks for pointing this out. I'll update the KIP > accordingly. > > It would also be possible to update the GetTelemetrySubscriptions and > PushTelemetry RPCs to reflect the new way that client instance ID is handled. > > Thanks, > Andrew > >> On 2026/04/21 06:21:42 Aditya Kousik wrote: >> Hi Andrew, Thanks for the KIP. I feel it will be very beneficial to all >> users. While the crux of the KIP concerns adding clientInstanceId to headers >> (and where the uuid is generated), I'd like to propose a slightly larger >> scope as part of this plan. AK1: To the point you’ve raised in the KIP, it’s >> fairly common for teams to stuff part of their infra into the client.id : >> I’ve seen a hyphenated concat of >> producer-applicationName-instanceId-podSHA-IP-hostname-UUID. On the flip >> side, client.id being optional means people do not set it and end up with >> unhelpful qualifiers like producer-1 or consumer-2. AK1_1: To that end, I >> propose a format that moves away from a plain uuid. clientInstanceId can be >> a single string field of the format prefix@uuid. The prefix would be client >> supplied with a client.instance.id.prefix commonconfig. To be honest the >> prefix alone would be considered client.id by another name but really the >> key difference is that clientInstanceId will always contain the unique id >> across each client/application instance making it genuinely useful on each >> RPC request whereas a plain uuid will always require more lookup to >> breakdown where the root of the issue lies. It will be self-contained in a >> log line with both helpful metatadata as well as the client uniqueness. >> AK1_2: There's also the documentation intent. Users have been stuffing infra >> metadata into the client.id for simple lack of a better place, in my >> opinion. client.instance.id.prefix gives them a blessed approach (and a >> migration plan) from the client to place it which offering a cleanup process >> of the client.id string. AK2: One of the critical changes you've proposed is >> that the broker is no longer the source of truth for the uuid, it's the >> client. I think it's the right direction for essentially what's 128bits >> generated over the wire before any RPC requests can actually be made. AK3: >> There are some API implications because of this: clients currently provide a >> Uuid clientInstanceId(Duration timeout); I think the timeout will no longer >> be required since it's generated client-side. So a new no-arg >> clientInstanceId() would suffice here. AK3_1: With a String >> clientInstanceId() method, we can move away from Uuid and deprecate that >> method. We can still support the uuid by picking the last bytes of the >> client.instance.id format until AK5.0 Thanks, Aditya On 20 Apr 2026, at >> 13:22, Andrew Schofield <[email protected]> wrote: Hi Kirk, Thanks for >> your feedback. 1. The clientInstanceId is just a UUID. If client ID is >> non-null, it's a string and could be quite chunky. I do accept that the >> change to client ID in this KIP is probably the most contentious part >> because it's not necessary for the principal intent of this KIP. 2. In Kafka >> Streams, each of the embedded clients (producer, consumer, admin) will have >> its own clientInstanceId. There is no separate clientInstanceId for the >> "logical" client which pulls them together. There could be, but that's not >> currently part of the KIP. 3. Because clientInstanceId is a tagged field in >> v2 of the request header, it applies to the latest versions of almost all of >> the RPCs. If you look at >> o.a.k.common.message.ApiMessageType.requestHeaderVersion(short), you'll see >> the mapping. The alternative of defining v3 of the RequestHeader would have >> necessitated a version bump of every RPC we wanted to have the new header >> version. Thanks, Andrew On 2026/04/17 23:50:50 Kirk True wrote: Thanks for >> the KIP Andrew! Some questions: 1. Why is it preferable to send the client >> ID only on the initial request but the client instance ID on every request? >> 2. Are "logical" clients (e.g. Kafka Streams) also assigned a client >> instance ID? If so, how does its client instance ID relate to the embedded >> clients’ instance IDs? 3. Is there any advice on "adoption" of the new >> client instance ID on the broker side? I could see a situation where some >> RPCs end up using the client instance ID, but not others. Thanks, Kirk On >> Fri, Apr 3, 2026, at 9:17 AM, Andrew Schofield wrote: > Hi, > I would like >> to start the discussion on KIP-1313. This adds a unique client instance ID >> to the request header of all Kafka protocol requests to give a unique >> identifier which can be used to correlate the requests from each client for >> the purposes of problem determination. > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313%3A+Client+instance+ID+in+all+request+headers >> > > Thanks, > Andrew >
