Hi, Andrew, Thanks for the reply.
JR14. Consider the case that a new client connects to an old broker. The new client generates a new clientInstanceId (uuidA) and includes it in the initial request. From the ApiVersionResponse, the client realizes that the broker only supports GetTelemetrySubscriptions V0. It issues that request and gets a different clientInstanceId (uuidB) in the GetTelemetrySubscriptions V0 response. What clientInstanceId should the client use in subsequent requests? If the client uses uuidA, that ID won't match the broker's registration for this client. Using uuidB violates the assumption that clientInstanceId is immutable and may also impact memberId. Jun On Tue, May 5, 2026 at 12:22 PM Andrew Schofield <[email protected]> wrote: > Hi Jun, > Thanks for the reply. > > JR7: I'm guessing at a detail of the implementation but I think it would > be reasonable to call a method on ChannelMetadataRegistry on every request > with a V2 header that sets or checks the clientInstanceId. > > JR13: Thanks. Updated. > > Thanks, > Andrew > > On 2026/05/05 19:06:52 Jun Rao via dev wrote: > > Hi, Andrew, > > > > Thanks for the reply. > > > > JR7. ChannelMetadataRegistry.registerClientInformation() is currently > only > > called on ApiVersionRequest. When registering clientInstanceId in > > ChannelMetadataRegistry, do we plan to do that on every request with V2 > > header? > > > > JR13. GetTelemetrySubscriptions has the following, which seems outdated. > > "After this KIP, the Apache Kafka Java client will send the > > ClientInstanceId in the request header and also in the request body" > > > > Jun > > > > On Tue, May 5, 2026 at 10:33 AM Andrew Schofield <[email protected]> > > wrote: > > > > > Hi Jun, > > > Thanks for your comments. > > > > > > JR5: This is a very interesting one. I did have some nervousness about > > > changing client ID without RPC version bumps, and you've brought up an > > > excellent example. > > > > > > I have removed the change to the client ID behavior from this KIP. > While I > > > still think it would be beneficial to stop sending the client ID > strings > > > repeatedly, it needs to be done properly. That was not the main > purpose of > > > this KIP, so I think it's best to pull back on this part. > > > > > > JR6: Done. > > > > > > JR7: I wonder why the authentication needs to have completed before the > > > client instance ID would be cached. The client instance ID is > generated by > > > the client and the client should use the same value for all of its > > > requests, and the broker will check that it has. > > > > > > JR8: As mentioned in my response to JR5, I've removed the client ID > > > changes. > > > > > > JR9: Done. > > > > > > JR10: I've clarified that the checking only applies to requests that > use > > > the v2 header. > > > > > > JR11: I've added v1 for GetTelemetrySubscriptions and PushTelemetry in > > > this KIP. > > > > > > JR12: Done. > > > > > > Thanks, > > > Andrew > > > > > > On 2026/05/05 00:04:13 Jun Rao via dev wrote: > > > > Hi, Andrew, > > > > > > > > Thanks for the updated KP. A few more comments. > > > > > > > > JR5. It seems that this design could break the clientId based quota > if a > > > > new client communicates with an old broker. The KIP proposes sending > > > > ClientId only on the first request, then null for all subsequent > > > requests, > > > > without bumping up the request version. So, the old server will > receive > > > the > > > > first request and ignore ClientId. The client doesn't know the server > > > > ignores the clientId and sends a null clientId to an old broker for > > > > subsequent requests. The broker will enforce quotas against a null > > > > clientId, which is incorrect. > > > > > > > > JR6. Motivation: It would be useful to motivate the benefit of > including > > > > clientInstanceID on every request. For example, one immediate > benefit is > > > > that it can be included in request logging for troubleshooting. > > > > > > > > JR7. "The initial client instance ID for each connection will be > cached > > > by > > > > the broker for checking" > > > > It would be useful to clarify that the caching occurs only after > > > > authentication completes. Some requests can be issued before > > > authentication > > > > completes. > > > > > > > > JR8. "This KIP also proposes a second change to the Kafka protocol. > It > > > > proposes sending the client ID only on the initial request on each > > > > connection, and then sending a null client ID for all subsequent > > > requests." > > > > Similar to the above, it would be useful to clarify that the > clientId is > > > > included on the first request after authentication completes. > > > > > > > > JR9. GetTelemetrySubscriptionsRequest > > > > "about": "Unique id for this client instance, must be set to 0 > on > > > the > > > > first request." > > > > Could we change the description above since clientInstanceId is no > > > longer 0 > > > > on first request? > > > > > > > > JR10. "Once a client has specified a client instance ID in the > request > > > > header of its first request, any subsequent requests which are > missing > > > the > > > > client instance ID or which specify a different value for the client > > > > instance ID will be rejected with error code INVALID_REQUEST." > > > > Hmm, does this work for a new AdminClient sending an > OffsetDeleteRequest > > > > that still uses a v1 request header? > > > > > > > > JR11. Since ClientInstanceId exists in the header, should we remove > > > > clientInstanceId from both GetTelemetrySubscriptions and > PushTelemetry? > > > > > > > > JR12. Could we add the description in RequestHeader to explain the > > > > difference between ClientId and ClientInstanceId? Historically, > clientId > > > is > > > > meant for identifying an application, not an instance of an > application. > > > > > > > > Jun > > > > > > > > On Thu, Apr 30, 2026 at 3:00 AM Andrew Schofield < > [email protected]> > > > > wrote: > > > > > > > > > Hi Apoorv, > > > > > Thanks for your suggestion. > > > > > > > > > > AM1: That's a great idea. Done. > > > > > > > > > > Thanks, > > > > > Andrew > > > > > > > > > > On 2026/04/27 12:24:12 Apoorv Mittal wrote: > > > > > > Hi Andrew, > > > > > > Thanks for the KIP. I have one question: > > > > > > > > > > > > AM1: With KIP-1082, the consumer now generates its own member ID > at > > > > > > construction time. With KIP-1313, the client will generate its > own > > > client > > > > > > instance ID as a Uuid at construction time. Both are > client-generated > > > > > > UUIDs, immediately available at startup, and remain constant for > the > > > > > > process's lifetime. For a consumer (or share consumer), these > two IDs > > > > > serve > > > > > > overlapping identification purposes at the same scope: > identifying > > > this > > > > > > particular client incarnation. The ClientInstanceIds interface in > > > Streams > > > > > > already maintains a separate client instance ID per consumer per > > > thread, > > > > > so > > > > > > there is no cardinality mismatch even in multi-threaded Streams > > > > > > applications. Has unifying these been considered? The consumer > could > > > > > > generate one UUID at startup and use it both as its member ID in > > > > > heartbeat > > > > > > requests and as the client instance ID in request headers and > > > telemetry. > > > > > > > > > > > > Regards, > > > > > > Apoorv Mittal > > > > > > > > > > > > > > > > > > On Fri, Apr 24, 2026 at 5:48 PM Jun Rao via dev < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > Thanks for the reply. > > > > > > > > > > > > > > JR1. Caching the clientID from the first request in > > > > > ChannelMetadataRegistry > > > > > > > sounds good. We can then use it to fill the request header in > > > > > subsequent > > > > > > > requests. > > > > > > > > > > > > > > JR2.1 Ok. It's fine to include clientInstanceID in the request > > > header. > > > > > It > > > > > > > would be useful to document that it's only set on the first > > > request and > > > > > > > cached in ChannelMetadataRegistry. > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > On Fri, Apr 24, 2026 at 6:28 AM Andrew Schofield < > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > > > JR1: I was thinking of attaching it to some channel-based > > > context, > > > > > such > > > > > > > as > > > > > > > > the ChannelMetadataRegistry. I've not dug into the code in > > > detail at > > > > > this > > > > > > > > point. However, as mentioned in the KIP, removing the client > ID > > > from > > > > > all > > > > > > > > except the initial request is perhaps overreach in this KIP. > I > > > wonder > > > > > > > > whether actually this would be better handled in a major > release. > > > > > Let me > > > > > > > > know your thoughts here. > > > > > > > > > > > > > > > > JR2.1: I did consider put it in the ApiVersionsRequest, but > I do > > > > > want it > > > > > > > > in every request to help with problem diagnosis. Also, > > > ApiVersions is > > > > > > > not a > > > > > > > > mandatory part of the protocol, so we can't rely on it being > the > > > > > first > > > > > > > > request. > > > > > > > > > > > > > > > > JR2.2: Similar to JR1, I was thinking of hanging it off some > > > > > > > channel-based > > > > > > > > context. > > > > > > > > > > > > > > > > JR3: Indeed. I've added share consumer. > > > > > > > > > > > > > > > > JR4: Thinking about migration more, it seems to me that it is > > > best > > > > > not to > > > > > > > > introduce new versions of the PushTelemetry and > > > > > GetTelemetrySubscriptions > > > > > > > > RPCs. For an AK 4.4 Java client, we know that the client will > > > > > generate a > > > > > > > > client instance ID before its initial connection. For any > other > > > > > client, > > > > > > > we > > > > > > > > do not know that. Consequently, even in the broker supports a > > > newer > > > > > > > version > > > > > > > > of PushTelemetry which omits ClientInstanceId from the > request > > > > > schema, we > > > > > > > > would still need to support the broker-side ClientInstanceId > > > > > > > initialization > > > > > > > > in which the response contains the ID created by the broker. > I > > > think > > > > > that > > > > > > > > leaving the slightly redundant ClientInstanceId in the > KIP-714 > > > RPCs > > > > > is > > > > > > > > best. I've changed the text accordingly. Let me know what you > > > think. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Andrew > > > > > > > > > > > > > > > > On 2026/04/23 19:04:37 Jun Rao via dev wrote: > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > Thanks for the KIP. A few comments. > > > > > > > > > > > > > > > > > > JR1. Currently, you can configure a client quota based on > > > > > > > > > clientId. Currently, the broker extracts the clientId from > the > > > > > request > > > > > > > > > header. If clientId is not included in every request, can > we > > > > > document > > > > > > > > where > > > > > > > > > the server will obtain it? > > > > > > > > > > > > > > > > > > JR2. client instance ID: > > > > > > > > > JR2.1 Is it only sent on the first request in a > connection? If > > > so, > > > > > have > > > > > > > > we > > > > > > > > > considered making it part of the ApiVersionRequest instead > of > > > the > > > > > > > request > > > > > > > > > header? > > > > > > > > > JR2.2 It would be useful to include the client instance ID > in > > > > > request > > > > > > > > > logging on the server side. Could we document how to make > that > > > > > > > available > > > > > > > > > for each request the server processes? > > > > > > > > > > > > > > > > > > JR3. "The client instance ID will be calculated during the > > > > > constructor > > > > > > > of > > > > > > > > > the Producer , Consumer and Admin". > > > > > > > > > What about ShareConsumer? > > > > > > > > > > > > > > > > > > JR4. GetTelemetrySubscriptions: Could we document the > changes > > > in > > > > > the > > > > > > > > > response too? > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 3, 2026 at 9:18 AM Andrew Schofield < > > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > I would like to start the discussion on KIP-1313. This > adds a > > > > > unique > > > > > > > > > > client instance ID to the request header of all Kafka > > > protocol > > > > > > > > requests to > > > > > > > > > > give a unique identifier which can be used to correlate > the > > > > > requests > > > > > > > > from > > > > > > > > > > each client for the purposes of problem determination. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313*3A*Client*instance*ID*in*all*request*headers__;JSsrKysrKys!!Ayb5sqE7!txpES9zmXlCh8usguNsiJLRnSXvMtgWuM6R1rx8VxYYyi-sThQTmaOrq3ZfhArG25CjhHKuaLaJ63z_dzny8$ > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
