Hi Jun, Thanks for the reply. JR27: Yes, I enhanced that section of the KIP. JR27.1: This KIP was unusual because we wanted to add a concept which existed in the KIP-714 RPCs to the request header, complicating the behavior of those RPCs, AND we wanted to change the handling of client ID without breaking older brokers. If we simply wanted to add a brand new field to the request header, we can do it as a tagged field in request header versions 2+. JR27.2: Covered this too. We'd need a major version API compatibility KIP.
JR31: Yes. KIP updated. JR32: Yes. This is a minor inconvenience I feel. KIP updated. JR33: I don't see the value of a string comparison here. Personally, I'm happy with "assume". Thanks, Andrew On 2026/06/23 22:52:15 Jun Rao via dev wrote: > Hi, Andrew, > > Thanks for the reply. > > JR27. The code that changed the request header parsing was introduced in > 2019 (https://github.com/apache/kafka/pull/7372). I agree that fixing it > now is too late. We should document the path forward for ApiVersion. Could > we answer at least the following questions in the Compatibility, > Deprecation, and Migration Plan section? > JR27.1 What happens to ApiVersion if we change the request header in the > future? > JR27.2 What happens if we ever want to deprecate v2 of the request header? > > JR28. The new MV needs to be covered in the Compatibility, Deprecation, and > Migration Plan section. > > JR31. "If a connection uses the v3 request header in its first request, it > must specify a non-zero client instance ID and it must specify the same > client instance ID for all subsequent requests." > Do we do the same if the first request is v6 of ApiVersion? > > JR32. "For subsequent requests using an earlier version of the request > header, the client sends the client ID". This means that subsequent > ApiVersion V6 requests will carry the non-null clientID unnecessarily. > > JR33. "After this KIP, the broker will assume that the client ID from the > initial request applies to all subsequent requests on a connection, and it > will ignore the client ID specified on any subsequent requests." > Should the broker verify that the clientId from subsequent request match > the first cached one, if not null? > > Jun > > On Tue, Jun 23, 2026 at 6:05 AM Andrew Schofield <[email protected]> > wrote: > > > Hi Jun, > > Thanks for your reply. > > > > JR27: Unfortunately, I don't think this is acceptable. We would be saying > > that AK 4.5 clients can only connect to AK 4.5 or later brokers, and that > > we'd create patch release for AK 4.0.x to AK 4.4.x. so AK 4.5 clients could > > connect to those too. We'd be preventing connection to AK 3.x brokers by AK > > 4.5 clients. I don't see how we can break compatibility in a minor release. > > Here's the existing compatibility table: > > https://urldefense.com/v3/__https://kafka.apache.org/41/getting-started/compatibility/__;!!Ayb5sqE7!ppWU_jI9f7EY9DYwcbiOESSBG2TmPn9CHfLnVM5TaIMwWqHUbUbIiGre1GGKMr-HmelQlyP6ibUSyORCAgte$ > > > > JR29: OK, makes sense to me. > > > > Thanks, > > Andrew > > > > On 2026/06/22 21:50:43 Jun Rao via dev wrote: > > > Hi, Andrew, > > > > > > Thanks for the reply. > > > > > > JR27. I am not sure I like the approach of pinning the header to v2 for > > > ApiVersionsRequest. This means that in the future, each change to the > > > request header must be duplicated in the request itself. Here is an > > > alternative. Note that to detect an unsupported ApiVersionRequest, we > > don't > > > need to parse the full v2 request header. Parsing the first two fields > > > ApiKey and ApiVersion is enough. So, we could patch all the 4.x releases > > > with a change that short-circuits parsing the full request header based > > on > > > the first two fields. Then, we recommend upgrading from the latest 4.x > > > releases to future releases. This way, we can still let > > ApiVersionsRequest > > > depend on the request header for current and future changes. > > > > > > JR29. The simplest thing is to upgrade the KRaft RPCs too to be > > consistent. > > > > > > Jun > > > > > > On Sat, Jun 20, 2026 at 3:38 AM Andrew Schofield <[email protected]> > > > wrote: > > > > > > > Hi Jun, > > > > Thanks for these insightful comments. I did prototype the earlier v2 > > > > request header with tagged fields variant. I have now prototyped the v3 > > > > request header variant. > > > > > > > > JR27: You were of course entirely correct that a new version of > > > > ApiVersions request cannot use a request header which is not > > > > binary-compatible with what has gone before. As a result, I have > > introduced > > > > a new version of ApiVersions request which includes ClientInstanceId > > in the > > > > request body but continues to use the v2 request header. > > > > > > > > JR28: Yes, we need a new metadata version, which is provisionally > > > > IBP_4_5_IV0. I'll update the KIP with the final value when it is > > assigned. > > > > > > > > JR29: I don't have any strong feelings on this. I suppose for each > > inter > > > > broker RPC version that is bumped, we need to introduce an MV check. We > > > > cannot avoid this for RPCs which are also used by clients such as > > Fetch. > > > > For purely inter-broker RPCs, we could just exempt them from the > > version > > > > bump. What do you recommend here? > > > > > > > > JR30: Done. INVALID_REQUEST. I did consider permitting zero client > > > > instance ID for inter-broker connections, but I decided that I > > preferred > > > > not creating another rule. > > > > > > > > Thanks, > > > > Andrew > > > > > > > > On 2026/06/17 22:45:07 Andrew Schofield wrote: > > > > > Hi Jun, > > > > > Thanks for your response. > > > > > > > > > > JR27: I am familiar with KIP-511 and the oddness of ApiVersions. You > > > > make an excellent point. I suspect that ApiVersionsRequest will have > > to be > > > > limited to the v2 request header, just as it is restricted to the v0 > > > > response header. > > > > > > > > > > I'll do some experimentation. > > > > > > > > > > Thanks, > > > > > Andrew > > > > > > > > > > On 2026/06/17 17:35:17 Jun Rao via dev wrote: > > > > > > Hi, Andrew, > > > > > > > > > > > > Thanks for the reply. A few more comments. > > > > > > > > > > > > JR27. It seems that we have a compatibility issue with > > > > ApiVersionRequest v6 > > > > > > depending on RequestHeader v3. This happens when a new client > > issues > > > > > > ApiVersionRequest v6 to an old broker (one that only supports > > > > > > ApiVersionRequest v5). The SocketServer in the broker will first > > try to > > > > > > read the request header. It tries to find the header version from > > the > > > > > > request version. Since the broker doesn't know ApiVersionRequest > > v6, it > > > > > > uses v2 to parse the request header. > > > > > > > > > > > > The v2's wire format for the request header is: ApiKey | > > ApiVersion | > > > > > > CorrelationId | ClientId | tagged-fields-varint > > > > > > The v3's wire format for the request header is: ApiKey | > > ApiVersion | > > > > > > CorrelationId | ClientId | ClientInstanceId(16 bytes) | > > tagged-fields > > > > varint > > > > > > The first 4 fields are identical, allowing the broker to parse > > them. > > > > The > > > > > > broker then tries to read the tagged-fields-varint using the > > serialized > > > > > > bytes intended for ClientInstanceId. This could lead to a parsing > > > > error and > > > > > > raise an invalid request error. The broker will close the > > connection > > > > > > instead of sending a v0 response for ApiVersionRequest. Then the > > client > > > > > > will never be able to detect supported API versions on the old > > broker. > > > > > > > > > > > > The same problem didn't exist when ApiVersionRequest changed from > > the > > > > v1 > > > > > > request header to v2 because the wire format for v1 is a prefix of > > v2. > > > > This > > > > > > problem exists for this KIP because the wire format of v2 is no > > longer > > > > a > > > > > > prefix of v3. > > > > > > > > > > > > JR28. Since we are bumping up the version of inter broker > > requests, we > > > > need > > > > > > to gate them by introducing a new MV. > > > > > > > > > > > > JR29. Could you specify whether the version for KRaft PRCs will be > > > > bumped > > > > > > too? > > > > > > > > > > > > JR30. "If a connection uses the v3 request header in its first > > > > request, it > > > > > > must specify a non-zero client instance ID" > > > > > > Could we define the broker's behavior when a v3 request header > > > > contains a 0 > > > > > > client instance ID? > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 16, 2026 at 1:25 AM Andrew Schofield < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Jun, > > > > > > > Thanks for your response. > > > > > > > > > > > > > > JR26: Yes, I was considering open-ended version ranges. I've > > updated > > > > the > > > > > > > KIP. > > > > > > > > > > > > > > Now that we have a new request header version, I think it's safe > > to > > > > > > > reintroduce the client ID optimisation. I've added that into the > > KIP > > > > also. > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > Thanks, > > > > > > > Andrew > > > > > > > > > > > > > > On 2026/06/15 17:46:00 Jun Rao via dev wrote: > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > Thanks for the reply. Just a minor comment. > > > > > > > > > > > > > > > > JR26. "The keys of the map are either single versions like "0" > > or > > > > closed > > > > > > > > version ranges like "1-2"." > > > > > > > > It will be useful to support open-ended versions. That way, > > "3": > > > > "3" > > > > > > > could > > > > > > > > be "3+": "3". This part won't need to change in the future > > when we > > > > bump > > > > > > > up > > > > > > > > the request version without changing the header. > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > On Fri, Jun 12, 2026 at 2:12 PM Andrew Schofield < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > > Thanks for your response. > > > > > > > > > > > > > > > > > > JR25: I agree. I've changed to a headerVersions map in both > > the > > > > request > > > > > > > > > and response schemas. > > > > > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > On 2026/06/12 03:39:39 Jun Rao via dev wrote: > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > > > Thanks for the updated KIP. > > > > > > > > > > > > > > > > > > > > JR25. I have some concerns about adding the > > > > clientInstanceIdVersions > > > > > > > > > field > > > > > > > > > > to the schema. First of all, it only applies to requests, > > but > > > > not > > > > > > > > > > responses. So, it's a bit weird to add it to the general > > > > schema. > > > > > > > > > Secondly, > > > > > > > > > > it's very specific to the new field added to the request > > > > header. If > > > > > > > we > > > > > > > > > > change the request header again in the future, we need > > another > > > > way > > > > > > > to map > > > > > > > > > > request versions to the new header version. Ideally, we > > should > > > > have a > > > > > > > > > > generic way to map request versions to all future header > > > > versions. > > > > > > > One > > > > > > > > > > possibility is to introduce a headerVersion field in the > > > > schema. > > > > > > > Then, > > > > > > > > > each > > > > > > > > > > request/response can use this field to map its versions to > > the > > > > header > > > > > > > > > > versions. > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > On Tue, Jun 9, 2026 at 4:55 AM Andrew Schofield < > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Matthias, > > > > > > > > > > > Thanks for your response. > > > > > > > > > > > > > > > > > > > > > > I have change the version property for introducing client > > > > instance > > > > > > > ID > > > > > > > > > into > > > > > > > > > > > the request headers to be `clientInstanceIdVersions` > > which > > > > is now > > > > > > > an > > > > > > > > > > > open-ended range, such as "3+", more closely matching > > what > > > > we have > > > > > > > for > > > > > > > > > > > flexible versions. > > > > > > > > > > > > > > > > > > > > > > I also added a change to the > > > > RebalanceConsumer.clientInstanceId > > > > > > > method > > > > > > > > > > > introduced in KIP-1306. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > On 2026/06/03 18:31:06 "Matthias J. Sax" wrote: > > > > > > > > > > > > Thanks Andrew. > > > > > > > > > > > > > > > > > > > > > > > > From a KIP-714 perspective, this is a great change, > > as it > > > > make > > > > > > > the > > > > > > > > > new > > > > > > > > > > > > RPCs and client/broker cross-version compatibility > > clean. > > > > > > > > > > > > > > > > > > > > > > > > One question about your `DeleteGroups` RPC example. > > Should > > > > the > > > > > > > new > > > > > > > > > > > > version of the RPC say > > > > > > > > > > > > > > > > > > > > > > > > > "clientInstanceIdVersion": "3+", > > > > > > > > > > > > > > > > > > > > > > > > ie, 3+ instead of 3? Similar for other request, which > > > > don't use > > > > > > > `+` > > > > > > > > > on > > > > > > > > > > > > `clientInstanceIdVersion` field. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Matthias > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 6/2/26 7:42 AM, Andrew Schofield wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > Following on from the discussion in the community, > > and > > > > the > > > > > > > > > difficulty > > > > > > > > > > > of trying to accommodate the client instance ID in a > > tagged > > > > field > > > > > > > with > > > > > > > > > > > acceptable complexity, I have made a significant change > > to > > > > the KIP. > > > > > > > > > Now, it > > > > > > > > > > > introduces v3 of the request header containing the client > > > > instance > > > > > > > ID > > > > > > > > > as a > > > > > > > > > > > proper field. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313*3A*Client*instance*ID*in*all*request*headers__;JSsrKysrKys!!Ayb5sqE7!pUZS25PuTKdhiqvTGcwDKbclqpG8dFOzw2Ax9WxVM6Sb09PVQ0gl1i0gjlr2_Krr66agbuVC2qLGgUcg7vrW$ > > > > > > > > > > > > > > > > > > > > > > > > > > Please review and let me know what you think. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > On 2026/05/28 21:51:01 "Matthias J. Sax" wrote: > > > > > > > > > > > > >> Thanks Andrew. I agree to the issue that tagged > > fields > > > > are > > > > > > > > > optional. > > > > > > > > > > > > >> > > > > > > > > > > > > >> Nevertheless, I find the proposed compatibility > > > > protocol too > > > > > > > > > complex > > > > > > > > > > > and > > > > > > > > > > > > >> unnecessary, and would still prefer a version bump > > for > > > > > > > > > > > > >> `GetTelemetrySubscriptionsRequest/Reponse` to clean > > it > > > > all > > > > > > > up. -- > > > > > > > > > For > > > > > > > > > > > > >> `PushTelemetryRequest` the argument from above > > applies, > > > > so > > > > > > > > > leaving it > > > > > > > > > > > > >> untouched is ok. > > > > > > > > > > > > >> > > > > > > > > > > > > >> I don't see why `GetTelemetrySubscriptionsRequest` > > > > would need > > > > > > > to > > > > > > > > > > > encode > > > > > > > > > > > > >> the `clientInstandId` in it's request body. Given > > that > > > > the > > > > > > > > > > > > >> `clientInstanceId` is a random UUID, the returned > > > > subscription > > > > > > > > > cannot > > > > > > > > > > > > >> really depend on it. So why would we need to send > > it to > > > > the > > > > > > > > > broker? > > > > > > > > > > > > >> Right now, it's set to ZERO only for the purpose to > > get > > > > the > > > > > > > > > broker to > > > > > > > > > > > > >> assign a new UUID. But it has nothing to do with the > > > > actual > > > > > > > "what > > > > > > > > > > > > >> metrics should I send" question. > > > > > > > > > > > > >> > > > > > > > > > > > > >> To be fair, KIP-714 lists `client_instance_id` as a > > > > field that > > > > > > > > > can be > > > > > > > > > > > > >> used to define a subscription. But it seems rather > > > > useless in > > > > > > > > > practice > > > > > > > > > > > > >> to me? I a user defines a subscription, they cannot > > > > know the > > > > > > > UUID. > > > > > > > > > > > Thus, > > > > > > > > > > > > >> I think we should actually drop supporting > > > > > > > `client_instance_id` > > > > > > > > > as a > > > > > > > > > > > > >> "subscription matching parameter"? Of course, there > > is a > > > > > > > backward > > > > > > > > > > > > >> compatibility question, but I think we can address > > > > this: if > > > > > > > > > brokers > > > > > > > > > > > are > > > > > > > > > > > > >> upgraded and they have an existing > > `client_instance_id` > > > > based > > > > > > > > > > > > >> subscription defined, they could advertise to only > > > > support > > > > > > > v0, and > > > > > > > > > > > they > > > > > > > > > > > > >> should log a warning that this feature is > > deprecated. > > > > New > > > > > > > brokers > > > > > > > > > > > would > > > > > > > > > > > > >> also not allow to use `client_instance_id` any > > longer to > > > > > > > define a > > > > > > > > > > > > >> subscription. > > > > > > > > > > > > >> > > > > > > > > > > > > >> Similarly, for `GetTelemetrySubscriptionsResponse`, > > we > > > > only > > > > > > > need > > > > > > > > > > > > >> `clientInstanceId` if the broker computes the UUID. > > But > > > > if we > > > > > > > let > > > > > > > > > the > > > > > > > > > > > > >> client compute it, this field is not needed any > > longer. > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> -Matthias > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> On 5/28/26 1:21 AM, Andrew Schofield wrote: > > > > > > > > > > > > >>> Hi Jun, > > > > > > > > > > > > >>> Thanks for your response. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> JR24: I'll update the KIP shortly. Here's a > > summary. > > > > The > > > > > > > various > > > > > > > > > > > modern group protocol heartbeat requests encode the > > member > > > > ID as a > > > > > > > > > string > > > > > > > > > > > for historical reasons, even though it's in practice a > > UUID. > > > > The > > > > > > > Apache > > > > > > > > > > > Kafka Java client happens to use > > > > org.apache.kafka.common.Uuid to > > > > > > > > > create the > > > > > > > > > > > member ID and convert it into a string. This means that > > when > > > > the > > > > > > > > > > > clientInstanceId is converted into a string, its encoding > > > > matches > > > > > > > the > > > > > > > > > > > member ID and it all lines up. However, there are other > > ways > > > > to > > > > > > > encode > > > > > > > > > a > > > > > > > > > > > UUID into a string and I know for a fact that the > > librdkafka > > > > client > > > > > > > > > does it > > > > > > > > > > > differently. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Thanks, > > > > > > > > > > > > >>> Andrew > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> On 2026/05/27 17:11:25 Jun Rao via dev wrote: > > > > > > > > > > > > >>>> Hi, Andrew, > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> Thanks for the reply. One more comment. > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> JR24. "The alignment of other identifiers is by > > > > convention > > > > > > > (and > > > > > > > > > the > > > > > > > > > > > Java > > > > > > > > > > > > >>>> client will follow the convention) rather than > > > > mandate." > > > > > > > Could > > > > > > > > > you > > > > > > > > > > > describe > > > > > > > > > > > > >>>> the convention to convert a clientInstanceId > > (UUID) > > > > to a > > > > > > > > > memberId > > > > > > > > > > > (String)? > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> Jun > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> On Tue, May 19, 2026 at 2:36 AM Andrew Schofield < > > > > > > > > > > > [email protected]> > > > > > > > > > > > > >>>> wrote: > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>>> Hi Jun, > > > > > > > > > > > > >>>>> Thanks for your response. > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> JR23: You are absolutely correct. It seems to me > > > > that not > > > > > > > > > sending a > > > > > > > > > > > > >>>>> clientInstanceId in the header and explicitly > > > > sending a > > > > > > > zero > > > > > > > > > UUID > > > > > > > > > > > as the > > > > > > > > > > > > >>>>> clientInstanceId in the header can be treated as > > > > > > > semantically > > > > > > > > > > > equivalent. > > > > > > > > > > > > >>>>> I've tweaked the words slightly. > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> Thanks, > > > > > > > > > > > > >>>>> Andrew > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> On 2026/05/19 03:42:16 Jun Rao via dev wrote: > > > > > > > > > > > > >>>>>> Hi, Andrew, > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> Thanks for the reply. > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> JR23. Our message protocol doc says "Any fields > > in > > > > the > > > > > > > message > > > > > > > > > > > object > > > > > > > > > > > > >>>>> that > > > > > > > > > > > > >>>>>> are not present in the version that you are > > > > deserializing > > > > > > > > > will be > > > > > > > > > > > reset > > > > > > > > > > > > >>>>> to > > > > > > > > > > > > >>>>>> default values. Unless a custom default has > > been > > > > set:". > > > > > > > Uuid > > > > > > > > > > > fields > > > > > > > > > > > > >>>>>> default to zero uuid. > > > > > > > > > > > > >>>>>> So if the server gets header.clientInstanceId=0 > > in > > > > the > > > > > > > > > > > deserialized > > > > > > > > > > > > >>>>> header, > > > > > > > > > > > > >>>>>> could it distinguish between the ID not being > > > > present > > > > > > > (since > > > > > > > > > > > client is > > > > > > > > > > > > >>>>> old) > > > > > > > > > > > > >>>>>> and the ID being explicitly set to 0 by the > > client? > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> Jun > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> On Mon, May 18, 2026 at 7:45 PM Andrew > > Schofield < > > > > > > > > > > > [email protected]> > > > > > > > > > > > > >>>>>> wrote: > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>>> Hi Jun, > > > > > > > > > > > > >>>>>>> Thanks for your reply. It's tricky squaring a > > > > circle. > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> JR23: For GetTelemetrySubscriptions, I have > > > > changed it so > > > > > > > > > that a > > > > > > > > > > > client > > > > > > > > > > > > >>>>>>> which omits the ClientInstanceId from the > > request > > > > header > > > > > > > is > > > > > > > > > > > permitted > > > > > > > > > > > > >>>>> to > > > > > > > > > > > > >>>>>>> specify a zero ClientInstanceId in the request > > > > body, > > > > > > > > > following > > > > > > > > > > > original > > > > > > > > > > > > >>>>>>> KIP-714 precedent. However, a client which > > > > specifies a > > > > > > > > > > > > >>>>> ClientInstanceId in > > > > > > > > > > > > >>>>>>> the request header MUST specify the same > > > > > > > ClientInstanceId in > > > > > > > > > the > > > > > > > > > > > > >>>>> request > > > > > > > > > > > > >>>>>>> body. This ensures that the header and > > telemetry > > > > UUIDs > > > > > > > are > > > > > > > > > the > > > > > > > > > > > same. > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> Thanks, > > > > > > > > > > > > >>>>>>> Andrew > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> On 2026/05/12 17:48:23 Andrew Schofield wrote: > > > > > > > > > > > > >>>>>>>> Hi Jun, > > > > > > > > > > > > >>>>>>>> Thanks for the reply and digging into the > > details. > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> JR23: Correct. The client telemetry component > > > > will use > > > > > > > > > UUID-B > > > > > > > > > > > as the > > > > > > > > > > > > >>>>>>> client instance ID. > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> JR23.1: Yes, I agree. It's not ideal. When I > > was > > > > > > > drawing up > > > > > > > > > the > > > > > > > > > > > > >>>>> tables, > > > > > > > > > > > > >>>>>>> I was thinking that this might be a > > possibility, > > > > but I'm > > > > > > > less > > > > > > > > > > > convinced > > > > > > > > > > > > >>>>>>> now. I think that I should mandate that if a > > client > > > > > > > specifies > > > > > > > > > > > > >>>>>>> header.ClientInstanceId on > > > > GetTelemetrySubscriptions > > > > > > > request, > > > > > > > > > > > then > > > > > > > > > > > > >>>>>>> request.ClientInstanceId must either be zero or > > > > equal to > > > > > > > > > > > > >>>>>>> header.ClientInstanceId. > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> JR23.2: This is perhaps the interesting one. > > From > > > > its > > > > > > > > > original > > > > > > > > > > > > >>>>> intent, > > > > > > > > > > > > >>>>>>> it should be UUID-B (the telemetry UUID), but > > then > > > > that > > > > > > > > > > > contradicts the > > > > > > > > > > > > >>>>>>> change in signature to remove the timeout. > > Unless > > > > I make > > > > > > > the > > > > > > > > > > > change > > > > > > > > > > > > >>>>> above, > > > > > > > > > > > > >>>>>>> in which case it will be UUID-H. > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>> Andrew > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> On 2026/05/12 17:23:58 Jun Rao via dev wrote: > > > > > > > > > > > > >>>>>>>>> Hi, Andrew, > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> Thanks for the reply. > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> JR23. In the new client -> old broker case, > > we > > > > have > > > > > > > > > > > > >>>>>>>>> header.ClientInstanceId=UUID-H > > > > > > > > > > > > >>>>>>>>> request.ClientInstanceId=UUID-B > > > > > > > > > > > > >>>>>>>>> response.ClientInstanceId=0 > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> On the server side, I guess the telemetry > > > > component > > > > > > > will > > > > > > > > > use > > > > > > > > > > > > >>>>> UUID-B as > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > >>>>>>>>> clientInstanceId? This has a couple of > > > > implications. > > > > > > > > > > > > >>>>>>>>> JR23.1 On the server side, we have two > > different > > > > > > > > > > > clientInstanceIds > > > > > > > > > > > > >>>>>>> used in > > > > > > > > > > > > >>>>>>>>> different places, UUID-H for request logging > > and > > > > > > > UUID-B in > > > > > > > > > > > > >>>>> telemetry. > > > > > > > > > > > > >>>>>>> This > > > > > > > > > > > > >>>>>>>>> seems confusing since we can't uniquely > > identify > > > > a > > > > > > > client > > > > > > > > > on > > > > > > > > > > > the > > > > > > > > > > > > >>>>> server > > > > > > > > > > > > >>>>>>>>> side. > > > > > > > > > > > > >>>>>>>>> JR23.2 On the client side. what uuid does > > > > > > > > > > > clientInstanceId(Duration > > > > > > > > > > > > >>>>>>>>> timeout) return? If it returns UUID-H, it > > will be > > > > > > > confusing > > > > > > > > > > > since > > > > > > > > > > > > >>>>> it > > > > > > > > > > > > >>>>>>>>> doesn't match the ID used for telemetry on > > the > > > > server. > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> Jun > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> On Tue, May 12, 2026 at 12:58 AM Andrew > > > > Schofield < > > > > > > > > > > > > >>>>>>> [email protected]> > > > > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> Hi Jun, > > > > > > > > > > > > >>>>>>>>>> Thanks for your response. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> JR20: I have improved (I hope) the wording. > > The > > > > client > > > > > > > > > sends > > > > > > > > > > > > >>>>>>>>>> request.clientInstanceId = 0 and > > > > > > > header.clientInstanceId = > > > > > > > > > > > > >>>>> UUID-H, > > > > > > > > > > > > >>>>>>> and the > > > > > > > > > > > > >>>>>>>>>> broker responds > > > > response.clientInstanceId=UUID-H. In > > > > > > > this > > > > > > > > > way, > > > > > > > > > > > > >>>>> the > > > > > > > > > > > > >>>>>>> broker > > > > > > > > > > > > >>>>>>>>>> will have taken the UUID-H from the header, > > and > > > > told > > > > > > > the > > > > > > > > > > > client > > > > > > > > > > > > >>>>> to > > > > > > > > > > > > >>>>>>> use it > > > > > > > > > > > > >>>>>>>>>> for client telemetry also. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> JR21: Done. Look for "henceforth". > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> JR22: Summary table added. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>>>> Andrew > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> On 2026/05/11 19:18:24 Jun Rao via dev > > wrote: > > > > > > > > > > > > >>>>>>>>>>> Hi, Andrew, > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Thanks for the reply. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> JR20. "If the client requests a new client > > > > instance > > > > > > > ID > > > > > > > > > on its > > > > > > > > > > > > >>>>>>> initial > > > > > > > > > > > > >>>>>>>>>>> GetTelemetrySubscriptions request and it > > > > sends a > > > > > > > client > > > > > > > > > > > > >>>>> instance > > > > > > > > > > > > >>>>>>> ID in > > > > > > > > > > > > >>>>>>>>>> the > > > > > > > > > > > > >>>>>>>>>>> request header, the broker will send back > > that > > > > client > > > > > > > > > > > instance > > > > > > > > > > > > >>>>> ID > > > > > > > > > > > > >>>>>>> rather > > > > > > > > > > > > >>>>>>>>>>> than generating a new UUID. This will > > > > automatically > > > > > > > > > align the > > > > > > > > > > > > >>>>> UUID > > > > > > > > > > > > >>>>>>> in the > > > > > > > > > > > > >>>>>>>>>>> request headers and client telemetry." > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> This seems inconsistent with what's in the > > > > table. In > > > > > > > the > > > > > > > > > > > > >>>>> table, for > > > > > > > > > > > > >>>>>>>>>>> example, if the client has the following: > > > > > > > > > > > > >>>>>>>>>>> GetTelemetrySubscriptions v0 > > > > > > > > > > > > >>>>>>>>>>> header.ClientInstanceId = UUID-H > > > > > > > > > > > > >>>>>>>>>>> request.ClientInstanceId = UUID-H > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> or > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> GetTelemetrySubscriptions v0 > > > > > > > > > > > > >>>>>>>>>>> header.ClientInstanceId = UUID-H > > > > > > > > > > > > >>>>>>>>>>> request.ClientInstanceId = UUID-R > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> the broker returns > > > > > > > > > > > > >>>>>>>>>>> response.ClientInstanceId = 0. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> JR21. It will be useful to document what > > the > > > > new > > > > > > > client > > > > > > > > > does > > > > > > > > > > > > >>>>> with > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > >>>>>>>>>>> returned response.ClientInstanceId. Note > > that > > > > return > > > > > > > > > value > > > > > > > > > > > may > > > > > > > > > > > > >>>>> or > > > > > > > > > > > > >>>>>>> may not > > > > > > > > > > > > >>>>>>>>>>> be 0. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> JR22. It's probably clearer if we could > > > > populate the > > > > > > > > > table > > > > > > > > > > > > >>>>> with 4 > > > > > > > > > > > > >>>>>>>>>>> combinations: old/new clients with old/new > > > > brokers. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Jun > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> On Fri, May 8, 2026 at 2:49 AM Andrew > > > > Schofield < > > > > > > > > > > > > >>>>>>> [email protected]> > > > > > > > > > > > > >>>>>>>>>>> wrote: > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> Hi Jun and Chia-Ping, > > > > > > > > > > > > >>>>>>>>>>>> I've overhauled part of the KIP to do with > > > > > > > alignment of > > > > > > > > > the > > > > > > > > > > > > >>>>>>> request > > > > > > > > > > > > >>>>>>>>>> header > > > > > > > > > > > > >>>>>>>>>>>> client instance ID, client telemetry > > client > > > > > > > instance ID > > > > > > > > > and > > > > > > > > > > > > >>>>> group > > > > > > > > > > > > >>>>>>>>>> protocol > > > > > > > > > > > > >>>>>>>>>>>> member IDs. The alignment is by > > convention, > > > > not > > > > > > > mandate > > > > > > > > > > > > >>>>> (SHOULD > > > > > > > > > > > > >>>>>>> not > > > > > > > > > > > > >>>>>>>>>> MUST). > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> It would be possible to go around the > > > > existing RPCs > > > > > > > > > such as > > > > > > > > > > > > >>>>>>>>>>>> ConsumerGroupHeartbeat and > > > > > > > GetTelemetrySubscriptions, > > > > > > > > > and > > > > > > > > > > > > >>>>> remove > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > >>>>>>>>>> fields > > > > > > > > > > > > >>>>>>>>>>>> containing the existing identifiers which > > are > > > > > > > intended > > > > > > > > > to be > > > > > > > > > > > > >>>>>>> aligned. > > > > > > > > > > > > >>>>>>>>>> Doing > > > > > > > > > > > > >>>>>>>>>>>> so would be a bad idea though, because we > > > > would then > > > > > > > > > have > > > > > > > > > > > RPC > > > > > > > > > > > > >>>>>>> versions > > > > > > > > > > > > >>>>>>>>>>>> which essentially depend upon the > > presence of > > > > a > > > > > > > tagged > > > > > > > > > field > > > > > > > > > > > > >>>>> in > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > >>>>>>>>>> request > > > > > > > > > > > > >>>>>>>>>>>> header. This is a protocol-compatibility > > > > nightmare. > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> I have removed the new versions of > > > > > > > > > GetTelemetrySubscriptions > > > > > > > > > > > > >>>>> and > > > > > > > > > > > > >>>>>>>>>>>> PushTelemetry. I have also explained the > > > > behavior of > > > > > > > > > > > > >>>>>>>>>>>> GetTelemetrySubscriptions in the presence > > and > > > > > > > absence > > > > > > > > > of a > > > > > > > > > > > > >>>>> client > > > > > > > > > > > > >>>>>>>>>> instance > > > > > > > > > > > > >>>>>>>>>>>> ID in the request header. > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> Let me know what you think. > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>>>>>> Andrew > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> On 2026/05/07 15:09:31 Andrew Schofield > > wrote: > > > > > > > > > > > > >>>>>>>>>>>>> Hi Jun and Chia-Ping, > > > > > > > > > > > > >>>>>>>>>>>>> I've been thinking and discussing the > > > > changes to > > > > > > > the > > > > > > > > > > > > >>>>> KIP-714 > > > > > > > > > > > > >>>>>>> RPCs. > > > > > > > > > > > > >>>>>>>>>> There > > > > > > > > > > > > >>>>>>>>>>>> are too many combinations for my liking > > at the > > > > > > > moment. I > > > > > > > > > > > > >>>>> want to > > > > > > > > > > > > >>>>>>> take > > > > > > > > > > > > >>>>>>>>>>>> another pass at this area and will make an > > > > update > > > > > > > in a > > > > > > > > > few > > > > > > > > > > > > >>>>> days. > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>> I intend to start a new vote once we have > > > > consensus > > > > > > > > > because > > > > > > > > > > > > >>>>>>> the spec > > > > > > > > > > > > >>>>>>>>>> has > > > > > > > > > > > > >>>>>>>>>>>> changed somewhat since the earliest votes. > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>>>>>>> Andrew > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>> On 2026/05/06 17:28:27 Chia-Ping Tsai > > wrote: > > > > > > > > > > > > >>>>>>>>>>>>>> hi Andrew > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>>> chia_0: If the consensus is to remove > > the > > > > > > > "duplicate" > > > > > > > > > > > > >>>>> field > > > > > > > > > > > > >>>>>>> from > > > > > > > > > > > > >>>>>>>>>> the > > > > > > > > > > > > >>>>>>>>>>>> RPC payloads, the tagged field in the > > header > > > > will > > > > > > > > > > > essentially > > > > > > > > > > > > >>>>>>> become a > > > > > > > > > > > > >>>>>>>>>>>> required field. This means the broker > > needs to > > > > > > > handle > > > > > > > > > the > > > > > > > > > > > > >>>>> edge > > > > > > > > > > > > >>>>>>> case > > > > > > > > > > > > >>>>>>>>>> where > > > > > > > > > > > > >>>>>>>>>>>> both the header and the request body have > > no > > > > > > > > > > > > >>>>> ClientInstanceId, > > > > > > > > > > > > >>>>>>> right? > > > > > > > > > > > > >>>>>>>>>> If > > > > > > > > > > > > >>>>>>>>>>>> so, would you mind clarifying the expected > > > > broker > > > > > > > > > behavior > > > > > > > > > > > in > > > > > > > > > > > > >>>>>>> the KIP? > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>>> Best, > > > > > > > > > > > > >>>>>>>>>>>>>> Chia-Ping > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>>> On 2026/04/03 16:17:37 Andrew Schofield > > > > wrote: > > > > > > > > > > > > >>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > >>>>>>>>>>>>>>> I would like to start the discussion on > > > > KIP-1313. > > > > > > > > > This > > > > > > > > > > > > >>>>>>> adds a > > > > > > > > > > > > >>>>>>>>>> unique > > > > > > > > > > > > >>>>>>>>>>>> client instance ID to the request header > > of > > > > all > > > > > > > Kafka > > > > > > > > > > > > >>>>> protocol > > > > > > > > > > > > >>>>>>>>>> requests to > > > > > > > > > > > > >>>>>>>>>>>> give a unique identifier which can be > > used to > > > > > > > correlate > > > > > > > > > the > > > > > > > > > > > > >>>>>>> requests > > > > > > > > > > > > >>>>>>>>>> from > > > > > > > > > > > > >>>>>>>>>>>> each client for the purposes of problem > > > > > > > determination. > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313*3A*Client*instance*ID*in*all*request*headers__;JSsrKysrKys!!Ayb5sqE7!uqWf0-b_X82WmpmCYImD2W2rht_s_q5vHcqB9ToMV4IaeQbZF42eMJyS5XC5b5qE_qJJUj3KTCXcqEvYbwYS$ > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>>>>>>>>> Andrew > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
