Hi Jun, JR1: I dug into the KIP-714 client metrics stuff a bit more and I have revised the KIP. I have added client-framework-name and client-framework-version to the set of keys in the matching for client metrics, and also added these broker-added metrics tags. This is how these pieces of information would have been incorporated into KIP_714 had they been part of Kafka at that point.
Thanks, Andrew On 2026/08/04 20:29:19 Andrew Schofield wrote: > Hi Jun, > Thanks for the response. > > JR1: My intent here is only request logging. It would be possible to add > client-framework-name and client-framework-version to the match criteria for > client metrics, which could conceivably permit scenarios such as capturing > metrics from specific versions of the Spring framework. This is beyond the > scope I have in mind. > > JR3: Thanks for clarifying the tagged field conventions. Version bump > reverted. > > Thanks, > Andrew > > On 2026/08/04 18:38:40 Jun Rao via dev wrote: > > Hi, Andrew, > > > > Thanks for the reply. > > > > JR1. ClientSoftwareName and ClientSoftwareVersion are used in metric names > > and match predicates for configuring client metrics. Are ClientFrameworkName > > and ClientFrameworkVersion used in those places too or are they only used > > in request logging? > > > > JR3. The general rule for adding a new field in RPC is that (1) if it's > > truly optional, we add it as a tagged field with no version bump; (2) > > otherwise, we add it as a non-tagged field with a version bump. One > > exception is when adding a non-optional field in the request header. > > Because of the limitation in existing implementation, we need to add it as > > a tagged field with a version bump. > > > > Jun > > > > On Tue, Aug 4, 2026 at 6:04 AM Andrew Schofield <[email protected]> > > wrote: > > > > > Hi Jun, > > > Thanks for your response. > > > > > > JR1: I've expanded the motivation section in the KIP. They are truly > > > optional, I feel. When diagnosing an issue using the client logs, having > > > this information can illuminate why the application is behaving in a > > > particular way. If the application is using a framework, behaviours such > > > as > > > retries will typically not be in the application code itself because > > > they're implemented in the framework. There's a difference between what > > > the > > > application developer thinks their code does and what we see in the logs. > > > That's the point. > > > > > > JR2: Done. They're Type.STRING, default null. I think they should be > > > importance LOW because that affects the prominence of the configs in the > > > documentation, but I wonder whether you agree. > > > > > > Thanks, > > > Andrew > > > > > > On 2026/08/03 21:39:06 Jun Rao via dev wrote: > > > > Hi, Andrew, > > > > > > > > Thanks for the KIP. > > > > > > > > JR1. Could you describe the use cases of the two new configs and are > > > > they > > > > truly optional? > > > > > > > > JR2. Could you add the type and the default value for the new configs? > > > > > > > > Jun > > > > > > > > > > > > On Fri, Jul 31, 2026 at 10:23 AM Matthias J. Sax <[email protected]> > > > wrote: > > > > > > > > > Thanks for the KIP Andrew. I think it will be very useful if we can > > > > > identify frameworks, especially our own ones (Connect and Streams). > > > > > > > > > > About Aditya's point: I am wondering where to draw the line. Many > > > > > examples seems to be metadata that belongs into he Kafka record > > > > > `Headers` at the application level, rather than the lower level > > > > > request > > > > > headers? > > > > > > > > > > There if of course no strict logical dividing line between both. And > > > > > yes, the broker does not access application level Kafka record > > > > > `Headers`. But if it would be useful to let the broker tap into > > > > > application level record `Headers` we should tackle it independently? > > > > > > > > > > Personally, I don't think it would be the right design to push too > > > > > many > > > > > thing into the lower level request headers. So I am in favor of only > > > > > added the two new propose `ClientFrameworkName` and > > > > > `ClientFrameworkVersion` fields. > > > > > > > > > > > > > > > -Matthias > > > > > > > > > > On 7/31/26 7:56 AM, Federico Valeri wrote: > > > > > > Changes look good. Thanks. > > > > > > > > > > > > On Fri, Jul 31, 2026 at 11:17 AM Andrew Schofield < > > > [email protected]> > > > > > wrote: > > > > > >> > > > > > >> Hi Fede, > > > > > >> Thanks for your response. > > > > > >> > > > > > >> FV1: They are set using the regular config properties. Yes, it is > > > > > possible for an end user to set arbitrary values, but then that would > > > also > > > > > be true of a builder or internal constructor. I've added a bit more > > > > > information in the KIP and beefed up the config descriptions to > > > discourage > > > > > application use. > > > > > >> > > > > > >> FV2: I've never encountered such nightmares myself. The framework > > > which > > > > > sets the config last would win. Maybe this is a motivation for having > > > > > a > > > > > programmatic way of setting the information. We could support > > > concatenation > > > > > of framework information, but that's just pandering to these people. > > > Let me > > > > > know what you think. > > > > > >> > > > > > >> I've also updated the KIP with a maximum length for these pieces of > > > > > information since all identifiers should have defined bounds. > > > > > >> > > > > > >> Thanks, > > > > > >> Andrew > > > > > >> > > > > > >> On 2026/07/31 08:50:11 Federico Valeri wrote: > > > > > >>> Hi Andrew, the motivation looks good. A couple of questions: > > > > > >>> > > > > > >>> FV1: The KIP says frameworks should set them, but it does not > > > specify > > > > > >>> the mechanism. If these are ordinary user-facing configs, nothing > > > > > >>> prevents an end user from setting arbitrary values, which defeats > > > the > > > > > >>> purpose. Should the framework set them programmatically (builder > > > > > >>> or > > > > > >>> internal constructor) or is user override intentional? > > > > > >>> > > > > > >>> FV2: What if we have multiple layers of frameworks? Let's say a > > > custom > > > > > >>> framework on top of SpringBoot. I've seen similar nightmares in > > > > > >>> the > > > > > >>> past. > > > > > >>> > > > > > >>> Thanks > > > > > >>> Fede > > > > > >>> > > > > > >>> On Tue, Jul 28, 2026 at 8:27 AM Aditya Kousik < > > > [email protected]> > > > > > wrote: > > > > > >>>> > > > > > >>>> Hi Andrew, > > > > > >>>> > > > > > >>>> We’re definitely agreed on the increased use of frameworks over > > > than > > > > > the client directly. I’ve cited the different patterns Spring, > > > Micronaut, > > > > > smallrye and company-internal frameworks have APIs built on top of the > > > > > Kafka client, in a couple of KIPs already. > > > > > >>>> > > > > > >>>> The project has enough traction that I think of it as its own > > > network > > > > > client with distributed log semantics for which frameworks are > > > > > written, > > > > > much like gRPC over netty. People want to just write the business > > > logic and > > > > > leave the plumbing and threading to the frameworks. > > > > > >>>> > > > > > >>>> All of this to say, I’m fine to ship the client framework as the > > > new > > > > > identifying parameter for frameworks to set. It will be mighty useful. > > > > > >>>> > > > > > >>>> My addendum, rather than a pushback is that: guilty as charged, > > > I’m > > > > > driven by the otel/DD telemetry/observability of using Apache Kafka in > > > > > applications. The client.framework.version config for instance can be > > > used > > > > > to detect regressions and isolate root causes. But I feel it is only > > > one of > > > > > many such facets. You mentioned that you would use client.id and > > > > > clientInstanceId to identify a client but that these do not help with > > > > > aggregate/fleet-wide issues. As an example, if I tag an app’s AZ it > > > > > can > > > > > help me write alerts on spike in latency in us-west-2. Or, detect > > > > > stuck > > > > > partitions across multiple client.id/application.names none of whom > > > share > > > > > the same client.framework.name. > > > > > >>>> > > > > > >>>> Other tags that come to mind: application.name, az, team, env, > > > rack. > > > > > All fields users usually hijack client.id for. > > > > > >>>> > > > > > >>>> An otel JavaAgent can capture the client metadata registered and > > > > > attach it as tags with each resource span. Users who use frameworks > > > > > but > > > > > rely on datadog/otel will get visibility into the client metadata for > > > free. > > > > > >>>> > > > > > >>>> The KIP as I read it, serves as a foundation for future use. So I > > > > > don’t want to shoehorn a new behaviour if it explodes the scope too > > > much. > > > > > >>>> > > > > > >>>> Best regards, > > > > > >>>> Aditya > > > > > >>>> > > > > > >>>>> On Jul 27, 2026, at 13:52, Andrew Schofield < > > > [email protected]> > > > > > wrote: > > > > > >>>>> > > > > > >>>>> Hi Aditya, > > > > > >>>>> Thanks for your response. > > > > > >>>>> > > > > > >>>>> AK1: I chose framework as the blessed abstraction because my > > > focus > > > > > was problem determination for client applications. We often find that > > > users > > > > > with client problems have not coded directly to the Kafka client > > > interface > > > > > because they are using a framework. As a result, their knowledge of > > > > > the > > > > > application code is one level removed from the Kafka client. > > > Frameworks can > > > > > override configuration defaults, introduce different retry behaviour > > > and so > > > > > on. Lots of companies have their own internal frameworks, so this KIP > > > can > > > > > be used by them too. I'm trying to make it easier to work out when a > > > user > > > > > is making use of a framework and knowing what it is. > > > > > >>>>> > > > > > >>>>> Sometimes, particularly for non-Java clients, people have > > > overridden > > > > > the ClientSoftwareName/Version themselves, which makes those concepts > > > much > > > > > less useful than they should be. By providing > > > ClientFrameworkName/Version, > > > > > there is no longer any need to do so. That's another motivation here, > > > even > > > > > though KIPs don't concern themselves with non-Java clients as such. > > > > > >>>>> > > > > > >>>>> We could go for a more flexible key-value approach, but a simple > > > > > name and version is sufficient for what I had in mind. Feel free to > > > push > > > > > back with additional justification and examples. > > > > > >>>>> > > > > > >>>>> AK2: Done. o.a.k.clients.CommonClientConfigs. > > > > > >>>>> > > > > > >>>>> AK3: To identify a particular client, I would use client ID and > > > > > client-instance ID. I think these are generally more useful concepts > > > than > > > > > the framework name and version which are extra information for the > > > person > > > > > trying to figure out why a client is not behaving as expected. > > > > > >>>>> > > > > > >>>>> AK4: I'm sure you have more experience of OTel/DataDog > > > collectors. > > > > > You may well be correct that they would be helpful for the collectors. > > > > > >>>>> > > > > > >>>>> Thanks, > > > > > >>>>> Andrew > > > > > >>>>> > > > > > >>>>>> On 2026/07/26 07:48:20 Aditya Kousik wrote: > > > > > >>>>>> Hello Andrew, > > > > > >>>>>> > > > > > >>>>>> I’m reminded of the client.id discussion we had back in > > > KIP-1313 > > > > > re: client instance id. After that discussion, I have a WIP KIP that > > > sets a > > > > > foundation for shipping client metadata tags to be sent for telemetry. > > > I > > > > > was hoping we could discuss if part of that approach could fit this > > > KIP. > > > > > >>>>>> > > > > > >>>>>> AK1. I had a note on the motivation of selecting a “framework” > > > as a > > > > > blessed abstraction. The KIP mentions it’s for the client metadata and > > > > > easier problem diagnosis. This is akin to an “application id” that > > > > > non-framework clients usually tag with. > > > > > >>>>>> > > > > > >>>>>> KIP-606 took an approach like metrics.context.<key>=<val>. If > > > > > >>>>>> we > > > > > allow client.metadata.<key>=<val>, then a framework/application name > > > and > > > > > version can sit in such a metadata context and be sent with ApiVersion > > > RPC. > > > > > Of course, this is an open box approach rather than just the framework > > > > > name/version (just two fields) we’re adding to the protocol. But I’m > > > > > curious about the tier of importance of framework alone. > > > > > >>>>>> > > > > > >>>>>> AK2. Can you clarify if this config goes into > > > > > CommonClientConfigs.java, referenced across producer/consumer/share > > > etc? I > > > > > know some share props have the “share.” prefix going. > > > > > >>>>>> > > > > > >>>>>> AK3. In another thread you mentioned that the broker may add it > > > to > > > > > the request context. In client logs, clientId is a very useful string > > > to > > > > > identify WARN logs when things go sideways like broker disconnected, > > > > > rebalance in progress etc. Can you cherry pick and highlight some > > > useful > > > > > log places that these strings can go? I suppose adding > > > clientId/framework > > > > > to the MDC context might be too voluminous. > > > > > >>>>>> > > > > > >>>>>> AK4. I foresee collectors like Datadog/OTel might find these > > > tags > > > > > useful in each span exported. > > > > > >>>>>> > > > > > >>>>>> Looking forward to your thoughts on this. > > > > > >>>>>> > > > > > >>>>>> Regards, > > > > > >>>>>> Aditya > > > > > >>>>>> > > > > > >>>>>>>> On Jul 17, 2026, at 11:53, Andrew Schofield < > > > > > [email protected]> wrote: > > > > > >>>>>>> > > > > > >>>>>>> Hi, > > > > > >>>>>>> I'd like to open discussion on KIP-1368: Client framework name > > > and > > > > > version. > > > > > >>>>>>> > > > > > >>>>>>> Applications often use application frameworks such as Spring > > > > > >>>>>>> to > > > > > connect to Kafka. To assist with problem diagnosis, this KIP > > > introduces a > > > > > way to provide the framework name and version as part of the metadata > > > the > > > > > client sends to the broker when it connects. > > > > > >>>>>>> > > > > > >>>>>>> Here's the KIP: > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/J4Q_Gg__;!!Ayb5sqE7!vs-_AkY76KFOqYK02q4f2tKSkFdnly7eklb5qfewIk841seg2S5cIOXxFhnAixEYsDVuFQyJx8-D5g$ > > > > > >>>>>>> > > > > > >>>>>>> Thanks, > > > > > >>>>>>> Andrew > > > > > >>>>>> > > > > > >>> > > > > > > > > > > > > > > > > > > > >
