Hi Andrew, We’re definitely agreed on the increased use of frameworks over than the client directly. I’ve cited the different patterns Spring, Micronaut, smallrye and company-internal frameworks have APIs built on top of the Kafka client, in a couple of KIPs already.
The project has enough traction that I think of it as its own network client with distributed log semantics for which frameworks are written, much like gRPC over netty. People want to just write the business logic and leave the plumbing and threading to the frameworks. All of this to say, I’m fine to ship the client framework as the new identifying parameter for frameworks to set. It will be mighty useful. My addendum, rather than a pushback is that: guilty as charged, I’m driven by the otel/DD telemetry/observability of using Apache Kafka in applications. The client.framework.version config for instance can be used to detect regressions and isolate root causes. But I feel it is only one of many such facets. You mentioned that you would use client.id and clientInstanceId to identify a client but that these do not help with aggregate/fleet-wide issues. As an example, if I tag an app’s AZ it can help me write alerts on spike in latency in us-west-2. Or, detect stuck partitions across multiple client.id/application.names none of whom share the same client.framework.name. Other tags that come to mind: application.name, az, team, env, rack. All fields users usually hijack client.id for. An otel JavaAgent can capture the client metadata registered and attach it as tags with each resource span. Users who use frameworks but rely on datadog/otel will get visibility into the client metadata for free. The KIP as I read it, serves as a foundation for future use. So I don’t want to shoehorn a new behaviour if it explodes the scope too much. Best regards, Aditya > On Jul 27, 2026, at 13:52, Andrew Schofield <[email protected]> wrote: > > Hi Aditya, > Thanks for your response. > > AK1: I chose framework as the blessed abstraction because my focus was > problem determination for client applications. We often find that users with > client problems have not coded directly to the Kafka client interface because > they are using a framework. As a result, their knowledge of the application > code is one level removed from the Kafka client. Frameworks can override > configuration defaults, introduce different retry behaviour and so on. Lots > of companies have their own internal frameworks, so this KIP can be used by > them too. I'm trying to make it easier to work out when a user is making use > of a framework and knowing what it is. > > Sometimes, particularly for non-Java clients, people have overridden the > ClientSoftwareName/Version themselves, which makes those concepts much less > useful than they should be. By providing ClientFrameworkName/Version, there > is no longer any need to do so. That's another motivation here, even though > KIPs don't concern themselves with non-Java clients as such. > > We could go for a more flexible key-value approach, but a simple name and > version is sufficient for what I had in mind. Feel free to push back with > additional justification and examples. > > AK2: Done. o.a.k.clients.CommonClientConfigs. > > AK3: To identify a particular client, I would use client ID and > client-instance ID. I think these are generally more useful concepts than the > framework name and version which are extra information for the person trying > to figure out why a client is not behaving as expected. > > AK4: I'm sure you have more experience of OTel/DataDog collectors. You may > well be correct that they would be helpful for the collectors. > > Thanks, > Andrew > >> On 2026/07/26 07:48:20 Aditya Kousik wrote: >> Hello Andrew, >> >> I’m reminded of the client.id discussion we had back in KIP-1313 re: client >> instance id. After that discussion, I have a WIP KIP that sets a foundation >> for shipping client metadata tags to be sent for telemetry. I was hoping we >> could discuss if part of that approach could fit this KIP. >> >> AK1. I had a note on the motivation of selecting a “framework” as a blessed >> abstraction. The KIP mentions it’s for the client metadata and easier >> problem diagnosis. This is akin to an “application id” that non-framework >> clients usually tag with. >> >> KIP-606 took an approach like metrics.context.<key>=<val>. If we allow >> client.metadata.<key>=<val>, then a framework/application name and version >> can sit in such a metadata context and be sent with ApiVersion RPC. Of >> course, this is an open box approach rather than just the framework >> name/version (just two fields) we’re adding to the protocol. But I’m curious >> about the tier of importance of framework alone. >> >> AK2. Can you clarify if this config goes into CommonClientConfigs.java, >> referenced across producer/consumer/share etc? I know some share props have >> the “share.” prefix going. >> >> AK3. In another thread you mentioned that the broker may add it to the >> request context. In client logs, clientId is a very useful string to >> identify WARN logs when things go sideways like broker disconnected, >> rebalance in progress etc. Can you cherry pick and highlight some useful log >> places that these strings can go? I suppose adding clientId/framework to the >> MDC context might be too voluminous. >> >> AK4. I foresee collectors like Datadog/OTel might find these tags useful in >> each span exported. >> >> Looking forward to your thoughts on this. >> >> Regards, >> Aditya >> >>>> On Jul 17, 2026, at 11:53, Andrew Schofield >>>> <[email protected]> wrote: >>> >>> Hi, >>> I'd like to open discussion on KIP-1368: Client framework name and version. >>> >>> Applications often use application frameworks such as Spring to connect to >>> Kafka. To assist with problem diagnosis, this KIP introduces a way to >>> provide the framework name and version as part of the metadata the client >>> sends to the broker when it connects. >>> >>> Here's the KIP: https://cwiki.apache.org/confluence/x/J4Q_Gg >>> >>> Thanks, >>> Andrew >>
