Thanks for the replies David and Magnus David:
02: From a client implementation perspective, it is easier to gate features based on the max version numbers returned per request, rather than any textual representation of a string. I’m not really envisioning a client implementation trying to match on an undefined string, especially if it’s documented as just metadata information. 03: Interesting, I may be one of the few that does query the version directly. Perhaps this can be some new information that is instead added to request 60, ClusterMetadata? The con with ClusterMetadata is that I’m interested in this information on a per-broker basis. We could add these fields per each broker in the Brokers field, though. Magnus: As far as I can see, only my franz-go client offers this ability to “guess” the version of a broker — and it’s historically done so through ApiVersions, which was the only way to do this. This feature was used in three projects by two people: my kcl project, and the formerly-known-as Kowl and Kminion projects. After reading through most of the discussion thread on KIP-35, it seems that the conversation about using a broker version string / cluster aggregate version was specifically related to having the client choose how to behave (i.e., choose what versions of requests to use). The discussion was not around having the broker version as a piece of information that a client can use in log lines or for end-user presentation purposes. It seems a bit of an misdirected worry that a client implementor may accidentally use an unstructured string field for versioning purposes, especially when another field (ApiKeys) exists for versioning purposes and is widely known. Implementing a Kafka client is quite complex and there are so many other areas an implementor can go wrong, I’m not sure that we should be worried about an unstructured and documented metadata field. "the existing ApiVersionsReq … this information is richer than a single version string" Agree, this true for clients. However, it’s completely useless visually for end users. The reason Kminion used the version guess was two fold: to emit log a log on startup that the process was talking to Kafka v#.#, and to emit a const gauge metric for Prometheus where people could monitor external to Kafka what version each broker was running. Kowl uses the version guess to display the Kafka version the process is talking to immediately when you go to the Brokers panel. I envision that this same UI display can be added to Conduktor, even Confluent, etc. kcl uses the version guess as an extremely quick debugging utility: software engineers (not cluster administrators) might not always know what version of Kafka they are talking to, but they are trying to use a client. I often receive questions about “why isn’t this xyz thing working”, I ask for their cluster version with kcl, and then we can jump into diagnosing the problem much quicker. I think, if we focus on the persona of a cluster administrator, it’s not obvious what the need for this KIP is. For me, focusing on the perspective of an end-user of a client makes the need a bit clearer. It is not the responsibility of an end-user to manage the cluster version, but it is the responsibility of an end-user to know which version of a cluster they are talking to so that they know which fields / requests / behaviors are supported in a client All that said: I think this information is worth it and unlikely to be misused. IMO, ApiVersions is the main place to include this information, but another alternative is ClusterMetadata. Adding these fields to ClusterMetadata might be a bit awkward and may return stale information sometimes during a rolling upgrade. Curious to know your thoughts, and again thank you for the consideration and replies, - Travis On 2022/11/11 13:07:37 Magnus Edenhill wrote: > Hi Travis and thanks for the KIP, two comments below: > > > Den fre 11 nov. 2022 kl 13:37 skrev David Jacot <da...@gmail.com>: > > > 02: I am a bit concerned by clients that could misuse these information. > > For instance, one may be tempted to rely on the version to decide whether a > > feature is enabled or not. The api versions should remain the source of > > truth but we cannot enforce it with the proposed approach. That may be > > acceptable though. > > > > We proposed including this in the original ApiVersionRequest KIP-35 (waaay > back), but it was rejected > for exactly this reason; that it may(will) be misused by clients. > > > > I would also like to question the actual end-user use of this information, > the existing ApiVersionsReq > already provides - on a detailed level - what features and functionality > the broker supports - > this information is richer than a single version string. > > The KIP says "End users can quickly check from a client if the cluster is > up to date" and > "Clients can also use the broker version in log lines so that end users can > quickly see > if a version looks about right or if something is seriously broken.": > > In my mind that's not typically the end-users role or responsibility, but > that of the Kafka cluster operator, > who'll already know the version being deployed. > > > Regards, > Magnus > > > > > Le jeu. 10 nov. 2022 à 19:10, Travis Bischel <tr...@gmail.com> a > > écrit : > > > > > Hi all, > > > > > > I've written a KIP to expose the BrokerSoftwareName and > > > BrokerSoftwareVersion to clients: > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-885%3A+Expose+Broker%27s+Name+and+Version+to+Clients > > > > > > If we agree this is useful, it would be great to have this in by 3.4. > > > > > > Thank you, > > > - Travis > > > > > >
