Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-11-27 Thread Anton Agestam
Sorry, the underscore was meant to refer to the https://github.com/apache/kafka/tree/trunk/clients/src/main/resources/common/message directory in the previous message. Den fre 24 nov. 2023 kl 14:30 skrev Anton Agestam : > Hi Ismael, > > This looks like a healthy KIP for Kafka  > > As the

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-11-24 Thread Ismael Juma
Hi Anton, The json schema definition files haven't been deemed public api yet although they perhaps should (KIPs welcome). Your suggestion to update `validVersions` sounds good to me. Longer-term, it would also make sense to consider a `deprecatedVersions` field (this is especially useful if we

RE: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-11-24 Thread Anton Agestam
Hi Ismael, This looks like a healthy KIP for Kafka  As the implementer of a Kafka Protocol library for Python, Aiven-Open/kio [1], I'm curious how this change will affect the library. We generate entities for the full protocol by introspecting the JSON schema definitions under _. How will the

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-11-21 Thread Ismael Juma
I started a vote thread for this since I addressed the comments so far and it doesn't seem like there were any major concerns. Ismael On Tue, Jan 3, 2023 at 8:17 AM Ismael Juma wrote: > Hi all, > > I would like to start a discussion regarding the removal of very old > client protocol API

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-11-21 Thread Ismael Juma
Hi Jose, I updated the KIP to include a new metric for deprecated request api versions and also a new attribute in the request log to make it easy to find such entries, Thanks, Ismael On Thu, Jan 12, 2023 at 1:03 AM Ismael Juma wrote: > Hi Jose, > > I think it's reasonable to add more

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-03-18 Thread Ismael Juma
Hi Jun, I updated the KIP to include the inter-broker protocol version deprecation. Ismael On Mon, Jan 9, 2023 at 3:53 PM Jun Rao wrote: > Hi, Ismael, > > Thanks for the reply. > > 2. Yes, that makes sense. > > Jun > > On Mon, Jan 9, 2023 at 11:26 AM Ismael Juma wrote: > > > Hi Jun, > > > >

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-12 Thread Ismael Juma
Hi Jose, I think it's reasonable to add more user-friendly metrics as you described. I'll update the KIP soon with that. I'll try to define them in a way where they track deprecated protocols for the next major release. That way, they can be useful even after AK 4.0 is released. Ismael On Wed,

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-11 Thread José Armando García Sancio
Thanks Ismael. > The following metrics are used to determine both questions: > > > >- Client name and version: > > > > kafka.server:clientSoftwareName=(client-software-name),clientSoftwareVersion=(client-software-version),listener=(listener),networkProcessor=(processor-index),type=(type)

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-10 Thread Ismael Juma
Hi Jose, The KIP describes a couple of existing metrics that can be used to answer that question: The following metrics are used to determine both questions: > >- Client name and version: > >

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-10 Thread José Armando García Sancio
Hi Ismael, Thanks for the improvement. I haven't been following the discussion in detail so it is possible that this was already discussed. If a user upgrades to Apache Kafka 4.0 it is possible for some of their clients to stop working because the request's version would not be a version that

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-09 Thread Jun Rao
Hi, Ismael, Thanks for the reply. 2. Yes, that makes sense. Jun On Mon, Jan 9, 2023 at 11:26 AM Ismael Juma wrote: > Hi Jun, > > Thanks for the feedback. > > 1. Good question. I was looking at the IBP issue as more of an inter broker > protocol communication thing and hence I thought we'd

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-09 Thread Ismael Juma
Hi Jun, Thanks for the feedback. 1. Good question. I was looking at the IBP issue as more of an inter broker protocol communication thing and hence I thought we'd cover that as part of the KIP that modernizes that aspect (yet to be written). I filed a JIRA recently that shares some of the

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-09 Thread Jun Rao
Hi, Ismael, Thanks for the KIP. A couple of comments. 1. Should we deprecate IBPs before 2.1 too? 2. There seem to be more requests than those listed in the KIP. Are they all introduced after 2.1? Thanks, Jun On Sat, Jan 7, 2023 at 5:13 PM Ismael Juma wrote: > Hi all, > > I updated the

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-07 Thread Ismael Juma
Hi all, I updated the KIP to make it explicit that the Java clients shipped with Apache Kafka 4.0 would only be guaranteed to work with brokers running Apache Kafka 2.1 or newer, since they too would no longer have access to the removed protocol APIs. Thanks to Jason for asking about this aspect

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-04 Thread Ismael Juma
Hi Viktor, Thanks for the feedback. See inline. I was gonna ask about whether you think it would be useful to automate > protocol deprecation but I see you'll have a separate KIP for that so I'll > just wait :). > With regards to the follow-up KIP, if you are interested in submitting that,

Re: [DISCUSS] KIP-896: Remove old client protocol API versions in Kafka 4.0

2023-01-04 Thread Viktor Somogyi-Vass
Hi Ismael, I think this is a good idea, it can also simplify the code quite a bit. I was gonna ask about whether you think it would be useful to automate protocol deprecation but I see you'll have a separate KIP for that so I'll just wait :). I see that in multiple places clients have older than