Hi Afshin, A couple observations: 1. The image you inserted doesn't get shown, please fix it 2. I'd like to clarify your proposal a bit. So for now we had (user, client), (user) or (client) combinations. You'd like to introduce topic-partitions in this framework. Would it extend the current behavior, so the previously 4 item set becomes a 6 item set like this: (tp, user, client), (tp, user), (tp, client), (tp), (user) or (client)? Or do these tp quotas behave differently? 3. How would your implementation work when the aggregate of topic quotas exceed the available bandwidth? Do topics get fair access or is it possible that some partitions can't be consumed because others eat the bandwidth? 4. I'm a bit confused about the motivation section. So you're saying that if you have a topic with 6 partitions where a quota is set to 2MB/s, are you expecting a 4MB/s throughput if 2 of those topic's partition leaders are hosted on the broker? Wouldn't that violate backward compatibility because with a client now I can produce at 4MB/s rate?
Thanks, Viktor On Wed, Feb 14, 2024 at 9:27 PM Afshin Moazami <amoaz...@salesforce.com.invalid> wrote: > Thanks Viktor, > > Hi folks, > I would like to propose a new feature to extend the quota management in > Kafka to support topic-partition based quotas. The following is the link to > the KIP > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1010%3A+Topic+Partition+Quota > > > Best, > Afshin Moazami > > On Wed, Feb 7, 2024 at 5:25 AM Viktor Somogyi-Vass > <viktor.somo...@cloudera.com.invalid> wrote: > > > Hi Afshin, > > > > We keep KIP discussions on dev@kafka.apache.org so please post this over > > there too. I'll go over this later this week but devs usually monitor > that > > list more frequently and you'll have better chances of getting a reply > > there. > > > > Regards, > > Viktor > > > > On Wed, Jan 17, 2024 at 12:03 AM Afshin Moazami > > <amoaz...@salesforce.com.invalid> wrote: > > > > > Hi folks, > > > I am not sure what is the KIP life-cycle and how we can get more > > attention > > > on them, so I just reply to this thread with the hope to get some > > > discussion started. > > > > > > Thanks, > > > Afshin > > > > > > On Mon, Dec 11, 2023 at 10:43 AM Afshin Moazami < > amoaz...@salesforce.com > > > > > > wrote: > > > > > > > Hi folks, > > > > I would like to propose a new feature to extend the quota management > in > > > > Kafka to support topic-partition based quotas. The following is the > > link > > > to > > > > the KIP > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1010*3A*Topic*Partition*Quota__;JSsrKw!!DCbAVzZNrAf4!BK-888ZjIeh53cmPcRZ_ZIpA6-02xIk5LXsT4cl82ieHRjWN31a-xsi36sN9I3P3LOhhpYCJU2FpbYkfg2YpGX2RXtPFAIjsHv0$ > > > > > > > > > > > > Best, > > > > Afshin Moazami > > > > > > > > > >
