[DISCUSS] KIP-977: Partition-Level Throughput Metrics

Hi All, Although this has been discussed many times, I would like to start a new discussion regarding the introduction of partition-level throughput metrics. Please review the KIP and I'm eager to know everyone's thoughts:

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

ntial impact if they decide to turn it on. > > > > Is there some guidance on the level of precision and detail expected > when providing the performance numbers in the KIP? > > > > This notion of proving out the performance impact is important, I agree. > Anecdotally, there wa

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Mon, Oct 30, 2023 at 6:26 PM Divij Vaidya wrote: > Hey *Qichao* > > Thank you for the update on the KIP. I like the idea of incremental > delivery and adding which metrics support this verbosity as a

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

given the benefits you mentioned earlier, I tried to make the config a JSON array. The change is to make it compatible with multi-level settings. Let me know what you think. Many thanks! Best, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Wed, Nov 1, 2023 at 9

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

Hi Divij, It would be very nice if you could take a look at the recent changes, thank you! If there's no more required changes, shall we move to vote stage? Best, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Thu, Nov 2, 2023 at 12:06 AM Qichao Chu

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

Hello Everyone, Can I ask for some feedback regarding KIP-977 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-977%3A+Partition-Level+Throughput+Metrics> ? Best, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Mon, Oct 16, 2023 at 7:34 P

Re: Feedback of using the 'confluent-kafka-go' for reliablity and performance

We are wondering if similar problems are spotted in your usage. Best, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Mon, Jul 31, 2023 at 10:57 AM Kenneth Eversole wrote: > Hello, > > Xinli, > > I can only speak for Cloudflare, but we use var

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

Thank you again for the nice suggestions, Jorge! I will wait for Divij's response and move it to the vote stage once the generic filter part reached concensus. Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Fri, Nov 10, 2023 at 6:49 AM Jorge Esteban Qu

Re: Request permission to contribute

Hi Bruno, Is it ok for you to set me up for contribution too? I would like to create a KIP. My account ID in Confluence and JIRA is ex172000 and my email is ex172...@gmail.com. Thank you in advance! Best, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/>

Re: Kafka trunk test & build stability

Hi Divij and all, Regarding the speeding up of the build & de-flaking tests, LinkedIn has done some great work which we probably can borrow ideas from. In the LinkedIn/Kafka repo, we can see one of their most recent PRs only took < 9 min(unit

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

s or suggestions. Looks good to move > forward > > from my perspective. > > > > > > > > -- > > Divij Vaidya > > > > > > > > On Fri, Nov 10, 2023 at 2:25 PM Qichao Chu > > wrote: > > > > > Thank you again for the

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

Hi Vahid, Thank you so much for the review and voting! Best, Qichao On Wed, Nov 29, 2023 at 2:55 PM Vahid Hashemian wrote: > Hi Qichao, > > Thanks for answering my questions and updating the KIP accordingly. > > It looks good to me. > > --Vahid > > > On Tue, No

[VOTE] KIP-997: Partition-Level Throughput Metrics

Hi all, I'd like to call a vote for KIP-977: Partition-Level Throughput Metrics. Please take a look here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-977%3A+Partition-Level+Throughput+Metrics Best, Qichao Chu

Re: [VOTE] KIP-997: Partition-Level Throughput Metrics

@Matthias: yeah it should be 977, sorry for the confusion. Btw, do you want to cast another binding vote for it? Best, Qichao Chu On Fri, Nov 17, 2023 at 12:45 AM Matthias J. Sax wrote: > This is KIP-977, right? Not as the subject says. > > Guess we won't be able to fix this now. Hop

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

have any more concerns. Best, Qichao Chu Software Engineer | Data - Kafka [image: Uber] <https://uber.com/> On Wed, Nov 8, 2023 at 6:29 AM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Hi Qichao, > > Thanks for the KIP! This will be a valuable con

Re: [VOTE] KIP-997: Partition-Level Throughput Metrics

Hi All, It would be nice if we could have more people to review and vote for this KIP. Many thanks! Qichao On Mon, Nov 20, 2023 at 2:43 PM Qichao Chu wrote: > @Matthias: yeah it should be 977, sorry for the confusion. > Btw, do you want to cast another binding vote for it? > > Be

[jira] [Created] (KAFKA-15447) Partition Level Throughput Metrics

Qichao Chu created KAFKA-15447: -- Summary: Partition Level Throughput Metrics Key: KAFKA-15447 URL: https://issues.apache.org/jira/browse/KAFKA-15447 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-16125) Add parser for the metrics.verbosity config

Qichao Chu created KAFKA-16125: -- Summary: Add parser for the metrics.verbosity config Key: KAFKA-16125 URL: https://issues.apache.org/jira/browse/KAFKA-16125 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-16129) Add integration test for KIP-977

Qichao Chu created KAFKA-16129: -- Summary: Add integration test for KIP-977 Key: KAFKA-16129 URL: https://issues.apache.org/jira/browse/KAFKA-16129 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-16127) Reset corresponding verbosity configs when topic is deleted

Qichao Chu created KAFKA-16127: -- Summary: Reset corresponding verbosity configs when topic is deleted Key: KAFKA-16127 URL: https://issues.apache.org/jira/browse/KAFKA-16127 Project: Kafka

[jira] [Created] (KAFKA-16124) Create metrics.verbosity dynamic config for controlling metrics verbosity

Qichao Chu created KAFKA-16124: -- Summary: Create metrics.verbosity dynamic config for controlling metrics verbosity Key: KAFKA-16124 URL: https://issues.apache.org/jira/browse/KAFKA-16124 Project: Kafka

[jira] [Created] (KAFKA-16128) Use metrics.verbosity in throughput metrics

Qichao Chu created KAFKA-16128: -- Summary: Use metrics.verbosity in throughput metrics Key: KAFKA-16128 URL: https://issues.apache.org/jira/browse/KAFKA-16128 Project: Kafka Issue Type: Task