Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi, Jose, Thanks for the reply. 20. I see the differences now. The metrics in KafkaController use Yammer metric and follow the camel case naming. The metrics in Raft use the client side Metrics package and follow the dash notation. So the naming in the KIP sounds good to me. 21. Sounds good.

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi Jun, Jun wrote: > 20. For the metric type and name, we use the camel names in some cases and > dashed lower names in some other cases. Should we make them consistent? For the metrics group `type=KafkaController`, I am using camel names like `MetadataLastAppliedRecordOffset` because it matches

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi, Jose, Thanks for the KIP. Just a couple of minor comments. 20. For the metric type and name, we use the camel names in some cases and dashed lower names in some other cases. Should we make them consistent? 21. Could you document the meaning of load-processing-time? Thanks, Jun On Mon,

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi all, Thanks for your feedback. I started a voting thread here: https://lists.apache.org/thread/x1cy5otpf7mj9ytghnktr5hog27hdf7k

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Thanks for all of the feedback. Some comments below: Luke wrote: > 1. Jason has asked but you didn't answer: What is the default value for ` > metadata.monitor.write.interval.ms`? Thanks for asking again. Looks like I missed this in my previous reply. In the implementation I am currently working

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi José, Thanks for the KIP! I think this will be a nice improvement. I had the same question as Luke and Jason: what's the default here for the NoOpRecord time? :) We should add a value here even if we think we'll adjust it later, just to give a feeling for how much traffic this would create.

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

José, thanks for the KIP! I think this is a good approach for proving the liveness of the quorum when metadata is not changing. 1. Based on the config name "metadata.monitor.write.interval.ms" I'm guessing the intention is to have a regularly scheduled write. If the quorum is busy with lots of

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi José, Thanks for the KIP! Some questions: 1. Jason has asked but you didn't answer: What is the default value for ` metadata.monitor.write.interval.ms`? 2. The `noopRecord` API key is `TBD`. Why can't we put the "currently used API Key nums + 1" into it? Any concern? 3. typo:

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Thanks for your feedback Jason, much appreciated. Here are the changes to the KIP: https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=211883219=5=4 On Tue, May 10, 2022 at 1:34 PM Jason Gustafson wrote: > The approach sounds reasonable. By the way, I think one of the

Re: [DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi Jose, Thanks for the KIP. The approach sounds reasonable. By the way, I think one of the gaps we have today is when the leader gets partitioned from the remaining voters. I believe it continues acting as a leader indefinitely. I was considering whether this periodic write can address the

[DISCUSS] KIP-835: Monitor KRaft Controller Quorum Health

Hi all, I created a KIP for adding a mechanism to monitor the health of the KRaft Controller quorum through metrics. See KIP-835: https://cwiki.apache.org/confluence/x/0xShD Thanks for your feedback, -José