I've updated the KIP doc and would start calling for a vote for this KIP
now.
On Tue, Oct 11, 2022 at 4:26 AM Nick Telford wrote:
> Hi Guozhang,
>
> What you propose sounds good to me. Having the more detailed Task-level
> metrics at DEBUG makes sense.
>
> Regards,
>
> Nick
>
--
-- Guozhang
Hi Guozhang,
What you propose sounds good to me. Having the more detailed Task-level
metrics at DEBUG makes sense.
Regards,
Nick
Hello Bruno / Nick,
Thanks for the great feedback (again)! I want to give my thoughts to them
and reply together here since some of them are correlated.
1) This KIP is indeed based on the separated restoration thread work,
trying to give more visibility as we move restoration to dedicated
Hi Guozhang,
The metrics you've described are currently "thread-level" metrics. I know
you're planning on splitting out the restoration from the StreamThread, so
I question the utility of some of these metrics being thread-level, as they
won't correlate with other thread-level metrics, which
Hi Guozhang!
Thanks for the updates!
1.
Why do you distinguish between active and standby for the total amount
of restored/updated records but not for the rate of restored/updated
records?
2.
Regarding "standby-records-remaining" I am not sure how useful this
metric is and I am not sure
Hello Bruno/Nick,
I've updated the KIP wiki to reflect the incorporated comments from you,
please feel free to take another look and let me know what you think.
Guozhang
On Tue, Sep 20, 2022 at 9:37 AM Guozhang Wang wrote:
> Hi Nick,
>
> Thanks for the reviews, and I think these are good
Hi Nick,
Thanks for the reviews, and I think these are good suggestions. Note that
currently the `restore-records-total/rate` would include both restoring
active tasks as well as updating standby tasks, I think for your purposes
you'd be more interested in active restoring tasks since they could
Hi Guozhang,
KIP looks great, I have one suggestion: in addition to
"restore-records-total", it would also be useful to track the number of
records *remaining*, that is, the records that have not yet been restored.
This is actually the metric I was attempting to implement in the
Hello Bruno,
Thanks for your comments!
1. Regarding the metrics group name: originally I put
"stream-state-metrics" as it's related to state store restorations, but
after a second thought I think I agree with you that this is quite
confusing and not right. About the metrics groups, right now I
Hi Guozhang,
Thanks for the KIP! I think this KIP is a really nice addition to better
understand what is going on in a Kafka Streams application.
1.
The metric names "paused-active-tasks" and "paused-standby-tasks" might
be a bit confusing since at least active tasks can be paused also
Hello everyone,
I'd like to start a discussion for the following KIP, aiming to improve
Kafka Stream's restoration visibility via new metrics and callback methods:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-869%3A+Improve+Streams+State+Restoration+Visibility
Thanks!
-- Guozhang
11 matches
Mail list logo
