Hey Guozhang, do we plan to add per partition latency in this KIP?
On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna <br...@confluent.io> wrote: > Hi Guozhang, > > Thank you for the KIP. > > 1) As far as I understand, the StreamsMetrics interface is there for > user-defined processors. Would it make sense to also add a method to > the interface to specify a sensor that records skipped records? > > 2) What are the semantics of active-task-process and standby-task-process > > 3) How do dropped-late-records and expired-window-record-drop relate > to each other? I guess the former is for records that fall outside the > grace period and the latter is for records that are processed after > the retention period of the window. Is this correct? > > 4) Is there an actual difference between skipped and dropped records? > If not, shall we unify the terminology? > > 5) What happens with removed metrics when the user sets the version of > "built.in.metrics.version" to 2.2- > > Best, > Bruno > > On Thu, Jun 27, 2019 at 6:11 PM Guozhang Wang <wangg...@gmail.com> wrote: > > > > Hello folks, > > > > As 2.3 is released now, I'd like to bump up this KIP discussion again for > > your reviews. > > > > > > Guozhang > > > > > > On Thu, May 23, 2019 at 4:44 PM Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > Hello Patrik, > > > > > > Since we are rolling out 2.3 and everyone is busy with the release now > > > this KIP does not have much discussion involved yet and will slip into > the > > > next release cadence. > > > > > > This KIP itself contains several parts itself: 1. refactoring the > existing > > > metrics hierarchy to cleanup some redundancy and also get more > clarity; 2. > > > add instance-level metrics like rebalance and state metrics, as well as > > > other static metrics. > > > > > > > > > Guozhang > > > > > > > > > > > > On Thu, May 23, 2019 at 5:34 AM Patrik Kleindl <pklei...@gmail.com> > wrote: > > > > > >> Hi Guozhang > > >> Thanks for the KIP, this looks very helpful. > > >> Could you please provide more detail on the metrics planned for the > state? > > >> We were just considering how to implement this ourselves because we > need > > >> to > > >> track the history of stage changes. > > >> The idea was to have an accumulated "seconds in state x" metric for > every > > >> state. > > >> The new rebalance metric might solve part of our use case, but it is > > >> interesting what you have planned for the state metric. > > >> best regards > > >> Patrik > > >> > > >> On Fri, 29 Mar 2019 at 18:56, Guozhang Wang <wangg...@gmail.com> > wrote: > > >> > > >> > Hello folks, > > >> > > > >> > I'd like to propose the following KIP to improve the Kafka Streams > > >> metrics > > >> > mechanism to users. This includes 1) a minor change in the public > > >> > StreamsMetrics API, and 2) a major cleanup on the Streams' own > built-in > > >> > metrics hierarchy. > > >> > > > >> > Details can be found here: > > >> > > > >> > > > >> > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams > > >> > > > >> > I'd love to hear your thoughts and feedbacks. Thanks! > > >> > > > >> > -- > > >> > -- Guozhang > > >> > > > >> > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > -- > > -- Guozhang >
