Works for me.
(Btw: I did not really vote, just stated my overall support :P )
On 9/5/19 2:50 PM, John Roesler wrote:
> Thanks, all.
>
> FWIW, the most recent formulation from Guozhang + Bruno's addendum would
> have my support.
>
> Thanks,
> -John
>
> On Thu, Sep 5, 2019 at 4:05 PM Bruno
Thanks, all.
FWIW, the most recent formulation from Guozhang + Bruno's addendum would
have my support.
Thanks,
-John
On Thu, Sep 5, 2019 at 4:05 PM Bruno Cadonna wrote:
> Hi Guozhang,
>
> Your summary corresponds to my proposal.
>
> A new value would only be added if in future we change the
Hi Guozhang,
Your summary corresponds to my proposal.
A new value would only be added if in future we change the metrics in
a backward-incompatible way, i.e., 2.4-. "latest" will always stay the default.
Best,
Bruno
On Thu, Sep 5, 2019 at 10:57 PM Guozhang Wang wrote:
>
> Hello Bruno,
>
> I
Hello Bruno,
I think your concern makes sense, let's adopt this suggestion in KIP-444
instead. Just to clarify:
1. The default value would be "latest".
2. The only other valid value is "0.10.0-2.3".
And moving forward this config may stay without any new values.
Guozhang
On Thu, Sep 5, 2019
Hi Guozhang,
I think user experience and code maintenance are tightly related. The
harder to maintain the code the worse the user experience will get.
Making the config optional does not solve the issue. Wouldn't users be
puzzled when we release 2.5 and they cannot set
built.in.metrics.version
Hi Bruno,
Thanks for raising this point. I think the main motivation behind this
proposal is, like Matthias said, to ease the understanding burden from
users to our own shoulders. Testing wise, I think we do not necessarily
need to explode the testing matrix but just test the last version before
I share Bruno's concern about future releases, however, I would make
slightly different proposal.
Instead of using "latest" we can just make the config optional and if
not set, we use the new metrics code? This way we don't need to add a
new version number each time we do a new release (note,
Hi,
I am sorry to restart the discussion here, but I came across a small
issue in the KIP.
I started to implement KIP-444 and I am bit concerned about the values
for the the config `built.in.metrics.version`. In the KIP the possible
values are specified as all Kafka Streams versions. I think
Hello Bruno,
I've updated the wiki page again per your comments, here's a brief summary:
1. added the list of removed metrics.
2. added a task-level INFO metric "dropped-records" that covers all
scenarios and merges in the existing "late-records-drop",
"skipped-records", and
Hi Bruno,
No it was not intentional, and we can definitely add the total amount
sensor as well -- they are just util functions to save users some lines of
code anyways, and should be straightforward.
Guozhang
On Tue, Aug 20, 2019 at 1:05 AM Bruno Cadonna wrote:
> Hi Guozhang,
>
> I totally
Hi Guozhang,
I totally missed the total invocation count metric in the javadoc.
Which brings me to a follow-up question. Should the names of the
methods reflect the included total invocation count? We have to rename
them anyways. One option would be to simply add `Total` to the method
names,
Hi Bruno,
Just realized that for `addRateSensor` and `addLatencyAndRateSensor` we've
actually added the total invocation metric already.
Guozhang
On Mon, Aug 19, 2019 at 4:11 PM Guozhang Wang wrote:
> Hi Bruno,
>
>
> On Tue, Aug 6, 2019 at 1:51 AM Bruno Cadonna wrote:
>
>> Hi Guozhang,
>>
Hi Bruno,
On Tue, Aug 6, 2019 at 1:51 AM Bruno Cadonna wrote:
> Hi Guozhang,
>
> I left my comments inline.
>
> On Thu, Jul 18, 2019 at 8:28 PM Guozhang Wang wrote:
> >
> > Hello Bruno,
> >
> > Thanks for the feedbacks, replied inline.
> >
> > On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna
Hi Guozhang,
I left my comments inline.
On Thu, Jul 18, 2019 at 8:28 PM Guozhang Wang wrote:
>
> Hello Bruno,
>
> Thanks for the feedbacks, replied inline.
>
> On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna wrote:
>
> > Hi Guozhang,
> >
> > Thank you for the KIP.
> >
> > 1) As far as I
For the existing releases, yes (with KIP-447 we are already going to do
that anyways), for future release maybe not --- hopefully we only do such
metrics refactoring once.
Guozhang
On Fri, Aug 2, 2019 at 3:23 PM Matthias J. Sax
wrote:
> Would this imply that we need to update the config in
Would this imply that we need to update the config in each release to
add a new accepted value?
-Matthias
On 8/2/19 1:07 PM, Guozhang Wang wrote:
> Hello Matthias,
>
> That's a good question. Thinking about a bit more, I'd like to propose that
> we just list all the version numbers since 0.10
Hello Matthias,
That's a good question. Thinking about a bit more, I'd like to propose that
we just list all the version numbers since 0.10 to 2.4 as accepted values,
and let Streams to decide if old / new set of metrics can be used
internally (implementation wise we can reuse the const values
Thanks for the KIP Guozhang.
I just re-read the wiki page and the DISCUSS thread. Overall LGTM.
The only nit is the naming of the new config values. With AK 2.3 being
released the versions numbers needs to be updated.
Additionally, I actually think that "2.2-" and "2.3" are not the best
names:
Thanks everyone for your inputs, I've updated the wiki page accordingly.
@Bruno: please let me know if you have any further thoughts per my replies
above.
Guozhang
On Mon, Jul 22, 2019 at 6:30 PM Guozhang Wang wrote:
> Thanks Boyang,
>
> I've thought about exposing time via metrics in
Thanks Boyang,
I've thought about exposing time via metrics in Streams. The tricky part
though is which layer of time we should expose: right now we have
task-level and partition-level stream time (what you suggested), and also
some processor internally maintain their own observed time. Today we
I mean the partition time.
On Thu, Jul 18, 2019 at 11:29 AM Guozhang Wang wrote:
> Hi Boyang,
>
> What do you mean by `per partition latency`?
>
> Guozhang
>
> On Mon, Jul 1, 2019 at 9:28 AM Boyang Chen
> wrote:
>
> > Hey Guozhang,
> >
> > do we plan to add per partition latency in this KIP?
>
Hi Boyang,
What do you mean by `per partition latency`?
Guozhang
On Mon, Jul 1, 2019 at 9:28 AM Boyang Chen
wrote:
> Hey Guozhang,
>
> do we plan to add per partition latency in this KIP?
>
> On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna wrote:
>
> > Hi Guozhang,
> >
> > Thank you for the
Good catch! Will update the wiki.
On Wed, Jul 3, 2019 at 6:19 AM Bruno Cadonna wrote:
> Hi Guozhang,
>
> I just noticed that the Per-State-Store tags are somehow mixed up in
> the KIP (e.g "client-id=[threadId]").
>
> Best,
> Bruno
>
> On Mon, Jul 1, 2019 at 6:28 PM Boyang Chen
> wrote:
> >
>
Hello Bruno,
Thanks for the feedbacks, replied inline.
On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna wrote:
> Hi Guozhang,
>
> Thank you for the KIP.
>
> 1) As far as I understand, the StreamsMetrics interface is there for
> user-defined processors. Would it make sense to also add a method to
>
Hi Guozhang,
I just noticed that the Per-State-Store tags are somehow mixed up in
the KIP (e.g "client-id=[threadId]").
Best,
Bruno
On Mon, Jul 1, 2019 at 6:28 PM Boyang Chen wrote:
>
> Hey Guozhang,
>
> do we plan to add per partition latency in this KIP?
>
> On Mon, Jul 1, 2019 at 7:08 AM
Hey Guozhang,
do we plan to add per partition latency in this KIP?
On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna wrote:
> Hi Guozhang,
>
> Thank you for the KIP.
>
> 1) As far as I understand, the StreamsMetrics interface is there for
> user-defined processors. Would it make sense to also add a
Hi Guozhang,
Thank you for the KIP.
1) As far as I understand, the StreamsMetrics interface is there for
user-defined processors. Would it make sense to also add a method to
the interface to specify a sensor that records skipped records?
2) What are the semantics of active-task-process and
Hello folks,
As 2.3 is released now, I'd like to bump up this KIP discussion again for
your reviews.
Guozhang
On Thu, May 23, 2019 at 4:44 PM Guozhang Wang wrote:
> Hello Patrik,
>
> Since we are rolling out 2.3 and everyone is busy with the release now
> this KIP does not have much
Hello Patrik,
Since we are rolling out 2.3 and everyone is busy with the release now this
KIP does not have much discussion involved yet and will slip into the next
release cadence.
This KIP itself contains several parts itself: 1. refactoring the existing
metrics hierarchy to cleanup some
Hi Guozhang
Thanks for the KIP, this looks very helpful.
Could you please provide more detail on the metrics planned for the state?
We were just considering how to implement this ourselves because we need to
track the history of stage changes.
The idea was to have an accumulated "seconds in state
There was a discussion about metric on KIP-439. Would be good if we
could incorporate those thought into this discussion.
-Matthias
On 3/29/19 10:56 AM, Guozhang Wang wrote:
> Hello folks,
>
> I'd like to propose the following KIP to improve the Kafka Streams metrics
> mechanism to users. This
Hello folks,
I'd like to propose the following KIP to improve the Kafka Streams metrics
mechanism to users. This includes 1) a minor change in the public
StreamsMetrics API, and 2) a major cleanup on the Streams' own built-in
metrics hierarchy.
Details can be found here:
32 matches
Mail list logo
