Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-06-30 Thread Matthias J. Sax
Thanks for the update Hao. Seems it's the best way forward. As we don't add it to `ReadOnlySessionStore` it's not exposed via IQ and thus the impact seems to be well contained. No concerns from my side. -Matthias On 6/29/22 11:51 AM, Hao Li wrote: Hi Dev Team, Just want to inform you

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-06-29 Thread Hao Li
Hi Dev Team, Just want to inform you here that we made an update to this KIP. To implement emit final in Session Window aggregation, we augment the `SessionStore` by adding an public API to range query from earliest session end time to latest session end time. API is like this: ``` public

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-15 Thread Hao Li
Hi Guozhang, Thanks for the suggestion. 1. I have no problem with window-aggregate-final-emit-(rate | total) and window-aggregate-final-emit-latency-(avg | max) 2. Good question, it should be the time to emit final when it COULD be emitted. Will update KIP and PR. Hao On Thu, Apr 14, 2022

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-14 Thread Guozhang Wang
Hello Hao, Just a bit further suggestion to align with KIP-444 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams): in the suppression processor node we have an existing metric as follows: suppression-emit (rate | total) So I'm feeling maybe we can

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-12 Thread Hao Li
Thanks for the feedback Bruno and John. 1. Proposed name by John sounds good to me! 2. I will use processor-node metrics with debug level since this doesn't seem like top level metrics and make more sense to not mix them in task. 3. Will update KIP as Bruno suggested. Thanks, Hao On Tue, Apr

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-12 Thread Bruno Cadonna
Hi Hao, Thanks for the addition! I second what John said about the naming. Could you please describe the metrics as has been previously done in KIP-471, KIP-613, or KIP-761? That would make the metrics more concise and clear. In addition, TaskMetrics is an internal class that is an

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-11 Thread John Roesler
Thanks, Hao! I have no concern about amending the KIP to add metrics. Thanks for thinking of it. Can you comment on the choice to add them as a task-level metric instead of a processor-level metric? This will cause the metrics for all windowed aggregations in a task that use final emission to be

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-04-11 Thread Hao Li
Hi all, I would like to introduce two metrics in this KIP as well to measure the latency and number of records emitted for emit final. They are named: `emit-final-latency` `emitted-records` I've updated the KIP with details in

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-25 Thread Hao Li
Got it. Forgot that. Yeah, it’s still open and ppl can still vote. Thanks for reminding! Hao Li > On Mar 25, 2022, at 8:22 AM, Guozhang Wang wrote: > > Hello Hao, > > According to bylaws the voting has to last for at least 72 business hours. > So let's wait a bit longer to see if there are

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-25 Thread Guozhang Wang
Hello Hao, According to bylaws the voting has to last for at least 72 business hours. So let's wait a bit longer to see if there are different opinions before calling it close. On Thu, Mar 24, 2022 at 4:20 PM Hao Li wrote: > The vote happened in the discussion thread since I started the vote

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Hao Li
The vote happened in the discussion thread since I started the vote there by mistake. But it passed there. To avoid having everyone vote again. I copied the content from that thread here: end of discussion thread vote == The vote passed with 5

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Hao Li
The vote passed with 5 binding votes from John, Guozhang, Bruno, Matthias and Bill. Thanks all for the feedback and vote. Hao On Thu, Mar 24, 2022 at 2:20 PM Bill Bejeck wrote: > Thanks for KIP Hao! > > Glad to see we settled on option 1 > > +1(binding) > > On Thu, Mar 24, 2022 at 5:13 PM

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Bill Bejeck
Thanks for KIP Hao! Glad to see we settled on option 1 +1(binding) On Thu, Mar 24, 2022 at 5:13 PM Matthias J. Sax wrote: > +1 (binding) > > > On 3/24/22 1:52 PM, Hao Li wrote: > > I hit reply on my phone in the mail app and changed the title and text > > hoping it will start a new thread.

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Matthias J. Sax
+1 (binding) On 3/24/22 1:52 PM, Hao Li wrote: I hit reply on my phone in the mail app and changed the title and text hoping it will start a new thread. Apparently it doesn't work... On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna wrote: Hi Hao, Actually, this is the VOTE thread. Do you

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
Thanks, Hao, I’m +1 (binding) -John On Thu, Mar 24, 2022, at 11:38, Hao Li wrote: > Hi all, > > I'd like to start a vote on Kafka Streams KIP-825: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced > > Thanks, > Hao

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Hao Li
I hit reply on my phone in the mail app and changed the title and text hoping it will start a new thread. Apparently it doesn't work... On Thu, Mar 24, 2022 at 12:36 PM Bruno Cadonna wrote: > Hi Hao, > > Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a > bit weird how it

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Bruno Cadonna
Hi Hao, Actually, this is the VOTE thread. Do you use GMail? Sometimes it is a bit weird how it shows e-mails in threads. Anyways, thank you for the KIP and your patience! +1 (binding) Best, Bruno On 24.03.22 17:36, Hao Li wrote: Thanks John and Guozhang. Didn't realize I used this

[VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Hao Li
Hi all, I'd like to start a vote on Kafka Streams KIP-825: https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced Thanks, Hao

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Hao Li
Thanks John and Guozhang. Didn't realize I used this discussion thread for voting. Let me start a new thread for voting. Will fix the KIP. On Thu, Mar 24, 2022 at 9:28 AM Guozhang Wang wrote: > +1 (binding). > > Thanks Hao! > > > Guozhang > > On Thu, Mar 24, 2022 at 9:20 AM John Roesler wrote:

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread Guozhang Wang
+1 (binding). Thanks Hao! Guozhang On Thu, Mar 24, 2022 at 9:20 AM John Roesler wrote: > Thanks, Hao! > > I'm +1 (binding) > > -John > > On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote: > > Hi all, > > > > I'd like to start a vote on Kafka Streams KIP-825: > > >

Re: [VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-24 Thread John Roesler
Thanks, Hao! I'm +1 (binding) -John On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote: > Hi all, > > I'd like to start a vote on Kafka Streams KIP-825: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced

[VOTE] KIP-825: introduce a new API to control when aggregated results are produced

2022-03-23 Thread Hao Li
Hi all, I'd like to start a vote on Kafka Streams KIP-825: https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced