Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Hello everyone, The KIP-862 vote has passed with: binding +1s (John, Guozhang, Bruno) non-binding +1s (Jim) Thank you everyone for reviewing the KIP and voting. Best, Vicky On Fri, Sep 16, 2022 at 11:44 AM Bruno Cadonna wrote: > Hi Vicky, > > Thanks for the KIP! > > I think the KIP looks good! > You described how the self-join is optimized when the names of the state > stores are automatically generated by Streams. I think for completeness > you should also mention what happens when users explicitly name the > state stores of the self-join and give an example. > > For the rest, I am +1 (binding). > > Best, > Bruno > > > On 13.09.22 22:50, Jim Hughes wrote: > > Hi Vicky, > > > > I'm +1 (non-binding); thanks for the KIP (and PR)! > > > > Cheers, > > > > Jim > > > > On Tue, Sep 13, 2022 at 12:05 PM Guozhang Wang > wrote: > > > >> Thank Vicky! I'm +1. > >> > >> Guozhang > >> > >> On Mon, Sep 12, 2022 at 7:02 PM John Roesler > wrote: > >> > >>> Thanks for the updates, Vicky! > >>> > >>> I've reviewed the KIP and your POC PR, > >>> and I'm +1 (binding). > >>> > >>> Thanks! > >>> -John > >>> > >>> On Mon, Sep 12, 2022, at 09:13, Vasiliki Papavasileiou wrote: > Hey Guozhang, > > Great suggestion, I made the change. > > Best, > Vicky > > On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang > >>> wrote: > > > Thanks Vicky, that reads much clearer now. > > > > Just regarding the value string name itself: "self.join" may be > >>> confusing > > compared to other values that people would think before this config > is > > enabled, self-join are not allowed at all. Maybe we can rename it to > > "single.store.self.join"? > > > > Guozhang > > > > On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou > > wrote: > > > >> Hey Guozhang, > >> > >> Ah it seems my text was not very clear :) > >> With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list > >>> of > >> optimization rule configs" I meant that it will accept the new value > >> strings for each optimization rule. Let me rephrase that in the KIP > >> to > > make > >> it clearer. > >> Is it better now? > >> > >> Best, > >> Vicky > >> > >> On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang > >>> wrote: > >> > >>> Thanks Vicky, > >>> > >>> I read through the KIP again and it looks good to me. Just a quick > >> question > >>> regarding the public config changes: you mentioned "No public > > interfaces > >>> will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be > >> extended > >>> to accept a list of optimization rule configs in addition to the > >>> global > >>> values "all" and "none" . But there are no new value strings > >>> mentioned > > in > >>> this KIP, so that means we will apply this optimization only when > >>> `all` > >> is > >>> specified in the config right? > >>> > >>> > >>> Guozhang > >>> > >>> > >>> On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou > >>> wrote: > >>> > Hello everyone, > > I'd like to open the vote for KIP-862, which proposes to > >> optimize > stream-stream self-joins by using a single state store for the > >>> join. > > The proposal is here: > > > >>> > >> > > > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > > Thanks to all who reviewed the proposal, and thanks in advance > >> for > >> taking > the time to vote! > > Thank you, > Vicky > > >>> > >>> > >>> -- > >>> -- Guozhang > >>> > >> > > > > > > -- > > -- Guozhang > > > >>> > >> > >> > >> -- > >> -- Guozhang > >> > > >
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Hi Vicky, Thanks for the KIP! I think the KIP looks good! You described how the self-join is optimized when the names of the state stores are automatically generated by Streams. I think for completeness you should also mention what happens when users explicitly name the state stores of the self-join and give an example. For the rest, I am +1 (binding). Best, Bruno On 13.09.22 22:50, Jim Hughes wrote: Hi Vicky, I'm +1 (non-binding); thanks for the KIP (and PR)! Cheers, Jim On Tue, Sep 13, 2022 at 12:05 PM Guozhang Wang wrote: Thank Vicky! I'm +1. Guozhang On Mon, Sep 12, 2022 at 7:02 PM John Roesler wrote: Thanks for the updates, Vicky! I've reviewed the KIP and your POC PR, and I'm +1 (binding). Thanks! -John On Mon, Sep 12, 2022, at 09:13, Vasiliki Papavasileiou wrote: Hey Guozhang, Great suggestion, I made the change. Best, Vicky On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang wrote: Thanks Vicky, that reads much clearer now. Just regarding the value string name itself: "self.join" may be confusing compared to other values that people would think before this config is enabled, self-join are not allowed at all. Maybe we can rename it to "single.store.self.join"? Guozhang On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou wrote: Hey Guozhang, Ah it seems my text was not very clear :) With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of optimization rule configs" I meant that it will accept the new value strings for each optimization rule. Let me rephrase that in the KIP to make it clearer. Is it better now? Best, Vicky On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang wrote: Thanks Vicky, I read through the KIP again and it looks good to me. Just a quick question regarding the public config changes: you mentioned "No public interfaces will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of optimization rule configs in addition to the global values "all" and "none" . But there are no new value strings mentioned in this KIP, so that means we will apply this optimization only when `all` is specified in the config right? Guozhang On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou wrote: Hello everyone, I'd like to open the vote for KIP-862, which proposes to optimize stream-stream self-joins by using a single state store for the join. The proposal is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins Thanks to all who reviewed the proposal, and thanks in advance for taking the time to vote! Thank you, Vicky -- -- Guozhang -- -- Guozhang -- -- Guozhang
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Hi Vicky, I'm +1 (non-binding); thanks for the KIP (and PR)! Cheers, Jim On Tue, Sep 13, 2022 at 12:05 PM Guozhang Wang wrote: > Thank Vicky! I'm +1. > > Guozhang > > On Mon, Sep 12, 2022 at 7:02 PM John Roesler wrote: > > > Thanks for the updates, Vicky! > > > > I've reviewed the KIP and your POC PR, > > and I'm +1 (binding). > > > > Thanks! > > -John > > > > On Mon, Sep 12, 2022, at 09:13, Vasiliki Papavasileiou wrote: > > > Hey Guozhang, > > > > > > Great suggestion, I made the change. > > > > > > Best, > > > Vicky > > > > > > On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang > > wrote: > > > > > >> Thanks Vicky, that reads much clearer now. > > >> > > >> Just regarding the value string name itself: "self.join" may be > > confusing > > >> compared to other values that people would think before this config is > > >> enabled, self-join are not allowed at all. Maybe we can rename it to > > >> "single.store.self.join"? > > >> > > >> Guozhang > > >> > > >> On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou > > >> wrote: > > >> > > >> > Hey Guozhang, > > >> > > > >> > Ah it seems my text was not very clear :) > > >> > With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list > > of > > >> > optimization rule configs" I meant that it will accept the new value > > >> > strings for each optimization rule. Let me rephrase that in the KIP > to > > >> make > > >> > it clearer. > > >> > Is it better now? > > >> > > > >> > Best, > > >> > Vicky > > >> > > > >> > On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang > > wrote: > > >> > > > >> > > Thanks Vicky, > > >> > > > > >> > > I read through the KIP again and it looks good to me. Just a quick > > >> > question > > >> > > regarding the public config changes: you mentioned "No public > > >> interfaces > > >> > > will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be > > >> > extended > > >> > > to accept a list of optimization rule configs in addition to the > > global > > >> > > values "all" and "none" . But there are no new value strings > > mentioned > > >> in > > >> > > this KIP, so that means we will apply this optimization only when > > `all` > > >> > is > > >> > > specified in the config right? > > >> > > > > >> > > > > >> > > Guozhang > > >> > > > > >> > > > > >> > > On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou > > >> > > wrote: > > >> > > > > >> > > > Hello everyone, > > >> > > > > > >> > > > I'd like to open the vote for KIP-862, which proposes to > optimize > > >> > > > stream-stream self-joins by using a single state store for the > > join. > > >> > > > > > >> > > > The proposal is here: > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > > >> > > > > > >> > > > Thanks to all who reviewed the proposal, and thanks in advance > for > > >> > taking > > >> > > > the time to vote! > > >> > > > > > >> > > > Thank you, > > >> > > > Vicky > > >> > > > > > >> > > > > >> > > > > >> > > -- > > >> > > -- Guozhang > > >> > > > > >> > > > >> > > >> > > >> -- > > >> -- Guozhang > > >> > > > > > -- > -- Guozhang >
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Thank Vicky! I'm +1. Guozhang On Mon, Sep 12, 2022 at 7:02 PM John Roesler wrote: > Thanks for the updates, Vicky! > > I've reviewed the KIP and your POC PR, > and I'm +1 (binding). > > Thanks! > -John > > On Mon, Sep 12, 2022, at 09:13, Vasiliki Papavasileiou wrote: > > Hey Guozhang, > > > > Great suggestion, I made the change. > > > > Best, > > Vicky > > > > On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang > wrote: > > > >> Thanks Vicky, that reads much clearer now. > >> > >> Just regarding the value string name itself: "self.join" may be > confusing > >> compared to other values that people would think before this config is > >> enabled, self-join are not allowed at all. Maybe we can rename it to > >> "single.store.self.join"? > >> > >> Guozhang > >> > >> On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou > >> wrote: > >> > >> > Hey Guozhang, > >> > > >> > Ah it seems my text was not very clear :) > >> > With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list > of > >> > optimization rule configs" I meant that it will accept the new value > >> > strings for each optimization rule. Let me rephrase that in the KIP to > >> make > >> > it clearer. > >> > Is it better now? > >> > > >> > Best, > >> > Vicky > >> > > >> > On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang > wrote: > >> > > >> > > Thanks Vicky, > >> > > > >> > > I read through the KIP again and it looks good to me. Just a quick > >> > question > >> > > regarding the public config changes: you mentioned "No public > >> interfaces > >> > > will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be > >> > extended > >> > > to accept a list of optimization rule configs in addition to the > global > >> > > values "all" and "none" . But there are no new value strings > mentioned > >> in > >> > > this KIP, so that means we will apply this optimization only when > `all` > >> > is > >> > > specified in the config right? > >> > > > >> > > > >> > > Guozhang > >> > > > >> > > > >> > > On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou > >> > > wrote: > >> > > > >> > > > Hello everyone, > >> > > > > >> > > > I'd like to open the vote for KIP-862, which proposes to optimize > >> > > > stream-stream self-joins by using a single state store for the > join. > >> > > > > >> > > > The proposal is here: > >> > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > >> > > > > >> > > > Thanks to all who reviewed the proposal, and thanks in advance for > >> > taking > >> > > > the time to vote! > >> > > > > >> > > > Thank you, > >> > > > Vicky > >> > > > > >> > > > >> > > > >> > > -- > >> > > -- Guozhang > >> > > > >> > > >> > >> > >> -- > >> -- Guozhang > >> > -- -- Guozhang
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Thanks for the updates, Vicky! I've reviewed the KIP and your POC PR, and I'm +1 (binding). Thanks! -John On Mon, Sep 12, 2022, at 09:13, Vasiliki Papavasileiou wrote: > Hey Guozhang, > > Great suggestion, I made the change. > > Best, > Vicky > > On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang wrote: > >> Thanks Vicky, that reads much clearer now. >> >> Just regarding the value string name itself: "self.join" may be confusing >> compared to other values that people would think before this config is >> enabled, self-join are not allowed at all. Maybe we can rename it to >> "single.store.self.join"? >> >> Guozhang >> >> On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou >> wrote: >> >> > Hey Guozhang, >> > >> > Ah it seems my text was not very clear :) >> > With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of >> > optimization rule configs" I meant that it will accept the new value >> > strings for each optimization rule. Let me rephrase that in the KIP to >> make >> > it clearer. >> > Is it better now? >> > >> > Best, >> > Vicky >> > >> > On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang wrote: >> > >> > > Thanks Vicky, >> > > >> > > I read through the KIP again and it looks good to me. Just a quick >> > question >> > > regarding the public config changes: you mentioned "No public >> interfaces >> > > will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be >> > extended >> > > to accept a list of optimization rule configs in addition to the global >> > > values "all" and "none" . But there are no new value strings mentioned >> in >> > > this KIP, so that means we will apply this optimization only when `all` >> > is >> > > specified in the config right? >> > > >> > > >> > > Guozhang >> > > >> > > >> > > On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou >> > > wrote: >> > > >> > > > Hello everyone, >> > > > >> > > > I'd like to open the vote for KIP-862, which proposes to optimize >> > > > stream-stream self-joins by using a single state store for the join. >> > > > >> > > > The proposal is here: >> > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins >> > > > >> > > > Thanks to all who reviewed the proposal, and thanks in advance for >> > taking >> > > > the time to vote! >> > > > >> > > > Thank you, >> > > > Vicky >> > > > >> > > >> > > >> > > -- >> > > -- Guozhang >> > > >> > >> >> >> -- >> -- Guozhang >>
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Hey Guozhang, Great suggestion, I made the change. Best, Vicky On Fri, Sep 9, 2022 at 10:43 PM Guozhang Wang wrote: > Thanks Vicky, that reads much clearer now. > > Just regarding the value string name itself: "self.join" may be confusing > compared to other values that people would think before this config is > enabled, self-join are not allowed at all. Maybe we can rename it to > "single.store.self.join"? > > Guozhang > > On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou > wrote: > > > Hey Guozhang, > > > > Ah it seems my text was not very clear :) > > With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of > > optimization rule configs" I meant that it will accept the new value > > strings for each optimization rule. Let me rephrase that in the KIP to > make > > it clearer. > > Is it better now? > > > > Best, > > Vicky > > > > On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang wrote: > > > > > Thanks Vicky, > > > > > > I read through the KIP again and it looks good to me. Just a quick > > question > > > regarding the public config changes: you mentioned "No public > interfaces > > > will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be > > extended > > > to accept a list of optimization rule configs in addition to the global > > > values "all" and "none" . But there are no new value strings mentioned > in > > > this KIP, so that means we will apply this optimization only when `all` > > is > > > specified in the config right? > > > > > > > > > Guozhang > > > > > > > > > On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou > > > wrote: > > > > > > > Hello everyone, > > > > > > > > I'd like to open the vote for KIP-862, which proposes to optimize > > > > stream-stream self-joins by using a single state store for the join. > > > > > > > > The proposal is here: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > > > > > > > > Thanks to all who reviewed the proposal, and thanks in advance for > > taking > > > > the time to vote! > > > > > > > > Thank you, > > > > Vicky > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > -- > -- Guozhang >
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Thanks Vicky, that reads much clearer now. Just regarding the value string name itself: "self.join" may be confusing compared to other values that people would think before this config is enabled, self-join are not allowed at all. Maybe we can rename it to "single.store.self.join"? Guozhang On Fri, Sep 9, 2022 at 2:15 AM Vasiliki Papavasileiou wrote: > Hey Guozhang, > > Ah it seems my text was not very clear :) > With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of > optimization rule configs" I meant that it will accept the new value > strings for each optimization rule. Let me rephrase that in the KIP to make > it clearer. > Is it better now? > > Best, > Vicky > > On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang wrote: > > > Thanks Vicky, > > > > I read through the KIP again and it looks good to me. Just a quick > question > > regarding the public config changes: you mentioned "No public interfaces > > will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be > extended > > to accept a list of optimization rule configs in addition to the global > > values "all" and "none" . But there are no new value strings mentioned in > > this KIP, so that means we will apply this optimization only when `all` > is > > specified in the config right? > > > > > > Guozhang > > > > > > On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou > > wrote: > > > > > Hello everyone, > > > > > > I'd like to open the vote for KIP-862, which proposes to optimize > > > stream-stream self-joins by using a single state store for the join. > > > > > > The proposal is here: > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > > > > > > Thanks to all who reviewed the proposal, and thanks in advance for > taking > > > the time to vote! > > > > > > Thank you, > > > Vicky > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Hey Guozhang, Ah it seems my text was not very clear :) With "TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of optimization rule configs" I meant that it will accept the new value strings for each optimization rule. Let me rephrase that in the KIP to make it clearer. Is it better now? Best, Vicky On Thu, Sep 8, 2022 at 9:07 PM Guozhang Wang wrote: > Thanks Vicky, > > I read through the KIP again and it looks good to me. Just a quick question > regarding the public config changes: you mentioned "No public interfaces > will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be extended > to accept a list of optimization rule configs in addition to the global > values "all" and "none" . But there are no new value strings mentioned in > this KIP, so that means we will apply this optimization only when `all` is > specified in the config right? > > > Guozhang > > > On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou > wrote: > > > Hello everyone, > > > > I'd like to open the vote for KIP-862, which proposes to optimize > > stream-stream self-joins by using a single state store for the join. > > > > The proposal is here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > > > > Thanks to all who reviewed the proposal, and thanks in advance for taking > > the time to vote! > > > > Thank you, > > Vicky > > > > > -- > -- Guozhang >
Re: [VOTE] KIP-862: Self-join optimization for stream-stream joins
Thanks Vicky, I read through the KIP again and it looks good to me. Just a quick question regarding the public config changes: you mentioned "No public interfaces will be impacted. The config TOPOLOGY_OPTIMIZATION_CONFIG will be extended to accept a list of optimization rule configs in addition to the global values "all" and "none" . But there are no new value strings mentioned in this KIP, so that means we will apply this optimization only when `all` is specified in the config right? Guozhang On Thu, Sep 8, 2022 at 12:02 PM Vasiliki Papavasileiou wrote: > Hello everyone, > > I'd like to open the vote for KIP-862, which proposes to optimize > stream-stream self-joins by using a single state store for the join. > > The proposal is here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins > > Thanks to all who reviewed the proposal, and thanks in advance for taking > the time to vote! > > Thank you, > Vicky > -- -- Guozhang
[VOTE] KIP-862: Self-join optimization for stream-stream joins
Hello everyone, I'd like to open the vote for KIP-862, which proposes to optimize stream-stream self-joins by using a single state store for the join. The proposal is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-862%3A+Self-join+optimization+for+stream-stream+joins Thanks to all who reviewed the proposal, and thanks in advance for taking the time to vote! Thank you, Vicky
