Re: [VOTE] KIP-904: Kafka Streams - Guarantee subtractor is called before adder if key has not changed
I will close the vote now. The results are 3 binding votes (Guozhang, Walker, Matthias) and 1 non-binding vote (Lucas). Therefore the vote on KIP-904 passes. Next steps are to revise the draft PR (https://github.com/apache/kafka/pull/10747) and go from there. Thank you everyone! Farooq On 2023/03/03 10:13:37 Lucas Brutschy wrote: > +1 (non-binding) > > Thanks! > > On Wed, Mar 1, 2023 at 9:03 PM Matthias J. Sax wrote: > > > > +1 (binding) > > > > Thanks for the KIP! > > > > On 3/1/23 10:58 AM, Walker Carlson wrote: > > > +1 Binding > > > > > > On Mon, Feb 27, 2023 at 12:48 PM Guozhang Wang > > > wrote: > > > > > >> +1. > > >> > > >> On Sun, Feb 26, 2023 at 4:27 PM Fq Public wrote: > > >>> > > >>> Hi everyone, > > >>> > > >>> I'd like to start the vote on KIP-904: Kafka Streams - Guarantee > > >> subtractor > > >>> is called before adder if key has not changed. > > >>> The KIP is available here: https://cwiki.apache.org/confluence/x/P5VbDg > > >>> The easiest way to view the entire discussion thread is via this search > > >>> link: https://lists.apache.org/list?dev@kafka.apache.org:lte=1M:KIP-904 > > >>> Please take a look and vote. > > >>> > > >>> Thank you, > > >>> Farooq > > >> > > > >
RE: Re: [DISCUSS] KIP-904 Kafka Streams - Guarantee subtractor is called before adder if key has not changed
I have started the voting thread here: https://lists.apache.org/thread/pn846910dovg3d0z3k8pmq5opj0tb9w5 Please vote :) Thanks, Farooq On 2023/02/23 07:27:14 "Matthias J. Sax" wrote: > Thanks for the KIP. Overall LGTM. > > I think you can start a VOTE. > > > -Matthias > > On 2/22/23 5:56 PM, Fq Public wrote: > > Just wanted to bump this thread for visbility. > > Thanks to everyone who has participated in the discussion so far. > > > > Thanks, > > Farooq > > > > On 2023/02/14 19:32:53 Guozhang Wang wrote: > >> Thanks Farooq, that looks good to me. > >> > >> Guozhang > >> > >> On Sun, Feb 12, 2023 at 9:01 AM Dharin Shah wrote: > >>> > >>> Hello Farooq, > >>> > >>> This is actually a great idea, we have dealt with this by using an array > >>> instead of a set. > >>> +1 to this :) > >>> > >>> Thank you, > >>> Dharin > >>> > >>> On Sun, Feb 12, 2023 at 8:11 PM Fq Public wrote: > >>> > >>>> Hi Guozhang, > >>>> > >>>> Thanks for reading over my proposal! > >>>> > >>>>> Regarding the format, I'm just thinking if we can change the type of > >>>> "INT newDataLength" to UINT32? > >>>> > >>>> Good idea, I've updated the KIP to reflect UINT32 since it makes clear > >>>> the > >>>> value can never be less than zero. > >>>> > >>>>> `.equals` default implementation on Object is by reference, so if the > >>>> groupBy did not generate a new object, that may still pass. This means > >>>> that > >>>> even if user does not implement the `.equals` function, if the same > >>>> object > >>>> is returned then this feature would still be triggered, is that correct? > >>>> > >>>> Correct, I've updated the KIP to call out this edge-case clearly as > >>>> follows: > >>>> > >>>>> Since the default `.equals` implementation for an `Object` is by > >>>> reference, if a user's `groupBy` returns the same reference for the key, > >>>> then the oldKey and the newKey will naturally `.equals` each other. This > >>>> will result in a single event being sent to the repartition topic. This > >>>> change in behaviour should be considered a "bug-fix" rather than a > >>>> "breaking change" as the semantics of the operation remain unchanged, the > >>>> only thing that changes for users is they no longer see transient > >>>> "inconsistent" states. In the worst case, users in this situation will > >>>> need to update any strict tests that check specifically for the presence > >>>> of > >>>> transient "inconsistent" states. > >>>> > >>>> What do you think? > >>>> > >>>> Thanks, > >>>> Farooq > >>>> > >>>> On 2023/02/07 18:02:24 Guozhang Wang wrote: > >>>>> Hello Farooq, > >>>>> > >>>>> Thanks for the very detailed proposal! I think this is a great idea. > >>>>> Just a few thoughts: > >>>>> > >>>>> 1. I regret that we over-optimized the Changed serde format for > >>>>> footprint while making it less extensible. It seems to me that a two > >>>>> rolling bounce migration is unavoidable.. Regarding the format, I'm > >>>>> just thinking if we can change the type of "INT newDataLength" to > >>>>> UINT32? > >>>>> > >>>>> 2. `.equals` default implementation on Object is by reference, so if > >>>>> the groupBy did not generate a new object, that may still pass. This > >>>>> means that even if user does not implement the `.equals` function, if > >>>>> the same object is returned then this feature would still be > >>>>> triggered, is that correct? > >>>>> > >>>>> > >>>>> Guozhang > >>>>> > >>>>> On Mon, Feb 6, 2023 at 5:05 AM Fq Public wrote: > >>>>>> > >>>>>> Hi everyone, > >>>>>> > >>>>>> I'd like to share a new KIP for discussion: > >>>>>> https://cwiki.apache.org/confluence/x/P5VbDg > >>>>>> > >>>>>> This could be considered mostly as a "bug fix" but we wanted to raise > >>>> a KIP > >>>>>> for discussion because it involves changes to the serialization format > >>>> of > >>>>>> an internal topic which raises backward compatibility considerations. > >>>>>> > >>>>>> Please take a look and let me know what you think. > >>>>>> > >>>>>> Thanks, > >>>>>> Farooq > >>>>> >
[VOTE] KIP-904: Kafka Streams - Guarantee subtractor is called before adder if key has not changed
Hi everyone, I'd like to start the vote on KIP-904: Kafka Streams - Guarantee subtractor is called before adder if key has not changed. The KIP is available here: https://cwiki.apache.org/confluence/x/P5VbDg The easiest way to view the entire discussion thread is via this search link: https://lists.apache.org/list?dev@kafka.apache.org:lte=1M:KIP-904 Please take a look and vote. Thank you, Farooq
RE: Re: Re: [DISCUSS] KIP-904 Kafka Streams - Guarantee subtractor is called before adder if key has not changed
Just wanted to bump this thread for visbility. Thanks to everyone who has participated in the discussion so far. Thanks, Farooq On 2023/02/14 19:32:53 Guozhang Wang wrote: > Thanks Farooq, that looks good to me. > > Guozhang > > On Sun, Feb 12, 2023 at 9:01 AM Dharin Shah wrote: > > > > Hello Farooq, > > > > This is actually a great idea, we have dealt with this by using an array > > instead of a set. > > +1 to this :) > > > > Thank you, > > Dharin > > > > On Sun, Feb 12, 2023 at 8:11 PM Fq Public wrote: > > > > > Hi Guozhang, > > > > > > Thanks for reading over my proposal! > > > > > > > Regarding the format, I'm just thinking if we can change the type of > > > "INT newDataLength" to UINT32? > > > > > > Good idea, I've updated the KIP to reflect UINT32 since it makes clear the > > > value can never be less than zero. > > > > > > > `.equals` default implementation on Object is by reference, so if the > > > groupBy did not generate a new object, that may still pass. This means > > > that > > > even if user does not implement the `.equals` function, if the same object > > > is returned then this feature would still be triggered, is that correct? > > > > > > Correct, I've updated the KIP to call out this edge-case clearly as > > > follows: > > > > > > > Since the default `.equals` implementation for an `Object` is by > > > reference, if a user's `groupBy` returns the same reference for the key, > > > then the oldKey and the newKey will naturally `.equals` each other. This > > > will result in a single event being sent to the repartition topic. This > > > change in behaviour should be considered a "bug-fix" rather than a > > > "breaking change" as the semantics of the operation remain unchanged, the > > > only thing that changes for users is they no longer see transient > > > "inconsistent" states. In the worst case, users in this situation will > > > need to update any strict tests that check specifically for the presence > > > of > > > transient "inconsistent" states. > > > > > > What do you think? > > > > > > Thanks, > > > Farooq > > > > > > On 2023/02/07 18:02:24 Guozhang Wang wrote: > > > > Hello Farooq, > > > > > > > > Thanks for the very detailed proposal! I think this is a great idea. > > > > Just a few thoughts: > > > > > > > > 1. I regret that we over-optimized the Changed serde format for > > > > footprint while making it less extensible. It seems to me that a two > > > > rolling bounce migration is unavoidable.. Regarding the format, I'm > > > > just thinking if we can change the type of "INT newDataLength" to > > > > UINT32? > > > > > > > > 2. `.equals` default implementation on Object is by reference, so if > > > > the groupBy did not generate a new object, that may still pass. This > > > > means that even if user does not implement the `.equals` function, if > > > > the same object is returned then this feature would still be > > > > triggered, is that correct? > > > > > > > > > > > > Guozhang > > > > > > > > On Mon, Feb 6, 2023 at 5:05 AM Fq Public wrote: > > > > > > > > > > Hi everyone, > > > > > > > > > > I'd like to share a new KIP for discussion: > > > > > https://cwiki.apache.org/confluence/x/P5VbDg > > > > > > > > > > This could be considered mostly as a "bug fix" but we wanted to raise > > > a KIP > > > > > for discussion because it involves changes to the serialization format > > > of > > > > > an internal topic which raises backward compatibility considerations. > > > > > > > > > > Please take a look and let me know what you think. > > > > > > > > > > Thanks, > > > > > Farooq > > > > >
RE: Re: [DISCUSS] KIP-904 Kafka Streams - Guarantee subtractor is called before adder if key has not changed
Hi Guozhang, Thanks for reading over my proposal! > Regarding the format, I'm just thinking if we can change the type of "INT > newDataLength" to UINT32? Good idea, I've updated the KIP to reflect UINT32 since it makes clear the value can never be less than zero. > `.equals` default implementation on Object is by reference, so if the groupBy > did not generate a new object, that may still pass. This means that even if > user does not implement the `.equals` function, if the same object is > returned then this feature would still be triggered, is that correct? Correct, I've updated the KIP to call out this edge-case clearly as follows: > Since the default `.equals` implementation for an `Object` is by reference, > if a user's `groupBy` returns the same reference for the key, then the oldKey > and the newKey will naturally `.equals` each other. This will result in a > single event being sent to the repartition topic. This change in behaviour > should be considered a "bug-fix" rather than a "breaking change" as the > semantics of the operation remain unchanged, the only thing that changes for > users is they no longer see transient "inconsistent" states. In the worst > case, users in this situation will need to update any strict tests that check > specifically for the presence of transient "inconsistent" states. What do you think? Thanks, Farooq On 2023/02/07 18:02:24 Guozhang Wang wrote: > Hello Farooq, > > Thanks for the very detailed proposal! I think this is a great idea. > Just a few thoughts: > > 1. I regret that we over-optimized the Changed serde format for > footprint while making it less extensible. It seems to me that a two > rolling bounce migration is unavoidable.. Regarding the format, I'm > just thinking if we can change the type of "INT newDataLength" to > UINT32? > > 2. `.equals` default implementation on Object is by reference, so if > the groupBy did not generate a new object, that may still pass. This > means that even if user does not implement the `.equals` function, if > the same object is returned then this feature would still be > triggered, is that correct? > > > Guozhang > > On Mon, Feb 6, 2023 at 5:05 AM Fq Public wrote: > > > > Hi everyone, > > > > I'd like to share a new KIP for discussion: > > https://cwiki.apache.org/confluence/x/P5VbDg > > > > This could be considered mostly as a "bug fix" but we wanted to raise a KIP > > for discussion because it involves changes to the serialization format of > > an internal topic which raises backward compatibility considerations. > > > > Please take a look and let me know what you think. > > > > Thanks, > > Farooq >
[DISCUSS] KIP-904 Kafka Streams - Guarantee subtractor is called before adder if key has not changed
Hi everyone, I'd like to share a new KIP for discussion: https://cwiki.apache.org/confluence/x/P5VbDg This could be considered mostly as a "bug fix" but we wanted to raise a KIP for discussion because it involves changes to the serialization format of an internal topic which raises backward compatibility considerations. Please take a look and let me know what you think. Thanks, Farooq
RE: Re: Request for JIRA account creation
Hi Luke, Sorry for the trouble. Here are some alternative usernames (in order of decreasing preference): fqpublic fqoss fqsnowstorm fqfuchsia Thanks, Farooq On 2023/02/05 06:29:54 Luke Chen wrote: > Hi Farooq, > > Your wiki ID is all set. > But for the JIRA account, the user name cannot allow the special character: > ".". > Could you provide another user name without special characters? > > Thank you. > Luke > > On Sun, Feb 5, 2023 at 2:02 AM Fq Public wrote: > > > Also, I would like permission to contribute to Apache Kafka. > > My wiki ID is also fq.public5 > > > > Thanks, > > Farooq > > > > On Sat, 4 Feb 2023 at 12:23, Fq Public wrote: > > > > > Hi, I would like to request a JIRA account in order to be able to file a > > > KIP and tickets related to this PR: > > > https://github.com/apache/kafka/pull/10747 > > > > > > Email: fq.publ...@gmail.com > > > User name: fq.public5 > > > Display name: Farooq Qaiser > > > > > > Thanks, > > > Farooq > > > > > >
Request for JIRA account creation
Hi, I would like to request a JIRA account in order to be able to file a KIP and tickets related to this PR: https://github.com/apache/kafka/pull/10747 Email: fq.publ...@gmail.com User name: fq.public5 Display name: Farooq Qaiser Thanks, Farooq
Re: Request for JIRA account creation
Also, I would like permission to contribute to Apache Kafka. My wiki ID is also fq.public5 Thanks, Farooq On Sat, 4 Feb 2023 at 12:23, Fq Public wrote: > Hi, I would like to request a JIRA account in order to be able to file a > KIP and tickets related to this PR: > https://github.com/apache/kafka/pull/10747 > > Email: fq.publ...@gmail.com > User name: fq.public5 > Display name: Farooq Qaiser > > Thanks, > Farooq >
