Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hi all, The vote for KIP-1004 passes with the following +1 votes and no +0 or -1 votes: - Hector Geraldino - Mickael Maison (binding) - Greg Harris (binding) - Yash Mayya (binding) - Federico Valeri With regards to the open discussion about whether to remove the deprecated tasks.max.enforce property in 4.0.0 or later, I've tweaked the KIP to clearly state that it may take place in 4.0.0 but may also be delayed. A deprecated property does not require a KIP for removal, so we have some wiggle room should the discussion continue, especially if people feel strongly that we should push to remove it in time for 4.0.0. Thanks all for your votes and discussion! Cheers, Chris On Fri, Jan 5, 2024 at 3:45 PM Greg Harris wrote: > Hey Chris, > > Thanks for keeping KIP-987 in-mind. > > The current design of KIP-987 doesn't take tasks.max.enforce into > account, but I think it may be possible to only allow the protocol > upgrade when tasks.max.enforce is true if we were to try to enforce > it. It may also be reasonable to just have a warning about it appended > to the documentation string for tasks.max.enforce. > I am fine with either keeping or removing it in 4.0, leaning towards > keeping it, for the same reasons you listed above. > > Thanks! > Greg > > On Fri, Jan 5, 2024 at 9:40 AM Chris Egerton > wrote: > > > > Hi Yash, > > > > Thanks for raising the possibility of a more aggressive removal schedule > > for the tasks.max.enforce property now that it seems a 3.8.x branch is > > likely--I was wondering if someone would bring that up! > > > > I think I'd prefer to err on the side of caution and give users more time > > to adjust, since some may skip 3.8.x and upgrade to 4.0.x, 4.1.x, etc. > > directly instead. It seems like the maintenance cost will be fairly low, > > and with the option to programmatically require it to be set to true in > > order to work with other features we may want to develop in the future, > it > > shouldn't block any progress in the meantime. Thoughts? I'd also be > curious > > what Greg Harris thinks about this, given that it seems relevant to > KIP-987 > > [1]. > > > > [1] - > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-987%3A+Connect+Static+Assignments > > > > Cheers, > > > > Chris > > > > On Thu, Jan 4, 2024 at 4:45 AM Federico Valeri > wrote: > > > > > Thanks! This will finally reconcile Javadoc and implementation. > > > > > > +1 (non binding) > > > > > > On Thu, Jan 4, 2024 at 6:49 AM Yash Mayya > wrote: > > > > > > > > Hi Chris, > > > > > > > > +1 (binding), thanks for the KIP. > > > > > > > > Based on discussion in other threads, it looks like the community is > > > > aligned with having a 3.8 release before the 4.0 release so we > should be > > > > able to remove the 'tasks.max.enforce' connector property in 4.0 > (we'd > > > > discussed potentially having to live with this property until 5.0 in > this > > > > KIP's discussion thread). Once we have confirmation of a 3.8 release, > > > will > > > > this KIP be updated to reflect the exact AK versions where the > deprecated > > > > property will be introduced and removed? > > > > > > > > Thanks, > > > > Yash > > > > > > > > On Wed, Jan 3, 2024 at 11:37 PM Greg Harris > > > > > > > > wrote: > > > > > > > > > Hey Chris, > > > > > > > > > > Thanks for the KIP! I think the aggressive default and deprecation > > > > > schedule is the right choice for this change. > > > > > > > > > > +1 (binding) > > > > > > > > > > On Wed, Jan 3, 2024 at 9:01 AM Mickael Maison < > > > mickael.mai...@gmail.com> > > > > > wrote: > > > > > > > > > > > > Hi Chris, > > > > > > > > > > > > +1 (binding), thanks for the KIP > > > > > > > > > > > > Mickael > > > > > > > > > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 > 3RD > > > A) > > > > > > wrote: > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > Thanks Chris! > > > > > > > > > > > > > > From: dev@kafka.apache.org At: 01/02/24 11:
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hey Chris, Thanks for keeping KIP-987 in-mind. The current design of KIP-987 doesn't take tasks.max.enforce into account, but I think it may be possible to only allow the protocol upgrade when tasks.max.enforce is true if we were to try to enforce it. It may also be reasonable to just have a warning about it appended to the documentation string for tasks.max.enforce. I am fine with either keeping or removing it in 4.0, leaning towards keeping it, for the same reasons you listed above. Thanks! Greg On Fri, Jan 5, 2024 at 9:40 AM Chris Egerton wrote: > > Hi Yash, > > Thanks for raising the possibility of a more aggressive removal schedule > for the tasks.max.enforce property now that it seems a 3.8.x branch is > likely--I was wondering if someone would bring that up! > > I think I'd prefer to err on the side of caution and give users more time > to adjust, since some may skip 3.8.x and upgrade to 4.0.x, 4.1.x, etc. > directly instead. It seems like the maintenance cost will be fairly low, > and with the option to programmatically require it to be set to true in > order to work with other features we may want to develop in the future, it > shouldn't block any progress in the meantime. Thoughts? I'd also be curious > what Greg Harris thinks about this, given that it seems relevant to KIP-987 > [1]. > > [1] - > https://cwiki.apache.org/confluence/display/KAFKA/KIP-987%3A+Connect+Static+Assignments > > Cheers, > > Chris > > On Thu, Jan 4, 2024 at 4:45 AM Federico Valeri wrote: > > > Thanks! This will finally reconcile Javadoc and implementation. > > > > +1 (non binding) > > > > On Thu, Jan 4, 2024 at 6:49 AM Yash Mayya wrote: > > > > > > Hi Chris, > > > > > > +1 (binding), thanks for the KIP. > > > > > > Based on discussion in other threads, it looks like the community is > > > aligned with having a 3.8 release before the 4.0 release so we should be > > > able to remove the 'tasks.max.enforce' connector property in 4.0 (we'd > > > discussed potentially having to live with this property until 5.0 in this > > > KIP's discussion thread). Once we have confirmation of a 3.8 release, > > will > > > this KIP be updated to reflect the exact AK versions where the deprecated > > > property will be introduced and removed? > > > > > > Thanks, > > > Yash > > > > > > On Wed, Jan 3, 2024 at 11:37 PM Greg Harris > > > > > wrote: > > > > > > > Hey Chris, > > > > > > > > Thanks for the KIP! I think the aggressive default and deprecation > > > > schedule is the right choice for this change. > > > > > > > > +1 (binding) > > > > > > > > On Wed, Jan 3, 2024 at 9:01 AM Mickael Maison < > > mickael.mai...@gmail.com> > > > > wrote: > > > > > > > > > > Hi Chris, > > > > > > > > > > +1 (binding), thanks for the KIP > > > > > > > > > > Mickael > > > > > > > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD > > A) > > > > > wrote: > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > Thanks Chris! > > > > > > > > > > > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > > > > dev@kafka.apache.org > > > > > > Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka > > > > Connect > > > > > > > > > > > > Hi all, > > > > > > > > > > > > Happy New Year! Wanted to give this a bump now that the holidays > > are > > > > over > > > > > > for a lot of us. Looking forward to people's thoughts! > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Chris > > > > > > > > > > > > On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I'd like to call for a vote on KIP-1004, which adds enforcement > > for > > > > the > > > > > > > tasks.max connector property in Kafka Connect. > > > > > > > > > > > > > > The KIP: > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > > > > > > property+in+Kafka+Connect > > > > > > > > > > > > > > The discussion thread: > > > > > > > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hi Yash, Thanks for raising the possibility of a more aggressive removal schedule for the tasks.max.enforce property now that it seems a 3.8.x branch is likely--I was wondering if someone would bring that up! I think I'd prefer to err on the side of caution and give users more time to adjust, since some may skip 3.8.x and upgrade to 4.0.x, 4.1.x, etc. directly instead. It seems like the maintenance cost will be fairly low, and with the option to programmatically require it to be set to true in order to work with other features we may want to develop in the future, it shouldn't block any progress in the meantime. Thoughts? I'd also be curious what Greg Harris thinks about this, given that it seems relevant to KIP-987 [1]. [1] - https://cwiki.apache.org/confluence/display/KAFKA/KIP-987%3A+Connect+Static+Assignments Cheers, Chris On Thu, Jan 4, 2024 at 4:45 AM Federico Valeri wrote: > Thanks! This will finally reconcile Javadoc and implementation. > > +1 (non binding) > > On Thu, Jan 4, 2024 at 6:49 AM Yash Mayya wrote: > > > > Hi Chris, > > > > +1 (binding), thanks for the KIP. > > > > Based on discussion in other threads, it looks like the community is > > aligned with having a 3.8 release before the 4.0 release so we should be > > able to remove the 'tasks.max.enforce' connector property in 4.0 (we'd > > discussed potentially having to live with this property until 5.0 in this > > KIP's discussion thread). Once we have confirmation of a 3.8 release, > will > > this KIP be updated to reflect the exact AK versions where the deprecated > > property will be introduced and removed? > > > > Thanks, > > Yash > > > > On Wed, Jan 3, 2024 at 11:37 PM Greg Harris > > > wrote: > > > > > Hey Chris, > > > > > > Thanks for the KIP! I think the aggressive default and deprecation > > > schedule is the right choice for this change. > > > > > > +1 (binding) > > > > > > On Wed, Jan 3, 2024 at 9:01 AM Mickael Maison < > mickael.mai...@gmail.com> > > > wrote: > > > > > > > > Hi Chris, > > > > > > > > +1 (binding), thanks for the KIP > > > > > > > > Mickael > > > > > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD > A) > > > > wrote: > > > > > > > > > > +1 (non-binding) > > > > > > > > > > Thanks Chris! > > > > > > > > > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > > > dev@kafka.apache.org > > > > > Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka > > > Connect > > > > > > > > > > Hi all, > > > > > > > > > > Happy New Year! Wanted to give this a bump now that the holidays > are > > > over > > > > > for a lot of us. Looking forward to people's thoughts! > > > > > > > > > > Cheers, > > > > > > > > > > Chris > > > > > > > > > > On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'd like to call for a vote on KIP-1004, which adds enforcement > for > > > the > > > > > > tasks.max connector property in Kafka Connect. > > > > > > > > > > > > The KIP: > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > > > > > property+in+Kafka+Connect > > > > > > > > > > > > The discussion thread: > > > > > > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Thanks! This will finally reconcile Javadoc and implementation. +1 (non binding) On Thu, Jan 4, 2024 at 6:49 AM Yash Mayya wrote: > > Hi Chris, > > +1 (binding), thanks for the KIP. > > Based on discussion in other threads, it looks like the community is > aligned with having a 3.8 release before the 4.0 release so we should be > able to remove the 'tasks.max.enforce' connector property in 4.0 (we'd > discussed potentially having to live with this property until 5.0 in this > KIP's discussion thread). Once we have confirmation of a 3.8 release, will > this KIP be updated to reflect the exact AK versions where the deprecated > property will be introduced and removed? > > Thanks, > Yash > > On Wed, Jan 3, 2024 at 11:37 PM Greg Harris > wrote: > > > Hey Chris, > > > > Thanks for the KIP! I think the aggressive default and deprecation > > schedule is the right choice for this change. > > > > +1 (binding) > > > > On Wed, Jan 3, 2024 at 9:01 AM Mickael Maison > > wrote: > > > > > > Hi Chris, > > > > > > +1 (binding), thanks for the KIP > > > > > > Mickael > > > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > > > wrote: > > > > > > > > +1 (non-binding) > > > > > > > > Thanks Chris! > > > > > > > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > > dev@kafka.apache.org > > > > Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka > > Connect > > > > > > > > Hi all, > > > > > > > > Happy New Year! Wanted to give this a bump now that the holidays are > > over > > > > for a lot of us. Looking forward to people's thoughts! > > > > > > > > Cheers, > > > > > > > > Chris > > > > > > > > On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'd like to call for a vote on KIP-1004, which adds enforcement for > > the > > > > > tasks.max connector property in Kafka Connect. > > > > > > > > > > The KIP: > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > > > > property+in+Kafka+Connect > > > > > > > > > > The discussion thread: > > > > > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > > > > > > > > > Cheers, > > > > > > > > > > Chris > > > > > > > > > > > > > > >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hi Chris, +1 (binding), thanks for the KIP. Based on discussion in other threads, it looks like the community is aligned with having a 3.8 release before the 4.0 release so we should be able to remove the 'tasks.max.enforce' connector property in 4.0 (we'd discussed potentially having to live with this property until 5.0 in this KIP's discussion thread). Once we have confirmation of a 3.8 release, will this KIP be updated to reflect the exact AK versions where the deprecated property will be introduced and removed? Thanks, Yash On Wed, Jan 3, 2024 at 11:37 PM Greg Harris wrote: > Hey Chris, > > Thanks for the KIP! I think the aggressive default and deprecation > schedule is the right choice for this change. > > +1 (binding) > > On Wed, Jan 3, 2024 at 9:01 AM Mickael Maison > wrote: > > > > Hi Chris, > > > > +1 (binding), thanks for the KIP > > > > Mickael > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > > wrote: > > > > > > +1 (non-binding) > > > > > > Thanks Chris! > > > > > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > dev@kafka.apache.org > > > Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka > Connect > > > > > > Hi all, > > > > > > Happy New Year! Wanted to give this a bump now that the holidays are > over > > > for a lot of us. Looking forward to people's thoughts! > > > > > > Cheers, > > > > > > Chris > > > > > > On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton wrote: > > > > > > > Hi all, > > > > > > > > I'd like to call for a vote on KIP-1004, which adds enforcement for > the > > > > tasks.max connector property in Kafka Connect. > > > > > > > > The KIP: > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > > > property+in+Kafka+Connect > > > > > > > > The discussion thread: > > > > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > > > > > > > Cheers, > > > > > > > > Chris > > > > > > > > > > >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hey Chris, Thanks for the KIP! I think the aggressive default and deprecation schedule is the right choice for this change. +1 (binding) On Wed, Jan 3, 2024 at 9:01 AM Mickael Maison wrote: > > Hi Chris, > > +1 (binding), thanks for the KIP > > Mickael > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > wrote: > > > > +1 (non-binding) > > > > Thanks Chris! > > > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > > dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect > > > > Hi all, > > > > Happy New Year! Wanted to give this a bump now that the holidays are over > > for a lot of us. Looking forward to people's thoughts! > > > > Cheers, > > > > Chris > > > > On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton wrote: > > > > > Hi all, > > > > > > I'd like to call for a vote on KIP-1004, which adds enforcement for the > > > tasks.max connector property in Kafka Connect. > > > > > > The KIP: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > > property+in+Kafka+Connect > > > > > > The discussion thread: > > > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > > > > > Cheers, > > > > > > Chris > > > > > > >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hi Chris, +1 (binding), thanks for the KIP Mickael On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) wrote: > > +1 (non-binding) > > Thanks Chris! > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > dev@kafka.apache.org > Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect > > Hi all, > > Happy New Year! Wanted to give this a bump now that the holidays are over > for a lot of us. Looking forward to people's thoughts! > > Cheers, > > Chris > > On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton wrote: > > > Hi all, > > > > I'd like to call for a vote on KIP-1004, which adds enforcement for the > > tasks.max connector property in Kafka Connect. > > > > The KIP: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > property+in+Kafka+Connect > > > > The discussion thread: > > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > > > Cheers, > > > > Chris > > > >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
+1 (non-binding) Thanks Chris! From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect Hi all, Happy New Year! Wanted to give this a bump now that the holidays are over for a lot of us. Looking forward to people's thoughts! Cheers, Chris On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton wrote: > Hi all, > > I'd like to call for a vote on KIP-1004, which adds enforcement for the > tasks.max connector property in Kafka Connect. > > The KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ property+in+Kafka+Connect > > The discussion thread: > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > Cheers, > > Chris >
Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hi all, Happy New Year! Wanted to give this a bump now that the holidays are over for a lot of us. Looking forward to people's thoughts! Cheers, Chris On Mon, Dec 4, 2023 at 10:36 AM Chris Egerton wrote: > Hi all, > > I'd like to call for a vote on KIP-1004, which adds enforcement for the > tasks.max connector property in Kafka Connect. > > The KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect > > The discussion thread: > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > Cheers, > > Chris >
[VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect
Hi all, I'd like to call for a vote on KIP-1004, which adds enforcement for the tasks.max connector property in Kafka Connect. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect The discussion thread: https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d Cheers, Chris
