Re: [VOTE] KIP-735: Increase default consumer session timeout
Calling the vote here. Thanks everyone! The total is +8 with no -1 votes. The binding total is +7 with votes from myself, Sophie, Gwen, Guozhang, Bill, David, and Bruno. -Jason On Thu, Apr 29, 2021 at 10:05 AM Jason Gustafson wrote: > Hey All, > > Thanks everyone for the votes. > > I had some offline discussion with Magnus Edenhill and he brought up a > potential problem with the change in behavior for group.(min|max). > session.timeout.ms. Unlike the java consumer, the librdkafka consumer > proactively revokes partitions after the session timeout expires locally > with no response from the coordinator. That means that the automatic > adjustment of the session timeout would not be respected by the client. > What is worse, after the consumer expires its local session timeout and > revokes partitions, it rejoins as a new member. However, the rebalance > would not be able to complete until the old member had expired, so the net > effect would be that rebalances get unexpectedly delayed by the adjusted > session timeout. > > I think this means we need to give this change some more thought, so I am > removing it from the KIP. Probably we will have to bump the JoinGroup API > so that the coordinator can make the client aware of the adjusted session > timeout (and probably the heartbeat interval as well). I will look into > doing this change in a separate proposal. > > Thanks, > Jason > > On Thu, Apr 29, 2021 at 1:43 AM Bruno Cadonna wrote: > >> Thank you for the KIP, Jason! >> >> +1 (binding) >> >> Best, >> Bruno >> >> On 29.04.21 10:10, Luke Chen wrote: >> > Hi Jason, >> > +1 (non-binding) >> > >> > Really need this KIP to save poor jenkins flaky tests. :) >> > >> > Luke >> > >> > On Thu, Apr 29, 2021 at 4:01 PM David Jacot > > >> > wrote: >> > >> >> +1 (binding) >> >> >> >> Thanks for the KIP. >> >> >> >> On Thu, Apr 29, 2021 at 2:27 AM Bill Bejeck wrote: >> >> >> >>> Thanks for the KIP Jason, +1(binding) >> >>> >> >>> -Bill >> >>> >> >>> On Wed, Apr 28, 2021 at 7:47 PM Guozhang Wang >> >> wrote: >> >>> >> +1. Thanks Jason！ >> >> On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira >> >> > >> wrote: >> >> > I love this improvement. >> > >> > +1 (binding) >> > >> > On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson >> > >> > wrote: >> > >> >> Hi All, >> >> >> >> I'd like to start a vote on KIP-735: >> >> >> >> >> > >> >> >>> >> >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout >> >> . >> >> +1 >> >> from myself obviously >> >> >> >> -Jason >> >> >> > >> > >> > -- >> > Gwen Shapira >> > Engineering Manager | Confluent >> > 650.450.2760 | @gwenshap >> > Follow us: Twitter | blog >> > >> >> >> -- >> -- Guozhang >> >> >>> >> >> >> > >> >
Re: [VOTE] KIP-735: Increase default consumer session timeout
Hey All, Thanks everyone for the votes. I had some offline discussion with Magnus Edenhill and he brought up a potential problem with the change in behavior for group.(min|max). session.timeout.ms. Unlike the java consumer, the librdkafka consumer proactively revokes partitions after the session timeout expires locally with no response from the coordinator. That means that the automatic adjustment of the session timeout would not be respected by the client. What is worse, after the consumer expires its local session timeout and revokes partitions, it rejoins as a new member. However, the rebalance would not be able to complete until the old member had expired, so the net effect would be that rebalances get unexpectedly delayed by the adjusted session timeout. I think this means we need to give this change some more thought, so I am removing it from the KIP. Probably we will have to bump the JoinGroup API so that the coordinator can make the client aware of the adjusted session timeout (and probably the heartbeat interval as well). I will look into doing this change in a separate proposal. Thanks, Jason On Thu, Apr 29, 2021 at 1:43 AM Bruno Cadonna wrote: > Thank you for the KIP, Jason! > > +1 (binding) > > Best, > Bruno > > On 29.04.21 10:10, Luke Chen wrote: > > Hi Jason, > > +1 (non-binding) > > > > Really need this KIP to save poor jenkins flaky tests. :) > > > > Luke > > > > On Thu, Apr 29, 2021 at 4:01 PM David Jacot > > > wrote: > > > >> +1 (binding) > >> > >> Thanks for the KIP. > >> > >> On Thu, Apr 29, 2021 at 2:27 AM Bill Bejeck wrote: > >> > >>> Thanks for the KIP Jason, +1(binding) > >>> > >>> -Bill > >>> > >>> On Wed, Apr 28, 2021 at 7:47 PM Guozhang Wang > >> wrote: > >>> > +1. Thanks Jason！ > > On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira > >> > wrote: > > > I love this improvement. > > > > +1 (binding) > > > > On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson > > > > wrote: > > > >> Hi All, > >> > >> I'd like to start a vote on KIP-735: > >> > >> > > > > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > >> . > >> +1 > >> from myself obviously > >> > >> -Jason > >> > > > > > > -- > > Gwen Shapira > > Engineering Manager | Confluent > > 650.450.2760 | @gwenshap > > Follow us: Twitter | blog > > > > > -- > -- Guozhang > > >>> > >> > > >
Re: [VOTE] KIP-735: Increase default consumer session timeout
Thank you for the KIP, Jason! +1 (binding) Best, Bruno On 29.04.21 10:10, Luke Chen wrote: Hi Jason, +1 (non-binding) Really need this KIP to save poor jenkins flaky tests. :) Luke On Thu, Apr 29, 2021 at 4:01 PM David Jacot wrote: +1 (binding) Thanks for the KIP. On Thu, Apr 29, 2021 at 2:27 AM Bill Bejeck wrote: Thanks for the KIP Jason, +1(binding) -Bill On Wed, Apr 28, 2021 at 7:47 PM Guozhang Wang wrote: +1. Thanks Jason！ On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira wrote: I love this improvement. +1 (binding) On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson wrote: Hi All, I'd like to start a vote on KIP-735: https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout . +1 from myself obviously -Jason -- Gwen Shapira Engineering Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog -- -- Guozhang
Re: [VOTE] KIP-735: Increase default consumer session timeout
Hi Jason, +1 (non-binding) Really need this KIP to save poor jenkins flaky tests. :) Luke On Thu, Apr 29, 2021 at 4:01 PM David Jacot wrote: > +1 (binding) > > Thanks for the KIP. > > On Thu, Apr 29, 2021 at 2:27 AM Bill Bejeck wrote: > > > Thanks for the KIP Jason, +1(binding) > > > > -Bill > > > > On Wed, Apr 28, 2021 at 7:47 PM Guozhang Wang > wrote: > > > > > +1. Thanks Jason！ > > > > > > On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira > > > > > > wrote: > > > > > > > I love this improvement. > > > > > > > > +1 (binding) > > > > > > > > On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson > > > > > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > I'd like to start a vote on KIP-735: > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > > > > > . > > > > > +1 > > > > > from myself obviously > > > > > > > > > > -Jason > > > > > > > > > > > > > > > > > -- > > > > Gwen Shapira > > > > Engineering Manager | Confluent > > > > 650.450.2760 | @gwenshap > > > > Follow us: Twitter | blog > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > >
Re: [VOTE] KIP-735: Increase default consumer session timeout
+1 (binding) Thanks for the KIP. On Thu, Apr 29, 2021 at 2:27 AM Bill Bejeck wrote: > Thanks for the KIP Jason, +1(binding) > > -Bill > > On Wed, Apr 28, 2021 at 7:47 PM Guozhang Wang wrote: > > > +1. Thanks Jason！ > > > > On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira > > > wrote: > > > > > I love this improvement. > > > > > > +1 (binding) > > > > > > On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson > > > > > > wrote: > > > > > > > Hi All, > > > > > > > > I'd like to start a vote on KIP-735: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > > > > . > > > > +1 > > > > from myself obviously > > > > > > > > -Jason > > > > > > > > > > > > > -- > > > Gwen Shapira > > > Engineering Manager | Confluent > > > 650.450.2760 | @gwenshap > > > Follow us: Twitter | blog > > > > > > > > > -- > > -- Guozhang > > >
Re: [VOTE] KIP-735: Increase default consumer session timeout
Thanks for the KIP Jason, +1(binding) -Bill On Wed, Apr 28, 2021 at 7:47 PM Guozhang Wang wrote: > +1. Thanks Jason！ > > On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira > wrote: > > > I love this improvement. > > > > +1 (binding) > > > > On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson > > > > wrote: > > > > > Hi All, > > > > > > I'd like to start a vote on KIP-735: > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > > > . > > > +1 > > > from myself obviously > > > > > > -Jason > > > > > > > > > -- > > Gwen Shapira > > Engineering Manager | Confluent > > 650.450.2760 | @gwenshap > > Follow us: Twitter | blog > > > > > -- > -- Guozhang >
Re: [VOTE] KIP-735: Increase default consumer session timeout
+1. Thanks Jason！ On Wed, Apr 28, 2021 at 12:50 PM Gwen Shapira wrote: > I love this improvement. > > +1 (binding) > > On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson > > wrote: > > > Hi All, > > > > I'd like to start a vote on KIP-735: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > > . > > +1 > > from myself obviously > > > > -Jason > > > > > -- > Gwen Shapira > Engineering Manager | Confluent > 650.450.2760 | @gwenshap > Follow us: Twitter | blog > -- -- Guozhang
Re: [VOTE] KIP-735: Increase default consumer session timeout
I love this improvement. +1 (binding) On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-735: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > . > +1 > from myself obviously > > -Jason > -- Gwen Shapira Engineering Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog
Re: [VOTE] KIP-735: Increase default consumer session timeout
Thanks for the KIP, +1 (binding) -Sophie On Wed, Apr 28, 2021 at 10:46 AM Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-735: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout > . > +1 > from myself obviously > > -Jason >
[VOTE] KIP-735: Increase default consumer session timeout
Hi All, I'd like to start a vote on KIP-735: https://cwiki.apache.org/confluence/display/KAFKA/KIP-735%3A+Increase+default+consumer+session+timeout. +1 from myself obviously -Jason
