Becket,
Thanks for the replies. Now I see you want to optimize with the heuristic
that "if I see a JoinGroup shortly enough after a rebalance is completed,
then likely there are more JoinGroups coming". I agree that it will help
with a single-instance console consumer for debugging etc, but still
Hi Guozhang,
Sorry for the confusion. I actually meant always "complete" the rebalance
immediately when the first consumer joining the group. i.e. the
configurable delta only kicks in after the first rebalance.
The concern I have was actually not the frequent rebalance for the users,
but the
Becket:
I think the problem is that when we have a single member joining an unknown
group for the first time ever, do we want to complete the rebalance
immediately or not; it does not matter if we want to "start" the rebalance,
since even for now if the group coordinator is in the SyncGroup phase
I am a little hesitant to add the configuration to the client. It would be
more flexible but this seems not the thing that users should worry about (I
imagine many people would simply set backoff to 0 just for fast rebalance).
I am wondering if the following variant of the current solution will
Found this thread after posting an alternative idea after we starting
hitting this issue ourselves for a job that has a lot of state stores and
topic partitions. My suggestion was to have consumer groups have a
configurable minimum member count before consumption begins, but that has
its own trade
Just recapping on client-side v.s. broker-side config: we did discuss about
adding this as a client-side config and bump up join-group request (I think
both Ismael and Ewen questioned about it) to include this configured value
to the broker. I cannot remember if there is any strong motivations
I forgot to address the broker versus client-side point that Jun raised. We
discussed passing the delay via the request and we were unable to make a
strong case for it. It seems like tools that use a single consumer and
group management (like the Console Consumer) are such a case though.
Changing
Hi Jun/Ismael,
Sounds good to me.
Thanks,
Damian
On Tue, 6 Jun 2017 at 23:08 Ismael Juma wrote:
> Hi Jun,
>
> The console consumer issue also came up in a conversation I was having
> recently. Seems like the config/server.properties change is a reasonable
> compromise given
Hi Jun,
The console consumer issue also came up in a conversation I was having
recently. Seems like the config/server.properties change is a reasonable
compromise given that we have other defaults that are for development.
Ismael
On Tue, Jun 6, 2017 at 10:59 PM, Jun Rao
Hi Onur,
It was in my previous email. But here it is again.
1. Better rebalance timing. We will try to rebalance only when all the
consumers in a group have joined. The challenge would be someone has to
define what does ALL consumers
Hi Damian.
Can you copy the point Becket made earlier that you say isn't addressed?
On Thu, Apr 6, 2017 at 2:51 AM, Damian Guy wrote:
> Thanks all, the Vote is now closed and the KIP has been accepted with 9 +1s
>
> 3 binding::
> Guozhang,
> Jason,
> Ismael
>
> 6
Thanks all, the Vote is now closed and the KIP has been accepted with 9 +1s
3 binding::
Guozhang,
Jason,
Ismael
6 non-binding:
Bill,
Eno,
Mathieu,
Matthias,
Dong,
Mickael
Thanks,
Damian
On Thu, 6 Apr 2017 at 09:26 Ismael Juma wrote:
> Thanks for the KIP, +1 (binding).
>
>
Thanks for the KIP, +1 (binding).
Ismael
On Thu, Mar 30, 2017 at 8:55 PM, Jason Gustafson wrote:
> +1 Thanks for the KIP!
>
> On Thu, Mar 30, 2017 at 12:51 PM, Guozhang Wang
> wrote:
>
> > +1
> >
> > Sorry about the previous email, Gmail seems be
Hi Onur,
Thanks for the update. I misunderstood what you said before. I believe what
you are suggesting sounds ok, though i don't think it addresses the point
Becket made earlier in the discussion thread. See below.
Thanks,
Damian
1.
Delaying the SyncGroupRequest is not what I had in mind.
What I was thinking was essentially a client-side stabilization window
where the client does nothing other than participate in the group
membership protocol and wait a bit for the group to stabilize.
During this window, several rounds of
Hi Damian.
After reading the discussion thread again, it still doesn't seem like the
thread discussed the option I mentioned earlier.
>From what I had understood from the broker-side vs. client-side config
debate was that the client-side config from the discussion would cause a
wire format
+1 (non-binding)
On Mon, Apr 3, 2017 at 9:53 AM, Mathieu Fenniak <
mathieu.fenn...@replicon.com> wrote:
> +1 (non-binding)
>
> This will be very helpful for me, looking forward to it! :-)
>
> On Thu, Mar 30, 2017 at 4:46 AM, Damian Guy wrote:
>
> > Hi All,
> >
> > I'd like
+1 (non-binding)
This will be very helpful for me, looking forward to it! :-)
On Thu, Mar 30, 2017 at 4:46 AM, Damian Guy wrote:
> Hi All,
>
> I'd like to start the voting thread on KIP-134:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>
+1 (non binding)
Thanks
On Fri, Mar 31, 2017 at 9:01 AM, Damian Guy wrote:
> Hi Onur,
>
> We already discussed similar on the Discussion thread and decided to move
> forward with a broker config.
>
> Thanks,
> Damian
>
> On Fri, 31 Mar 2017 at 02:40 Onur Karaman
Hi Onur,
We already discussed similar on the Discussion thread and decided to move
forward with a broker config.
Thanks,
Damian
On Fri, 31 Mar 2017 at 02:40 Onur Karaman
wrote:
> It seems like there generally has been the assumption that the broker needs
> to
It seems like there generally has been the assumption that the broker needs
to know about this delay either from its own config or provided over the
wire from clients. Is this actually true?
One alternative I don't think was mentioned was to make this delay concept
be completely client-side.
+1 (non-binding)
Thanks!
On Thu, Mar 30, 2017 at 6:03 PM, Becket Qin wrote:
> +1 Thanks for the KIP!
>
> On Thu, Mar 30, 2017 at 12:55 PM, Jason Gustafson
> wrote:
>
> > +1 Thanks for the KIP!
> >
> > On Thu, Mar 30, 2017 at 12:51 PM, Guozhang Wang
+1 Thanks for the KIP!
On Thu, Mar 30, 2017 at 12:55 PM, Jason Gustafson
wrote:
> +1 Thanks for the KIP!
>
> On Thu, Mar 30, 2017 at 12:51 PM, Guozhang Wang
> wrote:
>
> > +1
> >
> > Sorry about the previous email, Gmail seems be collapsing them into a
>
+1 Thanks for the KIP!
On Thu, Mar 30, 2017 at 12:51 PM, Guozhang Wang wrote:
> +1
>
> Sorry about the previous email, Gmail seems be collapsing them into a
> single thread on my inbox.
>
> Guozhang
>
> On Thu, Mar 30, 2017 at 11:34 AM, Guozhang Wang
>
+1
Sorry about the previous email, Gmail seems be collapsing them into a
single thread on my inbox.
Guozhang
On Thu, Mar 30, 2017 at 11:34 AM, Guozhang Wang wrote:
> Damian, could you create a new thread for the voting process?
>
> Thanks!
>
> Guozhang
>
> On Thu, Mar 30,
Hi Guozhang,
This was a new thread!
Damian
On Thu, 30 Mar 2017 at 19:34, Guozhang Wang wrote:
> Damian, could you create a new thread for the voting process?
>
> Thanks!
>
> Guozhang
>
> On Thu, Mar 30, 2017 at 10:33 AM, Bill Bejeck wrote:
>
> >
Damian, could you create a new thread for the voting process?
Thanks!
Guozhang
On Thu, Mar 30, 2017 at 10:33 AM, Bill Bejeck wrote:
> +1(non-binding)
>
> On Thu, Mar 30, 2017 at 1:30 PM, Eno Thereska
> wrote:
>
> > +1 (non binding)
> >
> > Thanks
>
+1(non-binding)
On Thu, Mar 30, 2017 at 1:30 PM, Eno Thereska
wrote:
> +1 (non binding)
>
> Thanks
> Eno
> > On 30 Mar 2017, at 18:01, Matthias J. Sax wrote:
> >
> > +1
> >
> > On 3/30/17 3:46 AM, Damian Guy wrote:
> >> Hi All,
> >>
> >> I'd like
+1 (non binding)
Thanks
Eno
> On 30 Mar 2017, at 18:01, Matthias J. Sax wrote:
>
> +1
>
> On 3/30/17 3:46 AM, Damian Guy wrote:
>> Hi All,
>>
>> I'd like to start the voting thread on KIP-134:
>>
+1
On 3/30/17 3:46 AM, Damian Guy wrote:
> Hi All,
>
> I'd like to start the voting thread on KIP-134:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-134%3A+Delay+initial+consumer+group+rebalance
>
> Thanks,
> Damian
>
signature.asc
Description: OpenPGP digital signature
Hi All,
I'd like to start the voting thread on KIP-134:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-134%3A+Delay+initial+consumer+group+rebalance
Thanks,
Damian
31 matches
Mail list logo