Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-26 Thread Mathieu Amblard
; > > > as soon as the poison pill is consumed, the consumer of this > other > > > > > instance > > > > > > will be stopped, and so on, and so on. It will take a few tens of > > > > seconds > > > > > > before the last consumer of the poi

Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-25 Thread Sophie Blee-Goldman
ration, we have planned to find a solution to avoid > > > this > > > > > time lapse between the consumer of the first instance being stopped > > and > > > > the > > > > > last one. Currently, I do not have a solution, but we are thinking > >

Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-25 Thread David Jacot
m. > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Mathieu > > > > > > > > Le ven. 7 oct. 2022 à 16:54, Hector Geraldino (BLOOMBERG/ 919 3RD A) > < > > > > hgerald...@bloomberg.n

Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-24 Thread Mathieu Amblard
> >> having single-partition topics solve this problem? > > >> > > >> You also mentioned that your goal is to minimize the potential of a > > >> poison pill message breaking all members of a group (by keeping track > of > > >> which

Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-24 Thread Sophie Blee-Goldman
3 members (A, B, C) > >> * Members are subscribed to 3 topics (T1, T2, T3) > >> * Each member is assigned one topic (A[T1], B[T2], C[T3]) > >> * One member fails to consume from a topic/partition (B[T2]), and goes > >> into failed state > >> > >&

Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-21 Thread Mathieu Amblard
T3) >> * Each member is assigned one topic (A[T1], B[T2], C[T3]) >> * One member fails to consume from a topic/partition (B[T2]), and goes >> into failed state >> >> How will the group leader know that T2 should not be re-assigned on the >> next rebalance? Can you

Re: [DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-11 Thread Mathieu Amblard
l the group leader know that T2 should not be re-assigned on the > next rebalance? Can you elaborate a bit more on the mechanisms used to > communicate this state to the other group members? > > Thanks > > From: dev@kafka.apache.org At: 10/05/22 03:47:33 UTC-4:00To: > dev@kafka

Re:[DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-07 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
? Thanks From: dev@kafka.apache.org At: 10/05/22 03:47:33 UTC-4:00To: dev@kafka.apache.org Subject: [DISCUSS] KIP-874: TopicRoundRobinAssignor Hi Kafka Developers, My proposal is to add a new partition assignment strategy at the topic level to : - have a better data consistency by consumed

[DISCUSS] KIP-874: TopicRoundRobinAssignor

2022-10-05 Thread Mathieu Amblard
Hi Kafka Developers, My proposal is to add a new partition assignment strategy at the topic level to : - have a better data consistency by consumed topic in case of exception - have a solution much thread safe for the consumer In case there are multiple consumers and multiple topics. Here is