Hi,
I see that Kafka uses zookeeper for group membership and leader election.
Curious to know if Kafka developers ever discussed to move away from
Zookeeper and use Gossip based protocols. Is there any specific advantage
of using Zookeeper over Gossip based implementation or vice versa?
Thanks,
rote:
> > > The reason for including LeaseStartTimeMs in the request is to ensure
> > > that the time required to communicate with the controller gets
> included in
> > > the lease time. Since requests can potentially be delayed in the
> network
> > > for a long t
e good to add some details on how
leases are maintained in case of active controller (raft leader) failure.
e.g. Attached a few diagrams considering leases are maintained with log
entries.
On Mon, Aug 31, 2020 at 6:28 PM Unmesh Joshi wrote:
> >>The reason for including LeaseStartTimeMs
that in a follow-on KIP, though, since this one is already
> > > pretty big.
> > >
> > > >
> > > > 4. Because we expose Raft log to all the brokers, any de-duplication
> of
> > > the
> > > > broker needs to happen before the requests are propo
Sat, Aug 29, 2020 at 10:49 AM Colin McCabe wrote:
> On Fri, Aug 28, 2020, at 19:36, Unmesh Joshi wrote:
> > Hi Colin,
> >
> > There were a few of questions I had..
>
> Hi Unmesh,
>
> Thanks for the response.
>
> >
> > 1. Were my comments on t
McCabe wrote:
> Hi all,
>
> I'm thinking of calling a vote on KIP-631 on Monday. Let me know if
> there's any more comments I should address before I start the vote.
>
> cheers,
> Colin
>
> On Tue, Aug 11, 2020, at 05:39, Unmesh Joshi wrote:
> > >>Hi Unmesh,
>
Thanks for the KIP.
+1 (non-binding)
On Tue, Sep 15, 2020 at 12:23 AM Colin McCabe wrote:
> Hi all,
>
> I'd like to call a vote on KIP-631: the quorum-based Kafka Controller.
> The KIP is here:
>
> https://cwiki.apache.org/confluence/x/4RV4CQ
>
> The DISCUSS thread is here:
>
>
>
s,
> > Unmesh
> >
> > On Sat, Sep 5, 2020 at 1:28 AM Colin McCabe wrote:
> >
> > > > Colin wrote:
> > > > > The reason for including LeaseStartTimeMs in the request is to
> ensure
> > > > > that the time required to communicate
Hi,
The LeaseStartTimeMs is expected to be the broker's
'System.currentTimeMillis()' at the point of the request. The active
controller will add its lease period to this in order to compute
the LeaseEndTimeMs.
I think this is a bit confusing. Monotonic clock on the active controller
ekafka/kip500/Kip631ControllerTest.scala
I hope it will be helpful in driving some discussions and doing quick
experimentation outside of the Kafka codebase.
Thanks,
Unmesh
On Tue, Aug 4, 2020 at 7:52 AM Unmesh Joshi wrote:
> Hi,
>
> >>>>The LeaseStartTimeMs is
; > > just using `broker.id`?
> > > > > 3. It sounds like you are imagining a stop-the-world approach to
> > > > > snapshotting, which is why we need the controller micromanaging
> > > snapshots
> > > > > on all followers. Alternatives include fuzzy
>>Hi Unmesh,
>>Thanks, I'll take a look.
Thanks. I will be adding more to the prototype and will be happy to help
and collaborate.
Thanks,
Unmesh
On Tue, Aug 11, 2020 at 12:28 AM Colin McCabe wrote:
> Hi Jose,
>
> That'a s good point that I hadn't considered. It's probably worth having
> a
I see that, when a new topic is created, two metadata records, a
TopicRecord (just the name and id of the topic) and a PartitionRecord (more
like LeaderAndIsr, with leader id and replica ids for the partition) are
created.
While creating the topic, log entries for both the records need to be
> Hi Unmesh,
>
> Yes, once the last stable offset advanced, we would consider the topic
> creation to be done, and then we could return success to the client.
>
> best,
> Colin
>
> On Thu, Jul 9, 2020, at 19:44, Unmesh Joshi wrote:
> > It still needs HighWaterMark / LastS
eTopic ||
||
||
On Fri, Jul 10, 2020 at 1:30 AM Colin McCabe wrote:
> On Thu, Jul 9, 2020, at 04:37, Unmesh Joshi wrote:
> > I see that, when a new topic is created, two metadata records, a
> > TopicRecord (just the name and id of the topic)
Hi,
In the FetchRequest Handling, how to make sure we handle scenarios where
the leader might have been disconnected from the cluster, but doesn't know
yet?
As discussed in the Raft Thesis section 6.4, the linearizable semantics of
read requests is implemented in LogCabin by sending heartbeat to
he controller micromanaging
> > > snapshots
> > > > > on all followers. Alternatives include fuzzy snapshots which can be
> > > done
> > > > > concurrently. If this has been rejected, can you add some detail
> > about
> > > why?
, that way, telling
client in time that it was disconnected from the quorum, and not keep on
sending state metadata to clients.
Thanks,
Unmesh
On Mon, Jul 27, 2020 at 9:31 AM Unmesh Joshi wrote:
> >>Could you clarify on this question? Which part of the raft group doesn't
> >>kno
Mon, Jul 27, 2020 at 6:23 AM Boyang Chen
wrote:
> Thanks for the questions Unmesh!
>
> On Sun, Jul 26, 2020 at 6:18 AM Unmesh Joshi
> wrote:
>
> > Hi,
> >
> > In the FetchRequest Handling, how to make sure we handle scenarios where
> > the leader might have bee
Hi Boyang,
Thanks for the KIP..
I think there are two aspects of linearizable read implementations in Raft.
1. Providing linearizable reads from the leader
Even read requests from the leader might not return the latest committed
results if the leader is partitioned. The leader needs to make
Went through the changes since the last discussion thread, and it's looking
in good shape. Thanks!.
+ 1 (non-binding)
On Wed, Dec 16, 2020 at 4:34 PM Tom Bentley wrote:
> Thanks for the KIP Colin, it does a great job of clearly explaining some
> pretty complex changes.
>
> +1 (non-binding)
>
>
allelForks` can be set via gradle argument:
> https://github.com/apache/kafka/blob/trunk/build.gradle#L78
>
> And in Jenkins, it looks like we default to 2.
> https://github.com/apache/kafka/blob/trunk/Jenkinsfile#L40
>
> Thank you.
> Luke
>
> On Fri, Apr 15, 2022 at 1:24 AM U
Hi,
I came across this issue which has discussion about capping
maxParallelForks while running tests.
https://issues.apache.org/jira/browse/KAFKA-2613
Is this still the case?
Thanks,
Unmesh
23 matches
Mail list logo
