Re: Regarding Distributed Kafka-connect cluster

Thank you, Yash. Your explanation makes sense To clarify, who initiates the step that assigns a connector to a specific worker? If this process is controlled by the leader, wouldn't it result in a failure to assign tasks to workers with whom it cannot communicate? Although it is not ideal, it i

Re: [kafka-clients] [VOTE] 3.6.0 RC1

Hi Jose, Sounds good to me. Let's have further discussion in JIRA/PR, and target to v3.6.1/v3.7.0. Thanks. Luke On Tue, Sep 26, 2023 at 1:35 AM José Armando García Sancio wrote: > On Sat, Sep 23, 2023 at 3:08 AM Luke Chen wrote: > > > > Hi Satish, > > > > I found the current KRaft implementat

Re: [kafka-clients] [VOTE] 3.6.0 RC1

On Sat, Sep 23, 2023 at 3:08 AM Luke Chen wrote: > > Hi Satish, > > I found the current KRaft implementation will have "split brain" issue when > network partition happens, which will cause inconsistent metadata returned > from the controller. > Filed KAFKA-15489

Re: Regarding Distributed Kafka-connect cluster

Hi Yeikel, Heartbeats and group coordination in Kafka Connect do occur through Kafka, but a Kafka Connect cluster where all workers cannot communicate with each other won't work very well. You'll be able to create / update / delete connectors by making requests to any workers that can communicate

Re: Regarding Distributed Kafka-connect cluster

Thank you, Nikhil. I did notice that challenge you're describing with the REST updates when I had more than one worker within the same datacenter. Luckily, solving that was relatively simple as all my workers can communicate within the same data center, and all I need to do is to ensure that th

Re: Regarding Distributed Kafka-connect cluster

Kudos to Nikhil. your explanation adds to my knowledge. 🙏 On Mon, 25 Sep 2023 at 7:16 PM, Nikhil Srivastava < nikhilsrivastava4...@gmail.com> wrote: > Hi Yeikel, > > Sharing my two cents. Would let others chime in to add to this. > > Based on my understanding, if connect workers (which are all p

Re: Regarding Distributed Kafka-connect cluster

Hi Yeikel, Sharing my two cents. Would let others chime in to add to this. Based on my understanding, if connect workers (which are all part of the same cluster) can communicate with the kafka brokers (which happens to be the Group Coordinator and facilitates Connect Leader Election via Group Mem

Re: No RSS available on the new Kafka Blog

Hi Alex We currently do not have plans but we can definitely use a helping hand with our website content ( e.g. our blogs are currently written in html which is quite painful to manage :) ). An RSS friendly blog would be a great addition. It would be great if the community could help us in adding

Current status of KIP-833 marking KRaft as production ready

Hi all, I have noticed that the latest update on KIP-833 (marking KRaft as production ready) was on April 2023. Because I am interested in the plan related to the listed Kafka versions for deprecating ZooK