Kafka - FindCoordinator error

2020-05-03 Thread Rajib Deb
Hi I have written a Python consumer using confluent-kafka package. After few hours of running the consumer is dying with the below error cimpl.KafkaException: KafkaError{code=_TIMED_OUT,val=-185,str="FindCoordinator response error: Local: Timed out"} Can anyone please help me understand why

Offset Management...

2020-05-11 Thread Rajib Deb
Hi, I wanted to know if it is a good practice to develop a custom offset management method while consuming from Kafka. I am thinking to develop it as below. 1. Create a PartitionInfo named tuple as below PartitionInfo("PartitionInfo",["header","custom writer","offset"] 1. Then populate

RE: Offset Management...

2020-05-12 Thread Rajib Deb
EGFjUW59I%2BPtNXsf2Cm02PWcXMjL3oBi0nLhgVE4KAg%3Dreserved=0> . -Bill On Mon, May 11, 2020 at 9:52 PM Rajib Deb wrote: > Hi, I wanted to know if it is a good practice to develop a custom > offset management method while consuming from Kafka. I am thinking to > develop it as below. > > > 1. Crea

RE: Kafka - FindCoordinator error

2020-05-07 Thread Rajib Deb
I wanted to check if anyone has faced this issue Thanks Rajib From: Rajib Deb Sent: Sunday, May 3, 2020 9:51 AM To: users@kafka.apache.org Subject: Kafka - FindCoordinator error Hi I have written a Python consumer using confluent-kafka package. After few hours of running the consumer is dying

RE: Change RF factor...

2020-05-08 Thread Rajib Deb
ign-partitions.sh --zookeeper zookeeper_hoost:2181 --execute --reassignment-json-file changeTest.json On 5/8/20, 12:22 PM, "Rajib Deb" wrote: [External] It has three brokers {"version":1,"partitions":[{"topic":"te_re&q

Change RF factor...

2020-05-08 Thread Rajib Deb
Hi I have by mistake created a topic with replication factor of 1. I am trying to increase the replication, but I get the below error. Can anyone please let me know if I am doing anything wrong. The Topic is created with single partition(te_re-0). ./kafka-reassign-partitions.sh --zookeeper

RE: Change RF factor...

2020-05-08 Thread Rajib Deb
PM To: users@kafka.apache.org Subject: Re: Change RF factor... [**EXTERNAL EMAIL**] How many broker you have on this cluster and what is content of -- increase-replication-factor.json On 5/8/20, 12:16 PM, "Rajib Deb" wrote: [External] Hi I have by mistake created a topic with replic

Kafka - producer writing to a certain broker...

2020-07-25 Thread Rajib Deb
Hi, I came across the below question and wanted to seek an answer on the same. If a producer needs to write to a certain broker only, is this possible. For example, if the producer is in Europe, it will write to the broker near to Europe, if US it will write to broker near to US. But consumers

Kafka - producer writing to a certain broker...

2020-07-26 Thread Rajib Deb
Hi, I came across the below question and wanted to seek an answer on the same. If a producer needs to write to a certain broker only, is this possible. For example, if the producer is in Europe, it will write to the broker near to Europe, if US it will write to broker near to US. But consumers