producer can't push msg sometimes with 1 broker recoved

2016-09-21 Thread FEI Aggie
Hi, I am testing with kafka 0.10.0 version. I set up 2 broker nodes. I shutdown 2 nodes and later bring up one of them, for sometimes producer can't push message to the live broker, though the live node does have lead partition for the specified topic. Log shows producer always try to connect

Re: Slow machine disrupting the cluster

2016-09-21 Thread Gerard Klijs
It turned out it was een over-provisioned VM. It was eventually solved by moving the VM to another cluster. He was also not a little slow but something in the magnitude of 100 times slower. We are now looking for some metrics to watch and alert in case it gets slow. On Fri, Sep 16, 2016 at 4:41

Migrating old consumer offsets to new consumer

2016-09-21 Thread Nick Kleinschmidt
I’m running the old 0.8 consumer storing offsets in Zookeeper, want to migrate to the new consumer introduced in 0.9 . I don’t see anything in the docs about how to do that while preserving offsets. Do I need to follow the steps from the FAQ to migrate to committing offsets to Kafka, then I can

Re: Kafka cluster behavior: broker no longer belonging to ISR, offsets reset to middle of partition

2016-09-21 Thread Chris Richardson
Turns out that ntp was not correctly setup and so the clocks were incorrect. I'd imagine that this would explain why brokers left the ISRs. On Mon, Sep 19, 2016 at 2:50 PM, Chris Richardson wrote: > Hi, > > One of my Kafka 0.9.0.1 clusters (3 brokers,

Re: Migrating old consumer offsets to new consumer

2016-09-21 Thread Stevo Slavić
Hello Nick, There are migration instructions in the docs (see http://kafka.apache.org/documentation.html#offsetmigration ) Kind regards, Stevo Slavic. On Wed, Sep 21, 2016 at 5:32 PM, Nick Kleinschmidt wrote: > I’m running the old 0.8 consumer storing offsets in

Error from SyncGroup: The request timed out

2016-09-21 Thread Jhon Davis
Recently we are experiencing "Error from SyncGroup: The request timed out" frequently with the Java Kafka APIs. This issue usually happens with few topic or consumer group in Kafka cluster. Any pointers on this one will be helpful. As a workaround, if I change the consumer group name I don't see

Migrating old consumer offsets to new consumer

2016-09-21 Thread Nick
I’m running the old 0.8 consumer storing offsets in Zookeeper, want to migrate to the new consumer introduced in 0.9 . I don’t see anything in the docs about how to do that while preserving offsets. Do I need to follow the steps from the FAQ to migrate to committing offsets to Kafka, then I can