Re: Got conflicted ephemeral node exception for several hours

2015-08-04 Thread Jaikiran Pai
I am on Kafka 0.8.2.1 (Java 8) and have happened to run into this same issue where the KafkaServer (broker) goes into a indefinite while loop writing out this message: [2015-08-04 15:45:12,350] INFO conflict in /brokers/ids/0 data:

Re: Got conflicted ephemeral node exception for several hours

2015-07-12 Thread Jiangjie Qin
Hi Tao, We see this error from time to time but did not think of this as a big issue. Any reason it bothers you much? I¹m not sure if throwing exception to user on this exception is a good handling or not. What are user supposed to do in that case other than retry? Thanks, Jiangjie (Becket) Qin

Re: Got conflicted ephemeral node exception for several hours

2015-07-12 Thread Mayuresh Gharat
Bouncing the consumers should solve this issue in most cases. Thanks, Mayuresh On Sun, Jul 12, 2015 at 8:21 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hi Tao, We see this error from time to time but did not think of this as a big issue. Any reason it bothers you much? I¹m not sure

Re: Got conflicted ephemeral node exception for several hours

2015-07-12 Thread tao xiao
Restart the consumers does fix the issue. But since the zk retry is wrapped in an infinite loop it doesn't give a chance to consumer to respond it until some one notices and restarts. Why I suggest to have a maximum retry policy is if max retry is reached it can invoke a customer handler which I

Re: Got conflicted ephemeral node exception for several hours

2015-07-12 Thread tao xiao
We saw the error again in our cluster. Anyone has the same issue before? On Fri, 10 Jul 2015 at 13:26 tao xiao xiaotao...@gmail.com wrote: Bump the thread. Any help would be appreciated. On Wed, 8 Jul 2015 at 20:09 tao xiao xiaotao...@gmail.com wrote: Additional info Kafka version:

Re: Got conflicted ephemeral node exception for several hours

2015-07-12 Thread Mayuresh Gharat
That would solve this. But it looks like a work around. We need to check why this happens exactly and get to the root cause. What do you think? Getting to the root cause of this might be really useful. Thanks, Mayuresh On Sun, Jul 12, 2015 at 8:45 PM, tao xiao xiaotao...@gmail.com wrote:

Re: Got conflicted ephemeral node exception for several hours

2015-07-12 Thread tao xiao
Finding out root cause is definitely something we should do and I hope the new Java consumer API will not have such issues anymore. Here are my observation of the issue: the zk retry was caused by session timeout. In theory the ephemeral node should be removed after the session is terminated by zk

Re: Got conflicted ephemeral node exception for several hours

2015-07-09 Thread tao xiao
Bump the thread. Any help would be appreciated. On Wed, 8 Jul 2015 at 20:09 tao xiao xiaotao...@gmail.com wrote: Additional info Kafka version: 0.8.2.1 zookeeper: 3.4.6 On Wed, 8 Jul 2015 at 20:07 tao xiao xiaotao...@gmail.com wrote: Hi team, I have 10 high level consumers connecting to

Re: Got conflicted ephemeral node exception for several hours

2015-07-08 Thread tao xiao
Additional info Kafka version: 0.8.2.1 zookeeper: 3.4.6 On Wed, 8 Jul 2015 at 20:07 tao xiao xiaotao...@gmail.com wrote: Hi team, I have 10 high level consumers connecting to Kafka and one of them kept complaining conflicted ephemeral node for about 8 hours. The log was filled with below

Got conflicted ephemeral node exception for several hours

2015-07-08 Thread tao xiao
Hi team, I have 10 high level consumers connecting to Kafka and one of them kept complaining conflicted ephemeral node for about 8 hours. The log was filled with below exception [2015-07-07 14:03:51,615] INFO conflict in /consumers/group/ids/test-1435856975563-9a9fdc6c data: