Re: Kafka ConsumerRebalanceFailedException, kafka.consumer.SimpleConsumer: Reconnect due to socket error: null, etc.

2015-05-26 Thread Jaikiran Pai
One way to narrow down the issue is to attach a debugger to the Kafka JVM and add a breakpoint in SimpleConsumer to see the real exception stacktrace which is causing the reconnect. I've filed a JIRA with a patch to improve this logging to include the entire cause stacktrace while logging this

Kafka ConsumerRebalanceFailedException, kafka.consumer.SimpleConsumer: Reconnect due to socket error: null, etc.

2015-05-21 Thread Kornel
Hi, I'm having trouble with a spark (1.2.1) streaming job which is using apache kafka (client: 0.8.1.1). The job is consuming messages, and the observed behaviour is that after processing some messages, the job stops processing, then restarts, then consumes the lag which build up during the