Re: kafka 0.8.1: Producer.send() can block forever when a broker is down

2014-09-17 Thread Neha Narkhede
Make sense. Please file a JIRA and attach a patch there. It will be great to add a simple test case as well. Thanks, Neha On Wed, Sep 17, 2014 at 8:25 AM, Jonathan Weeks Gmail jonathanbwe...@gmail.com wrote: The issue is that even with one down broker, the rest of the cluster is up, but

kafka 0.8.1: Producer.send() can block forever when a broker is down

2014-09-16 Thread Jack Foy
We observe that when a broker is down, Producer.send() can get into a state where it will block forever, even when using the async producer. When a Producer first sends data, it fetches topic metadata from the broker cluster. To do this, it shuffles the list of hosts in the cluster, then

Re: kafka 0.8.1: Producer.send() can block forever when a broker is down

2014-09-16 Thread Jun Rao
Jack, If the broker is down, channel.connect() should throw an IOException, instead of blocking forever. In your case, is the broker host down? In that case, the connect call will likely wait for the default tcp connection timeout, which is 8+ mins. Thanks, Jun On Tue, Sep 16, 2014 at 5:43 PM,