Possible DEAD LOCK for one day at broker controller?

2015-08-16 Thread Zhao Weinan
Hi guys, I got this problem, after changing one topic's config to retention.ms=86,400,000 from 864,000,000, the brokers start to shedule and do deletions of outdated index of that topic. Then for some reason some brokers' connection with zookeeper were expired, suddenly lots of ERRORs showed up

Re: 0.8.2 producer and single message requests

2015-08-16 Thread Kishore Senji
Adding to what Gwen already mentioned - The programming model for the Producer is send() with an optional callback and we get a Future. This model does not change whether behind the scenes batching is done or not. So your fault tolerance logic really should not depend on whether batching is done

Re: Possible DEAD LOCK for one day at broker controller?

2015-08-16 Thread Kishore Senji
Interesting problem you ran in to. It seems like this broker was chosen as the Controller and onControllerFailure() method was called. This will schedule the checkAndTriggerPartitionRebalance method to execute after 5 seconds (when auto rebalance enabled). In the mean time this broker lost

Re: [DISCUSSION] Kafka 0.8.2.2 release?

2015-08-16 Thread Gwen Shapira
BTW. I think it will be great for Apache Kafka to have a 0.8.2 release manager who's role is to cherrypick low-risk bug-fixes into the 0.8.2 branch and once enough bug fixes happened (or if sufficiently critical fixes happened) to roll out a new maintenance release (with every 3 month as a

Re: [DISCUSSION] Kafka 0.8.2.2 release?

2015-08-16 Thread Jeff Holoman
+1 for the release and also including https://issues.apache.org/jira/browse/KAFKA-2114 Thanks Jeff On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić ssla...@gmail.com wrote: +1 (non-binding) for 0.8.2.2 release Would be nice to include in that release new producer resiliency bug fixes

Re: [DISCUSSION] Kafka 0.8.2.2 release?

2015-08-16 Thread Grant Henke
+1 to that suggestion. Though I suspect that requires a committer to do. Making it part of the standard commit process could work too. On Aug 16, 2015 11:01 PM, Gwen Shapira g...@confluent.io wrote: BTW. I think it will be great for Apache Kafka to have a 0.8.2 release manager who's role is to

Re: [DISCUSSION] Kafka 0.8.2.2 release?

2015-08-16 Thread Stevo Slavić
+1 (non-binding) for 0.8.2.2 release Would be nice to include in that release new producer resiliency bug fixes https://issues.apache.org/jira/browse/KAFKA-1788 and https://issues.apache.org/jira/browse/KAFKA-2120 On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira g...@confluent.io wrote: Will be