[VOTE] 1.1.0 RC4

2018-03-23 Thread Rajini Sivaram
Hello Kafka users, developers and client-developers, This is the fifth candidate for release of Apache Kafka 1.1.0. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546 A few highlights: * Significant Controller improvements (much faster and session expiration edge cases

Re: can't consume from partitions due to KAFKA-3963

2018-03-23 Thread Jeff Widman
You could have had split brain where multiple brokers thought they were the controller. This has been a problematic piece of Kafka for a long time and only in the last release or two have some of the edge cases been cleaned up. To force a controller re-election, remove the "/controller" znode

Re: can't consume from partitions due to KAFKA-3963

2018-03-23 Thread Swapnil Gupta
As far as I have debugged my issues, server.log files were of the most use, they have errors and 'caused by' reasons. Probably, the leader broker was down and the followers were not able to replicate data now. 1. What are your min isr and replication settings? 2. Is your producer working? 3. What

RE: can't consume from partitions due to KAFKA-3963

2018-03-23 Thread Ryan O'Rourke
Thanks for the tips. As far as a broker being down, the situation is confused because a broker was definitely down, but I think we brought it down when we were trying to address the problem a few days ago. I can't say for sure that it wasn't down before, but we do have a monitor to catch that

Re: can't consume from partitions due to KAFKA-3963

2018-03-23 Thread Swapnil Gupta
Maybe a broker is down or unreachable which maybe breaking your min isr ratio and when consumers are set to ack all, the min isr has to be satisfied. Check your broker connect, or bring up a fresh broker and use preferred replica leader election tool ->

Re: Kafka bootstrap broker disconnected

2018-03-23 Thread Lehuede sebastien
Hi, Maybe you have make a typo with your IP address or you really use a Public IP for your zookeeper node ? Best regards, Sebastien 2018-03-23 15:15 GMT+01:00 Debraj Manna : > Some formatting issue in my first email. The dots are not showing up. > Correct IP -

Re: Kafka bootstrap broker disconnected

2018-03-23 Thread Debraj Manna
Some formatting issue in my first email. The dots are not showing up. Correct IP - 172.71.245.97:2181 On Fri, Mar 23, 2018 at 7:38 PM, adrien ruffie wrote: > Hi, > > > <+911727124597>:2181 disconnected means that your not is disconnected from > your zookeeper node.

RE: Kafka bootstrap broker disconnected

2018-03-23 Thread adrien ruffie
Hi, <+911727124597>:2181 disconnected means that your not is disconnected from your zookeeper node. Check if your zookeeper is alive, if you kafka node can reach zookeeper and vice-versa. But your zookeeper's ip <+911727124597> is very strange ... best regards, Adrien

Re: Kafka bootstrap broker disconnected

2018-03-23 Thread Debraj Manna
Anyone any thoughts My server.properties looks like below broker.id=0 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.retention.hours=168 log.retention.check.interval.ms=30

Re: Kafka streams failed for internal topics with FailOnInvalidTimestamp even with WallclockTimestampExtracto

2018-03-23 Thread Shelan Perera
Hi Mattias, Thank you very much for the explanation it really helps. We found out that one of the brokers were using older message formats and that caused the problem. Best Regards, Shelan. On 22 March 2018 at 23:28, Matthias J. Sax wrote: > Hi, > > the