Re: Stream processing meetup at LinkedIn (Mountain View) on Tuesday, August 23 at 6pm

ecording after the meetup. > Thanks for asking! > > Ed > > On Fri, Aug 12, 2016 at 2:56 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi, > > > > Thanks for the invitation. I won't be able to make it this soon. > > However, it'll b

Re: What is the benefit of using acks=all and minover e.g. acks=3

ize when things get clogged for some reason? > > Sorry for the quite messy description, we are all kind of new at kafka > here! > > BR > Andreas > > > On 28 Nov 2015, at 09:26, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > > > > Hi, > > > > This

Re: What is the benefit of using acks=all and minover e.g. acks=3

one of the > replicas will fall behind, you will have 3 out of 4 in sync. > If you set min.insync.replica = 3, produce requests will fail if the number > on in-sync replicas fall below 3. > > I hope this helps. > > Gwen > > On Fri, Nov 27, 2015 at 9:43 PM, Prabhjot Bhara

Re: SV: What is the benefit of using acks=all and minover e.g. acks=3

client.id=MZ > max.request.size=1048576 > acks=all > retries=0 > timeout.ms=3 > buffer.memory=67108864 > metadata.fetch.timeout.ms=3000 > > Do let me know what you think about it! We are currently setting up some > tests using the broker properties that you suggested. > &

Re: What is the benefit of using acks=all and minover e.g. acks=3

Hi Gwen, How about min.isr.replicas property? Is it still valid in the new version 0.9 ? We could get 3 out of 4 replicas in sync if we set it's value to 3. Correct? Thanks, Prabhjot On Nov 28, 2015 10:20 AM, "Gwen Shapira" wrote: > In your scenario, you are receiving acks

producer-consumer issues during deployments

Hi, We arrange our kafka machines in groups and deploy these phases. For kafka, we’ll have to map groups with phases. During each phase of the release, all the machines in that group can go down. When this happens, there are a couple of cases:- 1. All replicas are residing in a group of

Re: producer-consumer issues during deployments

Hi, Request your expertise on these doubts of mine Thanks, Prabhjot On Thu, Nov 26, 2015 at 4:43 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > We arrange our kafka machines in groups and deploy these phases. > > For kafka, we’ll have to map groups with phas

Re: Notable failure scenarios in high-level/new consumer

Hi, Request your expertise on these doubts of mine Thanks, Prabhjot On Thu, Nov 26, 2015 at 12:09 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hello Folks, > > I am trying to build fault tolerance on the consumer side, so as to make > sure that all failure sce

Re: producer-consumer issues during deployments

ine then you will be able to consumer from it. If the leader is on a > machine that goes offline then consumption will pause whilst leadership > switches over to a replica. > > All the best > B > > > On 26 Nov 2015, at 17:58, Prabhjot Bharaj <prabhbha...@gmail.com> wrote:

Re: Notable failure scenarios in high-level/new consumer

Hello Folks, Request your expertise on these doubts of mine Thanks, Prabhjot On Thu, Nov 26, 2015 at 11:28 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > Request your expertise on these doubts of mine > > Thanks, > Prabhjot > > On Thu, Nov 26, 201

Notable failure scenarios in high-level/new consumer

Hello Folks, I am trying to build fault tolerance on the consumer side, so as to make sure that all failure scenarios are handled. On Data integrity side, there are primary 2 requirements:- 1. No Data loss 2. No data duplication I'm particularly interested in data duplication. e.g. there are

Re: kafka producer and client use same groupid

Hi Surender, Please elaborate on your design Consumers don't talk to producers directly, Kafka is a brokered system, and Kafka sits between producers and consumers Also, consumers consume from partitions of a topic and producers write to partitions in a topic These partitions and the logical

Re: replication factor increased automatically !

Hi Sunil, Please paste the full output, including the first line of the output of kafka-topics.sh. e.g. root@x.x.x.x:~# kafka-topics.sh --describe --zookeeper localhost:2182 --topic part_2_repl_3 *Topic:part_2_repl_3 PartitionCount:2 ReplicationFactor:3 Configs:* Topic: part_2_repl_3

Re: All brokers are running but some partitions' leader is -1

Hi, With the information provided, these are the steps I can think of (based on the experience I had with kafka):- 1. do a describe on the topic. See if the partitions and replicas are evenly distributed amongst all. If not, you might want to try the 'Reassign Partitions Tool' -

Re: consumer offset tool and JMX metrics do not match

<allen.michael.c...@gmail.com> wrote: > I believe producers / brokers / and consumers has been restarted at > different times. > What do you think the issue is? > > On Sat, Nov 14, 2015 at 2:52 AM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi, > >

Re: Subscription Request

Hi Saurabh, Sure thing. You can follow the steps here: http://kafka.apache.org/contact.html Regards, Prabhjot On Nov 18, 2015 11:32 AM, "saurabh pratap singh" wrote: > Hi > > I have recently started using kafka and I want to subscribe to mailing list > . > > > Thanks >

Re: consumer offset tool and JMX metrics do not match

Hi, Did any of your brokers/consumers undergo a restart between the time after you had started the consumption and until you see this issue ? Thanks, Prabhjot On Sat, Nov 14, 2015 at 5:53 AM, allen chan wrote: > I also looked at this metric in JMX and it is

Re: Kafka Scalability & Partition

Hi, Having such a model will not scale. I think it it's mentioned in earlier posts as well as some wikis available out there Kafka works very well if you have more partitions, so you can massively parallel writes to Kafka. Also, your application need not send partition ids/numbers, unless you've

Re: Re: Re: Re: Re: Kafka lost data issue

Hi Gwen, If producer cant get a response but the message got committed, Because of retries, can this producer be committing messages more than once?? I'm trying to see this problem from the point of view of Idempotent Producer Thanks, Prabhjot On Nov 13, 2015 1:26 PM, "Hawin Jiang"

Re: Re: Kafka lost data issue

Hi, Just to confirm that the number of messages produced are what you are seeing, What does GetOffsetShell report for this topic ? Regards, Prabhjot On Thu, Nov 12, 2015 at 2:13 PM, Hawin Jiang wrote: > Hi Jinxing > > I don't think we can resolve this issue by

Re: Locality question

Hi Ben, I have a similar use case where data from processing layer gets onto storage layer, where data per db gets distributed amongst storage machines based on some logic. The architecture that we had agreed upon was to have topics as storage machine names so the processing layer can send all

Re: Kafka Out Of Memory Error

Hi, Not sure. But, I had hit OOM when using too many partitions and many topics with a lesser heap size assigned to the jvm Regards, Prabhjot On Thu, Nov 5, 2015 at 10:40 AM, Muqtafi Akhmad wrote: > Hello all, > Recently I got incident with kafka cluster, I found

Re: Kafka Out Of Memory Error

0 total partitions. Is there any rule of > thumb regarding this matter? Can you share your configuration (including > spec, jvm memory, etc) for kafka cluster? > > Thank you, > > On Thu, Nov 5, 2015 at 4:07 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi, &g

Re: Controlled shutdown

Hi Vadim, Did you see your hard disk partition getting full where kafka data directory is present ? It could be because you have set log retention to a larger value, whereas your input data may be taking up full disk space. In that case, move some data out from that disk partition, set log

Re: New and updated producers and consumers

Adding users as well On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > I'm using the latest update: 0.8.2.2 > I would like to use the latest producer and consumer apis > over the past few weeks, I have tried to do some performance

Debug kafka code in intellij

Hi, I'm using kafka 0.8.2.1 version with IntelliJ. Sometimes, I change the code and build it using this command: ./gradlew -PscalaVersion=2.11.7 releaseTarGz In some cases, I feel the need for debugging the code within IntelliJ. e.g. I, currently, want to debug the ConsumerOffsetCheker to see

Re: New and updated producers and consumers

as simple) Thanks, Prabhjot Thanks, Prabhjot On Thu, Nov 5, 2015 at 3:55 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Adding users as well > > On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > >> Hi, >> >> I'

Known issues or points of failure during consumer rebalance

Hello Folks, I am evaluating some failure scenarios during consumer rebalance in the high-level consumer. The idea of this test is to know what are the pain points from operational/maintanence stand poitn that I need to consider when a consumer rebalance takes place. Also, if there are any known

Re: New and updated producers and consumers

> be > > > the only consumer client that fully supports security features like SSL > > > that are slated to be released into the platform. > > > > > > Your choice of development language is entirely up to you. Note that > the > > > only version of clien

Re: Debug kafka code in intellij

Hello Folks, Requesting your expertise on this Thanks, Prabhjot On Thu, Nov 5, 2015 at 4:46 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > I'm using kafka 0.8.2.1 version with IntelliJ. > Sometimes, I change the code and build it using this command: > >

Re: Debug kafka code in intellij

y test to run in intellij. Say, to set some > breakpoints > > > and > > > > > > > debug... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >

Re: Debug kafka code in intellij

t: /Users/gwen/workspaces/kafka/clients/build/classes/main > Test output: /Users/gwen/workspaces/kafka/clients/build/classes/test > > Make sure you have something similar. > > Gwen > > On Thu, Nov 5, 2015 at 9:24 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrot

Re: Debug kafka code in intellij

; > > > Radek Gruchalski > > > > > ra...@gruchalski.com (mailto:ra...@gruchalski.com) (mailto: > > > > > ra...@gruchalski.com (mailto:ra...@gruchalski.com)) > > > > > de.linkedin.com/in/radgruchalski/ ( > > http://de.linkedin.com/in/radgruchal

Re: Kafka console producer stuck

Hi, It seems (from VerifiableProperties.scala - https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/utils/VerifiableProperties.scala#L224) that you are providing some property which kafka does not recognise Please share the full command that you are trying to use Regards,

Re: Queue Full

Hi, This is a type of problem where you operating more than network capacity. This can be handled at two places (you decide whichever is useful/practical for use case) :- 1. In case the bottleneck is because of the broker slowness, increase the number of partitions of your topic, balance them

Re: Urban Airship Connect

days/100GB and double replication. Most of the time end-to-end is less than > 500ms, but that's subject to the latency of the decoration services. > > On Fri, Oct 9, 2015 at 1:23 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi, > > > > Nice to

Re: Is kafka 8 and LeaderNotAvailableError

Wow. Nice. So, what all did you change in your cluster?? Regards, Prabhjot On Oct 11, 2015 7:58 PM, "David Montgomery" <davidmontgom...@gmail.com> wrote: > Its working great now. Thanks > > On Sat, Oct 10, 2015 at 3:35 AM, Prabhjot Bharaj <prabhbha...@gmail.com&g

Re: Is kafka 8 and LeaderNotAvailableError

Hi, Is this a valid zookeeper connection string 1.zk.do.production.test:2181,2.zk.do.production.test:2181,3.zk.do.production.test:2181 Is 2.zk.do.production.test or 3.zk.do.production.test a valid hostname?? Regards, Prabhjot On Oct 9, 2015 1:37 PM, "David Montgomery"

Re: Urban Airship Connect

Hi, Nice to know another usage of Kafka Could you tell how much load does the first and second level Kafka clusters intake? What is your write throughput, and end-to-end latency, for how long do you hold data in Kafka and how many minimum in-sync replicas you've chosen? Thanks, Prabhjot On Oct

Re: Is kafka 8 and LeaderNotAvailableError

:26665,"version":1,"leader_epoch":3,"isr":[26665]} > >> for example > {"version":1,"partitions":{"1":[2028,26665],"0":[26665,2028]}} > >> > >> But now I do get the below on describe: > >> roduct

Re: Is kafka 8 and LeaderNotAvailableError

describe: > > roduction PartitionCount:2 ReplicationFactor:2 Configs: > > Topic: topic-test-production Partition: 0 Leader: 26665 Replicas: > > 26665,2028 Isr: 26665 > > Topic: topic-test-production Partition: 1 Leader: 26665 Replicas: > > 2028,26665 Isr: 26665 > > > > Bu

Re: Is kafka 8 and LeaderNotAvailableError

kafka.conn:140388282590976:DEBUG:20143:Read > >> 42/42 bytes from Kafka > >> 2015-10-10 > >> > 00:17:38,143.143.313884735:kafka.client:140388282590976:DEBUG:20143:Response > >> 2: MetadataResponse(brokers=[], > >> topics=[TopicMetadata(topic='topic-

Re: Is kafka 8 and LeaderNotAvailableError

ar/lib/zookeeper > clientPort=2181 > > server.1=45.55.xxx.xxx:2888:3888 > server.3=104.236.xxx.xxx:2888:3888 > server.2=192.241.xxx.xxx:2888:3888 > > > On Sat, Oct 10, 2015 at 3:02 AM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi, > > >

Re: Producer becomes slow over time

s started on a > machine that is not part of the kafka cluster? > > Can you include your topic description/configuration, producer > configuration, and broker configuration? > > On 9/24/15, 1:44 AM, "Prabhjot Bharaj" <prabhbha...@gmail.com> wrote: > > >Hi, > >

Re: using kafka

Hi, If you put n different consumers in different consumer groups, each consumer will get the same message. Each consumer gets full data But, if you put n consumers in 1 consumer group, it'll act as a traditional distributed queue. Amortised, each consumer will get 1/n of the overall data

Re: Producer becomes slow over time

, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hello Folks, > > I've noticed that 2 producer machines, that I had configured, have become > very slow over time > They are giving 17-19 MB/s > > But, a producer that I setup today is giving 70MB/s as the write throughpu

Producer becomes slow over time

Hello Folks, I've noticed that 2 producer machines, that I had configured, have become very slow over time They are giving 17-19 MB/s But, a producer that I setup today is giving 70MB/s as the write throughput If I see the contents of bin, libs, config directories, nothing is different in the

Re: Will Mirror Maker only support 1 to 1?

Hi, Can I have 2 separate mirror maker processes in this format:- Process 1 - source: 2, target: 1 Process 2 - source: 3, target: 1 If this is not supported, then will this circular kind of setup work ? Process 1 (on cluster 1) - source: 2, target: 1 Process 1 (on cluster 2) - source: 1,

Re: Useful metric to check slow ISR catchup

created a new topic with 1 partition and 3 replicas, I see similar graph on the other set of machines. I'm not sure what parameter could be causing this. Any pointers are appreciated Thanks, Prabhjot On Mon, Sep 21, 2015 at 1:20 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > He

Useful metric to check slow ISR catchup

Hi, I've noticed that 1 follower replica node out of my kafka cluster catches up to the data form the leader pretty slowly. My topic has just 1 partition with 3 replicas. One other follower replica gets the full data from the leader pretty instantly It takes around 22 minutes to catch up 500MB

Re: Jumbled up ISR

KA/Replication+tools#Replicationtools-1.PreferredReplicaLeaderElectionTool > > > Thank you, > Grant > > On Tue, Sep 15, 2015 at 5:26 AM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi, > > > > Topic:part_1_repl_3_3 PartitionCount:1 ReplicationFactor:3 Co

Re: What can be reason for fetcher thread for slow response.

Hi, In addition to the parameters asked by Erik, it would be great if you could share your broker's server.properties as well Regards, Prabhjot On Fri, Sep 11, 2015 at 8:10 PM, Helleren, Erik wrote: > Hi Madhukar, > Some questions that can help understand whats

Re: Slow ISR catch-up

, it is taking way longer. Where am I doing wrong? Thanks, Prabhjot On Fri, Sep 4, 2015 at 1:45 AM, Gwen Shapira <g...@confluent.io> wrote: > Yes, this should work. Expect lower throughput though. > > On Thu, Sep 3, 2015 at 12:52 PM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrot

Re: Slow ISR catch-up

Hello friends, Request your expertise on this problem I'm facing Thanks On Sep 4, 2015 8:09 PM, "Prabhjot Bharaj" <prabhbha...@gmail.com> wrote: > Hi, > > I am experiencing super slow throughput when using acks=-1 > Some further progress in continuation to t

Slow ISR catch-up

Hi Folks, Request your expertise on my doubt here. *My setup:-* 5 node kafka cluster (4 cores, 8GB RAM) on RAID-6 (500 GB) Using Kafka 0.8.2.1 with modified ProducerPerformance.scala I've modified ProducerPerformance.scala to send custom ASCII data, instead of Byte Array of Zeroes

Re: Slow ISR catch-up

and doesn't wait for acks, so it > is possible that the messages were never acked... > > On Thu, Sep 3, 2015 at 7:56 AM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi Folks, > > > > Request your expertise on my doubt here. > > > > *M

Re: kafka producer-perf-test.sh compression-codec not working

compression is concerned. If you do implement an improvement, a patch might be helpful to the community. But something to consider is that threwput alone isn’t the only important performance measure. Round trip latency is also important. Thanks, -Erik From: Prabhjot Bharaj prabhbha

Re: kafka producer-perf-test.sh compression-codec not working

producer produces, and the consumer looks at the difference between between the timestamp in the body and the current timestamp. Or, if you were looking for ack latency, you can use the producer’s async callback to measure latency. -Erik From: Prabhjot Bharaj prabhbha...@gmail.commailto:prabhbha

Re: kafka producer-perf-test.sh compression-codec not working

that a compression algorithm can detect and compress. -Erik On 8/25/15, 6:47 AM, Prabhjot Bharaj prabhbha...@gmail.com wrote: Hi, I have bene trying to use kafka-producer-perf-test.sh to arrive at certain benchmarks. When I try to run it with --compression-codec values of 1, 2 and 3, I notice

Raid vs individual disks

Hi, I've gone through the details mentioned about Raid and individual disks in the ops section of the documentation But, I would like to know what performance boost we can get with individual disks. Is anybody using Kafka with multiple disks or all are raid into 1 big disk ? Regards, Prabcs

Re: SocketTimeoutException with kafka-producer-perf-test.sh

Hi, Never mind. I have solved this problem by referring to an earlier question, where Neha had suggested to use a higher value for request-timeout-ms Regards, prabcs On Fri, Aug 21, 2015 at 12:22 PM, Prabhjot Bharaj prabhbha...@gmail.com wrote: Hello Folks, I'm using Kafka 0.8.2.1

kafka-producer-perf-test.sh - No visible difference between request-num-acks 1 and -1

Hi, I'm using Kafka 0.8.2.1 with the default zookeeper build that comes along the bundle I have setup a 5 machine cluster and on the same 5 machines, I'm also running zookeeper as well I am trying to see what is the maximum produce throughput I can get on this 5 node cluster I have created

Re: SocketTimeoutException with kafka-producer-perf-test.sh

This link already had the answer I was searching for: http://grokbase.com/t/kafka/users/145vmz70cb/java-net-sockettimeoutexception-in-broker On Fri, Aug 21, 2015 at 3:03 PM, Prabhjot Bharaj prabhbha...@gmail.com wrote: Hi, Never mind. I have solved this problem by referring to an earlier

SocketTimeoutException with kafka-producer-perf-test.sh

Hello Folks, I'm using Kafka 0.8.2.1 with the default zookeeper build that comes along the bundle I have setup a 5 machine cluster and on the same 5 machines, I'm also running zookeeper as well I am trying to see what is the maximum produce throughput I can get on this 5 node cluster I have

Zookeeper jmx monitoring for kafka

Hi, Zookeeper monitoring - I was going through the 'Monitoring' section in http://kafka.apache.org/coding-guide.html I have setup Ganglia to monitor all the stats inocming from kafka jmx port But, in addition, there are a bunch of stats that Zookeeper also exposes in its jmx port. They are

Re: Inconsistency with Zookeeper

Have you set broker id in zookeeper data directory and some unique broker Id in server.properties ? Regards, Prabhjot On Aug 6, 2015 1:43 PM, Hemanth Abbina heman...@eiqnetworks.com wrote: Hi, I am running a Kafka POC with below details * 3 Node cluster (4 Core, 16 GB RAM each)

Re: Best practices - Using kafka (with http server) as source-of-truth

proxies for load balancing, HA, SSL termination, etc. This is yet another hop, so it might affect throughput and latency. -Ewen On Mon, Jul 27, 2015 at 6:55 AM, Prabhjot Bharaj prabhbha...@gmail.com wrote: Hi Folks, I would like to understand the best practices when using kafka

Re: Connection to zk shell on Kafka

wrote: You need the jline JAR file that ships with ZooKeeper. Chris IBM Tivoli Systems Research Triangle Park, NC (919) 224-2240 Internet: barl...@us.ibm.com From: Prabhjot Bharaj prabhbha...@gmail.com To: users@kafka.apache.org, u...@zookeeper.apache.org Date: 07/29/2015 01

Connection to zk shell on Kafka

Hi folks, */kafka/bin# ./zookeeper-shell.sh localhost:2182/* *Connecting to localhost:2182/* *Welcome to ZooKeeper!* *JLine support is disabled* *WATCHER::* *WatchedEvent state:SyncConnected type:None path:null* *The shell never says connected* I'm running 5 node zookeeper cluster on

Re: Number of kafka topics/partitions supported per cluster of n nodes

like to see these! On Mon, Jul 27, 2015 at 10:51 PM, Prabhjot Bharaj prabhbha...@gmail.com wrote: Hi, I'm looking forward to a benchmark which can explain how many total number of topics and partitions can be created in a cluster of n nodes, given the message size varies between x

Re: Number of kafka topics/partitions supported per cluster of n nodes

@Jiefu Gong, Are the results of your tests available publicly? Regards, Prabhjot On Tue, Jul 28, 2015 at 10:35 PM, Prabhjot Bharaj prabhbha...@gmail.com wrote: I would be using the servers available at my place of work. I dont have access to AWS servers. I would starting off with a small

KAfka Mirror Maker

Hi, I'm using Mirror Maker with a cluster of 3 nodes and cluster of 5 nodes. I would like to ask - is the number of nodes a restriction for Mirror Maker? Also, are there any other restrictions or properties that should be common across both the clusters so that they continue mirroring. I'm

Number of kafka topics/partitions supported per cluster of n nodes

Hi, I'm looking forward to a benchmark which can explain how many total number of topics and partitions can be created in a cluster of n nodes, given the message size varies between x and y bytes and how does it vary with varying heap sizes and how it affects the system performance. e.g. the

Best practices - Using kafka (with http server) as source-of-truth

Hi Folks, I would like to understand the best practices when using kafka as the source-of-truth, given the fact that I want to pump in data to Kafka using http methods. What are the current production configurations for such a use case:- 1. Kafka-http-client - is it scalable the way Nginx is ??

Re: multiple producer throughput

Hi, Have you tried with acks=1 and -1 as well? Please share the numbers and the message size Regards, Prabcs On Jul 27, 2015 10:24 PM, Yuheng Du yuheng.du.h...@gmail.com wrote: Hi, I am running 40 producers on 40 nodes cluster. The messages are sent to 6 brokers in another cluster. The

parametrized number of partitions for topics

Hello Folks, I would like to create custom partitions for various topics e.g. topic1 - 100 partitions, topic2 - 10 partitions I have using this github repository - https://github.com/edenhill/librdkafka I had posted a similar question: https://github.com/edenhill/librdkafka/issues/308, but I

Custom Zookeeper install with kafka

Hi, I've read on the Kafka documentation page that the zookeeper version used is 3.3.4 However, at my work, I've noticed certain problems with v3.3.4 (and the problems are documented here: http://zookeeper.apache.org/releases.html The latest stable version of zookeeper is 3.4.6 and has a lot of