Actually I did noticed something interesting. I have noticed that the majority of the client connection towards zookeeper are being made from a Rest Service (rest api for hbase). So I guess that i need to understand why Rest keep opening new sessions. Any ideas??
And yes, I sent the same question to an HBase mailing list - no comments yet. thanks, On Sun, Oct 16, 2011 at 5:58 PM, Camille Fournier <[email protected]>wrote: > Seems like a hbase problem to me... Zk doesn't randomly create or hang on > to > client connections. Did you ask on the hbase lists? > > C > > On Sunday, October 16, 2011, Ronen Itkin <[email protected]> wrote: > > Hey all! > > How are you? > > > > I am currently using Zookeeper 3.3.3-cdh3u1, installed as a distributed > > cluster with 3 nodes. > > Though I have installed Zookeeper separately, its only purpose right now > is > > to serve HBase clients (hbase-0.90.3-cdh3u1). > > Seems like everything is working great for a day or two and that out the > > blue, Zookeeper refuses to serve clients requests, > > alerting for the wide known issue of exceeding the defined maximum > > connections allowed for a client. > > I changed this value to 500 and it seems that it just held on for more > time > > and finally reached the threshold of 500 and then stopped working. > > So, I set it to 1000 and right now I am waiting to see what will happen. > > > > Meanwhile, by auditing the Zookeeper connections manually it seems like > they > > keep growing in a slow - promising pace towards the threshold of 1000 > > connections. > > > > Can someone think of a reason why it happens? Why it keep growing? Does > > someone has any other suggestions regarding this issue? > > What is the recommended value for Zookeeper maximum connections per > client? > > can it overload the server if set to a high value? what is a high value > in > > this case? > > > > If it helps, this is my environment design: > > > > - Server 1 - Hadoop NameNode, JobTracker, *Hbase Master (Including > Rest)*, > > *Zookeeper01* > > - Server 2 - Hadoop SecondaryMaster, HBase *Backup Master (Including > > Rest)*, *Zookeeper02* > > - Server 3 - *Zookeeper03* > > - Servers 4-6 - Hadoop DataNode,TaskTracker, *HBase Region Server.* > > > > The type of operations I am committing on HBase using Zookeeper are: > > > > - Simple read/writes/updates using SCAN / MapReduce / Cascading .. > > - Queries of the Rest service. > > > > > > Thanks a lot !!! > > * *Ronen Itkin > > > > <http://www.taykey.com/> > > > -- * Ronen Itkin* Taykey | www.taykey.com
