Hi, I slightly disagree with the priority for a "connection pool" for zookeeper. I had to implement a connection pool/reuse for a smaller environment (under 50 nodes). I have a toolchain that can be composed of parts, and each part needed some synchronization through zookeeper. So even on a smaller system these heavier toolchains started using a lot of physical sessions and connections. Hence the connection pool. Also from performance perspective establishing a TCP connection for every session adds latency overhead.
Cheers Peco On Fri, Feb 18, 2011 at 11:58 AM, Benjamin Reed <[email protected]> wrote: > it does seem like a good idea to make multiple zk handles share a > connection, but as ted points out, they may have different timeouts, which > would make the sharing logic quite complicated. i think the implementation > might also be quite complicated. having said that, if someone could come up > with a simple and correct connection sharing implementation, we (or at > least > i) would be open to it. > > ben > > On Fri, Feb 18, 2011 at 9:31 AM, Ted Dunning <[email protected]> > wrote: > > > On Fri, Feb 18, 2011 at 1:24 AM, YUNG-LIN HO <[email protected]> wrote: > > > > > Because zookeeper clients will try to keep session alive by sending a > > ping > > > request every 2 seconds. If libraries in an application do not share > > > connections with each other, they would flood the zookeeper server with > > > unnecessary requests and drag down performance of the server. > > > > > > > Make sure that you have a valid reason to worry first. > > > > Do you have thousands of clients? > > > > If not, these keep-alives are likely to be undetectable, load-wise. > > > > > > > I am wondering is there any connection manager exists in the > > > Hadoop/Zookeeper project that helps users to share connections? > > > > > > > Yes. Zookeeper. > > > > Just open a single connection and pass it around via a singleton of some > > kind or your favorite dependency injection technique. > > > > This isn't always a great idea since your disconnect and expiration > > strategies might differ between different uses in important ways. > > >
