10 connection maximum is too low. It has been recommended to go up to as many as 2000 connections
in the list. This doesn't fix your problem but is something you should probably have in your
configuration.
~Jeff
On 7/28/2011 10:00 AM, Stack wrote:
Try getting the ZooKeeperWatcher from the connection on your way out
and explicitly shutdown the zk connection (see TestZooKeeper unit test
for example).
St.Ack
On Thu, Jul 28, 2011 at 6:01 AM, Andre Reiter<[email protected]> wrote:
this issue is still not resolved...
unfortunatelly calling HConnectionManager.deleteConnection(conf, true);
after the MR job is finished, does not close the connection to the zookeeper
we have 3 zookeeper nodes
by default there is a limit of 10 connections allowed from a single client
so after running 30 MR jobs scheduled by our application, we have 30
unclosed connections, trying to start a new MR job results in a failure, the
connection to the zookeeper ensamble is droped...
the work around to restart the whole application after 30 MR jobss is not
very elegant... :-(
--
Jeff Whiting
Qualtrics Senior Software Engineer
[email protected]
