ZooKeeper-trunk-jdk8 - Build # 670 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/670/

###################################################################################
########################## LAST 60 LINES OF THE CONSOLE 
###########################
[...truncated 372548 lines...]
    [junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:16681
    [junit]     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit]     at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit]     at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit]     at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit]     at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit]     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    [junit]     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    [junit]     at java.lang.Thread.run(Thread.java:744)
    [junit] 2016-07-16 12:05:22,832 [myid:] - INFO  [New I/O boss 
#8184:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2016-07-16 12:05:22,832 [myid:127.0.0.1:16681] - INFO  
[main-SendThread(127.0.0.1:16681):ClientCnxn$SendThread@1231] - channel for 
sessionid 0x0 is lost, closing socket connection and attempting reconnect
    [junit] 2016-07-16 12:05:24,106 [myid:127.0.0.1:16681] - INFO  
[main-SendThread(127.0.0.1:16681):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:16681. Will not attempt to 
authenticate using SASL (unknown error)
    [junit] 2016-07-16 12:05:24,107 [myid:] - INFO  [New I/O boss 
#8184:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
    [junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:16681
    [junit]     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit]     at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit]     at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit]     at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit]     at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit]     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    [junit]     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    [junit]     at java.lang.Thread.run(Thread.java:744)
    [junit] 2016-07-16 12:05:24,107 [myid:] - WARN  [New I/O boss 
#8184:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0x051df62a] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:16681
    [junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:16681
    [junit]     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    [junit]     at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    [junit]     at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    [junit]     at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    [junit]     at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    [junit]     at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    [junit]     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    [junit]     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    [junit]     at java.lang.Thread.run(Thread.java:744)
    [junit] 2016-07-16 12:05:24,108 [myid:] - INFO  [New I/O boss 
#8184:ClientCnxnSocketNetty@208] - channel is told closing
    [junit] 2016-07-16 12:05:24,108 [myid:127.0.0.1:16681] - INFO  
[main-SendThread(127.0.0.1:16681):ClientCnxn$SendThread@1231] - channel for 
sessionid 0x0 is lost, closing socket connection and attempting reconnect
    [junit] Running org.apache.zookeeper.test.NettyNettySuiteTest in thread 3
    [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 
sec, Thread: 3, Class: org.apache.zookeeper.test.NettyNettySuiteTest
    [junit] Test org.apache.zookeeper.test.NettyNettySuiteTest FAILED (timeout)

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk-jdk8/trunk/build.xml:1207:
 Tests failed!

Total time: 19 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###################################################################################
############################## FAILED TESTS (if any) 
##############################
41 tests failed.
FAILED:  
org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testLaggingFollowerResyncsUnderNewEpoch

Error Message:
Waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: Waiting for server up
        at 
org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testLaggingFollowerResyncsUnderNewEpoch(FollowerResyncConcurrencyTest.java:126)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  
org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes

Error Message:
Did not connect

Stack Trace:
java.util.concurrent.TimeoutException: Did not connect
        at 
org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:132)
        at 
org.apache.zookeeper.test.FollowerResyncConcurrencyTest.createClient(FollowerResyncConcurrencyTest.java:547)
        at 
org.apache.zookeeper.test.FollowerResyncConcurrencyTest.followerResyncCrashTest(FollowerResyncConcurrencyTest.java:232)
        at 
org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:176)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  
org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors

Error Message:
KeeperErrorCode = ConnectionLoss for /invaliddir/test-

Stack Trace:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = 
ConnectionLoss for /invaliddir/test-
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1412)
        at 
org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors(LoadFromLogTest.java:452)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.LoadFromLogTest.testLoad

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:414)
        at sun.nio.ch.Net.bind(Net.java:406)
        at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.LoadFromLogTest.testLoad(LoadFromLogTest.java:87)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.LoadFromLogTest.testDatadirAutocreate

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:414)
        at sun.nio.ch.Net.bind(Net.java:406)
        at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.LoadFromLogTest.testDatadirAutocreate(LoadFromLogTest.java:495)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.LoadFromLogTest.testRestore

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:414)
        at sun.nio.ch.Net.bind(Net.java:406)
        at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.LoadFromLogTest.testRestore(LoadFromLogTest.java:361)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.LoadFromLogTest.testLoadFailure

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:414)
        at sun.nio.ch.Net.bind(Net.java:406)
        at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:153)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.LoadFromLogTest.testLoadFailure(LoadFromLogTest.java:148)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testHammer

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
org.apache.zookeeper.test.AsyncHammerTest.testHammer(AsyncHammerTest.java:177)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at 
org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:175)
        at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
        at 
org.apache.zookeeper.test.AsyncHammerTest.setUp(AsyncHammerTest.java:52)
        at 
org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer(AsyncHammerTest.java:203)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.NettyNettySuiteTest.testPortChange

Error Message:
Timeout occurred. Please note the time in the report does not reflect the time 
until the timeout.

Stack Trace:
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in 
the report does not reflect the time until the timeout.
        at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testHammer

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
        at 
org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
        at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
        at 
org.apache.zookeeper.test.AsyncHammerTest.setUp(AsyncHammerTest.java:52)
        at 
org.apache.zookeeper.test.AsyncHammerTest.testHammer(AsyncHammerTest.java:171)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:753)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testPortChange

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testPortChange(ReconfigTest.java:548)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  
org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:666)
        at 
org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader(ReconfigTest.java:662)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  
org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server 
thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion(ReconfigTest.java:829)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested 
exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:351)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)
        at 
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at 
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown 
Source)
        at 
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
        at 
javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1319)
        at 
javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:160)
        at 
javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:173)
        at 
javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
        at 
javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204)
        at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145)
        at 
org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne(ReconfigTest.java:222)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo(ReconfigTest.java:297)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  
org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:863)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:666)
        at 
org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower(ReconfigTest.java:658)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
        at 
javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne(ReconfigTest.java:848)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRoleChange

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server 
thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testRoleChange(ReconfigTest.java:474)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested 
exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:351)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)
        at 
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at 
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown 
Source)
        at 
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
        at 
javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1319)
        at 
javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:160)
        at 
javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:173)
        at 
javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
        at 
javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testBulkReconfig

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server 
thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testBulkReconfig(ReconfigTest.java:396)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested 
exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:351)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)
        at 
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at 
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown 
Source)
        at 
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
        at 
javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1319)
        at 
javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:160)
        at 
javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:173)
        at 
javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
        at 
javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange

Error Message:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server 
thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange(ReconfigTest.java:925)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Caused by: java.rmi.ServerError: Error occurred in server thread; nested 
exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:351)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)
        at 
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at 
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown 
Source)
        at 
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
        at 
javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1319)
        at 
javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:160)
        at 
javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:173)
        at 
javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
        at 
javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45)
        at 
javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1467)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90)
        at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127)
        at 
org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous(ReconfigTest.java:434)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testMinMaxSessionTimeout

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionMove

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSessionTimeoutAccess

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.SessionTest.testSession

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  
org.apache.zookeeper.test.SessionTest.testSessionStateNoDupStateReporting

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending

Error Message:
Error occurred in server thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread

Stack Trace:
java.rmi.ServerError: Error occurred in server thread; nested exception is: 
        java.lang.OutOfMemoryError: unable to create new native thread
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:351)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)
        at 
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
        at 
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
        at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown 
Source)
        at 
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2404)
        at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229)
        at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:454)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50)
        at 
javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1319)
        at 
javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:160)
        at 
javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:173)
        at 
javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250)
        at 
javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
        at sun.rmi.transport.Transport$1.run(Transport.java:178)
        at sun.rmi.transport.Transport$1.run(Transport.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
        at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
        at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherCorrectness

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:388)
        at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:467)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)


FAILED:  
org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithLocal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:388)
        at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:467)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithGlobal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:388)
        at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:467)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherCount

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:388)
        at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:467)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)


FAILED:  
org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithGlobal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:388)
        at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:467)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)


FAILED:  org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetWithLocal

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at 
org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
        at 
org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
        at 
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
        at 
org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131)
        at 
org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115)
        at 
org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at java.lang.Class.newInstance(Class.java:433)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:152)
        at 
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:146)
        at 
org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:388)
        at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:467)
        at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
        at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:73)


FAILED:  org.apache.zookeeper.test.QuorumHammerTest.testHammerBasic

Error Message:
Unable to connect to server

Stack Trace:
junit.framework.AssertionFailedError: Unable to connect to server
        at 
org.apache.zookeeper.test.ClientBase.createClient(ClientBase.java:188)
        at 
org.apache.zookeeper.test.ClientBase.createClient(ClientBase.java:177)
        at 
org.apache.zookeeper.test.ClientBase.createClient(ClientBase.java:162)
        at 
org.apache.zookeeper.test.ClientBase.createClient(ClientBase.java:155)
        at 
org.apache.zookeeper.test.ClientHammerTest.verifyHammer(ClientHammerTest.java:224)
        at 
org.apache.zookeeper.test.ClientHammerTest.runHammer(ClientHammerTest.java:142)
        at 
org.apache.zookeeper.test.ClientHammerTest.testHammerBasic(ClientHammerTest.java:120)
        at 
org.apache.zookeeper.test.QuorumHammerTest.testHammerBasic(QuorumHammerTest.java:49)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.QuorumMajorityTest.testMajQuorums

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
        at 
org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
        at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
        at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)


FAILED:  org.apache.zookeeper.test.QuorumTest.testViewContains

Error Message:
unable to create new native thread

Stack Trace:
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:713)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.createElectionAlgorithm(QuorumPeer.java:946)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.startLeaderElection(QuorumPeer.java:834)
        at 
org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:758)
        at 
org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:218)
        at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120)
        at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78)
        at org.apache.zookeeper.test.QuorumTest.setUp(QuorumTest.java:60)


FAILED:  org.apache.zookeeper.test.QuorumTest.testSessionMoved

Error Message:
KeeperErrorCode = ConnectionLoss for /

Stack Trace:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = 
ConnectionLoss for /
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:2253)
        at 
org.apache.zookeeper.test.QuorumTest.testSessionMoved(QuorumTest.java:214)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.QuorumZxidSyncTest.testLateLogs

Error Message:
waiting for server up

Stack Trace:
junit.framework.AssertionFailedError: waiting for server up
        at 
org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223)
        at 
org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:132)
        at 
org.apache.zookeeper.test.QuorumZxidSyncTest.testLateLogs(QuorumZxidSyncTest.java:127)
        at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)