See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/863/
################################################################################### ########################## LAST 60 LINES OF THE CONSOLE ########################### [...truncated 60.87 KB...] [junit] Running org.apache.zookeeper.test.SaslAuthMissingClientConfigTest in thread 3 [junit] Running org.apache.zookeeper.test.SaslClientTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.067 sec, Thread: 4, Class: org.apache.zookeeper.test.SaslClientTest [junit] Running org.apache.zookeeper.test.SaslSuperUserTest in thread 7 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.68 sec, Thread: 3, Class: org.apache.zookeeper.test.SaslAuthMissingClientConfigTest [junit] Running org.apache.zookeeper.test.ServerCnxnTest in thread 6 [junit] Running org.apache.zookeeper.test.SessionInvalidationTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.658 sec, Thread: 7, Class: org.apache.zookeeper.test.SaslSuperUserTest [junit] Running org.apache.zookeeper.test.SessionTest in thread 3 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.642 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionInvalidationTest [junit] Running org.apache.zookeeper.test.SessionTrackerCheckTest in thread 7 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec, Thread: 7, Class: org.apache.zookeeper.test.SessionTrackerCheckTest [junit] Running org.apache.zookeeper.test.SessionUpgradeTest in thread 4 [junit] Running org.apache.zookeeper.test.StandaloneTest in thread 7 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.504 sec, Thread: 6, Class: org.apache.zookeeper.test.ServerCnxnTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.524 sec, Thread: 7, Class: org.apache.zookeeper.test.StandaloneTest [junit] Running org.apache.zookeeper.test.StatTest in thread 6 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.7 sec, Thread: 6, Class: org.apache.zookeeper.test.StatTest [junit] Running org.apache.zookeeper.test.StaticHostProviderTest in thread 7 [junit] Running org.apache.zookeeper.test.StringUtilTest in thread 6 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.066 sec, Thread: 6, Class: org.apache.zookeeper.test.StringUtilTest [junit] Running org.apache.zookeeper.test.SyncCallTest in thread 6 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.651 sec, Thread: 6, Class: org.apache.zookeeper.test.SyncCallTest [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.773 sec, Thread: 7, Class: org.apache.zookeeper.test.StaticHostProviderTest [junit] Running org.apache.zookeeper.test.TruncateTest in thread 7 [junit] Running org.apache.zookeeper.test.WatchEventWhenAutoResetTest in thread 6 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.131 sec, Thread: 7, Class: org.apache.zookeeper.test.TruncateTest [junit] Running org.apache.zookeeper.test.WatchedEventTest in thread 7 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.071 sec, Thread: 7, Class: org.apache.zookeeper.test.WatchedEventTest [junit] Running org.apache.zookeeper.test.WatcherFuncTest in thread 7 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.982 sec, Thread: 7, Class: org.apache.zookeeper.test.WatcherFuncTest [junit] Running org.apache.zookeeper.test.WatcherTest in thread 7 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.66 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionUpgradeTest [junit] Running org.apache.zookeeper.test.X509AuthTest in thread 4 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.084 sec, Thread: 4, Class: org.apache.zookeeper.test.X509AuthTest [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.528 sec, Thread: 6, Class: org.apache.zookeeper.test.WatchEventWhenAutoResetTest [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest in thread 6 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.67 sec, Thread: 6, Class: org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 14, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 82.401 sec, Thread: 2, Class: org.apache.zookeeper.test.QuorumTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.726 sec, Thread: 5, Class: org.apache.zookeeper.test.QuorumZxidSyncTest [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.636 sec, Thread: 3, Class: org.apache.zookeeper.test.SessionTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.757 sec, Thread: 4, Class: org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.8 sec, Thread: 7, Class: org.apache.zookeeper.test.WatcherTest [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 251.421 sec, Thread: 1, Class: org.apache.zookeeper.test.ReconfigTest [junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 365.413 sec, Thread: 8, Class: org.apache.zookeeper.test.NettyNettySuiteTest fail.build.on.test.failure: BUILD FAILED /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build.xml:1276: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build.xml:1280: Tests failed! Total time: 9 minutes 16 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ################################################################################### ############################## FAILED TESTS (if any) ############################## 26 tests failed. FAILED: org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes Error Message: Waiting for server up Stack Trace: junit.framework.AssertionFailedError: Waiting for server up at org.apache.zookeeper.test.QuorumUtil.restart(QuorumUtil.java:216) at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.followerResyncCrashTest(FollowerResyncConcurrencyTest.java:237) at org.apache.zookeeper.test.FollowerResyncConcurrencyTest.testResyncBySnapThenDiffAfterFollowerCrashes(FollowerResyncConcurrencyTest.java:176) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) FAILED: org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:966) at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204) at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145) at org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:818) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.ReconfigTest.testPortChange Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:966) at org.apache.zookeeper.test.QuorumUtil.start(QuorumUtil.java:204) at org.apache.zookeeper.test.QuorumUtil.startAll(QuorumUtil.java:145) at org.apache.zookeeper.test.ReconfigTest.testPortChange(ReconfigTest.java:591) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader Error Message: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.lang.RuntimeException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:722) at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortLeader(ReconfigTest.java:718) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:966) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion Error Message: unable to create new native thread Stack Trace: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ClientCommunicatorAdmin.<init>(ClientCommunicatorAdmin.java:45) at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.<init>(RMIConnector.java:1493) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:339) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testInitialConfigHasPositiveVersion(ReconfigTest.java:893) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) FAILED: org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testRemoveAddOne(ReconfigTest.java:250) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testRemoveAddTwo(ReconfigTest.java:332) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPort(ReconfigTest.java:722) at org.apache.zookeeper.test.ReconfigTest.testPortChangeToBlockedPortFollower(ReconfigTest.java:714) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRemoveAddOne(ReconfigTest.java:912) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.ReconfigTest.testRoleChange Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testRoleChange(ReconfigTest.java:512) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.ReconfigTest.testBulkReconfig Error Message: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.lang.RuntimeException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testBulkReconfig(ReconfigTest.java:432) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:966) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:117) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testJMXBeanAfterRoleChange(ReconfigTest.java:991) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous Error Message: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:122) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:127) at org.apache.zookeeper.test.ReconfigTest.testRemoveOneAsynchronous(ReconfigTest.java:471) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) Caused by: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumUtil.<init>(QuorumUtil.java:90) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.SessionTest.testSessionTimeout Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.SessionTest.testMinMaxSessionTimeout Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.SessionTest.testSessionMove Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.SessionTest.testSessionTimeoutAccess Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.SessionTest.testSession Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.SessionTest.testSessionStateNoDupStateReporting Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.SessionTest.setUp(SessionTest.java:77) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending Error Message: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512) at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.WatcherTest.testWatcherCorrectness Error Message: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread Stack Trace: java.rmi.ServerError: Error occurred in server thread; nested exception is: java.lang.OutOfMemoryError: unable to create new native thread at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:386) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:512) at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at com.sun.jmx.remote.internal.ServerCommunicatorAdmin.<init>(ServerCommunicatorAdmin.java:50) at javax.management.remote.rmi.RMIConnectionImpl$RMIServerCommunicatorAdmin.<init>(RMIConnectionImpl.java:1321) at javax.management.remote.rmi.RMIConnectionImpl.<init>(RMIConnectionImpl.java:164) at javax.management.remote.rmi.RMIJRMPServerImpl.makeClient(RMIJRMPServerImpl.java:207) at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:250) at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.zookeeper.test.WatcherTest.testWatcherAutoResetDisabledWithLocal Error Message: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory Stack Trace: java.io.IOException: Couldn't instantiate org.apache.zookeeper.server.NettyServerCnxnFactory at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:138) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:154) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:148) at org.apache.zookeeper.test.ClientBase.createNewServerInstance(ClientBase.java:446) at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:525) at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:518) at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76) Caused by: java.lang.reflect.InvocationTargetException at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:134) Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378) at org.jboss.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:368) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52) at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44) at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28) at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39) at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:149) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:131) at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.<init>(NioServerSocketChannelFactory.java:115) at org.apache.zookeeper.server.NettyServerCnxnFactory.<init>(NettyServerCnxnFactory.java:332) FAILED: org.apache.zookeeper.test.QuorumHammerTest.testHammerBasic Error Message: waiting for server up Stack Trace: junit.framework.AssertionFailedError: waiting for server up at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78) at org.apache.zookeeper.test.QuorumHammerTest.setUp(QuorumHammerTest.java:36) FAILED: org.apache.zookeeper.test.QuorumMajorityTest.testMajQuorums Error Message: unable to create new native thread Stack Trace: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at org.apache.zookeeper.server.quorum.QuorumPeer.createElectionAlgorithm(QuorumPeer.java:1049) at org.apache.zookeeper.server.quorum.QuorumPeer.startLeaderElection(QuorumPeer.java:937) at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:861) at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:214) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78) FAILED: org.apache.zookeeper.test.QuorumMajorityTest.testNull Error Message: waiting for server up Stack Trace: junit.framework.AssertionFailedError: waiting for server up at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:223) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:120) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78) FAILED: org.apache.zookeeper.test.QuorumQuotaTest.testQuotaWithQuorum Error Message: error during JRMP connection establishment; nested exception is: java.net.SocketException: Connection reset Stack Trace: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: java.net.SocketException: Connection reset at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:307) at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at org.apache.zookeeper.test.JMXEnv.setUp(JMXEnv.java:58) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:85) at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:78) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) at java.io.DataInputStream.readByte(DataInputStream.java:265) at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:246)