Bill created GEODE-5816:
---------------------------
Summary: ClusterStartupRule fails to launch JMX manager (port
already in use)
Key: GEODE-5816
URL: https://issues.apache.org/jira/browse/GEODE-5816
Project: Geode
Issue Type: Bug
Components: tests
Reporter: Bill
We see these related failures on a couple tests in
RegionMembershipMBeanOverHttpDUnitTest from our recent mass-test-run.
>From looking at the stack traces though, we surmise that the problem occurs in
>the ClusterStartupRule _before_ the actual tests run. Since it occurs before
>the tests run, we think the problem lies outside the
>RegionMembershipMBeanOverHttpDUnitTest class.
{noformat}
4 failures (99.600% success
rate)org.apache.geode.management.internal.cli.commands.RegionMembershipMBeanOverHttpDUnitTest
| .testAddRmNewMemberWithReplicatedRegionsAndSubregions: 1 failures (99.900%
success rate)
| | Failed build 24 at
https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/24
| .testMultiplePartitionedRegions: 3 failures (99.700% success rate)
| | Failed build 982 at
https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/982
| | Failed build 256 at
https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/256
{noformat}
Here's a stack trace:
{noformat}
java.lang.AssertionError: Suspicious strings were written to the log during
this run.
Fix the strings or use IgnoredException.addIgnoredException to ignore.
-----------------------------------------------------------------------
Found suspect string in log4j at line 358
[error 2018/10/01 06:28:30.869 UTC <RMI TCP Connection(1)-172.17.0.20>
tid=0x20] Jmx manager could not be started because
java.rmi.server.ExportException: Port already in use: 25305; nested exception
is:
java.net.BindException: Failed to create server socket on
ffd50f3577c5/172.17.0.20[25305]
org.apache.geode.management.ManagementException:
java.rmi.server.ExportException: Port already in use: 25305; nested exception
is:
java.net.BindException: Failed to create server socket on
ffd50f3577c5/172.17.0.20[25305]
at
org.apache.geode.management.internal.ManagementAgent.startAgent(ManagementAgent.java:162)
at
org.apache.geode.management.internal.SystemManagementService.startManager(SystemManagementService.java:435)
at
org.apache.geode.management.internal.beans.ManagementAdapter.handleCacheCreation(ManagementAdapter.java:173)
at
org.apache.geode.management.internal.beans.ManagementListener.handleEvent(ManagementListener.java:118)
at
org.apache.geode.distributed.internal.InternalDistributedSystem.notifyResourceEventListeners(InternalDistributedSystem.java:2201)
at
org.apache.geode.distributed.internal.InternalDistributedSystem.handleResourceEvent(InternalDistributedSystem.java:591)
at
org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1218)
at
org.apache.geode.internal.cache.GemFireCacheImpl.basicCreate(GemFireCacheImpl.java:793)
at
org.apache.geode.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:779)
at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:177)
at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:224)
at
org.apache.geode.distributed.internal.InternalLocator.startCache(InternalLocator.java:662)
at
org.apache.geode.distributed.internal.InternalLocator.startDistributedSystem(InternalLocator.java:649)
at
org.apache.geode.distributed.internal.InternalLocator.startLocator(InternalLocator.java:311)
at org.apache.geode.distributed.Locator.startLocator(Locator.java:253)
at
org.apache.geode.distributed.Locator.startLocatorAndDS(Locator.java:140)
at
org.apache.geode.test.junit.rules.LocatorStarterRule.startLocator(LocatorStarterRule.java:87)
at
org.apache.geode.test.junit.rules.LocatorStarterRule.before(LocatorStarterRule.java:68)
at
org.apache.geode.test.dunit.rules.ClusterStartupRule.lambda$startLocatorVM$22d9b8a8$1(ClusterStartupRule.java:206)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hydra.MethExecutor.executeObject(MethExecutor.java:244)
at
org.apache.geode.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:70)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.rmi.server.ExportException: Port already in use: 25305; nested
exception is:
java.net.BindException: Failed to create server socket on
ffd50f3577c5/172.17.0.20[25305]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:346)
at
sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254)
at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
at
sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236)
at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:173)
at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:144)
at
java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
at
org.apache.geode.management.internal.ManagementAgent.configureAndStart(ManagementAgent.java:456)
at
org.apache.geode.management.internal.ManagementAgent.startAgent(ManagementAgent.java:160)
... 40 more
Caused by: java.net.BindException: Failed to create server socket on
ffd50f3577c5/172.17.0.20[25305]
at
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:758)
at
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:715)
at
org.apache.geode.management.internal.ManagementAgent$GemFireRMIServerSocketFactory.createServerSocket(ManagementAgent.java:615)
at
sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335)
... 50 more
Caused by: java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:754)
... 54 more
-----------------------------------------------------------------------
Found suspect string in log4j at line 431
[error 2018/10/01 06:28:30.872 UTC <RMI TCP Connection(1)-172.17.0.20>
tid=0x20] org.apache.geode.management.ManagementException:
java.rmi.server.ExportException: Port already in use: 25305; nested exception
is:
java.net.BindException: Failed to create server socket on
ffd50f3577c5/172.17.0.20[25305]
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.geode.test.dunit.standalone.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:411)
at
org.apache.geode.test.dunit.rules.ClusterStartupRule.after(ClusterStartupRule.java:172)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
at
org.apache.geode.test.junit.rules.DescribedExternalResource$1.evaluate(DescribedExternalResource.java:40)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
at
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
at
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137)
at
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The three stack traces are almost identical.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
