Dan Smith created GEODE-3958:
--------------------------------
Summary: TomcatSessionBackwardsCompatibilityTest fails to shutdown
server due to non daemon client queue initialization thread
Key: GEODE-3958
URL: https://issues.apache.org/jira/browse/GEODE-3958
Project: Geode
Issue Type: Bug
Components: client/server
Reporter: Dan Smith
TomcatSessionBackwardsCompatibilityTest has been failing the nightly builds
with the below error message.
{noformat}
org.junit.ComparisonFailure: [....The Cache Server process terminated
unexpectedly with exit status 1. Please refer to the log file in
/Users/dsmith/Documents/Code/gemfire/open/geode-assembly/build/distributedTest/server
for full details.
Exception in thread "main" java.lang.RuntimeException: A PID file already
exists and a Server may be running in
/Users/dsmith/Documents/Code/gemfire/open/geode-assembly/build/distributedTest/server
on 10.118.19.21[0].
at
org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:810)
at
org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:692)
at
org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:226)
Caused by: org.apache.geode.internal.process.FileAlreadyExistsException: Pid
file already exists:
/Users/dsmith/Documents/Code/gemfire/open/geode-assembly/build/distributedTest/server/vf.gf.server.pid
for process 4524
at
org.apache.geode.internal.process.LocalProcessLauncher.checkOtherPid(LocalProcessLauncher.java:151)
at
org.apache.geode.internal.process.LocalProcessLauncher.writePid(LocalProcessLauncher.java:122)
at
org.apache.geode.internal.process.LocalProcessLauncher.<init>(LocalProcessLauncher.java:70)
at
org.apache.geode.internal.process.ControllableProcess.createLocalProcessLauncher(ControllableProcess.java:158)
at
org.apache.geode.internal.process.ControllableProcess.<init>(ControllableProcess.java:55)
at
org.apache.geode.internal.process.ControllableProcess.<init>(ControllableProcess.java:47)
at
org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:746)
... 2 more
{noformat}
I was able to reproduce this failure locally. What's happening is that the
first test runs successfully, and then the remaining tests fail with this error
because there is a server process running. The test is using gfsh to start and
stop the server in the same directory.
I got a stack trace and logsfrom the running server. From the the logs of the
server, it did get the shutdown message.
{noformat}
[info 2017/11/06 17:57:49.690 PST server <Thread-22> tid=0x74]
DistributionManager stopped in 109ms.
[info 2017/11/06 17:57:49.691 PST server <Thread-22> tid=0x74] Marking
DistributionManager 10.118.19.21(server:4524)<v1>:1025 as closed.
{noformat}
However, the process did not die because there is still a non daemon thread
running
{noformat}
"Client Queue Initialization Thread 0" #89 prio=5 os_prio=31
tid=0x00007f8b1be8f800 nid=0xb903 waiting on condition [0x000070000bf17000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000076b5ea308> (a
java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl$3$1.run(AcceptorImpl.java:610)
at java.lang.Thread.run(Thread.java:748)
{noformat}
This thread was added recently for GEODE-3637. It needs to be cleaned up as
part of member shutdown.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
