> On June 8, 2015, 4:57 p.m., John Speidel wrote: > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/DeadlockWarningThread.java, > > line 79 > > <https://reviews.apache.org/r/35074/diff/2/?file=980027#file980027line79> > > > > why not findDeadlockedThreads()? > > Dmytro Shkvyra wrote: > It is not work if deadlock caused by use ReadWriteLock > > John Speidel wrote: > I am a bit confused by your response. > > The reason that I suggested findDeadlockedThreads() be used instead of > findMonitorDeadlockedThreads() is that it does detect deadlock involving > ReadWriteLock (as I understand) in addition to monitor locks. > > From the ThreadMXBean javadoc: > long[] findDeadlockedThreads() > Finds cycles of threads that are in deadlock waiting to acquire object > monitors or ownable synchronizers. Threads are deadlocked in a cycle waiting > for a lock of these two types if each thread owns one lock while trying to > acquire another lock already held by another thread in the cycle. > > I believe that ReadWriteLock is an example of an 'ownable synchronizer' > that is supported. > From the LockInfo javadoc: > An ownable synchronizer is a synchronizer that may be exclusively owned > by a thread and uses AbstractOwnableSynchronizer (or its subclass) to > implement its synchronization property. ReentrantLock and > ReentrantReadWriteLock are two examples of ownable synchronizers provided by > the platform. > > Have you seen behavior that would indicate otherwise? > > Dmytro Shkvyra wrote: > I have tried findDeadlockedThreads() first of all, seems to > http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6207928 was not fixed > properly, so findDeadlockedThreads() works the same as > findMonitorDeadlockedThreads()
ok, I wasn't aware that it didn't work. > On June 8, 2015, 4:57 p.m., John Speidel wrote: > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/DeadlockWarningThread.java, > > line 121 > > <https://reviews.apache.org/r/35074/diff/2/?file=980027#file980027line121> > > > > See earlier message questioning why we have 2 deadlock detection > > mechanisms here. > > > > Comparing the current call stack against the last call stack to > > determine whether a deadlock has occurred seems dangerous here and likely > > to result in occasional detected deadlocks in cases where none exist. For > > example if there is an extended GC during this test and none of the > > monitored threads make progress but this thread does, a false deadlock will > > be reported. > > Dmytro Shkvyra wrote: > GC stop the world will stop DeadlockWarningThread too, so it will not > report false deadlock. > > Dmytro Shkvyra wrote: > ``` > try { > Thread.sleep(3000); > } catch (InterruptedException ex) { > ``` > Also should prevent false deadlock report I still believe that it is possible that under heavy load that the DeadlockWarningThread could make progress but not the worker threads. I am ok with this for now and if it does happen we can address then. - John ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35074/#review87040 ----------------------------------------------------------- On June 9, 2015, 12:58 p.m., Vitalyi Brodetskyi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/35074/ > ----------------------------------------------------------- > > (Updated June 9, 2015, 12:58 p.m.) > > > Review request for Ambari, Jonathan Hurley and John Speidel. > > > Bugs: AMBARI-11692 > https://issues.apache.org/jira/browse/AMBARI-11692 > > > Repository: ambari > > > Description > ------- > > Fails on 2 ubuntu workstations > > {noformat} > > Tests in error: > > testDeadlockWhileRestartingComponents(org.apache.ambari.server.state.cluster.ClusterDeadlockTest): > test timed out after 75000 milliseconds > > Tests run: 3016, Failures: 0, Errors: 1, Skipped: 21 > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Views ...................................... SUCCESS [4.863s] > [INFO] Ambari Server ..................................... FAILURE > [1:49:50.358s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > > {noformat} > > {noformat} > java.lang.Exception: test timed out after 75000 milliseconds > at java.lang.Object.wait(Native Method) > at java.lang.Thread.join(Thread.java:1281) > at java.lang.Thread.join(Thread.java:1355) > at > org.apache.ambari.server.state.cluster.ClusterDeadlockTest.testDeadlockWhileRestartingComponents(ClusterDeadlockTest.java:241) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62) > > at > org.apache.ambari.server.state.svccomphost.ServiceComponentHostImpl.getDesiredStateEntity(ServiceComponentHostImpl.java:1555) > at > org.apache.ambari.server.state.svccomphost.ServiceComponentHostImpl.isRestartRequired(ServiceComponentHostImpl.java:1478) > at > org.apache.ambari.server.state.ConfigHelper.calculateIsStaleConfigs(ConfigHelper.java:927) > at > org.apache.ambari.server.state.ConfigHelper.isStaleConfigs(ConfigHelper.java:376) > at > org.apache.ambari.server.state.cluster.ClusterImpl.getClusterHealthReport(ClusterImpl.java:2580) > {noformat} > > Reproduced in trunk, last commit > {noformat} > commit a52eb51d1af0edc9273a947535a2a36886e625da > Author: Oleg Nechiporenko <[email protected]> > Date: Thu May 28 18:02:28 2015 +0300 > > AMBARI-11484. Configs: when doing override, it's hard to find config > override (onechiporenko) > > {noformat} > > > Diffs > ----- > > > ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java > 2f064ab > > ambari-server/src/test/java/org/apache/ambari/server/testing/DeadlockWarningThread.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/testing/DeadlockedThreadsTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/35074/diff/ > > > Testing > ------- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
