On 3/17/14, 09:07 , Stefan Egli wrote:
Hi Richard,
(see below)
On 3/17/14 1:34 PM, "Richard S. Hall" <[email protected]> wrote:
On 3/14/14, 05:26 , Stefan Egli wrote:
Hi,
I just ran into the famous deadlock with Java 6 where the same class is
being loaded from two different bundles (also see FELIX-3953). This
happened on a startup of a CQ instance.
I'll add more analysis details but my question is: what is the
suggested best practice to handle this issue?
* Use '-XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass' as
suggested in [0] and [1]? But then again [1] says "You must treat it as
unstable, experimental, temporary, unsupported".
This is your only option if you are not able to use Java 7.
* Or would it be worth while to reconsider fixing FELIX-3953?
Unless we modify the JRE, there is no way to fix it. The Felix framework
doesn't hold locks while class loading.
I obviously don't know the details of the BundleClassLoader, so I can only
speculate. But it seems the problem is its 'class locking' (m_classLocks).
The assumption there seems, that BundleClassLoader.findClass() is always
called unsynchronized. But this is clearly not the case in Java 6.
The underlying issue of eliminating class loader cycle deadlocks is not
possible.
The m_classLocks structure was put in place to support Java 7 parallel
class loaders. So, I guess the question is, does this approach make
deadlock more likely on older JVMs? I don't know the answer to that nor
do I have the time to look into it, but it would be worthwhile to
investigate/understand.
So maybe Java 6 would need a special class loader which is
synchronize-aware.. (in which case you could for example synchronize the
BundleClassLoaderJava6.findClass() - given that the synchronization would
be done by the VM anyway).
Or introduce a 'proxy class loader' such that the VM would always only (be
able to) synchronize the proxy.. (if that's feasible)
Again, the underlying issue is not resolvable. Before we did
m_classLocks, we were actually locking on "this" (i.e., the class
loader) before calling findLoadedClass() and that was the only time we
held a lock and we were still getting into deadlocks because of the
JVM-held locks.
As I state above, the only real issue in my mind is if we were to, for
example, effectively go back to locking on "this" for pre-Java7 JVMs,
would this solve the issue in your specific use case?
What we were trying to do was to keep a single approach for class
loading, but one that would automatically gain the parallel benefits on
Java 7. It could be possible that it interferes somehow.
If you want to do a test, modify the class loader code so lock on "this"
instead of m_classLocks when accessing the m_classLocks data structure,
I think that is all that would be necessary to simulate it. If it turns
out to help your situation, then it might be an easy patch. We just have
to modify the class loader to detect which lock it should use.
-> richard
-> richard
Cheers,
Stefan
-> richard
Here's more details (see thread-dumps below):
* Thread 158 (on behalf on bundle X) wants to load class A which is
part of bundle Y - hence goes via getClassByDelegation - then does a
BundleWiringImpl$BundleClassLoader.findClass
* before Thread 158 continues VM threading decides to give Thread 156
some execution time:
* Thread 156 now wants to load same class A, but this time directly
on behalf of bundle Y.
* Due to [1] Thread 156 internally does a synchronized
loadClassInternal, thus holds the lock on 6f025d710 (the
BundleClassLoader of bundle Y) (I was able to verify this by hooking the
deadlocked-VM to a debugger, which indicated exactly this)
* Thread 156 now continues into
BundleClassLoader.findClass(BundleWiringImpl.java:2115) where it has to
do a m_classLocks.wait() - since Thread 158 earlier marked the
BundleClassLoader 'in use/locked for class A'.
* Thread 158 on the other hand continues the findClass execution,
eventually walks into checkCerts, which is synchronized(this), which
Thread 156 holds
* and we have the famous classloading deadlock.
Cheers,
Stefan
--
[0] https://issues.apache.org/jira/browse/FELIX-3953
[1]
http://underlap.blogspot.de/2006/11/experimental-fix-for-sunbug-4670071.h
tml
PS: Used org.apache.felix.framework 4.3.0.R1558704 - which is a
snapshot-release as of SVN revision 1558704
"Thread-158" daemon prio=5 tid=7f83a4cbd800 nid=0x25b73e000 waiting for
monitor entry [25b73c000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.ClassLoader.checkCerts(ClassLoader.java:788)
- waiting to lock <6f025d710> (a
org.apache.felix.framework.BundleWiringImpl$BundleClassLoaderJava5)
at java.lang.ClassLoader.preDefineClass(ClassLoader.java:493)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.findClass(B
undleWiringImpl.java:2297)
at
org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegati
on(BundleWiringImpl.java:1519)
at
org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.j
ava:77)
at
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(B
undleWiringImpl.java:1973)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at
org.apache.felix.framework.BundleWiringImpl.getClassByDelegation(BundleWi
ringImpl.java:1392)
at
org.apache.felix.framework.BundleWiringImpl.searchImports(BundleWiringImp
l.java:1571)
at
org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegati
on(BundleWiringImpl.java:1502)
at
org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.j
ava:77)
at
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(B
undleWiringImpl.java:1973)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
[class A being loaded from a class of bundle X]
"Thread-156" daemon prio=5 tid=7f83a57ac800 nid=0x25b36b000 in
Object.wait() [25b369000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.findClass(B
undleWiringImpl.java:2115)
- locked <6f788bf40> (a java.util.HashMap)
at
org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegati
on(BundleWiringImpl.java:1519)
at
org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.j
ava:77)
at
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(B
undleWiringImpl.java:1973)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
[class A being loaded from a class of bundle Y]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
