Thanks for the quick response! We have managed to come up with a work-around
for our problem. I will document it here in case someone else comes across the
same issue we had, although I did have some questions (see last paragraph of
e-mail).
I created a test environment to reproduce the bug that avoided spawning any
threads; I made necessary calls within the Activator. Using this test
environment, I was able to narrow down the issue to an embedded dependency
being used by our bundle. Tracing through the code, I saw the embedded
dependency dynamically adds a Java Security provider if one is not already
present. The code looks something like:
if( Security.getProvider(BouncyCastleProvider.PROVIDER_NAME) == null ) {
Security.addProvider(new BouncyCastleProvider());
}
This happens upon loading one of the classes in the embedded dependency. Now
this is pure educated guessing on my part, but I think the problem arises
because the BouncyCastleProvider class is being loaded by the current bundle's
ClassLoader, but getting added statically to a class, java.security.Security,
that is handled by one of the core/boot felix ClassLoaders (since it is a base
JVM class). So now this class, java.security.Security, that is accessible by
all the bundles has a reference to a class, BouncyCastleProvider, that is tied
to a specific bundle's bootloader. When I do the uninstall and re-install of
the bundle, the above code never adds a new provider because it can still see
the old one it set previously (this is why I think the java.security.Security
class is being loaded outside of the bundle's ClassLoader).
Now when the newly re-installed bundle makes some crypto calls that require the
BouncyCastle provider, some classes get loaded
(sun.reflect.ConstructorAccessorImpl, specifically) using the old bundle's
ClassLoader. This triggers the NullPointerException because the old
ClassLoader is disposed. Again, purely an educated guess.
WORKAROUND: The workaround we set in place was to add the BouncyCastleProvider
as a Java Security provider in our bootstrapping bundle that starts before all
our other bundles. This bundle removes any existing BouncyCastleProvider then
adds one. When the bootstrapping bundle is stopped, it removes the
BouncyCastleProvider. This ensures the BouncyCastle provider always has a
valid ClassLoader since it is only ever available as long as our bootstrapping
bundle is active and started. Additionally, this lets us managed the version
of BouncyCastleProvider we want to use.
Note: we had a strict requirement that prevented us from being able to restart
our OSGi framework. A simpler work-around was to configure the boot delegation
property for Felix:
-Dorg.osgi.framework.bootdelegation="sun.*"
In either case, the issue no longer appears in our test environment using
either fix mentioned above. The boot delegation fix makes me a bit nervous
since presumably you are still referencing an old ClassLoader.
Is there is a recommended practice for using the Java Crypto API in OSGi with
custom providers? Should providers be added statically outside of Felix?
Should the Java Crypto APIs be avoided? Or is boot delegation meant to be set
when using these types of frameworks?
-Karl
________________________________________
From: Richard S. Hall [[email protected]]
Sent: Saturday, November 23, 2013 3:25 PM
To: [email protected]
Subject: Re: Stale BundleWiringImpl (ClassLoader) used for
sun.reflect.ConstructorAccessorImpl
On 11/23/13, 13:29 , Karl Rieb wrote:
> Hi,
>
> We are using felix framwork 3.0.4 and are having problems when one of our
> bundle upgrades. We noticed the same issue also occurs if we uninstall, then
> later re-install the bundle.
>
> The bundle has an embedded dependency (e.g. a non-OSGi jar) that makes a call
> to MessageDigest.getInstance("MD4"). When the bundle is first installed,
> This internal call goes through without a problem. But after upgrading the
> bundle or uninstalling and re-installing it, we start to get the following
> exceptions:
>
> Caused by: java.security.NoSuchAlgorithmException: Error constructing
> implementation (algorithm: MD4, provider: BC, class:
> org.bouncycastle.jce.provider.JDKMessageDigest$MD4)
> at java.security.Provider$Service.newInstance(Provider.java:1262)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
> at java.security.Security.getImpl(Security.java:695)
> at java.security.MessageDigest.getInstance(MessageDigest.java:159)
> at rpc.security.ntlm.Responses.ntlmHash(Responses.java:186)
> at rpc.security.ntlm.Responses.ntlmv2Hash(Responses.java:203)
> at rpc.security.ntlm.Responses.getLMv2Response(Responses.java:102)
> at
> rpc.security.ntlm.NtlmAuthentication.createType3(NtlmAuthentication.java:270)
> ... 23 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1432)
> at
> org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
> at
> org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at java.security.Provider$Service.getImplClass(Provider.java:1279)
> at java.security.Provider$Service.newInstance(Provider.java:1237)
>
> Using a debugger, we traced through this call and found that the
> NullPointerException gets triggered when trying to load the
> sun.reflect.ConstructorAccessorImpl class for creating a new instance of
> org.bouncycastle.jce.provider.JDKMessageDigest$MD4. Note that the
> classloader has no problems finding
> org.bouncycastle.jce.provider.JDKMessageDigest$MD4, it only has an issue with
> sun.reflect.ConstructorAccessorImpl.
>
> Stepping through the BundleWiringImpl calls, we noticed that the
> NullPointerException occurs because the BundleWiringImpl instance has been
> disposed (e.g. it is stale). The revision of the BundleWiringImpl confirms
> this since it matches the revision of the original bundle install (when we
> first start up the felix container). For example, the revision will be
> something like [25] when the currently running bundle revision is actually
> [25.2] or [27] (depending if we do upgrades or uninstall/re-install).
>
> All the other non-sun classes required by the bundle go against the correct
> version of BundleWiringImpl except for sun.reflect.ConstructorAccessorImpl.
> Some how, no matter how many times we uninstall and re-install the bundle,
> the sun.reflect.ConstructorAccessorImpl class always gets loaded by the same
> stale instance of BundleWiringImpl (e.g. [25]). I imagine there must be some
> sort of caching going on somewhere that is keeping a reference to this old
> classloader, but I don't understand where or how.
The framework isn't caching it, could be in some class loader somewhere.
But still, this would indicate that when you uninstall and refresh, that
you aren't really refreshing everything. In most cases, someone has left
a thread running somewhere that continues to use the stale class loader.
Also, you could try on a new framework version, but if it is a runaway
thread, it probably won't help.
-> richard
>
>
> Note that we are able to avoid this issue if we set:
>
> org.osgi.framework.bootdelegation=sun.*
>
> Note that just setting 'sun.reflect.*' does not work. We tried using dynamic
> imports for all sun packages, but this did not work:
>
> <DynamicImport-Package>sun.*</DynamicImport-Package>
>
> Unfortunately we can not easily restart the Felix container to set the boot
> delegation property. Is there a way to avoid this class loading issue
> without requiring a restart of Felix?
>
> -Karl
> This electronic message contains information which may be confidential or
> privileged. The information is intended for the use of the individual or
> entity named above. If you are not the intended recipient, be aware that any
> disclosure, copying, distribution or use of the contents of this information
> is prohibited. If you have received this electronic transmission in error,
> please notify us by e-mail at ([email protected]) immediately.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
This electronic message contains information which may be confidential or
privileged. The information is intended for the use of the individual or entity
named above. If you are not the intended recipient, be aware that any
disclosure, copying, distribution or use of the contents of this information is
prohibited. If you have received this electronic transmission in error, please
notify us by e-mail at ([email protected]) immediately.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
