On 11/25/13, 22:13 , Karl Rieb wrote:
Thanks for the quick response! We have managed to come up with a work-around for our problem. I will document it here in case someone else comes across the same issue we had, although I did have some questions (see last paragraph of e-mail).I created a test environment to reproduce the bug that avoided spawning any threads; I made necessary calls within the Activator. Using this test environment, I was able to narrow down the issue to an embedded dependency being used by our bundle. Tracing through the code, I saw the embedded dependency dynamically adds a Java Security provider if one is not already present. The code looks something like: if( Security.getProvider(BouncyCastleProvider.PROVIDER_NAME) == null ) { Security.addProvider(new BouncyCastleProvider()); } This happens upon loading one of the classes in the embedded dependency. Now this is pure educated guessing on my part, but I think the problem arises because the BouncyCastleProvider class is being loaded by the current bundle's ClassLoader, but getting added statically to a class, java.security.Security, that is handled by one of the core/boot felix ClassLoaders (since it is a base JVM class). So now this class, java.security.Security, that is accessible by all the bundles has a reference to a class, BouncyCastleProvider, that is tied to a specific bundle's bootloader. When I do the uninstall and re-install of the bundle, the above code never adds a new provider because it can still see the old one it set previously (this is why I think the java.security.Security class is being loaded outside of the bundle's ClassLoader). Now when the newly re-installed bundle makes some crypto calls that require the BouncyCastle provider, some classes get loaded (sun.reflect.ConstructorAccessorImpl, specifically) using the old bundle's ClassLoader. This triggers the NullPointerException because the old ClassLoader is disposed. Again, purely an educated guess. WORKAROUND: The workaround we set in place was to add the BouncyCastleProvider as a Java Security provider in our bootstrapping bundle that starts before all our other bundles. This bundle removes any existing BouncyCastleProvider then adds one. When the bootstrapping bundle is stopped, it removes the BouncyCastleProvider. This ensures the BouncyCastle provider always has a valid ClassLoader since it is only ever available as long as our bootstrapping bundle is active and started. Additionally, this lets us managed the version of BouncyCastleProvider we want to use. Note: we had a strict requirement that prevented us from being able to restart our OSGi framework. A simpler work-around was to configure the boot delegation property for Felix: -Dorg.osgi.framework.bootdelegation="sun.*" In either case, the issue no longer appears in our test environment using either fix mentioned above. The boot delegation fix makes me a bit nervous since presumably you are still referencing an old ClassLoader. Is there is a recommended practice for using the Java Crypto API in OSGi with custom providers? Should providers be added statically outside of Felix? Should the Java Crypto APIs be avoided? Or is boot delegation meant to be set when using these types of frameworks?
The tricky aspect here is that the embedded library automatically adds the provider when someone loads a class. It seems you'd have to modify the library to be aware that it has active/inactive states and it can only provide the crypto algorithm when it is active. There is no other way that I can think of because a non-activated bundle cannot get any events and it really isn't possible for the bundle to learn that it is being uninstalled (or refresh).
A slight different way to do it would be an extender pattern, where the provider bundles could offer their provider objects and the extender bundle could provide them when the bundle is resolved and remove them when the bundle is uninstalled/refreshed.
There is no real way around this issue, you cannot pass objects around from bundles and then not clean up after them after an uninstall/refresh.
-> richard
-Karl ________________________________________ From: Richard S. Hall [[email protected]] Sent: Saturday, November 23, 2013 3:25 PM To: [email protected] Subject: Re: Stale BundleWiringImpl (ClassLoader) used for sun.reflect.ConstructorAccessorImpl On 11/23/13, 13:29 , Karl Rieb wrote:Hi, We are using felix framwork 3.0.4 and are having problems when one of our bundle upgrades. We noticed the same issue also occurs if we uninstall, then later re-install the bundle. The bundle has an embedded dependency (e.g. a non-OSGi jar) that makes a call to MessageDigest.getInstance("MD4"). When the bundle is first installed, This internal call goes through without a problem. But after upgrading the bundle or uninstalling and re-installing it, we start to get the following exceptions: Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: MD4, provider: BC, class: org.bouncycastle.jce.provider.JDKMessageDigest$MD4) at java.security.Provider$Service.newInstance(Provider.java:1262) at sun.security.jca.GetInstance.getInstance(GetInstance.java:236) at sun.security.jca.GetInstance.getInstance(GetInstance.java:164) at java.security.Security.getImpl(Security.java:695) at java.security.MessageDigest.getInstance(MessageDigest.java:159) at rpc.security.ntlm.Responses.ntlmHash(Responses.java:186) at rpc.security.ntlm.Responses.ntlmv2Hash(Responses.java:203) at rpc.security.ntlm.Responses.getLMv2Response(Responses.java:102) at rpc.security.ntlm.NtlmAuthentication.createType3(NtlmAuthentication.java:270) ... 23 more Caused by: java.lang.NullPointerException at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1432) at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.security.Provider$Service.getImplClass(Provider.java:1279) at java.security.Provider$Service.newInstance(Provider.java:1237) Using a debugger, we traced through this call and found that the NullPointerException gets triggered when trying to load the sun.reflect.ConstructorAccessorImpl class for creating a new instance of org.bouncycastle.jce.provider.JDKMessageDigest$MD4. Note that the classloader has no problems finding org.bouncycastle.jce.provider.JDKMessageDigest$MD4, it only has an issue with sun.reflect.ConstructorAccessorImpl. Stepping through the BundleWiringImpl calls, we noticed that the NullPointerException occurs because the BundleWiringImpl instance has been disposed (e.g. it is stale). The revision of the BundleWiringImpl confirms this since it matches the revision of the original bundle install (when we first start up the felix container). For example, the revision will be something like [25] when the currently running bundle revision is actually [25.2] or [27] (depending if we do upgrades or uninstall/re-install). All the other non-sun classes required by the bundle go against the correct version of BundleWiringImpl except for sun.reflect.ConstructorAccessorImpl. Some how, no matter how many times we uninstall and re-install the bundle, the sun.reflect.ConstructorAccessorImpl class always gets loaded by the same stale instance of BundleWiringImpl (e.g. [25]). I imagine there must be some sort of caching going on somewhere that is keeping a reference to this old classloader, but I don't understand where or how.The framework isn't caching it, could be in some class loader somewhere. But still, this would indicate that when you uninstall and refresh, that you aren't really refreshing everything. In most cases, someone has left a thread running somewhere that continues to use the stale class loader. Also, you could try on a new framework version, but if it is a runaway thread, it probably won't help. -> richardNote that we are able to avoid this issue if we set: org.osgi.framework.bootdelegation=sun.* Note that just setting 'sun.reflect.*' does not work. We tried using dynamic imports for all sun packages, but this did not work: <DynamicImport-Package>sun.*</DynamicImport-Package> Unfortunately we can not easily restart the Felix container to set the boot delegation property. Is there a way to avoid this class loading issue without requiring a restart of Felix? -Karl This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at ([email protected]) immediately. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at ([email protected]) immediately. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
