I will look into that. @Xikui, Do you have a reproducable use case where such a behavior can be observed?
Cheers, Abdullah. On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover <[email protected]> wrote: > Hi > > Each external library has an separately associated custom class loader > (URLClassLoader) that loads the contained functions and any dependencies > (jar files) packaged inside the library. The custom loader fits into the > natural hierarchy with system (classpath based) classloader as the > immediate parent and bootstrap classloader as the root. As the delegation > model works, a class loader attempts to load a class only after parent(s) > in the lineage have failed. > > In your specific case, the system class loader is able to find the jackson > library from the configured classpath and so the library classloader is not > involved at all for this jar. The problem arising due to different versions > of a jar present across the hierarchy of classloaders is similar to one > faced in web containers such as Tomcat or application servers such as > Websphere where system loaded classes can override any application specific > classes due to name conflict. > > The way you are defining a library is correct. The fix needs to be in the > way the classes are loaded by AsterixDB (here acting as an application > server). > > I see two ways:- > > a) Skipping the system class loader: AsterixDB uses a URLClassloader to > load libraries. Setting the parent of the this classloader as null implies > the bootrstrap classloader (root) as the immediate parent. The bootstrap > classloader loads rt.jar (java runtime classes) and i18n.jar and these > definitely do not have conflicts with your library dependencies (if they > do, not sure why one would override the classes in rt.jar and so we need to > fix the library). Note that the system classloader is out of the hierarchy > here and so any other jars loaded by AsterixDB from its classpath (pom.xml) > would not be visible. > > b) Defining a Custom Class loader: Inside AsterixDB, we provide a custom > implementation of classloader that attempts to load a class prior to > delegating to the parent. This way, the library classes and packaged > dependencies override any system level classes from the class path and even > the classes contained in rt.jar and i18n.jar. > > I am opening the discussion here to suggest further alternatives or provide > preferences. > > I have a preference for (a) (skipping the system class loader) for two > reasons: > > a) it is simpler > > b) the other option allows a custom class loader to override classes in > rt.jar., which is ok but not how classloaders are supposed to work in > principle. > > Regards, > > Raman > > > > > > > > On Jun 29, 2016 11:07 PM, "Mike Carey" <[email protected]> wrote: > > > Any classloader experts have suggestions...? > > On Jun 29, 2016 10:26 PM, "Xikui Wang" <[email protected]> wrote: > > > > > Hi Devs, > > > > > > We found a problem when trying to build external functions for the > > > cloudberry demo. > > > > > > When the external function is depend on certain library, the library > that > > > comes with the external function will be blocked by same library in > > > AsterixDB. In our case, our external function 'geoTag' uses jackson > > v2.7.1, > > > and we packed all dependencies into one single jar. When running > 'geoTag' > > > on Asterix, it will call jackson v2.0.0 in AsterixDB which causes > > > NullPointerException. We have to manually change pom.xml in AsterixDB > to > > > fix that. > > > > > > We are wondering is that because we load the external function in a > wrong > > > way, or this could be one possible interesting problem which is worth > > > noticing. Thanks. > > > > > > Best, > > > Xikui > > > > > >
