Re: ClassLoader Architecture

[Paul Hammant]

Dain, OSGi is a Dependency Lookup father than Dependency Injection form of Inversion of control (running with the  the common mistake that component assembly is what defines IoC, ignoring configuration etc). I think the industry long term future is in Dependency Injection, particularly CDI. That is for static languages like Java. Experience gleaned from the worlds of Ruby and Python is that component assembly issues are less of a problem there. I just don't like OSGi and think even the original Avalon (Stefano's baby from 1998 and onwards) has a better dependency lookup API. Anyway,  In the NanoContainer project we've always been pushing to have hierarchies of components and containers that have class and implementation visibilities sorted out. PicoContainer does not provide any reflection-esque way of instantiating components. NanoContainer (as well as many other things) wraps PicoContainer and does do that. In the same timeline as this tread we wrote a tiny boot class that allows for components in a tree to not even see the classes for NanoContainer & Groovy (our preferred but not mandated composition language)... Primordial--|             |--Boot             |                      |             |--Pico--|--Nano, Groovy,                      |   ASM, CLI                      |                      |                      |--Common--|                           API   |--Comp A                                 |                                 |--Comp B Comp A and B (in separate classloaders) can see Common-API, which can see the classloader that contains PicoContainer who's parent is the classes in rt.jar On a separate branch, is the Boot classloader (once main() is executed the thing plays no further part in the running of the 'machine'). It creates NanoContainer and all the associated libs it is going to use to create trees of comps, in one classloader giving it PicoContainer (etc) as its parent classloader. Anyway, it works well and allows us to implement separations. It also allows us to have say Component-B depend on Component-A provided the type A implemented was in the Common-API classloader or higher. Making this more complex is the need to depend on things not there at boot time. We've experimented with the notion of hot deployment (in MicroContainer as it happens), but we'll talk about the mechanics here (not sell another product). A hypothetical Kernel could be running is the same space as the Nano/Groovy thing, and hot deploy (via some user action or archive dropped into a dir) a component. The component could introduce a new thing for others to avail of. For example, the component could be DefaultSpellChecker and implement SpellChecker that it ships with. If the mechanism for delivering the component allows for a diff jar each for interface and implementaion, and the former was in some way marked, the Kernel could take it and allow it to join the classloader at Common-API level while mounting its implementation in a new classloader alongside Comp-A and Comp-B.  This is a once off deal though. The interface itself, once mounted cannot be replaced in that classloader. So we gained late deploy and hot deploy via that. We also have re-hot-deploy sorted via the same mechanism, but not if the API changed. What clearly is not done via that mechanism is redeployment of a second or superseding version of the same interface. This is possible of course. Just it is pushed elsewhere I think.  Bob McWhirter's and Jason van Zyl's ClassWorlds will do it. (directed/constrained graphs with selection package inheritance). Peter Donald has something inside Loom that does it too called ClassMan. With those and cunning use of some context during classloading, you should be able to lace things correctly. Stick with DI lads - it has a rosy future... Incidentally we aim with NanoContainer to deliver whole applications that leverage cherry picked components from projects (Geronimo, Spring, Jetty, Hibernate etc). Composed like so :-   builder = NanoContainerBuilder()   parent = builder.container(parent:parent,class:ImplementationHidingNanoPicoContainer) {       classpathelement(path:"lib/api.jar")       hidden() {           classpathelement(path:"lib/a.jar")           component(key:"A", class:"AImpl")       }       hidden() {           classpathelement(path:"lib/b.jar")           component(class:"BImpl")       }   } - Paul On Jun 16, 2005, at 2:00 PM, Dain Sundstrom wrote: Stephane and I chatted about this on irc, so I wanted to post up the notes. The issues Stephane points are big deal.  This is a problem Paul Hammant first pointed out to me a the last OSCon, and I have been thinking about how to address for a while.  Of course I never brought it up for discussion here :( In the long run (for Geronimo 2.0), I'm currently leaning towards using OSGI to solve this problem.  OSGI is the framework that eclipse plugins use, and they have really solved the class loader problems.  This would require a major architectural change so I don't think it can be done in 1.x.  For those of you that are interested, the current OSGi r3 spec chapter 4 describes the class loader architecture.  The next yet to be published r4 spec expands on the model addressing some of the bad assumptions (e.g., furure versions of any library are backwards compatible with all previous versions). In the short term, I like Stephane's idea of adding an optional child first delegation class loader to applications (not just wep application).  This would help applications avoid conflicts with Geronimo system classes.  Anyone want to take a look at implementing this? -dain On Jun 14, 2005, at 8:18 AM, Stephane Bailliez wrote: Hi all, I'd like to clarify  a couple of points about the Geronimo ClassLoader architecture. I have had a discussion about it with Gianny Damour on IRC, so i'm bringing the discussion here to have some feedback about the existing implementation and my thoughts. It looks like as of now the CL architecture is exposing its internal implementation to applications [1] if I'm reading correctly. Say for example, if Geronimo is using component 1.x it will be visible to applications. There are 2 aspects: 1. if the application uses component 2.x which is incompatible with 1.x you could then be in a blocker situation. 2. if the application incidentally forgets to ship with component 1.x, in a normal world, this would end up with a CNFE. In our case it would simply mask it to the user, actually run by accident..and may blow up later, say when upgrading the apps, or Geronimo. 3. some component unfortunately autoconfigure based on the presence or absence of another component (a good example is ActiveMQ autoconfiguring itself with derby persistence if available in the classpath). I don't like too much autoconfiguration, as it will often lead to problems later. The  rational being to put as little configuration as possible and thus forgetting to say 'even if you find that component don't do that, use this instead' (for activemq it would be explicity configuring vmpersistence). My rational is to avoid magic. A component should always start the same way. To be more general: In the OSS J2EE CL land, we have as of now 2 different hierarchies: - JBoss with its Unified ClassLoader [2] AFAIK what has been said publicly to defend this design was originally to 'please' users and to avoid them the dreaded ClassCast or ClassNotFound, VeryErrors,etc..... by stuffing as much components as they could find into a gigantic 'unified' pot. This led to a terrible thing which any serious user has been fighting against in production...not mentioning that classloader isolation settings have been changing between micro releases. - JOnAS It has a similar classloader hierarchy of Geronimo's [3] (or vice-versa) but is unfortunately exposing its implementation to applications. There is now a very serious problems with deployment of Hibernate based apps which use ASM and the ASM version used within JOnAS internals...which are incompatible. Web applications (WARs) deployed in those J2EE containers generally can change the delegation model from the java2 delegation model (parent first) to a servlet 2.3+ model (current first), but this is insufficient in itself because it is only for the WAR part..you could very well have this problem in the EJB tiers .. so you need to be able to change the model type as well in the EJB tier. Being able to change the delegation model of the EJB tiers of course does not prevent actually having the case described in (2) and (3) happening. Unless we can filter for sure what can be delegated which is not so straightforward to me. Indeed, it does not mean filtering out to allow 'only' J2EE apis and implementation. In your app, you may perfectly need visibility of components located in the boostrap classpath or in jre/lib/ext (for example a crypto provider, etc...) Shouldn't the classloader architecture have a child classloader to load internals (just like it does for the database driver) Any thoughts on that ? I would really like to avoid having a release with a ClassLoader architecture that is already known to expose the users to some serious problems. Otherwise this may lead to tightly coupled components that will be very hard to decouple later (if not now). Any comment is welcome. Cheers, Stephane [1] http://chariotsolutions.com/geronimo/elements-classloaders.html [2] http://wiki.jboss.org/wiki/Wiki.jsp?page=ClassLoadingConfiguration [3] http://www.redhat.com/docs/manuals/rhaps/jonas-guide/s1-conclusion.html