Yes you would have to do it with classloaders (not 'hello world' but not 'rocket science' either).
You'll be limited on using native libraries, even if you use classloaders properly as native libs can be loaded only once. You will have to ensure you get rid of the task classloader once the task is over (thus removing all singleton stuff that may be in it). You will have to put in place a security manager for the code running out the task classloader. You'll end up doing somemthing similar to servlet containers webapp classloading model with the extra burden of hot-loading for each task run. Which in the end may have a similar overhead of bootstrapping a JVM for the task, this should be measured to see what is the time delta to see if it is worth the effort. A On Mon, Jun 2, 2008 at 3:53 PM, Steve Loughran <[EMAIL PROTECTED]> wrote: > Christophe Taton wrote: > >> Actually Hadoop could be made more friendly to such realtime Map/Reduce >> jobs. >> For instance, we could consider running all tasks inside the task tracker >> jvm as separate threads, which could be implemented as another personality >> of the TaskRunner. >> I have been looking into this a couple of weeks ago... >> Would you be interested in such a feature? >> > > Why does that have benefits? So that you can share stuff via local data > structures? Because you'd better be sharing classloaders if you are going to > play that game. And that is very hard to get right (to the extent that I > dont think any apache project other than Felix does it well) >
