On 17 February 2012 10:21, Simone Tripodi <[email protected]> wrote:
> Hi all guys, > > The ML at ASF is the best place where we can discuss ;) Also because > the more people can access to the info, then easier would be involving > new volunteers that wants to help :) > > Anyway, speaking more about technical details: I had a look at the > current codebase and even if we restrict the package where performing > the plugins research, this operation has always the same complexity, > because it needs to visit the whole ClassPath > We can scan only the directory meant to contain plugins, the current CLI configuration scans the entire classpath and takes about 2 secs, I expect better performances for a smaller set of JARs. > > Moreover, it consumes memory: it loads ALL classes for any recognized > item that has to be passed to the filter, that checks if the current > item is a plugin or not. > Agree, this can be avoided implementing a cache on the classpath, the PluginManager can check if the cache is valid and then access it. > > Just for the record, have a look at > < > http://lemnik.wordpress.com/2007/09/14/why-classpath-scanning-is-bad-for-you/ > > > > The Jersey approach is a little different, IIRC they did a little > trick on < > http://docs.oracle.com/javase/6.0/docs/api/java/lang/ClassLoader.html#getResources(java.lang.String) > > > (and is not efficient as well) > > What I am proposing is something really simpler, using the > ServiceLoader < > http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html:> > that would still allow dynamic definition of the plugins loaded at > runtime, but since plugins are already known at compile time, we can > drastically reduce the class loading. > That's why Phython guys joke Java developers :) > > There are tools as well that will help us on generating > META-INF/services metadata: > <http://weblogs.java.net/blog/kohsuke/archive/2009/03/my_project_of_t.html > > > > Ah, and just for the record: I prototyped a general purpose lib for > scanning the classpath, using fluent APIs, @commons, see > <https://svn.apache.org/repos/asf/commons/sandbox/meiyo/trunk/> - > commons sandbox is open to all ASF committers, in the case you are > interested... ;) > Will give a look. > > All the best, > -Simo > The best. Bye Mic > > http://people.apache.org/~simonetripodi/ > http://simonetripodi.livejournal.com/ > http://twitter.com/simonetripodi > http://www.99soft.org/ > > > > On Fri, Feb 17, 2012 at 1:14 AM, Michele Mostarda > <[email protected]> wrote: > > Hi Simo, > > > > we scan a set of packages that can be specified programmatically, as > done > > by Jersey for example to detect providers. > > There are several optimizations that can be done like building a local > > cache to avoid a full rescan every time (for example while > > running CLI tools). ATM the best choice is to use the more strictest > > packages as possible. > > > > I would like to spend some time optimizing the PluginManager, > implementing > > for example a local cache based on, classpath > > files modifications. I think we could arrive to a general purpose lib > > for classpath scanning in Java (that ATM is missing AFAIK). > > > > Let me know when you're available for a deeper discussion about it. > > > > The best. > > > > Mic > > > > On 16 February 2012 23:24, Lewis John Mcgibbney > > <[email protected]>wrote: > > > >> I'm here Simo, but of little help.. I'm nearly burnt out for the day.... > >> > >> If we can initiate a discussion it is better than nothing at all I > suppose. > >> > >> > >> On Thu, Feb 16, 2012 at 10:21 PM, Simone Tripodi > >> <[email protected]>wrote: > >> > >> > Hi all, > >> > > >> > I started having a look at the current plugins architecture and just > >> > noticed we scan the classpath: that is the most expensive operation, > >> > because no optimization can be applied and it is always linear - that > >> > also means that more jars plugin we add, the more is memory/time > >> > consumption. > >> > > >> > There is an improvement we can apply switching over the ServiceLoader > >> > pattern, I need anyway more details about the plugins architecture... > >> > anyone available? > >> > > >> > TIA, > >> > -Simo > >> > > >> > http://people.apache.org/~simonetripodi/ > >> > http://simonetripodi.livejournal.com/ > >> > http://twitter.com/simonetripodi > >> > http://www.99soft.org/ > >> > > >> > >> > >> > >> -- > >> *Lewis* > >> > > > > > > > > -- > > Michele Mostarda > > Senior Software Engineer > > skype: michele.mostarda > > twitter: micmos > > mail: [email protected] > > site : http://www.michelemostarda.com > -- Michele Mostarda Senior Software Engineer skype: michele.mostarda twitter: micmos mail: [email protected] site : http://www.michelemostarda.com
