This is fantastic stuff. I just gave it a try. The performance improvements are really dramatic when downloading from Maven repos.
Hans -- Hans Dockter Founder, Gradle http://www.gradle.org, http://twitter.com/gradleware CEO, Gradleware - Gradle Training, Support, Consulting http://www.gradleware.com On Sat, Nov 5, 2011 at 10:40 PM, Daz DeBoer <[email protected]>wrote: > G'day > > Over the past couple of weeks we've made some significant improvements to > the way dependency resolution works in Gradle. There are some things that > are just refactorings, some performance improvements, and a couple of new > features. These changes should make it into M6, which is due out soon. > * > * > *Caching Improvements:* > The big new change in M6 is that Gradle now takes control of the caching > of Dynamic Versions (eg 1.+) and Changing Modules (eg SNAPSHOT). A new > metadata file in the cache, "dynamic-versions.bin", caches information > about module resolution on a per-resolver basis. > *Dynamic Versions: * > If a resolver tells us that 1.+ is actually 1.5, we remember that for next > time. That way, we can look directly in the module cache to get the > descriptor for 1.+, rather than first having to resolve 1.+ to 1.5. > Dynamic Version cache entries expire by default in 24 hours. This value > can be modified using configuration.resolutionStrategy.cacheDynamicVersionsFor > 2, "minutes" > > *Changing Modules:* > Similarly, if a dependency is flagged as changing, or a resolver tells us > the module is changing (eg SNAPSHOT), we remember when we last fetched the > module meta-data and artifacts. If we find a cache entry indicating it's ok > to use the cached version, we use them. Otherwise, we bypass the cache and > download the descriptor and artifacts again. > Changing Module cache entries expire by default in 24 hours. This value > can be modified using configuration.resolutionStrategy.cacheChangingModulesFor > 2, "minutes" > > *Maven Snapshots:* > By implementing these resolution caches ourselves, we have been able to > remove the previous snapshot ttl that was provided by the > WharfIbiblioResolver. This has greatly simplified things internally. > > *Offline Mode:* > Although nothing official has been added, we should probably try out an > offline switch in our build that sets these cache values very high. In > theory, no requests should be required when everything has be previously > retrieved. > > > *Performance Improvements:* > Although the new caching improvements help a lot with performance when > things have been resolved before, we've also done some work to improve the > performance when resolving dependencies against maven repositories. Namely: > - Don't request javadoc and source jars when resolving the POM > - Don't request .sha1 and .md5 files when not required > - Don't use HTTP HEAD to check existence of file prior to downloading them. > > Together, these improvements have reduced the number of HTTP requests > required to resolve a static module from 11 down to 2, and a simple > SNAPSHOT module from 25 requests down to 4! > > *Refactoring:* > Adam spent a bunch of time replacing the core Ivy ResolveEngine with our > own implementation. This has already proven beneficial in making the other > improvements simpler. > For most of the internals, we've started using our own from-scratch > implementation of org.apache.ivy.plugins.resolver.DependencyResolver, in > place of extending AbstractResolver, BasicResolver, ChainResolver etc. Our > own implementations need only implement 3 key methods on the interface, and > we've removed a whole bunch of ivy black magic in the process. > > Another improvement has been to remove some of the power of the low-level > ivy resolvers that perform the actual resolution. Previously, each resolver > could look at what had been resolved earlier in the chain, and determine if > it do anything or accept the earlier resolution. Now each resolver behaves > independently, with the high level UserResolverChain doing the work of > choosing the best resolved version; short-circuiting resolution when > looking up static versions. > > --------- > > I've been using the latest code as my local gradle install and I'm pretty > sure I've ironed out most of the kinks for M6. Please give it a go and > shout if anything isn't behaving. Note that the artifact cache version is > changed, so you're initial builds will be slow due to the need to download > everything again. > > Enjoy! > > -- > Darrell (Daz) DeBoer > Principal Engineer, Gradleware > http://www.gradleware.com > >
