G'day Over the past couple of weeks we've made some significant improvements to the way dependency resolution works in Gradle. There are some things that are just refactorings, some performance improvements, and a couple of new features. These changes should make it into M6, which is due out soon. * * *Caching Improvements:* The big new change in M6 is that Gradle now takes control of the caching of Dynamic Versions (eg 1.+) and Changing Modules (eg SNAPSHOT). A new metadata file in the cache, "dynamic-versions.bin", caches information about module resolution on a per-resolver basis. *Dynamic Versions: * If a resolver tells us that 1.+ is actually 1.5, we remember that for next time. That way, we can look directly in the module cache to get the descriptor for 1.+, rather than first having to resolve 1.+ to 1.5. Dynamic Version cache entries expire by default in 24 hours. This value can be modified using configuration.resolutionStrategy.cacheDynamicVersionsFor 2, "minutes"
*Changing Modules:* Similarly, if a dependency is flagged as changing, or a resolver tells us the module is changing (eg SNAPSHOT), we remember when we last fetched the module meta-data and artifacts. If we find a cache entry indicating it's ok to use the cached version, we use them. Otherwise, we bypass the cache and download the descriptor and artifacts again. Changing Module cache entries expire by default in 24 hours. This value can be modified using configuration.resolutionStrategy.cacheChangingModulesFor 2, "minutes" *Maven Snapshots:* By implementing these resolution caches ourselves, we have been able to remove the previous snapshot ttl that was provided by the WharfIbiblioResolver. This has greatly simplified things internally. *Offline Mode:* Although nothing official has been added, we should probably try out an offline switch in our build that sets these cache values very high. In theory, no requests should be required when everything has be previously retrieved. *Performance Improvements:* Although the new caching improvements help a lot with performance when things have been resolved before, we've also done some work to improve the performance when resolving dependencies against maven repositories. Namely: - Don't request javadoc and source jars when resolving the POM - Don't request .sha1 and .md5 files when not required - Don't use HTTP HEAD to check existence of file prior to downloading them. Together, these improvements have reduced the number of HTTP requests required to resolve a static module from 11 down to 2, and a simple SNAPSHOT module from 25 requests down to 4! *Refactoring:* Adam spent a bunch of time replacing the core Ivy ResolveEngine with our own implementation. This has already proven beneficial in making the other improvements simpler. For most of the internals, we've started using our own from-scratch implementation of org.apache.ivy.plugins.resolver.DependencyResolver, in place of extending AbstractResolver, BasicResolver, ChainResolver etc. Our own implementations need only implement 3 key methods on the interface, and we've removed a whole bunch of ivy black magic in the process. Another improvement has been to remove some of the power of the low-level ivy resolvers that perform the actual resolution. Previously, each resolver could look at what had been resolved earlier in the chain, and determine if it do anything or accept the earlier resolution. Now each resolver behaves independently, with the high level UserResolverChain doing the work of choosing the best resolved version; short-circuiting resolution when looking up static versions. --------- I've been using the latest code as my local gradle install and I'm pretty sure I've ironed out most of the kinks for M6. Please give it a go and shout if anything isn't behaving. Note that the artifact cache version is changed, so you're initial builds will be slow due to the need to download everything again. Enjoy! -- Darrell (Daz) DeBoer Principal Engineer, Gradleware http://www.gradleware.com
