On 07/11/2011, at 8:18 AM, Magnus Rundberget wrote: > Cool stuff ! > > Just tested 1.0-milestone-6-20111106000016+0100 > Unfortunately I had trimmed down the number of repositories and snapshot > usage (after milestone4 and partially still milestone 5 resolution > performance). > However I could still see a improvement (albeit small compared to groovy > compilation times) in dependency resolution performance. > > I must have missed something in the versioning strategy stuff or a bug has > been introduced. > > Suspicious behavior: > - In my build script I have set up dependency configuration for groovy(-all) > to 1.8.3 > - I have a testCompile dependency to spock which again has a transitive > dependency to a snapshot version (1.8.0-SNAPSHOT) of groovy-all > - When running my build with milestone-6 it complains that it can't find the > 1.8.0 SNAPSHOT version > - Disabling the transitive deps for the spock libs ensures it works. > > > So I'm guessing there might be a slight regression going on here (latest > version strategy not handling transitives with snapshot versions perhaps) ? > Bug or feature ?
It's a bug: http://issues.gradle.org/browse/GRADLE-1897 > > > cheers > Magnus > > > > > > > dependencies { > groovy group: 'org.codehaus.groovy', name: 'groovy-all', version: > '1.8.3' > > > providedCompile 'javax.servlet:servlet-api:2.4' > // consolidate logging to use slf4 api and logback as implementation > // Note ordering of dependencies to ensure logback initialized properly > as early as possible > compile 'ch.qos.logback:logback-core:0.9.29', > 'ch.qos.logback:logback-classic:0.9.29', > 'org.slf4j:jcl-over-slf4j:1.6.1', > 'org.slf4j:jul-to-slf4j:1.6.1' > > compile("org.grails:grails-gorm:$grailsVersion") { > exclude group: 'org.springframework', module: 'spring-webmvc' > exclude group: 'org.springframework', module: 'spring-jms' > } > compile "org.grails:grails-bootstrap:$grailsVersion", > "org.springframework:spring-beans:$springVersion", > "org.apache.tapestry:tapestry-core:$tapestryVersion", > "org.apache.tapestry:tapestry-spring:$tapestryVersion" > compile('org.got5:tapestry5-jquery:2.6.0') //{changing=true} > compile 'joda-time:joda-time:1.6.2', 'joda-time:joda-time-hibernate:1.2' > > compile 'com.h2database:h2:1.3.157' > testCompile 'junit:junit:4.8.2', > 'org.spockframework:spock-core:0.5-groovy-1.8@jar', /* NOTE I > HAVE TO DISABLE TRANSITIVE DEPS HERE in milestone 6 */ > 'org.spockframework:spock-spring:0.5-groovy-1.8@jar', /* NOTE I > HAVE TO DISABLE TRANSITIVE DEPS HERE in milestone 6 */ > "org.grails:grails-test:$grailsVersion" > } > > > > > > > > > Date: Sat, 5 Nov 2011 15:40:07 -0600 > From: [email protected] > To: [email protected] > Subject: [gradle-dev] Dependency resolution improvements > > G'day > > Over the past couple of weeks we've made some significant improvements to the > way dependency resolution works in Gradle. There are some things that are > just refactorings, some performance improvements, and a couple of new > features. These changes should make it into M6, which is due out soon. > > Caching Improvements: > The big new change in M6 is that Gradle now takes control of the caching of > Dynamic Versions (eg 1.+) and Changing Modules (eg SNAPSHOT). A new metadata > file in the cache, "dynamic-versions.bin", caches information about module > resolution on a per-resolver basis. > Dynamic Versions: > If a resolver tells us that 1.+ is actually 1.5, we remember that for next > time. That way, we can look directly in the module cache to get the > descriptor for 1.+, rather than first having to resolve 1.+ to 1.5. > Dynamic Version cache entries expire by default in 24 hours. This value can > be modified using configuration.resolutionStrategy.cacheDynamicVersionsFor 2, > "minutes" > > Changing Modules: > Similarly, if a dependency is flagged as changing, or a resolver tells us the > module is changing (eg SNAPSHOT), we remember when we last fetched the module > meta-data and artifacts. If we find a cache entry indicating it's ok to use > the cached version, we use them. Otherwise, we bypass the cache and download > the descriptor and artifacts again. > Changing Module cache entries expire by default in 24 hours. This value can > be modified using configuration.resolutionStrategy.cacheChangingModulesFor 2, > "minutes" > > Maven Snapshots: > By implementing these resolution caches ourselves, we have been able to > remove the previous snapshot ttl that was provided by the > WharfIbiblioResolver. This has greatly simplified things internally. > > Offline Mode: > Although nothing official has been added, we should probably try out an > offline switch in our build that sets these cache values very high. In > theory, no requests should be required when everything has be previously > retrieved. > > > Performance Improvements: > Although the new caching improvements help a lot with performance when things > have been resolved before, we've also done some work to improve the > performance when resolving dependencies against maven repositories. Namely: > - Don't request javadoc and source jars when resolving the POM > - Don't request .sha1 and .md5 files when not required > - Don't use HTTP HEAD to check existence of file prior to downloading them. > > Together, these improvements have reduced the number of HTTP requests > required to resolve a static module from 11 down to 2, and a simple SNAPSHOT > module from 25 requests down to 4! > > Refactoring: > Adam spent a bunch of time replacing the core Ivy ResolveEngine with our own > implementation. This has already proven beneficial in making the other > improvements simpler. > For most of the internals, we've started using our own from-scratch > implementation of org.apache.ivy.plugins.resolver.DependencyResolver, in > place of extending AbstractResolver, BasicResolver, ChainResolver etc. Our > own implementations need only implement 3 key methods on the interface, and > we've removed a whole bunch of ivy black magic in the process. > > Another improvement has been to remove some of the power of the low-level ivy > resolvers that perform the actual resolution. Previously, each resolver could > look at what had been resolved earlier in the chain, and determine if it do > anything or accept the earlier resolution. Now each resolver behaves > independently, with the high level UserResolverChain doing the work of > choosing the best resolved version; short-circuiting resolution when looking > up static versions. > > --------- > > I've been using the latest code as my local gradle install and I'm pretty > sure I've ironed out most of the kinks for M6. Please give it a go and shout > if anything isn't behaving. Note that the artifact cache version is changed, > so you're initial builds will be slow due to the need to download everything > again. > > Enjoy! > > -- > Darrell (Daz) DeBoer > Principal Engineer, Gradleware > http://www.gradleware.com > -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
