On 26/09/2011, at 6:34 AM, Adam Murdoch wrote: > When resolving a dependency descriptor: > * We look for a matching dependency in the cache for each resolver. We don't > invoke the resolvers at this stage. If a matching cached value is found that > has not expired, we use the cached value, and stop looking. > * Otherwise, we attempt to resolve the dependency using each resolver in > turn. We stop on the first resolver that can resolve the dependency. > * If this fails, and an expired entry was found in the cache, use that > instead. > * We remember which resolver found the module, for downloading the artifacts > later. > > When resolving an artifact, we delegate directly to the resolver where the > artifact's module was found. > > Also, we apply the same expiry time for all dynamic revisions. This includes > snapshot revisions, changing = true, anything that matches a changing > pattern, version ranges, dynamic revisions (1.2+, etc) and statuses > (latest.integration). When we resolve a dynamic dependency descriptor, we > persist the module that we ended up resolving to, and we use that value until > the expiry is reached. > > Some implications of this: > > * We're making a performance-accuracy trade-off here. Which means we'll > probably need some way to tweak the behaviour. Not sure exactly how this > might look, yet. I'd start with a simple time-to-live property on each > configuration, and let the use cases drive anything beyond that. > > * For dynamic revisions, stopping on the first resolver means we may miss a > newer revision that happens to be in a later repository. An alternate > approach might be to use all resolvers for dynamic revisions, but only when > there is no unexpired value in the cache. We could do this search in > parallel, and just pick the latest out of those we have found at the end of > some timeout. Perhaps we could do the search in parallel for all revisions, > dynamic or not.
If that timeout as expired, we should try all resolvers I think. If parallel is achievable in our timeframes then I can't see a reason not to. Also, we could save some time if we can specify that some repositories will never have newer snapshot versions. There is no point checking maven central for a newer version of the same version number of anything if it is already cached. However, this isn't likely to offer much of a saving on small - medium projects. > * We fetch artifacts only from the same repository as their module was found > in (but this repository can have multiple patterns/base urls/etc). I think > this is a good thing, from an accuracy/repeatability point of view. Makes sense. > * The fact that all dynamic revisions have the same time-to-live is a change > in behaviour, but a good one, I think. Agreed. > Thoughts? I want to get on to this as soon as milestone 5 is out. Have we considered mapping dependencies to specific repositories? i.e. specifying that dependencies can only come from certain repositories would certainly make resolution faster for large projects but is less convenient. Perhaps it could be optional, with the default behaviour being that a dependency will be searched for in each repository. Another way to achieve this might be to have include/exclude patterns on repositories that are checked before attempting to search it for a particular artifact. However, this is probably not necessary for 1.0 and I think the improvements you outline will be good enough and will give enough people “good performance”. -- Luke Daley Principal Engineer, Gradleware http://gradleware.com
