On 17/01/2012, at 12:15 PM, Luke Daley wrote: > On 16 Jan 2012, at 17:09, Daz DeBoer wrote: > >> G'day >> Currently, we are caching changing modules for 24 hours by default. This >> means that even though we know the module may change, we only check for >> updated version once per day. >> >> I think this confuses users, who expect changing modules (SNAPSHOTs) to be >> up-to-date. I'm guessing that maven works this way? > > Maven by default has a ttl of 24 hours. > >> I propose we make 0 seconds the default cache timeout for changing modules >> (user configurable). >> >> When we do a check, we follow this process for the meta-data file as well as >> any artifacts:1) Attempt to download an SHA1 key for the artifact2) If the >> downloaded key matches the current snapshot artifact, do nothing >> 3) If there is no SHA1 key, or it does not match, download the >> module/artifact >> This means that the process is pretty efficient when SHA1 keys are >> published.But when no SHA1 keys are published, this would mean that we'd be >> downloading the artifacts every time*. (Users could then do >> cacheChangingModulesFor 24, 'hours') >> >> Thoughts? > > I'm not for this. I think 24 hours is a reasonable default. Even though the > amount of traffic is low for checksum checking, it could still be painful > over a slow/flaky connection.
I agree. This will be painful without a reasonable timeout on all network requests. I don't want to wait 3 minutes for tcp to timeout when my network connection is stuck, so that Gradle can figure out that nothing's changed. Here's some things I think we could do, to allow us to make the cache timeout shorter. I wouldn't make the timeout much shorter without doing some of these things: * Introduce a timeout on network requests, and make it quite short when we're checking for changed artifacts. * Check for changed artifacts in background in the daemon. * Check for online/offline repositories in the background in the daemon, and check for changed artifacts more frequently for online repositories. * Do dependency resolution in parallel to task execution, so that we can be doing useful work while dependency resolution is hitting the network. * Do dependency resolution in parallel to network requests, so that we can have multiple network requests in progress at any given time. * Reduce the cost of a missing SHA1, by checking etag and/or modified date. * Reduce the likelihood of a missing SHA1, by publishing checksums to ivy repositories (in addition to maven repositories). * Work with repository manager vendors to define a more efficient protocol (over http), so we can do a bulk check. -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
