On Sun, Sep 8, 2013 at 3:34 PM, Adam Murdoch <adam.murd...@gradleware.com>wrote:
> > On 09/09/2013, at 5:21 AM, Daz DeBoer <darrell.deb...@gradleware.com> > wrote: > > G'day > Since we're currently changing the way we manage locks in the Gradle > cache, it would be great if we could separate the 'filestore' part of the > cache (that hasn't changed in structure since 1.0) from the binary metadata > part (which changes quite frequently). > > What this means is that we'd have these in separate directories: > ~/.gradle/caches/metadata-1 > ~/.gradle/caches/filestore-1 > > The 'filestore' would simply contain the content that is currently in > caches/artifacts-n/filestore. > > The benefit of this change is that the actual downloaded files would not > need to be copied into a new location every time we change the metadata > file format. > > The only complication I see is that we'd need a separate (cross-version) > lock for the filestore since it would be shared by different Gradle > versions. We'd need to bump the filestore version whenever the locking > mechanism changes, rather than every time the metadata store format changes. > > > I think having two locks would make it easy to deadlock when there are 2 > processes (or threads) resolving concurrently. We could address this by > enforcing a certain lock order (eg always lock the files if you need to > lock the metadata). > > Another option would be to have a single lock for both the files and > metadata, versioned on the locking protocol, something like: > > ~/.gradle/caches/files-1 > ~/.gradle/caches/files-1/store-1 > ~/.gradle/caches/files-1/metadata-1 > > It could even look at bit more intentional if we used eg: caches/modules-1 caches/modules-1/artifacts-1.0 caches/modules-1/metadata-1.3 This would be a cosmetic difference, but I think it would look clearer to users. Either way, any chance we can slip this into the cache locking changes that are currently in progress? Or add it as a story to be done soon? The way things currently work results in cache explosion over time: this change would slow that down somewhat. Daz
