On 11/09/2013, at 12:35 AM, Daz DeBoer <darrell.deb...@gradleware.com> wrote:
> On Mon, Sep 9, 2013 at 10:48 PM, Adam Murdoch <adam.murd...@gradleware.com> > wrote: > > On 10/09/2013, at 2:42 PM, Daz DeBoer <darrell.deb...@gradleware.com> wrote: > >> >> On 9 Sep 2013 13:52, "Adam Murdoch" <adam.murd...@gradleware.com> wrote: >> > >> > >> > On 10/09/2013, at 5:32 AM, Daz DeBoer <darrell.deb...@gradleware.com> >> > wrote: >> > >> >> On Sun, Sep 8, 2013 at 3:34 PM, Adam Murdoch >> >> <adam.murd...@gradleware.com> wrote: >> >>> >> >>> >> >>> On 09/09/2013, at 5:21 AM, Daz DeBoer <darrell.deb...@gradleware.com> >> >>> wrote: >> >>> >> >>>> G'day >> >>>> Since we're currently changing the way we manage locks in the Gradle >> >>>> cache, it would be great if we could separate the 'filestore' part of >> >>>> the cache (that hasn't changed in structure since 1.0) from the binary >> >>>> metadata part (which changes quite frequently). >> >>>> >> >>>> What this means is that we'd have these in separate directories: >> >>>> ~/.gradle/caches/metadata-1 >> >>>> ~/.gradle/caches/filestore-1 >> >>>> >> >>>> The 'filestore' would simply contain the content that is currently in >> >>>> caches/artifacts-n/filestore. >> >>>> >> >>>> The benefit of this change is that the actual downloaded files would >> >>>> not need to be copied into a new location every time we change the >> >>>> metadata file format. >> >>>> >> >>>> The only complication I see is that we'd need a separate >> >>>> (cross-version) lock for the filestore since it would be shared by >> >>>> different Gradle versions. We'd need to bump the filestore version >> >>>> whenever the locking mechanism changes, rather than every time the >> >>>> metadata store format changes. >> >>> >> >>> >> >>> I think having two locks would make it easy to deadlock when there are 2 >> >>> processes (or threads) resolving concurrently. We could address this by >> >>> enforcing a certain lock order (eg always lock the files if you need to >> >>> lock the metadata). >> >>> >> >>> Another option would be to have a single lock for both the files and >> >>> metadata, versioned on the locking protocol, something like: >> >>> >> >>> ~/.gradle/caches/files-1 >> >>> ~/.gradle/caches/files-1/store-1 >> >>> ~/.gradle/caches/files-1/metadata-1 >> >>> >> >> >> >> It could even look at bit more intentional if we used eg: >> >> >> >> caches/modules-1 >> >> caches/modules-1/artifacts-1.0 >> >> caches/modules-1/metadata-1.3 >> > >> > >> > This isn't really going to work. If a format change is backwards >> > compatible, then we should not change the directory name, otherwise we >> > have two sets of processes that are using different stores even though >> > they can share them. So we wouldn't use the minor version in file names, >> > only major versions. >> > >> >> I don't really understand how my proposal is any different, except in the >> actual numbers used. When the lock file format changes, we need to create a >> new top level directory with new file store and metadata. When the file >> store or metadata formats change we bump that version only. The actual >> numbers uses are arbitrary. > > So we'd use either `thing-n.0` or `thing-1.n` as the naming scheme, where `n` > is the version number for the thing? > > I was thinking: > > caches/modules-x/filestore-x.y > caches/modules-x/metadata-x.z > I get it now. That could work. -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com Join us at the Gradle eXchange 2013, Oct 28th in London, UK: http://skillsmatter.com/event/java-jee/gradle-exchange-2013
