Re: reftable [v5]: new ref storage format

2017-08-14 Thread Jeff King
On Mon, Aug 14, 2017 at 04:05:05PM +, David Turner wrote: > > All that aside, we could simply add an EXCLUSIVE open-flag to LMDB, and > > prevent multiple processes from using the DB concurrently. In that case, > > maintaining coherence with other NFS clients is a non-issue. It strikes me > >

RE: reftable [v5]: new ref storage format

2017-08-14 Thread David Turner
ag...@alum.mit.edu; p...@peff.net; > sbel...@google.com; sto...@gmail.com > Subject: Re: reftable [v5]: new ref storage format > > Howard Chu wrote: > > The primary issue with using LMDB over NFS is with performance. All > > reads are performed thru accesses of mapped memory, and in

Re: reftable [v5]: new ref storage format

2017-08-14 Thread Howard Chu
Howard Chu wrote: The primary issue with using LMDB over NFS is with performance. All reads are performed thru accesses of mapped memory, and in general, NFS implementations don't cache mmap'd pages. I believe this is a consequence of the fact that they also can't guarantee cache coherence, so

Re: reftable [v5]: new ref storage format

2017-08-09 Thread Howard Chu
Shawn Pearce wrote: On Sun, Aug 6, 2017 at 4:37 PM, Ben Alex wrote: > Just on the LmdbJava specific pieces: > > On Mon, Aug 7, 2017 at 8:56 AM, Shawn Pearce wrote: I don't know if we need a larger key size. $DAY_JOB limits ref names to ~200 bytes in a hook. I think GitHub does similar. But

Re: reftable [v5]: new ref storage format

2017-08-08 Thread Shawn Pearce
On Tue, Aug 8, 2017 at 12:52 AM, Jeff King wrote: > On Mon, Aug 07, 2017 at 03:40:48PM +, David Turner wrote: > >> > -Original Message- >> > From: Shawn Pearce [mailto:spea...@spearce.org] >> > In git-core, I'm worried about the caveats related to locking. Git tries >> > to work >> >

Re: reftable [v5]: new ref storage format

2017-08-08 Thread Jeff King
On Mon, Aug 07, 2017 at 03:40:48PM +, David Turner wrote: > > -Original Message- > > From: Shawn Pearce [mailto:spea...@spearce.org] > > In git-core, I'm worried about the caveats related to locking. Git tries to > > work > > nicely on NFS, and it seems LMDB wouldn't. Git also runs fi

Re: reftable [v5]: new ref storage format

2017-08-08 Thread Jeff King
On Mon, Aug 07, 2017 at 07:41:43AM -0700, Shawn Pearce wrote: > > As such if JGit wanted to use a longer key size, it is possible to implement > > similar automatic builds and packaging into JGit. > > I don't know if we need a larger key size. $DAY_JOB limits ref names > to ~200 bytes in a hook.

RE: reftable [v5]: new ref storage format

2017-08-07 Thread David Turner
> -Original Message- > From: Shawn Pearce [mailto:spea...@spearce.org] > In git-core, I'm worried about the caveats related to locking. Git tries to > work > nicely on NFS, and it seems LMDB wouldn't. Git also runs fine on a read-only > filesystem, and LMDB gets a little weird about that.

Re: reftable [v5]: new ref storage format

2017-08-07 Thread Shawn Pearce
On Sun, Aug 6, 2017 at 4:37 PM, Ben Alex wrote: > Just on the LmdbJava specific pieces: > > On Mon, Aug 7, 2017 at 8:56 AM, Shawn Pearce wrote: >> >> Looks pretty complete. Its a Java wrapper around the C implementation >> of LMDB, which may be sufficient for reference storage. Keys are >> limite

Re: reftable [v5]: new ref storage format

2017-08-06 Thread Shawn Pearce
On Sun, Aug 6, 2017 at 9:56 AM, Ævar Arnfjörð Bjarmason wrote: > On Sun, Aug 06 2017, Shawn Pearce jotted: > >> 5th iteration of the reftable storage format. > > I haven't kept up with all of the discussion, sorry if these comments > repeat something that's already mentioned. > >> ### Version 1 >>

Re: reftable [v5]: new ref storage format

2017-08-06 Thread Ævar Arnfjörð Bjarmason
On Sun, Aug 06 2017, Shawn Pearce jotted: > 5th iteration of the reftable storage format. I haven't kept up with all of the discussion, sorry if these comments repeat something that's already mentioned. > ### Version 1 > > A repository must set its `$GIT_DIR/config` to configure reftable: > >

Re: reftable [v5]: new ref storage format

2017-08-05 Thread Shawn Pearce
5th iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Significant changes from v4: - Supported Michael Haggerty's multi-level index. - Restart table now appears at start