Re: reftable [v6]: new ref storage format

2017-08-18 Thread Michael Haggerty
On Wed, Aug 16, 2017 at 12:47 AM, Shawn Pearce wrote: > On Mon, Aug 14, 2017 at 5:13 AM, Michael Haggerty > wrote: >> On 08/07/2017 03:47 AM, Shawn Pearce wrote: >>> 6th iteration of the reftable storage format. > [...] >>> index record >>> >>> An index record describes the last entry in an

Re: reftable [v6]: new ref storage format

2017-08-15 Thread Shawn Pearce
On Mon, Aug 14, 2017 at 5:13 AM, Michael Haggerty wrote: > On 08/07/2017 03:47 AM, Shawn Pearce wrote: >> 6th iteration of the reftable storage format. > > Thanks! > >> Changes from v5: >> - extensions.refStorage = reftable is used to select this format. >> >> - Log records can be explicitly delet

Re: reftable [v6]: new ref storage format

2017-08-14 Thread Michael Haggerty
On 08/07/2017 03:47 AM, Shawn Pearce wrote: > 6th iteration of the reftable storage format. Thanks! > Changes from v5: > - extensions.refStorage = reftable is used to select this format. > > - Log records can be explicitly deleted (for refs/stash). > - Log records may use Michael Haggerty's chai

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Shawn Pearce
On Tue, Aug 8, 2017 at 4:34 PM, Junio C Hamano wrote: > Shawn Pearce writes: > >> Given that the index can now also be multi-level, I don't expect to >> see a 2G index. A 2G index forces the reader to load the entire 2G to >> take advantage of the restart table. It may be more efficient for such

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Stefan Beller
On Mon, Aug 7, 2017 at 11:30 AM, Shawn Pearce wrote: > On Mon, Aug 7, 2017 at 11:27 AM, Stefan Beller wrote: >> On Sun, Aug 6, 2017 at 6:47 PM, Shawn Pearce wrote: >>> 6th iteration of the reftable storage format. >>> >>> You can read a rendered version of this here: >>> https://googlers.googles

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Junio C Hamano
Shawn Pearce writes: > Given that the index can now also be multi-level, I don't expect to > see a 2G index. A 2G index forces the reader to load the entire 2G to > take advantage of the restart table. It may be more efficient for such > a reader to have had the writer make a mutli-level index, i

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Shawn Pearce
On Tue, Aug 8, 2017 at 12:25 PM, Junio C Hamano wrote: > Shawn Pearce writes: > >> For `log_type = 0x4..0x7` the `log_chained` section is used instead to >> compress information that already appeared in a prior log record. The >> `log_chained` always includes `old_id` for this record, as `new_id

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Shawn Pearce
On Tue, Aug 8, 2017 at 12:01 PM, Junio C Hamano wrote: > I notice that you raised the location of restart table within a > block in this iteration (or maybe it happened in v5). > > This forces you to hold all contents in core before the first byte > is written out. You start from the first entry

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Junio C Hamano
Shawn Pearce writes: > For `log_type = 0x4..0x7` the `log_chained` section is used instead to > compress information that already appeared in a prior log record. The > `log_chained` always includes `old_id` for this record, as `new_id` is > implied by the prior (by file order, more recent) recor

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Junio C Hamano
I notice that you raised the location of restart table within a block in this iteration (or maybe it happened in v5). This forces you to hold all contents in core before the first byte is written out. You start from the first entry (which will become the first restart entry), emit a handful as

Re: reftable [v6]: new ref storage format

2017-08-08 Thread Jeff King
On Sun, Aug 06, 2017 at 06:47:06PM -0700, Shawn Pearce wrote: > Changes from v5: > - extensions.refStorage = reftable is used to select this format. Thanks, I think this is a better scheme going forward. Just a few notes on compatibility while I'm thinking about it: - existing versions will co

Re: reftable [v6]: new ref storage format

2017-08-07 Thread Shawn Pearce
On Mon, Aug 7, 2017 at 11:27 AM, Stefan Beller wrote: > On Sun, Aug 6, 2017 at 6:47 PM, Shawn Pearce wrote: >> 6th iteration of the reftable storage format. >> >> You can read a rendered version of this here: >> https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftabl

Re: reftable [v6]: new ref storage format

2017-08-07 Thread Stefan Beller
On Sun, Aug 6, 2017 at 6:47 PM, Shawn Pearce wrote: > 6th iteration of the reftable storage format. > > You can read a rendered version of this here: > https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md > > Changes from v5: > - extensions.refStorage = reftabl

Re: reftable [v6]: new ref storage format

2017-08-06 Thread Shawn Pearce
6th iteration of the reftable storage format. You can read a rendered version of this here: https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md Changes from v5: - extensions.refStorage = reftable is used to select this format. - Log records can be explicitly