Re: [PATCH v3] Refactor recv_sideband()

2016-06-22 Thread Nicolas Pitre
utput which is then > printed using a single write() call, such that the atomicity of the > output is preserved. See 9ac13ec (atomic write for sideband remote > messages, 2006-10-11) for details. > > Helped-by: Nicolas Pitre > Signed-off-by: Lukas Fleischer The patch is buggy.

Re: [PATCH v3] Refactor recv_sideband()

2016-06-22 Thread Nicolas Pitre
On Wed, 22 Jun 2016, Nicolas Pitre wrote: > On Wed, 22 Jun 2016, Lukas Fleischer wrote: > > > Before this patch, we used character buffer manipulations to split > > messages from the sideband at line breaks and insert "remote: " at the > > beginning of e

Re: [PATCH v3] Refactor recv_sideband()

2016-06-23 Thread Nicolas Pitre
On Thu, 23 Jun 2016, Lukas Fleischer wrote: > On Thu, 23 Jun 2016 at 00:47:39, Nicolas Pitre wrote: > > On Wed, 22 Jun 2016, Nicolas Pitre wrote: > > [...] > > > if (*b) { > > > xwrite(STDE

Re: [PATCH v2] Refactor recv_sideband()

2016-06-27 Thread Nicolas Pitre
On Mon, 27 Jun 2016, Lukas Fleischer wrote: > On Mon, 27 Jun 2016 at 19:50:13, Junio C Hamano wrote: > > Jeff King writes: > > > > > On Mon, Jun 27, 2016 at 08:54:22AM -0700, Junio C Hamano wrote: > > > > > >> It's just you used xwrite() there that introduced a different issue. > > >> Wouldn't r

Re: [PATCH v4] Refactor recv_sideband()

2016-06-28 Thread Nicolas Pitre
On Tue, 28 Jun 2016, Junio C Hamano wrote: > And then that made me stare at the patch even more. We still write > some error messages to stderr in the updated code (without my crap > SQUASH) inside "while (!retval)" loop: > > while (retval == 0) { > int band, len; >

Re: [PATCH v4] Refactor recv_sideband()

2016-06-28 Thread Nicolas Pitre
On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre writes: > > >> When we exit the loop because we set retval to a non-zero value, > >> should we still drain the outbuf? > > > > I would think so. Anything that the remote sent before any error shoul

Re: [PATCH v4] Refactor recv_sideband()

2016-06-28 Thread Nicolas Pitre
On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Without this, the error and remaining buffer would be reversed as > > mentioned previously. With this, the order is restored, but a newline > > is added to unterminated lines whereas the erro

Re: [PATCH v4] Refactor recv_sideband()

2016-06-28 Thread Nicolas Pitre
On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre writes: > > >> There is something else going on. I cannot quite explain why I am > >> getting this failure from t5401-update-hooks.sh, for example: > >> > >> --- expect 2016-06-28 19

Re: [PATCH v4] Refactor recv_sideband()

2016-06-28 Thread Nicolas Pitre
On Tue, 28 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre writes: > > >> The basic structure of the code (without the "SQUASH" we discussed) > >> looks like this: > >> > >>strbuf_addf(&outbuf, "%s", PREFIX); > >>

Re: [PATCH v4] Refactor recv_sideband()

2016-06-28 Thread Nicolas Pitre
On Tue, 28 Jun 2016, Junio C Hamano wrote: > Junio C Hamano writes: > > > It's just that if you take the latter, then the conditional after > > the loop exits (i.e. the last transmission was an incomplete line) > > cannot be "is outbuf empty?", as your base state is "has PREFIX and > > can never

Re: [PATCH v4] Refactor recv_sideband()

2016-07-05 Thread Nicolas Pitre
On Wed, 29 Jun 2016, Junio C Hamano wrote: > Nicolas Pitre writes: > > > To make it clearer, here's a patch on top of pu that fixes all the > > issues I think are remaining. All tests pass now. > > Lukas, can you see what is in 'pu' after I push out

Re: [PATCH v4] Refactor recv_sideband()

2016-07-06 Thread Nicolas Pitre
On Wed, 6 Jul 2016, Junio C Hamano wrote: > Thanks; will apply with a miniscule fix. > > > - >8 > > Subject: sideband.c: small optimization of strbuf usage > > > > Signed-off-by: Nicolas Pitre > > ... > > @@ -97,7 +97,7 @@ int recv_si

Re: [PATCH] Refactor recv_sideband()

2016-06-13 Thread Nicolas Pitre
ace a custom implementation of strpbrk() with a call > to the standard C library function. > > Signed-off-by: Lukas Fleischer The previous code was a total abomination, even if I happen to know who wrote it. Acked-by: Nicolas Pitre > I had a really hard time reading and understanding thi

Re: [PATCH] Refactor recv_sideband()

2016-06-14 Thread Nicolas Pitre
On Tue, 14 Jun 2016, Johannes Schindelin wrote: > Hi, > > On Mon, 13 Jun 2016, Nicolas Pitre wrote: > > > On Mon, 13 Jun 2016, Lukas Fleischer wrote: > > > > > Improve the readability of recv_sideband() significantly by replacing > > > fragile bu

Re: [PATCH] Refactor recv_sideband()

2016-06-14 Thread Nicolas Pitre
On Tue, 14 Jun 2016, Johannes Schindelin wrote: > Hi Nico, > > On Tue, 14 Jun 2016, Nicolas Pitre wrote: > > > On Tue, 14 Jun 2016, Johannes Schindelin wrote: > > > > > On Mon, 13 Jun 2016, Nicolas Pitre wrote: > > > > >

Re: [PATCH] Refactor recv_sideband()

2016-06-14 Thread Nicolas Pitre
On Mon, 13 Jun 2016, Nicolas Pitre wrote: > On Mon, 13 Jun 2016, Lukas Fleischer wrote: > > > Improve the readability of recv_sideband() significantly by replacing > > fragile buffer manipulations with more sophisticated format strings. > > Also, reorganize the overall c

Re: [PATCH] Refactor recv_sideband()

2016-06-14 Thread Nicolas Pitre
On Tue, 14 Jun 2016, Lukas Fleischer wrote: > Hi Nicolas, > > On Tue, 14 Jun 2016 at 19:09:15, Nicolas Pitre wrote: > > I just looked again at all the contraptions _I_ wrote (not Junio's) for > > a reason why I went to such extremes in making this code co complicated

Re: [PATCH] Refactor recv_sideband()

2016-06-14 Thread Nicolas Pitre
On Tue, 14 Jun 2016, Lukas Fleischer wrote: > On Tue, 14 Jun 2016 at 19:55:06, Nicolas Pitre wrote: > > It is not buffered as it writes to stderr. And some C libs do separate > > calls to write() for every string format specifier. So "%s%s%c" may end > > up calli

Re: [PATCH 2/2] index-pack: prefetch missing REF_DELTA bases

2019-05-20 Thread Nicolas Pitre
On Sat, 18 May 2019, Duy Nguyen wrote: > On Fri, May 17, 2019 at 3:55 PM Jeff King wrote: > > > > On Fri, May 17, 2019 at 02:20:42PM +0700, Duy Nguyen wrote: > > > > > On Fri, May 17, 2019 at 12:35 PM Jeff King wrote: > > > > As it turns out, index-pack does not handle these complicated cases at

Re: [PATCH 0/5] handle corruption in patch-delta

2018-08-30 Thread Nicolas Pitre
uption > > Your related trailing-garbage fix. I split this into two in order to > better demonstrate the cases this part covers. > > [5/5]: patch-delta: handle truncated copy parameters > > My fix for the cp_size read. > > I hope you don't mind me hacking up your patches a bit. Thanks again for > your original report and patch. Looks good to me (feels like traveling back in time). Reviewed-by: Nicolas Pitre > > -Peff >

Re: [PATCH 3/6] find_pack_entry(): document last_found_pack

2014-02-21 Thread Nicolas Pitre
; > Signed-off-by: Michael Haggerty REviewed-by: Nicolas Pitre > --- > sha1_file.c | 16 > 1 file changed, 12 insertions(+), 4 deletions(-) > > diff --git a/sha1_file.c b/sha1_file.c > index 6e8c05d..0910939 100644 > --- a/sha1_file.c > +++ b/sha1

Re: [PATCH 5/6] Document a bunch of functions defined in sha1_file.c

2014-02-21 Thread Nicolas Pitre
On Fri, 21 Feb 2014, Michael Haggerty wrote: > Signed-off-by: Michael Haggerty Acked-by: Nicolas Pitre > --- > cache.h | 66 > ++--- > sha1_file.c | 26 +--- > 2 files changed, 78 insertion

Re: [PATCH 6/6] Document some functions defined in object.c

2014-02-21 Thread Nicolas Pitre
- /* Assumes power-of-2 hash sizes in grow_object_hash */ > return hash & (n - 1); > } Other than that... Reviewed-by: Nicolas Pitre > > +/* > + * Insert obj into the hash table hash, which has length size (which > + * must be a power of 2). On collisions, sim

Re: [PATCH v2 2/2] Document some functions defined in object.c

2014-02-28 Thread Nicolas Pitre
On Fri, 28 Feb 2014, Michael Haggerty wrote: > Signed-off-by: Michael Haggerty Acked-by: Nicolas Pitre > --- > object.c | 29 - > object.h | 7 +++ > 2 files changed, 35 insertions(+), 1 deletion(-) > > diff --git a/object.c b/obj

Re: Pack v4 again..

2015-02-15 Thread Nicolas Pitre
On Fri, 13 Feb 2015, Duy Nguyen wrote: > After taking 1.5 years "vacation" from pack v4, I plan to do something > about it again. Will post more when I have some patches to discuss. > Only one question for now (forgive me if I asked already, it's been > quite some time) Yeah. I had to re-study m

Re: [ANNOUNCE] Git v1.8.4

2013-08-26 Thread Nicolas Pitre
On Fri, 23 Aug 2013, Junio C Hamano wrote: > The latest feature release Git v1.8.4 is now available at the usual > places. It contains 870+ changes from ~100 contributors (among > which 33 people are new) since v1.8.3. > > We will have two more releases til the end of this year; the release > af

Re: [ANNOUNCE] Git v1.8.4

2013-08-26 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Duy Nguyen wrote: > On Tue, Aug 27, 2013 at 7:06 AM, Nicolas Pitre wrote: > > Yes, after being vaporware for many many years (I don't even remember > > when I started making references to a possible pack format version 4 -- > > certainly more t

[PATCH 00/23] Preliminary pack v4 support

2013-08-26 Thread Nicolas Pitre
Subject says it all... at last ! This can also be fetched here: git://git.linaro.org/people/nico/git Demonstration of what it does at the moment: http://article.gmane.org/gmane.comp.version-control.git/233038 I'd like to preserve the author time stamps as they relate to where i

[PATCH 06/23] pack v4: split the object list and dictionary creation

2013-08-26 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 58 +++-- 1 file changed, 44 insertions(+), 14 deletions(-) diff --git a/packv4-create.c b/packv4-create.c index 5c08871..20d97a4 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -261,7

[PATCH 03/23] pack v4: scan tree objects

2013-08-26 Thread Nicolas Pitre
From: Nicolas Pitre Let's read a pack to feed our dictionary with all the path strings contained in all the tree objects. Dump the resulting dictionary sorted by frequency to stdout. Signed-off-by: Nicolas Pitre --- Makefile| 1 + packv4-create.c

[PATCH 07/23] pack v4: move to struct pack_idx_entry and get rid of our own struct idx_entry

2013-08-26 Thread Nicolas Pitre
Let's create a struct pack_idx_entry list with sorted sha1 which will be useful later. The offset sorted list is now a separate indirect list. Signed-off-by: Nicolas Pitre --- packv4-create.c | 72 + 1 file changed, 42 insertions(+

[PATCH 16/23] pack v4: tree object delta encoding

2013-08-26 Thread Nicolas Pitre
at the moment though. The code isn't optimal at the moment as it doesn't consider the case where a borrow sequence could be larger than the local sequence it means to replace. Signed-off-by: Nicolas Pitre --- packv4-create.c | 115 ++--

[PATCH 02/23] export packed_object_info()

2013-08-26 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- cache.h | 1 + sha1_file.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/cache.h b/cache.h index 85b544f..b6634c4 100644 --- a/cache.h +++ b/cache.h @@ -1160,6 +1160,7 @@ struct object_info { } u; }; extern int

[PATCH 10/23] pack v4: tree object encoding

2013-08-26 Thread Nicolas Pitre
is some kind of delta encoding for multiple tree objects with only small differences between them. But that'll come later. Signed-off-by: Nicolas Pitre --- packv4-create.c | 63 + 1 file changed, 63 insertions(+) diff --git a/p

[PATCH 23/23] initial pack index v3 support on the read side

2013-08-26 Thread Nicolas Pitre
A bit crud but good enough for now. Signed-off-by: Nicolas Pitre --- cache.h | 1 + sha1_file.c | 58 +++--- 2 files changed, 52 insertions(+), 7 deletions(-) diff --git a/cache.h b/cache.h index b6634c4..63066a1 100644 --- a/cache.h

[PATCH 05/23] pack v4: add commit object parsing

2013-08-26 Thread Nicolas Pitre
ed to get the string length as the provided string pointer is not always be null terminated. Signed-off-by: Nicolas Pitre --- packv4-create.c | 98 +++-- 1 file changed, 89 insertions(+), 9 deletions(-) diff --git a/packv4-create.c b/packv4-cre

[PATCH 08/23] pack v4: basic references encoding

2013-08-26 Thread Nicolas Pitre
SHA1 table encoded using the variable length number encoding, or the literal 20 bytes SHA1 prefixed with a 0. The index 0 discriminates between an actual index value or the literal SHA1. Therefore when the index is used its value must be increased by 1. Signed-off-by: Nicolas Pitre --- p

[PATCH 22/23] pack v4: add progress display

2013-08-26 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index 6801e21..22e14da 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -13,6 +13,7 @@ #include "tree-walk.h" #include "pa

[PATCH 20/23] pack index v3

2013-08-26 Thread Nicolas Pitre
This is a minor change over pack index v2. Since pack v4 already contains the sorted SHA1 table, it is therefore ommitted from the index file. Signed-off-by: Nicolas Pitre --- pack-write.c| 6 +- packv4-create.c | 10 +- 2 files changed, 14 insertions(+), 2 deletions(-) diff

[PATCH 09/23] pack v4: commit object encoding

2013-08-26 Thread Nicolas Pitre
index file requiring no SHA1 search into the pack index file. Signed-off-by: Nicolas Pitre --- packv4-create.c | 119 1 file changed, 119 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index bf33d15..cedbbd9 100644 --- a/p

[PATCH 21/23] pack v4: normalize pack name to properly generate the pack index file name

2013-08-26 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 73 +++-- 1 file changed, 34 insertions(+), 39 deletions(-) diff --git a/packv4-create.c b/packv4-create.c index bb171c5..6801e21 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -949,56

[PATCH 19/23] pack v4: relax commit parsing a bit

2013-08-26 Thread Nicolas Pitre
At least commit af25e94d4dcfb9608846242fabdd4e6014e5c9f0 in the Linux kernel repository has "author <> 1120285620 -0700" Signed-off-by: Nicolas Pitre --- packv4-create.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packv4-create.c b/packv4-create

[PATCH 13/23] pack v4: object headers

2013-08-26 Thread Nicolas Pitre
future, especially since pack v4 has only one type of delta object instead of two. Signed-off-by: Nicolas Pitre --- packv4-create.c | 26 ++ 1 file changed, 26 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index 5211f9c..6e0bb1d 100644 --- a/packv4-create.c

[PATCH 15/23] pack v4: object writing

2013-08-26 Thread Nicolas Pitre
once they're written to the destination file. This will be needed later for writing the pack index file. Signed-off-by: Nicolas Pitre --- packv4-create.c | 74 ++--- 1 file changed, 71 insertions(+), 3 deletions(-) diff --git a/p

[PATCH 18/23] pack v4: honor pack.compression config option

2013-08-26 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index ed67eb6..2d46d11 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -15,6 +15,7 @@ #include "pack-revindex.h" +

[PATCH 11/23] pack v4: dictionary table output

2013-08-26 Thread Nicolas Pitre
Here's the code to dump a table into a pack. Table entries are written according to the current sort order. This is important as objects use this order to index into the table. Signed-off-by: Nicolas Pitre --- packv4-create.c | 49 + 1

[PATCH 14/23] pack v4: object data copy

2013-08-26 Thread Nicolas Pitre
not found in the table (see add_sha1_ref). This is true for both REF_DELTA as well as OFS_DELTA. Object payload is validated against the recorded CRC32 in the source pack index file when possible. Signed-off-by: Nicolas Pitre --- packv4-create.c | 66

[PATCH 01/23] pack v4: initial pack dictionary structure and code

2013-08-26 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 137 1 file changed, 137 insertions(+) create mode 100644 packv4-create.c diff --git a/packv4-create.c b/packv4-create.c new file mode 100644 index 000..2de6d41 --- /dev/null +++ b

[PATCH 17/23] pack v4: load delta candidate for encoding tree objects

2013-08-26 Thread Nicolas Pitre
investigated eventually. Signed-off-by: Nicolas Pitre --- packv4-create.c | 69 ++--- 1 file changed, 66 insertions(+), 3 deletions(-) diff --git a/packv4-create.c b/packv4-create.c index 744514c..ed67eb6 100644 --- a/packv4-create.c +++ b/packv4

[PATCH 04/23] pack v4: add tree entry mode support to dictionary entries

2013-08-26 Thread Nicolas Pitre
Augment dict entries with a 16-bit prefix in order to store the file mode value of tree entries. Signed-off-by: Nicolas Pitre --- packv4-create.c | 56 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/packv4-create.c b

[PATCH 12/23] pack v4: creation code

2013-08-26 Thread Nicolas Pitre
late those objects first. The SHA1 table size is obtained by multiplying the number of objects by 20. And then the commit and path dictionary tables are written right after the SHA1 table. Signed-off-by: Nicolas Pitre --- packv4-create.c | 60 ++

Re: [PATCH 08/23] pack v4: basic references encoding

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Add variable length number encoding. Let's apply the same encoding > > currently used for the offset to base of OBJ_OFS_DELTA objects which > > is the most efficient way to do this, an

Re: [PATCH 00/23] Preliminary pack v4 support

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Subject says it all... at last ! > > > > This can also be fetched here: > > > > git://git.linaro.org/people/nico/git > > > > Demonstration of what it does at

Re: [PATCH 01/23] pack v4: initial pack dictionary structure and code

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Signed-off-by: Nicolas Pitre > > --- > > Was there a reason not to reuse the hash-table Linus did in > hash.[ch]? Well... Most likely because when I started that code (which used to be qu

Re: [PATCH 05/23] pack v4: add commit object parsing

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Let's create another dictionary table to hold the author and committer > > entries. We use the same table format used for tree entries where the > > 16 bit data prefix is conveniently use

Re: [PATCH 10/23] pack v4: tree object encoding

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > This goes as follows: > > > > - Number of tree entries: variable length encoded. > > > > Then for each tree entry: > > > > - Path component reference: variable length enc

Re: [PATCH 12/23] pack v4: creation code

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Let's actually open the destination pack file and write the header and > > the tables. > > > > The header isn't much different from pack v3, except for the pack version > > nu

Re: [PATCH 09/23] pack v4: commit object encoding

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > This goes as follows: > > > > - Tree reference: either variable length encoding of the index > > into the SHA1 table or the literal SHA1 prefixed by 0 (see > > add_sha1_ref()). >

Re: [PATCH 14/23] pack v4: object data copy

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > Blob and tag objects have no particular changes except for their object > > header. > > > > Delta objects are also copied as is, except for their delta base reference > > which is conver

Re: [PATCH] Document pack v4 format

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Nguyễn Thái Ngọc Duy wrote: > > Signed-off-by: Nguyễn Thái Ngọc Duy > --- > For my education but may help people who are interested in the > format. Most is gathered from commit messages, except the delta tree > entries. Excellent! That's the kind of thing I need help w

Re: [PATCH 09/23] pack v4: commit object encoding

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > + /* parse the "tree" line */ > > + if (in + 46 >= tail || memcmp(in, "tree ", 5) || in[45] != '\n') > > + goto bad_data; > > + if (get_sha

Re: [PATCH 09/23] pack v4: commit object encoding

2013-08-27 Thread Nicolas Pitre
On Tue, 27 Aug 2013, Junio C Hamano wrote: > Nicolas Pitre writes: > > > On Tue, 27 Aug 2013, Junio C Hamano wrote: > > > >> Nicolas Pitre writes: > >> > >> > +/* parse the "tree" line */ > >> > +

Re: [PATCH 00/23] Preliminary pack v4 support

2013-08-27 Thread Nicolas Pitre
On Wed, 28 Aug 2013, Duy Nguyen wrote: > On Tue, Aug 27, 2013 at 11:44 PM, Junio C Hamano wrote: > > As you have "0-index" escape hatch for SHA-1 table, but no similar > > escape hatch for the people's name table, I can see why it may be > > cumbersome to fix a thin pack by only appending to a re

Re: [PATCH v2] Document pack v4 format

2013-09-02 Thread Nicolas Pitre
On Sat, 31 Aug 2013, Nguyễn Thái Ngọc Duy wrote: > > Signed-off-by: Nguyễn Thái Ngọc Duy > --- > Incorporated suggestions by Nico and Junio. I went ahead and added > escape hatches for converting thin packs to full ones so the document > does not really match the code (I've been watching Nico

Re: [PATCH 23/23] initial pack index v3 support on the read side

2013-09-02 Thread Nicolas Pitre
On Sat, 31 Aug 2013, Duy Nguyen wrote: > On Tue, Aug 27, 2013 at 11:26 AM, Nicolas Pitre wrote: > > A bit crud but good enough for now. > > I wonder if we should keep a short SHA-1 table in .idx. An entry in > the original .idx v1 table is + then offset moved out > t

Re: [PATCH 09/23] pack v4: commit object encoding

2013-09-02 Thread Nicolas Pitre
On Tue, 3 Sep 2013, Duy Nguyen wrote: > On Tue, Aug 27, 2013 at 11:25 AM, Nicolas Pitre wrote: > > This goes as follows: > > > > - Tree reference: either variable length encoding of the index > > into the SHA1 table or the literal SHA1 prefixed by 0 (see > >

Re: [PATCH v2] Document pack v4 format

2013-09-02 Thread Nicolas Pitre
On Tue, 3 Sep 2013, Nicolas Pitre wrote: > On Sat, 31 Aug 2013, Nguyễn Thái Ngọc Duy wrote: > > > > > Signed-off-by: Nguyễn Thái Ngọc Duy > > --- > > Incorporated suggestions by Nico and Junio. I went ahead and added > > escape hatches for converting thi

Re: [PATCH 09/23] pack v4: commit object encoding

2013-09-04 Thread Nicolas Pitre
On Tue, 3 Sep 2013, Duy Nguyen wrote: > On Tue, Sep 3, 2013 at 1:30 PM, Nicolas Pitre wrote: > > On Tue, 3 Sep 2013, Duy Nguyen wrote: > > > >> On Tue, Aug 27, 2013 at 11:25 AM, Nicolas Pitre wrote: > >> > This goes as follows: > >> > > >&g

Re: [PATCH v2] Document pack v4 format

2013-09-04 Thread Nicolas Pitre
On Tue, 3 Sep 2013, Duy Nguyen wrote: > On Tue, Sep 3, 2013 at 6:49 PM, Duy Nguyen wrote: > > On Tue, Sep 3, 2013 at 1:46 PM, Nicolas Pitre wrote: > >> So... looks like pack v4 is now "functional". > >> > >> However something is still wrong as it o

Re: [PATCH v2] Document pack v4 format

2013-09-04 Thread Nicolas Pitre
On Thu, 5 Sep 2013, Duy Nguyen wrote: > On Thu, Sep 5, 2013 at 11:12 AM, Nicolas Pitre wrote: > > Many other bugs have now been fixed. A git.git repository with packs > > version 4 appears to be functional and passes git-fsck --full --strict. > > Yeah I was looking at the

Re: [PATCH v2] Document pack v4 format

2013-09-04 Thread Nicolas Pitre
On Thu, 5 Sep 2013, Duy Nguyen wrote: > On Thu, Sep 5, 2013 at 11:40 AM, Nicolas Pitre wrote: > > On Thu, 5 Sep 2013, Duy Nguyen wrote: > > > >> On Thu, Sep 5, 2013 at 11:12 AM, Nicolas Pitre wrote: > >> > Many other bugs have now been fixed. A git.git re

[PATCH 00/38] pack version 4 basic functionalities

2013-09-04 Thread Nicolas Pitre
After the initial posting here: http://news.gmane.org/group/gmane.comp.version-control.git/thread=233061 This is a repost plus the basic read side working, at least to validate the write side and the pack format itself. And many many bug fixes. This can also be fetched here: git://git.lina

[PATCH 34/38] pack v4: code to retrieve a path component

2013-09-04 Thread Nicolas Pitre
Because the path dictionary table is located right after the name dictionary table, we currently need to load the later to find the former. Signed-off-by: Nicolas Pitre --- cache.h| 2 ++ packv4-parse.c | 36 2 files changed, 38 insertions(+) diff

[PATCH 28/38] pack v4: code to load and prepare a pack dictionary table for use

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-parse.c | 77 ++ 1 file changed, 77 insertions(+) diff --git a/packv4-parse.c b/packv4-parse.c index 299fc48..26894bc 100644 --- a/packv4-parse.c +++ b/packv4-parse.c @@ -28,3 +28,80 @@ const

[PATCH 32/38] pack v4: parse delta base reference

2013-09-04 Thread Nicolas Pitre
There is only one type of delta with pack v4. The base reference encoding already handles either an offset (via the pack index) or a literal SHA1. We assume in the literal SHA1 case that the object lives in the same pack, just like with previous pack versions. Signed-off-by: Nicolas Pitre

[PATCH 33/38] pack v4: we can read commit objects now

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- Makefile | 1 + packv4-parse.c | 1 + packv4-parse.h | 7 +++ sha1_file.c| 10 ++ 4 files changed, 19 insertions(+) create mode 100644 packv4-parse.h diff --git a/Makefile b/Makefile index ba6cafc..22fc276 100644 --- a/Makefile +++ b

[PATCH 38/38] packv4-create: add a command line argument to limit tree copy sequences

2013-09-04 Thread Nicolas Pitre
tree deltas entirely. This allows for experiments with the delta width and see the influence on pack size vs runtime access cost. Signed-off-by: Nicolas Pitre --- packv4-create.c | 27 --- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/packv4-create.c b/p

[PATCH 35/38] pack v4: decode tree objects

2013-09-04 Thread Nicolas Pitre
objects without going through the SHA1 search. Signed-off-by: Nicolas Pitre --- packv4-parse.c | 137 ++--- 1 file changed, 131 insertions(+), 6 deletions(-) diff --git a/packv4-parse.c b/packv4-parse.c index b80b73e..04eab46 100644 --- a/packv4

[PATCH 12/38] pack v4: dictionary table output

2013-09-04 Thread Nicolas Pitre
Here's the code to dump a table into a pack. Table entries are written according to the current sort order. This is important as objects use this order to index into the table. Signed-off-by: Nicolas Pitre --- packv4-create.c | 49 + 1

[PATCH 31/38] sha1_file.c: make use of decode_varint()

2013-09-04 Thread Nicolas Pitre
... replacing the equivalent open coded loop. Signed-off-by: Nicolas Pitre --- sha1_file.c | 14 +++--- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/sha1_file.c b/sha1_file.c index a298933..67eb903 100644 --- a/sha1_file.c +++ b/sha1_file.c @@ -1687,20 +1687,12

[PATCH 29/38] pack v4: code to retrieve a name

2013-09-04 Thread Nicolas Pitre
The name dictionary is loaded if not already done. We know it is located right after the SHA1 table (20 bytes per object) which is itself right after the 12-byte header. Then the index is parsed from the input buffer and a pointer to the corresponding entry is returned. Signed-off-by: Nicolas

[PATCH 37/38] pack v4: introduce "escape hatches" in the name and path indexes

2013-09-04 Thread Nicolas Pitre
-by: Nicolas Pitre --- packv4-create.c | 6 +++--- packv4-parse.c | 28 ++-- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/packv4-create.c b/packv4-create.c index fd16222..9d6ffc0 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -343,7 +343,7 @@ void

[PATCH 36/38] pack v4: get tree objects

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-parse.c | 25 + packv4-parse.h | 2 ++ sha1_file.c| 2 +- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/packv4-parse.c b/packv4-parse.c index 04eab46..4c218d2 100644 --- a/packv4-parse.c +++ b/packv4-parse.c

[PATCH 10/38] pack v4: commit object encoding

2013-09-04 Thread Nicolas Pitre
o the pack index file requiring no SHA1 search into the pack index file. Signed-off-by: Nicolas Pitre --- packv4-create.c | 119 1 file changed, 119 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index 12527c0..d4a79f4 100644

[PATCH 01/38] pack v4: initial pack dictionary structure and code

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 137 1 file changed, 137 insertions(+) create mode 100644 packv4-create.c diff --git a/packv4-create.c b/packv4-create.c new file mode 100644 index 000..2de6d41 --- /dev/null +++ b

[PATCH 22/38] pack index v3

2013-09-04 Thread Nicolas Pitre
This is a minor change over pack index v2. Since pack v4 already contains the sorted SHA1 table, it is therefore ommitted from the index file. Signed-off-by: Nicolas Pitre --- pack-write.c| 6 +- packv4-create.c | 10 +- 2 files changed, 14 insertions(+), 2 deletions(-) diff

[PATCH 20/38] pack v4: honor pack.compression config option

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index c8d3053..45f8427 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -16,6 +16,7 @@ #include "varint.h" +

[PATCH 21/38] pack v4: relax commit parsing a bit

2013-09-04 Thread Nicolas Pitre
At least commit af25e94d4dcfb9608846242fabdd4e6014e5c9f0 in the Linux kernel repository has "author <> 1120285620 -0700" Signed-off-by: Nicolas Pitre --- packv4-create.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packv4-create.c b/packv4-create

[PATCH 14/38] pack v4: object headers

2013-09-04 Thread Nicolas Pitre
future, especially since pack v4 has only one type of delta object instead of two. Signed-off-by: Nicolas Pitre --- packv4-create.c | 27 +++ 1 file changed, 27 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index 61b70c8..6098062 100644 --- a/packv4-create.c

[PATCH 03/38] pack v4: scan tree objects

2013-09-04 Thread Nicolas Pitre
Let's read a pack to feed our dictionary with all the path strings contained in all the tree objects. Dump the resulting dictionary sorted by frequency to stdout. Signed-off-by: Nicolas Pitre --- Makefile| 1 + packv4-create.c

[PATCH 16/38] pack v4: object writing

2013-09-04 Thread Nicolas Pitre
once they're written to the destination file. This will be needed later for writing the pack index file. Signed-off-by: Nicolas Pitre --- packv4-create.c | 74 ++--- 1 file changed, 71 insertions(+), 3 deletions(-) diff --git a/p

[PATCH 07/38] pack v4: move to struct pack_idx_entry and get rid of our own struct idx_entry

2013-09-04 Thread Nicolas Pitre
Let's create a struct pack_idx_entry list with sorted sha1 which will be useful later. The offset sorted list is now a separate indirect list. Signed-off-by: Nicolas Pitre --- packv4-create.c | 72 + 1 file changed, 42 insertions(+

[PATCH 04/38] pack v4: add tree entry mode support to dictionary entries

2013-09-04 Thread Nicolas Pitre
Augment dict entries with a 16-bit prefix in order to store the file mode value of tree entries. Signed-off-by: Nicolas Pitre --- packv4-create.c | 56 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/packv4-create.c b

[PATCH 26/38] pack v4: object header decode

2013-09-04 Thread Nicolas Pitre
For this we need the pack version. However only open_packed_git_1() has been audited for pack v4 so far, hence the version validation is not added to pack_version_ok() just yet. Signed-off-by: Nicolas Pitre --- cache.h | 1 + sha1_file.c | 14 -- 2 files changed, 13 insertions

[PATCH 23/38] packv4-create: normalize pack name to properly generate the pack index file name

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- packv4-create.c | 73 +++-- 1 file changed, 34 insertions(+), 39 deletions(-) diff --git a/packv4-create.c b/packv4-create.c index 22cdf8e..c23c791 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -956,56

[PATCH 08/38] pack v4: basic SHA1 reference encoding

2013-09-04 Thread Nicolas Pitre
. Signed-off-by: Nicolas Pitre --- packv4-create.c | 29 + 1 file changed, 29 insertions(+) diff --git a/packv4-create.c b/packv4-create.c index 012129b..12527c0 100644 --- a/packv4-create.c +++ b/packv4-create.c @@ -12,6 +12,7 @@ #include "object.h" #include &q

[PATCH 17/38] pack v4: tree object delta encoding

2013-09-04 Thread Nicolas Pitre
se where a copy sequence could be larger than the local sequence it means to replace. Signed-off-by: Nicolas Pitre --- packv4-create.c | 108 +--- 1 file changed, 103 insertions(+), 5 deletions(-) diff --git a/packv4-create.c b/packv4-create.c

[PATCH 13/38] pack v4: creation code

2013-09-04 Thread Nicolas Pitre
late those objects first. The SHA1 table size is obtained by multiplying the number of objects by 20. And then the commit and path dictionary tables are written right after the SHA1 table. Signed-off-by: Nicolas Pitre --- packv4-create.c | 60 ++

[PATCH 02/38] export packed_object_info()

2013-09-04 Thread Nicolas Pitre
Signed-off-by: Nicolas Pitre --- cache.h | 1 + sha1_file.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/cache.h b/cache.h index 85b544f..b6634c4 100644 --- a/cache.h +++ b/cache.h @@ -1160,6 +1160,7 @@ struct object_info { } u; }; extern int

[PATCH 09/38] introduce get_sha1_lowhex()

2013-09-04 Thread Nicolas Pitre
This is like get_sha1_hex() but stricter in accepting lowercase letters only. Signed-off-by: Nicolas Pitre --- cache.h | 3 +++ hex.c | 11 +++ 2 files changed, 14 insertions(+) diff --git a/cache.h b/cache.h index b6634c4..4231dfa 100644 --- a/cache.h +++ b/cache.h @@ -850,8

  1   2   >