Re: Git Project Leadership Committee

2018-08-20 Thread Jeff King
On Thu, Aug 16, 2018 at 06:41:38PM -0400, Jeff King wrote: > - we should avoid anyone who is affiliated with a company that already >has a member on the committee. So nobody from Google and nobody from >GitHub. I would extend that to Microsoft, too, given a certain >

Re: Antw: Re: non-smooth progress indication for git fsck and git gc

2018-08-20 Thread Jeff King
On Mon, Aug 20, 2018 at 10:57:13AM +0200, Ævar Arnfjörð Bjarmason wrote: > > That seems to apply. BTW: Is there a way go get some repository statistics > > like a histogram of object sizes (or whatever that might be useful to help > > making decisions)? > > The git-sizer program is really

Re: [PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects

2018-08-20 Thread Jeff King
On Mon, Aug 20, 2018 at 02:03:53PM -0700, Junio C Hamano wrote: > > So taking all of those options into account, what I ended up > > with is a separate list of "external bases" that are not > > part of the main packing list. Each delta entry that points > > to an external base has a single-bit

Re: [PATCH] t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test

2018-08-19 Thread Jeff King
On Sun, Aug 19, 2018 at 11:37:59PM +0200, Andrei Rybak wrote: > On 19/08/18 22:32, Jeff King wrote: > > On Sun, Aug 19, 2018 at 07:50:42PM +0200, Andrei Rybak wrote: > > > >> 1. Check both files at the same time (combination with Gábor's > >>

Re: git-bug: Distributed bug tracker embedded in git

2018-08-19 Thread Jeff King
On Sat, Aug 18, 2018 at 03:08:21PM -0700, Jonathan Nieder wrote: > > So we don't get to say "you never asked us about git-annex, we're using > > that name now" without considering how widely used it is. It's us who > > decided to expose the API of seamlessly integrating 3rd party tools. > > I

Re: [PATCH] t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test

2018-08-19 Thread Jeff King
On Sun, Aug 19, 2018 at 07:50:42PM +0200, Andrei Rybak wrote: > > I actually think the above gives way too confusing output, when the > > actual output is empty and we are expecting some output. > > > > The tester wants to hear from test_cmp "your 'git cmd' produced some > > output when we are

Re: Git Project Leadership Committee

2018-08-18 Thread Jeff King
On Sat, Aug 18, 2018 at 07:32:38PM -0700, Elijah Newren wrote: > On Thu, Aug 16, 2018 at 3:43 PM Jeff King wrote: > > - we should avoid anyone who is affiliated with a company that already > >has a member on the committee. So nobody from Google and nobody from > >G

Re: Re* [PATCH v7 1/1] sideband: highlight keywords in remote sideband output

2018-08-18 Thread Jeff King
On Sat, Aug 18, 2018 at 09:16:28AM -0700, Junio C Hamano wrote: > -- >8 -- > Subject: [PATCH] sideband: do not read beyond the end of input > > The caller of maybe_colorize_sideband() gives a counted buffer > , but the callee checked src[] as if it were a NUL terminated > buffer. If src[] had

Re: [RFC/PATCH] drop vcs-svn experiment

2018-08-18 Thread Jeff King
On Fri, Aug 17, 2018 at 10:26:05PM -0700, Jonathan Nieder wrote: > > We also ship contrib/svn-fe, which builds on the vcs-svn > > work. However, it does not seem to build out of the box for > > me, as the link step misses some required libraries for > > using libgit.a. > > What libraries do you

Re: [PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 03:57:18PM -0700, Stefan Beller wrote: > On Fri, Aug 17, 2018 at 2:06 PM Jeff King wrote: > > > > When we serve a fetch, we pass the "wants" and "haves" from > > the fetch negotiation to pack-objects. That tells us not > >

Re: [PATCH 5/6] pack-bitmap: save "have" bitmap from walk

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 03:39:29PM -0700, Stefan Beller wrote: > > diff --git a/pack-bitmap.h b/pack-bitmap.h > > index 4555907dee..02a60ce670 100644 > > --- a/pack-bitmap.h > > +++ b/pack-bitmap.h > > @@ -50,6 +50,13 @@ int rebuild_existing_bitmaps(struct bitmap_index *, > > struct packing_data

[PATCH 6/6] pack-objects: reuse on-disk deltas for thin "have" objects

2018-08-17 Thread Jeff King
it flag to do so; we have a little breathing room in the bitfield section of object_entry. This lets us limit the change primarily to the oe_delta() and oe_set_delta_ext() functions. And as a bonus, most of the rest of the code does not consider these dummy entries at all, saving both runtime CPU and cod

[PATCH 5/6] pack-bitmap: save "have" bitmap from walk

2018-08-17 Thread Jeff King
ends up being fixed and adjust this interface accordingly. Signed-off-by: Jeff King --- Funny story: the earlier version of this series called it bitmap_have(). That caused a bug later when somebody tried to build on it, thinking it was "does the bitmap have this object in the result". Oops. Henc

[PATCH 4/6] t/perf: add perf tests for fetches from a bitmapped server

2018-08-17 Thread Jeff King
clock time for the operation (though the script does not do this automatically). Signed-off-by: Jeff King --- t/perf/p5311-pack-bitmaps-fetch.sh | 45 ++ 1 file changed, 45 insertions(+) create mode 100755 t/perf/p5311-pack-bitmaps-fetch.sh diff --git a/t/perf/

[PATCH 3/6] t/perf: add infrastructure for measuring sizes

2018-08-17 Thread Jeff King
ric output from the test and shows it in the aggregated output (with pretty printing and relative size comparison). Signed-off-by: Jeff King --- t/perf/README | 25 ++ t/perf/aggregate.perl | 48 ++- t/perf/perf-lib.

[PATCH 2/6] t/perf: factor out percent calculations

2018-08-17 Thread Jeff King
This will let us reuse the code when we add new values to aggregate besides times. Signed-off-by: Jeff King --- t/perf/aggregate.perl | 21 - 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl index bc865160e7

[PATCH 1/6] t/perf: factor boilerplate out of test_perf

2018-08-17 Thread Jeff King
About half of test_perf() is boilerplate preparing to run _any_ test, and the other half is specifically running a timing test. Let's split it into two functions, so that we can reuse the boilerplate in future commits. Signed-off-by: Jeff King --- Best viewed with "-w". t/perf/perf-l

[PATCH 0/6] reuse on-disk deltas for fetches with bitmaps

2018-08-17 Thread Jeff King
This series more aggressively reuses on-disk deltas to serve fetches when reachability bitmaps tell us a more complete picture of what the client has. That saves server CPU and results in smaller packs. See the final patch for numbers and more discussion. It's a resurrection of this very old

[RFC/PATCH] drop vcs-svn experiment

2018-08-17 Thread Jeff King
Let's drop it and let anybody who wants to pick it up do so by resurrecting it from the git history. [1] https://public-inbox.org/git/calkwk0mphzkfzfkkpzkfaus3yvc9nfydbfnt+5jqyvkipk3...@mail.gmail.com/ Signed-off-by: Jeff King --- Of course, I could be completely wrong about people using this.

Re: [PATCH] refactor various if (x) FREE_AND_NULL(x) to just FREE_AND_NULL(x)

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 01:33:08PM -0400, Jeff King wrote: > > And revised even more to also spell "E" as "E != NULL" (and "!E" as > > "E == NULL"), which seems to make a difference, which is even more > > sad. I do not want to wonder if

Re: [PATCH] refactor various if (x) FREE_AND_NULL(x) to just FREE_AND_NULL(x)

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 01:39:51PM -0400, Jeff King wrote: > > I wonder if there is a way to "relax" a pattern where these semantically > > equivalent cases can all be covered automatically. I don't know enough > > about the tool to say. > > Hmm. They seem to

Re: [PATCH] refactor various if (x) FREE_AND_NULL(x) to just FREE_AND_NULL(x)

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 10:07:36AM -0700, Junio C Hamano wrote: > Junio C Hamano writes: > > > It is a bit sad that > > > > - if (E) > > FREE_AND_NULL(E); > > > > is not sufficient to catch it. Shouldn't we be doing the same for > > regular free(E) as well? IOW, like the attached

Re: Syncing HEAD

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 09:28:59AM -0700, Junio C Hamano wrote: > Jeff King writes: > > > So all of this really implies to me that you want to be able to say > > "take this symref on the other side and update this one on the local > > side". I.e., some w

Re: [PATCH/RFC] commit: add short option for --amend

2018-08-17 Thread Jeff King
On Fri, Aug 17, 2018 at 04:33:30PM +0200, Duy Nguyen wrote: > On Fri, Aug 17, 2018 at 8:47 AM Jonathan Nieder wrote: > > > > Nguyễn Thái Ngọc Duy wrote: > > > > > --- a/builtin/commit.c > > > +++ b/builtin/commit.c > > > @@ -1489,7 +1489,7 @@ int cmd_commit(int argc, const char **argv, const >

Re: Syncing HEAD

2018-08-16 Thread Jeff King
On Wed, Aug 15, 2018 at 07:49:25AM +0200, Christian Couder wrote: > > And so here the convention is simpler, because we're talking about the > > main HEAD. But we still have know if you want to do that, and not update > > some refs/remotes/ symref in a bare repo. > > We could maybe look at the

Git Project Leadership Committee

2018-08-16 Thread Jeff King
This is a followup to the issue I raised back in March[1], which is that our project committee at Software Freedom Conservancy has two members, but is required by the charter to have at least three. There wasn't any substantive discussion in response to that email or at the contributor summit. I

Re: non-smooth progress indication for git fsck and git gc

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 04:55:56PM -0400, Jeff King wrote: > > * We spend the majority of the ~30s on this: > > > > https://github.com/git/git/blob/63749b2dea5d1501ff85bab7b8a7f64911d21dea/pack-check.c#L70-L79 > > This is hashing the actual packfile. This i

Re: non-smooth progress indication for git fsck and git gc

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 10:35:53PM +0200, Ævar Arnfjörð Bjarmason wrote: > This is all interesting, but I think unrelated to what Ulrich is talking > about. Quote: > > Between the two phases of "git fsck" (checking directories and > checking objects) there was a break of several seconds

Re: non-smooth progress indication for git fsck and git gc

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 11:57:14AM -0400, Jeff King wrote: > The only way to solve that is to count bytes. We don't have a total byte > count in most cases, and it wouldn't always make sense (e.g., the > "Compressing objects" meter can show the same issue, but it's not reall

Re: [PATCH 0/7] speeding up cat-file by reordering object access

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 11:52:13AM -0700, Junio C Hamano wrote: > Jeff King writes: > > > I think that makes sense. We already see duplicates from > > for_each_packed_object() when they're in multiple packs, and callers > > just need to be ready to deal with it (and

Re: [PATCH/RFC] commit: add short option for --amend

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 08:31:17PM +0200, Nguyễn Thái Ngọc Duy wrote: > I just realized how often I type "git ci --amend". Looking back at my > ~/.bash_history (only 10k lines) this is the second most often git > command I type which may justify a short option for it (assuming that > other people

Re: [PATCHv4 0/6] Add missing includes and forward declares

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 01:19:17AM +0100, Ramsay Jones wrote: > As a quick ("just before bedtime") exercise, I tried adding > a Makefile target to perform a similar check. The result is > given below, but I haven't had time to look too closely at > the results: The resulting patch doesn't look

Re: [PATCH 0/7] speeding up cat-file by reordering object access

2018-08-16 Thread Jeff King
On Wed, Aug 15, 2018 at 10:05:04AM -0400, Derrick Stolee wrote: > One thing that I realized while reading it is that the multi-pack-index is > not integrated into the for_each_packed_object method. I was already going > to work on some cleanups in that area [1][2]. > > When using the new flag

Re: [PATCH 4/7] for_each_packed_object: support iterating in pack-order

2018-08-16 Thread Jeff King
On Wed, Aug 15, 2018 at 09:28:44AM -0400, Derrick Stolee wrote: > On 8/10/2018 7:15 PM, Jeff King wrote: > > diff --git a/commit-graph.c b/commit-graph.c > > index b0a55ad128..69a0d1c203 100644 > > --- a/commit-graph.c > > +++ b/commit-graph.c > > @@ -730,7 +730,7

Re: non-smooth progress indication for git fsck and git gc

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 05:18:51PM +0200, Duy Nguyen wrote: > > During "git gc" the writing objects phase did not update for some > > seconds, but then the percentage counter jumped like from 15% to > > 42%. > [...] > > Is it possible to make this repository public? You can also use "git >

Re: non-smooth progress indication for git fsck and git gc

2018-08-16 Thread Jeff King
On Thu, Aug 16, 2018 at 08:54:25AM +0200, Ulrich Windl wrote: > I'd like to point out some minor issue observed while processing some > 5-object repository with many binary objects, but most are rather > small: > > Between the two phases of "git fsck" (checking directories and > checking

Re: [PATCH 0/9] Add missing includes and forward declares

2018-08-14 Thread Jeff King
ust be git-compat-util.h or an equivalent". This isn't actually a problem, as test-tool.h doesn't define anything tricky, but we should probably follow our own rule. Rather than change them all, let's instead make test-tool.h one of those equivalents, just like we do for builtin.h (which many of

Re: [PATCH v4 6/6] list-objects-filter: implement filter tree:0

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 04:55:34PM -0700, Matthew DeVore wrote: > > - it's expensive to compute, because you have to actually walk all of > > the possible commits and trees that could refer to it. This > > prohibits a lot of other optimizations like reachability bitmaps > > (though

Re: t5570-git-daemon fails with SIGPIPE on OSX

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 06:32:47PM -0400, Jeff King wrote: > I suspect the (largely untested) patch below would make your test > problems go away. Or instead, we could simply add sigpipe=ok to the > test_must_fail invocation, but I agree with you that the current > behavior on OS X

Re: t5570-git-daemon fails with SIGPIPE on OSX

2018-08-14 Thread Jeff King
On Mon, Aug 06, 2018 at 05:11:13PM +0200, SZEDER Gábor wrote: > - 'git upload-pack' receives the request, parses the want line, > notices the corrupt pack, responds with an 'ERR upload-pack: not > our ref' pkt-line, and die()s right away. > > - 'git fetch' finally approaches the end

Re: [PATCH] t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 01:47:21PM +0200, SZEDER Gábor wrote: > The test 'pack-objects to file can use bitmap' added in 645c432d61 > (pack-objects: use reachability bitmap index when generating > non-stdout pack, 2016-09-10) is silently buggy and doesn't check what > it's supposed to. > > In

Re: Syncing HEAD

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 05:06:16PM -0400, Jeff King wrote: > On Tue, Aug 14, 2018 at 10:09:37PM +0200, Christian Couder wrote: > > > When cloning with --mirror, the clone gets its HEAD initialized with > > the value HEAD has in its origin remote. After that if HEAD cha

Re: Syncing HEAD

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 10:09:37PM +0200, Christian Couder wrote: > When cloning with --mirror, the clone gets its HEAD initialized with > the value HEAD has in its origin remote. After that if HEAD changes in > origin there is no simple way to sync HEAD at the same time as the > refs are synced.

Re: Measuring Community Involvement (was Re: Contributor Summit planning)

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 12:47:59PM -0700, Stefan Beller wrote: > On Tue, Aug 14, 2018 at 12:36 PM Jeff King wrote: > > > Thanks, it was nice to see a more comprehensive list in one spot. > > > > It would be neat to have a tool that presents all of these > > automa

Re: [PATCH v4 6/6] list-objects-filter: implement filter tree:0

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 10:28:13AM -0700, Matthew DeVore wrote: > The name "tree:0" allows later filtering based on depth, i.e. "tree:1" > would filter out all but the root tree and blobs. In order to avoid > confusion between 0 and capital O, the documentation was worded in a > somewhat

Re: [PATCH v4 2/5] unpack-trees: add performance tracing

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 08:51:41PM +0200, Duy Nguyen wrote: > > But AFAICT these series are all about putting the sampling points into the > > code base, so formatting would be orthogonal to it? > > It's not just sampling points. There's things like index id being > shown in the message for

Re: [PATCH 3/4] cat-file: use a single strbuf for all output

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 09:30:57PM +0200, René Scharfe wrote: > > -static void batch_object_write(const char *obj_name, struct batch_options > > *opt, > > +static void batch_object_write(const char *obj_name, > > + struct strbuf *scratch, > > +

Re: Measuring Community Involvement (was Re: Contributor Summit planning)

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 01:43:38PM -0400, Derrick Stolee wrote: > On 8/13/2018 5:54 PM, Jeff King wrote: > > So I try not to think too hard on metrics, and just use them to get a > > rough view on who is active. > > I've been very interested in measuring

Re: [PATCH] mingw: enable atomic O_APPEND

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 08:29:04PM +0200, Johannes Sixt wrote: > Am 14.08.2018 um 00:37 schrieb Jeff King: > > And then you can do something like: > > > >for size in 4097 8193 16385 32769 65537 131073 262145 524289 1048577; do > > >out ;# clean up from

Re: [PATCH 2/2] submodule: munge paths to submodule git directories

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 11:04:06AM -0700, Brandon Williams wrote: > > I think this backwards-compatibility is necessary to avoid pain. But > > until it goes away, I don't think this is helping the vulnerability from > > 0383bbb901. Because there the issue was that the submodule name pointed > >

[PATCH 4/4] for_each_*_object: move declarations to object-store.h

2018-08-14 Thread Jeff King
they also make sense in object-store.h. Signed-off-by: Jeff King --- This patch also happens to be a nice showcase for --color-moved. builtin/prune-packed.c | 1 + cache.h| 75 --- object-store.h | 90 ++

[PATCH 3/4] cat-file: use a single strbuf for all output

2018-08-14 Thread Jeff King
tra parameter to the callstack. That's more verbose, but it makes it a bit more obvious what is going on, which in turn makes it easy to see where we need to be releasing the string in the caller (right after the loop which uses it in each case). Based-on-a-patch-by: René Scharfe Signed-off-b

[PATCH 2/4] cat-file: split batch "buf" into two variables

2018-08-14 Thread Jeff King
n we start using the "output" buffer for more expansions. Signed-off-by: Jeff King --- René, in the patch you sent earlier, I noticed that for the non-batch-all-objects case we use the same strbuf for input and output. That'd probably be OK most of the time (the first thing we do is resolve

[PATCH 1/4] cat-file: use oidset check-and-insert

2018-08-14 Thread Jeff King
is simpler to read, too. Signed-off-by: Jeff King --- builtin/cat-file.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 45992c9be9..04b5cda191 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -443,9 +443,8 @@ static int

[PATCH 0/4] finishing touches on jk/for-each-object-iteration

2018-08-14 Thread Jeff King
On Mon, Aug 13, 2018 at 11:45:06AM -0700, Jonathan Tan wrote: > > [1/7]: for_each_*_object: store flag definitions in a single location > > [2/7]: for_each_*_object: take flag arguments as enum > > [3/7]: for_each_*_object: give more comprehensive docstrings > > [4/7]:

Re: [PATCH v6 11/21] range-diff: add tests

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 11:03:10AM -0400, Jeff King wrote: > > Hmm. I hoped that my commit message was enough to indicate that while he > > is the author, I assembled this. Maybe I should move him to the footer, as > > an Original-Authored-By:? > > I think the "Autho

Re: [PATCH v6 11/21] range-diff: add tests

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 04:53:51PM +0200, Johannes Schindelin wrote: > > > These are essentially lifted from https://github.com/trast/tbdiff, with > > > light touch-ups to account for the command now being named `git > > > range-diff`. > [...] > > Just noticed while reading the whole series again

Re: [PATCH] mingw: enable atomic O_APPEND

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 03:47:54PM +0200, Ævar Arnfjörð Bjarmason wrote: > The relevant POSIX docs are here: > http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html > > Write requests of {PIPE_BUF} bytes or less shall not be interleaved > with data from other processes

Re: Contributor Summit planning

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 04:30:09PM +0200, Duy Nguyen wrote: > > I frequently using "git shortlog -ns" to see who is active (especially > > coupled with "--since=". > > > > I also use "--no-merges", because it makes me look a lot better when > > compared relatively to Junio. :) > > --no-merges

Re: Contributor Summit planning

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 06:31:50AM +0200, Christian Couder wrote: > > We have been kicking around the thought of reviving the GitTogethers > > like back in the olden days (I only know them from hearsay), in > > Mountain View or Sunnyvale at the Google Campus, but we have not yet > > spent enough

Re: Contributor Summit planning

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 04:06:23PM +0200, Ævar Arnfjörð Bjarmason wrote: > I very much regret sending that 'git log' command without some further > explanation. > > It was only meant as a *very* rough shortlist of people in the context > of a discussion of why some active contributors don't come

Re: Contributor Summit planning

2018-08-14 Thread Jeff King
On Tue, Aug 14, 2018 at 09:25:41AM -0400, Randall S. Becker wrote: > Unrelated directly to above, I noticed that I actually showed up on > the list for 2018 based on git log anyway. Does this mean I'd be > welcome? Personally, it's actually easier to get approval to travel to > Brussels now than

Re: [PATCH 2/2] fsck: use oidset for skiplist

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 09:58:42PM -0400, Jeff King wrote: > > builtin/cat-file.c | 93 +++--- > > 1 file changed, 88 insertions(+), 5 deletions(-) > > By the way, your patch seemed damaged (wouldn't apply, and "am -3"

Re: [PATCH 2/2] fsck: use oidset for skiplist

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 07:15:19PM +0200, René Scharfe wrote: > Getting sidetracked here, but the following patch helps both sides a bit: > > -- >8 -- > Subject: [PATCH] cat-file: reuse strbuf in batch_object_write() > > Avoid allocating and then releasing memory for constructing the output >

Re: [PATCH 2/2] fsck: use oidset for skiplist

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 07:15:23PM +0200, René Scharfe wrote: > Am 11.08.2018 um 22:59 schrieb René Scharfe: > > If the current oidset implementation is so bad, why not replace it with > > one based on oid_array? ;-) > > > > Intuitively I'd try a hashmap with no payload and open addressing via >

Re: [PATCH] mingw: enable atomic O_APPEND

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 11:22:10PM +0200, Ævar Arnfjörð Bjarmason wrote: > > O_APPEND is POSIX and means race-free append. If you mark some call > > sites with O_APPEND, then that must be the ones that need race-free > > append. Hence, you would have to go the other route: Mark those call > >

Re: Contributor Summit planning

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 02:19:07PM -0700, Stefan Beller wrote: > However the mailing list participation numbers there doesn't really > help me: > > ~/git-ml$ git shortlog --since 2017 -sne > 3721 Junio C Hamano > 2166 Stefan Beller > 2071 Jeff King > > and

Re: [PATCH v4 2/5] unpack-trees: add performance tracing

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 09:52:41PM +0200, Duy Nguyen wrote: > I don't think I have really fully mastered 'perf'. In this case for > example, I don't think the default event 'cycles' is the right one > because we are hit hard by I/O as well. I think at least I now have an > excuse to try that

Re: Contributor Summit planning

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 01:41:51PM -0700, Stefan Beller wrote: > > Oh, using "git shortlog" might be also simpler ;-) > > I guess you'd need to memorize a different set of flags for that > as without -s it would be harder to parse than the oneliner above. I frequently using "git shortlog -ns"

Re: Contributor Summit planning

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 08:49:33PM +0200, Ævar Arnfjörð Bjarmason wrote: > * I think we should tread carefully when it comes to say some form of >remote A/V participation open to the Internet. > >It was fine to have Dscho on a chair, but it would be quite different >if this were say

Re: [PATCH v4 2/5] unpack-trees: add performance tracing

2018-08-13 Thread Jeff King
On Sun, Aug 12, 2018 at 10:15:48AM +0200, Nguyễn Thái Ngọc Duy wrote: > We're going to optimize unpack_trees() a bit in the following > patches. Let's add some tracing to measure how long it takes before > and after. This is the baseline ("git checkout -" on webkit.git, 275k > files on worktree)

Re: [PATCH v4 1/7] Add delta-islands.{c,h}

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 05:33:59AM +0200, Christian Couder wrote: > >> + memcpy(_core, oid->hash, sizeof(uint64_t)); > >> + rl->hash += sha_core; > > > > Hmm, so the first 64-bits of the oid of each ref that is part of > > this island is added together as a 'hash' for the island. And this

Re: [PATCH v4 1/7] Add delta-islands.{c,h}

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 01:17:18PM +0100, Ramsay Jones wrote: > >>> +struct island_bitmap { > >>> + uint32_t refcount; > >>> + uint32_t bits[]; > >> > >> Use FLEX_ARRAY here? We are slowly moving toward requiring > >> certain C99 features, but I can't remember a flex array > >>

Re: [RFC] implement branch.sort config option

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 11:56:58AM +0200, Samuel Maftoul wrote: > Currently, you can: > > git tag --sort=$sorting_key > > You can also do this on branches: > > git branch --sort=$sorting_key > > For tags, you can also configure it with a config key: > > git config tag.sort $sorting_key > >

Re: Contributor Summit planning

2018-08-13 Thread Jeff King
On Mon, Aug 13, 2018 at 12:58:54PM -0400, Derrick Stolee wrote: > I would be up for two meetings a year. I would expect that the variety of > locations would allow a larger set of contributors to make at least one > meeting a year. This may come at a cost of a smaller group in each summit. Yeah,

Contributor Summit planning

2018-08-13 Thread Jeff King
For the past several years, we've held a Git Contributor Summit as part of the Git Merge conference. I'd like to get opinions from the community to help plan future installments. Any feedback or opinion is welcome, but some obvious things to think about: - where, when, and how often? Plans

Re: function get_delta_base() is a file-local symbol

2018-08-11 Thread Jeff King
On Sun, Aug 12, 2018 at 01:30:02AM +0100, Ramsay Jones wrote: > Hi Christian, > > My static-check.pl script has pinged me about the get_delta_base() > symbol from packfile.[co]. The first patch from your 'cc/delta-islands' > branch exports this symbol, saying that it will soon be called from >

Re: [PATCH 1/2] fsck: use strbuf_getline() to read skiplist file

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 05:39:27PM +0200, René Scharfe wrote: > The char array named "buffer" is unlikely to contain a NUL character, so > printing its contents using %s in a die() format is unsafe. Clang's > ASan reports running over the end of buffer in the recently added > skiplist tests in

Re: [PATCH 1/9] Add missing includes and forward declares

2018-08-11 Thread Jeff King
On Fri, Aug 10, 2018 at 09:32:10PM -0700, Elijah Newren wrote: > diff --git a/argv-array.h b/argv-array.h > index a39ba43f57..c46238784c 100644 > --- a/argv-array.h > +++ b/argv-array.h > @@ -1,6 +1,8 @@ > #ifndef ARGV_ARRAY_H > #define ARGV_ARRAY_H > > +#include "git-compat-util.h" /* for

Re: [PATCH 0/9] Add missing includes and forward declares

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 01:59:50AM -0700, Elijah Newren wrote: > The part of my story you snipped in the ellipsis is kind of important, > though: "...and decided to determine which header files were missing > their own necessary #include's and forward declarations." The way I > did so was making

Re: [PATCH 2/2] fsck: use oidset for skiplist

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 01:02:48PM -0400, Jeff King wrote: > - we could probably improve the speed of oidset. Two things I notice > about its implementation: > > - it has to malloc for each entry, which I suspect is the main > bottleneck. We could probably pool-

Re: [PATCH 2/2] fsck: use oidset for skiplist

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 05:47:56PM +0200, René Scharfe wrote: > Object IDs to skip are stored in a shared static oid_array. Lookups do > a binary search on the sorted array. The code checks if the object IDs > are already in the correct order while loading and skips sorting in that > case. > >

Re: Help with "fatal: unable to read ...." error during GC?

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 04:38:00PM +0200, Duy Nguyen wrote: > On Sat, Aug 11, 2018 at 4:25 PM Jeff King wrote: > > Responding myself and adding Duy to the cc to increase visibility among > > worktree experts. :) > > I do silently watch this thread (and yes I still have to

Re: Help with "fatal: unable to read ...." error during GC?

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 10:23:41AM -0400, Jeff King wrote: > > I do still have these warnings and no amount of git gc/git fsck/etc. > > has reduced them in any way: > > > > $ git gc > > warning: reflog of 'HEAD' references pruned commits > > warning: reflog

Re: Help with "fatal: unable to read ...." error during GC?

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 08:13:17AM -0400, Paul Smith wrote: > I rebuilt Git 2.18.0 without optimization to try to get more debug > information. Unfortunately I didn't think to create a backup of my > problematic .git directory. > > When I ran the above command under the debugger using the

Re: [PATCH v3 2/8] Add delta-islands.{c,h}

2018-08-11 Thread Jeff King
On Sat, Aug 11, 2018 at 12:32:32PM +0200, Christian Couder wrote: > Ok, I have made the following changes in the branch I will send next. > > diff --git a/delta-islands.c b/delta-islands.c > index 92137f2eca..22e4360810 100644 > --- a/delta-islands.c > +++ b/delta-islands.c > @@ -322,8 +322,7 @@

Re: [PATCH 1/7] for_each_*_object: store flag definitions in a single location

2018-08-10 Thread Jeff King
On Fri, Aug 10, 2018 at 04:39:14PM -0700, Stefan Beller wrote: > > IMHO the whole for_each_*_object() interface should go in there (it even > > has packed_git defined there already!). I think I'd still just as soon > > do it on top of this series, but it might not be too bad to do as part > > of

Re: [PATCH 1/7] for_each_*_object: store flag definitions in a single location

2018-08-10 Thread Jeff King
On Fri, Aug 10, 2018 at 07:31:33PM -0400, Jeff King wrote: > On Fri, Aug 10, 2018 at 04:27:25PM -0700, Stefan Beller wrote: > > > > cache.h| 13 - > > > packfile.h | 8 ++-- > > > 2 files changed, 14 insertions(+), 7 deletions(

Re: [PATCH 1/7] for_each_*_object: store flag definitions in a single location

2018-08-10 Thread Jeff King
On Fri, Aug 10, 2018 at 04:27:25PM -0700, Stefan Beller wrote: > > cache.h| 13 - > > packfile.h | 8 ++-- > > 2 files changed, 14 insertions(+), 7 deletions(-) > > rubs me the wrong way. ;-) > > cache.h is such a misnomer of a name, and a kitchen sink > of a file in the

[PATCH 7/7] cat-file: support "unordered" output for --batch-all-objects

2018-08-10 Thread Jeff King
e without having to add another compatibility option. Signed-off-by: Jeff King --- Documentation/git-cat-file.txt | 10 ++ builtin/cat-file.c | 56 +++--- t/t1006-cat-file.sh| 11 +++ 3 files changed, 72 insertions(+), 5 deletions

[PATCH 6/7] cat-file: rename batch_{loose,packed}_object callbacks

2018-08-10 Thread Jeff King
We're not really doing the batch-show operation in these callbacks, but just collecting the set of objects. That distinction will become more important in a future patch, so let's rename them now to avoid cluttering that diff. Signed-off-by: Jeff King --- builtin/cat-file.c | 18

[PATCH 5/7] t1006: test cat-file --batch-all-objects with duplicates

2018-08-10 Thread Jeff King
(because we'd show their loose counterparts). Signed-off-by: Jeff King --- t/t1006-cat-file.sh | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 13dd510b2e..4fb5e098f2 100755 --- a/t/t1006-cat-file.sh +++ b/t/t100

[PATCH 4/7] for_each_packed_object: support iterating in pack-order

2018-08-10 Thread Jeff King
it an optional flag. You can't actually do any interesting timings yet, as it's not plumbed through to any user-facing tools like cat-file. That will come in a later patch. Signed-off-by: Jeff King --- cache.h| 5 + commit-graph.c | 2 +- packfile.c | 21 --

[PATCH 3/7] for_each_*_object: give more comprehensive docstrings

2018-08-10 Thread Jeff King
) - clarify that reachability doesn't matter here; this is all accessible objects - what ordering/uniqueness guarantees we give - how pack-specific flags are handled for the loose case Signed-off-by: Jeff King --- cache.h| 8 +--- packfile.h | 12 2 files changed, 13 insertions

[PATCH 2/7] for_each_*_object: take flag arguments as enum

2018-08-10 Thread Jeff King
eclarations, per our modern coding style. Signed-off-by: Jeff King --- cache.h | 3 ++- packfile.c | 3 ++- packfile.h | 5 +++-- sha1-file.c | 3 ++- 4 files changed, 9 insertions(+), 5 deletions(-) diff --git a/cache.h b/cache.h index 4187238ecf..9e02fc494e 100644 --- a/cache.h +++ b/cache.

[PATCH 1/7] for_each_*_object: store flag definitions in a single location

2018-08-10 Thread Jeff King
These flags were split between cache.h and packfile.h, because some of the flags apply only to packs. However, they share a single numeric namespace, since both are respected for the packed variant. Let's make sure they're defined together so that nobody accidentally adds a new flag in one

[PATCH 0/7] speeding up cat-file by reordering object access

2018-08-10 Thread Jeff King
This series is meant to replace the RFC discussion in: https://public-inbox.org/git/20180808231210.242120-1-jonathanta...@google.com/ and https://public-inbox.org/git/20180808155045.gb1...@sigill.intra.peff.net/ The general idea is that accessing objects in packfile order is way kinder to

Re: [RFC PATCH] packfile: iterate packed objects in pack order

2018-08-10 Thread Jeff King
On Thu, Aug 09, 2018 at 03:03:24PM -0700, Jonathan Tan wrote: > On Wed, Aug 8, 2018 at 4:25 PM, Jeff King wrote: > > Even if you just use the oid to do a separate lookup in the object > > database, there's still a benefit in accessing the objects in pack > > order. >

Re: [PATCH 0/4] t5552: fix flakiness by introducing proper locking for GIT_TRACE

2018-08-10 Thread Jeff King
On Fri, Aug 10, 2018 at 11:34:59AM -0700, Junio C Hamano wrote: > Johannes Sixt writes: > > > As this buglet looks like a recurring theme, and a proper fix is > > preferable over repeated work-arounds. To me it looks like we need > > some sort of locking on Windows. Unless your friends at

Re: Bug? Git won't apply a split hunk that went through a text editor

2018-08-10 Thread Jeff King
On Thu, Aug 09, 2018 at 08:17:36PM -0700, Philip White wrote: > I’d like to report what I suspect is a bug in Git, tested in 2.18 and > 2.14. (I’d be delighted to be corrected if it is my own > misunderstanding.) I’m reporting it here based on guidance from > https://git-scm.com/community. > > I

<    4   5   6   7   8   9   10   11   12   13   >