Re: [PATCH v2 0/4] git-commit-graph.txt: various cleanups

2018-09-27 Thread Derrick Stolee
erns and looks good to me. Reviewed-by: Derrick Stolee

Re: Git Evolve

2018-10-01 Thread Derrick Stolee
On 9/29/2018 7:00 PM, Stefan Xenos wrote: Hello, List! Hello! Welcome. I'm interested in porting something like Mercurial's evolve command to Git. I'll be following up with a formal proposal shortly, but before I do I thought I'd introduce myself to the list and find out if anyone else is int

Re: [PATCH] read-cache: fix division by zero core-dump

2018-10-01 Thread Derrick Stolee
On 9/28/2018 11:31 AM, Ramsay Jones wrote: Also, this is not the first time some multi-threaded code in git has 'failed' by assuming more than one cpu, so ... I wonder if this is a good time to create a GIT_TEST_CPU_COUNT variable so we can mock out single-processor environments instead of rely

Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear

2018-10-01 Thread Derrick Stolee
On 10/1/2018 3:16 PM, René Scharfe wrote: Am 28.06.2018 um 14:31 schrieb Derrick Stolee via GitGitGadget: diff --git a/commit-reach.c b/commit-reach.c index c58e50fbb..ac132c8e4 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -513,65 +513,88 @@ int commit_contains(struct ref_filter *filter

Re: [PATCH] more oideq/hasheq conversions

2018-10-02 Thread Derrick Stolee
it's a reasonable time to apply it. There are a few lingering cases in pu, so another option is to wait a week or two and see if they get merged. These conversions look good to me! Reviewed-by: Derrick Stolee

Re: [PATCH 1/2] commit-graph: clean up leaked memory during write

2018-10-03 Thread Derrick Stolee
On 10/2/2018 6:44 PM, Stefan Beller wrote: My preference is to avoid them in the name of simplicity. If you're using "make SANITIZE=leak test" to check for leaks, it will skip these cases. If you're using valgrind, I think these may be reported as "reachable". But that number already isn't useful

Re: We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-03 Thread Derrick Stolee
On 10/3/2018 9:36 AM, SZEDER Gábor wrote: On Wed, Oct 03, 2018 at 03:23:57PM +0200, Ævar Arnfjörð Bjarmason wrote: Don't have time to patch this now, but thought I'd send a note / RFC about this. Now that we have the commit graph it's nice to be able to set e.g. core.commitGraph=true & gc.write

Re: [PATCH 0/2] commit-graph: more leak fixes

2018-10-03 Thread Derrick Stolee
On 10/3/2018 11:36 AM, Martin Ågren wrote: Hi Derrick, These two patches on top of yours make the test suite (i.e., the subset of it that I run) leak-free with respect to builtin/commit-graph.c and commit-graph.c. Thanks! The first could be squashed into your patch 1/2. It touches the same f

Re: We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-03 Thread Derrick Stolee
On 10/3/2018 2:51 PM, Jeff King wrote: On Wed, Oct 03, 2018 at 08:47:11PM +0200, Ævar Arnfjörð Bjarmason wrote: On Wed, Oct 03 2018, Stefan Beller wrote: So we wouldn't be spending 5 minutes repacking linux.git right after cloning it, just ~10s generating the commit graph, and the same would

Re: [RFC PATCH] We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-05 Thread Derrick Stolee
On 10/4/2018 5:42 PM, Ævar Arnfjörð Bjarmason wrote: I don't have time to polish this up for submission now, but here's a WIP patch that implements this, highlights: * There's a gc.clone.autoDetach=false default setting which overrides gc.autoDetach if 'git gc --auto' is run via git-clone

Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear

2018-10-05 Thread Derrick Stolee
On 10/4/2018 6:59 PM, René Scharfe wrote: Am 01.10.2018 um 22:37 schrieb René Scharfe: Am 01.10.2018 um 21:26 schrieb Derrick Stolee: Good catch! I'm disappointed that we couldn't use type-checking here, as it is quite difficult to discover that the types are wrong here. Generic

Re: [RFC PATCH] We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-05 Thread Derrick Stolee
On 10/5/2018 9:05 AM, Ævar Arnfjörð Bjarmason wrote: On Fri, Oct 05 2018, Derrick Stolee wrote: On 10/4/2018 5:42 PM, Ævar Arnfjörð Bjarmason wrote: I don't have time to polish this up for submission now, but here's a WIP patch that implements this, highlights: *

Re: [RFC PATCH] We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-05 Thread Derrick Stolee
On 10/5/2018 3:21 PM, Jeff King wrote: On Fri, Oct 05, 2018 at 09:45:47AM -0400, Derrick Stolee wrote: My misunderstanding was that your proposed change to gc computes the commit-graph in either of these two cases: (1) The auto-GC threshold is met. (2) There is no commit-graph file. And

Re: [RFC PATCH] We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-05 Thread Derrick Stolee
On 10/5/2018 3:47 PM, Jeff King wrote: On Fri, Oct 05, 2018 at 03:41:40PM -0400, Derrick Stolee wrote: So can we really just take (total_objects - commit_graph_objects) and compare it to some threshold? The commit-graph only stores the number of _commits_, not total objects. Oh, right, of

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-10-08 Thread Derrick Stolee
On 10/8/2018 9:43 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Aug 28 2018, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-10-08 Thread Derrick Stolee
On 10/8/2018 10:58 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Oct 08 2018, Derrick Stolee wrote: On 10/8/2018 9:43 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Aug 28 2018, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The commit-graph feature is tested in isolation by t5318

Re: We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-08 Thread Derrick Stolee
On 10/8/2018 12:41 PM, SZEDER Gábor wrote: On Wed, Oct 03, 2018 at 03:18:05PM -0400, Jeff King wrote: I'm still excited about the prospect of a bloom filter for paths which each commit touches. I think that's the next big frontier in getting things like "git log -- path" to a reasonable run-time

Re: [PATCH][Outreachy] remove all the inclusions of git-compat-util.h in header files

2018-10-08 Thread Derrick Stolee
On 10/8/2018 1:05 PM, Ananya Krishna Maram wrote: Hi All, Hello, Ananya! Welcome. I was searching through #leftovers and found this. https://public-inbox.org/git/cabpp-bgvvxcbzx44er6to-pusfen_6gnyj1u5cuon9deaa4...@mail.gmail.com/ This patch address the task discussed in the above link. The di

Re: We should add a "git gc --auto" after "git clone" due to commit graph

2018-10-08 Thread Derrick Stolee
On 10/8/2018 2:10 PM, SZEDER Gábor wrote: On Mon, Oct 08, 2018 at 12:57:34PM -0400, Derrick Stolee wrote: Nice! These numbers make sense to me, in terms of how many TREESAME queries we actually need to perform for such a query. Yeah... because you didn't notice that I deliberately ch

Bloom Filters (was Re: We should add a "git gc --auto" after "git clone" due to commit graph)

2018-10-09 Thread Derrick Stolee
(Changing title to reflect the new topic.) On 10/8/2018 11:08 PM, Jeff King wrote: On Mon, Oct 08, 2018 at 02:29:47PM -0400, Derrick Stolee wrote: There are two questions that I was hoping to answer by looking at your code: 1. How do you store your Bloom filter? Is it connected to the commit

Re: [PATCH 2/3] midx: close multi-pack-index on repack

2018-10-09 Thread Derrick Stolee
On 10/9/2018 5:10 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: diff --git a/builtin/repack.c b/builtin/repack.c index c6a7943d5c..7925bb976e 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -432,6 +432,10 @@ int cmd_repack(int argc, const char **argv,

Re: Bloom Filters (was Re: We should add a "git gc --auto" after "git clone" due to commit graph)

2018-10-09 Thread Derrick Stolee
On 10/9/2018 2:46 PM, Jeff King wrote: On Tue, Oct 09, 2018 at 09:48:20AM -0400, Derrick Stolee wrote: [I snipped all of the parts about bloom filters that seemed entirely reasonable to me ;) ] Imagine we have that list. Is a bloom filter still the best data structure for each commit? At

Re: [PATCH 0/4] Bloom filter experiment

2018-10-09 Thread Derrick Stolee
On 10/9/2018 3:34 PM, SZEDER Gábor wrote: To keep the ball rolling, here is my proof of concept in a somewhat cleaned-up form, with still plenty of rough edges. You can play around with it like this: $ GIT_USE_POC_BLOOM_FILTER=$((8*1024*1024*8)) git commit-graph write Computing commit gra

Re: [RFC PATCH 6/6] commit-reach: fix first-parent heuristic

2018-10-11 Thread Derrick Stolee
On 10/10/2018 9:50 PM, Jonathan Nieder wrote: Hi, Derrick Stolee wrote: commit-reach.c| 4 +++- t/t6600-test-reach.sh | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) I like this testing technique, and the test passes for me. Except: if I put CC = cc -m32

Re: What's cooking in git.git (Oct 2018, #01; Wed, 10)

2018-10-11 Thread Derrick Stolee
On 10/10/2018 1:43 AM, Junio C Hamano wrote: * ds/reachable-topo-order (2018-09-21) 7 commits - revision.c: refactor basic topo-order logic - revision.h: add whitespace in flag definitions - commit/revisions: bookkeeping before refactoring - revision.c: begin refactoring --topo-order log

Re: Bloom Filters

2018-10-11 Thread Derrick Stolee
On 10/9/2018 7:12 PM, Jeff King wrote: On Tue, Oct 09, 2018 at 05:14:50PM -0400, Jeff King wrote: Hmph. It really sounds like we could do better with a custom RLE solution. But that makes me feel like I'm missing something, because surely I can't invent something better than the state of the ar

Re: [PATCH 1/2] One filter per commit

2018-10-11 Thread Derrick Stolee
On 10/10/2018 9:21 PM, Jonathan Tan wrote: diff --git a/commit-graph.c b/commit-graph.c index f415d3b41f..90b0b3df90 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -715,13 +715,11 @@ static int add_ref_to_list(const char *refname, static void add_changes_to_bloom_filter(struct bloom_filter

Re: [PATCH v3 7/7] revision.c: refactor basic topo-order logic

2018-10-11 Thread Derrick Stolee
On 10/11/2018 11:35 AM, Jeff King wrote: On Fri, Sep 21, 2018 at 10:39:36AM -0700, Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee When running a command like 'git rev-list --topo-order HEAD', Git performed the following steps: [...] In the new algorithm, these t

Re: [PATCH v4 0/1] contrib: Add script to show uncovered "new" lines

2018-10-12 Thread Derrick Stolee
On 10/11/2018 11:01 PM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: CHANGES IN V4: I reduced the blame output using -s which decreases the width. I include a summary of the commit authors at the end to help people see the lines they wrote. This version is also c

Re: [PATCH v3 4/7] revision.c: begin refactoring --topo-order logic

2018-10-12 Thread Derrick Stolee
On 10/12/2018 2:33 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: * revs->limited implies we run limit_list() to walk the entire reachable set. There are some short-cuts here, such as if we perform a range query like 'git rev-list COMPARE..HEAD&#

Git Test Coverage Report (Friday, Oct 12)

2018-10-12 Thread Derrick Stolee
unusable todo list: '%s'"), todo_file); b97e187364 4780) todo_list_release(&todo_list); b97e187364 4781) return -1; b97e187364 4785) return error(_("could not copy '%s' to '%s'."), todo_file, b97e187364 4789) return error(_("could not transfo

Re: [PATCH v3 1/2] commit-graph write: add progress output

2018-10-12 Thread Derrick Stolee
On 10/12/2018 11:07 AM, Ævar Arnfjörð Bjarmason wrote: On Fri, Oct 12 2018, Junio C Hamano wrote: Makes sense. If this second iteration were also time consuming, then it probably is a good idea to split these into two separate phases? "Counting 1...N" followed by "Inspecting 1...N" or somethin

Re: [PATCH v2 0/3] Add GIT_TEST_MULTI_PACK_INDEX environment variable

2018-10-12 Thread Derrick Stolee
On 10/12/2018 1:34 PM, Derrick Stolee via GitGitGadget wrote: To increase coverage of the multi-pack-index feature, add a GIT_TEST_MULTI_PACK_INDEX environment variable similar to other GIT_TEST_* variables. After creating the environment variable and running the test suite with it enabled, I

Re: Git Test Coverage Report (Friday, Oct 12)

2018-10-15 Thread Derrick Stolee
On 10/15/2018 4:09 AM, Johannes Schindelin wrote: Hi Stolee, On Fri, 12 Oct 2018, Derrick Stolee wrote: In an effort to ensure new code is reasonably covered by the test suite, we now have contrib/coverage-diff.sh to combine the gcov output from 'make coverage-test ; make coverage-r

Re: [PATCH 0/4] Bloom filter experiment

2018-10-15 Thread Derrick Stolee
On 10/9/2018 3:34 PM, SZEDER Gábor wrote: To keep the ball rolling, here is my proof of concept in a somewhat cleaned-up form, with still plenty of rough edges. Peff, Szeder, and Jonathan, Thanks for giving me the kick in the pants to finally write a proof of concept for my personal take on h

Re: [PATCH v2 09/13] commit-graph: convert to using the_hash_algo

2018-10-15 Thread Derrick Stolee
On 10/14/2018 10:18 PM, brian m. carlson wrote: Instead of using hard-coded constants for object sizes, use the_hash_algo to look them up. In addition, use a function call to look up the object ID version and produce the correct value. This looks good and I can see already how this new organizat

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-15 Thread Derrick Stolee
On 10/14/2018 10:19 PM, brian m. carlson wrote: Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants themselves, as they are internal and could change in the

Re: [PATCH 15/16] commit-reach: make can_all_from_reach... linear

2018-10-15 Thread Derrick Stolee
On 10/14/2018 10:29 AM, René Scharfe wrote: It still has some repetition, converted code is a bit longer than the current one, and I don't know how to build a Coccinelle rule that would do that conversion. Looked for a possibility to at least leave QSORT call-sites alone by enhancing that macro,

Git Test Coverage Report (Monday, Oct 15)

2018-10-15 Thread Derrick Stolee
et); 2c58483a59 3373) return error(_("could not checkout %s"), commit); 4df66c40b0 3387) return error(_("%s: not a valid OID"), orig_head); 71f82465b1 3407) fprintf(stderr, _("Stopped at HEAD\n")); b97e187364 4771) return -1; b97e187364 4774) return -1; b97e187364 4780) ret

Re: Git Test Coverage Report (Monday, Oct 15)

2018-10-15 Thread Derrick Stolee
On 10/15/2018 12:24 PM, Derrick Stolee wrote: Uncovered code in 'jch' (22f2f0f) and not in 'next' (152ad8e) - prio-queue.c 2d181390f3 94) return queue->array[queue->nr - 1].data; (I have a fix to cover thi

Re: [PATCH 0/4] Bloom filter experiment

2018-10-16 Thread Derrick Stolee
On 10/16/2018 12:45 AM, Junio C Hamano wrote: Derrick Stolee writes: 2. The filters are sized according to the number of changes in each commit, with a minimum of one 64-bit word. ... 6. When we compute the Bloom filters, we don't store a filter for commits whose first-parent diff has

Re: [PATCH 0/4] Bloom filter experiment

2018-10-16 Thread Derrick Stolee
On 10/16/2018 8:57 AM, Ævar Arnfjörð Bjarmason wrote: On Tue, Oct 16 2018, Derrick Stolee wrote: On 10/16/2018 12:45 AM, Junio C Hamano wrote: Derrick Stolee writes: 2. The filters are sized according to the number of changes in each commit, with a minimum of one 64-bit word. ... 6. When

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-16 Thread Derrick Stolee
On 10/16/2018 11:35 AM, Duy Nguyen wrote: On Mon, Oct 15, 2018 at 4:23 AM brian m. carlson wrote: Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants them

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-17 Thread Derrick Stolee
On 10/14/2018 10:19 PM, brian m. carlson wrote: Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants themselves, as they are internal and could change in the

Re: [PATCH 00/19] Bring more repository handles into our code base

2018-10-17 Thread Derrick Stolee
On 10/16/2018 7:35 PM, Stefan Beller wrote: This series takes another approach as it doesn't change the signature of functions, but introduces new functions that can deal with arbitrary repositories, keeping the old function signature around using a shallow wrapper.

Re: [PATCH] test-tool: show tool list on error

2018-10-17 Thread Derrick Stolee
This will improve contributors' lives! Thanks. Reviewed-by: Derrick Stolee

Re: [PATCH 0/1] Run GIT_TEST_COMMIT_GRAPH and GIT_TEST_MULTI_PACK_INDEX during CI

2018-10-17 Thread Derrick Stolee
On 10/17/2018 9:00 AM, Derrick Stolee via GitGitGadget wrote: [1] https://git.visualstudio.com/git/_build?definitionId=4 Newlines are hard. Sorry for the formatting issues when translating from a PR description. Build definition that tests Git with different arrangements of GIT_TEST_

Git Test Coverage Report (Wednesday, Oct 17)

2018-10-17 Thread Derrick Stolee
-- builtin/gc.c 3029970275 builtin/gc.c 461) ret = error_errno(_("cannot stat '%s'"), gc_log_path); 3029970275 builtin/gc.c 470) ret = error_errno(_("cannot read '%s'"), gc_log_path); fec2ed2187 builtin/gc.c 495) die(FAILED_RUN

Re: commit-graph is cool (overcoming add_missing_tags() perf issues)

2018-10-17 Thread Derrick Stolee
On 10/17/2018 2:00 PM, Elijah Newren wrote: Hi, Just wanted to give a shout-out for the commit-graph work and how impressive it is. I had an internal report from a user that git pushes containing only one new tiny commit were taking over a minute (in a moderate size repo with good network conne

Re: [PATCH 1/3] t6501: use --quiet when testing gc stderr

2018-10-18 Thread Derrick Stolee
On 10/18/2018 1:23 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: From: Derrick Stolee The test script t6501-freshen-objects.sh has some tests that care if 'git gc' has any output to stderr. This is intended to say that no warnings occurred rel

Re: [PATCH 0/3] Use commit-graph by default

2018-10-18 Thread Derrick Stolee
On 10/17/2018 11:47 PM, Junio C Hamano wrote: If I recall correctly, one more task that was discussed but hasn't been addressed well is how the generation and incremental update of it should integrate with the normal repository maintenance workflow (perhaps "gc --auto"). If we are going to turn

Re: [PATCH v2 13/13] commit-graph: specify OID version for SHA-256

2018-10-18 Thread Derrick Stolee
On 10/17/2018 8:06 PM, brian m. carlson wrote: On Wed, Oct 17, 2018 at 04:31:19PM +0200, Duy Nguyen wrote: On Wed, Oct 17, 2018 at 12:44 AM brian m. carlson wrote: Honestly, anything in the .git directory that is not the v3 pack indexes or the loose object file should be in exactly one hash al

Re: [PATCH] multi-pack-index: avoid dead store for struct progress

2018-10-18 Thread Derrick Stolee
On 10/18/2018 2:59 PM, Carlo Marcelo Arenas Belón wrote: it is initialized unconditionally by a call to start_progress below. Signed-off-by: Carlo Marcelo Arenas Belón --- midx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/midx.c b/midx.c index ea2f3ffe2e..4fac0cd08a

Re: [PATCH 1/1] commit-reach: fix first-parent heuristic

2018-10-19 Thread Derrick Stolee
On 10/19/2018 1:24 AM, Junio C Hamano wrote: "Derrick Stolee via GitGitGadget" writes: We can also re-run the performance tests from commit 4fbcca4e "commit-reach: make can_all_from_reach... linear". Performance was measured on the Linux repository using 'test-tool

Re: What's cooking in git.git (Oct 2018, #04; Fri, 19)

2018-10-19 Thread Derrick Stolee
On 10/19/2018 2:02 AM, Junio C Hamano wrote: * ds/ci-commit-graph-and-midx (2018-10-19) 1 commit - ci: add optional test variables One of our CI tests to run with "unusual/experimental/random" settings now also uses commti-graph and midx. Will merge to 'next'. s/commti-graph/commit-g

Re: [PATCH v3 06/20] commit-graph: add 'verify' subcommand

2018-06-04 Thread Derrick Stolee
On 6/2/2018 5:19 PM, Jakub Narebski wrote: Derrick Stolee writes: Do we have a way to run individual steps of the test suite? I am unfamiliar with that process. The t/README describes three such ways in "Skipping Tests" section: - GIT_SKIP_TESTS environment variable, which can

Re: [PATCH v3 09/20] commit-graph: verify corrupt OID fanout and lookup

2018-06-04 Thread Derrick Stolee
On 6/2/2018 12:38 AM, Duy Nguyen wrote: On Thu, May 24, 2018 at 6:25 PM, Derrick Stolee wrote: + if (i && oidcmp(&prev_oid, &cur_oid) >= 0) + graph_report("commit-graph has incorrect OID order: %s then %s", +

Re: [PATCH v3 13/20] commit-graph: verify generation number

2018-06-04 Thread Derrick Stolee
On 6/2/2018 8:23 AM, Jakub Narebski wrote: Derrick Stolee writes: While iterating through the commit parents, perform the generation number calculation and compare against the value stored in the commit-graph. All right, that's good. What about commit-graph files that

Re: [PATCH v3 16/20] commit-graph: verify contents match checksum

2018-06-04 Thread Derrick Stolee
On 6/2/2018 11:52 AM, Jakub Narebski wrote: Derrick Stolee writes: The commit-graph file ends with a SHA1 hash of the previous contents. If a commit-graph file has errors but the checksum hash is correct, then we know that the problem is a bug in Git and not simply file corruption after-the

Re: [PATCH v3 17/20] fsck: verify commit-graph

2018-06-04 Thread Derrick Stolee
On 6/2/2018 12:17 PM, Jakub Narebski wrote: Derrick Stolee writes: If core.commitGraph is true, verify the contents of the commit-graph during 'git fsck' using the 'git commit-graph verify' subcommand. Run this check on all alternates, as well. All right, so we have on

[PATCH] t5318-commit-graph.sh: use core.commitGraph

2018-06-04 Thread Derrick Stolee
ebski Signed-off-by: Derrick Stolee --- t/t5318-commit-graph.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 77d85aefe7..59d0be2877 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -28,8 +28,8 @@

Re: [PATCH v3 18/20] commit-graph: add '--reachable' option

2018-06-04 Thread Derrick Stolee
On 6/2/2018 1:34 PM, Jakub Narebski wrote: Derrick Stolee writes: When writing commit-graph files, it can be convenient to ask for all reachable commits (starting at the ref set) in the resulting file. This is particularly helpful when writing to stdin is complicated, such as a future

Re: [PATCH v3 19/20] gc: automatically write commit-graph files

2018-06-04 Thread Derrick Stolee
On 6/2/2018 2:03 PM, Jakub Narebski wrote: Derrick Stolee writes: The commit-graph file is a very helpful feature for speeding up git operations. In order to make it more useful, write the commit-graph file by default during standard garbage collection operations. I think you meant here

Re: [PATCH v3 15/20] commit-graph: test for corrupted octopus edge

2018-06-04 Thread Derrick Stolee
On 6/2/2018 8:39 AM, Jakub Narebski wrote: Derrick Stolee writes: The commit-graph file has an extra chunk to store the parent int-ids for parents beyond the first parent for octopus merges. Our test repo has a single octopus merge that we can manipulate to demonstrate the 'verify'

[PATCH v4 01/21] commit-graph: UNLEAK before die()

2018-06-04 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 37420ae0fd..f0875b8bf3 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -51,8 +51,11 @@ static

[PATCH v4 02/21] commit-graph: fix GRAPH_MIN_SIZE

2018-06-04 Thread Derrick Stolee
n the magic constants. Signed-off-by: Derrick Stolee --- commit-graph.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index bb54c1214c..c09e87c3c2 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -34,10 +34,11 @@ #define GRAPH_LAST

[PATCH v4 06/21] commit-graph: add 'verify' subcommand

2018-06-04 Thread Derrick Stolee
graph file exists, this is an acceptable state. Do not report any errors. Helped-by: Ramsay Jones Signed-off-by: Derrick Stolee --- Documentation/git-commit-graph.txt | 6 + builtin/commit-graph.c | 38 ++ commit-graph.c

[PATCH v4 07/21] commit-graph: verify catches corrupt signature

2018-06-04 Thread Derrick Stolee
will be marked for translation, as that is the only message that is intended for a typical user. Helped-by: Szeder Gábor Signed-off-by: Derrick Stolee --- t/t5318-commit-graph.sh | 43 + 1 file changed, 43 insertions(+) diff --git a/t/t5318-commit-gra

[PATCH v4 03/21] commit-graph: parse commit from chosen graph

2018-06-04 Thread Derrick Stolee
Before verifying a commit-graph file against the object database, we need to parse all commits from the given commit-graph file. Create parse_commit_in_graph_one() to target a given struct commit_graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 18 +++--- 1 file changed, 15

[PATCH v4 08/21] commit-graph: verify required chunks are present

2018-06-04 Thread Derrick Stolee
The commit-graph file requires the following three chunks: * OID Fanout * OID Lookup * Commit Data If any of these are missing, then the 'verify' subcommand should report a failure. This includes the chunk IDs malformed or the chunk count is truncated. Signed-off-by: Derrick Stolee -

[PATCH v4 05/21] commit-graph: load a root tree from specific graph

2018-06-04 Thread Derrick Stolee
When lazy-loading a tree for a commit, it will be important to select the tree from a specific struct commit_graph. Create a new method that specifies the commit-graph file and use that in get_commit_tree_in_graph(). Signed-off-by: Derrick Stolee --- commit-graph.c | 12 +--- 1 file

[PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc'

2018-06-04 Thread Derrick Stolee
y large, but I'll send it in a follow-up PR. Thanks, -Stolee Derrick Stolee (21): commit-graph: UNLEAK before die() commit-graph: fix GRAPH_MIN_SIZE commit-graph: parse commit from chosen graph commit: force commit to parse from object database commit-graph: load a root tree from specifi

[PATCH v4 11/21] commit-graph: verify root tree OIDs

2018-06-04 Thread Derrick Stolee
ectly from the object database. Add checks for the root tree OID. Signed-off-by: Derrick Stolee --- commit-graph.c | 17 - t/t5318-commit-graph.sh | 7 +++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index

[PATCH v4 09/21] commit-graph: verify corrupt OID fanout and lookup

2018-06-04 Thread Derrick Stolee
In the commit-graph file, the OID fanout chunk provides an index into the OID lookup. The 'verify' subcommand should find incorrect values in the fanout. Similarly, the 'verify' subcommand should find out-of-order values in the OID lookup. Signed-off-by: Derrick Stolee

[PATCH v4 12/21] commit-graph: verify parent list

2018-06-04 Thread Derrick Stolee
one parsed from the object database. Test these checks for corrupt parents, too many parents, and wrong parents. Add a boundary check to insert_parent_or_die() for when the parent position value is out of range. The octopus merge will be tested in a later commit. Signed-off-by: Derrick Stolee -

[PATCH v4 10/21] commit-graph: verify objects exist

2018-06-04 Thread Derrick Stolee
In the 'verify' subcommand, load commits directly from the object database to ensure they exist. Parse by skipping the commit-graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 17 + t/t5318-commit-graph.sh | 7 +++ 2 files changed, 24 insertion

[PATCH v4 18/21] commit-graph: use string-list API for input

2018-06-04 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 39 +-- commit-graph.c | 15 +++ commit-graph.h | 7 +++ 3 files changed, 23 insertions(+), 38 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit

[PATCH v4 13/21] commit-graph: verify generation number

2018-06-04 Thread Derrick Stolee
have a mix of zero and non-zero generation numbers. Create a test that sets one commit to generation zero and all following commits report a failure as they have non-zero generation in a file that contains generation number zero. Signed-off-by: Derrick Stolee --- commit-graph.c | 34

[PATCH v4 20/21] gc: automatically write commit-graph files

2018-06-04 Thread Derrick Stolee
mason Signed-off-by: Derrick Stolee --- Documentation/config.txt | 10 +- Documentation/git-gc.txt | 4 builtin/gc.c | 6 ++ t/t5318-commit-graph.sh | 14 ++ 4 files changed, 33 insertions(+), 1 deletion(-) diff --git a/Documentation/config.txt b/Docu

[PATCH v4 19/21] commit-graph: add '--reachable' option

2018-06-04 Thread Derrick Stolee
When writing commit-graph files, it can be convenient to ask for all reachable commits (starting at the ref set) in the resulting file. This is particularly helpful when writing to stdin is complicated, such as a future integration with 'git gc'. Signed-off-by: Derrick Stolee --- Doc

[PATCH v4 16/21] commit-graph: verify contents match checksum

2018-06-04 Thread Derrick Stolee
our tests, as we break the checksum as we modify bytes of the commit-graph file. Signed-off-by: Derrick Stolee --- commit-graph.c | 16 ++-- t/t5318-commit-graph.sh | 6 ++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-gra

[PATCH v4 17/21] fsck: verify commit-graph

2018-06-04 Thread Derrick Stolee
erifying a commit-graph file from the other fsck details. 2. The commit-graph verification requires the commits to be loaded in a specific order to guarantee we parse from the commit-graph file for some objects and from the object database for others. Signed-off-by: Derrick Stolee --

[PATCH v4 14/21] commit-graph: verify commit date

2018-06-04 Thread Derrick Stolee
Signed-off-by: Derrick Stolee --- commit-graph.c | 6 ++ t/t5318-commit-graph.sh | 6 ++ 2 files changed, 12 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index 5faecae2a7..47fdd62e88 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -982,6 +982,12 @@ int

[PATCH v4 21/21] commit-graph: update design document

2018-06-04 Thread Derrick Stolee
The commit-graph feature is now integrated with 'fsck' and 'gc', so remove those items from the "Future Work" section of the commit-graph design document. Also remove the section on lazy-loading trees, as that was completed in an earlier patch series.

[PATCH v4 15/21] commit-graph: test for corrupted octopus edge

2018-06-04 Thread Derrick Stolee
The commit-graph file has an extra chunk to store the parent int-ids for parents beyond the first parent for octopus merges. Our test repo has a single octopus merge that we can manipulate to demonstrate the 'verify' subcommand detects incorrect values in that chunk. Signed-off-by: Derr

[PATCH v4 04/21] commit: force commit to parse from object database

2018-06-04 Thread Derrick Stolee
is explicit in avoiding commits from the commit-graph file. Signed-off-by: Derrick Stolee --- commit.c | 9 +++-- commit.h | 1 + 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/commit.c b/commit.c index 1d28677dfb..6eaed0174c 100644 --- a/commit.c +++ b/commit.c @@ -392,7

Re: [PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc'

2018-06-04 Thread Derrick Stolee
Sorry I forgot to --in-reply-to the previous version [1] [1] https://public-inbox.org/git/20180524162504.158394-1-dsto...@microsoft.com/T/#u On 6/4/2018 12:52 PM, Derrick Stolee wrote: Thanks for the feedback on v3. There were several small cleanups, but perhaps the biggest change is the

Re: [PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc'

2018-06-05 Thread Derrick Stolee
On 6/5/2018 10:51 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Jun 4, 2018 at 6:52 PM, Derrick Stolee wrote: Thanks for the feedback on v3. There were several small cleanups, but perhaps the biggest change is the addition of "commit-graph: use string-list API for input" which makes &qu

Re: [RFC PATCH 00/18] Multi-pack index (MIDX)

2018-06-06 Thread Derrick Stolee
On 6/6/2018 4:13 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Jan 08 2018, Derrick Stolee wrote: On 1/7/2018 5:42 PM, Ævar Arnfjörð Bjarmason wrote: On Sun, Jan 07 2018, Derrick Stolee jotted: git log --oneline --raw --parents Num Packs | Before MIDX | After MIDX | Rel % | 1 pack

Re: [PATCH v4 17/21] fsck: verify commit-graph

2018-06-06 Thread Derrick Stolee
On 6/6/2018 7:08 AM, Ævar Arnfjörð Bjarmason wrote: On Mon, Jun 04 2018, Derrick Stolee wrote: + prepare_alt_odb(); + for (alt = alt_odb_list; alt; alt = alt->next) { + verify_argv[2] = "--object-dir"; +

[PATCH v5 00/21] Integrate commit-graph into 'fsck' and 'gc'

2018-06-06 Thread Derrick Stolee
ier version of this patch. Thanks, -Stolee Derrick Stolee (21): commit-graph: UNLEAK before die() commit-graph: fix GRAPH_MIN_SIZE commit-graph: parse commit from chosen graph commit: force commit to parse from object database commit-graph: load a root tree from specific graph

[PATCH v5 05/21] commit-graph: load a root tree from specific graph

2018-06-06 Thread Derrick Stolee
When lazy-loading a tree for a commit, it will be important to select the tree from a specific struct commit_graph. Create a new method that specifies the commit-graph file and use that in get_commit_tree_in_graph(). Signed-off-by: Derrick Stolee --- commit-graph.c | 12 +--- 1 file

[PATCH v5 15/21] commit-graph: test for corrupted octopus edge

2018-06-06 Thread Derrick Stolee
The commit-graph file has an extra chunk to store the parent int-ids for parents beyond the first parent for octopus merges. Our test repo has a single octopus merge that we can manipulate to demonstrate the 'verify' subcommand detects incorrect values in that chunk. Signed-off-by: Derr

[PATCH v5 21/21] commit-graph: update design document

2018-06-06 Thread Derrick Stolee
The commit-graph feature is now integrated with 'fsck' and 'gc', so remove those items from the "Future Work" section of the commit-graph design document. Also remove the section on lazy-loading trees, as that was completed in an earlier patch series.

[PATCH v5 17/21] fsck: verify commit-graph

2018-06-06 Thread Derrick Stolee
erifying a commit-graph file from the other fsck details. 2. The commit-graph verification requires the commits to be loaded in a specific order to guarantee we parse from the commit-graph file for some objects and from the object database for others. Signed-off-by: Derrick Stolee --

[PATCH v5 20/21] gc: automatically write commit-graph files

2018-06-06 Thread Derrick Stolee
mason Signed-off-by: Derrick Stolee --- Documentation/config.txt | 10 +- Documentation/git-gc.txt | 4 builtin/gc.c | 6 ++ t/t5318-commit-graph.sh | 14 ++ 4 files changed, 33 insertions(+), 1 deletion(-) diff --git a/Documentation/config.txt b/Docu

[PATCH v5 13/21] commit-graph: verify generation number

2018-06-06 Thread Derrick Stolee
have a mix of zero and non-zero generation numbers. Create a test that sets one commit to generation zero and all following commits report a failure as they have non-zero generation in a file that contains generation number zero. Signed-off-by: Derrick Stolee --- commit-graph.c | 34

[PATCH v5 03/21] commit-graph: parse commit from chosen graph

2018-06-06 Thread Derrick Stolee
Before verifying a commit-graph file against the object database, we need to parse all commits from the given commit-graph file. Create parse_commit_in_graph_one() to target a given struct commit_graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 18 +++--- 1 file changed, 15

[PATCH v5 10/21] commit-graph: verify objects exist

2018-06-06 Thread Derrick Stolee
In the 'verify' subcommand, load commits directly from the object database to ensure they exist. Parse by skipping the commit-graph. Signed-off-by: Derrick Stolee --- commit-graph.c | 17 + t/t5318-commit-graph.sh | 7 +++ 2 files changed, 24 insertion

[PATCH v5 19/21] commit-graph: add '--reachable' option

2018-06-06 Thread Derrick Stolee
When writing commit-graph files, it can be convenient to ask for all reachable commits (starting at the ref set) in the resulting file. This is particularly helpful when writing to stdin is complicated, such as a future integration with 'git gc'. Signed-off-by: Derrick Stolee --- Doc

<    1   2   3   4   5   6   7   8   9   10   >