[PATCH 1/1] sha1-file: split OBJECT_INFO_FOR_PREFETCH

2019-05-28 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The OBJECT_INFO_FOR_PREFETCH bitflag was added to sha1-file.c in 0f4a4fb1 (sha1-file: support OBJECT_INFO_FOR_PREFETCH, 2019-03-29) and is used to prevent the fetch_objects() method when enabled. However, there is a problem with the current use. The definition of

[PATCH 0/1] sha1-file: split OBJECT_INFO_FOR_PREFETCH

2019-05-28 Thread Derrick Stolee via GitGitGadget
, too. Thanks, -Stolee [1] https://github.com/microsoft/git/pull/140 Derrick Stolee (1): sha1-file: split OBJECT_INFO_FOR_PREFETCH object-store.h | 10 +++--- sha1-file.c| 2 +- 2 files changed, 8 insertions(+), 4 deletions(-) base-commit: 0f4a4fb1c4239a2aa46343add84ad6f99f6f3aae Publis

[PATCH] pack-bitmap: look for an uninteresting bitmap

2019-05-24 Thread Derrick Stolee
E: this is only a proof-of-concept, as it fails a test in t5310-pack-bitmaps.sh (clearly marked as a failure now). Reported-by: Ævar Arnfjörð Bjarmason Helped-by: Jeff King Signed-off-by: Derrick Stolee --- On 5/23/2019 7:30 AM, Jeff King wrote:> + /* > + * First traverse the rel

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-24 Thread Derrick Stolee
On 5/24/2019 3:24 AM, Jeff King wrote: > On Thu, May 23, 2019 at 08:53:56AM -0400, Derrick Stolee wrote: > >>> I spent a while thinking and experimenting with this tonight. The result >>> is the patch below. Ævar, do you still have a copy of the repo that >>> misb

Re: standalone library/tool to query commit-graph?

2019-05-23 Thread Derrick Stolee
On 5/23/2019 6:20 PM, SZEDER Gábor wrote: > On Thu, May 23, 2019 at 11:54:22PM +0200, Ævar Arnfjörð Bjarmason wrote: >> >> On Thu, May 23 2019, Jakub Narebski wrote: >> >>> Derrick Stolee writes: >>>> On 5/22/2019 2:49 PM, Karl Ostmo wrote: >>>

Re: [PATCH v2 09/11] commit-graph: merge commit-graph chains

2019-05-23 Thread Derrick Stolee
On 5/22/2019 8:43 PM, Ævar Arnfjörð Bjarmason wrote: > > On Wed, May 22 2019, Derrick Stolee via GitGitGadget wrote: > >> To keep lookups fast, but also keep most incremental writes fast, create >> a strategy for merging levels of the commit-graph chain. The strategy is >

Re: [PATCH v3] repack: enable bitmaps by default on bare repos

2019-05-23 Thread Derrick Stolee
On 5/23/2019 7:30 AM, Jeff King wrote: > On Wed, Apr 10, 2019 at 06:57:21PM -0400, Jeff King wrote: > >> 2. The answer we get from a bitmap versus a regular traversal are not >> apples-to-apples equivalent. The regular traversal walks down to >> the UNINTERESTING commits, marks the bou

[PATCH v2 00/11] [RFC] Commit-graph: Write incremental files

2019-05-22 Thread Derrick Stolee via GitGitGadget
file. (This is the main reason this is a priority to the VFS for Git team.) This is based on ds/commit-graph-write-refactor. Thanks, -Stolee [1] https://github.com/git/git/commit/43d356180556180b4ef6ac232a14498a5bb2b446 commit-graph write: don't die if the existing graph is co

[PATCH v2 10/11] commit-graph: allow cross-alternate chains

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee In an environment like a fork network, it is helpful to have a commit-graph chain that spans both the base repo and the fork repo. The fork is usually a small set of data on top of the large repo, but sometimes the fork is much larger. For example, git-for-windows/git has

[PATCH v2 04/11] commit-graph: load commit-graph chains

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Prepare the logic for reading a chain of commit-graphs. First, look for a file at $OBJDIR/info/commit-graph. If it exists, then use that file and stop. Next, look for the chain file at $OBJDIR/info/commit-graphs/commit-graph-chain. If this file exists, then load the hash

[PATCH v2 09/11] commit-graph: merge commit-graph chains

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee When searching for a commit in a commit-graph chain of G graphs with N commits, the search takes O(G log N) time. If we always add a new tip graph with every write, the linear G term will start to dominate and slow the lookup process. To keep lookups fast, but also keep

[PATCH v2 11/11] commit-graph: expire commit-graph files

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee As we merge commit-graph files in a commit-graph chain, we should clean up the files that are no longer used. This change introduces an 'expiry_window' value to the context, which is always zero (for now). We then check the modified time of each graph-{hash}.gra

[PATCH v2 06/11] commit-graph: rearrange chunk count logic

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The number of chunks in a commit-graph file can change depending on whether we need the Extra Edges Chunk. We are going to add more optional chunks, and it will be helpful to rearrange this logic around the chunk count before doing so. Specifically, we need to finalize the

[PATCH v2 08/11] commit-graph: add --split option to builtin

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Add a new "--split" option to the 'git commit-graph write' subcommand. This option allows the optional behavior of writing a commit-graph chain. The current behavior will add a tip commit-graph containing any commits that are not in the existing commit-g

[PATCH v2 01/11] commit-graph: document commit-graph chains

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Add a basic description of commit-graph chains. More details about the feature will be added as we add functionality. This introduction gives a high-level overview to the goals of the feature and the basic layout of commit-graph chains. Signed-off-by: Derrick Stolee

[PATCH v2 02/11] commit-graph: prepare for commit-graph chains

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee To prepare for a chain of commit-graph files, augment the commit_graph struct to point to a base commit_graph. As we load commits from the graph, we may actually want to read from a base file according to the graph position. The "graph position" of a commit i

[PATCH v2 07/11] commit-graph: write commit-graph chains

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Extend write_commit_graph() to write a commit-graph chain when given the COMMIT_GRAPH_SPLIT flag. This implementation is purposefully simplistic in how it creates a new chain. The commits not already in the chain are added to a new tip commit-graph file. Much of the logic

[PATCH v2 05/11] commit-graph: add base graphs chunk

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee To quickly verify a commit-graph chain is valid on load, we will read from the new "Base Graphs Chunk" of each file in the chain. This will prevent accidentally loading incorrect data from manually editing the commit-graph-chain file or renaming graph-{hash}.g

[PATCH v2 03/11] commit-graph: rename commit_compare to oid_compare

2019-05-22 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The helper function commit_compare() actually compares object_id structs, not commits. A future change to commit-graph.c will need to sort commit structs, so rename this function in advance. Signed-off-by: Derrick Stolee --- commit-graph.c | 4 ++-- 1 file changed, 2

Re: Revision walking, commit dates, slop

2019-05-22 Thread Derrick Stolee
On 5/22/2019 2:29 PM, Jakub Narebski wrote: > Derrick Stolee writes: >> On 5/20/2019 7:27 PM, Jakub Narebski wrote: > Restating it yet again: > >A. corrected_date(C) = max(committer_date(C), >max_P(committer_date(P) + offset(P)) + 1)

Re: standalone library/tool to query commit-graph?

2019-05-22 Thread Derrick Stolee
On 5/22/2019 2:49 PM, Karl Ostmo wrote: > After producing the file ".git/objects/info/commit-graph" with the > command "git commit-graph write", is there a way to answer queries > like "git merge-base --is-ancestor" without having a .git directory? > E.g. is there a library that will operate on the

[PATCH 2/2] revision: keep topo-walk free of unintersting commits

2019-05-21 Thread Derrick Stolee
git rev-list --topo-order HEAD~1..HEAD will expand the topo walk for all commits reachable from HEAD, and not just one commit. This change should speed up these cases, but there is still a need for corrected commit-date for some A..B queries. Signed-off-by: Derrick Stolee --- Sorry for the patch

[PATCH] revision: use generation for A..B --topo-order queries

2019-05-21 Thread Derrick Stolee
truct rev_info forces a full walk of the commit history (after discovering the A..B range). Previosuly, it is enabled whenever we see an uninteresting commit. We prevent enabling the parameter when we are planning to use the reachability index for a topo-order. Signed-off-by: Derrick Stolee

Re: Revision walking, commit dates, slop

2019-05-20 Thread Derrick Stolee
On 5/20/2019 7:27 PM, Jakub Narebski wrote: > Jakub Narebski writes: >> Derrick Stolee writes: >>> On 5/20/2019 7:02 AM, Jakub Narebski wrote: >>>> >>>> Are there any blockers that prevent the switch to this >>>> "generation number v2

Re: Finer timestamps and serialization in git

2019-05-20 Thread Derrick Stolee
On 5/20/2019 12:36 PM, Eric S. Raymond wrote: > Michal Suchánek : >> On Wed, 15 May 2019 21:25:46 -0400 >> Derrick Stolee wrote: >> >>> On 5/15/2019 8:28 PM, Eric S. Raymond wrote: >>>> Derrick Stolee : >>>>> What problem are you tr

Re: [PATCH 3/3] packfile: close_all_packs to close_object_store

2019-05-20 Thread Derrick Stolee
On 5/20/2019 6:01 AM, Johannes Schindelin wrote: > Hi Stolee, > > *really* minor nit: the commit subject probably wants to have a "rename" > after the colon ;-) I did put that there, but then the subject line was too long. I'm not opposed to putting it back. > The patch looks sensible to me. Si

Re: Revision walking, commit dates, slop

2019-05-20 Thread Derrick Stolee
On 5/20/2019 7:02 AM, Jakub Narebski wrote: > > Are there any blockers that prevent the switch to this > "generation number v2"? > > - Is it a problem with insufficient data to choose the correct numbering > as "generation number v2' (there can be only one)? > - Is it a problem with selected "ge

Re: [PATCH 0/3] Close commit-graph before calling 'gc'

2019-05-20 Thread Derrick Stolee
On 5/18/2019 10:04 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" writes: > >> This series adds close_commit_graph() to close_all_packs() and then renames >> close_all_packs() to close_object_store(). This name is more descriptive of >> its

Re: Revision walking, commit dates, slop

2019-05-19 Thread Derrick Stolee
On 5/18/2019 12:17 AM, Mike Hommey wrote: > On Sat, May 18, 2019 at 12:58:28PM +0900, Mike Hommey wrote: >> On Sat, May 18, 2019 at 03:50:05AM +0200, SZEDER Gábor wrote: >>> >>> All the above is without commit-graph, I presume? If so, then you >>> should give it a try, as it might bring immediate

[PATCH 0/3] Close commit-graph before calling 'gc'

2019-05-17 Thread Derrick Stolee via GitGitGadget
rpose. This is based on ds/commit-graph-write-refactor to avoid conflicts. Thanks, -Stolee Derrick Stolee (3): commit-graph: use raw_object_store when closing packfile: close commit-graph in close_all_packs packfile: close_all_packs to close_object_store builtin/am.c | 2 +- bu

[PATCH 2/3] packfile: close commit-graph in close_all_packs

2019-05-17 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The close_all_packs() method is used to close all read handles to pack-files and the multi-pack-index before running 'git gc --auto'. This is particularly important on the Windows platform, where read handles block any writes to those files. Replacing one of these

[PATCH 1/3] commit-graph: use raw_object_store when closing

2019-05-17 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The close_commit_graph() method took a repository struct, but then only uses the raw_object_store within. Change the function prototype to make the method more flexible. Signed-off-by: Derrick Stolee --- commit-graph.c | 8 commit-graph.h | 2 +- upload-pack.c

[PATCH 3/3] packfile: close_all_packs to close_object_store

2019-05-17 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The close_all_packs() method is now responsible for more than just pack-files. It also closes the commit-graph and the multi-pack-index. Rename the function to be more descriptive of its larger role. The name also fits because the input parameter is a raw_object_store

Re: Finer timestamps and serialization in git

2019-05-15 Thread Derrick Stolee
On 5/15/2019 8:28 PM, Eric S. Raymond wrote: > Derrick Stolee : >> What problem are you trying to solve where commit date is important? > > I don't know what Jason's are. I know what mine are. > > A. Portable commit identifiers > > 1. When I in-migrate a

Re: Finer timestamps and serialization in git

2019-05-15 Thread Derrick Stolee
On 5/15/2019 7:32 PM, Eric S. Raymond wrote: > Derrick Stolee : >> On 5/15/2019 3:16 PM, Eric S. Raymond wrote: >>> The deeper problem is that I want something from Git that I cannot >>> have with 1-second granularity. That is: a unique timestamp on each >>> c

Re: Finer timestamps and serialization in git

2019-05-15 Thread Derrick Stolee
On 5/15/2019 4:28 PM, Jason Pyeron wrote: > (please don’t cc me) Ok. I'll "To" you. > and we follow the rule that: > > 1. any trailing zero after the decimal point MUST be omitted > 2. if there are no digits after the decimal point, it MUST be omitted > > This would allow: > > committer Name

Re: Finer timestamps and serialization in git

2019-05-15 Thread Derrick Stolee
On 5/15/2019 3:16 PM, Eric S. Raymond wrote: > The deeper problem is that I want something from Git that I cannot > have with 1-second granularity. That is: a unique timestamp on each > commit in a repository. This is impossible in a distributed version control system like Git (where the commits a

[PATCH v6 05/11] midx: refactor permutation logic and pack sorting

2019-05-14 Thread Derrick Stolee
ere tracking in that struct. Finally, update get_sorted_entries() to not permute the pack-int-id and instead supply the permutation to write_midx_object_offsets(). This requires sorting the packs after get_sorted_entries(). Signed-off-by: Derrick Stole

[PATCH v6 08/11] midx: implement midx_repack()

2019-05-14 Thread Derrick Stolee
l size of the referenced objects, or can create a large pack if the actual size of the referenced objects is larger than the expected size. Signed-off-by: Derrick Stolee --- midx.c | 151 +++- t/t5319-multi-pack-index.sh | 28 +++ 2 file

[PATCH v6 11/11] t5319-multi-pack-index.sh: test batch size zero

2019-05-14 Thread Derrick Stolee
pack-files, as they no longer contain any referenced objects in the multi-pack-index. We must remove the .keep file that was added in the previous test in order to expire that pack-file. Also test that a 'repack' will do nothing if there is only one pack-file. Signed-off-by: Derrick Stole

[PATCH v6 10/11] midx: add test that 'expire' respects .keep files

2019-05-14 Thread Derrick Stolee
pack that contains every object in the repo, then add it to the multi-pack-index. Then create a .keep file for a pack starting with "a-pack" that was added in the previous test. Finally, expire and verify that the pack remains and the other packs were expired. Signed-off-by: Derrick

[PATCH v6 00/11] Create 'expire' and 'repack' verbs for git-multi-pack-index

2019-05-14 Thread Derrick Stolee
in the packed_git list after opening. * I noticed some whitespace problems. I also expect GMail to munge my added "From:" tags, so it will look like the author is "sto...@gmail.com" instead of "dsto...@microsoft.com". Sorry for the continued inconvenience here. Thanks

[PATCH v6 03/11] multi-pack-index: prepare for 'expire' subcommand

2019-05-14 Thread Derrick Stolee
hey have a specific order when sorted by size. This will be important in a later test. Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 5 +++ builtin/multi-pack-index.c | 4 ++- midx.c | 5 +++ midx.h

[PATCH v6 09/11] multi-pack-index: test expire while adding packs

2019-05-14 Thread Derrick Stolee
the multi-pack-index. Add a test to t5319-multi-pack-index.sh that catches this issue. The test adds new packs that cause another pack to be expired, and creates new packs that are lexicographically sorted before and after the existing packs. Signed-off-by: Derrick Stolee --- t/t5319-multi-pack

[PATCH v6 04/11] midx: simplify computation of pack name lengths

2019-05-14 Thread Derrick Stolee
g the file. Signed-off-by: Derrick Stolee --- midx.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/midx.c b/midx.c index 3b7da1a360..62404620ad 100644 --- a/midx.c +++ b/midx.c @@ -433,7 +433,6 @@ struct pack_list { uint32_t nr; uint32_t

[PATCH v6 01/11] repack: refactor pack deletion for future use

2019-05-14 Thread Derrick Stolee
enabled for the use in repack, but will be important for a future caller. Signed-off-by: Derrick Stolee --- builtin/repack.c | 14 ++ packfile.c | 28 packfile.h | 7 +++ 3 files changed, 37 insertions(+), 12 deletions(-) diff --git

[PATCH v6 07/11] multi-pack-index: prepare 'repack' subcommand

2019-05-14 Thread Derrick Stolee
rted by size (ascending). This will be important in a future test. Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 17 + builtin/multi-pack-index.c | 12 ++-- midx.c | 5 + midx.h

[PATCH v6 06/11] multi-pack-index: implement 'expire' subcommand

2019-05-14 Thread Derrick Stolee
ndex to ensure it no longer references those packs. Signed-off-by: Derrick Stolee --- midx.c | 119 +--- t/t5319-multi-pack-index.sh | 20 ++ 2 files changed, 129 insertions(+), 10 deletions(-) diff --git a/midx.c b/midx.c index 6d4b84e243..9b0

[PATCH v6 02/11] Docs: rearrange subcommands for multi-pack-index

2019-05-14 Thread Derrick Stolee
Gábor Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt index f7778a2c85..1af406aca2 100644 --- a/Documentation/git-

Git Test Coverage Report (May 13, 2019)

2019-05-13 Thread Derrick Stolee
7;); 722abec7 215) BUG("sanitizing refname '%s' check returned error", refname); sha1-name.c af96fe33 161) return; t/helper/test-read-cache.c dc76852d 29) die("%s not in index", name); upload-pack.c 8e712ef6 1073) precomposed_unicode = git_config_bool(var,

Git Test Coverage Report (v2.22.0-rc0)

2019-05-13 Thread Derrick Stolee
Please see the test-coverage report for code added between v2.21.0 and v2.22.0-rc0. These lines are not tested by the test suite on Linux with standard build options. Thanks, -Stolee --- builtin/blame.c a544fb08f8b builtin/blame.c 1005) die("no such ref: HEAD"); builtin/branch.c 0ecb1fc7269

Re: [PATCH v4 02/11] commit-graph: return with errors during write

2019-05-13 Thread Derrick Stolee
On 5/13/2019 7:04 AM, Derrick Stolee wrote: > On 5/12/2019 11:13 PM, Junio C Hamano wrote: >> "Derrick Stolee via GitGitGadget" writes: >> >>> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c >>> @@ -188,14 +187,14 @@ static

Re: [PATCH v4 02/11] commit-graph: return with errors during write

2019-05-13 Thread Derrick Stolee
On 5/12/2019 11:13 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" writes: > >> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c >> @@ -188,14 +187,14 @@ static int graph_write(int argc, const char **argv

Re: [PATCH v4 03/11] commit-graph: collapse parameters into flags

2019-05-13 Thread Derrick Stolee
On 5/12/2019 11:44 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" writes: > >> From: Derrick Stolee >> >> The write_commit_graph() and write_commit_graph_reachable() methods >> currently take two boolean parameters: 'append'

Re: [PATCH v7 02/15] trace2: create new combined trace facility

2019-05-10 Thread Derrick Stolee
On 5/10/2019 1:28 PM, SZEDER Gábor wrote: > On Fri, Feb 22, 2019 at 02:25:01PM -0800, Jeff Hostetler via GitGitGadget > wrote: >> From: Jeff Hostetler >> >> * GIT_TR2_EVENT is a new structured format. It writes event data as a >> series of JSON records. > > Please document these new environmen

[PATCH 0/1] trace2: Add variable description to git.txt

2019-05-10 Thread Derrick Stolee via GitGitGadget
Jeff is out this week, so I picked up this request from Szeder. I'm very open to edit suggestions. Thanks, -Stolee In-Reply-To: 20190510172824.gr14...@szeder.dev Derrick Stolee (1): trace2: Add variable description to git.txt Documentation/git.txt | 21 + 1 file ch

[PATCH 1/1] trace2: Add variable description to git.txt

2019-05-10 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Documentation/technical/api-trace2.txt contains the full details of the trace2 API and the GIT_TR2* environment variables. However, most environment variables are included in Documentation/git.txt, including the GIT_TRACE* variables. Add a brief description of the GIT_TR2

Re: I made a flame graph renderer for git's trace2 output

2019-05-10 Thread Derrick Stolee
On 5/10/2019 11:09 AM, Ævar Arnfjörð Bjarmason wrote: > Here's a flamegraph of where git's test suite spends its time on my box: > https://vm.nix.is/~avar/noindex/git-tests.svg > > I hacked up a script for this today to plot trace2 production data, as > noted there it's at: > https://github.com/av

Re: [PATCH] trace2: fix up a missing "leave" entry point

2019-05-10 Thread Derrick Stolee
On 5/10/2019 9:37 AM, Ævar Arnfjörð Bjarmason wrote: > Fix a trivial bug that's been here since the shared/do_write_index > tracing was added in 42fee7a388 ("trace2:data: add trace2 > instrumentation to index read/write", 2019-02-22). We should have > enter/leave points, not two enter/enter points.

Re: [PATCH 12/17] Documentation: describe split commit-graphs

2019-05-10 Thread Derrick Stolee
On 5/9/2019 5:45 PM, Ævar Arnfjörð Bjarmason wrote: > > On Thu, May 09 2019, Derrick Stolee wrote: > >> How far apart can these concurrency issues happen in the file system? >> One benefit to Option 0 is that there is only one file _write_ that matters. >> The other o

Re: [PATCH 14/14] verify-commit: simplify parameters to run_gpg_verify()

2019-05-10 Thread Derrick Stolee
On 5/9/2019 5:32 PM, Jeff King wrote: > Instead, let's do our type check by loading the object via > parse_object(). That will attach the buffer to the struct so it can be > used later by check_commit_signature(). And it ensures that > lookup_commit() will return something sane. This is a good ide

Re: [PATCH 0/14] "final" batch of unused parameter cleanups

2019-05-10 Thread Derrick Stolee
On 5/9/2019 5:25 PM, Jeff King wrote: > This is a continuation of my efforts to get us compiling with > -Wunused-parameter. This round finishes most of the cleanups and fixes > I intend to do (though I have a handful of cleanup cases that I'm still > poking at to make sure they're not in fact bugs

Re: [PATCH 03/14] builtin: consistently pass cmd_* prefix to parse_options

2019-05-10 Thread Derrick Stolee
On 5/9/2019 5:28 PM, Jeff King wrote: > If a builtin uses RUN_SETUP to request that git.c enter the repository > directory, we'll get passed in a "prefix" variable with the path to the > original directory. It's important to pass this to parse_options(), > since we may use it to fix up relative OP

Re: [PATCH 01/14] cmd_{read,write}_tree: rename "unused" variable that is used

2019-05-10 Thread Derrick Stolee
On 5/9/2019 5:27 PM, Jeff King wrote: > I kind of hate "cmd_prefix"; I was tempted to just call it "prefix" so > that all of the cmd_* functions were consistent, but I worried that it > really would get confused with the local variables (even if those > variables are renamed, as I do here). cmd_pr

Re: [PATCH 12/17] Documentation: describe split commit-graphs

2019-05-09 Thread Derrick Stolee
On 5/9/2019 11:48 AM, Ævar Arnfjörð Bjarmason wrote: > > On Thu, May 09 2019, Derrick Stolee wrote: > [snip] >> >> I still don't think this is a problem, with the idea below. >> >>>> 4. It would be helpful to add a new optional chunk that contains th

[PATCH v4 10/11] commit-graph: extract copy_oids_to_commits()

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract copy_oids_to_commits(), which fills the commits list with the distinct commits from the oids list. During this loop, it also counts the number of "extra" edges fr

[PATCH v4 06/11] commit-graph: extract fill_oids_from_packs()

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. This extracts fill_oids_from_packs() that reads the given pack-file list and fills the oid list in the context. Signed-off-by: Derrick Stolee --- commit-graph.c | 83

[PATCH v4 05/11] commit-graph: create write_commit_graph_context

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too large and complex. To simplify it, we should extract several small methods. However, we will risk repeating a lot of declarations related to progress incidators and object id or commit lists. Create a new write_commit_graph_context

[PATCH v4 08/11] commit-graph: extract fill_oids_from_all_packs()

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract fill_oids_from_all_packs() that reads all pack-files for commits and fills the oid list in the context. Signed-off-by: Derrick Stolee --- commit-graph.c | 26

[PATCH v4 07/11] commit-graph: extract fill_oids_from_commit_hex()

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract fill_oids_from_commit_hex() that reads the given commit id list and fille the oid list in the context. Signed-off-by: Derrick Stolee --- commit-graph.c | 72

[PATCH v4 11/11] commit-graph: extract write_commit_graph_file()

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract write_commit_graph_file() that takes all of the information in the context struct and writes the data to a commit-graph file. Signed-off-by: Derrick Stolee --- commit-graph.c

[PATCH v4 09/11] commit-graph: extract count_distinct_commits()

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract count_distinct_commits(), which sorts the oids list, then iterates through to find duplicates. Signed-off-by: Derrick Stolee --- commit-graph.c | 35

[PATCH v4 04/11] commit-graph: remove Future Work section

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph feature began with a long list of planned benefits, most of which are now complete. The future work section has only a few items left. As for making more algorithms aware of generation numbers, some are only waiting for generation number v2 to ensure the

[PATCH v4 03/11] commit-graph: collapse parameters into flags

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() and write_commit_graph_reachable() methods currently take two boolean parameters: 'append' and 'report_progress'. We will soon expand the possible options to send to these methods, so instead of complicating the parameter lis

[PATCH v4 01/11] commit-graph: fix the_repository reference

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The parse_commit_buffer() method takes a repository pointer, so it should not refer to the_repository anymore. Signed-off-by: Derrick Stolee --- commit.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commit.c b/commit.c index a5333c7ac6..e4d1233226

[PATCH v4 02/11] commit-graph: return with errors during write

2019-05-09 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method uses die() to report failure and exit when confronted with an unexpected condition. This use of die() in a library function is incorrect and is now replaced by error() statements and an int return type. Now that we use 'goto cleanup

[PATCH v4 00/11] Commit-graph write refactor (was: Create commit-graph file format v2)

2019-05-09 Thread Derrick Stolee via GitGitGadget
have missed. Thanks, -Stolee [0] https://public-inbox.org/git/pull.184.git.gitgitgad...@gmail.com/ [1] https://public-inbox.org/git/87o94mql0a@evledraar.gmail.com/ [2] https://public-inbox.org/git/87pnp2qlkv@evledraar.gmail.com/ Derrick Stolee (11): commit-graph: fix the_reposito

Re: [PATCH 17/17] fetch: add fetch.writeCommitGraph config setting

2019-05-09 Thread Derrick Stolee
On 5/9/2019 4:07 AM, Ævar Arnfjörð Bjarmason wrote: > > So rather than have this patch I'd like to as noted in 00/17 get the > refactoring bits of the commit-graph in first. Refactor-only series coming soon. > Then some version of my WIP patch in > https://public-inbox.org/git/87lfzprkfc@evl

Re: [PATCH 12/17] Documentation: describe split commit-graphs

2019-05-09 Thread Derrick Stolee
On 5/8/2019 4:11 PM, Ævar Arnfjörð Bjarmason wrote: > > On Wed, May 08 2019, Derrick Stolee wrote: > >> I'll ignore the improbability of this turn of events (two writes happening >> during the span of trying to read two files) and focus on the fact that >> we can p

Re: [PATCH 12/17] Documentation: describe split commit-graphs

2019-05-09 Thread Derrick Stolee
On 5/9/2019 12:49 AM, Junio C Hamano wrote: > Ævar Arnfjörð Bjarmason writes: > >> With commit-graph- all these unlink() race conditions go away, >> partial reads due to concurrent graph writing becomes a non-issue (we'd >> just leave the old files, "gc" deals with them later..), no need to >> ca

Re: [PATCH 01/19] revision.h: avoid bit fields in struct rev_info

2019-05-09 Thread Derrick Stolee
On 5/9/2019 5:56 AM, Duy Nguyen wrote: > On Wed, May 8, 2019 at 10:52 PM Derrick Stolee wrote: >> >> The biggest issue with my suggestion is that it requires changing the >> consumers of the options, as they would no longer live directly on the >> rev_info struct. That

Git Test Coverage Report (May 9)

2019-05-09 Thread Derrick Stolee
misor-remote: use repository_format_partial_clone Christian Couder54248706 Add initial support for many promisor remotes Christian Couder7b6e1b04 Move core_partial_clone_filter_default to promisor-remote.c Christian Couder0ba08c05 Remove fetch-object.{c,h} in favor of pro

Re: [PATCH] Doc: add missing backticks to pull.rebase value

2019-05-08 Thread Derrick Stolee
On 5/8/2019 2:43 PM, Denton Liu wrote: > In the documentation for pull.rebase, all of the valid values are > surrounded with backticks except for "true". Surround "true" with these > missing backticks. This would make the rendered docs [1] look a bit better, but I think this series is incomplete.

Re: [PATCH 12/17] Documentation: describe split commit-graphs

2019-05-08 Thread Derrick Stolee
On 5/8/2019 1:20 PM, SZEDER Gábor wrote: > On Wed, May 08, 2019 at 08:53:57AM -0700, Derrick Stolee via GitGitGadget > wrote: >> From: Derrick Stolee > Consider the following sequence of events: > > 1. There are three commit-graph files in the repository. > >

[PATCH 10/17] commit-graph: extract copy_oids_to_commits()

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract copy_oids_to_commits(), which fills the commits list with the distinct commits from the oids list. During this loop, it also counts the number of "extra" edges fr

[PATCH 15/17] commit-graph: write split commit-graph files

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- commit-graph.c | 248 +++- commit-graph.h | 1 + t/t5318-commit-graph.sh | 2 +- 3 files changed, 244 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c

[PATCH 16/17] commit-graph: add --split option

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- builtin/commit-graph.c | 10 +++--- t/t5318-commit-graph.sh | 26 ++ 2 files changed, 33 insertions(+), 3 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 828b1a713f..c2c07d3917

[PATCH 00/17] [RFC] Commit-graph: Write incremental files

2019-05-08 Thread Derrick Stolee via GitGitGadget
with --split, the append logic will trigger a merge with the current tip (at minimum). Some care should be taken to make this more clear. Thanks, -Stolee [1] https://github.com/git/git/commit/43d356180556180b4ef6ac232a14498a5bb2b446 commit-graph write: don't die if the ex

[PATCH 05/17] commit-graph: create write_commit_graph_context

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too large and complex. To simplify it, we should extract several small methods. However, we will risk repeating a lot of declarations related to progress incidators and object id or commit lists. Create a new write_commit_graph_context

[PATCH 12/17] Documentation: describe split commit-graphs

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The design for the split commit-graphs uses file names to force a "stack" of commit-graph files. This allows incremental writes without updating the file format. Signed-off-by: Derrick Stolee --- Documentation/technical/commit-graph.txt | 142

[PATCH 03/17] commit-graph: collapse parameters into flags

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() and write_commit_graph_reachable() methods currently take two boolean parameters: 'append' and 'report_progress'. We will soon expand the possible options to send to these methods, so instead of complicating the parameter lis

[PATCH 02/17] commit-graph: return with errors during write

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method uses die() to report failure and exit when confronted with an unexpected condition. This use of die() in a library function is incorrect and is now replaced by error() statements and an int return type. Now that we use 'goto cleanup

[PATCH 09/17] commit-graph: extract count_distinct_commits()

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract count_distinct_commits(), which sorts the oids list, then iterates through to find duplicates. Signed-off-by: Derrick Stolee --- commit-graph.c | 35

[PATCH 08/17] commit-graph: extract fill_oids_from_all_packs()

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract fill_oids_from_all_packs() that reads all pack-files for commits and fills the oid list in the context. Signed-off-by: Derrick Stolee --- commit-graph.c | 26

[PATCH 04/17] commit-graph: remove Future Work section

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The commit-graph feature began with a long list of planned benefits, most of which are now complete. The future work section has only a few items left. As for making more algorithms aware of generation numbers, some are only waiting for generation number v2 to ensure the

[PATCH 11/17] commit-graph: extract write_commit_graph_file()

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract write_commit_graph_file() that takes all of the information in the context struct and writes the data to a commit-graph file. Signed-off-by: Derrick Stolee --- commit-graph.c

[PATCH 06/17] commit-graph: extract fill_oids_from_packs()

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. This extracts fill_oids_from_packs() that reads the given pack-file list and fills the oid list in the context. Signed-off-by: Derrick Stolee --- commit-graph.c | 83

[PATCH 07/17] commit-graph: extract fill_oids_from_commit_hex()

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee The write_commit_graph() method is too complex, so we are extracting methods one by one. Extract fill_oids_from_commit_hex() that reads the given commit id list and fille the oid list in the context. Signed-off-by: Derrick Stolee --- commit-graph.c | 72

[PATCH 13/17] commit-graph: lay groundwork for incremental files

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- commit-graph.c | 66 -- commit-graph.h | 4 +++ 2 files changed, 62 insertions(+), 8 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 7723156964..f790f44a9c 100644 --- a

[PATCH 17/17] fetch: add fetch.writeCommitGraph config setting

2019-05-08 Thread Derrick Stolee via GitGitGadget
From: Derrick Stolee Signed-off-by: Derrick Stolee --- builtin/fetch.c | 17 + 1 file changed, 17 insertions(+) diff --git a/builtin/fetch.c b/builtin/fetch.c index b620fd54b4..cf0944bad5 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -23,6 +23,7 @@ #include

<    1   2   3   4   5   6   7   8   9   10   >