[PATCH v3] unpack-trees: fix accidentally quadratic behavior

, number of changes, and packfile packing decisions. Signed-off-by: David Turner <dtur...@twopensource.com> --- A colleague pointed out that the info->prev test was unnecessary as ce_in_traverse_path already does that. --- unpack-trees.c | 13 - 1 file changed, 12 insertions(+), 1

Re: [PATCH] unpack-trees: fix accidentally quadratic behavior

On Wed, 2016-01-20 at 20:58 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: > > > While unpacking trees (e.g. during git checkout), when we hit a > > cache > > entry that's past and outside our path, we cut off iteration. > > &

Re: [PATCH v4 20/21] refs: add LMDB refs storage backend

On Fri, 2016-02-12 at 18:01 +0100, Michael Haggerty wrote: > On 02/05/2016 08:44 PM, David Turner wrote: > > Add a database backend for refs using LMDB. This backend runs git > > for-each-ref about 30% faster than the files backend with fully > > -packed > > refs on a

Re: [PATCH v4 20/21] refs: add LMDB refs storage backend

On Mon, 2016-02-15 at 16:57 +0700, Duy Nguyen wrote: > On Sun, Feb 14, 2016 at 7:04 PM, Duy Nguyen <pclo...@gmail.com> > wrote: > > On Sat, Feb 6, 2016 at 2:44 AM, David Turner < > > dtur...@twopensource.com> wrote: > > > +static char *get_refdb_path(const ch

Re: [PATCH v4 20/21] refs: add LMDB refs storage backend

On Sun, 2016-02-14 at 19:04 +0700, Duy Nguyen wrote: > On Sat, Feb 6, 2016 at 2:44 AM, David Turner < > dtur...@twopensource.com> wrote: > > +static char *get_refdb_path(const char *base) > > +{ > > + static struct strbuf path_buf = STRBUF_INIT;

Re: [PATCH v4 19/21] refs: add register_ref_storage_backends()

On Fri, 2016-02-12 at 16:42 +0100, Michael Haggerty wrote: > On 02/05/2016 08:44 PM, David Turner wrote: > > This new function will register all known ref storage backends... > > once > > there are any other than the default. For now, it's a no-op. > > > >

Re: [PATCH v4 15/21] init: allow alternate ref strorage to be set for new repos

On Fri, 2016-02-12 at 16:26 +0100, Michael Haggerty wrote: > On 02/05/2016 08:44 PM, David Turner wrote: > > git init learns a new argument --ref-storage. Presently, only > > "files" is supported, but later we will add other storage backends. > > > > When th

Re: [PATCH v4 13/21] refs: resolve symbolic refs first

On Fri, 201-02-12 at 15:09 +0100, Michael Haggerty wrote:] > On 02/05/2016 08:44 PM, David Turner wrote: > > Before committing ref updates, split symbolic ref updates into two > > parts: an update to the underlying ref, and a log-only update to > > the > > symbolic r

Re: [PATCH v4 14/21] refs: always handle non-normal refs in files backend

On Fri, 2016-02-12 at 16:07 +0100, Michael Haggerty wrote: > On 02/05/2016 08:44 PM, David Turner wrote: > > Always handle non-normal (per-worktree or pseudo) refs in the files > > backend instead of alternate backends. > > > > Sometimes a ref transaction will upd

[PATCH v5 01/27] refs: Move head_ref{,_submodule} to the common code

These don't use any backend-specific functions. These were previously defined in terms of the do_head_ref helper function, but since they are otherwise identical, we don't need that function. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c

[PATCH v5 00/27] refs backends

s everyone to run the tests under lmdb easily. David Turner (25): refs: Move head_ref{,_submodule} to the common code refs: move for_each_*ref* functions into common code refs: add method for do_for_each_ref refs: add do_for_each_per_worktree_ref refs: add methods for reflog refs:

[PATCH v5 13/27] refs: make lock generic

Instead of using a files-backend-specific struct ref_lock, the generic ref_transaction struct should provide a void pointer that backends can use for their own lock data. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs/files-backend.c | 29 - ref

[PATCH v5 17/27] refs: on symref reflog expire, lock symref not referrent

When locking a symbolic ref to expire a reflog, lock the symbolic ref (using REF_NODEREF) instead of its referent. Add a test for this. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs/files-backend.c | 3 ++- t/t1410-reflog.sh| 10 ++ 2 files changed, 12 inse

[PATCH v5 10/27] refs: add methods to init refs db

Alternate refs backends might not need the refs/heads directory and so on, so we make ref db initialization part of the backend. Signed-off-by: David Turner <dtur...@twopensource.com> --- builtin/init-db.c| 20 ++-- refs.c | 5 + refs.h

[PATCH v5 15/27] refs: allow log-only updates

The refs infrastructure learns about log-only ref updates, which only update the reflog. Later, we will use this to separate symbolic reference resolution from ref updating. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs/files-backend.c | 15 ++- ref

[PATCH v5 08/27] refs: add method for initial ref transaction commit

Signed-off-by: Ronnie Sahlberg <rsahlb...@google.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 6 ++ refs/files-backend.c | 5 +++-- refs/refs-internal.h | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/refs.c b/

[PATCH v5 14/27] refs: move duplicate check to common code

The check for duplicate refnames in a transaction is needed for all backends, so move it to the common code. ref_transaction_commit_fn gains a new argument, the sorted string_list of affected refnames. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c

[PATCH v5 19/27] refs: always handle non-normal refs in files backend

). Updates to normal refs continue to go through the chosen backend. Updates to non-normal refs are moved to a separate files backend transaction. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 110 + 1 file chang

[PATCH v5 04/27] refs: add methods for misc ref operations

From: Ronnie Sahlberg <sahlb...@google.com> Add ref backend methods for: resolve_ref_unsafe, verify_refname_available, pack_refs, peel_ref, create_symref, resolve_gitlink_ref. Signed-off-by: Ronnie Sahlberg <sahlb...@google.com> Signed-off-by: David Turner <dtur...@twopensource.c

[PATCH v5 09/27] refs: add method for delete_refs

In the file-based backend, delete_refs has some special optimization to deal with packed refs. In other backends, we might be able to make ref deletion faster by putting all deletions into a single transaction. So we need a special backend function for this. Signed-off-by: David Turner <d

[PATCH v5 16/27] refs: don't dereference on rename

the pointed-to ref name for the moved reflog. Add a test for the reflog bug. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs/files-backend.c | 15 +-- t/t3200-branch.sh| 9 + 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/refs/files-bac

[PATCH v5 02/27] refs: move for_each_*ref* functions into common code

Make do_for_each_ref take a submodule as an argument instead of a ref_cache. Since all for_each_*ref* functions are defined in terms of do_for_each_ref, we can then move them into the common code. Later, we can simply make do_for_each_ref into a backend function. Signed-off-by: David Turner

[PATCH v5 06/27] refs: add do_for_each_per_worktree_ref

Alternate refs backends might still use files to store per-worktree refs. So the files backend's ref-loading infrastructure should be available to those backends, just for use on per-worktree refs. Add do_for_each_per_worktree_ref, which iterates over per-worktree refs. Signed-off-by: David

[PATCH v5 21/27] refs: check submodules' ref storage config

All submodules must have the same ref storage (for now). Confirm that this is so before attempting to do anything with submodule refs. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 54

[PATCH v5 25/27] refs: add LMDB refs storage backend

, are skipped), the only remaining failing tests are the git-new-workdir tests and the gitweb tests. Signed-off-by: David Turner <dtur...@twopensource.com> --- .gitignore |1 + Documentation/config.txt |9 + Documentation/git-clo

[PATCH v5 07/27] refs: add methods for reflog

In the file-based backend, the reflog piggybacks on the ref lock. Since other backends won't have the same sort of ref lock, ref backends must also handle reflogs. Signed-off-by: Ronnie Sahlberg <rsahlb...@google.com> Signed-off-by: David Turner <dtur...@twopensource.com>

[PATCH v5 11/27] refs: add method to rename refs

Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 5 + refs/files-backend.c | 4 +++- refs/refs-internal.h | 8 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/refs.c b/refs.c index e222d02..f5754f2 100644 --- a/refs.c +++ b/

[PATCH v5 24/27] refs: add register_ref_storage_backends()

This new function will register all known ref storage backends... once there are any other than the default. For now, it's a no-op. Signed-off-by: David Turner <dtur...@twopensource.com> --- builtin/init-db.c | 3 +++ config.c | 25 + refs.c

[PATCH v5 20/27] init: allow alternate ref strorage to be set for new repos

backend's initdb function is used to set up the ref database. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: SZEDER Gábor <sze...@ira.uka.de> --- Documentation/git-init-db.txt | 2 +- Documentation/git-init.txt | 7 +++

[PATCH v5 26/27] refs: tests for lmdb backend

Add tests for the database backend. Signed-off-by: David Turner <dtur...@twopensource.com> Helped-by: Dennis Kaarsemaker <den...@kaarsemaker.net> --- t/t1460-refs-lmdb-backend.sh| 1109 +++ t/t1470-refs-lmdb-backend-reflog.sh | 359 +

[PATCH v5 27/27] tests: add ref-storage argument

are the following: * The rearrangement of commands in t1401 is because without HEAD in the right place, git doesn't recognized the trash dir as a git repo, so no git commands work. * t1430-bad-ref-name specifically blocks lmdb because other alternate backends might want to keep this test. Signed-off-by: David

[PATCH v5 23/27] svn: learn ref-storage argument

git svn learns to pass the ref-storage command-line argument (to init and clone) through to git init. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: SZEDER Gábor <sze...@ira.uka.de> --- contrib/completion/git-completion.bash | 2 +-

[PATCH v5 18/27] refs: resolve symbolic refs first

REF_NODEREF. And ref_transaction_commit only happens on already-derefernced refs. So we can assume REF_NODEREF when resolving inside this function. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 69 ++ refs/files-backend.c

[PATCH v5 22/27] clone: allow ref storage backend to be set for clone

Add a new option, --ref-storage, to allow the ref storage backend to be set on new clones. Submodules must use the same ref storage as the parent repository, so we also pass the --ref-storage option option when cloning submodules. Signed-off-by: David Turner <dtur...@twopensource.com> Sign

[PATCH v5 12/27] refs: forbid cross-backend ref renames

This would be pretty weird, but since it will break, we should prevent it. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/refs.c b/refs.c index f5754f2..8eb04da 100644 --- a/refs.c +++ b/refs.c @@ -1306,5 +1

[PATCH v5 03/27] refs: add a backend method structure with transaction functions

ed-off-by: Ronnie Sahlberg <sahlb...@google.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 40 refs.h | 7 +++ refs/files-backend.c | 10 -- refs/refs-internal.h | 14 +- 4

[PATCH v5 05/27] refs: add method for do_for_each_ref

Add a ref backend method for do_for_each_ref. Signed-off-by: David Turner <dtur...@twopensource.com> --- refs.c | 8 refs/files-backend.c | 7 +-- refs/refs-internal.h | 5 + 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/refs.c b/refs.c

Re: [PATCH v4 14/21] refs: always handle non-normal refs in files backend

On Thu, 2016-02-18 at 13:07 +0100, Michael Haggerty wrote: > On 02/18/2016 03:44 AM, David Turner wrote: > > On Fri, 2016-02-12 at 16:07 +0100, Michael Haggerty wrote: > > > On 02/05/2016 08:44 PM, David Turner wrote: > > > > Always handle non-normal

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

On Thu, 2016-02-18 at 15:50 +0700, Duy Nguyen wrote: [snip] Thanks; applied the above > This permission makes me wonder if we need adjust_shared_perm() here > and some other places. So just add this after every mkdir? if (shared_repository) adjust_shared_perm(db_path);

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

On Fri, 2016-02-19 at 09:54 +0700, Duy Nguyen wrote: > On Fri, Feb 19, 2016 at 3:23 AM, David Turner < > dtur...@twopensource.com> wrote: > > > > +static int read_per_worktree_ref(const char *submodule, const > > > > char > > > > *refname, > >

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

> + *flags |= REF_ISBROKEN; > > + } > > + return refname; > > + } > > + if (flags) > > + *flags |= REF_ISSYMREF; > > + buf = ref_data + 4

Re: [PATCH v7 30/33] refs: break out resolve_ref_unsafe_submodule

On Tue, 2016-03-01 at 17:21 +, Ramsay Jones wrote: > > On 01/03/16 00:53, David Turner wrote: > > It will soon be useful for resolve_ref_unsafe to support > > submodules. > > But since it is called from so many places, changing it would have > > been painful.

Re: [PATCH v7 29/33] setup: configure ref storage on setup

On Tue, 2016-03-01 at 17:18 +, Ramsay Jones wrote: > > On 01/03/16 00:53, David Turner wrote: > > This sets up the existing backend early, so that other code which > > reads refs is reading from the right place. > > > > Signed-off-by: David Turner <dtur...

Re: [PATCH v7 31/33] refs: add LMDB refs storage backend

On Tue, 2016-03-01 at 08:31 +0700, Duy Nguyen wrote: > On Tue, Mar 1, 2016 at 7:53 AM, David Turner < > dtur...@twopensource.com> wrote: > > +Weaknesses: > > +--- > > + > > +The reflog format is somewhat inefficient: a binary format could > &g

Re: [PATCH 10/10] setup: drop GIT_REPO_VERSION constants

On Tue, 2016-03-01 at 09:45 -0500, Jeff King wrote: > - char repo_version_string[10]; > > /* This forces creation of new config file */ > - xsnprintf(repo_version_string, sizeof(repo_version_string), > - "%d", GIT_REPO_VERSION); > -

Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs

On Tue, 2016-03-01 at 18:47 -0500, David Turner wrote: > On Tue, 2016-03-01 at 03:35 -0500, Jeff King wrote: > > On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote: > > > > > Usually, git calls some form of setup_git_directory at startup. > > >

Re: [PATCH 0/10] cleaning up check_repository_format_gently

On Tue, 2016-03-01 at 09:35 -0500, Jeff King wrote: > After the discussion in: > > http://thread.gmane.org/gmane.comp.version-control.git/287949/focus > =288002 > > I came up with this series to try to address some of the warts in > check_repository_format_gently(). > > I had hoped to come

Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs

On Tue, 2016-03-01 at 03:35 -0500, Jeff King wrote: > On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote: > > > Usually, git calls some form of setup_git_directory at startup. > > But > > sometimes, it doesn't. Usually, that's OK because it's not really

Re: [PATCH 06/10] setup: refactor repo format reading and verification

On Tue, 2016-03-01 at 09:42 -0500, Jeff King wrote: > +/* > + * Read the repository format characteristics from the config file > "path". If > + * the version cannot be extracted from the file for any reason, the > version > + * field will be set to -1, and all other fields are undefined. > + */ >

[PATCH 04/19] index-helper: new daemon for caching index and related stuff

From: Nguyễn Thái Ngọc Duy Instead of reading the index from disk and worrying about disk corruption, the index is cached in memory (memory bit-flips happen too, but hopefully less often). The result is faster read. Read time is reduced by 70%. The biggest gain is not having

[PATCH 06/19] index-helper: add --strict

From: Nguyễn Thái Ngọc Duy There's are "holes" in the index-helper approach because the shared memory is not verified again by git. If $USER is compromised, shared memory could be modified. But then they can already modify $GIT_DIR/index. A more realistic risk is some bugs in

Re: [PATCH 04/19] index-helper: new daemon for caching index and related stuff

On Thu, 2016-03-10 at 18:17 +0700, Duy Nguyen wrote: > On Thu, Mar 10, 2016 at 6:21 AM, Junio C Hamano <gits...@pobox.com> > wrote: > > Junio C Hamano <gits...@pobox.com> writes: > > > > > David Turner <dtur...@twopensource.com> writes: > > >

[PATCH v2 17/17] read-cache: config for waiting for index-helper

. Signed-off-by: David Turner <dtur...@twopensource.com> --- cache.h | 1 + config.c | 5 + environment.c | 5 + read-cache.c | 5 - 4 files changed, 15 insertions(+), 1 deletion(-) diff --git a/cache.h b/cache.h index 4ae7dd0..f8b8dbf 100644 --- a/cache.h +++ b/cache.h @@

[PATCH v2 14/17] index-helper: don't run if already running

Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 7 +++ t/t7900-index-helper.sh | 9 + 2 files changed, 16 insertions(+) diff --git a/index-helper.c b/index-helper.c index ffa3c2a..7362abb 100644 --- a/index-helper.c +++ b/index-helper.c @@

[PATCH v2 12/17] unpack-trees: preserve index extensions

if the untracked cache were entirely invalidated by such an operation, the user has signaled their intention to have such a cache, and we don't want to throw it away. The same logic applies to the watchman state. Signed-off-by: David Turner <dtur...@twopensource.com> --- c

[PATCH v2 15/17] index-helper: autorun mode

Soon, we'll want to automatically start index-helper, so we need a mode that silently exists if it can't start up (either because it's not in a git dir, or because another one is already running). Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c

[PATCH v2 09/17] Add watchman support to reduce index refresh cost

From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> The previous patch has the logic to clear bits in 'WAMA' bitmap. This patch has logic to set bits as told by watchman. The missing bit, _using_ these bits, are not here yet. A lot of this code is written by David Turner originally, mostly f

[PATCH v2 01/17] read-cache.c: fix constness of verify_hdr()

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/read-cache.c b/read-cache.c index d9fb78b..16cc487 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1345,7

[PATCH v2 08/17] read-cache: invalidate untracked cache data when reading WAMA

When reading the watchman extension, invalidate the listed untracked-cache entries. We don't clear these entries yet; we can only do that once we know that they came from a live watchman (rather than from disk). Signed-off-by: David Turner <dtur...@twopensource.com> --- dir.c

[PATCH v2 02/17] read-cache: allow to keep mmap'd memory after reading

From: Nguyễn Thái Ngọc Duy Later, we will introduce git index-helper to share this memory with other git processes. Since the memory will be shared, it will never be unmapped (although the kernel may of course choose to page it out). Signed-off-by: Nguyễn Thái Ngọc Duy

[PATCH v2 04/17] index-helper: add --strict

From: Nguyễn Thái Ngọc Duy There are "holes" in the index-helper approach because the shared memory is not verified again by git. If $USER is compromised, shared memory could be modified. But anyone who could do this could already modify $GIT_DIR/index. A more realistic risk

[PATCH v2 07/17] read-cache: add watchman 'WAMA' extension

From: Nguyễn Thái Ngọc Duy The extension contains a bitmap, one bit for each entry in the index. If the n-th bit is zero, the n-th entry is considered unchanged, we can ce_mark_uptodate() it without refreshing. If the bit is non-zero and we found out the corresponding file is

[PATCH v2 10/17] index-helper: use watchman to avoid refreshing index with lstat()

ễn Thái Ngọc Duy gmail.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- Documentation/git-index-helper.txt | 6 ++ cache.h| 2 + dir.c | 12 index-helper.c | 128 ++

[PATCH v2 11/17] update-index: enable/disable watchman support

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/update-index.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 1c94ca5..7c08e1c 100644 ---

[PATCH v2 13/17] index-helper: kill mode

Add a new command (and command-line arg) to allow index-helpers to exit cleanly. This is mainly useful for tests. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 39 +-- t/t7900-index-helper.sh | 9 + 2

[PATCH v2 16/17] index-helper: optionally automatically run

Introduce a new config option, indexhelper.autorun, to automatically run git index-helper before starting up a builtin git command. This enables users to keep index-helper running without manual intervention. Signed-off-by: David Turner <dtur...@twopensource.com> ---

Re: [PATCH] merge: refuse to create too cool a merge by default

On Fri, 2016-03-18 at 13:21 -0700, Junio C Hamano wrote: > Many tests that are updated by this patch does the > pass-through manually by turning: Nit: Should be many tests ... do Also, I didn't notice a test that shows that "cool" merges without allow-unrelated-histories are forbidden. -- To

[PATCH v2 06/17] index-helper: add --detach

From: Nguyễn Thái Ngọc Duy We detach after creating and opening the named pipe, because otherwise we might return control to the shell before index-helper is ready to accept commands. This might lead to flaky tests. Signed-off-by: Nguyễn Thái Ngọc Duy ---

[PATCH v2 05/17] daemonize(): set a flag before exiting the main process

From: Nguyễn Thái Ngọc Duy This allows signal handlers and atexit functions to realize this situation and not clean up. Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/gc.c | 2 +- cache.h | 2 +- daemon.c | 2 +- setup.c | 4 +++- 4 files

[PATCH v2 03/17] index-helper: new daemon for caching index and related stuff

tingly with index v4, we get less out of index-helper. It makes sense as v4 requires more processing after loading the index: (vanilla) 0.72249 s: read_index_from .git/index (index-helper) 0.302741500 s: read_index_from .git/index Signed-off-by: Nguyễn Thái Ngọc Duy <pcl

[PATCH v2 00/15] index-helper, watchman

want to do that, and because it would lead to multiple readers on the same named pipe, which has undefined behavior. Just kill the old one if you're done with it. David Turner (7): read-cache: invalidate untracked cache data when reading WAMA unpack-trees: preserve index extensions index-helper: kill mo

[PATCH 03/19] read-cache: allow to keep mmap'd memory after reading

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- cache.h | 3 +++ read-cache.c | 13 - 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/cache.h b/cache.h index 7e01403..c43ef3d 100644 --- a/cache.h +++

[PATCH 02/19] read-cache.c: fix constness of verify_hdr()

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/read-cache.c b/read-cache.c index d9fb78b..16cc487 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1345,7

[PATCH 01/19] trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics

From: Nguyễn Thái Ngọc Duy trace_stats() is intended for GIT_TRACE_*_STATS variable group and GIT_TRACE_PACK_STATS is more like an example how new vars can be added. Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git.txt | 3 +++ cache.h

[PATCH 00/19] index-helper, watchman

but they provide a benefit for us, so we still use them. David Turner (5): unpack-trees: preserve index extensions index-helper: rewrite pidfile after daemonizing index-helper: process management index-helper: autorun hack: watchman/untracked cache mashup Nguyễn Thái Ngọc Duy (14): trace.c: add

[PATCH 19/19] hack: watchman/untracked cache mashup

to see if other folks think it's the right idea first. Signed-off-by: David Turner <dtur...@twopensource.com> --- dir.c | 23 +++-- dir.h | 6 +++ read-cache.c | 142 +++-- watchman-support.c | 21 +

[PATCH 09/19] index-helper: add Windows support

From: Nguyễn Thái Ngọc Duy Windows supports shared memory, but the semantics is a bit different than POSIX shm. The most noticeable thing is there's no way to get the shared memory's size by the reader, and wrapping fstat to do that would be hell. So the shm size is added near

[PATCH 15/19] unpack-trees: preserve index extensions

if the untracked cache were entirely invalidated by such an operation, the user has signaled their intention to have such a cache, and we don't want to throw it away. The same logic applies to the watchman state. Signed-off-by: David Turner <dtur...@twopensource.com> --- c

[PATCH 12/19] read-cache: allow index-helper to prepare shm before git reads it

<pclo...@gmail.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- read-cache.c | 36 ++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/read-cache.c b/read-cache.c index 85ef15b..57c5df9 100644 --- a/read-cache.c +++ b/read-c

[PATCH 17/19] index-helper: process management

a test for index-helper's pid-file writing and for --kill. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 95 + t/t7900-index-helper.sh | 23 2 files changed, 111 insertions(+), 7 deletions(-)

[PATCH 14/19] update-index: enable/disable watchman support

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/update-index.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 1c94ca5..7c08e1c 100644 ---

[PATCH 18/19] index-helper: autorun

Introduce a new config option, indexhelper.autorun, to automatically run git index-helper before starting up a builtin git command. This enables users to keep index-helper running without manual intervention. Signed-off-by: David Turner <dtur...@twopensource.com> ---

[PATCH 05/19] trace.c: add GIT_TRACE_INDEX_STATS for index statistics

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git.txt | 1 + cache.h | 1 + read-cache.c | 16 trace.c | 5 - 4 files changed, 22 insertions(+), 1 deletion(-)

[PATCH 11/19] Add watchman support to reduce index refresh cost

From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> The previous patch has the logic to clear bits in 'WAMA' bitmap. This patch has logic to set bits as told by watchman. The missing bit, _using_ these bits, are not here yet. A lot of this code is written by David Turner originally, mostly f

[PATCH 07/19] daemonize(): set a flag before exiting the main process

From: Nguyễn Thái Ngọc Duy This allows signal handlers and atexit functions to realize this situation and not clean up. Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/gc.c | 2 +- cache.h | 2 +- daemon.c | 2 +- setup.c | 4 +++- 4 files

[PATCH 08/19] index-helper: add --detach

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-index-helper.txt | 3 +++ index-helper.c | 10 -- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git

[PATCH 13/19] index-helper: use watchman to avoid refreshing index with lstat()

From: Nguyễn Thái Ngọc Duy Watchman is hidden behind index-helper. Before git tries to read the index from shm, it notifies index-helper with SIGHUP and sleep, waiting for index-helper to prepare shm. index-helper then contacts watchman, updates 'WAMA' extension and put it in

[PATCH 10/19] read-cache: add watchman 'WAMA' extension

From: Nguyễn Thái Ngọc Duy The extension contains a bitmap, one bit for each entry in the index. If the n-th bit is zero, the n-th entry is considered unchanged, we can ce_mark_uptodate() it without refreshing. If the bit is non-zero and we found out the corresponding file is

[PATCH 16/19] index-helper: rewrite pidfile after daemonizing

When we daemonize, our pid changes, so we need to rewrite the pid file. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 48 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/index-helper.c b/index-he

Re: [PATCH 18/19] index-helper: autorun

On Tue, 2016-03-15 at 15:26 +0100, Johannes Schindelin wrote: > Hi Duy, > > On Tue, 15 Mar 2016, Duy Nguyen wrote: > > > On Thu, Mar 10, 2016 at 1:36 AM, David Turner < > > dtur...@twopensource.com> wrote: > > > Introduce a new config option, indexhelper.au

Re: [PATCH 19/19] hack: watchman/untracked cache mashup

On Tue, 2016-03-15 at 19:31 +0700, Duy Nguyen wrote: > On Thu, Mar 10, 2016 at 1:36 AM, David Turner < > dtur...@twopensource.com> wrote: > > static struct watchman_query *make_query(const char *last_update) > > @@ -60,8 +61,24 @@ static void update_index(struct

[PATCH] read-cache: increase write buffer size

Each write() has syscall overhead, and writing a large index entails many such calls. A larger write buffer reduces the overhead, leading to increased performance. On my repo, which has an index size of 30m, this saves about 10ms of time writing the index. Signed-off-by: David Turner <d

Re: [PATCH 18/19] index-helper: autorun

On Fri, 2016-03-18 at 14:44 +0700, Duy Nguyen wrote: > > So yeah, this is the challenge: to make Git work at real-world > > scale > > (didn't we hear a lot about this at the latest Git Merge?) > > I'm all for making Junio cry by using Git for what it is (or was) not > intended for, but this seems

Re: [PATCH 04/19] index-helper: new daemon for caching index and related stuff

On Wed, 2016-03-09 at 15:21 -0800, Junio C Hamano wrote: > Junio C Hamano <gits...@pobox.com> writes: > > > David Turner <dtur...@twopensource.com> writes: > > > > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > > >

Re: [PATCH 01/19] trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics

On Wed, 2016-03-09 at 14:58 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: ... > > trace_stats() is intended for GIT_TRACE_*_STATS variable group and > > GIT_TRACE_PACK_STATS is more like an example how new vars can be > > added.

Re: [PATCH 03/19] read-cache: allow to keep mmap'd memory after reading

On Wed, 2016-03-09 at 15:02 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: > > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > --- > > As

Re: [PATCH 04/19] index-helper: new daemon for caching index and related stuff

On Wed, 2016-03-09 at 15:09 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: > > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > Instead of reading the index from disk and worrying about disk > > corruption, the i

Re: [PATCH 00/21] replacement for dt/refs-backend-lmdb v7 patch 04/33

On Wed, 2016-03-30 at 08:37 +0200, Michael Haggerty wrote: > On 03/29/2016 10:12 PM, David Turner wrote: > > On Sun, 2016-03-27 at 07:22 +0200, Michael Haggerty wrote: > > > On 03/24/2016 07:47 AM, David Turner wrote: > > > > [...] > > > > I incor

[PATCH 15/24] read_raw_ref(): manage own scratch space

From: Michael Haggerty Instead of creating scratch space in resolve_ref_unsafe() and passing it down through resolve_ref_1 to read_raw_ref(), teach read_raw_ref() to manage its own scratch space. This reduces coupling across the functions at the cost of some extra

[PATCH 00/24] Yet another pre-refs-backend series

going to be harder to move and can wait until after the vtable. I have reviewed all of Michael's patches. David Turner (5): refs: move head_ref{,_submodule} to the common code refs: move for_each_*ref* functions into common code files-backend: break out ref reading refs: move

<    5   6   7   8   9   10   11   12   13   14   >