[PATCH v3 12/16] index-helper: kill mode

Add a new command (and command-line arg) to allow index-helpers to exit cleanly. This is mainly useful for tests. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 31 ++- t/t7900-index-helper.sh | 13 + 2 files c

[PATCH v3 04/16] index-helper: add --strict

From: Nguyễn Thái Ngọc Duy There are "holes" in the index-helper approach because the shared memory is not verified again by git. If $USER is compromised, shared memory could be modified. But anyone who could do this could already modify $GIT_DIR/index. A more realistic risk

[PATCH v3 08/16] Add watchman support to reduce index refresh cost

From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> The previous patch has the logic to clear bits in 'WAMA' bitmap. This patch has logic to set bits as told by watchman. The missing bit, _using_ these bits, are not here yet. A lot of this code is written by David Turner originally, mostly f

[PATCH v3 07/16] read-cache: add watchman 'WAMA' extension

From: Nguyễn Thái Ngọc Duy The extension contains a bitmap, one bit for each entry in the index. If the n-th bit is zero, the n-th entry is considered unchanged, we can ce_mark_uptodate() it without refreshing. If the bit is non-zero and we found out the corresponding file is

[PATCH v3 10/16] update-index: enable/disable watchman support

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/update-index.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 1c94ca5..7c08e1c 100644 ---

[PATCH v3 06/16] index-helper: add --detach

From: Nguyễn Thái Ngọc Duy We detach after creating and opening the socket, because otherwise we might return control to the shell before index-helper is ready to accept commands. This might lead to flaky tests. Signed-off-by: Nguyễn Thái Ngọc Duy ---

[PATCH v3 00/16] index-helper, watchman

moved/squashed at Duy's suggestion (some of?) my spelling errors corrected David Turner (6): unpack-trees: preserve index extensions index-helper: kill mode index-helper: don't run if already running index-helper: autorun mode index-helper: optionally automatically run read-cache: config

[PATCH v3 02/16] read-cache: allow to keep mmap'd memory after reading

From: Nguyễn Thái Ngọc Duy Later, we will introduce git index-helper to share this memory with other git processes. Since the memory will be shared, it will never be unmapped (although the kernel may of course choose to page it out). Signed-off-by: Nguyễn Thái Ngọc Duy

[PATCH v3 03/16] index-helper: new daemon for caching index and related stuff

x_from .git/index [1] http://thread.gmane.org/gmane.comp.version-control.git/247268/focus=248771 Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- .gitignore | 1 + Documentation/git-index-helper.txt | 46 ++

[PATCH v3 01/16] read-cache.c: fix constness of verify_hdr()

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/read-cache.c b/read-cache.c index d9fb78b..16cc487 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1345,7

Re: [PATCH 00/21] replacement for dt/refs-backend-lmdb v7 patch 04/33

On Thu, 2016-03-31 at 18:37 -0700, Stefan Beller wrote: > On Wed, Mar 30, 2016 at 1:05 PM, David Turner < > dtur...@twopensource.com> wrote: > > On Wed, 2016-03-30 at 08:37 +0200, Michael Haggerty wrote: > > > On 03/29/2016 10:12 PM, David Turner wrote: > > >

Re: [PATCH 00/21] replacement for dt/refs-backend-lmdb v7 patch 04/33

On Thu, 2016-03-31 at 18:14 +0200, Michael Haggerty wrote: > On 03/30/2016 10:05 PM, David Turner wrote: > > On Wed, 2016-03-30 at 08:37 +0200, Michael Haggerty wrote: > > > On 03/29/2016 10:12 PM, David Turner wrote: > > > > On Sun, 2016-03-27 at 07:22

Re: RFC: New reference iteration paradigm

On Thu, 2016-03-31 at 18:13 +0200, Michael Haggerty wrote: > Currently the way to iterate over references is via a family of > for_each_ref()-style functions. You pass some arguments plus a > callback > function and cb_data to the function, and your callback is called for > each reference that is

Re: [PATCH 00/21] replacement for dt/refs-backend-lmdb v7 patch 04/33

On Wed, 2016-03-30 at 08:37 +0200, Michael Haggerty wrote: > On 03/29/2016 10:12 PM, David Turner wrote: > > On Sun, 2016-03-27 at 07:22 +0200, Michael Haggerty wrote: > > > On 03/24/2016 07:47 AM, David Turner wrote: > > > > [...] > > > > I incor

Re: [PATCH v2 03/17] index-helper: new daemon for caching index and related stuff

On Tue, 2016-03-29 at 09:31 +0700, Duy Nguyen wrote: > On Sat, Mar 19, 2016 at 8:04 AM, David Turner < > dtur...@twopensource.com> wrote: > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > Instead of reading the index from disk and worrying about disk

Re: [PATCH 00/19] index-helper, watchman

On Tue, 2016-03-29 at 19:09 +0200, Torsten Bögershausen wrote: > On 2016-03-09 19.36, David Turner wrote: > > This is a rebase and extension of Duy's work on git index-helper > > and > > watchman support. > > > Somewhere we need to tweak something: > t7900

Re: [PATCH v2 08/17] read-cache: invalidate untracked cache data when reading WAMA

On Tue, 2016-03-29 at 09:50 +0700, Duy Nguyen wrote: > On Sat, Mar 19, 2016 at 8:04 AM, David Turner < > dtur...@twopensource.com> wrote: > > @@ -1407,10 +1472,24 @@ static int read_watchman_ext(struct > > index_state *istate, const void *data, > > ewah_eac

Re: [PATCH 00/21] replacement for dt/refs-backend-lmdb v7 patch 04/33

On Sun, 2016-03-27 at 07:22 +0200, Michael Haggerty wrote: > On 03/24/2016 07:47 AM, David Turner wrote: > > [...] > > I incorporated your changes into the lmdb backend. To make merging > > later more convenient, I rebased on top of pu -- I think this > > mainly > &g

Re: [PATCH v3 1/2] refs: add a new function set_worktree_head_symref

On Mon, 2016-03-28 at 10:48 -0700, Junio C Hamano wrote: > Kazuki Yamaguchi writes: > > > Add a new function set_worktree_head_symref, to update HEAD symref > > for > > the specified worktree. > > > > To update HEAD of a linked working tree, > >

Re: [PATCH v7 09/33] refs: reduce the visibility of do_for_each_ref()

On Thu, 2016-03-24 at 08:07 +0100, Michael Haggerty wrote: > > +/* > > + * The common backend for the for_each_*ref* functions > > + */ > > +static int do_for_each_ref(const char *submodule, const char > > *base, > > + each_ref_fn fn, int trim, int flags, > > + void

Re: [PATCH v7 14/33] refs: add methods to init refs db

On Thu, 2016-03-24 at 08:28 +0100, Michael Haggerty wrote: > > if (shared_repository) { > > adjust_shared_perm(get_git_dir()); > > - adjust_shared_perm(git_path_buf(, "refs")); > > Given that this function is creating the "refs" directory, it seems > like >

Re: [PATCH v2 09/17] Add watchman support to reduce index refresh cost

On Thu, 2016-03-24 at 09:47 -0400, Jeff Hostetler wrote: > I'm seeing wm->name have value ".git" rather than ".git/" on Linux. > > > On 03/18/2016 09:04 PM, David Turner wrote: > > + if (!strncmp(wm->name, ".git/&qu

Re: [PATCH 00/21] replacement for dt/refs-backend-lmdb v7 patch 04/33

On Wed, 2016-03-23 at 11:04 +0100, Michael Haggerty wrote: > Patch 04/33 in David Turner's refs-backend-lmdb series v7 [1] did way > too much in a single patch, and in fact got a few minor things wrong. > Instead of that patch, I suggest this patch series, which > ... LGTM. I think I would

Re: [PATCH] read-cache: increase write buffer size

On Sat, 2016-03-19 at 18:18 +0700, Duy Nguyen wrote: > On Sat, Mar 19, 2016 at 8:19 AM, David Turner < > dtur...@twopensource.com> wrote: > > Each write() has syscall overhead, and writing a large index > > entails > > many such calls. A larger write buffer red

Re: [PATCH 18/19] index-helper: autorun

On Thu, 2016-03-17 at 15:43 +0100, Johannes Schindelin wrote: > Hi Duy, > > On Thu, 17 Mar 2016, Duy Nguyen wrote: > > > On Thu, Mar 17, 2016 at 1:27 AM, Johannes Schindelin > > wrote: > > > I am much more concerned about concurrent accesses and the > > >

Re: [PATCH 19/19] hack: watchman/untracked cache mashup

On Thu, 2016-03-17 at 20:06 +0700, Duy Nguyen wrote: > On Thu, Mar 17, 2016 at 7:56 AM, David Turner < > dtur...@twopensource.com> wrote: > > > So if we detect an updated file that's not in the index, we are > > > prepared to invalidate that path, correct? We

Re: [PATCH 18/19] index-helper: autorun

On Fri, 2016-03-18 at 14:44 +0700, Duy Nguyen wrote: > > So yeah, this is the challenge: to make Git work at real-world > > scale > > (didn't we hear a lot about this at the latest Git Merge?) > > I'm all for making Junio cry by using Git for what it is (or was) not > intended for, but this seems

[PATCH] read-cache: increase write buffer size

Each write() has syscall overhead, and writing a large index entails many such calls. A larger write buffer reduces the overhead, leading to increased performance. On my repo, which has an index size of 30m, this saves about 10ms of time writing the index. Signed-off-by: David Turner <d

Re: [PATCH 19/19] hack: watchman/untracked cache mashup

On Tue, 2016-03-15 at 19:31 +0700, Duy Nguyen wrote: > On Thu, Mar 10, 2016 at 1:36 AM, David Turner < > dtur...@twopensource.com> wrote: > > static struct watchman_query *make_query(const char *last_update) > > @@ -60,8 +61,24 @@ static void update_index(struct

Re: [PATCH 18/19] index-helper: autorun

On Tue, 2016-03-15 at 15:26 +0100, Johannes Schindelin wrote: > Hi Duy, > > On Tue, 15 Mar 2016, Duy Nguyen wrote: > > > On Thu, Mar 10, 2016 at 1:36 AM, David Turner < > > dtur...@twopensource.com> wrote: > > > Introduce a new config option, indexhelper.au

[PATCH v2 12/17] unpack-trees: preserve index extensions

if the untracked cache were entirely invalidated by such an operation, the user has signaled their intention to have such a cache, and we don't want to throw it away. The same logic applies to the watchman state. Signed-off-by: David Turner <dtur...@twopensource.com> --- c

[PATCH v2 15/17] index-helper: autorun mode

Soon, we'll want to automatically start index-helper, so we need a mode that silently exists if it can't start up (either because it's not in a git dir, or because another one is already running). Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c

[PATCH v2 17/17] read-cache: config for waiting for index-helper

. Signed-off-by: David Turner <dtur...@twopensource.com> --- cache.h | 1 + config.c | 5 + environment.c | 5 + read-cache.c | 5 - 4 files changed, 15 insertions(+), 1 deletion(-) diff --git a/cache.h b/cache.h index 4ae7dd0..f8b8dbf 100644 --- a/cache.h +++ b/cache.h @@

[PATCH v2 14/17] index-helper: don't run if already running

Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 7 +++ t/t7900-index-helper.sh | 9 + 2 files changed, 16 insertions(+) diff --git a/index-helper.c b/index-helper.c index ffa3c2a..7362abb 100644 --- a/index-helper.c +++ b/index-helper.c @@

[PATCH v2 06/17] index-helper: add --detach

From: Nguyễn Thái Ngọc Duy We detach after creating and opening the named pipe, because otherwise we might return control to the shell before index-helper is ready to accept commands. This might lead to flaky tests. Signed-off-by: Nguyễn Thái Ngọc Duy ---

[PATCH v2 05/17] daemonize(): set a flag before exiting the main process

From: Nguyễn Thái Ngọc Duy This allows signal handlers and atexit functions to realize this situation and not clean up. Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/gc.c | 2 +- cache.h | 2 +- daemon.c | 2 +- setup.c | 4 +++- 4 files

[PATCH v2 03/17] index-helper: new daemon for caching index and related stuff

tingly with index v4, we get less out of index-helper. It makes sense as v4 requires more processing after loading the index: (vanilla) 0.72249 s: read_index_from .git/index (index-helper) 0.302741500 s: read_index_from .git/index Signed-off-by: Nguyễn Thái Ngọc Duy <pcl

[PATCH v2 00/15] index-helper, watchman

want to do that, and because it would lead to multiple readers on the same named pipe, which has undefined behavior. Just kill the old one if you're done with it. David Turner (7): read-cache: invalidate untracked cache data when reading WAMA unpack-trees: preserve index extensions index-helper: kill mo

Re: [PATCH] merge: refuse to create too cool a merge by default

On Fri, 2016-03-18 at 13:21 -0700, Junio C Hamano wrote: > Many tests that are updated by this patch does the > pass-through manually by turning: Nit: Should be many tests ... do Also, I didn't notice a test that shows that "cool" merges without allow-unrelated-histories are forbidden. -- To

[PATCH v2 16/17] index-helper: optionally automatically run

Introduce a new config option, indexhelper.autorun, to automatically run git index-helper before starting up a builtin git command. This enables users to keep index-helper running without manual intervention. Signed-off-by: David Turner <dtur...@twopensource.com> ---

[PATCH v2 13/17] index-helper: kill mode

Add a new command (and command-line arg) to allow index-helpers to exit cleanly. This is mainly useful for tests. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 39 +-- t/t7900-index-helper.sh | 9 + 2

[PATCH v2 10/17] index-helper: use watchman to avoid refreshing index with lstat()

ễn Thái Ngọc Duy gmail.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- Documentation/git-index-helper.txt | 6 ++ cache.h| 2 + dir.c | 12 index-helper.c | 128 ++

[PATCH v2 11/17] update-index: enable/disable watchman support

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/update-index.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 1c94ca5..7c08e1c 100644 ---

[PATCH v2 07/17] read-cache: add watchman 'WAMA' extension

From: Nguyễn Thái Ngọc Duy The extension contains a bitmap, one bit for each entry in the index. If the n-th bit is zero, the n-th entry is considered unchanged, we can ce_mark_uptodate() it without refreshing. If the bit is non-zero and we found out the corresponding file is

[PATCH v2 02/17] read-cache: allow to keep mmap'd memory after reading

From: Nguyễn Thái Ngọc Duy Later, we will introduce git index-helper to share this memory with other git processes. Since the memory will be shared, it will never be unmapped (although the kernel may of course choose to page it out). Signed-off-by: Nguyễn Thái Ngọc Duy

[PATCH v2 04/17] index-helper: add --strict

From: Nguyễn Thái Ngọc Duy There are "holes" in the index-helper approach because the shared memory is not verified again by git. If $USER is compromised, shared memory could be modified. But anyone who could do this could already modify $GIT_DIR/index. A more realistic risk

[PATCH v2 08/17] read-cache: invalidate untracked cache data when reading WAMA

When reading the watchman extension, invalidate the listed untracked-cache entries. We don't clear these entries yet; we can only do that once we know that they came from a live watchman (rather than from disk). Signed-off-by: David Turner <dtur...@twopensource.com> --- dir.c

[PATCH v2 09/17] Add watchman support to reduce index refresh cost

From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> The previous patch has the logic to clear bits in 'WAMA' bitmap. This patch has logic to set bits as told by watchman. The missing bit, _using_ these bits, are not here yet. A lot of this code is written by David Turner originally, mostly f

[PATCH v2 01/17] read-cache.c: fix constness of verify_hdr()

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/read-cache.c b/read-cache.c index d9fb78b..16cc487 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1345,7

Re: [PATCH 04/19] index-helper: new daemon for caching index and related stuff

On Thu, 2016-03-10 at 18:17 +0700, Duy Nguyen wrote: > On Thu, Mar 10, 2016 at 6:21 AM, Junio C Hamano <gits...@pobox.com> > wrote: > > Junio C Hamano <gits...@pobox.com> writes: > > > > > David Turner <dtur...@twopensource.com> writes: > > >

Re: [PATCH 04/19] index-helper: new daemon for caching index and related stuff

On Wed, 2016-03-09 at 15:09 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: > > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > Instead of reading the index from disk and worrying about disk > > corruption, the i

Re: [PATCH 03/19] read-cache: allow to keep mmap'd memory after reading

On Wed, 2016-03-09 at 15:02 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: > > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > Signed-off-by: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > --- > > As

Re: [PATCH 01/19] trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics

On Wed, 2016-03-09 at 14:58 -0800, Junio C Hamano wrote: > David Turner <dtur...@twopensource.com> writes: ... > > trace_stats() is intended for GIT_TRACE_*_STATS variable group and > > GIT_TRACE_PACK_STATS is more like an example how new vars can be > > added.

Re: [PATCH 04/19] index-helper: new daemon for caching index and related stuff

On Wed, 2016-03-09 at 15:21 -0800, Junio C Hamano wrote: > Junio C Hamano <gits...@pobox.com> writes: > > > David Turner <dtur...@twopensource.com> writes: > > > > > From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> > > > > > >

[PATCH 00/19] index-helper, watchman

but they provide a benefit for us, so we still use them. David Turner (5): unpack-trees: preserve index extensions index-helper: rewrite pidfile after daemonizing index-helper: process management index-helper: autorun hack: watchman/untracked cache mashup Nguyễn Thái Ngọc Duy (14): trace.c: add

[PATCH 01/19] trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics

From: Nguyễn Thái Ngọc Duy trace_stats() is intended for GIT_TRACE_*_STATS variable group and GIT_TRACE_PACK_STATS is more like an example how new vars can be added. Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git.txt | 3 +++ cache.h

[PATCH 02/19] read-cache.c: fix constness of verify_hdr()

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/read-cache.c b/read-cache.c index d9fb78b..16cc487 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1345,7

[PATCH 03/19] read-cache: allow to keep mmap'd memory after reading

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- cache.h | 3 +++ read-cache.c | 13 - 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/cache.h b/cache.h index 7e01403..c43ef3d 100644 --- a/cache.h +++

[PATCH 19/19] hack: watchman/untracked cache mashup

to see if other folks think it's the right idea first. Signed-off-by: David Turner <dtur...@twopensource.com> --- dir.c | 23 +++-- dir.h | 6 +++ read-cache.c | 142 +++-- watchman-support.c | 21 +

[PATCH 15/19] unpack-trees: preserve index extensions

if the untracked cache were entirely invalidated by such an operation, the user has signaled their intention to have such a cache, and we don't want to throw it away. The same logic applies to the watchman state. Signed-off-by: David Turner <dtur...@twopensource.com> --- c

[PATCH 12/19] read-cache: allow index-helper to prepare shm before git reads it

<pclo...@gmail.com> Signed-off-by: David Turner <dtur...@twopensource.com> --- read-cache.c | 36 ++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/read-cache.c b/read-cache.c index 85ef15b..57c5df9 100644 --- a/read-cache.c +++ b/read-c

[PATCH 17/19] index-helper: process management

a test for index-helper's pid-file writing and for --kill. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 95 + t/t7900-index-helper.sh | 23 2 files changed, 111 insertions(+), 7 deletions(-)

[PATCH 14/19] update-index: enable/disable watchman support

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/update-index.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 1c94ca5..7c08e1c 100644 ---

[PATCH 18/19] index-helper: autorun

Introduce a new config option, indexhelper.autorun, to automatically run git index-helper before starting up a builtin git command. This enables users to keep index-helper running without manual intervention. Signed-off-by: David Turner <dtur...@twopensource.com> ---

[PATCH 05/19] trace.c: add GIT_TRACE_INDEX_STATS for index statistics

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git.txt | 1 + cache.h | 1 + read-cache.c | 16 trace.c | 5 - 4 files changed, 22 insertions(+), 1 deletion(-)

[PATCH 11/19] Add watchman support to reduce index refresh cost

From: Nguyễn Thái Ngọc Duy <pclo...@gmail.com> The previous patch has the logic to clear bits in 'WAMA' bitmap. This patch has logic to set bits as told by watchman. The missing bit, _using_ these bits, are not here yet. A lot of this code is written by David Turner originally, mostly f

[PATCH 07/19] daemonize(): set a flag before exiting the main process

From: Nguyễn Thái Ngọc Duy This allows signal handlers and atexit functions to realize this situation and not clean up. Signed-off-by: Nguyễn Thái Ngọc Duy --- builtin/gc.c | 2 +- cache.h | 2 +- daemon.c | 2 +- setup.c | 4 +++- 4 files

[PATCH 13/19] index-helper: use watchman to avoid refreshing index with lstat()

From: Nguyễn Thái Ngọc Duy Watchman is hidden behind index-helper. Before git tries to read the index from shm, it notifies index-helper with SIGHUP and sleep, waiting for index-helper to prepare shm. index-helper then contacts watchman, updates 'WAMA' extension and put it in

[PATCH 10/19] read-cache: add watchman 'WAMA' extension

From: Nguyễn Thái Ngọc Duy The extension contains a bitmap, one bit for each entry in the index. If the n-th bit is zero, the n-th entry is considered unchanged, we can ce_mark_uptodate() it without refreshing. If the bit is non-zero and we found out the corresponding file is

[PATCH 16/19] index-helper: rewrite pidfile after daemonizing

When we daemonize, our pid changes, so we need to rewrite the pid file. Signed-off-by: David Turner <dtur...@twopensource.com> --- index-helper.c | 48 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/index-helper.c b/index-he

[PATCH 09/19] index-helper: add Windows support

From: Nguyễn Thái Ngọc Duy Windows supports shared memory, but the semantics is a bit different than POSIX shm. The most noticeable thing is there's no way to get the shared memory's size by the reader, and wrapping fstat to do that would be hell. So the shm size is added near

[PATCH 08/19] index-helper: add --detach

From: Nguyễn Thái Ngọc Duy Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-index-helper.txt | 3 +++ index-helper.c | 10 -- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git

[PATCH 04/19] index-helper: new daemon for caching index and related stuff

From: Nguyễn Thái Ngọc Duy Instead of reading the index from disk and worrying about disk corruption, the index is cached in memory (memory bit-flips happen too, but hopefully less often). The result is faster read. Read time is reduced by 70%. The biggest gain is not having

[PATCH 06/19] index-helper: add --strict

From: Nguyễn Thái Ngọc Duy There's are "holes" in the index-helper approach because the shared memory is not verified again by git. If $USER is compromised, shared memory could be modified. But then they can already modify $GIT_DIR/index. A more realistic risk is some bugs in

Re: [PATCH 0/10] cleaning up check_repository_format_gently

On Tue, 2016-03-01 at 09:35 -0500, Jeff King wrote: > After the discussion in: > > http://thread.gmane.org/gmane.comp.version-control.git/287949/focus > =288002 > > I came up with this series to try to address some of the warts in > check_repository_format_gently(). > > I had hoped to come

Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs

On Tue, 2016-03-01 at 18:47 -0500, David Turner wrote: > On Tue, 2016-03-01 at 03:35 -0500, Jeff King wrote: > > On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote: > > > > > Usually, git calls some form of setup_git_directory at startup. > > >

Re: [PATCH 10/10] setup: drop GIT_REPO_VERSION constants

On Tue, 2016-03-01 at 09:45 -0500, Jeff King wrote: > - char repo_version_string[10]; > > /* This forces creation of new config file */ > - xsnprintf(repo_version_string, sizeof(repo_version_string), > - "%d", GIT_REPO_VERSION); > -

Re: [PATCH v7 01/33] setup: call setup_git_directory_gently before accessing refs

On Tue, 2016-03-01 at 03:35 -0500, Jeff King wrote: > On Mon, Feb 29, 2016 at 07:52:34PM -0500, David Turner wrote: > > > Usually, git calls some form of setup_git_directory at startup. > > But > > sometimes, it doesn't. Usually, that's OK because it's not really

Re: [PATCH 06/10] setup: refactor repo format reading and verification

On Tue, 2016-03-01 at 09:42 -0500, Jeff King wrote: > +/* > + * Read the repository format characteristics from the config file > "path". If > + * the version cannot be extracted from the file for any reason, the > version > + * field will be set to -1, and all other fields are undefined. > + */ >

Re: [PATCH v7 29/33] setup: configure ref storage on setup

On Tue, 2016-03-01 at 17:18 +, Ramsay Jones wrote: > > On 01/03/16 00:53, David Turner wrote: > > This sets up the existing backend early, so that other code which > > reads refs is reading from the right place. > > > > Signed-off-by: David Turner <dtur...

Re: [PATCH v7 30/33] refs: break out resolve_ref_unsafe_submodule

On Tue, 2016-03-01 at 17:21 +, Ramsay Jones wrote: > > On 01/03/16 00:53, David Turner wrote: > > It will soon be useful for resolve_ref_unsafe to support > > submodules. > > But since it is called from so many places, changing it would have > > been painful.

Re: [PATCH v7 31/33] refs: add LMDB refs storage backend

On Tue, 2016-03-01 at 08:31 +0700, Duy Nguyen wrote: > On Tue, Mar 1, 2016 at 7:53 AM, David Turner < > dtur...@twopensource.com> wrote: > > +Weaknesses: > > +--- > > + > > +The reflog format is somewhat inefficient: a binary format could > &g

[PATCH v7 09/33] refs: reduce the visibility of do_for_each_ref()

From: Ramsay Jones Now that we have moved do_for_each_ref into refs.c, it no longer needs to be public. Signed-off-by: Ramsay Jones Signed-off-by: Junio C Hamano --- refs.c | 19 +++

[PATCH v7 28/33] refs: register ref storage backends

Add new function register_ref_storage_backends(). This new function registers all known ref storage backends... once there are any other than the default. For now, it just registers the files backend. Call the function before calling set_ref_storage_backend. Signed-off-by: David Turner <d

[PATCH v7 33/33] tests: add ref-storage argument

in the right place, git doesn't recognized the trash dir as a git repo, so no git commands work. * t1430-bad-ref-name specifically blocks lmdb because other alternate backends might want to keep this test. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano

[PATCH v7 08/33] refs: add method for do_for_each_ref

Add a ref backend method for do_for_each_ref. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs.c | 8 refs/files-backend.c | 7 +-- refs/refs-internal.h | 5 + 3 files changed, 18 inse

[PATCH v7 32/33] refs: tests for lmdb backend

Add tests for the database backend. Signed-off-by: David Turner <dtur...@twopensource.com> Helped-by: Dennis Kaarsemaker <den...@kaarsemaker.net> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- t/t1460-refs-lmdb-backend.sh| 1109 +++

[PATCH v7 25/33] refs: check submodules' ref storage config

All submodules must have the same ref storage (for now). Confirm that this is so before attempting to do anything with submodule refs. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs.c

[PATCH v7 13/33] refs: add method for delete_refs

In the file-based backend, delete_refs has some special optimization to deal with packed refs. In other backends, we might be able to make ref deletion faster by putting all deletions into a single transaction. So we need a special backend function for this. Signed-off-by: David Turner <d

[PATCH v7 21/33] refs: on symref reflog expire, lock symref not referrent

When locking a symbolic ref to expire a reflog, lock the symbolic ref (using REF_NODEREF) instead of its referent. Add a test for this. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs/files-backend.c | 3 ++- t/t14

[PATCH v7 17/33] refs: make lock generic

Instead of using a files-backend-specific struct ref_lock, the generic ref_transaction struct should provide a void pointer that backends can use for their own lock data. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> ---

[PATCH v7 14/33] refs: add methods to init refs db

Alternate refs backends might not need the refs/heads directory and so on, so we make ref db initialization part of the backend. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- builtin/init-db.c| 20 ++--

[PATCH v7 19/33] refs: allow log-only updates

The refs infrastructure learns about log-only ref updates, which only update the reflog. Later, we will use this to separate symbolic reference resolution from ref updating. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> ---

[PATCH v7 22/33] refs: resolve symbolic refs first

REF_NODEREF. And ref_transaction_commit only happens on already-derefernced refs. So we can assume REF_NODEREF when resolving inside this function. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs.c

[PATCH v7 23/33] refs: always handle non-normal refs in files backend

). Updates to normal refs continue to go through the chosen backend. Updates to non-normal refs are moved to a separate files backend transaction. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> ---

[PATCH v7 20/33] refs: don't dereference on rename

the pointed-to ref name for the moved reflog. Add a test for the reflog bug. Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs/files-backend.c | 15 +-- t/t3200-branch.sh| 9 + 2 files changed, 1

[PATCH v7 10/33] refs: add do_for_each_per_worktree_ref

Alternate refs backends might still use files to store per-worktree refs. So the files backend's ref-loading infrastructure should be available to those backends, just for use on per-worktree refs. Add do_for_each_per_worktree_ref, which iterates over per-worktree refs. Signed-off-by: David

[PATCH v7 12/33] refs: add method for initial ref transaction commit

Signed-off-by: Ronnie Sahlberg <rsahlb...@google.com> Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs.c | 6 ++ refs/files-backend.c | 5 +++-- refs/refs-internal.h | 1 + 3 files changed, 1

[PATCH v7 15/33] refs: add method to rename refs

Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> --- refs.c | 5 + refs/files-backend.c | 4 +++- refs/refs-internal.h | 8 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/refs.c b/

[PATCH v7 06/33] refs: add a backend method structure with transaction functions

ed-off-by: Ronnie Sahlberg <sahlb...@google.com> Signed-off-by: David Turner <dtur...@twopensource.com> Signed-off-by: Junio C Hamano <gits...@pobox.com> Signed-off-by: Jeff King <p...@peff.net> --- refs.c | 40 refs.h

<    1   2   3   4   5   6   7   8   9   10   >