Re: Subject: Something like cat-file for the index?

2014-01-10 Thread Thomas Gummerer
Hi, Enno Weichert enno.weich...@gmail.com writes: Hi, I'd like to have a more technical look into the index file and what/how it stores data; call it educational spelunking. I know the index-format.txt but I'd really like to save me the work to implement a pretty-printed output based on

Re: [PATCH/WIP v2 02/14] read-cache: new extension to mark what file is watched

2014-01-17 Thread Thomas Gummerer
Nguyễn Thái Ngọc Duy pclo...@gmail.com writes: If an entry is watched, git lets an external program decide if the entry is modified or not. It's more like --assume-unchanged, but designed to be controlled by machine. We are running out of on-disk ce_flags, so instead of extending on-disk

[PATCH 0/3] Wider exposure for index-v4

2014-02-15 Thread Thomas Gummerer
the environment or the config variables are set. I'm not sure about the precedence in patch 3, right now the environment variable has precedence, but it should be easy to give the config option precedence over that. Thomas Gummerer (3): introduce GIT_INDEX_VERSION environment variable test-lib: allow

[PATCH 2/3] test-lib: allow setting the index format version

2014-02-15 Thread Thomas Gummerer
version under which t2104 is run to 3. This test only tests functionality specific to version 2 and 3 of the index file and would fail if the test suite is run with any other version. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile | 7 +++ t/t2104

[PATCH 1/3] introduce GIT_INDEX_VERSION environment variable

2014-02-15 Thread Thomas Gummerer
. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/git.txt | 5 + read-cache.c | 18 +- t/t1600-index.sh | 24 3 files changed, 46 insertions(+), 1 deletion(-) create mode 100755 t/t1600-index.sh diff --git

[PATCH 3/3] read-cache: add index.version config variable

2014-02-15 Thread Thomas Gummerer
Add a config variable that allows setting the default index version when initializing a new index file. Similar to the GIT_INDEX_VERSION environment variable this only affects new index files. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/config.txt | 4 read

Re: [PATCH 0/3] Wider exposure for index-v4

2014-02-16 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sun, Feb 16, 2014 at 2:23 AM, Thomas Gummerer t.gumme...@gmail.com wrote: Hi, since index-v5 didn't seem to generate enough interest to be merged, I I thought there were some comments last time that you were going to address and resubmit? Yes

Re: [PATCH 1/3] introduce GIT_INDEX_VERSION environment variable

2014-02-21 Thread Thomas Gummerer
Junio C Hamano gits...@pobox.com writes: Thomas Gummerer t.gumme...@gmail.com writes: diff --git a/Documentation/git.txt b/Documentation/git.txt index aec3726..bc9eeea 100644 --- a/Documentation/git.txt +++ b/Documentation/git.txt @@ -712,6 +712,11 @@ Git so take care if using Cogito etc

[PATCH v2 0/3] Easier access to index-v4

2014-02-23 Thread Thomas Gummerer
. Thomas Gummerer (3): introduce GIT_INDEX_VERSION environment variable test-lib: allow setting the index format version read-cache: add index.version config variable Documentation/config.txt | 4 ++ Documentation/git.txt | 5 +++ Makefile

[PATCH v2 3/3] read-cache: add index.version config variable

2014-02-23 Thread Thomas Gummerer
Add a config variable that allows setting the default index version when initializing a new index file. Similar to the GIT_INDEX_VERSION environment variable this only affects new index files. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/config.txt | 4 read

[PATCH v2 2/3] test-lib: allow setting the index format version

2014-02-23 Thread Thomas Gummerer
version under which t2104 is run to 3. This test only tests functionality specific to version 2 and 3 of the index file and would fail if the test suite is run with any other version. Helped-by: Junio C Hamano gits...@pobox.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile

[PATCH v2 1/3] introduce GIT_INDEX_VERSION environment variable

2014-02-23 Thread Thomas Gummerer
. Helped-by: Junio C Hamano gits...@pobox.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/git.txt | 5 + read-cache.c | 21 - t/t1600-index.sh | 49 + 3 files changed, 74 insertions(+), 1

Re: [PATCH v2 0/3] Easier access to index-v4

2014-02-24 Thread Thomas Gummerer
Junio C Hamano gits...@pobox.com writes: Junio C Hamano gits...@pobox.com writes: Thomas Gummerer t.gumme...@gmail.com writes: previous round was at $gmane/242198. Since then I've squashed the fixes suggested by Junio, added a test showing what should happen if an index file is present

Re: [PATCH 5.5/22] Add documentation for the index api

2013-07-11 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Wed, Jul 10, 2013 at 3:10 AM, Thomas Gummerer t.gumme...@gmail.com wrote: If you happen to know that certain entries match the given pathspec, you could help the caller avoid match_pathspec'ing again by set a bit in ce_flags. I currently don't know

Re: [PATCH 13/22] documentation: add documentation of the index-v5 file format

2013-07-11 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sun, Jul 7, 2013 at 3:11 PM, Thomas Gummerer t.gumme...@gmail.com wrote: +== File entry (fileentries) + + File entries are sorted in ascending order on the name field, after the + respective offset given by the directory entries. All file names

Re: [PATCH 13/22] documentation: add documentation of the index-v5 file format

2013-07-11 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Thu, Jul 11, 2013 at 6:39 PM, Thomas Gummerer t.gumme...@gmail.com wrote: Question about the possibility of updating index file directly. If git updates a few fields of an entry (but not entrycrc yet) and crashes, the entry would become corrupt because

[PATCH v2 01/19] t2104: Don't fail for index versions other than [23]

2013-07-12 Thread Thomas Gummerer
-worktree flags for the default index format is still covered by t7011 and t7012. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- t/t2104-update-index-skip-worktree.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/t/t2104-update-index-skip-worktree.sh b/t/t2104-update-index-skip

[PATCH v2 04/19] read-cache: Re-read index if index file changed

2013-07-12 Thread Thomas Gummerer
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 91 +--- 1 file changed, 57 insertions(+), 34 deletions(-) diff --git a/read-cache.c b/read-cache.c index 1e7ffc2..3e3a0e2 100644 --- a/read-cache.c +++ b/read-cache.c

[PATCH v2 03/19] read-cache: move index v2 specific functions to their own file

2013-07-12 Thread Thomas Gummerer
Move index version 2 specific functions to their own file. The non-index specific functions will be in read-cache.c, while the index version 2 specific functions will be in read-cache-v2.c. Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com

[PATCH v2 05/19] Add documentation for the index api

2013-07-12 Thread Thomas Gummerer
Add documentation for the index reading api. This also includes documentation for the new api functions introduced in the next patch. Helped-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/api-in-core-index.txt | 54

[PATCH v2 06/19] read-cache: add index reading api

2013-07-12 Thread Thomas Gummerer
functions for iterating over and accessing cache entries. This simplifies future improvements to the in-memory format, as changes will be concentrated on one file, instead of the whole git source code. Helped-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com

[PATCH v2 07/19] make sure partially read index is not changed

2013-07-12 Thread Thomas Gummerer
part of the index file instead of re-reading it when changing it, gives a bit of a performance advantage, by avoiding to read parts of the index twice. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/update-index.c | 4 cache.h| 1 + read-cache-v2.c| 2

[PATCH v2 08/19] grep.c: Use index api

2013-07-12 Thread Thomas Gummerer
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/grep.c | 71 ++ 1 file changed, 37 insertions(+), 34 deletions(-) diff --git a/builtin/grep.c b/builtin/grep.c index a419cda..8b02644 100644 --- a/builtin/grep.c +++ b/builtin

[PATCH v2 09/19] ls-files.c: use index api

2013-07-12 Thread Thomas Gummerer
Use the index api to read only part of the index, if the on-disk version of the index is index-v5. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/ls-files.c | 31 --- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/builtin/ls-files.c b

[PATCH v2 11/19] read-cache: make in-memory format aware of stat_crc

2013-07-12 Thread Thomas Gummerer
Make the in-memory format aware of the stat_crc used by index-v5. It is simply ignored by index version prior to v5. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h | 1 + read-cache.c | 25 + 2 files changed, 26 insertions(+) diff --git a/cache.h b

[PATCH v2 10/19] documentation: add documentation of the index-v5 file format

2013-07-12 Thread Thomas Gummerer
robin.rosenb...@dewire.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/index-file-format-v5.txt | 296 +++ 1 file changed, 296 insertions(+) create mode 100644 Documentation/technical/index-file-format-v5.txt diff --git a/Documentation

[PATCH v2 14/19] read-cache: read cache-tree in index-v5

2013-07-12 Thread Thomas Gummerer
is sorted lexically. So we have to rebuild this format from the on-disk directory list. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache-tree.c| 2 +- cache-tree.h| 6 read-cache-v5.c | 100 3 files changed, 107

[PATCH v2 13/19] read-cache: read resolve-undo data

2013-07-12 Thread Thomas Gummerer
-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache-v5.c | 39 +++ 1 file changed, 39 insertions(+) diff --git a/read-cache-v5.c b/read-cache-v5.c index 00112ea..853b97d 100644 --- a/read-cache-v5.c +++ b/read-cache

[PATCH v2 12/19] read-cache: read index-v5

2013-07-12 Thread Thomas Gummerer
...@gmail.com Helped-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile| 1 + cache.h | 75 ++- read-cache-v5.c | 638 read-cache.h| 1 + 4 files changed, 714 insertions

[PATCH v2 16/19] read-cache: write index-v5 cache-tree data

2013-07-12 Thread Thomas Gummerer
Write the cache-tree data for the index version 5 file format. The in-memory cache-tree data is converted to the ondisk format, by adding it to the directory entries, that were compiled from the cache-entries in the step before. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache

[PATCH v2 18/19] update-index.c: rewrite index when index-version is given

2013-07-12 Thread Thomas Gummerer
Make update-index always rewrite the index when a index-version is given, even if the index already has the right version. This option is used for performance testing the writer and reader. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/update-index.c | 4 ++-- 1 file changed, 2

[PATCH v2 17/19] read-cache: write resolve-undo data for index-v5

2013-07-12 Thread Thomas Gummerer
-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache-v5.c | 94 + 1 file changed, 94 insertions(+) diff --git a/read-cache-v5.c b/read-cache-v5.c index cd819b4..093ee1a 100644 --- a/read-cache-v5.c

[PATCH v2 15/19] read-cache: write index-v5

2013-07-12 Thread Thomas Gummerer
. Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com Helped-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h | 8 + read-cache-v5.c | 594 +++- read-cache.c| 11 +- read-cache.h

[PATCH v2 19/19] p0003-index.sh: add perf test for the index formats

2013-07-12 Thread Thomas Gummerer
From: Thomas Rast tr...@inf.ethz.ch Add a performance test for index version [23]/4/5 by using git update-index --index-version=x, thus testing both the reader and the writer speed of all index formats. Signed-off-by: Thomas Rast tr...@inf.ethz.ch Signed-off-by: Thomas Gummerer t.gumme

Re: [PATCH v2 00/19] Index-v5

2013-07-15 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: t/perf/p0003-index.sh| 59 + t/t2104-update-index-skip-worktree.sh|1 + For such a big code addition, the test part seems modest

Re: [PATCH v2 08/19] grep.c: Use index api

2013-07-15 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +static int grep_cache(struct cache_entry *ce, void *cb_data) { - int hit = 0; - int nr; - read_cache(); + struct grep_opts *opts = cb_data

Re: [PATCH v2 00/19] Index-v5

2013-07-17 Thread Thomas Gummerer
Ramsay Jones ram...@ramsay1.demon.co.uk writes: Thomas Gummerer wrote: Hi, previous rounds (without api) are at $gmane/202752, $gmane/202923, $gmane/203088 and $gmane/203517, the previous round with api was at $gmane/229732. Thanks to Junio, Duy and Eric for their comments on the previous

Re: [PATCH v2 09/19] ls-files.c: use index api

2013-07-17 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: + if (!with_tree) { + memset(opts, 0, sizeof(*opts)); + opts-pathspec = pathspec_struct; + opts-read_staged = 1

Re: [PATCH v2 10/19] documentation: add documentation of the index-v5 file format

2013-07-17 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +== Directory offsets (diroffsets) + + diroffset (32-bits): offset to the directory relative to the beginning +of the index file. There are ndir + 1 offsets

Re: [PATCH v2 12/19] read-cache: read index-v5

2013-07-17 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: [..] +static int read_entries(struct index_state *istate, struct directory_entry **de, + unsigned int *entry_offset, void **mmap, + unsigned long mmap_size, unsigned int *nr, + unsigned

Re: [PATCH v2 00/19] Index-v5

2013-07-17 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Mon, Jul 15, 2013 at 4:30 PM, Thomas Gummerer t.gumme...@gmail.com wrote: Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: t/perf/p0003-index.sh| 59

Re: [PATCH v2 07/19] make sure partially read index is not changed

2013-07-17 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: A partially read index file currently cannot be written to disk. Make sure that never happens, by erroring out when a caller tries to change a partially read index

Re: [PATCH v2 03/19] read-cache: move index v2 specific functions to their own file

2013-07-19 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: @@ -489,8 +479,8 @@ extern void *read_blob_data_from_index(struct index_state *, const char *, unsig #define CE_MATCH_RACY_IS_DIRTY 02 /* do stat comparison even

Re: [PATCH v2 00/19] Index-v5

2013-07-19 Thread Thomas Gummerer
Junio C Hamano gits...@pobox.com writes: Thomas Gummerer t.gumme...@gmail.com writes: Ah ok, I understand. I think it's best to add a GIT_INDEX_VERSION=x config option to config.mak, where x is the index version that should be tested. Whatever you do, please do not call

Re: [PATCH v2 10/19] documentation: add documentation of the index-v5 file format

2013-08-04 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +== Header + sig (32-bits): Signature: + The signature is { 'D', 'I', 'R', 'C' } (stands for dircache) + + vnr (32-bits): Version number: + The current supported

Re: [PATCH v2 12/19] read-cache: read index-v5

2013-08-07 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +struct directory_entry { + struct directory_entry *next; + struct directory_entry *next_hash; + struct cache_entry *ce; + struct cache_entry *ce_last

Re: [PATCH v2 12/19] read-cache: read index-v5

2013-08-07 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: A little bit more.. On Sat, Jul 13, 2013 at 12:26 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +static void ce_queue_push(struct cache_entry **head, +struct cache_entry **tail, +struct

Re: [PATCH v2 12/19] read-cache: read index-v5

2013-08-08 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Wed, Jul 17, 2013 at 3:11 PM, Thomas Gummerer t.gumme...@gmail.com wrote: Duy Nguyen pclo...@gmail.com writes: [..] +static int read_entries(struct index_state *istate, struct directory_entry **de, + unsigned int *entry_offset

Re: [PATCH v2 12/19] read-cache: read index-v5

2013-08-09 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Wed, Jul 17, 2013 at 3:11 PM, Thomas Gummerer t.gumme...@gmail.com wrote: Duy Nguyen pclo...@gmail.com writes: [..] +static int read_entries(struct index_state *istate, struct directory_entry **de, + unsigned int *entry_offset

[PATCH v3 00/24] Index-v5

2013-08-18 Thread Thomas Gummerer
and ino on cygwin - introduce GIT_INDEX_VERSION environment variable - test-lib: allow setting the index format version Thomas Gummerer (23): t2104: Don't fail for index versions other than [23] read-cache: use fixed width integer types read-cache: split index file version specific

[PATCH v3 06/24] read-cache: Don't compare uid, gid and ino on cygwin

2013-08-18 Thread Thomas Gummerer
Cygwin doesn't have uid, gid and ino stats fields. Therefore we should never check them in the match_stat_data when working on the CYGWIN platform. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- This patch was not tested on Cygwin yet. I think it's needed though, because the re

[PATCH v3 05/24] read-cache: move index v2 specific functions to their own file

2013-08-18 Thread Thomas Gummerer
Move index version 2 specific functions to their own file. The non-index specific functions will be in read-cache.c, while the index version 2 specific functions will be in read-cache-v2.c. Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com

[PATCH v3 03/24] read-cache: split index file version specific functionality

2013-08-18 Thread Thomas Gummerer
Split index file version specific functionality to their own functions, to prepare for moving the index file version specific parts to their own file. This makes it easier to add a new index file format later. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 114

[PATCH v3 04/24] read-cache: clear version in discard_index()

2013-08-18 Thread Thomas Gummerer
All fields except index_state-version are reset in discard_index. Reset the version too. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 1 + 1 file changed, 1 insertion(+) diff --git a/read-cache.c b/read-cache.c index de0bbcd..1e22f6f 100644 --- a/read-cache.c +++ b

[PATCH v3 01/24] t2104: Don't fail for index versions other than [23]

2013-08-18 Thread Thomas Gummerer
-worktree flags for the default index format is still covered by t7011 and t7012. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- t/t2104-update-index-skip-worktree.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/t/t2104-update-index-skip-worktree.sh b/t/t2104-update-index-skip

[PATCH v3 02/24] read-cache: use fixed width integer types

2013-08-18 Thread Thomas Gummerer
Use the fixed width integer types uint16_t and uint32_t for ondisk structures, because unsigned short and unsigned int do not hae a guaranteed size. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h | 10 +- read-cache.c | 30 +++--- 2 files

[PATCH v3 23/24] introduce GIT_INDEX_VERSION environment variable

2013-08-18 Thread Thomas Gummerer
Respect a GIT_INDEX_VERSION environment variable, when a new index is initialized. Setting the environment variable will not cause existing index files to be converted to another format for additional safety. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 9 +++-- 1

[PATCH v3 18/24] read-cache: write index-v5

2013-08-18 Thread Thomas Gummerer
. Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com Helped-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h | 2 + read-cache-v5.c | 596 +++- read-cache.c| 4 +- read-cache.h

[PATCH v3 14/24] read-cache: make in-memory format aware of stat_crc

2013-08-18 Thread Thomas Gummerer
Make the in-memory format aware of the stat_crc used by index-v5. It is simply ignored by index version prior to v5. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h | 1 + read-cache.c | 25 + 2 files changed, 26 insertions(+) diff --git a/cache.h b

[PATCH v3 22/24] p0003-index.sh: add perf test for the index formats

2013-08-18 Thread Thomas Gummerer
From: Thomas Rast tr...@inf.ethz.ch Add a performance test for index version [23]/4/5 by using git update-index --index-version=x, thus testing both the reader and the writer speed of all index formats. Signed-off-by: Thomas Rast tr...@inf.ethz.ch Signed-off-by: Thomas Gummerer t.gumme

[PATCH v3 12/24] ls-files.c: use index api

2013-08-18 Thread Thomas Gummerer
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/ls-files.c | 36 +--- 1 file changed, 33 insertions(+), 3 deletions(-) diff --git a/builtin/ls-files.c b/builtin/ls-files.c index bebc9c2..fbf9c47 100644 --- a/builtin/ls-files.c +++ b/builtin/ls

[PATCH v3 10/24] make sure partially read index is not changed

2013-08-18 Thread Thomas Gummerer
. Forcing the caller to load the right part of the index file instead of re-reading it when changing it, gives a bit of a performance advantage, by avoiding to read parts of the index twice. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 4 1 file changed, 4 insertions

[PATCH v3 16/24] read-cache: read resolve-undo data

2013-08-18 Thread Thomas Gummerer
-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache-v5.c | 130 1 file changed, 130 insertions(+) diff --git a/read-cache-v5.c b/read-cache-v5.c index 799b8e7..85a2069 100644 --- a/read-cache-v5

[PATCH v3 17/24] read-cache: read cache-tree in index-v5

2013-08-18 Thread Thomas Gummerer
is sorted lexically. So we have to rebuild this format from the on-disk directory list. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache-tree.c| 2 +- cache-tree.h| 1 + read-cache-v5.c | 78 + 3 files changed, 80 insertions

[PATCH v3 07/24] read-cache: Re-read index if index file changed

2013-08-18 Thread Thomas Gummerer
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 65 +++- 1 file changed, 34 insertions(+), 31 deletions(-) diff --git a/read-cache.c b/read-cache.c index aa17ce7..2d12601 100644 --- a/read-cache.c +++ b/read-cache.c

[PATCH v3 13/24] documentation: add documentation of the index-v5 file format

2013-08-18 Thread Thomas Gummerer
robin.rosenb...@dewire.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/index-file-format-v5.txt | 301 +++ 1 file changed, 301 insertions(+) create mode 100644 Documentation/technical/index-file-format-v5.txt diff --git a/Documentation

[PATCH v3 19/24] read-cache: write index-v5 cache-tree data

2013-08-18 Thread Thomas Gummerer
Write the cache-tree data for the index version 5 file format. The in-memory cache-tree data is converted to the ondisk format, by adding it to the directory entries, that were compiled from the cache-entries in the step before. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache

[PATCH v3 15/24] read-cache: read index-v5

2013-08-18 Thread Thomas Gummerer
...@gmail.com Helped-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile| 1 + cache.h | 32 +++- read-cache-v5.c | 473 read-cache.h| 1 + 4 files changed, 506 insertions(+), 1

[PATCH v3 21/24] update-index.c: rewrite index when index-version is given

2013-08-18 Thread Thomas Gummerer
Make update-index always rewrite the index when a index-version is given, even if the index already has the right version. This option is used for performance testing the writer and reader. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/update-index.c | 4 ++-- 1 file changed, 2

[PATCH v3 24/24] test-lib: allow setting the index format version

2013-08-18 Thread Thomas Gummerer
version 3). Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile| 7 +++ t/test-lib-functions.sh | 5 + t/test-lib.sh | 3 +++ 3 files changed, 15 insertions(+) diff --git a/Makefile b/Makefile index a55206d..ecae6b8 100644 --- a/Makefile +++ b/Makefile

[PATCH v3 20/24] read-cache: write resolve-undo data for index-v5

2013-08-18 Thread Thomas Gummerer
-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache-v5.c | 89 + 1 file changed, 89 insertions(+) diff --git a/read-cache-v5.c b/read-cache-v5.c index ed52b7c..10960fd 100644 --- a/read-cache-v5.c

[PATCH v3 11/24] grep.c: use index api

2013-08-18 Thread Thomas Gummerer
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/grep.c | 69 +- 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/builtin/grep.c b/builtin/grep.c index 7dc0389..1114fe8 100644 --- a/builtin/grep.c +++ b/builtin

[PATCH v3 08/24] add documentation for the index api

2013-08-18 Thread Thomas Gummerer
Add documentation for the index reading api. This also includes documentation for the new api functions introduced in the next patch. Helped-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/api-in-core-index.txt | 54

[PATCH v3 09/24] read-cache: add index reading api

2013-08-18 Thread Thomas Gummerer
functions for iterating over and accessing cache entries. This simplifies future improvements to the in-memory format, as changes will be concentrated on one file, instead of the whole git source code. Helped-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com

Re: [PATCH v3 06/24] read-cache: Don't compare uid, gid and ino on cygwin

2013-08-20 Thread Thomas Gummerer
Ramsay Jones ram...@ramsay1.demon.co.uk writes: On 18/08/2013 08:41 PM, Thomas Gummerer wrote: Cygwin doesn't have uid, gid and ino stats fields. Therefore we should never check them in the match_stat_data when working on the CYGWIN platform. Hmm, this is simply not true ... ;-) The need

Re: [PATCH v3 10/24] make sure partially read index is not changed

2013-08-20 Thread Thomas Gummerer
Eric Sunshine sunsh...@sunshineco.com writes: On Sun, Aug 18, 2013 at 3:41 PM, Thomas Gummerer t.gumme...@gmail.com wrote: A partially read index file currently cannot be written to disk. Make sure that never happens, by erroring out when a caller tries to write a s/,// partially read

Re: [PATCH v3 15/24] read-cache: read index-v5

2013-08-20 Thread Thomas Gummerer
that in the re-roll. On Mon, Aug 19, 2013 at 2:42 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +static struct directory_entry *directory_entry_from_ondisk(struct ondisk_directory_entry *ondisk, + const char *name

Re: [PATCH v3 15/24] read-cache: read index-v5

2013-08-20 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Mon, Aug 19, 2013 at 2:42 AM, Thomas Gummerer t.gumme...@gmail.com wrote: +static int read_entry(struct cache_entry **ce, char *pathname, size_t pathlen, + void *mmap, unsigned long mmap_size, + unsigned int

Re: [PATCH v3 02/24] read-cache: use fixed width integer types

2013-08-20 Thread Thomas Gummerer
Junio C Hamano gits...@pobox.com writes: Thomas Gummerer t.gumme...@gmail.com writes: Use the fixed width integer types uint16_t and uint32_t for ondisk structures, because unsigned short and unsigned int do not hae a guaranteed size. This sounds like an independent fix to me. I'd queue

Re: [PATCH v3 04/24] read-cache: clear version in discard_index()

2013-08-20 Thread Thomas Gummerer
Junio C Hamano gits...@pobox.com writes: Thomas Gummerer t.gumme...@gmail.com writes: All fields except index_state-version are reset in discard_index. Reset the version too. What is the practical consequence of not clearing this field? I somehow have a feeling that this was done

Re: [PATCH v3 23/24] introduce GIT_INDEX_VERSION environment variable

2013-08-20 Thread Thomas Gummerer
Duy Nguyen pclo...@gmail.com writes: On Mon, Aug 19, 2013 at 2:42 AM, Thomas Gummerer t.gumme...@gmail.com wrote: Respect a GIT_INDEX_VERSION environment variable, when a new index is initialized. Setting the environment variable will not cause existing index files to be converted to another

Re: [PATCH v3 00/24] Index-v5

2013-08-30 Thread Thomas Gummerer
Junio C Hamano gits...@pobox.com writes: Duy Nguyen pclo...@gmail.com writes: On Mon, Aug 19, 2013 at 2:41 AM, Thomas Gummerer t.gumme...@gmail.com wrote: I'm done reviewing this version (I neglected the extension writing patches because after spending hours on the main write patch I

Re: [PATCH/RFC v2 15/16] update-index.c: add a force-rewrite option

2012-08-08 Thread Thomas Gummerer
On 08/05, Junio C Hamano wrote: Thomas Gummerer t.gumme...@gmail.com writes: Add a force-rewrite option to update-index, which allows the user to rewrite the index, even if there are no changes. This can be used to do performance tests of both the reader and the writer. Signed-off

Re: [PATCH/RFC v2 09/16] Read index-v5

2012-08-08 Thread Thomas Gummerer
On 08/05, Junio C Hamano wrote: Thomas Gummerer t.gumme...@gmail.com writes: +static struct directory_entry *read_directories_v5(unsigned int *dir_offset, + unsigned int *dir_table_offset, + void *mmap

Re: [PATCH 2/2] Add index-v5

2012-08-08 Thread Thomas Gummerer
On 08/07, Robin Rosenberg wrote: Nguyễn Thái Ngọc Duy skrev 2012-08-06 16.36: +++ b/read-cache-v5.c @@ -0,0 +1,1170 @@ +#include cache.h +#include read-cache.h +#include resolve-undo.h +#include cache-tree.h + +struct cache_header_v5 { +unsigned int hdr_ndir; +unsigned int

[PATCH/RFC v3 05/13] Make in-memory format aware of stat_crc

2012-08-08 Thread Thomas Gummerer
Make the in-memory format aware of the stat_crc used by index-v5. It is simply ignored by index version prior to v5. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h |1 + read-cache.c | 25 + 2 files changed, 26 insertions(+) diff --git

[PATCH/RFC v3 0/13] Introduce index file format version 5

2012-08-08 Thread Thomas Gummerer
Previous rounds of this series were on $gmane/202752 and $gmane/202923. This round includes a major change, as it is splitting up read-cache.c into read-cache.c, read-cache-v2.c and read-cache-v5.c. It uses index-ops to call the specific functions as suggested by Duy. It also changes the index

[PATCH/RFC v3 02/13] t2104: Don't fail for index versions other than [23]

2012-08-08 Thread Thomas Gummerer
-worktree flags for the default index format is still covered by t7011 and t7012. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- t/t2104-update-index-skip-worktree.sh |1 + 1 file changed, 1 insertion(+) diff --git a/t/t2104-update-index-skip-worktree.sh b/t/t2104-update-index-skip

[PATCH/RFC v3 13/13] p0002-index.sh: add perf test for the index formats

2012-08-08 Thread Thomas Gummerer
From: Thomas Rast tr...@student.ethz.ch Add a performance test for index version [23]/4/5 by using git update-index --update-index=[345], thus testing both the reader and the writer speed of all index formats. Signed-off-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer

[PATCH/RFC v3 06/13] Read index-v5

2012-08-08 Thread Thomas Gummerer
...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile|1 + cache.h | 72 +++ read-cache-v5.c | 589 +++ read-cache.c|1 - 4 files changed, 662 insertions(+), 1 deletion(-) create mode 100644 read

[PATCH/RFC v3 08/13] Read cache-tree in index-v5

2012-08-08 Thread Thomas Gummerer
is sorted lexically. So we have to rebuild this format from the on-disk directory list. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache-tree.c| 93 +++ cache-tree.h| 10 ++ read-cache-v5.c |1 + 3 files changed, 104

[PATCH/RFC v3 10/13] Write index-v5 cache-tree data

2012-08-08 Thread Thomas Gummerer
Write the cache-tree data for the index version 5 file format. The in-memory cache-tree data is converted to the ondisk format, by adding it to the directory entries, that were compiled from the cache-entries in the step before. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache

[PATCH/RFC v3 04/13] Add documentation of the index-v5 file format

2012-08-08 Thread Thomas Gummerer
robin.rosenb...@dewire.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/index-file-format-v5.txt | 285 ++ 1 file changed, 285 insertions(+) create mode 100644 Documentation/technical/index-file-format-v5.txt diff --git a/Documentation

[PATCH/RFC v3 03/13] t3700: Avoid interfering with the racy code

2012-08-08 Thread Thomas Gummerer
interfering with the racy code, we use a time relative to the time returned by time(3), instead of a time relative to the mtime of the cache entries. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- t/t3700-add.sh |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t3700-add.sh

[PATCH/RFC v3 01/13] Move index v2 specific functions to their own file

2012-08-08 Thread Thomas Gummerer
...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Makefile |2 + cache.h | 13 +- read-cache-v2.c | 581 +++ read-cache.c | 613 +++--- read-cache.h

[PATCH/RFC v3 07/13] Read resolve-undo data

2012-08-08 Thread Thomas Gummerer
-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache-v5.c |1 + resolve-undo.c | 36 resolve-undo.h |2 ++ 3 files changed, 39 insertions(+) diff --git a/read-cache-v5.c b/read-cache-v5.c index ec1201d

[PATCH/RFC v3 11/13] Write resolve-undo data for index-v5

2012-08-08 Thread Thomas Gummerer
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache-v5.c |3 ++ resolve-undo.c | 93 +++ resolve-undo.h |1 + 3 files changed, 97 insertions(+) diff --git a/read-cache-v5.c b/read-cache-v5.c index 45f7acd..3d03111 100644

[PATCH/RFC v3 12/13] update-index.c: always rewrite the index when index-version is given

2012-08-08 Thread Thomas Gummerer
Make git update-index always rewrite the index, if a index-version is given. This is used for performance testing, to have a reader and writer for the whole index. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- builtin/update-index.c |4 +++- 1 file changed, 3 insertions(+), 1

[PATCH/RFC v3 09/13] Write index-v5

2012-08-08 Thread Thomas Gummerer
. Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com Helped-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- cache.h | 10 +- read-cache-v5.c | 589 ++- read-cache.c| 19 +- read-cache.h

Re: [PATCH/RFC v2 09/16] Read index-v5

2012-08-08 Thread Thomas Gummerer
On 08/08, Junio C Hamano wrote: Thomas Gummerer t.gumme...@gmail.com writes: +name = (char *)mmap + *dir_offset; +beginning = mmap + *dir_table_offset; Notice how you computed name with pointer arithmetic by first casting mmap (which is void *) and when computing

  1   2   3   4   5   6   7   8   >