Re: [PATCH] tag: Use OPT_BOOL instead of OPT_BOOLEAN to allow one action multiple times
Jonathan Nieder jrnie...@gmail.com writes: Stefano Lattarini wrote: Why not encourage the use of a standardized '--action' option instead? Because it's an unpleasant UI. :) This can work with lesser compatibility headaches for both the commands taking mode options and the commands taking mode words: git submodule init becomes git submodule --action=init git tag --delete TAG becomes git tag --action=delete TAGNAME That looks like a bad change in both cases --- it involves more typing without much upside to go along with it. But git submodule init gains synonym git submodule --init git tag --delete TAG stays as git tag --delete TAG looks fine to me. I agree 100% with the above that illustrates why --action=name is a bad idea. As I already said, adding action-option like --init, if doing so might help people, I am not opposed to it. Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Does Git now have any C struct version history tracking mechanism?
Hi, all. * Background Such a requirement came into my mind when I am tracking a gloomy C struct , with lengthy list of elements which are either elaborated or opaque. So I use git blame to track it down and found that its original version is quite simple and intuitive. So I think I could just slice out every snapshot of this struct, reading every changelog , to get a better knowledge of what it is and why it should be like this. It seems quite helpful but the process is quite cumbersome. So I wonder if there is already some mechanism fulfilling my requirement in Git. If it doesn't, would it be worthy adding one ? Thanks -- Regards, Zhan Jianyu -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
netrc credential helper promotion out of contrib?
A while has passed since contrib/credential/netrc was added. Is it OK to promote it to be part of the main installation? In that directory there's also gnome-keyring, osxkeychain, and wincred; I don't know if those are ready for promotion. Thanks Ted -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: What's cooking in git.git (Aug 2013, #04; Thu, 15)
Am 16.08.2013 00:36, schrieb Junio C Hamano: Due to unfortunate regressions, two topics had to be reverted: * An attempted fix to git stash save, to detect that going back to the state of the HEAD needs to lose killed files, and/or untracked files in a killed directory, to prevent the command from proceeding without --force. This used ls-files -k that was unusably slow. * An attempted enhancement to allow @ to be used to name HEAD. This rewrote @ in a ref where it shouldn't have, e.g. refs/@/foo. You still need to remove corresponding paragraphs from the release notes. -- Hannes -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH gitk 0/4] gitk support for git log -L
Hi Thomas, On Wed, Jul 31, 2013 at 03:17:41PM +0200, Thomas Rast wrote: Jens Lehmann jens.lehm...@web.de writes: Am 29.07.2013 21:37, schrieb Thomas Rast: Thomas Rast tr...@inf.ethz.ch writes: Thomas Rast tr...@inf.ethz.ch writes: Now that git log -L has hit master, I figure it's time to discuss the corresponding change to gitk. Paul, any news on this? Any chance we can get it into the next release, since that will also be the first release to ship with 'git log -L'? Jens pointed out privately that the handling of unstuck -L options is unfortunate, to put it mildly. I'll send a reroll. But as soon as that is fixed I'd really like to see this applied, as I think gitk is the perfect tool to show history information. One thing I worry about is having gitk storing in memory not just the history graph but also all the diffs (assuming I have understood correctly what you're doing). Gitk's memory consumption is already pretty large. However, I can't see an alternative at this point. Unfortunately it's turning out to be harder than I hoped. gitk runs the arguments through git-rev-parse, which only knows that -n gets an unstuck argument. Consequently, gitk accepts an unstuck -n but only stuck forms of -S and -G. Excuse my ignorance, but what do you mean by stuck vs. unstuck? Thanks, Paul. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCHv2] repack: rewrite the shell script in C.
On 08/17/2013 03:34 PM, René Scharfe wrote: Hmm, stepping back a bit, why not just build the paths and call unlink for them right away, without readdir? The shell version only ever deletes existing .pack files (those in $existing alias existing_packs) as well as their .idx and .keep files, if present. It doesn't use a glob pattern, unlike remove_pack here. I'll meditate on that. Thanks for all the other remarks. Now the code looks much more git-ish, similar to other commands. The lines of code went down from 411 to 385, I guess we can cut off more inefficiencies there. As you suggested, maybe we should juts have one helper function to read in the pack directory and keeping all the information (complete filename), so we do not need to find the exact filename later again by looping over the directory again. Stefan -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCHv3] repack: rewrite the shell script in C.
This is the beginning of the rewrite of the repacking.
* replace all plain string handling functions
by git helper functions, most often mkpathdup
* use argv-array structs to pass arguments to
other git invocations.
Only test t7701 (2nd) fails now with this patch.
Signed-off-by: Stefan Beller stefanbel...@googlemail.com
---
Makefile| 2 +-
builtin.h | 1 +
builtin/repack.c| 385
git-repack.sh = contrib/examples/git-repack.sh | 0
git.c | 1 +
5 files changed, 388 insertions(+), 1 deletion(-)
create mode 100644 builtin/repack.c
rename git-repack.sh = contrib/examples/git-repack.sh (100%)
diff --git a/Makefile b/Makefile
index ef442eb..4ec5bbe 100644
--- a/Makefile
+++ b/Makefile
@@ -464,7 +464,6 @@ SCRIPT_SH += git-pull.sh
SCRIPT_SH += git-quiltimport.sh
SCRIPT_SH += git-rebase.sh
SCRIPT_SH += git-remote-testgit.sh
-SCRIPT_SH += git-repack.sh
SCRIPT_SH += git-request-pull.sh
SCRIPT_SH += git-stash.sh
SCRIPT_SH += git-submodule.sh
@@ -972,6 +971,7 @@ BUILTIN_OBJS += builtin/reflog.o
BUILTIN_OBJS += builtin/remote.o
BUILTIN_OBJS += builtin/remote-ext.o
BUILTIN_OBJS += builtin/remote-fd.o
+BUILTIN_OBJS += builtin/repack.o
BUILTIN_OBJS += builtin/replace.o
BUILTIN_OBJS += builtin/rerere.o
BUILTIN_OBJS += builtin/reset.o
diff --git a/builtin.h b/builtin.h
index 8afa2de..b56cf07 100644
--- a/builtin.h
+++ b/builtin.h
@@ -102,6 +102,7 @@ extern int cmd_reflog(int argc, const char **argv, const
char *prefix);
extern int cmd_remote(int argc, const char **argv, const char *prefix);
extern int cmd_remote_ext(int argc, const char **argv, const char *prefix);
extern int cmd_remote_fd(int argc, const char **argv, const char *prefix);
+extern int cmd_repack(int argc, const char **argv, const char *prefix);
extern int cmd_repo_config(int argc, const char **argv, const char *prefix);
extern int cmd_rerere(int argc, const char **argv, const char *prefix);
extern int cmd_reset(int argc, const char **argv, const char *prefix);
diff --git a/builtin/repack.c b/builtin/repack.c
new file mode 100644
index 000..190eb5f
--- /dev/null
+++ b/builtin/repack.c
@@ -0,0 +1,385 @@
+/*
+ * The shell version was written by Linus Torvalds (2005) and many others.
+ * This is a translation into C by Stefan Beller (2013)
+ */
+
+#include builtin.h
+#include cache.h
+#include dir.h
+#include parse-options.h
+#include run-command.h
+#include sigchain.h
+#include strbuf.h
+#include string-list.h
+
+#include argv-array.h
+
+static const char *const git_repack_usage[] = {
+ N_(git repack [options]),
+ NULL
+};
+
+/* enabled by default since 22c79eab (2008-06-25) */
+static int delta_base_offset = 1;
+
+static int repack_config(const char *var, const char *value, void *cb)
+{
+ if (!strcmp(var, repack.usedeltabaseoffset)) {
+ delta_base_offset = git_config_bool(var, value);
+ return 0;
+ }
+ return git_default_config(var, value, cb);
+}
+
+static void remove_temporary_files() {
+ DIR *dir;
+ struct dirent *e;
+ char *prefix, *path;
+
+ prefix = mkpathdup(.tmp-%d-pack, getpid());
+ path = mkpathdup(%s/pack, get_object_directory());
+
+ dir = opendir(path);
+ while ((e = readdir(dir)) != NULL) {
+ if (!prefixcmp(e-d_name, prefix)) {
+ struct strbuf fname = STRBUF_INIT;
+ strbuf_addf(fname, %s/%s, path, e-d_name);
+ unlink(strbuf_detach(fname, NULL));
+ }
+ }
+ free(prefix);
+ free(path);
+ closedir(dir);
+}
+
+static void remove_pack_on_signal(int signo)
+{
+ remove_temporary_files();
+ sigchain_pop(signo);
+ raise(signo);
+}
+
+void get_pack_sha1_list(char *packdir, struct string_list *sha1_list)
+{
+ DIR *dir;
+ struct dirent *e;
+ char *path, *suffix;
+
+ path = mkpathdup(%s/pack, get_object_directory());
+ suffix = .pack;
+
+ dir = opendir(path);
+ while ((e = readdir(dir)) != NULL) {
+ if (!suffixcmp(e-d_name, suffix)) {
+ char *buf, *sha1;
+ buf = xmemdupz(e-d_name, strlen(e-d_name));
+ buf[strlen(e-d_name) - strlen(suffix)] = '\0';
+ if (strlen(e-d_name) - strlen(suffix) 40) {
+ sha1 = buf[strlen(e-d_name) - strlen(suffix)
- 40];
+ string_list_append_nodup(sha1_list, sha1);
+ } else {
+ /*TODO: what should happen to pack files having
no 40 char sha1 specifier?*/
+ }
+ }
+ }
+ free(path);
+ closedir(dir);
+}
+
+/*
+ * remove_pack will remove any files following the pattern *${SHA1}.{EXT}
Re: What's cooking in git.git (Aug 2013, #03; Tue, 13)
gits...@pobox.com wrote on Tue, 13 Aug 2013 15:06 -0700: * ks/p4-view-spec (2013-08-11) 3 commits - WAITING FOR ACK - git p4: implement view spec wildcards with p4 where - git p4 test: sanitize P4CHARSET Waiting for an ack. I'm still running perf tests on the 3-patch version. It looks good. You should expect a reroll of the 2nd and 3rd commits, combining them into one patch. Thanks for tracking these. -- Pete -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCHv3] repack: rewrite the shell script in C.
On Aug 18, 2013, at 07:36, Stefan Beller wrote: + fprintf(stderr, + WARNING: Some packs in use have been renamed by\n + WARNING: prefixing old- to their name, in order to\n + WARNING: replace them with the new version of the\n + WARNING: file. But the operation failed, and\n + WARNING: attempt to rename them back to their\n + WARNING: original names also failed.\n Bad grammar But the operation failed, and attempt to rename them How about But the operation failed, and the attempt to rename them ... instead. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCHv3] repack: rewrite the shell script in C.
+static void remove_temporary_files() {
+ DIR *dir;
+ struct dirent *e;
+ char *prefix, *path;
+
+ prefix = mkpathdup(.tmp-%d-pack, getpid());
+ path = mkpathdup(%s/pack, get_object_directory());
+
+ dir = opendir(path);
+ while ((e = readdir(dir)) != NULL) {
+ if (!prefixcmp(e-d_name, prefix)) {
+ struct strbuf fname = STRBUF_INIT;
+ strbuf_addf(fname, %s/%s, path, e-d_name);
+ unlink(strbuf_detach(fname, NULL));
I'm not sure I like the memory allocation done here for each file to be
deleted, but it's probably not worth worrying about.
+void get_pack_sha1_list(char *packdir, struct string_list *sha1_list)
+{
+ DIR *dir;
+ struct dirent *e;
+ char *path, *suffix;
+
+ path = mkpathdup(%s/pack, get_object_directory());
+ suffix = .pack;
+
+ dir = opendir(path);
+ while ((e = readdir(dir)) != NULL) {
+ if (!suffixcmp(e-d_name, suffix)) {
+ char *buf, *sha1;
+ buf = xmemdupz(e-d_name, strlen(e-d_name));
+ buf[strlen(e-d_name) - strlen(suffix)] = '\0';
+ if (strlen(e-d_name) - strlen(suffix) 40) {
+ sha1 = buf[strlen(e-d_name) - strlen(suffix)
- 40];
+ string_list_append_nodup(sha1_list, sha1);
Unless sha1 == buf, this will crash when that string_list is freed
because sha1 was not returned by malloc. If it doesn't crash for
you then I guess sha1_list is never freed. :) How about just
taking the part of d_name we need, like this?
size_t len = strlen(e-d_name) - strlen(suffix);
if (len 40) {
char *sha1 = xmemdupz(e-d_name + len - 40, 40);
string_list_append_nodup(sha1_list, sha1);
}
+ } else {
+ /*TODO: what should happen to pack files having
no 40 char sha1 specifier?*/
What does the current code do with them? From a quick glance it
looks like it deletes them in the end, right?
René
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 0/2] fix cases of rebase -i ignoring core.commentchar
This is a reroll of [1] which fixes a couple cases where rebase -i ignores core.commentchar. One case was an oversight from the original patch which added core.commentchar support to rebase -i, and is already in 'maint'. The other is a regression introduced by [2] which is still in 'next'. In this reroll, the single patch from v1 is split into two: patch 1/2 is suitable for 'maint'; and patch 2/2 is for 'next'. There are no other changes. I wanted to add tests to t3404 for these bugs but couldn't figure out how to do it using the external interface of rebase -i. I was able to verify before and after behavior by adding temporary echo's to the code in order to observe the internal functioning. [1]: http://git.661346.n2.nabble.com/PATCH-rebase-i-fix-cases-ignoring-core-commentchar-td7593965.html [2]: http://git.661346.n2.nabble.com/PATCH-0-3-fix-interactive-rebase-short-SHA-1-collision-bug-td7593673.html#a7593676 Eric Sunshine (2): rebase -i: fix skip_unnecessary_picks() to respect core.commentchar rebase -i: fix core.commentchar regression git-rebase--interactive.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 1.8.4.rc3.500.gc3113b0 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Urgent..
I have a business deal to discuss with you please get back to me for more info. Josh Brandon -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 2/2] rebase -i: fix core.commentchar regression
9a46c25bdbf79744 (rebase: interactive: fix short SHA-1 collision,
2013-08-12) added transform_todo_ids() which assumes incorrectly that
the comment character is always '#', however, this has not been the case
since 180bad3d10fe3a7f (rebase -i: respect core.commentchar, 2013-02-11)
patched rebase -i to respect core.commentchar. Fix this regression.
Signed-off-by: Eric Sunshine sunsh...@sunshineco.com
---
Intended for 'next'.
git-rebase--interactive.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 3733312..f246810 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -693,7 +693,7 @@ transform_todo_ids () {
while read -r command rest
do
case $command in
- '#'* | exec)
+ $comment_char* | exec)
# Be careful for oddball commands like 'exec'
# that do not have a SHA-1 at the beginning of $rest.
;;
--
1.8.4.rc3.500.gc3113b0
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 1/2] rebase -i: fix skip_unnecessary_picks() to respect core.commentchar
eff80a9fd990de36 (Allow custom comment char, 2013-01-16) added general
core.commentchar support but forgot to update git-rebase--interactive to
respect it. 180bad3d10fe3a7f (rebase -i: respect core.commentchar,
2013-02-11) addressed this oversight but missed one instance of
hard-coded '#' comment character in skip_unnecessary_picks(). Fix this.
Signed-off-by: Eric Sunshine sunsh...@sunshineco.com
---
Suitable for 'maint'.
git-rebase--interactive.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index ea11e62..3733312 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -671,7 +671,7 @@ skip_unnecessary_picks () {
;;
esac
;;
- 3,#*|3,)
+ 3,$comment_char*|3,)
# copy comments
;;
*)
--
1.8.4.rc3.500.gc3113b0
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] submodule: prevent warning in summary output
When git submodule summary is run and there is a deleted submodule, there is an
warning from git rev-parse:
fatal: Not a git repository: '.vim/pathogen/.git'
Silence this warning, since it is fully expected that a deleted submodule will
not be a git repository.
Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
---
git-submodule.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/git-submodule.sh b/git-submodule.sh
index 2979197..66ee621 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1071,7 +1071,7 @@ cmd_summary() {
missing_dst=
test $mod_src = 16
- ! GIT_DIR=$name/.git git-rev-parse -q --verify $sha1_src^0
/dev/null
+ ! GIT_DIR=$name/.git git-rev-parse -q --verify $sha1_src^0
/dev/null 21
missing_src=t
test $mod_dst = 16
--
1.8.4.rc2.564.g10ce5ae
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git fmt-merge-msg usage
Michael Haggerty mhag...@alum.mit.edu writes: On 07/15/2013 11:43 PM, Junio C Hamano wrote: [...] This was a good exercise for git-imerge.[...] A few things I noticed: [...] - The final step imerge finish gave me this ugliness: Merge commit 93d9353... into commit cb5d2fc7 Perhaps you can at least use the initial branch name nd/magic-pathspec I gave you, and use git fmt-merge-msg? I tried to implement this but it is not obvious from the documentation (to say the least) how to use git fmt-merge-msg. It appears that this program takes, on standard input, something like sha1 TAB TAB text1 LF sha1 TAB TAB text2 LF sha1 TAB TAB text3 LF ... (the two TABs are required!). Correct; fmt-merge-msg is designed to read FETCH_HEAD that can have 'not-for-merge' marker between these two HTs. text$N are also expected to be in a specific format to explain where the object being merged described by the line came from. But a bit of the magic of these merge messages is how the text are generated in the first place; e.g., refs/heads/foo - branch 'foo' refs/remotes/bar/baz - remote-tracking branch 'bar/baz' Is this magic available via any Git commands, or do I have to replicate it? This is all internal to fetch and git merge, which are the only things that need to know the specifics. store_updated_refs() is where entries in FETCH_HEAD are written. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 00/24] Index-v5
Hi, previous rounds (without api) are at $gmane/202752, $gmane/202923, $gmane/203088 and $gmane/203517, the previous rounds with api were at $gmane/229732 and $gmane/230210. Thanks to Duy for reviewing the the last round and Junio and Ramsay for additional comments. Changes since the previous round: read-cache: move index v2 specific functions to their own file - set istate-ops to NULL in discard_index read-cache: add index reading api - style fixes - instead of using internal_ops struct, do for_each_index_entry in read-cache.c grep.c: use index api - remove duplicate call to match_pathspec_depth ls-files.c: use index api - load the whole index if there is a trai documentation: add documentation of the index-v5 file format - fix typo - change the position of nfile and ndir in the index file - document that the conflicts are also stored in the fileentries block - document invalid flag read-cache: read index-v5 - restrict partial loading a bit more, by being more careful when adjusting the pathspec - move the ondisk structs from cache.h to read-cache-v5.c - merge for and while loop in read_entries - keep a directory tree instead of a flat list when reading the directories - ce_queue_push moved to read-cache: write index-v5 using a next_ce pointer instead of the next pointer that's already used by name-hash. - fix reading if there are extensions that are not yet supported - ignore entries that have the invalid flag set read-cache: read cache-tree in index-v5 - use the tree structure which is now used in read index-v5 read-cache: write index-v5 - simplify compile_directory_data changes to the index file format: - store the number of files before the number of directories in the header, so that the file command still can recognize the number of files in the repository correctly. - store all staged entries in the fileentries block. Doesn't hurt the performance a lot but simplifies the code. - add an invalid flag for entries that should be ignored. currently unused but respected when reading. will be used once the conflict resolution is done by flipping a bit in the conflict entries at the end of the index. added commits: - read-cache: use fixed width integer types - read-cache: clear version in discard_index() - read-cache: Don't compare uid, gid and ino on cygwin - introduce GIT_INDEX_VERSION environment variable - test-lib: allow setting the index format version Thomas Gummerer (23): t2104: Don't fail for index versions other than [23] read-cache: use fixed width integer types read-cache: split index file version specific functionality read-cache: clear version in discard_index() read-cache: move index v2 specific functions to their own file read-cache: Don't compare uid, gid and ino on cygwin read-cache: Re-read index if index file changed add documentation for the index api read-cache: add index reading api make sure partially read index is not changed grep.c: use index api ls-files.c: use index api documentation: add documentation of the index-v5 file format read-cache: make in-memory format aware of stat_crc read-cache: read index-v5 read-cache: read resolve-undo data read-cache: read cache-tree in index-v5 read-cache: write index-v5 read-cache: write index-v5 cache-tree data read-cache: write resolve-undo data for index-v5 update-index.c: rewrite index when index-version is given introduce GIT_INDEX_VERSION environment variable test-lib: allow setting the index format version Thomas Rast (1): p0003-index.sh: add perf test for the index formats Documentation/technical/api-in-core-index.txt| 54 +- Documentation/technical/index-file-format-v5.txt | 301 + Makefile | 10 + builtin/apply.c |2 + builtin/grep.c | 69 +- builtin/ls-files.c | 36 +- builtin/update-index.c |6 +- cache-tree.c |2 +- cache-tree.h |1 + cache.h | 93 +- read-cache-v2.c | 550 + read-cache-v5.c | 1417 ++ read-cache.c | 685 +++ read-cache.h | 61 + t/perf/p0003-index.sh| 63 + t/t2104-update-index-skip-worktree.sh|1 + t/test-lib-functions.sh |5 + t/test-lib.sh|3 + test-index-version.c |6 + unpack-trees.c |3 +- 20 files changed, 2786 insertions(+), 582 deletions(-) create mode 100644
[PATCH v3 06/24] read-cache: Don't compare uid, gid and ino on cygwin
Cygwin doesn't have uid, gid and ino stats fields. Therefore we should
never check them in the match_stat_data when working on the CYGWIN
platform.
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
This patch was not tested on Cygwin yet. I think it's needed though,
because the re-reading of the index if it changed will no longer use
it's own index_changed function, but use the stat_validity_check
function instead. Would be great if someone running Cygwin could test
this.
read-cache.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 1f827de..aa17ce7 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -82,6 +82,7 @@ int match_stat_data(const struct stat_data *sd, struct stat
*st)
changed |= CTIME_CHANGED;
#endif
+#if !defined (__CYGWIN__)
if (check_stat) {
if (sd-sd_uid != (unsigned int) st-st_uid ||
sd-sd_gid != (unsigned int) st-st_gid)
@@ -89,6 +90,7 @@ int match_stat_data(const struct stat_data *sd, struct stat
*st)
if (sd-sd_ino != (unsigned int) st-st_ino)
changed |= INODE_CHANGED;
}
+#endif
#ifdef USE_STDEV
/*
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 05/24] read-cache: move index v2 specific functions to their own file
Move index version 2 specific functions to their own file. The non-index
specific functions will be in read-cache.c, while the index version 2
specific functions will be in read-cache-v2.c.
Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
Makefile | 2 +
builtin/apply.c| 2 +
builtin/update-index.c | 2 +-
cache.h| 13 +-
read-cache-v2.c| 544 ++
read-cache.c | 576 +
read-cache.h | 58 +
test-index-version.c | 6 +
unpack-trees.c | 3 +-
9 files changed, 669 insertions(+), 537 deletions(-)
create mode 100644 read-cache-v2.c
create mode 100644 read-cache.h
diff --git a/Makefile b/Makefile
index 6b446e7..afae23e 100644
--- a/Makefile
+++ b/Makefile
@@ -712,6 +712,7 @@ LIB_H += progress.h
LIB_H += prompt.h
LIB_H += quote.h
LIB_H += reachable.h
+LIB_H += read-cache.h
LIB_H += reflog-walk.h
LIB_H += refs.h
LIB_H += remote.h
@@ -855,6 +856,7 @@ LIB_OBJS += prompt.o
LIB_OBJS += quote.o
LIB_OBJS += reachable.o
LIB_OBJS += read-cache.o
+LIB_OBJS += read-cache-v2.o
LIB_OBJS += reflog-walk.o
LIB_OBJS += refs.o
LIB_OBJS += remote.o
diff --git a/builtin/apply.c b/builtin/apply.c
index 50912c9..3d5a5dc 100644
--- a/builtin/apply.c
+++ b/builtin/apply.c
@@ -3682,6 +3682,8 @@ static void build_fake_ancestor(struct patch *list, const
char *filename)
die (Could not add %s to temporary index, name);
}
+ if (!result.initialized)
+ initialize_index(result, 0);
fd = open(filename, O_WRONLY | O_CREAT, 0666);
if (fd 0 || write_index(result, fd) || close(fd))
die (Could not write temporary index to %s, filename);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index e3a10d7..c5bb889 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -863,7 +863,7 @@ int cmd_update_index(int argc, const char **argv, const
char *prefix)
if (the_index.version != preferred_index_format)
active_cache_changed = 1;
- the_index.version = preferred_index_format;
+ change_cache_version(preferred_index_format);
}
if (read_from_stdin) {
diff --git a/cache.h b/cache.h
index 9ef778a..d4dae21 100644
--- a/cache.h
+++ b/cache.h
@@ -95,16 +95,8 @@ unsigned long git_deflate_bound(git_zstream *, unsigned
long);
*/
#define DEFAULT_GIT_PORT 9418
-/*
- * Basic data structures for the directory cache
- */
#define CACHE_SIGNATURE 0x44495243 /* DIRC */
-struct cache_header {
- uint32_t hdr_signature;
- uint32_t hdr_version;
- uint32_t hdr_entries;
-};
#define INDEX_FORMAT_LB 2
#define INDEX_FORMAT_UB 4
@@ -279,6 +271,7 @@ struct index_state {
initialized : 1;
struct hash_table name_hash;
struct hash_table dir_hash;
+ struct index_ops *ops;
};
extern struct index_state the_index;
@@ -296,6 +289,8 @@ extern void free_name_hash(struct index_state *istate);
#define active_cache_changed (the_index.cache_changed)
#define active_cache_tree (the_index.cache_tree)
+#define initialize_cache() initialize_index(the_index, 0)
+#define change_cache_version(version) change_index_version(the_index,
(version))
#define read_cache() read_index(the_index)
#define read_cache_from(path) read_index_from(the_index, (path))
#define read_cache_preload(pathspec) read_index_preload(the_index, (pathspec))
@@ -454,6 +449,8 @@ extern void sanitize_stdfds(void);
} while (0)
/* Initialize and use the cache information */
+extern void initialize_index(struct index_state *istate, int version);
+extern void change_index_version(struct index_state *istate, int version);
extern int read_index(struct index_state *);
extern int read_index_preload(struct index_state *, const struct pathspec
*pathspec);
extern int read_index_from(struct index_state *, const char *path);
diff --git a/read-cache-v2.c b/read-cache-v2.c
new file mode 100644
index 000..070d468
--- /dev/null
+++ b/read-cache-v2.c
@@ -0,0 +1,544 @@
+#include cache.h
+#include read-cache.h
+#include resolve-undo.h
+#include cache-tree.h
+#include varint.h
+
+/* Mask for the name length in ce_flags in the on-disk index */
+#define CE_NAMEMASK (0x0fff)
+
+/*
+ * Index File I/O
+ */
+
+/*
+ * dev/ino/uid/gid/size are also just tracked to the low 32 bits
+ * Again - this is just a (very strong in practice) heuristic that
+ * the inode hasn't changed.
+ *
+ * We save the fields in big-endian order to allow using the
+ * index file over NFS transparently.
+ */
+struct ondisk_cache_entry {
+ struct cache_time ctime;
+ struct cache_time mtime;
+ uint32_t
[PATCH v3 03/24] read-cache: split index file version specific functionality
Split index file version specific functionality to their own functions,
to prepare for moving the index file version specific parts to their own
file. This makes it easier to add a new index file format later.
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache.c | 114 ++-
1 file changed, 74 insertions(+), 40 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 0df5b31..de0bbcd 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1269,10 +1269,8 @@ struct ondisk_cache_entry_extended {
ondisk_cache_entry_extended_size(ce_namelen(ce)) : \
ondisk_cache_entry_size(ce_namelen(ce)))
-static int verify_hdr(struct cache_header *hdr, unsigned long size)
+static int verify_hdr_version(struct cache_header *hdr, unsigned long size)
{
- git_SHA_CTX c;
- unsigned char sha1[20];
int hdr_version;
if (hdr-hdr_signature != htonl(CACHE_SIGNATURE))
@@ -1280,10 +1278,21 @@ static int verify_hdr(struct cache_header *hdr,
unsigned long size)
hdr_version = ntohl(hdr-hdr_version);
if (hdr_version INDEX_FORMAT_LB || INDEX_FORMAT_UB hdr_version)
return error(bad index version %d, hdr_version);
+ return 0;
+}
+
+static int verify_hdr(void *mmap, unsigned long size)
+{
+ git_SHA_CTX c;
+ unsigned char sha1[20];
+
+ if (size sizeof(struct cache_header) + 20)
+ die(index file smaller than expected);
+
git_SHA1_Init(c);
- git_SHA1_Update(c, hdr, size - 20);
+ git_SHA1_Update(c, mmap, size - 20);
git_SHA1_Final(sha1, c);
- if (hashcmp(sha1, (unsigned char *)hdr + size - 20))
+ if (hashcmp(sha1, (unsigned char *)mmap + size - 20))
return error(bad index file sha1 signature);
return 0;
}
@@ -1425,44 +1434,14 @@ static struct cache_entry *create_from_disk(struct
ondisk_cache_entry *ondisk,
return ce;
}
-/* remember to discard_cache() before reading a different cache! */
-int read_index_from(struct index_state *istate, const char *path)
+static int read_index_v2(struct index_state *istate, void *mmap, unsigned long
mmap_size)
{
- int fd, i;
- struct stat st;
+ int i;
unsigned long src_offset;
struct cache_header *hdr;
- void *mmap;
- size_t mmap_size;
struct strbuf previous_name_buf = STRBUF_INIT, *previous_name;
- if (istate-initialized)
- return istate-cache_nr;
-
- istate-timestamp.sec = 0;
- istate-timestamp.nsec = 0;
- fd = open(path, O_RDONLY);
- if (fd 0) {
- if (errno == ENOENT)
- return 0;
- die_errno(index file open failed);
- }
-
- if (fstat(fd, st))
- die_errno(cannot stat the open index);
-
- mmap_size = xsize_t(st.st_size);
- if (mmap_size sizeof(struct cache_header) + 20)
- die(index file smaller than expected);
-
- mmap = xmmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd,
0);
- if (mmap == MAP_FAILED)
- die_errno(unable to map index file);
- close(fd);
-
hdr = mmap;
- if (verify_hdr(hdr, mmap_size) 0)
- goto unmap;
istate-version = ntohl(hdr-hdr_version);
istate-cache_nr = ntohl(hdr-hdr_entries);
@@ -1488,8 +1467,6 @@ int read_index_from(struct index_state *istate, const
char *path)
src_offset += consumed;
}
strbuf_release(previous_name_buf);
- istate-timestamp.sec = st.st_mtime;
- istate-timestamp.nsec = ST_MTIME_NSEC(st);
while (src_offset = mmap_size - 20 - 8) {
/* After an array of active_nr index entries,
@@ -1509,6 +1486,58 @@ int read_index_from(struct index_state *istate, const
char *path)
src_offset += 8;
src_offset += extsize;
}
+ return 0;
+unmap:
+ munmap(mmap, mmap_size);
+ die(index file corrupt);
+}
+
+/* remember to discard_cache() before reading a different cache! */
+int read_index_from(struct index_state *istate, const char *path)
+{
+ int fd;
+ struct stat st;
+ struct cache_header *hdr;
+ void *mmap;
+ size_t mmap_size;
+
+ errno = EBUSY;
+ if (istate-initialized)
+ return istate-cache_nr;
+
+ errno = ENOENT;
+ istate-timestamp.sec = 0;
+ istate-timestamp.nsec = 0;
+ fd = open(path, O_RDONLY);
+ if (fd 0) {
+ if (errno == ENOENT)
+ return 0;
+ die_errno(index file open failed);
+ }
+
+ if (fstat(fd, st))
+ die_errno(cannot stat the open index);
+
+ errno = EINVAL;
+ mmap_size = xsize_t(st.st_size);
+ if (mmap_size sizeof(struct cache_header) + 20)
+ die(index file smaller than
[PATCH v3 04/24] read-cache: clear version in discard_index()
All fields except index_state-version are reset in discard_index. Reset the version too. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- read-cache.c | 1 + 1 file changed, 1 insertion(+) diff --git a/read-cache.c b/read-cache.c index de0bbcd..1e22f6f 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1558,6 +1558,7 @@ int discard_index(struct index_state *istate) for (i = 0; i istate-cache_nr; i++) free(istate-cache[i]); resolve_undo_clear_index(istate); + istate-version = 0; istate-cache_nr = 0; istate-cache_changed = 0; istate-timestamp.sec = 0; -- 1.8.3.4.1231.g9fbf354.dirty -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 01/24] t2104: Don't fail for index versions other than [23]
t2104 currently checks for the exact index version 2 or 3, depending if there is a skip-worktree flag or not. Other index versions do not use extended flags and thus cannot be tested for version changes. Make this test update the index to version 2 at the beginning of the test. Testing the skip-worktree flags for the default index format is still covered by t7011 and t7012. Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- t/t2104-update-index-skip-worktree.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/t/t2104-update-index-skip-worktree.sh b/t/t2104-update-index-skip-worktree.sh index 1d0879b..bd9644f 100755 --- a/t/t2104-update-index-skip-worktree.sh +++ b/t/t2104-update-index-skip-worktree.sh @@ -22,6 +22,7 @@ H sub/2 EOF test_expect_success 'setup' ' + git update-index --index-version=2 mkdir sub touch ./1 ./2 sub/1 sub/2 git add 1 2 sub/1 sub/2 -- 1.8.3.4.1231.g9fbf354.dirty -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 02/24] read-cache: use fixed width integer types
Use the fixed width integer types uint16_t and uint32_t for ondisk
structures, because unsigned short and unsigned int do not hae a
guaranteed size.
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
cache.h | 10 +-
read-cache.c | 30 +++---
2 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/cache.h b/cache.h
index bd6fb9f..9ef778a 100644
--- a/cache.h
+++ b/cache.h
@@ -101,9 +101,9 @@ unsigned long git_deflate_bound(git_zstream *, unsigned
long);
#define CACHE_SIGNATURE 0x44495243 /* DIRC */
struct cache_header {
- unsigned int hdr_signature;
- unsigned int hdr_version;
- unsigned int hdr_entries;
+ uint32_t hdr_signature;
+ uint32_t hdr_version;
+ uint32_t hdr_entries;
};
#define INDEX_FORMAT_LB 2
@@ -115,8 +115,8 @@ struct cache_header {
* check it for equality in the 32 bits we save.
*/
struct cache_time {
- unsigned int sec;
- unsigned int nsec;
+ uint32_t sec;
+ uint32_t nsec;
};
struct stat_data {
diff --git a/read-cache.c b/read-cache.c
index ceaf207..0df5b31 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1230,14 +1230,14 @@ static struct cache_entry *refresh_cache_entry(struct
cache_entry *ce, int reall
struct ondisk_cache_entry {
struct cache_time ctime;
struct cache_time mtime;
- unsigned int dev;
- unsigned int ino;
- unsigned int mode;
- unsigned int uid;
- unsigned int gid;
- unsigned int size;
+ uint32_t dev;
+ uint32_t ino;
+ uint32_t mode;
+ uint32_t uid;
+ uint32_t gid;
+ uint32_t size;
unsigned char sha1[20];
- unsigned short flags;
+ uint16_t flags;
char name[FLEX_ARRAY]; /* more */
};
@@ -1249,15 +1249,15 @@ struct ondisk_cache_entry {
struct ondisk_cache_entry_extended {
struct cache_time ctime;
struct cache_time mtime;
- unsigned int dev;
- unsigned int ino;
- unsigned int mode;
- unsigned int uid;
- unsigned int gid;
- unsigned int size;
+ uint32_t dev;
+ uint32_t ino;
+ uint32_t mode;
+ uint32_t uid;
+ uint32_t gid;
+ uint32_t size;
unsigned char sha1[20];
- unsigned short flags;
- unsigned short flags2;
+ uint16_t flags;
+ uint16_t flags2;
char name[FLEX_ARRAY]; /* more */
};
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 23/24] introduce GIT_INDEX_VERSION environment variable
Respect a GIT_INDEX_VERSION environment variable, when a new index is
initialized. Setting the environment variable will not cause existing
index files to be converted to another format for additional safety.
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache.c | 9 +++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 1d9b615..f820d8a 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1235,8 +1235,13 @@ static struct cache_entry *refresh_cache_entry(struct
cache_entry *ce, int reall
void initialize_index(struct index_state *istate, int version)
{
istate-initialized = 1;
- if (!version)
- version = INDEX_FORMAT_DEFAULT;
+ if (!version) {
+ char *envversion = getenv(GIT_INDEX_VERSION);
+ if (!envversion)
+ version = INDEX_FORMAT_DEFAULT;
+ else
+ version = atoi(envversion);
+ }
istate-version = version;
set_istate_ops(istate);
}
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 18/24] read-cache: write index-v5
Write the index version 5 file format to disk. This version doesn't
write the cache-tree data and resolve-undo data to the file.
The main work is done when filtering out the directories from the
current in-memory format, where in the same turn also the conflicts
and the file data is calculated.
Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com
Helped-by: Thomas Rast tr...@student.ethz.ch
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
cache.h | 2 +
read-cache-v5.c | 596 +++-
read-cache.c| 4 +-
read-cache.h| 1 +
4 files changed, 601 insertions(+), 2 deletions(-)
diff --git a/cache.h b/cache.h
index 89f556b..a109f35 100644
--- a/cache.h
+++ b/cache.h
@@ -138,6 +138,7 @@ struct cache_entry {
unsigned char sha1[20];
uint32_t ce_stat_crc;
struct cache_entry *next; /* used by name_hash */
+ struct cache_entry *next_ce;
char name[FLEX_ARRAY]; /* more */
};
@@ -532,6 +533,7 @@ extern int unmerged_index(const struct index_state *);
extern int verify_path(const char *path);
extern struct cache_entry *index_name_exists(struct index_state *istate, const
char *name, int namelen, int igncase);
extern int index_name_pos(const struct index_state *, const char *name, int
namelen);
+extern struct directory_entry *init_directory_entry(char *pathname, int len);
#define ADD_CACHE_OK_TO_ADD 1 /* Ok to add */
#define ADD_CACHE_OK_TO_REPLACE 2 /* Ok to replace file/directory */
#define ADD_CACHE_SKIP_DFCHECK 4 /* Ok to skip DF conflict checks */
diff --git a/read-cache-v5.c b/read-cache-v5.c
index b14505a..85b912b 100644
--- a/read-cache-v5.c
+++ b/read-cache-v5.c
@@ -673,9 +673,603 @@ static int read_index_v5(struct index_state *istate, void
*mmap,
return 0;
}
+#define WRITE_BUFFER_SIZE 8192
+static unsigned char write_buffer[WRITE_BUFFER_SIZE];
+static unsigned long write_buffer_len;
+
+static int ce_write_flush(int fd)
+{
+ unsigned int buffered = write_buffer_len;
+ if (buffered) {
+ if (write_in_full(fd, write_buffer, buffered) != buffered)
+ return -1;
+ write_buffer_len = 0;
+ }
+ return 0;
+}
+
+static int ce_write(uint32_t *crc, int fd, void *data, unsigned int len)
+{
+ if (crc)
+ *crc = crc32(*crc, (Bytef*)data, len);
+ while (len) {
+ unsigned int buffered = write_buffer_len;
+ unsigned int partial = WRITE_BUFFER_SIZE - buffered;
+ if (partial len)
+ partial = len;
+ memcpy(write_buffer + buffered, data, partial);
+ buffered += partial;
+ if (buffered == WRITE_BUFFER_SIZE) {
+ write_buffer_len = buffered;
+ if (ce_write_flush(fd))
+ return -1;
+ buffered = 0;
+ }
+ write_buffer_len = buffered;
+ len -= partial;
+ data = (char *) data + partial;
+ }
+ return 0;
+}
+
+static int ce_flush(int fd)
+{
+ unsigned int left = write_buffer_len;
+
+ if (left)
+ write_buffer_len = 0;
+
+ if (write_in_full(fd, write_buffer, left) != left)
+ return -1;
+
+ return 0;
+}
+
+static void ce_smudge_racily_clean_entry(struct cache_entry *ce)
+{
+ /*
+* This method shall only be called if the timestamp of ce
+* is racy (check with is_racy_timestamp). If the timestamp
+* is racy, the writer will set the CE_SMUDGED flag.
+*
+* The reader (match_stat_basic) will then take care
+* of checking if the entry is really changed or not, by
+* taking into account the size and the stat_crc and if
+* that hasn't changed checking the sha1.
+*/
+ ce-ce_flags |= CE_SMUDGED;
+}
+
+char *super_directory(const char *filename)
+{
+ char *slash;
+
+ slash = strrchr(filename, '/');
+ if (slash)
+ return xmemdupz(filename, slash-filename);
+ return NULL;
+}
+
+struct directory_entry *init_directory_entry(char *pathname, int len)
+{
+ struct directory_entry *de = xmalloc(directory_entry_size(len));
+
+ memcpy(de-pathname, pathname, len);
+ de-pathname[len] = '\0';
+ de-de_flags = 0;
+ de-de_foffset= 0;
+ de-de_cr = 0;
+ de-de_ncr= 0;
+ de-de_nsubtrees = 0;
+ de-de_nfiles = 0;
+ de-de_nentries = 0;
+ memset(de-sha1, 0, 20);
+ de-de_pathlen= len;
+ de-next = NULL;
+ de-next_hash = NULL;
+ de-ce= NULL;
+ de-ce_last = NULL;
+ de-conflict = NULL;
+ de-conflict_last = NULL;
+ de-conflict_size = 0;
+ return de;
+}
+
+static void ondisk_from_directory_entry(struct
[PATCH v3 22/24] p0003-index.sh: add perf test for the index formats
From: Thomas Rast tr...@inf.ethz.ch Add a performance test for index version [23]/4/5 by using git update-index --index-version=x, thus testing both the reader and the writer speed of all index formats. Signed-off-by: Thomas Rast tr...@inf.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- t/perf/p0003-index.sh | 63 +++ 1 file changed, 63 insertions(+) create mode 100755 t/perf/p0003-index.sh diff --git a/t/perf/p0003-index.sh b/t/perf/p0003-index.sh new file mode 100755 index 000..5360175 --- /dev/null +++ b/t/perf/p0003-index.sh @@ -0,0 +1,63 @@ +#!/bin/sh + +test_description=Tests index versions [23]/4/5 + +. ./perf-lib.sh + +test_perf_large_repo + +test_expect_success convert to v3 + git update-index --index-version=2 + + +test_perf v[23]: update-index + git update-index --index-version=2 /dev/null + + +subdir=$(git ls-files | sed 's#/[^/]*$##' | grep -v '^$' | uniq | tail -n 30 | head -1) + +test_perf v[23]: grep nonexistent -- subdir + test_must_fail git grep nonexistent -- $subdir /dev/null + + +test_perf v[23]: ls-files -- subdir + git ls-files $subdir /dev/null + + +test_expect_success convert to v4 + git update-index --index-version=4 + + +test_perf v4: update-index + git update-index --index-version=4 /dev/null + + +test_perf v4: grep nonexistent -- subdir + test_must_fail git grep nonexistent -- $subdir /dev/null + + +test_perf v4: ls-files -- subdir + git ls-files $subdir /dev/null + + +test_expect_success convert to v5 + git update-index --index-version=5 + + +test_perf v5: update-index + git update-index --index-version=5 /dev/null + + +test_perf v5: ls-files + git ls-files /dev/null + + +test_perf v5: grep nonexistent -- subdir + test_must_fail git grep nonexistent -- $subdir /dev/null + + +test_perf v5: ls-files -- subdir + git ls-files $subdir /dev/null + + +test_done -- 1.8.3.4.1231.g9fbf354.dirty -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 10/24] make sure partially read index is not changed
A partially read index file currently cannot be written to disk. Make
sure that never happens, by erroring out when a caller tries to write a
partially read index. Do the same when trying to re-read a partially
read index without having discarded it first to avoid loosing any
information.
Forcing the caller to load the right part of the index file instead of
re-reading it when changing it, gives a bit of a performance advantage,
by avoiding to read parts of the index twice.
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache.c | 4
1 file changed, 4 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 38b9a04..7a27f9b 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1332,6 +1332,8 @@ int read_index_filtered_from(struct index_state *istate,
const char *path,
void *mmap;
size_t mmap_size;
+ if (istate-filter_opts)
+ die(BUG: Can't re-read partially read index);
errno = EBUSY;
if (istate-initialized)
return istate-cache_nr;
@@ -1455,6 +1457,8 @@ void update_index_if_able(struct index_state *istate,
struct lock_file *lockfile
int write_index(struct index_state *istate, int newfd)
{
+ if (istate-filter_opts)
+ die(BUG: index: cannot write a partially read index);
return istate-ops-write_index(istate, newfd);
}
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 07/24] read-cache: Re-read index if index file changed
Add the possibility of re-reading the index file, if it changed
while reading.
The index file might change during the read, causing outdated
information to be displayed. We check if the index file changed
by using its stat data as heuristic.
Helped-by: Ramsay Jones ram...@ramsay1.demon.co.uk
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache.c | 65 +++-
1 file changed, 34 insertions(+), 31 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index aa17ce7..2d12601 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1296,8 +1296,8 @@ int read_index(struct index_state *istate)
/* remember to discard_cache() before reading a different cache! */
int read_index_from(struct index_state *istate, const char *path)
{
- int fd;
- struct stat st;
+ int fd, err, i;
+ struct stat_validity sv;
struct cache_header *hdr;
void *mmap;
size_t mmap_size;
@@ -1309,43 +1309,46 @@ int read_index_from(struct index_state *istate, const
char *path)
errno = ENOENT;
istate-timestamp.sec = 0;
istate-timestamp.nsec = 0;
-
- fd = open(path, O_RDONLY);
- if (fd 0) {
- if (errno == ENOENT) {
- initialize_index(istate, 0);
- return 0;
+ sv.sd = NULL;
+ for (i = 0; i 50; i++) {
+ err = 0;
+ fd = open(path, O_RDONLY);
+ if (fd 0) {
+ if (errno == ENOENT) {
+ initialize_index(istate, 0);
+ return 0;
+ }
+ die_errno(index file open failed);
}
- die_errno(index file open failed);
- }
- if (fstat(fd, st))
- die_errno(cannot stat the open index);
+ stat_validity_update(sv, fd);
+ if (!sv.sd)
+ die_errno(cannot stat the open index);
- errno = EINVAL;
- mmap_size = xsize_t(st.st_size);
- mmap = xmmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd,
0);
- close(fd);
- if (mmap == MAP_FAILED)
- die_errno(unable to map index file);
+ errno = EINVAL;
+ mmap_size = xsize_t(sv.sd-sd_size);
+ mmap = xmmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE, fd, 0);
+ close(fd);
+ if (mmap == MAP_FAILED)
+ die_errno(unable to map index file);
- hdr = mmap;
- if (verify_hdr_version(istate, hdr, mmap_size) 0)
- goto unmap;
+ hdr = mmap;
+ if (verify_hdr_version(istate, hdr, mmap_size) 0)
+ err = 1;
- if (istate-ops-verify_hdr(mmap, mmap_size) 0)
- goto unmap;
+ if (istate-ops-verify_hdr(mmap, mmap_size) 0)
+ err = 1;
- if (istate-ops-read_index(istate, mmap, mmap_size) 0)
- goto unmap;
- istate-timestamp.sec = st.st_mtime;
- istate-timestamp.nsec = ST_MTIME_NSEC(st);
+ if (istate-ops-read_index(istate, mmap, mmap_size) 0)
+ err = 1;
+ istate-timestamp.sec = sv.sd-sd_mtime.sec;
+ istate-timestamp.nsec = sv.sd-sd_mtime.nsec;
- munmap(mmap, mmap_size);
- return istate-cache_nr;
+ munmap(mmap, mmap_size);
+ if (stat_validity_check(sv, path) !err)
+ return istate-cache_nr;
+ }
-unmap:
- munmap(mmap, mmap_size);
die(index file corrupt);
}
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 13/24] documentation: add documentation of the index-v5 file format
Add a documentation of the index file format version 5 to
Documentation/technical.
Helped-by: Michael Haggerty mhag...@alum.mit.edu
Helped-by: Junio C Hamano gits...@pobox.com
Helped-by: Thomas Rast tr...@student.ethz.ch
Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com
Helped-by: Robin Rosenberg robin.rosenb...@dewire.com
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
Documentation/technical/index-file-format-v5.txt | 301 +++
1 file changed, 301 insertions(+)
create mode 100644 Documentation/technical/index-file-format-v5.txt
diff --git a/Documentation/technical/index-file-format-v5.txt
b/Documentation/technical/index-file-format-v5.txt
new file mode 100644
index 000..5209c02
--- /dev/null
+++ b/Documentation/technical/index-file-format-v5.txt
@@ -0,0 +1,301 @@
+GIT index format
+
+
+== The git index
+
+ The git index file (.git/index) documents the status of the files
+ in the git staging area.
+
+ The staging area is used for preparing commits, merging, etc.
+
+== The git index file format
+
+ All binary numbers are in network byte order. Version 5 is described
+ here. The index file consists of various sections. They appear in
+ the following order in the file.
+
+ - header: the description of the index format, including it's signature,
+ version and various other fields that are used internally.
+
+ - diroffsets (ndir entries of direcotry offset): A 4-byte offset
+ relative to the beginning of the direntries block (see below)
+ for each of the ndir directories in the index, sorted by pathname
+ (of the directory it's pointing to). [1]
+
+ - direntries (ndir entries of directory offset): A directory entry
+ for each of the ndir directories in the index, sorted by pathname
+ (see below). [2]
+
+ - fileoffsets (nfile entries of file offset): A 4-byte offset
+ relative to the beginning of the fileentries block (see below)
+ for each of the nfile files in the index. [1]
+
+ - fileentries (nfile entries of file entry): A file entry for
+ each of the nfile files in the index (see below).
+
+ - crdata: A number of entries for conflicted data/resolved conflicts
+ (see below).
+
+ - Extensions (Currently none, see below in the future)
+
+ Extensions are identified by signature. Optional extensions can
+ be ignored if GIT does not understand them.
+
+ GIT supports an arbitrary number of extension, but currently none
+ is implemented. [3]
+
+ extsig (32-bits): extension signature. If the first byte is 'A'..'Z'
+ the extension is optional and can be ignored.
+
+ extsize (32-bits): size of the extension, excluding the header
+ (extsig, extsize, extchecksum).
+
+ extchecksum (32-bits): crc32 checksum of the extension signature
+ and size.
+
+- Extension data.
+
+== Header
+ sig (32-bits): Signature:
+ The signature is { 'D', 'I', 'R', 'C' } (stands for dircache)
+
+ vnr (32-bits): Version number:
+ The current supported versions are 2, 3, 4 and 5.
+
+ nfile (32-bits): number of file entries in the index.
+
+ ndir (32-bits): number of directories in the index.
+
+ fblockoffset (32-bits): offset to the file block, relative to the
+ beginning of the file.
+
+ - Offset to the extensions.
+
+ nextensions (32-bits): number of extensions.
+
+ extoffset (32-bits): offset to the extension. (Possibly none, as
+ many as indicated in the 4-byte number of extensions)
+
+ headercrc (32-bits): crc checksum including the header and the
+ offsets to the extensions.
+
+
+== Directory offsets (diroffsets)
+
+ diroffset (32-bits): offset to the directory relative to the
+beginning of the index file. There are ndir + 1 offsets in the
+diroffset table, the last is pointing to the end of the last
+direntry. With this last entry, we are able to replace the strlen
+of the directory name when reading the directory name, by
+calculating it from diroffset[n+1]-diroffset[n]-61. 61 is the
+size of the directory data, which follows each each directory +
+the crc sum + the NUL byte.
+
+ This part is needed for making the directory entries bisectable and
+thus allowing a binary search.
+
+== Directory entry (direntries)
+
+ Directory entries are sorted in lexicographic order by the name
+of their path starting with the root.
+
+ pathname (variable length, nul terminated): relative to top level
+directory (without the leading slash). '/' is used as path
+separator. A string of length 0 ('') indicates the root directory.
+The special path components ., and .. (without quotes) are
+disallowed. The path also includes a trailing slash. [9]
+
+ foffset (32-bits): offset to the lexicographically first file in
+the file offsets (fileoffsets), relative to the beginning of
+the fileoffset block.
+
+ cr (32-bits): offset to conflicted/resolved data at the end
[PATCH v3 19/24] read-cache: write index-v5 cache-tree data
Write the cache-tree data for the index version 5 file format. The
in-memory cache-tree data is converted to the ondisk format, by adding
it to the directory entries, that were compiled from the cache-entries
in the step before.
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache-v5.c | 53 +
1 file changed, 53 insertions(+)
diff --git a/read-cache-v5.c b/read-cache-v5.c
index 85b912b..ed52b7c 100644
--- a/read-cache-v5.c
+++ b/read-cache-v5.c
@@ -891,6 +891,57 @@ static struct conflict_entry
*create_conflict_entry_from_ce(struct cache_entry *
return create_new_conflict(ce-name, ce_namelen(ce), pathlen);
}
+static void convert_one_to_ondisk_v5(struct hash_table *table, struct
cache_tree *it,
+ const char *path, int pathlen, uint32_t crc)
+{
+ int i;
+ struct directory_entry *found, *search;
+
+ crc = crc32(crc, (Bytef*)path, pathlen);
+ found = lookup_hash(crc, table);
+ search = found;
+ while (search strcmp(path, search-pathname + search-de_pathlen -
strlen(path)) != 0)
+ search = search-next_hash;
+ if (!search)
+ return;
+ /*
+* The number of subtrees is already calculated by
+* compile_directory_data, therefore we only need to
+* add the entry_count
+*/
+ search-de_nentries = it-entry_count;
+ if (0 = it-entry_count)
+ hashcpy(search-sha1, it-sha1);
+ if (strcmp(path, ) != 0)
+ crc = crc32(crc, (Bytef*)/, 1);
+
+#if DEBUG
+ if (0 = it-entry_count)
+ fprintf(stderr, cache-tree %.*s (%d ent, %d subtree) %s\n,
+ pathlen, path, it-entry_count, it-subtree_nr,
+ sha1_to_hex(it-sha1));
+ else
+ fprintf(stderr, cache-tree %.*s (%d subtree) invalid\n,
+ pathlen, path, it-subtree_nr);
+#endif
+
+ for (i = 0; i it-subtree_nr; i++) {
+ struct cache_tree_sub *down = it-down[i];
+ if (i) {
+ struct cache_tree_sub *prev = it-down[i-1];
+ if (subtree_name_cmp(down-name, down-namelen,
+prev-name, prev-namelen) = 0)
+ die(fatal - unsorted cache subtree);
+ }
+ convert_one_to_ondisk_v5(table, down-cache_tree, down-name,
down-namelen, crc);
+ }
+}
+
+static void cache_tree_to_ondisk_v5(struct hash_table *table, struct
cache_tree *root)
+{
+ convert_one_to_ondisk_v5(table, root, , 0, 0);
+}
+
static void ce_queue_push(struct cache_entry **head,
struct cache_entry **tail,
struct cache_entry *ce)
@@ -961,6 +1012,8 @@ static struct directory_entry
*compile_directory_data(struct index_state *istate
add_part_to_conflict_entry(search, conflict_entry,
conflict_part);
}
}
+ if (istate-cache_tree)
+ cache_tree_to_ondisk_v5(table, istate-cache_tree);
return de;
}
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 15/24] read-cache: read index-v5
Make git read the index file version 5 without complaining.
This version of the reader doesn't read neither the cache-tree
nor the resolve undo data, but doesn't choke on an index that
includes such data.
Helped-by: Junio C Hamano gits...@pobox.com
Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com
Helped-by: Thomas Rast tr...@student.ethz.ch
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
Makefile| 1 +
cache.h | 32 +++-
read-cache-v5.c | 473
read-cache.h| 1 +
4 files changed, 506 insertions(+), 1 deletion(-)
create mode 100644 read-cache-v5.c
diff --git a/Makefile b/Makefile
index afae23e..a55206d 100644
--- a/Makefile
+++ b/Makefile
@@ -857,6 +857,7 @@ LIB_OBJS += quote.o
LIB_OBJS += reachable.o
LIB_OBJS += read-cache.o
LIB_OBJS += read-cache-v2.o
+LIB_OBJS += read-cache-v5.o
LIB_OBJS += reflog-walk.o
LIB_OBJS += refs.o
LIB_OBJS += remote.o
diff --git a/cache.h b/cache.h
index 714a334..89f556b 100644
--- a/cache.h
+++ b/cache.h
@@ -99,7 +99,7 @@ unsigned long git_deflate_bound(git_zstream *, unsigned long);
#define CACHE_SIGNATURE 0x44495243 /* DIRC */
#define INDEX_FORMAT_LB 2
-#define INDEX_FORMAT_UB 4
+#define INDEX_FORMAT_UB 5
/*
* The cache_time is just the low 32 bits of the
@@ -121,6 +121,15 @@ struct stat_data {
unsigned int sd_size;
};
+/*
+ * The *next pointer is used in read_entries_v5 for holding
+ * all the elements of a directory, and points to the next
+ * cache_entry in a directory.
+ *
+ * It is reset by the add_name_hash call in set_index_entry
+ * to set it to point to the next cache_entry in the
+ * correct in-memory format ordering.
+ */
struct cache_entry {
struct stat_data ce_stat_data;
unsigned int ce_mode;
@@ -132,11 +141,17 @@ struct cache_entry {
char name[FLEX_ARRAY]; /* more */
};
+#define CE_NAMEMASK (0x0fff)
#define CE_STAGEMASK (0x3000)
#define CE_EXTENDED (0x4000)
#define CE_VALID (0x8000)
+#define CE_SMUDGED (0x0400) /* index v5 only flag */
#define CE_STAGESHIFT 12
+#define CONFLICT_CONFLICTED (0x8000)
+#define CONFLICT_STAGESHIFT 13
+#define CONFLICT_STAGEMASK (0x6000)
+
/*
* Range 0x in ce_flags is divided into
* two parts: in-memory flags and on-disk ones.
@@ -173,6 +188,19 @@ struct cache_entry {
#define CE_EXTENDED_FLAGS (CE_INTENT_TO_ADD | CE_SKIP_WORKTREE)
/*
+ * Representation of the extended on-disk flags in the v5 format.
+ * They must not collide with the ordinary on-disk flags, and need to
+ * fit in 16 bits. Note however that v5 does not save the name
+ * length.
+ */
+#define CE_INTENT_TO_ADD_V5 (0x4000)
+#define CE_SKIP_WORKTREE_V5 (0x0800)
+#define CE_INVALID_V5(0x0200)
+#if (CE_VALID|CE_STAGEMASK)
(CE_INTENTTOADD_V5|CE_SKIPWORKTREE_V5|CE_INVALID_V5)
+#error v5 on-disk flags collide with ordinary on-disk flags
+#endif
+
+/*
* Safeguard to avoid saving wrong flags:
* - CE_EXTENDED2 won't get saved until its semantic is known
* - Bits in 0x have been saved in ce_flags already
@@ -213,6 +241,8 @@ static inline unsigned create_ce_flags(unsigned stage)
#define ce_skip_worktree(ce) ((ce)-ce_flags CE_SKIP_WORKTREE)
#define ce_mark_uptodate(ce) ((ce)-ce_flags |= CE_UPTODATE)
+#define conflict_stage(c) ((CONFLICT_STAGEMASK (c)-flags)
CONFLICT_STAGESHIFT)
+
#define ce_permissions(mode) (((mode) 0100) ? 0755 : 0644)
static inline unsigned int create_ce_mode(unsigned int mode)
{
diff --git a/read-cache-v5.c b/read-cache-v5.c
new file mode 100644
index 000..799b8e7
--- /dev/null
+++ b/read-cache-v5.c
@@ -0,0 +1,473 @@
+#include cache.h
+#include read-cache.h
+#include resolve-undo.h
+#include cache-tree.h
+#include dir.h
+#include pathspec.h
+
+#define ptr_add(x,y) ((void *)(((char *)(x)) + (y)))
+
+struct cache_header_v5 {
+ uint32_t hdr_ndir;
+ uint32_t hdr_fblockoffset;
+ uint32_t hdr_nextension;
+};
+
+struct directory_entry {
+ struct directory_entry **sub;
+ struct directory_entry *next;
+ struct directory_entry *next_hash;
+ struct cache_entry *ce;
+ struct cache_entry *ce_last;
+ struct conflict_entry *conflict;
+ struct conflict_entry *conflict_last;
+ uint32_t conflict_size;
+ uint32_t de_foffset;
+ uint32_t de_cr;
+ uint32_t de_ncr;
+ uint32_t de_nsubtrees;
+ uint32_t de_nfiles;
+ uint32_t de_nentries;
+ unsigned char sha1[20];
+ uint16_t de_flags;
+ uint32_t de_pathlen;
+ char pathname[FLEX_ARRAY];
+};
+
+struct conflict_part {
+ struct conflict_part *next;
+ uint16_t flags;
+ uint16_t entry_mode;
+ unsigned char sha1[20];
+};
+
+struct conflict_entry {
+ struct conflict_entry *next;
+ uint32_t nfileconflicts;
+ struct conflict_part *entries;
+ uint32_t namelen;
+ uint32_t pathlen;
+ char name[FLEX_ARRAY];
+};
+
+#define
[PATCH v3 20/24] read-cache: write resolve-undo data for index-v5
Make git read the resolve-undo data from the index.
Since the resolve-undo data is joined with the conflicts in
the ondisk format of the index file version 5, conflicts and
resolved data is read at the same time, and the resolve-undo
data is then converted to the in-memory format.
Helped-by: Thomas Rast tr...@student.ethz.ch
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache-v5.c | 89 +
1 file changed, 89 insertions(+)
diff --git a/read-cache-v5.c b/read-cache-v5.c
index ed52b7c..10960fd 100644
--- a/read-cache-v5.c
+++ b/read-cache-v5.c
@@ -942,6 +942,94 @@ static void cache_tree_to_ondisk_v5(struct hash_table
*table, struct cache_tree
convert_one_to_ondisk_v5(table, root, , 0, 0);
}
+static void resolve_undo_to_ondisk_v5(struct hash_table *table,
+ struct string_list *resolve_undo,
+ unsigned int *ndir,
+ unsigned int *total_dir_len,
+ struct directory_entry *de)
+{
+ struct string_list_item *item;
+ struct directory_entry *search;
+
+ if (!resolve_undo)
+ return;
+ for_each_string_list_item(item, resolve_undo) {
+ struct conflict_entry *conflict_entry;
+ struct resolve_undo_info *ui = item-util;
+ char *super;
+ int i, dir_len, len;
+ uint32_t crc;
+ struct directory_entry *found, *current, *new_tree;
+
+ if (!ui)
+ continue;
+
+ super = super_directory(item-string);
+ dir_len = super ? strlen(super) : 0;
+ crc = crc32(0, (Bytef*)super, dir_len);
+ found = lookup_hash(crc, table);
+ current = NULL;
+ new_tree = NULL;
+
+ while (!found) {
+ struct directory_entry *new;
+
+ new = init_directory_entry(super, dir_len);
+ if (!current)
+ current = new;
+ insert_directory_entry(new, table, total_dir_len, ndir,
crc);
+ if (new_tree != NULL)
+ new-de_nsubtrees = 1;
+ new-next = new_tree;
+ new_tree = new;
+ super = super_directory(super);
+ dir_len = super ? strlen(super) : 0;
+ crc = crc32(0, (Bytef*)super, dir_len);
+ found = lookup_hash(crc, table);
+ }
+ search = found;
+ while (search-next_hash strcmp(super, search-pathname) !=
0)
+ search = search-next_hash;
+ if (search !current)
+ current = search;
+ if (!search !current)
+ current = new_tree;
+ if (!super new_tree) {
+ new_tree-next = de-next;
+ de-next = new_tree;
+ de-de_nsubtrees++;
+ } else if (new_tree) {
+ struct directory_entry *temp;
+
+ search = de-next;
+ while (strcmp(super, search-pathname))
+ search = search-next;
+ temp = new_tree;
+ while (temp-next)
+ temp = temp-next;
+ search-de_nsubtrees++;
+ temp-next = search-next;
+ search-next = new_tree;
+ }
+
+ len = strlen(item-string);
+ conflict_entry = create_new_conflict(item-string, len,
current-de_pathlen);
+ add_conflict_to_directory_entry(current, conflict_entry);
+ for (i = 0; i 3; i++) {
+ if (ui-mode[i]) {
+ struct conflict_part *cp;
+
+ cp = xmalloc(sizeof(struct conflict_part));
+ cp-flags = (i + 1) CONFLICT_STAGESHIFT;
+ cp-entry_mode = ui-mode[i];
+ cp-next = NULL;
+ hashcpy(cp-sha1, ui-sha1[i]);
+ add_part_to_conflict_entry(current,
conflict_entry, cp);
+ }
+ }
+ }
+}
+
static void ce_queue_push(struct cache_entry **head,
struct cache_entry **tail,
struct cache_entry *ce)
@@ -1014,6 +1102,7 @@ static struct directory_entry
*compile_directory_data(struct index_state *istate
}
if (istate-cache_tree)
cache_tree_to_ondisk_v5(table, istate-cache_tree);
+ resolve_undo_to_ondisk_v5(table, istate-resolve_undo,
[PATCH v3 08/24] add documentation for the index api
Add documentation for the index reading api. This also includes documentation for the new api functions introduced in the next patch. Helped-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/api-in-core-index.txt | 54 +-- 1 file changed, 50 insertions(+), 4 deletions(-) diff --git a/Documentation/technical/api-in-core-index.txt b/Documentation/technical/api-in-core-index.txt index adbdbf5..9b8c37c 100644 --- a/Documentation/technical/api-in-core-index.txt +++ b/Documentation/technical/api-in-core-index.txt @@ -1,14 +1,60 @@ in-core index API = +Reading API +--- + +`cache`:: + + An array of cache entries. This is used to access the cache + entries directly. Use `index_name_pos` to search for the + index of a specific cache entry. + +`read_index_filtered`:: + + Read a part of the index, filtered by the pathspec given in + the opts. The function may load more than necessary, so the + caller still responsible to apply filters appropriately. The + filtering is only done for performance reasons, as it's + possible to only read part of the index when the on-disk + format is index-v5. + + To iterate only over the entries that match the pathspec, use + the for_each_index_entry function. + +`read_index`:: + + Read the whole index file from disk. + +`index_name_pos`:: + + Find a cache_entry with name in the index. Returns pos if an + entry is matched exactly and -1-pos if an entry is matched + partially. + e.g. + index: + file1 + file2 + path/file1 + zzz + + index_name_pos(path/file1, 10) returns 2, while + index_name_pos(path, 4) returns -3 + +`for_each_index_entry`:: + + Iterates over all cache_entries in the index filtered by + filter_opts in the index_state. For each cache entry fn is + executed with cb_data as callback data. From within the loop + do `return 0` to continue, or `return 1` to break the loop. + +TODO + Talk about read-cache.c and cache-tree.c, things like: -* cache - the_index macros -* read_index() * write_index() * ie_match_stat() and ie_modified(); how they are different and when to use which. -* index_name_pos() * remove_index_entry_at() * remove_file_from_index() * add_file_to_index() @@ -18,4 +64,4 @@ Talk about read-cache.c and cache-tree.c, things like: * cache_tree_invalidate_path() * cache_tree_update() -(JC, Linus) +(JC, Linus, Thomas Gummerer) -- 1.8.3.4.1231.g9fbf354.dirty -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v7 0/3] some enhancements for reporting branch tracking info
Jiang Xin worldhello@gmail.com writes: Changes since v6: * s/broken/gone/ in [PATCH 2/3] (branch: mark missing tracking branch as gone) * rewrite commit log for [PATCH 3/3] (status: always show tracking branch even no change) Jiang Xin (3): branch: not report invalid tracking branch branch: mark missing tracking branch as gone status: always show tracking branch even no change builtin/branch.c | 36 remote.c | 72 +-- t/t6040-tracking-info.sh | 89 +--- wt-status.c | 26 +++--- 4 files changed, 175 insertions(+), 48 deletions(-) Thanks; getting clearer to read. I however feel that [1/3] is introducing a regression (what we used to report, gone branches, are hidden), only to correct the regression immediately after it with [2/3]. I wonder if these patches should be combined into one, with justification like with the current code, a branch that is in sync with its upstream, a branch whose upstream no longer exists and a branch that does not build on anything else cannot be distinguished; show the first class as 'same', mark the second class as 'gone', and show only the last one as not having any uptream', or something. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] rebase -i: fix cases ignoring core.commentchar
Eric Sunshine sunsh...@sunshineco.com writes: One of the fixes in this patch addresses an oversight in 180bad3d10fe3a7f (rebase -i: respect core.commentchar, 2013-02-11) which is already in 'maint'. Should I split this patch in two so that the one fix can be applied to 'maint'? We are so close to 1.8.4 final and I do not think it makes much difference in practice, but if we were shooting for the ideal, yeah, as that part of the patch could make core.commentchar useful for rebase-i users who are still on 1.8.3.X maintenance track. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] rebase -i: fix cases ignoring core.commentchar
On Sun, Aug 18, 2013 at 4:29 PM, Junio C Hamano gits...@pobox.com wrote: Eric Sunshine sunsh...@sunshineco.com writes: One of the fixes in this patch addresses an oversight in 180bad3d10fe3a7f (rebase -i: respect core.commentchar, 2013-02-11) which is already in 'maint'. Should I split this patch in two so that the one fix can be applied to 'maint'? We are so close to 1.8.4 final and I do not think it makes much difference in practice, but if we were shooting for the ideal, yeah, as that part of the patch could make core.commentchar useful for rebase-i users who are still on 1.8.3.X maintenance track. I sent a v2 [1] of this submission which does split the patch into two parts: one for 'maint' and one for 'next', with appropriate commit messages. The actual changes to git-rebase--interactive.sh are identical between v1 and v2. [1]: http://git.661346.n2.nabble.com/PATCH-v2-0-2-fix-cases-of-rebase-i-ignoring-core-commentchar-td7594009.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 08/24] add documentation for the index api
On Aug 18, 2013, at 3:41 PM, Thomas Gummerer wrote: Add documentation for the index reading api. This also includes documentation for the new api functions introduced in the next patch. Helped-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com Signed-off-by: Thomas Gummerer t.gumme...@gmail.com --- Documentation/technical/api-in-core-index.txt | 54 +-- 1 file changed, 50 insertions(+), 4 deletions(-) diff --git a/Documentation/technical/api-in-core-index.txt b/Documentation/technical/api-in-core-index.txt index adbdbf5..9b8c37c 100644 --- a/Documentation/technical/api-in-core-index.txt +++ b/Documentation/technical/api-in-core-index.txt @@ -1,14 +1,60 @@ in-core index API = +Reading API +--- + +`cache`:: + + An array of cache entries. This is used to access the cache + entries directly. Use `index_name_pos` to search for the + index of a specific cache entry. + +`read_index_filtered`:: + + Read a part of the index, filtered by the pathspec given in + the opts. The function may load more than necessary, so the + caller still responsible to apply filters appropriately. The Grammatical nit: ...the caller is still responsible for applying… + filtering is only done for performance reasons, as it's + possible to only read part of the index when the on-disk + format is index-v5. + + To iterate only over the entries that match the pathspec, use + the for_each_index_entry function. + +`read_index`:: + + Read the whole index file from disk. + +`index_name_pos`:: + + Find a cache_entry with name in the index. Returns pos if an + entry is matched exactly and -1-pos if an entry is matched + partially. + e.g. + index: + file1 + file2 + path/file1 + zzz + + index_name_pos(path/file1, 10) returns 2, while + index_name_pos(path, 4) returns -3 A couple of these entries won't format correctly. You may want to squash in something like this (sans whitespace damage): --8-- diff --git a/Documentation/technical/api-in-core-index.txt b/Documentation/technical/api-in-core-index.txt index 9b8c37c..d2518c8 100644 --- a/Documentation/technical/api-in-core-index.txt +++ b/Documentation/technical/api-in-core-index.txt @@ -18,9 +18,9 @@ Reading API filtering is only done for performance reasons, as it's possible to only read part of the index when the on-disk format is index-v5. - - To iterate only over the entries that match the pathspec, use - the for_each_index_entry function. ++ +To iterate only over the entries that match the pathspec, use +the for_each_index_entry function. `read_index`:: @@ -30,16 +30,18 @@ Reading API Find a cache_entry with name in the index. Returns pos if an entry is matched exactly and -1-pos if an entry is matched - partially. - e.g. - index: + partially. e.g. ++ + +index: file1 file2 path/file1 zzz - - index_name_pos(path/file1, 10) returns 2, while - index_name_pos(path, 4) returns -3 + ++ +`index_name_pos(path/file1, 10)` returns 2, while +`index_name_pos(path, 4)` returns -3 `for_each_index_entry`:: -- 1.8.4.rc3.500.gc3113b0 --8-- + +`for_each_index_entry`:: + + Iterates over all cache_entries in the index filtered by + filter_opts in the index_state. For each cache entry fn is + executed with cb_data as callback data. From within the loop + do `return 0` to continue, or `return 1` to break the loop. + +TODO + Talk about read-cache.c and cache-tree.c, things like: -* cache - the_index macros -* read_index() * write_index() * ie_match_stat() and ie_modified(); how they are different and when to use which. -* index_name_pos() * remove_index_entry_at() * remove_file_from_index() * add_file_to_index() @@ -18,4 +64,4 @@ Talk about read-cache.c and cache-tree.c, things like: * cache_tree_invalidate_path() * cache_tree_update() -(JC, Linus) +(JC, Linus, Thomas Gummerer) -- 1.8.3.4.1231.g9fbf354.dirty -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 10/24] make sure partially read index is not changed
On Sun, Aug 18, 2013 at 3:41 PM, Thomas Gummerer t.gumme...@gmail.com wrote:
A partially read index file currently cannot be written to disk. Make
sure that never happens, by erroring out when a caller tries to write a
s/,//
partially read index. Do the same when trying to re-read a partially
read index without having discarded it first to avoid loosing any
s/loosing/losing/
information.
Forcing the caller to load the right part of the index file instead of
re-reading it when changing it, gives a bit of a performance advantage,
s/it,/it/ (or s/file instead/file, instead/)
s/advantage,/advantage/
by avoiding to read parts of the index twice.
/to read/reading/
More below...
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
read-cache.c | 4
1 file changed, 4 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 38b9a04..7a27f9b 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1332,6 +1332,8 @@ int read_index_filtered_from(struct index_state
*istate, const char *path,
void *mmap;
size_t mmap_size;
+ if (istate-filter_opts)
+ die(BUG: Can't re-read partially read index);
errno = EBUSY;
if (istate-initialized)
return istate-cache_nr;
@@ -1455,6 +1457,8 @@ void update_index_if_able(struct index_state *istate,
struct lock_file *lockfile
int write_index(struct index_state *istate, int newfd)
{
+ if (istate-filter_opts)
+ die(BUG: index: cannot write a partially read index);
Consistency nit:
In the preceding hunk, the error message starts BUG: Can't..., but
in this hunk we have BUG: index: cannot
So, BUG: is the prefix of one, but BUG: index: is the prefix of the other.
Spelling difference: Can't vs. cannot.
Capitalization difference: Can't vs. cannot.
return istate-ops-write_index(istate, newfd);
}
--
1.8.3.4.1231.g9fbf354.dirty
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 2/2] rebase -i: fix core.commentchar regression
Eric Sunshine sunsh...@sunshineco.com writes:
Intended for 'next'.
Thanks.
Will queue on top of es/rebase-i-no-abbrev, but we have a chance to
rebuild 'next' after 1.8.4 release, so we may want to squash this
into the problematic commit when it happens.
git-rebase--interactive.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 3733312..f246810 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -693,7 +693,7 @@ transform_todo_ids () {
while read -r command rest
do
case $command in
- '#'* | exec)
+ $comment_char* | exec)
# Be careful for oddball commands like 'exec'
# that do not have a SHA-1 at the beginning of $rest.
;;
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add command `git bisect state` that checks if the current bisection process has reached the first bad commit.
Mattias Andrée maand...@member.fsf.org writes: This patch allows you to an automated by section by just like if it was a manual, i.e. stating with `git bisect start git bisect bad git bisect good commit` but then type: while ! git bisect state; do test command git bisect good || git bisect bad done Hmph, so this is git bisect run turned inside out? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCHv4] repack: rewrite the shell script in C.
This is the beginning of the rewrite of the repacking.
* rename get_pack_sha1_list to get_pack_filename_list, which
* reads the pack directory only once as suggested by Rene.
* fix the grammar as suggested by Kyle.
All tests have been positive at least once now.
However there is still one non-deterministic error occuring,
I am tired to search for it now, I'll get it working tommorow.
Signed-off-by: Stefan Beller stefanbel...@googlemail.com
---
Makefile| 2 +-
builtin.h | 1 +
builtin/repack.c| 372
git-repack.sh = contrib/examples/git-repack.sh | 0
git.c | 1 +
5 files changed, 375 insertions(+), 1 deletion(-)
create mode 100644 builtin/repack.c
rename git-repack.sh = contrib/examples/git-repack.sh (100%)
diff --git a/Makefile b/Makefile
index ef442eb..4ec5bbe 100644
--- a/Makefile
+++ b/Makefile
@@ -464,7 +464,6 @@ SCRIPT_SH += git-pull.sh
SCRIPT_SH += git-quiltimport.sh
SCRIPT_SH += git-rebase.sh
SCRIPT_SH += git-remote-testgit.sh
-SCRIPT_SH += git-repack.sh
SCRIPT_SH += git-request-pull.sh
SCRIPT_SH += git-stash.sh
SCRIPT_SH += git-submodule.sh
@@ -972,6 +971,7 @@ BUILTIN_OBJS += builtin/reflog.o
BUILTIN_OBJS += builtin/remote.o
BUILTIN_OBJS += builtin/remote-ext.o
BUILTIN_OBJS += builtin/remote-fd.o
+BUILTIN_OBJS += builtin/repack.o
BUILTIN_OBJS += builtin/replace.o
BUILTIN_OBJS += builtin/rerere.o
BUILTIN_OBJS += builtin/reset.o
diff --git a/builtin.h b/builtin.h
index 8afa2de..b56cf07 100644
--- a/builtin.h
+++ b/builtin.h
@@ -102,6 +102,7 @@ extern int cmd_reflog(int argc, const char **argv, const
char *prefix);
extern int cmd_remote(int argc, const char **argv, const char *prefix);
extern int cmd_remote_ext(int argc, const char **argv, const char *prefix);
extern int cmd_remote_fd(int argc, const char **argv, const char *prefix);
+extern int cmd_repack(int argc, const char **argv, const char *prefix);
extern int cmd_repo_config(int argc, const char **argv, const char *prefix);
extern int cmd_rerere(int argc, const char **argv, const char *prefix);
extern int cmd_reset(int argc, const char **argv, const char *prefix);
diff --git a/builtin/repack.c b/builtin/repack.c
new file mode 100644
index 000..bfaaad7
--- /dev/null
+++ b/builtin/repack.c
@@ -0,0 +1,372 @@
+/*
+ * The shell version was written by Linus Torvalds (2005) and many others.
+ * This is a translation into C by Stefan Beller (2013)
+ */
+
+#include builtin.h
+#include cache.h
+#include dir.h
+#include parse-options.h
+#include run-command.h
+#include sigchain.h
+#include strbuf.h
+#include string-list.h
+
+#include argv-array.h
+
+static const char *const git_repack_usage[] = {
+ N_(git repack [options]),
+ NULL
+};
+
+/* enabled by default since 22c79eab (2008-06-25) */
+static int delta_base_offset = 1;
+
+static int repack_config(const char *var, const char *value, void *cb)
+{
+ if (!strcmp(var, repack.usedeltabaseoffset)) {
+ delta_base_offset = git_config_bool(var, value);
+ return 0;
+ }
+ return git_default_config(var, value, cb);
+}
+
+static void remove_temporary_files() {
+ DIR *dir;
+ struct dirent *e;
+ char *prefix, *path;
+
+ prefix = mkpathdup(.tmp-%d-pack, getpid());
+ path = mkpathdup(%s/pack, get_object_directory());
+
+ dir = opendir(path);
+ while ((e = readdir(dir)) != NULL) {
+ if (!prefixcmp(e-d_name, prefix)) {
+ struct strbuf fname = STRBUF_INIT;
+ strbuf_addf(fname, %s/%s, path, e-d_name);
+ unlink(strbuf_detach(fname, NULL));
+ }
+ }
+ free(prefix);
+ free(path);
+ closedir(dir);
+}
+
+static void remove_pack_on_signal(int signo)
+{
+ remove_temporary_files();
+ sigchain_pop(signo);
+ raise(signo);
+}
+
+/*
+ * Fills the filename list with all the files found in the pack directory
+ * ending with .pack, without that extension.
+ */
+void get_pack_filenames(char *packdir, struct string_list *fname_list)
+{
+ DIR *dir;
+ struct dirent *e;
+ char *path, *suffix, *fname;
+
+ path = mkpathdup(%s/pack, get_object_directory());
+ suffix = .pack;
+
+ dir = opendir(path);
+ while ((e = readdir(dir)) != NULL) {
+ if (!suffixcmp(e-d_name, suffix)) {
+ size_t len = strlen(e-d_name) - strlen(suffix);
+ fname = xmemdupz(e-d_name, len);
+ string_list_append_nodup(fname_list, fname);
+ }
+ }
+ free(path);
+ closedir(dir);
+}
+
+/*
+ * remove_pack will remove any files following the pattern *${SHA1}.{EXT}
+ * where EXT is one of {pack, idx, keep}. The SHA1 consists of 40 chars and
+ * is specified by the sha1 parameter.
+ * path
Re: [PATCH v3 06/24] read-cache: Don't compare uid, gid and ino on cygwin
On 18/08/2013 08:41 PM, Thomas Gummerer wrote:
Cygwin doesn't have uid, gid and ino stats fields. Therefore we should
never check them in the match_stat_data when working on the CYGWIN
platform.
Hmm, this is simply not true ... ;-)
The need to omit the uid, gid and ino fields from the stat checks in
your original code was caused by the schizophrenic stat implementation
in cygwin. (This was also before core.checkstat was implemented; note
the 'check_stat' conditional below ...)
However, since commit f66450ae (cygwin: Remove the Win32 l/stat()
implementation, 22-06-2013), this patch is no longer necessary and
can simply be dropped from this series.
[I have not had time to read your new patches yet, but I seem to remember
being concerned about those platforms which have UNRELIABLE_FSTAT set.
(ie cygwin, MinGW and Windows.)]
ATB,
Ramsay Jones
Signed-off-by: Thomas Gummerer t.gumme...@gmail.com
---
This patch was not tested on Cygwin yet. I think it's needed though,
because the re-reading of the index if it changed will no longer use
it's own index_changed function, but use the stat_validity_check
function instead. Would be great if someone running Cygwin could test
this.
read-cache.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 1f827de..aa17ce7 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -82,6 +82,7 @@ int match_stat_data(const struct stat_data *sd, struct stat
*st)
changed |= CTIME_CHANGED;
#endif
+#if !defined (__CYGWIN__)
if (check_stat) {
if (sd-sd_uid != (unsigned int) st-st_uid ||
sd-sd_gid != (unsigned int) st-st_gid)
@@ -89,6 +90,7 @@ int match_stat_data(const struct stat_data *sd, struct stat
*st)
if (sd-sd_ino != (unsigned int) st-st_ino)
changed |= INODE_CHANGED;
}
+#endif
#ifdef USE_STDEV
/*
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 15/24] read-cache: read index-v5
On Sun, Aug 18, 2013 at 3:42 PM, Thomas Gummerer t.gumme...@gmail.com wrote: Make git read the index file version 5 without complaining. This version of the reader doesn't read neither the cache-tree nor the resolve undo data, but doesn't choke on an index that includes such data. The double-negatives are difficult to digest. Grammatical fixup: --8-- This version of the reader reads neither the cache-tree nor the resolve undo data, however, it won't choke on an index that includes such data. --8-- Helped-by: Junio C Hamano gits...@pobox.com Helped-by: Nguyen Thai Ngoc Duy pclo...@gmail.com Helped-by: Thomas Rast tr...@student.ethz.ch Signed-off-by: Thomas Gummerer t.gumme...@gmail.com -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Does Git now have any C struct version history tracking mechanism?
On Sun, Aug 18, 2013 at 6:33 PM, Zhan Jianyu nasa4...@gmail.com wrote: Such a requirement came into my mind when I am tracking a gloomy C struct , with lengthy list of elements which are either elaborated or opaque. So I use git blame to track it down and found that its original version is quite simple and intuitive. So I think I could just slice out every snapshot of this struct, reading every changelog , to get a better knowledge of what it is and why it should be like this. It seems quite helpful but the process is quite cumbersome. So I wonder if there is already some mechanism fulfilling my requirement in Git. If it doesn't, would it be worthy adding one ? It's already merged to git.git's master quite recently in ed73fe56428eecd2b635473f6a517a183c4713a3 (back in June). You'd invoke git log like this: $ git log -L :struct_or_function_name:filename.c and it will show you the commits and the specific hunks that affect the struct or function name. It's still a bit rough on the edges, for example, doing the following in git.git: $ git log -L :rev_cmdline_info:revision.h Shows three commits (a765499, ca92e59 and 281eee4) where the second one does not touch the struct at all (if you do git show ca92e59 you might gain an insight as to how -L works). But that is not to say that it's not good. The gain really shadows the roughness. nazri -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
