[PATCH 09/24] ewah: add convenient wrapper ewah_serialize_strbuf()

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 ewah/ewah_io.c | 13 +
 ewah/ewok.h|  2 ++
 split-index.c  | 11 ++-
 3 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/ewah/ewah_io.c b/ewah/ewah_io.c
index 1c2d7af..43481b9 100644
--- a/ewah/ewah_io.c
+++ b/ewah/ewah_io.c
@@ -19,6 +19,7 @@
  */
 #include git-compat-util.h
 #include ewok.h
+#include strbuf.h
 
 int ewah_serialize_native(struct ewah_bitmap *self, int fd)
 {
@@ -110,6 +111,18 @@ int ewah_serialize(struct ewah_bitmap *self, int fd)
return ewah_serialize_to(self, write_helper, (void *)(intptr_t)fd);
 }
 
+static int write_strbuf(void *user_data, const void *data, size_t len)
+{
+   struct strbuf *sb = user_data;
+   strbuf_add(sb, data, len);
+   return len;
+}
+
+int ewah_serialize_strbuf(struct ewah_bitmap *self, struct strbuf *sb)
+{
+   return ewah_serialize_to(self, write_strbuf, sb);
+}
+
 int ewah_read_mmap(struct ewah_bitmap *self, const void *map, size_t len)
 {
const uint8_t *ptr = map;
diff --git a/ewah/ewok.h b/ewah/ewok.h
index f6ad190..4d7f5e9 100644
--- a/ewah/ewok.h
+++ b/ewah/ewok.h
@@ -30,6 +30,7 @@
 #  define ewah_calloc xcalloc
 #endif
 
+struct strbuf;
 typedef uint64_t eword_t;
 #define BITS_IN_WORD (sizeof(eword_t) * 8)
 
@@ -97,6 +98,7 @@ int ewah_serialize_to(struct ewah_bitmap *self,
  void *out);
 int ewah_serialize(struct ewah_bitmap *self, int fd);
 int ewah_serialize_native(struct ewah_bitmap *self, int fd);
+int ewah_serialize_strbuf(struct ewah_bitmap *self, struct strbuf *);
 
 int ewah_deserialize(struct ewah_bitmap *self, int fd);
 int ewah_read_mmap(struct ewah_bitmap *self, const void *map, size_t len);
diff --git a/split-index.c b/split-index.c
index 21485e2..968b780 100644
--- a/split-index.c
+++ b/split-index.c
@@ -41,13 +41,6 @@ int read_link_extension(struct index_state *istate,
return 0;
 }
 
-static int write_strbuf(void *user_data, const void *data, size_t len)
-{
-   struct strbuf *sb = user_data;
-   strbuf_add(sb, data, len);
-   return len;
-}
-
 int write_link_extension(struct strbuf *sb,
 struct index_state *istate)
 {
@@ -55,8 +48,8 @@ int write_link_extension(struct strbuf *sb,
strbuf_add(sb, si-base_sha1, 20);
if (!si-delete_bitmap  !si-replace_bitmap)
return 0;
-   ewah_serialize_to(si-delete_bitmap, write_strbuf, sb);
-   ewah_serialize_to(si-replace_bitmap, write_strbuf, sb);
+   ewah_serialize_strbuf(si-delete_bitmap, sb);
+   ewah_serialize_strbuf(si-replace_bitmap, sb);
return 0;
 }
 
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 06/24] untracked cache: record/validate dir mtime and reuse cached output

2015-01-20 Thread Nguyễn Thái Ngọc Duy
The main readdir loop in read_directory_recursive() is replaced with a
new one that checks if cached results of a directory is still valid.

If a file is added or removed from the index, the containing directory
is invalidated (but not its subdirs). If directory's mtime is changed,
the same happens. If a .gitignore is updated, the containing directory
and all subdirs are invalidated recursively. If dir_struct#flags or
other conditions change, the cache is ignored.

If a directory is invalidated, we opendir/readdir/closedir and run the
exclude machinery on that directory listing as usual. If untracked
cache is also enabled, we'll update the cache along the way. If a
directory is validated, we simply pull the untracked listing out from
the cache. The cache also records the list of direct subdirs that we
have to recurse in. Fully excluded directories are seen as untracked
files.

In the best case when no dirs are invalidated, read_directory()
becomes a series of

  stat(dir), open(.gitignore), fstat(), read(), close() and optionally
  hash_sha1_file()

For comparison, standard read_directory() is a sequence of

  opendir(), readdir(), open(.gitignore), fstat(), read(), close(), the
  expensive last_exclude_matching() and closedir().

We already try not to open(.gitignore) if we know it does not exist,
so open/fstat/read/close sequence does not apply to every
directory. The sequence could be reduced further, as noted in
prep_exclude() in another patch. So in theory, the entire best-case
read_directory sequence could be reduced to a series of stat() and
nothing else.

This is not a silver bullet approach. When you compile a C file, for
example, the old .o file is removed and a new one with the same name
created, effectively invalidating the containing directory's cache
(but not its subdirectories). If your build process touches every
directory, this cache adds extra overhead for nothing, so it's a good
idea to separate generated files from tracked files.. Editors may use
the same strategy for saving files. And of course you're out of luck
running your repo on an unsupported filesystem and/or operating system.

Helped-by: Eric Sunshine sunsh...@sunshineco.com
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 121 --
 dir.h |   2 ++
 2 files changed, 121 insertions(+), 2 deletions(-)

diff --git a/dir.c b/dir.c
index fb6ed86..8c989e3 100644
--- a/dir.c
+++ b/dir.c
@@ -37,7 +37,12 @@ enum path_treatment {
 struct cached_dir {
DIR *fdir;
struct untracked_cache_dir *untracked;
+   int nr_files;
+   int nr_dirs;
+
struct dirent *de;
+   const char *file;
+   struct untracked_cache_dir *ucd;
 };
 
 static enum path_treatment read_directory_recursive(struct dir_struct *dir,
@@ -607,6 +612,14 @@ static void invalidate_gitignore(struct untracked_cache 
*uc,
do_invalidate_gitignore(dir);
 }
 
+static void invalidate_directory(struct untracked_cache *uc,
+struct untracked_cache_dir *dir)
+{
+   uc-dir_invalidated++;
+   dir-valid = 0;
+   dir-untracked_nr = 0;
+}
+
 /*
  * Given a file with name fname, read it (either from disk, or from
  * the index if check_index is non-zero), parse it and store the
@@ -1424,6 +1437,39 @@ static enum path_treatment treat_one_path(struct 
dir_struct *dir,
}
 }
 
+static enum path_treatment treat_path_fast(struct dir_struct *dir,
+  struct untracked_cache_dir 
*untracked,
+  struct cached_dir *cdir,
+  struct strbuf *path,
+  int baselen,
+  const struct path_simplify *simplify)
+{
+   strbuf_setlen(path, baselen);
+   if (!cdir-ucd) {
+   strbuf_addstr(path, cdir-file);
+   return path_untracked;
+   }
+   strbuf_addstr(path, cdir-ucd-name);
+   /* treat_one_path() does this before it calls treat_directory() */
+   if (path-buf[path-len - 1] != '/')
+   strbuf_addch(path, '/');
+   if (cdir-ucd-check_only)
+   /*
+* check_only is set as a result of treat_directory() getting
+* to its bottom. Verify again the same set of directories
+* with check_only set.
+*/
+   return read_directory_recursive(dir, path-buf, path-len,
+   cdir-ucd, 1, simplify);
+   /*
+* We get path_recurse in the first run when
+* directory_exists_in_index() returns index_nonexistent. We
+* are sure that new changes in the index does not impact the
+* outcome. Return now.
+*/
+   return path_recurse;
+}
+
 static enum path_treatment treat_path(struct 

[PATCH 08/24] untracked cache: don't open non-existent .gitignore

2015-01-20 Thread Nguyễn Thái Ngọc Duy
This cuts down a signficant number of open(.gitignore) because most
directories usually don't have .gitignore files.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 26 +-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/dir.c b/dir.c
index 02cdc26..95a0f0a 100644
--- a/dir.c
+++ b/dir.c
@@ -1019,7 +1019,21 @@ static void prep_exclude(struct dir_struct *dir, const 
char *base, int baselen)
/* Try to read per-directory file */
hashclr(sha1_stat.sha1);
sha1_stat.valid = 0;
-   if (dir-exclude_per_dir) {
+   if (dir-exclude_per_dir 
+   /*
+* If we know that no files have been added in
+* this directory (i.e. valid_cached_dir() has
+* been executed and set untracked-valid) ..
+*/
+   (!untracked || !untracked-valid ||
+/*
+ * .. and .gitignore does not exist before
+ * (i.e. null exclude_sha1 and skip_worktree is
+ * not set). Then we can skip loading .gitignore,
+ * which would result in ENOENT anyway.
+ * skip_worktree is taken care in read_directory()
+ */
+!is_null_sha1(untracked-exclude_sha1))) {
/*
 * dir-basebuf gets reused by the traversal, but we
 * need fname to remain unchanged to ensure the src
@@ -1782,6 +1796,7 @@ static struct untracked_cache_dir 
*validate_untracked_cache(struct dir_struct *d
  const struct pathspec 
*pathspec)
 {
struct untracked_cache_dir *root;
+   int i;
 
if (!dir-untracked)
return NULL;
@@ -1833,6 +1848,15 @@ static struct untracked_cache_dir 
*validate_untracked_cache(struct dir_struct *d
if (dir-exclude_list_group[EXC_CMDL].nr)
return NULL;
 
+   /*
+* An optimization in prep_exclude() does not play well with
+* CE_SKIP_WORKTREE. It's a rare case anyway, if a single
+* entry has that bit set, disable the whole untracked cache.
+*/
+   for (i = 0; i  active_nr; i++)
+   if (ce_skip_worktree(active_cache[i]))
+   return NULL;
+
if (!dir-untracked-root) {
const int len = sizeof(*dir-untracked-root);
dir-untracked-root = xmalloc(len);
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 10/24] untracked cache: save to an index extension

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/technical/index-format.txt |  58 +
 cache.h  |   3 +
 dir.c| 136 +++
 dir.h|   1 +
 read-cache.c |  12 +++
 5 files changed, 210 insertions(+)

diff --git a/Documentation/technical/index-format.txt 
b/Documentation/technical/index-format.txt
index fe6f316..b97ac8d 100644
--- a/Documentation/technical/index-format.txt
+++ b/Documentation/technical/index-format.txt
@@ -233,3 +233,61 @@ Git index format
   The remaining index entries after replaced ones will be added to the
   final index. These added entries are also sorted by entry namme then
   stage.
+
+== Untracked cache
+
+  Untracked cache saves the untracked file list and necessary data to
+  verify the cache. The signature for this extension is { 'U', 'N',
+  'T', 'R' }.
+
+  The extension starts with
+
+  - Stat data of $GIT_DIR/info/exclude. See Index entry section from
+ctime field until file size.
+
+  - Stat data of core.excludesfile
+
+  - 32-bit dir_flags (see struct dir_struct)
+
+  - 160-bit SHA-1 of $GIT_DIR/info/exclude. Null SHA-1 means the file
+does not exist.
+
+  - 160-bit SHA-1 of core.excludesfile. Null SHA-1 means the file does
+not exist.
+
+  - NUL-terminated string of per-dir exclude file name. This usually
+is .gitignore.
+
+  - The number of following directory blocks, variable width
+encoding. If this number is zero, the extension ends here with a
+following NUL.
+
+  - A number of directory blocks in depth-first-search order, each
+consists of
+
+- The number of untracked entries, variable witdh encoding.
+
+- The number of sub-directory blocks, variable with encoding.
+
+- The directory name terminated by NUL.
+
+- A number of untrached file/dir names terminated by NUL.
+
+The remaining data of each directory block is grouped by type:
+
+  - An ewah bitmap, the n-th bit marks whether the n-th directory has
+valid untracked cache entries.
+
+  - An ewah bitmap, the n-th bit records check-only bit of
+read_directory_recursive() for the n-th directory.
+
+  - An ewah bitmap, the n-th bit indicates whether SHA-1 and stat data
+is valid for the n-th directory and exists in the next data.
+
+  - An array of stat data. The n-th data corresponds with the n-th
+one bit in the previous ewah bitmap.
+
+  - An array of SHA-1. The n-th SHA-1 corresponds with the n-th one bit
+in the previous ewah bitmap.
+
+  - One NUL.
diff --git a/cache.h b/cache.h
index dcf3a2a..b14d6e2 100644
--- a/cache.h
+++ b/cache.h
@@ -297,6 +297,8 @@ static inline unsigned int canon_mode(unsigned int mode)
 #define SPLIT_INDEX_ORDERED(1  6)
 
 struct split_index;
+struct untracked_cache;
+
 struct index_state {
struct cache_entry **cache;
unsigned int version;
@@ -310,6 +312,7 @@ struct index_state {
struct hashmap name_hash;
struct hashmap dir_hash;
unsigned char sha1[20];
+   struct untracked_cache *untracked;
 };
 
 extern struct index_state the_index;
diff --git a/dir.c b/dir.c
index 95a0f0a..1f2d701 100644
--- a/dir.c
+++ b/dir.c
@@ -12,6 +12,8 @@
 #include refs.h
 #include wildmatch.h
 #include pathspec.h
+#include varint.h
+#include ewah/ewok.h
 
 struct path_simplify {
int len;
@@ -2139,3 +2141,137 @@ void clear_directory(struct dir_struct *dir)
}
strbuf_release(dir-basebuf);
 }
+
+struct ondisk_untracked_cache {
+   struct stat_data info_exclude_stat;
+   struct stat_data excludes_file_stat;
+   uint32_t dir_flags;
+   unsigned char info_exclude_sha1[20];
+   unsigned char excludes_file_sha1[20];
+   char exclude_per_dir[FLEX_ARRAY];
+};
+
+#define ouc_size(len) (offsetof(struct ondisk_untracked_cache, 
exclude_per_dir) + len + 1)
+
+struct write_data {
+   int index; /* number of written untracked_cache_dir */
+   struct ewah_bitmap *check_only; /* from untracked_cache_dir */
+   struct ewah_bitmap *valid;  /* from untracked_cache_dir */
+   struct ewah_bitmap *sha1_valid; /* set if exclude_sha1 is not null */
+   struct strbuf out;
+   struct strbuf sb_stat;
+   struct strbuf sb_sha1;
+};
+
+static void stat_data_to_disk(struct stat_data *to, const struct stat_data 
*from)
+{
+   to-sd_ctime.sec  = htonl(from-sd_ctime.sec);
+   to-sd_ctime.nsec = htonl(from-sd_ctime.nsec);
+   to-sd_mtime.sec  = htonl(from-sd_mtime.sec);
+   to-sd_mtime.nsec = htonl(from-sd_mtime.nsec);
+   to-sd_dev= htonl(from-sd_dev);
+   to-sd_ino= htonl(from-sd_ino);
+   to-sd_uid= htonl(from-sd_uid);
+   to-sd_gid= htonl(from-sd_gid);
+   to-sd_size   = htonl(from-sd_size);
+}
+
+static void write_one_dir(struct untracked_cache_dir *untracked,
+ 

[PATCH 04/24] untracked cache: invalidate dirs recursively if .gitignore changes

2015-01-20 Thread Nguyễn Thái Ngọc Duy
It's easy to see that if an existing .gitignore changes, its SHA-1
would be different and invalidate_gitignore() is called.

If .gitignore is removed, add_excludes() will treat it like an empty
.gitignore, which again should invalidate the cached directory data.

if .gitignore is added, lookup_untracked() already fills initial
.gitignore SHA-1 as empty file, so again invalidate_gitignore() is
called.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/dir.c b/dir.c
index 44ed9f2..6e91315 100644
--- a/dir.c
+++ b/dir.c
@@ -1010,7 +1010,23 @@ static void prep_exclude(struct dir_struct *dir, const 
char *base, int baselen)
add_excludes(el-src, el-src, stk-baselen, el, 1,
 untracked ? sha1_stat : NULL);
}
-   if (untracked) {
+   /*
+* NEEDSWORK: when untracked cache is enabled, prep_exclude()
+* will first be called in valid_cached_dir() then maybe many
+* times more in last_exclude_matching(). When the cache is
+* used, last_exclude_matching() will not be called and
+* reading .gitignore content will be a waste.
+*
+* So when it's called by valid_cached_dir() and we can get
+* .gitignore SHA-1 from the index (i.e. .gitignore is not
+* modified on work tree), we could delay reading the
+* .gitignore content until we absolutely need it in
+* last_exclude_matching(). Be careful about ignore rule
+* order, though, if you do that.
+*/
+   if (untracked 
+   hashcmp(sha1_stat.sha1, untracked-exclude_sha1)) {
+   invalidate_gitignore(dir-untracked, untracked);
hashcpy(untracked-exclude_sha1, sha1_stat.sha1);
}
dir-exclude_stack = stk;
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 05/24] untracked cache: make a wrapper around {open,read,close}dir()

2015-01-20 Thread Nguyễn Thái Ngọc Duy
This allows us to feed different info to read_directory_recursive()
based on untracked cache in the next patch.

Helped-by: Ramsay Jones ram...@ramsay1.demon.co.uk
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 55 +++
 1 file changed, 47 insertions(+), 8 deletions(-)

diff --git a/dir.c b/dir.c
index 6e91315..fb6ed86 100644
--- a/dir.c
+++ b/dir.c
@@ -31,6 +31,15 @@ enum path_treatment {
path_untracked
 };
 
+/*
+ * Support data structure for our opendir/readdir/closedir wrappers
+ */
+struct cached_dir {
+   DIR *fdir;
+   struct untracked_cache_dir *untracked;
+   struct dirent *de;
+};
+
 static enum path_treatment read_directory_recursive(struct dir_struct *dir,
const char *path, int len, struct untracked_cache_dir *untracked,
int check_only, const struct path_simplify *simplify);
@@ -1417,12 +1426,13 @@ static enum path_treatment treat_one_path(struct 
dir_struct *dir,
 
 static enum path_treatment treat_path(struct dir_struct *dir,
  struct untracked_cache_dir *untracked,
- struct dirent *de,
+ struct cached_dir *cdir,
  struct strbuf *path,
  int baselen,
  const struct path_simplify *simplify)
 {
int dtype;
+   struct dirent *de = cdir-de;
 
if (is_dot_or_dotdot(de-d_name) || !strcmp(de-d_name, .git))
return path_none;
@@ -1444,6 +1454,37 @@ static void add_untracked(struct untracked_cache_dir 
*dir, const char *name)
dir-untracked[dir-untracked_nr++] = xstrdup(name);
 }
 
+static int open_cached_dir(struct cached_dir *cdir,
+  struct dir_struct *dir,
+  struct untracked_cache_dir *untracked,
+  struct strbuf *path,
+  int check_only)
+{
+   memset(cdir, 0, sizeof(*cdir));
+   cdir-untracked = untracked;
+   cdir-fdir = opendir(path-len ? path-buf : .);
+   if (!cdir-fdir)
+   return -1;
+   return 0;
+}
+
+static int read_cached_dir(struct cached_dir *cdir)
+{
+   if (cdir-fdir) {
+   cdir-de = readdir(cdir-fdir);
+   if (!cdir-de)
+   return -1;
+   return 0;
+   }
+   return -1;
+}
+
+static void close_cached_dir(struct cached_dir *cdir)
+{
+   if (cdir-fdir)
+   closedir(cdir-fdir);
+}
+
 /*
  * Read a directory tree. We currently ignore anything but
  * directories, regular files and symlinks. That's because git
@@ -1460,23 +1501,21 @@ static enum path_treatment 
read_directory_recursive(struct dir_struct *dir,
struct untracked_cache_dir *untracked, int 
check_only,
const struct path_simplify *simplify)
 {
-   DIR *fdir;
+   struct cached_dir cdir;
enum path_treatment state, subdir_state, dir_state = path_none;
-   struct dirent *de;
struct strbuf path = STRBUF_INIT;
 
strbuf_add(path, base, baselen);
 
-   fdir = opendir(path.len ? path.buf : .);
-   if (!fdir)
+   if (open_cached_dir(cdir, dir, untracked, path, check_only))
goto out;
 
if (untracked)
untracked-check_only = !!check_only;
 
-   while ((de = readdir(fdir)) != NULL) {
+   while (!read_cached_dir(cdir)) {
/* check how the file or directory should be treated */
-   state = treat_path(dir, untracked, de, path, baselen, 
simplify);
+   state = treat_path(dir, untracked, cdir, path, baselen, 
simplify);
 
if (state  dir_state)
dir_state = state;
@@ -1529,7 +1568,7 @@ static enum path_treatment 
read_directory_recursive(struct dir_struct *dir,
break;
}
}
-   closedir(fdir);
+   close_cached_dir(cdir);
  out:
strbuf_release(path);
 
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 07/24] untracked cache: mark what dirs should be recursed/saved

2015-01-20 Thread Nguyễn Thái Ngọc Duy
If we redo this thing in a functional style, we would have one struct
untracked_dir as input tree and another as output. The input is used
for verification. The output is a brand new tree, reflecting current
worktree.

But that means recreate a lot of dir nodes even if a lot could be
shared between input and output trees in good cases. So we go with the
messy but efficient way, combining both input and output trees into
one. We need a way to know which node in this combined tree belongs to
the output. This is the purpose of this recurse flag.

valid bit can't be used for this because it's about data of the node
except the subdirs. When we invalidate a directory, we want to keep
cached data of the subdirs intact even though we don't really know
what subdir still exists (yet). Then we check worktree to see what
actual subdir remains on disk. Those will have 'recurse' bit set
again. If cached data for those are still valid, we may be able to
avoid computing exclude files for them. Those subdirs that are deleted
will have 'recurse' remained clear and their 'valid' bits do not
matter.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 14 +-
 dir.h |  3 ++-
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/dir.c b/dir.c
index 8c989e3..02cdc26 100644
--- a/dir.c
+++ b/dir.c
@@ -615,9 +615,12 @@ static void invalidate_gitignore(struct untracked_cache 
*uc,
 static void invalidate_directory(struct untracked_cache *uc,
 struct untracked_cache_dir *dir)
 {
+   int i;
uc-dir_invalidated++;
dir-valid = 0;
dir-untracked_nr = 0;
+   for (i = 0; i  dir-dirs_nr; i++)
+   dir-dirs[i]-recurse = 0;
 }
 
 /*
@@ -1577,6 +1580,10 @@ static int read_cached_dir(struct cached_dir *cdir)
}
while (cdir-nr_dirs  cdir-untracked-dirs_nr) {
struct untracked_cache_dir *d = 
cdir-untracked-dirs[cdir-nr_dirs];
+   if (!d-recurse) {
+   cdir-nr_dirs++;
+   continue;
+   }
cdir-ucd = d;
cdir-nr_dirs++;
return 0;
@@ -1598,8 +1605,10 @@ static void close_cached_dir(struct cached_dir *cdir)
 * We have gone through this directory and found no untracked
 * entries. Mark it valid.
 */
-   if (cdir-untracked)
+   if (cdir-untracked) {
cdir-untracked-valid = 1;
+   cdir-untracked-recurse = 1;
+   }
 }
 
 /*
@@ -1842,6 +1851,9 @@ static struct untracked_cache_dir 
*validate_untracked_cache(struct dir_struct *d
invalidate_gitignore(dir-untracked, root);
dir-untracked-ss_excludes_file = dir-ss_excludes_file;
}
+
+   /* Make sure this directory is not dropped out at saving phase */
+   root-recurse = 1;
return root;
 }
 
diff --git a/dir.h b/dir.h
index ff3d99b..95baf01 100644
--- a/dir.h
+++ b/dir.h
@@ -115,8 +115,9 @@ struct untracked_cache_dir {
unsigned int untracked_alloc, dirs_nr, dirs_alloc;
unsigned int untracked_nr;
unsigned int check_only : 1;
-   /* all data in this struct are good */
+   /* all data except 'dirs' in this struct are good */
unsigned int valid : 1;
+   unsigned int recurse : 1;
/* null SHA-1 means this directory does not have .gitignore */
unsigned char exclude_sha1[20];
char name[FLEX_ARRAY];
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 11/24] untracked cache: load from UNTR index extension

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 dir.c| 220 +++
 dir.h|   2 +
 read-cache.c |   5 ++
 3 files changed, 227 insertions(+)

diff --git a/dir.c b/dir.c
index 1f2d701..e7d7df3 100644
--- a/dir.c
+++ b/dir.c
@@ -2275,3 +2275,223 @@ void write_untracked_extension(struct strbuf *out, 
struct untracked_cache *untra
strbuf_release(wd.sb_stat);
strbuf_release(wd.sb_sha1);
 }
+
+static void free_untracked(struct untracked_cache_dir *ucd)
+{
+   int i;
+   if (!ucd)
+   return;
+   for (i = 0; i  ucd-dirs_nr; i++)
+   free_untracked(ucd-dirs[i]);
+   for (i = 0; i  ucd-untracked_nr; i++)
+   free(ucd-untracked[i]);
+   free(ucd-untracked);
+   free(ucd-dirs);
+   free(ucd);
+}
+
+void free_untracked_cache(struct untracked_cache *uc)
+{
+   if (uc)
+   free_untracked(uc-root);
+   free(uc);
+}
+
+struct read_data {
+   int index;
+   struct untracked_cache_dir **ucd;
+   struct ewah_bitmap *check_only;
+   struct ewah_bitmap *valid;
+   struct ewah_bitmap *sha1_valid;
+   const unsigned char *data;
+   const unsigned char *end;
+};
+
+static void stat_data_from_disk(struct stat_data *to, const struct stat_data 
*from)
+{
+   to-sd_ctime.sec  = get_be32(from-sd_ctime.sec);
+   to-sd_ctime.nsec = get_be32(from-sd_ctime.nsec);
+   to-sd_mtime.sec  = get_be32(from-sd_mtime.sec);
+   to-sd_mtime.nsec = get_be32(from-sd_mtime.nsec);
+   to-sd_dev= get_be32(from-sd_dev);
+   to-sd_ino= get_be32(from-sd_ino);
+   to-sd_uid= get_be32(from-sd_uid);
+   to-sd_gid= get_be32(from-sd_gid);
+   to-sd_size   = get_be32(from-sd_size);
+}
+
+static int read_one_dir(struct untracked_cache_dir **untracked_,
+   struct read_data *rd)
+{
+#define NEXT(x) \
+   next = data + (x); \
+   if (next  rd-end) \
+   return -1;
+
+   struct untracked_cache_dir ud, *untracked;
+   const unsigned char *next, *data = rd-data, *end = rd-end;
+   unsigned int value;
+   int i, len;
+
+   memset(ud, 0, sizeof(ud));
+
+   next = data;
+   value = decode_varint(next);
+   if (next  end)
+   return -1;
+   ud.recurse = 1;
+   ud.untracked_alloc = value;
+   ud.untracked_nr= value;
+   if (ud.untracked_nr)
+   ud.untracked = xmalloc(sizeof(*ud.untracked) * ud.untracked_nr);
+   data = next;
+
+   next = data;
+   ud.dirs_alloc = ud.dirs_nr = decode_varint(next);
+   if (next  end)
+   return -1;
+   ud.dirs = xmalloc(sizeof(*ud.dirs) * ud.dirs_nr);
+   data = next;
+
+   len = strlen((const char *)data);
+   NEXT(len + 1);
+   *untracked_ = untracked = xmalloc(sizeof(*untracked) + len);
+   memcpy(untracked, ud, sizeof(ud));
+   memcpy(untracked-name, data, len + 1);
+   data = next;
+
+   for (i = 0; i  untracked-untracked_nr; i++) {
+   len = strlen((const char *)data);
+   NEXT(len + 1);
+   untracked-untracked[i] = xstrdup((const char*)data);
+   data = next;
+   }
+
+   rd-ucd[rd-index++] = untracked;
+   rd-data = data;
+
+   for (i = 0; i  untracked-dirs_nr; i++) {
+   len = read_one_dir(untracked-dirs + i, rd);
+   if (len  0)
+   return -1;
+   }
+   return 0;
+}
+
+static void set_check_only(size_t pos, void *cb)
+{
+   struct read_data *rd = cb;
+   struct untracked_cache_dir *ud = rd-ucd[pos];
+   ud-check_only = 1;
+}
+
+static void read_stat(size_t pos, void *cb)
+{
+   struct read_data *rd = cb;
+   struct untracked_cache_dir *ud = rd-ucd[pos];
+   if (rd-data + sizeof(struct stat_data)  rd-end) {
+   rd-data = rd-end + 1;
+   return;
+   }
+   stat_data_from_disk(ud-stat_data, (struct stat_data *)rd-data);
+   rd-data += sizeof(struct stat_data);
+   ud-valid = 1;
+}
+
+static void read_sha1(size_t pos, void *cb)
+{
+   struct read_data *rd = cb;
+   struct untracked_cache_dir *ud = rd-ucd[pos];
+   if (rd-data + 20  rd-end) {
+   rd-data = rd-end + 1;
+   return;
+   }
+   hashcpy(ud-exclude_sha1, rd-data);
+   rd-data += 20;
+}
+
+static void load_sha1_stat(struct sha1_stat *sha1_stat,
+  const struct stat_data *stat,
+  const unsigned char *sha1)
+{
+   stat_data_from_disk(sha1_stat-stat, stat);
+   hashcpy(sha1_stat-sha1, sha1);
+   sha1_stat-valid = 1;
+}
+
+struct untracked_cache *read_untracked_extension(const void *data, unsigned 
long sz)
+{
+   const struct ondisk_untracked_cache *ouc;
+   struct untracked_cache *uc;
+   struct read_data rd;
+   const 

[PATCH 03/24] untracked cache: initial untracked cache validation

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Make sure the starting conditions and all global exclude files are
good to go. If not, either disable untracked cache completely, or wipe
out the cache and start fresh.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 113 --
 dir.h |   4 +++
 2 files changed, 114 insertions(+), 3 deletions(-)

diff --git a/dir.c b/dir.c
index 27734f0..44ed9f2 100644
--- a/dir.c
+++ b/dir.c
@@ -582,6 +582,22 @@ static struct untracked_cache_dir *lookup_untracked(struct 
untracked_cache *uc,
return d;
 }
 
+static void do_invalidate_gitignore(struct untracked_cache_dir *dir)
+{
+   int i;
+   dir-valid = 0;
+   dir-untracked_nr = 0;
+   for (i = 0; i  dir-dirs_nr; i++)
+   do_invalidate_gitignore(dir-dirs[i]);
+}
+
+static void invalidate_gitignore(struct untracked_cache *uc,
+struct untracked_cache_dir *dir)
+{
+   uc-gitignore_invalidated++;
+   do_invalidate_gitignore(dir);
+}
+
 /*
  * Given a file with name fname, read it (either from disk, or from
  * the index if check_index is non-zero), parse it and store the
@@ -697,6 +713,13 @@ static void add_excludes_from_file_1(struct dir_struct 
*dir, const char *fname,
 struct sha1_stat *sha1_stat)
 {
struct exclude_list *el;
+   /*
+* catch setup_standard_excludes() that's called before
+* dir-untracked is assigned. That function behaves
+* differently when dir-untracked is non-NULL.
+*/
+   if (!dir-untracked)
+   dir-unmanaged_exclude_files++;
el = add_exclude_list(dir, EXC_FILE, fname);
if (add_excludes(fname, , 0, el, 0, sha1_stat)  0)
die(cannot use %s as an exclude file, fname);
@@ -704,6 +727,7 @@ static void add_excludes_from_file_1(struct dir_struct 
*dir, const char *fname,
 
 void add_excludes_from_file(struct dir_struct *dir, const char *fname)
 {
+   dir-unmanaged_exclude_files++; /* see validate_untracked_cache() */
add_excludes_from_file_1(dir, fname, NULL);
 }
 
@@ -1572,9 +1596,87 @@ static int treat_leading_path(struct dir_struct *dir,
return rc;
 }
 
+static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct 
*dir,
+ int base_len,
+ const struct pathspec 
*pathspec)
+{
+   struct untracked_cache_dir *root;
+
+   if (!dir-untracked)
+   return NULL;
+
+   /*
+* We only support $GIT_DIR/info/exclude and core.excludesfile
+* as the global ignore rule files. Any other additions
+* (e.g. from command line) invalidate the cache. This
+* condition also catches running setup_standard_excludes()
+* before setting dir-untracked!
+*/
+   if (dir-unmanaged_exclude_files)
+   return NULL;
+
+   /*
+* Optimize for the main use case only: whole-tree git
+* status. More work involved in treat_leading_path() if we
+* use cache on just a subset of the worktree. pathspec
+* support could make the matter even worse.
+*/
+   if (base_len || (pathspec  pathspec-nr))
+   return NULL;
+
+   /* Different set of flags may produce different results */
+   if (dir-flags != dir-untracked-dir_flags ||
+   /*
+* See treat_directory(), case index_nonexistent. Without
+* this flag, we may need to also cache .git file content
+* for the resolve_gitlink_ref() call, which we don't.
+*/
+   !(dir-flags  DIR_SHOW_OTHER_DIRECTORIES) ||
+   /* We don't support collecting ignore files */
+   (dir-flags  (DIR_SHOW_IGNORED | DIR_SHOW_IGNORED_TOO |
+  DIR_COLLECT_IGNORED)))
+   return NULL;
+
+   /*
+* If we use .gitignore in the cache and now you change it to
+* .gitexclude, everything will go wrong.
+*/
+   if (dir-exclude_per_dir != dir-untracked-exclude_per_dir 
+   strcmp(dir-exclude_per_dir, dir-untracked-exclude_per_dir))
+   return NULL;
+
+   /*
+* EXC_CMDL is not considered in the cache. If people set it,
+* skip the cache.
+*/
+   if (dir-exclude_list_group[EXC_CMDL].nr)
+   return NULL;
+
+   if (!dir-untracked-root) {
+   const int len = sizeof(*dir-untracked-root);
+   dir-untracked-root = xmalloc(len);
+   memset(dir-untracked-root, 0, len);
+   }
+
+   /* Validate $GIT_DIR/info/exclude and core.excludesfile */
+   root = dir-untracked-root;
+   if (hashcmp(dir-ss_info_exclude.sha1,
+   dir-untracked-ss_info_exclude.sha1)) {
+   invalidate_gitignore(dir-untracked, root);

[PATCHv2] rebase -i: respect core.abbrev for real

2015-01-20 Thread Kirill A. Shutemov
I have tried to fix this before: see 568950388be2, but it doesn't
really work.

I don't know how it happend, but that commit makes interactive rebase to
respect core.abbrev only during --edit-todo, but not the initial todo
list edit.

For this time I've included a test-case to avoid this frustration again.

Signed-off-by: Kirill A. Shutemov kirill.shute...@linux.intel.com
---
 v2: fix -chain in the test-case
---
 git-rebase--interactive.sh| 4 ++--
 t/t3404-rebase-interactive.sh | 7 +++
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index c6a4629cbc2b..1855e12f1ada 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -962,7 +962,7 @@ else
shortrevisions=$shorthead
 fi
 git rev-list $merges_option --pretty=oneline --abbrev-commit \
-   --abbrev=7 --reverse --left-right --topo-order \
+   --reverse --left-right --topo-order \
$revisions ${restrict_revision+^$restrict_revision} | \
sed -n s/^//p |
 while read -r shortsha1 rest
@@ -1020,7 +1020,7 @@ then
# just the history of its first-parent for others that 
will
# be rebasing on top of it
git rev-list --parents -1 $rev | cut -d' ' -s -f2  
$dropped/$rev
-   short=$(git rev-list -1 --abbrev-commit --abbrev=7 $rev)
+   short=$(git rev-list -1 --abbrev-commit $rev)
sane_grep -v ^[a-z][a-z]* $short $todo  
${todo}2 ; mv ${todo}2 $todo
rm $rewritten/$rev
fi
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 8197ed29a9ec..a31f7e0430e1 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -1039,4 +1039,11 @@ test_expect_success 'short SHA-1 collide' '
)
 '
 
+test_expect_success 'respect core.abbrev' '
+   git config core.abbrev 12 
+   set_cat_todo_editor 
+   test_must_fail git rebase -i HEAD~4 todo-list 
+   test 4 = $(grep -c pick [0-9a-f]\{12,\} todo-list)
+'
+
 test_done
-- 
2.1.4

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 19/24] update-index: manually enable or disable untracked cache

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Overall time saving on git status is about 40% in the best case
scenario, removing ..collect_untracked() as the most time consuming
function. read and refresh index operations are now at the top (which
should drop when index-helper and/or watchman support is added). More
numbers and analysis below.

webkit.git
==

169k files. 6k dirs. Lots of test data (i.e. not touched most of the
time)

Base status
---

Index version 4 in split index mode and cache-tree populated. No
untracked cache. It shows how time is consumed by git status. The
same settings are used for other repos below.

18:28:10.199679 builtin/commit.c:1394   performance: 0.00451 s: 
cmd_status:setup
18:28:10.474847 read-cache.c:1407   performance: 0.274873831 s: read_index
18:28:10.475295 read-cache.c:1407   performance: 0.00656 s: read_index
18:28:10.728443 preload-index.c:131 performance: 0.253147487 s: 
read_index_preload
18:28:10.741422 read-cache.c:1254   performance: 0.012868340 s: 
refresh_index
18:28:10.752300 wt-status.c:623 performance: 0.010421357 s: 
wt_status_collect_changes_worktree
18:28:10.762069 wt-status.c:629 performance: 0.009644748 s: 
wt_status_collect_changes_index
18:28:11.601019 wt-status.c:632 performance: 0.838859547 s: 
wt_status_collect_untracked
18:28:11.605939 builtin/commit.c:1421   performance: 0.004835004 s: 
cmd_status:update_index
18:28:11.606580 trace.c:415 performance: 1.407878388 s: git 
command: 'git' 'status'

Populating status
-

This is after enabling untracked cache and the cache is still empty.
We see a slight increase in .._collect_untracked() and update_index
(because new cache has to be written to $GIT_DIR/index).

18:28:18.915213 builtin/commit.c:1394   performance: 0.00326 s: 
cmd_status:setup
18:28:19.197364 read-cache.c:1407   performance: 0.281901416 s: read_index
18:28:19.197754 read-cache.c:1407   performance: 0.00546 s: read_index
18:28:19.451355 preload-index.c:131 performance: 0.253599607 s: 
read_index_preload
18:28:19.464400 read-cache.c:1254   performance: 0.012935336 s: 
refresh_index
18:28:19.475115 wt-status.c:623 performance: 0.010236920 s: 
wt_status_collect_changes_worktree
18:28:19.486022 wt-status.c:629 performance: 0.010801685 s: 
wt_status_collect_changes_index
18:28:20.362660 wt-status.c:632 performance: 0.876551366 s: 
wt_status_collect_untracked
18:28:20.396199 builtin/commit.c:1421   performance: 0.033447969 s: 
cmd_status:update_index
18:28:20.396939 trace.c:415 performance: 1.482695902 s: git 
command: 'git' 'status'

Populated status


After the cache is populated, wt_status_collect_untracked() drops 82%
from 0.838s to 0.144s. Overall time drops 45%. Top offenders are now
read_index() and read_index_preload().

18:28:20.408605 builtin/commit.c:1394   performance: 0.00457 s: 
cmd_status:setup
18:28:20.692864 read-cache.c:1407   performance: 0.283980458 s: read_index
18:28:20.693273 read-cache.c:1407   performance: 0.00661 s: read_index
18:28:20.958814 preload-index.c:131 performance: 0.265540254 s: 
read_index_preload
18:28:20.972375 read-cache.c:1254   performance: 0.013437429 s: 
refresh_index
18:28:20.983959 wt-status.c:623 performance: 0.011146646 s: 
wt_status_collect_changes_worktree
18:28:20.993948 wt-status.c:629 performance: 0.009879094 s: 
wt_status_collect_changes_index
18:28:21.138125 wt-status.c:632 performance: 0.144084737 s: 
wt_status_collect_untracked
18:28:21.173678 builtin/commit.c:1421   performance: 0.035463949 s: 
cmd_status:update_index
18:28:21.174251 trace.c:415 performance: 0.766707355 s: git 
command: 'git' 'status'

gentoo-x86.git
==

This repository is a strange one with a balanced, wide and shallow
worktree (about 100k files and 23k dirs) and no .gitignore in
worktree. .._collect_untracked() time drops 88%, total time drops 56%.

Base status
---
18:20:40.828642 builtin/commit.c:1394   performance: 0.00496 s: 
cmd_status:setup
18:20:41.027233 read-cache.c:1407   performance: 0.198130532 s: read_index
18:20:41.027670 read-cache.c:1407   performance: 0.00581 s: read_index
18:20:41.171716 preload-index.c:131 performance: 0.144045594 s: 
read_index_preload
18:20:41.179171 read-cache.c:1254   performance: 0.007320424 s: 
refresh_index
18:20:41.185785 wt-status.c:623 performance: 0.006144638 s: 
wt_status_collect_changes_worktree
18:20:41.192701 wt-status.c:629 performance: 0.006780184 s: 
wt_status_collect_changes_index
18:20:41.991723 wt-status.c:632 performance: 0.798927029 s: 
wt_status_collect_untracked
18:20:41.994664 builtin/commit.c:1421   performance: 0.002852772 s: 
cmd_status:update_index
18:20:41.995458 trace.c:415 performance: 1.168427502 s: git 
command: 'git' 'status'
Populating status
-
18:20:48.968848 

[PATCH 16/24] untracked cache: mark index dirty if untracked cache is updated

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 cache.h  | 1 +
 dir.c| 9 +
 read-cache.c | 2 +-
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/cache.h b/cache.h
index f8b3dc5..fca979b 100644
--- a/cache.h
+++ b/cache.h
@@ -295,6 +295,7 @@ static inline unsigned int canon_mode(unsigned int mode)
 #define RESOLVE_UNDO_CHANGED   (1  4)
 #define CACHE_TREE_CHANGED (1  5)
 #define SPLIT_INDEX_ORDERED(1  6)
+#define UNTRACKED_CHANGED  (1  7)
 
 struct split_index;
 struct untracked_cache;
diff --git a/dir.c b/dir.c
index c5ca5ce..1c3db0b 100644
--- a/dir.c
+++ b/dir.c
@@ -1933,6 +1933,15 @@ int read_directory(struct dir_struct *dir, const char 
*path, int len, const stru
 dir-untracked-gitignore_invalidated,
 dir-untracked-dir_invalidated,
 dir-untracked-dir_opened);
+   if (dir-untracked == the_index.untracked 
+   (dir-untracked-dir_opened ||
+dir-untracked-gitignore_invalidated ||
+dir-untracked-dir_invalidated))
+   the_index.cache_changed |= UNTRACKED_CHANGED;
+   if (dir-untracked != the_index.untracked) {
+   free(dir-untracked);
+   dir-untracked = NULL;
+   }
}
return dir-nr;
 }
diff --git a/read-cache.c b/read-cache.c
index 0ecba05..71d8e20 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -42,7 +42,7 @@ static struct cache_entry *refresh_cache_entry(struct 
cache_entry *ce,
 /* changes that can be kept in $GIT_DIR/index (basically all extensions) */
 #define EXTMASK (RESOLVE_UNDO_CHANGED | CACHE_TREE_CHANGED | \
 CE_ENTRY_ADDED | CE_ENTRY_REMOVED | CE_ENTRY_CHANGED | \
-SPLIT_INDEX_ORDERED)
+SPLIT_INDEX_ORDERED | UNTRACKED_CHANGED)
 
 struct index_state the_index;
 static const char *alternate_index_output;
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 24/24] git-status.txt: advertisement for untracked cache

2015-01-20 Thread Nguyễn Thái Ngọc Duy
When a good user sees the too long, consider -uno advice when
running `git status`, they should check out the man page to find out
more. This change suggests they try untracked cache before -uno.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/git-status.txt | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt
index def635f..7850f53 100644
--- a/Documentation/git-status.txt
+++ b/Documentation/git-status.txt
@@ -58,7 +58,10 @@ When `-u` option is not used, untracked files and 
directories are
 shown (i.e. the same as specifying `normal`), to help you avoid
 forgetting to add newly created files.  Because it takes extra work
 to find untracked files in the filesystem, this mode may take some
-time in a large working tree.  You can use `no` to have `git status`
+time in a large working tree.
+Consider to enable untracked cache and split index if supported (see
+`git update-index --untracked-cache` and `git update-index
+--split-index`), Otherwise you can use `no` to have `git status`
 return more quickly without showing untracked files.
 +
 The default can be changed using the status.showUntrackedFiles
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 12/24] untracked cache: invalidate at index addition or removal

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Ideally we should implement untracked_cache_remove_from_index() and
untracked_cache_add_to_index() so that they update untracked cache
right away instead of invalidating it and wait for read_directory()
next time to deal with it. But that may need some more work in
unpack-trees.c. So stay simple as the first step.

The new call in add_index_entry_with_check() may look strange because
new calls usually stay close to cache_tree_invalidate_path(). We do it
a bit later than c_t_i_p() in this function because if it's about
replacing the entry with the same name, we don't care (but cache-tree
does).

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c  | 31 +++
 dir.h  |  4 
 read-cache.c   |  4 
 unpack-trees.c |  7 +--
 4 files changed, 44 insertions(+), 2 deletions(-)

diff --git a/dir.c b/dir.c
index e7d7df3..30e23f8 100644
--- a/dir.c
+++ b/dir.c
@@ -2495,3 +2495,34 @@ done2:
}
return uc;
 }
+
+void untracked_cache_invalidate_path(struct index_state *istate,
+const char *path)
+{
+   const char *sep;
+   struct untracked_cache_dir *d;
+   if (!istate-untracked || !istate-untracked-root)
+   return;
+   sep = strrchr(path, '/');
+   if (sep)
+   d = lookup_untracked(istate-untracked,
+istate-untracked-root,
+path, sep - path);
+   else
+   d = istate-untracked-root;
+   istate-untracked-dir_invalidated++;
+   d-valid = 0;
+   d-untracked_nr = 0;
+}
+
+void untracked_cache_remove_from_index(struct index_state *istate,
+  const char *path)
+{
+   untracked_cache_invalidate_path(istate, path);
+}
+
+void untracked_cache_add_to_index(struct index_state *istate,
+ const char *path)
+{
+   untracked_cache_invalidate_path(istate, path);
+}
diff --git a/dir.h b/dir.h
index 40a679a..2ce7dd3 100644
--- a/dir.h
+++ b/dir.h
@@ -298,6 +298,10 @@ static inline int dir_path_match(const struct dir_entry 
*ent,
  has_trailing_dir);
 }
 
+void untracked_cache_invalidate_path(struct index_state *, const char *);
+void untracked_cache_remove_from_index(struct index_state *, const char *);
+void untracked_cache_add_to_index(struct index_state *, const char *);
+
 void free_untracked_cache(struct untracked_cache *);
 struct untracked_cache *read_untracked_extension(const void *data, unsigned 
long sz);
 void write_untracked_extension(struct strbuf *out, struct untracked_cache 
*untracked);
diff --git a/read-cache.c b/read-cache.c
index 3736a56..d643a3f 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -78,6 +78,7 @@ void rename_index_entry_at(struct index_state *istate, int 
nr, const char *new_n
memcpy(new-name, new_name, namelen + 1);
 
cache_tree_invalidate_path(istate, old-name);
+   untracked_cache_remove_from_index(istate, old-name);
remove_index_entry_at(istate, nr);
add_index_entry(istate, new, 
ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE);
 }
@@ -537,6 +538,7 @@ int remove_file_from_index(struct index_state *istate, 
const char *path)
if (pos  0)
pos = -pos-1;
cache_tree_invalidate_path(istate, path);
+   untracked_cache_remove_from_index(istate, path);
while (pos  istate-cache_nr  !strcmp(istate-cache[pos]-name, 
path))
remove_index_entry_at(istate, pos);
return 0;
@@ -968,6 +970,8 @@ static int add_index_entry_with_check(struct index_state 
*istate, struct cache_e
}
pos = -pos-1;
 
+   untracked_cache_add_to_index(istate, ce-name);
+
/*
 * Inserting a merged entry (stage 0) into the index
 * will always replace all non-merged entries..
diff --git a/unpack-trees.c b/unpack-trees.c
index 629c658..e5ddb0c 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -9,6 +9,7 @@
 #include refs.h
 #include attr.h
 #include split-index.h
+#include dir.h
 
 /*
  * Error messages expected by scripts out of plumbing commands such as
@@ -1255,8 +1256,10 @@ static int verify_uptodate_sparse(const struct 
cache_entry *ce,
 static void invalidate_ce_path(const struct cache_entry *ce,
   struct unpack_trees_options *o)
 {
-   if (ce)
-   cache_tree_invalidate_path(o-src_index, ce-name);
+   if (!ce)
+   return;
+   cache_tree_invalidate_path(o-src_index, ce-name);
+   untracked_cache_invalidate_path(o-src_index, ce-name);
 }
 
 /*
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 13/24] read-cache.c: split racy stat test to a separate function

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 read-cache.c | 24 +++-
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index d643a3f..f12a185 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -270,20 +270,26 @@ static int ce_match_stat_basic(const struct cache_entry 
*ce, struct stat *st)
return changed;
 }
 
-static int is_racy_timestamp(const struct index_state *istate,
-const struct cache_entry *ce)
+static int is_racy_stat(const struct index_state *istate,
+   const struct stat_data *sd)
 {
-   return (!S_ISGITLINK(ce-ce_mode) 
-   istate-timestamp.sec 
+   return (istate-timestamp.sec 
 #ifdef USE_NSEC
 /* nanosecond timestamped files can also be racy! */
-   (istate-timestamp.sec  ce-ce_stat_data.sd_mtime.sec ||
-(istate-timestamp.sec == ce-ce_stat_data.sd_mtime.sec 
- istate-timestamp.nsec = ce-ce_stat_data.sd_mtime.nsec))
+   (istate-timestamp.sec  sd-sd_mtime.sec ||
+(istate-timestamp.sec == sd-sd_mtime.sec 
+ istate-timestamp.nsec = sd-sd_mtime.nsec))
 #else
-   istate-timestamp.sec = ce-ce_stat_data.sd_mtime.sec
+   istate-timestamp.sec = sd-sd_mtime.sec
 #endif
-);
+   );
+}
+
+static int is_racy_timestamp(const struct index_state *istate,
+const struct cache_entry *ce)
+{
+   return (!S_ISGITLINK(ce-ce_mode) 
+   is_racy_stat(istate, ce-ce_stat_data));
 }
 
 int ie_match_stat(const struct index_state *istate,
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 22/24] mingw32: add uname()

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Helped-by: Eric Sunshine sunsh...@sunshineco.com
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 compat/mingw.c | 11 +++
 compat/mingw.h |  9 +
 2 files changed, 20 insertions(+)

diff --git a/compat/mingw.c b/compat/mingw.c
index c5c37e5..88140e4 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -2128,3 +2128,14 @@ void mingw_startup()
/* initialize Unicode console */
winansi_init();
 }
+
+int uname(struct utsname *buf)
+{
+   DWORD v = GetVersion();
+   memset(buf, 0, sizeof(*buf));
+   strcpy(buf-sysname, Windows);
+   sprintf(buf-release, %u.%u, v  0xff, (v  8)  0xff);
+   /* assuming NT variants only.. */
+   sprintf(buf-version, %u, (v  16)  0x7fff);
+   return 0;
+}
diff --git a/compat/mingw.h b/compat/mingw.h
index df0e320..d00ba7a 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -77,6 +77,14 @@ struct itimerval {
 };
 #define ITIMER_REAL 0
 
+struct utsname {
+   char sysname[16];
+   char nodename[1];
+   char release[16];
+   char version[16];
+   char machine[1];
+};
+
 /*
  * sanitize preprocessor namespace polluted by Windows headers defining
  * macros which collide with git local versions
@@ -166,6 +174,7 @@ struct passwd *getpwuid(uid_t uid);
 int setitimer(int type, struct itimerval *in, struct itimerval *out);
 int sigaction(int sig, struct sigaction *in, struct sigaction *out);
 int link(const char *oldpath, const char *newpath);
+int uname(struct utsname *buf);
 
 /*
  * replacements of existing functions
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 23/24] untracked cache: guard and disable on system changes

2015-01-20 Thread Nguyễn Thái Ngọc Duy
If the user enables untracked cache, then

 - move worktree to an unsupported filesystem
 - or simply upgrade OS
 - or move the whole (portable) disk from one machine to another
 - or access a shared fs from another machine

there's no guarantee that untracked cache can still function properly.
Record the worktree location and OS footprint in the cache. If it
changes, err on the safe side and disable the cache. The user can
'update-index --untracked-cache' again to make sure all conditions are
met.

This adds a new requirement that setup_git_directory* must be called
before read_cache() because we need worktree location by then, or the
cache is dropped.

This change does not cover all bases, you can fool it if you try
hard. The point is to stop accidents.

Helped-by: Eric Sunshine sunsh...@sunshineco.com
Helped-by: brian m. carlson sand...@crustytoothpaste.net
Helped-by: Torsten Bögershausen tbo...@web.de
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
---
 Documentation/technical/index-format.txt |  4 +++
 builtin/update-index.c   | 16 ++
 dir.c| 55 +++-
 dir.h|  2 ++
 git-compat-util.h|  1 +
 test-dump-untracked-cache.c  |  1 +
 6 files changed, 72 insertions(+), 7 deletions(-)

diff --git a/Documentation/technical/index-format.txt 
b/Documentation/technical/index-format.txt
index b97ac8d..0045b89 100644
--- a/Documentation/technical/index-format.txt
+++ b/Documentation/technical/index-format.txt
@@ -242,6 +242,10 @@ Git index format
 
   The extension starts with
 
+  - A sequence of NUL-terminated strings, preceded by the size of the
+sequence in variable width encoding. Each string describes the
+environment where the cache can be used.
+
   - Stat data of $GIT_DIR/info/exclude. See Index entry section from
 ctime field until file size.
 
diff --git a/builtin/update-index.c b/builtin/update-index.c
index f23ec83..e76740d 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1083,7 +1083,7 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
the_index.split_index = NULL;
the_index.cache_changed |= SOMETHING_CHANGED;
}
-   if (untracked_cache  0  !the_index.untracked) {
+   if (untracked_cache  0) {
struct untracked_cache *uc;
 
if (untracked_cache  2) {
@@ -1091,11 +1091,15 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
if (!test_if_untracked_cache_is_supported())
return 1;
}
-   uc = xcalloc(1, sizeof(*uc));
-   uc-exclude_per_dir = .gitignore;
-   /* should be the same flags used by git-status */
-   uc-dir_flags = DIR_SHOW_OTHER_DIRECTORIES | 
DIR_HIDE_EMPTY_DIRECTORIES;
-   the_index.untracked = uc;
+   if (!the_index.untracked) {
+   uc = xcalloc(1, sizeof(*uc));
+   strbuf_init(uc-ident, 100);
+   uc-exclude_per_dir = .gitignore;
+   /* should be the same flags used by git-status */
+   uc-dir_flags = DIR_SHOW_OTHER_DIRECTORIES | 
DIR_HIDE_EMPTY_DIRECTORIES;
+   the_index.untracked = uc;
+   }
+   add_untracked_ident(the_index.untracked);
the_index.cache_changed |= UNTRACKED_CHANGED;
} else if (!untracked_cache  the_index.untracked) {
the_index.untracked = NULL;
diff --git a/dir.c b/dir.c
index 5b9dd5d..b8a4f9e 100644
--- a/dir.c
+++ b/dir.c
@@ -1793,6 +1793,40 @@ static int treat_leading_path(struct dir_struct *dir,
return rc;
 }
 
+static const char *get_ident_string(void)
+{
+   static struct strbuf sb = STRBUF_INIT;
+   struct utsname uts;
+
+   if (sb.len)
+   return sb.buf;
+   if (uname(uts))
+   die_errno(_(failed to get kernel name and information));
+   strbuf_addf(sb, Location %s, system %s %s %s, get_git_work_tree(),
+   uts.sysname, uts.release, uts.version);
+   return sb.buf;
+}
+
+static int ident_in_untracked(const struct untracked_cache *uc)
+{
+   const char *end = uc-ident.buf + uc-ident.len;
+   const char *p   = uc-ident.buf;
+
+   for (p = uc-ident.buf; p  end; p += strlen(p) + 1)
+   if (!strcmp(p, get_ident_string()))
+   return 1;
+   return 0;
+}
+
+void add_untracked_ident(struct untracked_cache *uc)
+{
+   if (ident_in_untracked(uc))
+   return;
+   strbuf_addstr(uc-ident, get_ident_string());
+   /* this strbuf contains a list of strings, save NUL too */
+   strbuf_addch(uc-ident, 0);
+}
+
 static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct 
*dir,
   

[PATCH 21/24] t7063: tests for untracked cache

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 .gitignore |   1 +
 Makefile   |   1 +
 t/t7063-status-untracked-cache.sh (new +x) | 353 +
 test-dump-untracked-cache.c (new)  |  61 +
 4 files changed, 416 insertions(+)
 create mode 100755 t/t7063-status-untracked-cache.sh
 create mode 100644 test-dump-untracked-cache.c

diff --git a/.gitignore b/.gitignore
index 81e12c0..e2bb375 100644
--- a/.gitignore
+++ b/.gitignore
@@ -182,6 +182,7 @@
 /test-delta
 /test-dump-cache-tree
 /test-dump-split-index
+/test-dump-untracked-cache
 /test-scrap-cache-tree
 /test-genrandom
 /test-hashmap
diff --git a/Makefile b/Makefile
index 9f984a9..fa58a53 100644
--- a/Makefile
+++ b/Makefile
@@ -555,6 +555,7 @@ TEST_PROGRAMS_NEED_X += test-date
 TEST_PROGRAMS_NEED_X += test-delta
 TEST_PROGRAMS_NEED_X += test-dump-cache-tree
 TEST_PROGRAMS_NEED_X += test-dump-split-index
+TEST_PROGRAMS_NEED_X += test-dump-untracked-cache
 TEST_PROGRAMS_NEED_X += test-genrandom
 TEST_PROGRAMS_NEED_X += test-hashmap
 TEST_PROGRAMS_NEED_X += test-index-version
diff --git a/t/t7063-status-untracked-cache.sh 
b/t/t7063-status-untracked-cache.sh
new file mode 100755
index 000..2b2ffd7
--- /dev/null
+++ b/t/t7063-status-untracked-cache.sh
@@ -0,0 +1,353 @@
+#!/bin/sh
+
+test_description='test untracked cache'
+
+. ./test-lib.sh
+
+avoid_racy() {
+   sleep 1
+}
+
+git update-index --untracked-cache
+# It's fine if git update-index returns an error code other than one,
+# it'll be caught in the first test.
+if test $? -eq 1; then
+   skip_all='This system does not support untracked cache'
+   test_done
+fi
+
+test_expect_success 'setup' '
+   git init worktree 
+   cd worktree 
+   mkdir done dtwo dthree 
+   touch one two three done/one dtwo/two dthree/three 
+   git add one two done/one 
+   : .git/info/exclude 
+   git update-index --untracked-cache
+'
+
+test_expect_success 'untracked cache is empty' '
+   test-dump-untracked-cache ../actual 
+   cat ../expect EOF 
+info/exclude 
+core.excludesfile 
+exclude_per_dir .gitignore
+flags 0006
+EOF
+   test_cmp ../expect ../actual
+'
+
+cat ../status.expect EOF 
+A  done/one
+A  one
+A  two
+?? dthree/
+?? dtwo/
+?? three
+EOF
+
+cat ../dump.expect EOF 
+info/exclude e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+core.excludesfile 
+exclude_per_dir .gitignore
+flags 0006
+/  recurse valid
+dthree/
+dtwo/
+three
+/done/  recurse valid
+/dthree/  recurse check_only valid
+three
+/dtwo/  recurse check_only valid
+two
+EOF
+
+test_expect_success 'status first time (empty cache)' '
+   avoid_racy 
+   : ../trace 
+   GIT_TRACE_UNTRACKED_STATS=$TRASH_DIRECTORY/trace \
+   git status --porcelain ../actual 
+   test_cmp ../status.expect ../actual 
+   cat ../trace.expect EOF 
+node creation: 3
+gitignore invalidation: 1
+directory invalidation: 0
+opendir: 4
+EOF
+   test_cmp ../trace.expect ../trace
+'
+
+test_expect_success 'untracked cache after first status' '
+   test-dump-untracked-cache ../actual 
+   test_cmp ../dump.expect ../actual
+'
+
+test_expect_success 'status second time (fully populated cache)' '
+   avoid_racy 
+   : ../trace 
+   GIT_TRACE_UNTRACKED_STATS=$TRASH_DIRECTORY/trace \
+   git status --porcelain ../actual 
+   test_cmp ../status.expect ../actual 
+   cat ../trace.expect EOF 
+node creation: 0
+gitignore invalidation: 0
+directory invalidation: 0
+opendir: 0
+EOF
+   test_cmp ../trace.expect ../trace
+'
+
+test_expect_success 'untracked cache after second status' '
+   test-dump-untracked-cache ../actual 
+   test_cmp ../dump.expect ../actual
+'
+
+test_expect_success 'modify in root directory, one dir invalidation' '
+   avoid_racy 
+   : four 
+   : ../trace 
+   GIT_TRACE_UNTRACKED_STATS=$TRASH_DIRECTORY/trace \
+   git status --porcelain ../actual 
+   cat ../status.expect EOF 
+A  done/one
+A  one
+A  two
+?? dthree/
+?? dtwo/
+?? four
+?? three
+EOF
+   test_cmp ../status.expect ../actual 
+   cat ../trace.expect EOF 
+node creation: 0
+gitignore invalidation: 0
+directory invalidation: 1
+opendir: 1
+EOF
+   test_cmp ../trace.expect ../trace
+
+'
+
+test_expect_success 'verify untracked cache dump' '
+   test-dump-untracked-cache ../actual 
+   cat ../expect EOF 
+info/exclude e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+core.excludesfile 
+exclude_per_dir .gitignore
+flags 0006
+/  

[PATCH 15/24] untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATS

2015-01-20 Thread Nguyễn Thái Ngọc Duy
This could be used to verify correct behavior in tests

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 12 
 1 file changed, 12 insertions(+)

diff --git a/dir.c b/dir.c
index 439ff22..c5ca5ce 100644
--- a/dir.c
+++ b/dir.c
@@ -1922,6 +1922,18 @@ int read_directory(struct dir_struct *dir, const char 
*path, int len, const stru
free_simplify(simplify);
qsort(dir-entries, dir-nr, sizeof(struct dir_entry *), cmp_name);
qsort(dir-ignored, dir-ignored_nr, sizeof(struct dir_entry *), 
cmp_name);
+   if (dir-untracked) {
+   static struct trace_key trace_untracked_stats = 
TRACE_KEY_INIT(UNTRACKED_STATS);
+   trace_printf_key(trace_untracked_stats,
+node creation: %u\n
+gitignore invalidation: %u\n
+directory invalidation: %u\n
+opendir: %u\n,
+dir-untracked-dir_created,
+dir-untracked-gitignore_invalidated,
+dir-untracked-dir_invalidated,
+dir-untracked-dir_opened);
+   }
return dir-nr;
 }
 
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 14/24] untracked cache: avoid racy timestamps

2015-01-20 Thread Nguyễn Thái Ngọc Duy
When a directory is updated within the same second that its timestamp
is last saved, we cannot realize the directory has been updated by
checking timestamps. Assume the worst (something is update). See
29e4d36 (Racy GIT - 2005-12-20) for more information.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 cache.h  | 2 ++
 dir.c| 4 ++--
 read-cache.c | 8 
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/cache.h b/cache.h
index b14d6e2..f8b3dc5 100644
--- a/cache.h
+++ b/cache.h
@@ -561,6 +561,8 @@ extern void fill_stat_data(struct stat_data *sd, struct 
stat *st);
  * INODE_CHANGED, and DATA_CHANGED.
  */
 extern int match_stat_data(const struct stat_data *sd, struct stat *st);
+extern int match_stat_data_racy(const struct index_state *istate,
+   const struct stat_data *sd, struct stat *st);
 
 extern void fill_stat_cache_info(struct cache_entry *ce, struct stat *st);
 
diff --git a/dir.c b/dir.c
index 30e23f8..439ff22 100644
--- a/dir.c
+++ b/dir.c
@@ -682,7 +682,7 @@ static int add_excludes(const char *fname, const char 
*base, int baselen,
if (sha1_stat) {
int pos;
if (sha1_stat-valid 
-   !match_stat_data(sha1_stat-stat, st))
+   !match_stat_data_racy(the_index, sha1_stat-stat, 
st))
; /* no content change, ss-sha1 still good */
else if (check_index 
 (pos = cache_name_pos(fname, strlen(fname))) 
= 0 
@@ -1538,7 +1538,7 @@ static int valid_cached_dir(struct dir_struct *dir,
return 0;
}
if (!untracked-valid ||
-   match_stat_data(untracked-stat_data, st)) {
+   match_stat_data_racy(the_index, untracked-stat_data, st)) {
if (untracked-valid)
invalidate_directory(dir-untracked, untracked);
fill_stat_data(untracked-stat_data, st);
diff --git a/read-cache.c b/read-cache.c
index f12a185..0ecba05 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -292,6 +292,14 @@ static int is_racy_timestamp(const struct index_state 
*istate,
is_racy_stat(istate, ce-ce_stat_data));
 }
 
+int match_stat_data_racy(const struct index_state *istate,
+const struct stat_data *sd, struct stat *st)
+{
+   if (is_racy_stat(istate, sd))
+   return MTIME_CHANGED;
+   return match_stat_data(sd, st);
+}
+
 int ie_match_stat(const struct index_state *istate,
  const struct cache_entry *ce, struct stat *st,
  unsigned int options)
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 20/24] update-index: test the system before enabling untracked cache

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Helped-by: Eric Sunshine sunsh...@sunshineco.com
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 Documentation/git-update-index.txt |   6 ++
 builtin/update-index.c | 148 +
 2 files changed, 154 insertions(+)

diff --git a/Documentation/git-update-index.txt 
b/Documentation/git-update-index.txt
index f9a35cd..ed32bae 100644
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -180,6 +180,12 @@ may not support it yet.
system must change `st_mtime` field of a directory if files
are added or deleted in that directory.
 
+--force-untracked-cache::
+   For safety, `--untracked-cache` performs tests on the working
+   directory to make sure untracked cache can be used. These
+   tests can take a few seconds. `--force-untracked-cache` can be
+   used to skip the tests.
+
 \--::
Do not interpret any more arguments as options.
 
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 3d2dedd..f23ec83 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -47,6 +47,147 @@ static void report(const char *fmt, ...)
va_end(vp);
 }
 
+static void remove_test_directory(void)
+{
+   struct strbuf sb = STRBUF_INIT;
+   strbuf_addstr(sb, dir-mtime-test);
+   remove_dir_recursively(sb, 0);
+   strbuf_release(sb);
+}
+
+static void xmkdir(const char *path)
+{
+   if (mkdir(path, 0700))
+   die_errno(_(failed to create directory %s), path);
+}
+
+static int xstat(const char *path, struct stat *st)
+{
+   if (stat(path, st))
+   die_errno(_(failed to stat %s), path);
+   return 0;
+}
+
+static int create_file(const char *path)
+{
+   int fd = open(path, O_CREAT | O_RDWR, 0644);
+   if (fd  0)
+   die_errno(_(failed to create file %s), path);
+   return fd;
+}
+
+static void xunlink(const char *path)
+{
+   if (unlink(path))
+   die_errno(_(failed to delete file %s), path);
+}
+
+static void xrmdir(const char *path)
+{
+   if (rmdir(path))
+   die_errno(_(failed to delete directory %s), path);
+}
+
+static void avoid_racy(void)
+{
+   /*
+* not use if we could usleep(10) if USE_NSEC is defined. The
+* field nsec could be there, but the OS could choose to
+* ignore it?
+*/
+   sleep(1);
+}
+
+static int test_if_untracked_cache_is_supported(void)
+{
+   struct stat st;
+   struct stat_data base;
+   int fd;
+
+   fprintf(stderr, _(Testing ));
+   xmkdir(dir-mtime-test);
+   atexit(remove_test_directory);
+   xstat(dir-mtime-test, st);
+   fill_stat_data(base, st);
+   fputc('.', stderr);
+
+   avoid_racy();
+   fd = create_file(dir-mtime-test/newfile);
+   xstat(dir-mtime-test, st);
+   if (!match_stat_data(base, st)) {
+   close(fd);
+   fputc('\n', stderr);
+   fprintf_ln(stderr,_(directory stat info does not 
+   change after adding a new file));
+   return 0;
+   }
+   fill_stat_data(base, st);
+   fputc('.', stderr);
+
+   avoid_racy();
+   xmkdir(dir-mtime-test/new-dir);
+   xstat(dir-mtime-test, st);
+   if (!match_stat_data(base, st)) {
+   close(fd);
+   fputc('\n', stderr);
+   fprintf_ln(stderr, _(directory stat info does not change 
+after adding a new directory));
+   return 0;
+   }
+   fill_stat_data(base, st);
+   fputc('.', stderr);
+
+   avoid_racy();
+   write_or_die(fd, data, 4);
+   close(fd);
+   xstat(dir-mtime-test, st);
+   if (match_stat_data(base, st)) {
+   fputc('\n', stderr);
+   fprintf_ln(stderr, _(directory stat info changes 
+after updating a file));
+   return 0;
+   }
+   fputc('.', stderr);
+
+   avoid_racy();
+   close(create_file(dir-mtime-test/new-dir/new));
+   xstat(dir-mtime-test, st);
+   if (match_stat_data(base, st)) {
+   fputc('\n', stderr);
+   fprintf_ln(stderr, _(directory stat info changes after 
+adding a file inside subdirectory));
+   return 0;
+   }
+   fputc('.', stderr);
+
+   avoid_racy();
+   xunlink(dir-mtime-test/newfile);
+   xstat(dir-mtime-test, st);
+   if (!match_stat_data(base, st)) {
+   fputc('\n', stderr);
+   fprintf_ln(stderr, _(directory stat info does not 
+change after deleting a file));
+   return 0;
+   }
+   fill_stat_data(base, st);
+   fputc('.', stderr);
+
+   avoid_racy();
+   xunlink(dir-mtime-test/new-dir/new);
+   

[PATCH 17/24] untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHE

2015-01-20 Thread Nguyễn Thái Ngọc Duy
This can be used to double check if results with untracked cache are
correctly, compared to vanilla version. Untracked cache remains in
index, but not used.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dir.c b/dir.c
index 1c3db0b..5b9dd5d 100644
--- a/dir.c
+++ b/dir.c
@@ -1800,7 +1800,7 @@ static struct untracked_cache_dir 
*validate_untracked_cache(struct dir_struct *d
struct untracked_cache_dir *root;
int i;
 
-   if (!dir-untracked)
+   if (!dir-untracked || getenv(GIT_DISABLE_UNTRACKED_CACHE))
return NULL;
 
/*
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 18/24] status: enable untracked cache

2015-01-20 Thread Nguyễn Thái Ngọc Duy
update_index_if_able() is moved down so that the updated untracked
cache could be written out.

Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 builtin/commit.c | 5 +++--
 wt-status.c  | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 5ed6036..bdcfa61 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1372,13 +1372,14 @@ int cmd_status(int argc, const char **argv, const char 
*prefix)
refresh_index(the_index, REFRESH_QUIET|REFRESH_UNMERGED, s.pathspec, 
NULL, NULL);
 
fd = hold_locked_index(index_lock, 0);
-   if (0 = fd)
-   update_index_if_able(the_index, index_lock);
 
s.is_initial = get_sha1(s.reference, sha1) ? 1 : 0;
s.ignore_submodule_arg = ignore_submodule_arg;
wt_status_collect(s);
 
+   if (0 = fd)
+   update_index_if_able(the_index, index_lock);
+
if (s.relative_paths)
s.prefix = prefix;
 
diff --git a/wt-status.c b/wt-status.c
index 27da529..8880c3b 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -585,6 +585,8 @@ static void wt_status_collect_untracked(struct wt_status *s)
DIR_SHOW_OTHER_DIRECTORIES | DIR_HIDE_EMPTY_DIRECTORIES;
if (s-show_ignored_files)
dir.flags |= DIR_SHOW_IGNORED_TOO;
+   else
+   dir.untracked = the_index.untracked;
setup_standard_excludes(dir);
 
fill_directory(dir, s-pathspec);
-- 
2.2.0.84.ge9c7a8a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 01/24] dir.c: optionally compute sha-1 of a .gitignore file

2015-01-20 Thread Nguyễn Thái Ngọc Duy
This is not used anywhere yet. But the goal is to compare quickly if a
.gitignore file has changed when we have the SHA-1 of both old (cached
somewhere) and new (from index or a tree) versions.

Helped-by: Junio C Hamano gits...@pobox.com
Helped-by: Torsten Bögershausen tbo...@web.de
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 53 ++---
 dir.h |  6 ++
 2 files changed, 52 insertions(+), 7 deletions(-)

diff --git a/dir.c b/dir.c
index fcb6872..4cc936b 100644
--- a/dir.c
+++ b/dir.c
@@ -466,7 +466,8 @@ void add_exclude(const char *string, const char *base,
x-el = el;
 }
 
-static void *read_skip_worktree_file_from_index(const char *path, size_t *size)
+static void *read_skip_worktree_file_from_index(const char *path, size_t *size,
+   struct sha1_stat *sha1_stat)
 {
int pos, len;
unsigned long sz;
@@ -485,6 +486,10 @@ static void *read_skip_worktree_file_from_index(const char 
*path, size_t *size)
return NULL;
}
*size = xsize_t(sz);
+   if (sha1_stat) {
+   memset(sha1_stat-stat, 0, sizeof(sha1_stat-stat));
+   hashcpy(sha1_stat-sha1, active_cache[pos]-sha1);
+   }
return data;
 }
 
@@ -529,11 +534,18 @@ static void trim_trailing_spaces(char *buf)
*last_space = '\0';
 }
 
-int add_excludes_from_file_to_list(const char *fname,
-  const char *base,
-  int baselen,
-  struct exclude_list *el,
-  int check_index)
+/*
+ * Given a file with name fname, read it (either from disk, or from
+ * the index if check_index is non-zero), parse it and store the
+ * exclude rules in el.
+ *
+ * If ss is not NULL, compute SHA-1 of the exclude file and fill
+ * stat data from disk (only valid if add_excludes returns zero). If
+ * ss_valid is non-zero, ss must contain good value as input.
+ */
+static int add_excludes(const char *fname, const char *base, int baselen,
+   struct exclude_list *el, int check_index,
+   struct sha1_stat *sha1_stat)
 {
struct stat st;
int fd, i, lineno = 1;
@@ -547,7 +559,7 @@ int add_excludes_from_file_to_list(const char *fname,
if (0 = fd)
close(fd);
if (!check_index ||
-   (buf = read_skip_worktree_file_from_index(fname, size)) == 
NULL)
+   (buf = read_skip_worktree_file_from_index(fname, size, 
sha1_stat)) == NULL)
return -1;
if (size == 0) {
free(buf);
@@ -560,6 +572,11 @@ int add_excludes_from_file_to_list(const char *fname,
} else {
size = xsize_t(st.st_size);
if (size == 0) {
+   if (sha1_stat) {
+   fill_stat_data(sha1_stat-stat, st);
+   hashcpy(sha1_stat-sha1, EMPTY_BLOB_SHA1_BIN);
+   sha1_stat-valid = 1;
+   }
close(fd);
return 0;
}
@@ -571,6 +588,21 @@ int add_excludes_from_file_to_list(const char *fname,
}
buf[size++] = '\n';
close(fd);
+   if (sha1_stat) {
+   int pos;
+   if (sha1_stat-valid 
+   !match_stat_data(sha1_stat-stat, st))
+   ; /* no content change, ss-sha1 still good */
+   else if (check_index 
+(pos = cache_name_pos(fname, strlen(fname))) 
= 0 
+!ce_stage(active_cache[pos]) 
+ce_uptodate(active_cache[pos]))
+   hashcpy(sha1_stat-sha1, 
active_cache[pos]-sha1);
+   else
+   hash_sha1_file(buf, size, blob, 
sha1_stat-sha1);
+   fill_stat_data(sha1_stat-stat, st);
+   sha1_stat-valid = 1;
+   }
}
 
el-filebuf = buf;
@@ -589,6 +621,13 @@ int add_excludes_from_file_to_list(const char *fname,
return 0;
 }
 
+int add_excludes_from_file_to_list(const char *fname, const char *base,
+  int baselen, struct exclude_list *el,
+  int check_index)
+{
+   return add_excludes(fname, base, baselen, el, check_index, NULL);
+}
+
 struct exclude_list *add_exclude_list(struct dir_struct *dir,
  int group_type, const char *src)
 {
diff --git a/dir.h b/dir.h
index 6c45e9d..cdca71b 100644
--- a/dir.h
+++ b/dir.h
@@ -73,6 +73,12 @@ struct 

[PATCH 02/24] untracked cache: record .gitignore information and dir hierarchy

2015-01-20 Thread Nguyễn Thái Ngọc Duy
The idea is if we can capture all input and (non-rescursive) output of
read_directory_recursive(), and can verify later that all the input is
the same, then the second r_d_r() should produce the same output as in
the first run.

The requirement for this to work is stat info of a directory MUST
change if an entry is added to or removed from that directory (and
should not change often otherwise). If your OS and filesystem do not
meet this requirement, untracked cache is not for you. Most file
systems on *nix should be fine. On Windows, NTFS is fine while FAT may
not be [1] even though FAT on Linux seems to be fine.

The list of input of r_d_r() is in the big comment block in dir.h. In
short, the output of a directory (not counting subdirs) mainly depends
on stat info of the directory in question, all .gitignore leading to
it and the check_only flag when r_d_r() is called recursively. This
patch records all this info (and the output) as r_d_r() runs.

Two hash_sha1_file() are required for $GIT_DIR/info/exclude and
core.excludesfile unless their stat data matches. hash_sha1_file() is
only needed when .gitignore files in the worktree are modified,
otherwise their SHA-1 in index is used (see the previous patch).

We could store stat data for .gitignore files so we don't have to
rehash them if their content is different from index, but I think
.gitignore files are rarely modified, so not worth extra cache data
(and hashing penalty read-cache.c:verify_hdr(), as we will be storing
this as an index extension).

The implication is, if you change .gitignore, you better add it to the
index soon or you lose all the benefit of untracked cache because a
modified .gitignore invalidates all subdirs recursively. This is
especially bad for .gitignore at root.

This cached output is about untracked files only, not ignored files
because the number of tracked files is usually small, so small cache
overhead, while the number of ignored files could go really high
(e.g. *.o files mixing with source code).

[1] Description of NTFS date and time stamps for files and folders
http://support.microsoft.com/kb/299648

Helped-by: Torsten Bögershausen tbo...@web.de
Helped-by: David Turner dtur...@twopensource.com
Signed-off-by: Nguyễn Thái Ngọc Duy pclo...@gmail.com
Signed-off-by: Junio C Hamano gits...@pobox.com
---
 dir.c | 142 +-
 dir.h |  60 
 2 files changed, 183 insertions(+), 19 deletions(-)

diff --git a/dir.c b/dir.c
index 4cc936b..27734f0 100644
--- a/dir.c
+++ b/dir.c
@@ -32,7 +32,7 @@ enum path_treatment {
 };
 
 static enum path_treatment read_directory_recursive(struct dir_struct *dir,
-   const char *path, int len,
+   const char *path, int len, struct untracked_cache_dir *untracked,
int check_only, const struct path_simplify *simplify);
 static int get_dtype(struct dirent *de, const char *path, int len);
 
@@ -535,6 +535,54 @@ static void trim_trailing_spaces(char *buf)
 }
 
 /*
+ * Given a subdirectory name and dir of the current directory,
+ * search the subdir in dir and return it, or create a new one if it
+ * does not exist in dir.
+ *
+ * If name has the trailing slash, it'll be excluded in the search.
+ */
+static struct untracked_cache_dir *lookup_untracked(struct untracked_cache *uc,
+   struct untracked_cache_dir 
*dir,
+   const char *name, int len)
+{
+   int first, last;
+   struct untracked_cache_dir *d;
+   if (!dir)
+   return NULL;
+   if (len  name[len - 1] == '/')
+   len--;
+   first = 0;
+   last = dir-dirs_nr;
+   while (last  first) {
+   int cmp, next = (last + first)  1;
+   d = dir-dirs[next];
+   cmp = strncmp(name, d-name, len);
+   if (!cmp  strlen(d-name)  len)
+   cmp = -1;
+   if (!cmp)
+   return d;
+   if (cmp  0) {
+   last = next;
+   continue;
+   }
+   first = next+1;
+   }
+
+   uc-dir_created++;
+   d = xmalloc(sizeof(*d) + len + 1);
+   memset(d, 0, sizeof(*d));
+   memcpy(d-name, name, len);
+   d-name[len] = '\0';
+
+   ALLOC_GROW(dir-dirs, dir-dirs_nr + 1, dir-dirs_alloc);
+   memmove(dir-dirs + first + 1, dir-dirs + first,
+   (dir-dirs_nr - first) * sizeof(*dir-dirs));
+   dir-dirs_nr++;
+   dir-dirs[first] = d;
+   return d;
+}
+
+/*
  * Given a file with name fname, read it (either from disk, or from
  * the index if check_index is non-zero), parse it and store the
  * exclude rules in el.
@@ -645,14 +693,20 @@ struct exclude_list *add_exclude_list(struct dir_struct 
*dir,
 /*
  * Used to set up core.excludesfile and .git/info/exclude lists.
  */
-void add_excludes_from_file(struct dir_struct 

[PATCH 00/24] nd/untracked-cache update

2015-01-20 Thread Nguyễn Thái Ngọc Duy
Sorry for this really late update. This fixes bugs in extension
writing code (10/24), support using the same cache from different
hosts (23/24), and adds a new bug to point the user to untracked cache from
'git status -uno' (new patch 24/24)

Diff from 'pu'
-- 8 --
diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt
index def635f..7850f53 100644
--- a/Documentation/git-status.txt
+++ b/Documentation/git-status.txt
@@ -58,7 +58,10 @@ When `-u` option is not used, untracked files and 
directories are
 shown (i.e. the same as specifying `normal`), to help you avoid
 forgetting to add newly created files.  Because it takes extra work
 to find untracked files in the filesystem, this mode may take some
-time in a large working tree.  You can use `no` to have `git status`
+time in a large working tree.
+Consider to enable untracked cache and split index if supported (see
+`git update-index --untracked-cache` and `git update-index
+--split-index`), Otherwise you can use `no` to have `git status`
 return more quickly without showing untracked files.
 +
 The default can be changed using the status.showUntrackedFiles
diff --git a/Documentation/technical/index-format.txt 
b/Documentation/technical/index-format.txt
index 5dc2bee..0045b89 100644
--- a/Documentation/technical/index-format.txt
+++ b/Documentation/technical/index-format.txt
@@ -242,8 +242,9 @@ Git index format
 
   The extension starts with
 
-  - A NUL-terminated string describing the environment when the cache
-is created.
+  - A sequence of NUL-terminated strings, preceded by the size of the
+sequence in variable width encoding. Each string describes the
+environment where the cache can be used.
 
   - Stat data of $GIT_DIR/info/exclude. See Index entry section from
 ctime field until file size.
diff --git a/builtin/update-index.c b/builtin/update-index.c
index f23ec83..e76740d 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1083,7 +1083,7 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
the_index.split_index = NULL;
the_index.cache_changed |= SOMETHING_CHANGED;
}
-   if (untracked_cache  0  !the_index.untracked) {
+   if (untracked_cache  0) {
struct untracked_cache *uc;
 
if (untracked_cache  2) {
@@ -1091,11 +1091,15 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
if (!test_if_untracked_cache_is_supported())
return 1;
}
-   uc = xcalloc(1, sizeof(*uc));
-   uc-exclude_per_dir = .gitignore;
-   /* should be the same flags used by git-status */
-   uc-dir_flags = DIR_SHOW_OTHER_DIRECTORIES | 
DIR_HIDE_EMPTY_DIRECTORIES;
-   the_index.untracked = uc;
+   if (!the_index.untracked) {
+   uc = xcalloc(1, sizeof(*uc));
+   strbuf_init(uc-ident, 100);
+   uc-exclude_per_dir = .gitignore;
+   /* should be the same flags used by git-status */
+   uc-dir_flags = DIR_SHOW_OTHER_DIRECTORIES | 
DIR_HIDE_EMPTY_DIRECTORIES;
+   the_index.untracked = uc;
+   }
+   add_untracked_ident(the_index.untracked);
the_index.cache_changed |= UNTRACKED_CHANGED;
} else if (!untracked_cache  the_index.untracked) {
the_index.untracked = NULL;
diff --git a/dir.c b/dir.c
index 95ff3f0..b8a4f9e 100644
--- a/dir.c
+++ b/dir.c
@@ -1793,6 +1793,40 @@ static int treat_leading_path(struct dir_struct *dir,
return rc;
 }
 
+static const char *get_ident_string(void)
+{
+   static struct strbuf sb = STRBUF_INIT;
+   struct utsname uts;
+
+   if (sb.len)
+   return sb.buf;
+   if (uname(uts))
+   die_errno(_(failed to get kernel name and information));
+   strbuf_addf(sb, Location %s, system %s %s %s, get_git_work_tree(),
+   uts.sysname, uts.release, uts.version);
+   return sb.buf;
+}
+
+static int ident_in_untracked(const struct untracked_cache *uc)
+{
+   const char *end = uc-ident.buf + uc-ident.len;
+   const char *p   = uc-ident.buf;
+
+   for (p = uc-ident.buf; p  end; p += strlen(p) + 1)
+   if (!strcmp(p, get_ident_string()))
+   return 1;
+   return 0;
+}
+
+void add_untracked_ident(struct untracked_cache *uc)
+{
+   if (ident_in_untracked(uc))
+   return;
+   strbuf_addstr(uc-ident, get_ident_string());
+   /* this strbuf contains a list of strings, save NUL too */
+   strbuf_addch(uc-ident, 0);
+}
+
 static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct 
*dir,
  int base_len,
  const struct pathspec 

[ANNOUNCE] git-arr 0.14

2015-01-20 Thread Alberto Bertogli

Hi!

git-arr is a git repository browser that can generate static HTML
instead of having to run dynamically.

I've just released version 0.14, which includes minor fixes and
performance improvements, as well as the following features:

 - Improved max_pages handling, including saner defaults.
 - Show a creation event for the root commit (optionally).
 - Support for hierarchical branch names.
 - Render hexdump(1)-style binary blob content.

You can find it at http://blitiri.com.ar/p/git-arr.


Thanks specially to Eric Sunshine who wrote almost all the changes in
this release, including all the features mentioned above, as well as
many code cleanups.

Cheers!
Alberto

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


『BODY雜誌金纖獎』專家評鑑超人氣口碑絕對有信心!ixccbew05qzz

2015-01-20 Thread gsafgasaewtfjdsf agssaasgf
陶鈴立ixccbew05qzz



全新第四代金善美有效提升代謝,

還妳窈宨曲線,享受28腰真的不難。

『BODY雜誌金纖獎』專家評鑑超人氣口碑,

絕對有信心,天然配方安全無慮,超強力提升基礎代謝。

金善美官方網站 http://ow.ly/HpafR 












acezm陶鈴立
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3] remote-curl: fall back to Basic auth if Negotiate fails

2015-01-20 Thread Dan Langille (dalangil)
I did not test this patch.  Is that holding up a commit?
— 
Dan Langille
Infrastructure  Operations
Talos Group
Sourcefire, Inc.

 On Jan 7, 2015, at 7:29 PM, brian m. carlson sand...@crustytoothpaste.net 
 wrote:
 
 Apache servers using mod_auth_kerb can be configured to allow the user
 to authenticate either using Negotiate (using the Kerberos ticket) or
 Basic authentication (using the Kerberos password).  Often, one will
 want to use Negotiate authentication if it is available, but fall back
 to Basic authentication if the ticket is missing or expired.
 
 However, libcurl will try very hard to use something other than Basic
 auth, even over HTTPS.  If Basic and something else are offered, libcurl
 will never attempt to use Basic, even if the other option fails.
 Teach the HTTP client code to stop trying authentication mechanisms that
 don't use a password (currently Negotiate) after the first failure,
 since if they failed the first time, they will never succeed.
 
 Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
 Signed-off-by: Jeff King p...@peff.net
 ---
 Peff's original change was to get_curl_handle; however, we retry the
 second time with the same slot and we may not call get_curl_handle
 again, so I had to move that change to get_active_slot.  This has been
 tested pushing with both Negotiate and Basic against an HTTPS server
 both when info/refs was protected and when it was not.
 
 http.c | 10 ++
 1 file changed, 10 insertions(+)
 
 diff --git a/http.c b/http.c
 index 040f362..44b130c 100644
 --- a/http.c
 +++ b/http.c
 @@ -62,6 +62,9 @@ static const char *user_agent;
 
 static struct credential cert_auth = CREDENTIAL_INIT;
 static int ssl_cert_password_required;
 +#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY
 +static unsigned long http_auth_methods = CURLAUTH_ANY;
 +#endif
 
 static struct curl_slist *pragma_header;
 static struct curl_slist *no_pragma_header;
 @@ -580,6 +583,9 @@ struct active_request_slot *get_active_slot(void)
   curl_easy_setopt(slot-curl, CURLOPT_UPLOAD, 0);
   curl_easy_setopt(slot-curl, CURLOPT_HTTPGET, 1);
   curl_easy_setopt(slot-curl, CURLOPT_FAILONERROR, 1);
 +#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY
 + curl_easy_setopt(slot-curl, CURLOPT_HTTPAUTH, http_auth_methods);
 +#endif
   if (http_auth.password)
   init_curl_http_auth(slot-curl);
 
 @@ -870,6 +876,9 @@ int handle_curl_result(struct slot_results *results)
   credential_reject(http_auth);
   return HTTP_NOAUTH;
   } else {
 +#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY
 + http_auth_methods = ~CURLAUTH_GSSNEGOTIATE;
 +#endif
   return HTTP_REAUTH;
   }
   } else {
 @@ -986,6 +995,7 @@ static void extract_content_type(struct strbuf *raw, 
 struct strbuf *type,
   strbuf_addstr(charset, ISO-8859-1);
 }
 
 +
 /* http_request() targets */
 #define HTTP_REQUEST_STRBUF   0
 #define HTTP_REQUEST_FILE 1
 -- 
 2.2.1.209.g41e5f3a
 



[PATCH] move MAXDEPTH definition to the cache.h

2015-01-20 Thread Alexander Kuleshov
There are a couple of source code files as abspath.c, lockfile.c and etc...,
which defines MAXDEPTH macro. All of these definitions are the same, so let's
move MAXDEPTH definition to the cache.h instead of directly declaration of
this macro in all these files.

Signed-off-by: Alexander Kuleshov kuleshovm...@gmail.com
---
 abspath.c   | 3 ---
 cache.h | 1 +
 http-push.c | 3 ---
 lockfile.c  | 4 
 refs.c  | 2 --
 5 files changed, 1 insertion(+), 12 deletions(-)

diff --git a/abspath.c b/abspath.c
index 5edb4e7..9209522 100644
--- a/abspath.c
+++ b/abspath.c
@@ -11,9 +11,6 @@ int is_directory(const char *path)
return (!stat(path, st)  S_ISDIR(st.st_mode));
 }
 
-/* We allow recursive symbolic links. Only within reason, though. */
-#define MAXDEPTH 5
-
 /*
  * Return the real path (i.e., absolute path, with symlinks resolved
  * and extra slashes removed) equivalent to the specified path.  (If
diff --git a/cache.h b/cache.h
index 64aa287..cac85b7 100644
--- a/cache.h
+++ b/cache.h
@@ -1010,6 +1010,7 @@ extern int read_ref(const char *refname, unsigned char 
*sha1);
  * Caps and underscores refers to the special refs, such as HEAD,
  * FETCH_HEAD and friends, that all live outside of the refs/ directory.
  */
+#define MAXDEPTH 5
 #define RESOLVE_REF_READING 0x01
 #define RESOLVE_REF_NO_RECURSE 0x02
 #define RESOLVE_REF_ALLOW_BAD_NAME 0x04
diff --git a/http-push.c b/http-push.c
index 0beb7ab..bb1f82e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -70,9 +70,6 @@ enum XML_Status {
 #define FETCHING (1u18)
 #define PUSHING  (1u19)
 
-/* We allow recursive symbolic refs. Only within reason, though */
-#define MAXDEPTH 5
-
 static int pushing;
 static int aborted;
 static signed char remote_dir_exists[256];
diff --git a/lockfile.c b/lockfile.c
index 9889277..88d0102 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -59,10 +59,6 @@ static void trim_last_path_component(struct strbuf *path)
strbuf_setlen(path, i);
 }
 
-
-/* We allow recursive symbolic links. Only within reason, though */
-#define MAXDEPTH 5
-
 /*
  * path contains a path that might be a symlink.
  *
diff --git a/refs.c b/refs.c
index 872cb26..c37879f 100644
--- a/refs.c
+++ b/refs.c
@@ -1335,8 +1335,6 @@ static struct ref_dir *get_loose_refs(struct ref_cache 
*refs)
return get_ref_dir(refs-loose);
 }
 
-/* We allow recursive symbolic refs. Only within reason, though */
-#define MAXDEPTH 5
 #define MAXREFLEN (1024)
 
 /*
-- 
2.3.0.rc0.286.ga3dc223.dirty

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] hash-object: add -t and --no-filters options to the hash-object synopsis

2015-01-20 Thread Alexander Kuleshov
Signed-off-by: Alexander Kuleshov kuleshovm...@gmail.com
---
 builtin/hash-object.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index 207b90c..a8100a7 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -80,7 +80,7 @@ int cmd_hash_object(int argc, const char **argv, const char 
*prefix)
 {
static const char * const hash_object_usage[] = {
N_(git hash-object [-t type] [-w] [--path=file | 
--no-filters] [--stdin] [--] file...),
-   N_(git hash-object  --stdin-paths  list-of-paths),
+   N_(git hash-object [-t type] [-w] --stdin-paths 
[--no-filters]  list-of-paths),
NULL
};
const char *type = blob_type;
-- 
2.3.0.rc0.286.ga3dc223.dirty

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Torsten Bögershausen
On 2015-01-20 20.46, Noralf Trønnes wrote:
could it be that your ø is not encoded as UTF-8,
but in ISO-8859-15 (or so)

 $ git log -1
 commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
 Author: Noralf TrF8nnes no...@tronnes.org
What does 
git config -l | grep Noralf | xxd
say ?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Noralf Trønnes

Den 20.01.2015 21:07, skrev Torsten Bögershausen:

On 2015-01-20 20.46, Noralf Trønnes wrote:
could it be that your ø is not encoded as UTF-8,
but in ISO-8859-15 (or so)


$ git log -1
commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
Author: Noralf TrF8nnes no...@tronnes.org

What does
git config -l | grep Noralf | xxd
say ?


$ git config -l | grep Noralf | xxd
000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
010: 2054 72f8 6e6e 6573 0aTr.nnes.

$ file ~/.gitconfig
/home/pi/.gitconfig: ISO-8859 text

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Ævar Arnfjörð Bjarmason
On Tue, Jan 20, 2015 at 10:23 PM, Noralf Trønnes no...@tronnes.org wrote:
 Den 20.01.2015 21:45, skrev Ævar Arnfjörð Bjarmason:

 On Tue, Jan 20, 2015 at 9:17 PM, Noralf Trønnes no...@tronnes.org wrote:

 Den 20.01.2015 21:07, skrev Torsten Bögershausen:

 On 2015-01-20 20.46, Noralf Trønnes wrote:
 could it be that your ø is not encoded as UTF-8,
 but in ISO-8859-15 (or so)

 $ git log -1
 commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
 Author: Noralf TrF8nnes no...@tronnes.org

 What does
 git config -l | grep Noralf | xxd
 say ?

 $ git config -l | grep Noralf | xxd
 000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
 010: 2054 72f8 6e6e 6573 0aTr.nnes.

 $ file ~/.gitconfig
 /home/pi/.gitconfig: ISO-8859 text

 What's happened here is that:

   1. You've authored your commit in ISO-8859-1
   2. Git itself has no place for the encoding of the author name in the
 commit object format
   3. git-format-patch has a --compose-encoding which I think would sort
 this out if you set it to ISO-8859-1, but it defaults to UTF-8
   4. Your patch is actually a ISO-8859-1 byte sequence, but is
 advertised as UTF-8
   5. You end up with a screwed-up commit

 You could work around this, but I suggest just joining the 21st
 century and working exclusively in UTF-8, it makes things much easier,
 speaking as someone with 3x more non-ASCII characters their his name
 than you :)


 Ok, then the question is: How do I switch to UTF-8?

 To me it seems I'm already using it:
 $ locale charmap
 UTF-8

Your .gitconfig has an ISO-8859-1 string, from an earlier mail of yours:

 $ git config -l | grep Noralf | xxd
 000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
 010: 2054 72f8 6e6e 6573 0aTr.nnes.

On a system configured for UTF-8 this would be:

$ echo Noralf Trønnes | xxd
000: 4e6f 7261 6c66 2054 72c3 b86e 6e65 730a  Noralf Tr..nnes.

Note the f8 v.s. c3 b8.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Ævar Arnfjörð Bjarmason
On Tue, Jan 20, 2015 at 10:38 PM, Noralf Trønnes no...@tronnes.org wrote:
 Den 20.01.2015 22:26, skrev Ævar Arnfjörð Bjarmason:

 On Tue, Jan 20, 2015 at 10:23 PM, Noralf Trønnes no...@tronnes.org
 wrote:

 Den 20.01.2015 21:45, skrev Ævar Arnfjörð Bjarmason:

 On Tue, Jan 20, 2015 at 9:17 PM, Noralf Trønnes no...@tronnes.org
 wrote:

 Den 20.01.2015 21:07, skrev Torsten Bögershausen:

 On 2015-01-20 20.46, Noralf Trønnes wrote:
 could it be that your ø is not encoded as UTF-8,
 but in ISO-8859-15 (or so)

 $ git log -1
 commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
 Author: Noralf TrF8nnes no...@tronnes.org

 What does
 git config -l | grep Noralf | xxd
 say ?

 $ git config -l | grep Noralf | xxd
 000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
 010: 2054 72f8 6e6e 6573 0aTr.nnes.

 $ file ~/.gitconfig
 /home/pi/.gitconfig: ISO-8859 text

 What's happened here is that:

1. You've authored your commit in ISO-8859-1
2. Git itself has no place for the encoding of the author name in the
 commit object format
3. git-format-patch has a --compose-encoding which I think would sort
 this out if you set it to ISO-8859-1, but it defaults to UTF-8
4. Your patch is actually a ISO-8859-1 byte sequence, but is
 advertised as UTF-8
5. You end up with a screwed-up commit

 You could work around this, but I suggest just joining the 21st
 century and working exclusively in UTF-8, it makes things much easier,
 speaking as someone with 3x more non-ASCII characters their his name
 than you :)

 Ok, then the question is: How do I switch to UTF-8?

 To me it seems I'm already using it:
 $ locale charmap
 UTF-8

 Your .gitconfig has an ISO-8859-1 string, from an earlier mail of yours:

 $ git config -l | grep Noralf | xxd
 000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
 010: 2054 72f8 6e6e 6573 0aTr.nnes.

 On a system configured for UTF-8 this would be:

 $ echo Noralf Trønnes | xxd
 000: 4e6f 7261 6c66 2054 72c3 b86e 6e65 730a  Noralf Tr..nnes.

 Note the f8 v.s. c3 b8.


 Yes:
 $ echo Noralf Trønnes | xxd
 000: 4e6f 7261 6c66 2054 72f8 6e6e 6573 0aNoralf Tr.nnes.

 Is there a command I can run that shows that I'm using ISO-8859-1 ?
 I need something to google with, my previous search only gave locale stuff,
 which seems fine.

What does this give you, this is UTF-8.

$ echo git commit --author=Noralf Trønnes no...@tronnes.org | xxd
000: 6769 7420 636f 6d6d 6974 202d 2d61 7574  git commit --aut
010: 686f 723d 4e6f 7261 6c66 2054 72c3 b86e  hor=Noralf Tr..n
020: 6e65 7320 3c6e 6f74 726f 4074 726f 6e6e  nes notro@tronn
030: 6573 2e6f 7267 3e0a  es.org.

To see if you're using UTF-8 just look at the codepoints for the
non-ASCII characters you're using and check if they're valid UTF-8.
E.g. you can check this out:
http://en.wikipedia.org/wiki/%C3%98#Computers

Which shows you that the UTF-8 hex version is C3 B8, but the Latin-1
is F8, you're emitting F8, I'm emitting C3 B8.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] parse_color: fix return value for numeric color values 0-8

2015-01-20 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 Eek. Definitely an unintended regression. The fix is below. Thanks for
 reporting (and especially for catching during the -rc period!).

 You should not need it, but for reference, using 0 is the same as
 black (both in old git and new).

 -- 8 --
 When commit 695d95d refactored the color parsing, it missed
 a return 0 when parsing literal numbers 0-8 (which
 represent basic ANSI colors), leading us to report these
 colors as an error.

 Signed-off-by: Jeff King p...@peff.net
 ---

Thanks; somebody should have caught this before we applied and
merged to 'master', but the process obviously did not work well.

Sorry and thanks.

  color.c  | 1 +
  t/t4026-color.sh | 4 
  2 files changed, 5 insertions(+)

 diff --git a/color.c b/color.c
 index 809b359..9027352 100644
 --- a/color.c
 +++ b/color.c
 @@ -112,6 +112,7 @@ static int parse_color(struct color *out, const char 
 *name, int len)
   } else if (val  8) {
   out-type = COLOR_ANSI;
   out-value = val;
 + return 0;
   } else if (val  256) {
   out-type = COLOR_256;
   out-value = val;
 diff --git a/t/t4026-color.sh b/t/t4026-color.sh
 index 267c43b..4d20fea 100755
 --- a/t/t4026-color.sh
 +++ b/t/t4026-color.sh
 @@ -60,6 +60,10 @@ test_expect_success 'absurdly long color specification' '
 [1;2;4;5;7;22;24;25;27;38;2;255;255;255;48;2;255;255;255m
  '
  
 +test_expect_success '0-7 are aliases for basic ANSI color names' '
 + color 0 7 [30;47m
 +'
 +
  test_expect_success '256 colors' '
   color 254 bold 255 [1;38;5;254;48;5;255m
  '
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] move MAXDEPTH definition to the cache.h

2015-01-20 Thread Junio C Hamano
Alexander Kuleshov kuleshovm...@gmail.com writes:

 There are a couple of source code files as abspath.c, lockfile.c and etc...,
 which defines MAXDEPTH macro. All of these definitions are the same,...

Are they the same by design (because there are logical linkage
between these values and there is a reason why they must share the
same value), or are they happen to be the same (i.e. one can be
raised for some reason without affecting others)?

I am guessing that it is the latter, and if that is the case, then
they should not be made into a single symbol.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Greg Kroah-Hartman
On Tue, Jan 20, 2015 at 09:45:46PM +0100, Ævar Arnfjörð Bjarmason wrote:
 On Tue, Jan 20, 2015 at 9:17 PM, Noralf Trønnes no...@tronnes.org wrote:
  Den 20.01.2015 21:07, skrev Torsten Bögershausen:
 
  On 2015-01-20 20.46, Noralf Trønnes wrote:
  could it be that your ø is not encoded as UTF-8,
  but in ISO-8859-15 (or so)
 
  $ git log -1
  commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
  Author: Noralf TrF8nnes no...@tronnes.org
 
  What does
  git config -l | grep Noralf | xxd
  say ?
 
  $ git config -l | grep Noralf | xxd
  000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
  010: 2054 72f8 6e6e 6573 0aTr.nnes.
 
  $ file ~/.gitconfig
  /home/pi/.gitconfig: ISO-8859 text
 
 What's happened here is that:
 
  1. You've authored your commit in ISO-8859-1
  2. Git itself has no place for the encoding of the author name in the
 commit object format
  3. git-format-patch has a --compose-encoding which I think would sort
 this out if you set it to ISO-8859-1, but it defaults to UTF-8
  4. Your patch is actually a ISO-8859-1 byte sequence, but is
 advertised as UTF-8
  5. You end up with a screwed-up commit
 
 You could work around this, but I suggest just joining the 21st
 century and working exclusively in UTF-8, it makes things much easier,
 speaking as someone with 3x more non-ASCII characters their his name
 than you :)

So how exactly do you fix this using UTF-8?  Git is exporting a UTF-8
From: line so it thinks the character is correct, but it's not
creating something properly here.

confused,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: git --recurse-submodule does not recurse to sub-submodules (etc.)

2015-01-20 Thread Jens Lehmann

Am 19.01.2015 um 21:19 schrieb Maximilian Held:

I have a directory with nested submodules, such as:

supermodule/submodule/sub-submodule/sub-sub-submodule

When I cd to supermodule and do:

git push --recurse-submodule=check (or on-demand),

git only pushes the submodule, but not the sub-submodule etc.

Maybe this is expected behavior and not a bug, but I thought it was
pretty unintuitive. I expected that git would push, well, recursively.


I agree this is unexpected and should be fixed. I suspect the fix
would be to teach the push_submodule() function to use the same
flags that were used for the push in the superproject.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Noralf Trønnes

Den 20.01.2015 22:26, skrev Ævar Arnfjörð Bjarmason:

On Tue, Jan 20, 2015 at 10:23 PM, Noralf Trønnes no...@tronnes.org wrote:

Den 20.01.2015 21:45, skrev Ævar Arnfjörð Bjarmason:


On Tue, Jan 20, 2015 at 9:17 PM, Noralf Trønnes no...@tronnes.org wrote:

Den 20.01.2015 21:07, skrev Torsten Bögershausen:

On 2015-01-20 20.46, Noralf Trønnes wrote:
could it be that your ø is not encoded as UTF-8,
but in ISO-8859-15 (or so)


$ git log -1
commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
Author: Noralf TrF8nnes no...@tronnes.org

What does
git config -l | grep Noralf | xxd
say ?


$ git config -l | grep Noralf | xxd
000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
010: 2054 72f8 6e6e 6573 0aTr.nnes.

$ file ~/.gitconfig
/home/pi/.gitconfig: ISO-8859 text

What's happened here is that:

   1. You've authored your commit in ISO-8859-1
   2. Git itself has no place for the encoding of the author name in the
commit object format
   3. git-format-patch has a --compose-encoding which I think would sort
this out if you set it to ISO-8859-1, but it defaults to UTF-8
   4. Your patch is actually a ISO-8859-1 byte sequence, but is
advertised as UTF-8
   5. You end up with a screwed-up commit

You could work around this, but I suggest just joining the 21st
century and working exclusively in UTF-8, it makes things much easier,
speaking as someone with 3x more non-ASCII characters their his name
than you :)


Ok, then the question is: How do I switch to UTF-8?

To me it seems I'm already using it:
$ locale charmap
UTF-8

Your .gitconfig has an ISO-8859-1 string, from an earlier mail of yours:


$ git config -l | grep Noralf | xxd
000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
010: 2054 72f8 6e6e 6573 0aTr.nnes.

On a system configured for UTF-8 this would be:

$ echo Noralf Trønnes | xxd
000: 4e6f 7261 6c66 2054 72c3 b86e 6e65 730a  Noralf Tr..nnes.

Note the f8 v.s. c3 b8.



Yes:
$ echo Noralf Trønnes | xxd
000: 4e6f 7261 6c66 2054 72f8 6e6e 6573 0aNoralf Tr.nnes.

Is there a command I can run that shows that I'm using ISO-8859-1 ?
I need something to google with, my previous search only gave locale 
stuff, which seems fine.


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] parse_color: fix return value for numeric color values 0-8

2015-01-20 Thread Jeff King
On Tue, Jan 20, 2015 at 03:57:13PM -0800, Junio C Hamano wrote:

  -- 8 --
  When commit 695d95d refactored the color parsing, it missed
  a return 0 when parsing literal numbers 0-8 (which
  represent basic ANSI colors), leading us to report these
  colors as an error.
 
  Signed-off-by: Jeff King p...@peff.net
  ---
 
 Thanks; somebody should have caught this before we applied and
 merged to 'master', but the process obviously did not work well.

I am not too surprised. The use of numeric values for colors was
completely undocumented, and we did not have any test coverage for it. I
did not even know it existed until I started refactoring the function,
and wondered what was going on (though I did try to preserve it once I
found it).

So I suspect that almost nobody is using this undocumented feature,
which is why it was not caught while cooking in 'next'.  The system
cannot always have perfect output, but hopefully the number of people
affected by a bug is proportional to the quickness with which it is
caught.

-Peff

PS All that being said, I think it is a good example of why it is a good
   idea to beef up test coverage in an area before refactoring. A
   trivial test would have caught this.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Pretty format specifier for commit count?

2015-01-20 Thread josh
On Tue, Jan 20, 2015 at 04:49:53PM -0500, Jeff King wrote:
 On Mon, Jan 19, 2015 at 05:17:25PM -0800, Josh Triplett wrote:
 
   Can you be a bit more specific about the type count that you are after?
   git describe counts commits since the most recent tag (possibly within
   a specific subset of all tags). Is that your desired format?
  
  That might work, since the repository in question has no tags; I'd
  actually like commits since root commit.
 
 That's basically a generation number. But I'm not sure if that's really
 what you want; in a non-linear history it's not unique (two children of
 commit X are both X+1).

That would actually be perfectly fine.  If I need to distinguish
branches, I can either use branch/tag names, or append a commit hash.  I
don't mind the following:

 /-B-\
A D
 \-C-/

A=1
B=C=2
D=3

I could (and probably should) append +hash to the version number for
uniqueness, and if I care what order B and C sort in, I can use tags,
branches, or some other more clever mechanism.

 It sounds like you really just want commits
 counting up from the root, and with side branches to have their own
 unique numbers. So something like:
 
C
   /
   A--B--D
 
   A=1
   B=2
   C=3
   D=4
 
 except the last two are assigned arbitrarily. You need some rules for
 linearizing the commits.

I don't care about the numbers assigned to anything not reachable from
the committish I start from.

 But that's not deterministic as you add more starting points (either new
 ref tips, or just new merges we have to cross). For example, imagine
 this:
 
  G--H
 /\
C--E   \
   /\   \
   A--B--D---F---I
 
 If we start at I, then we might visit H and G first, meaning we learn
 about C much earlier than we otherwise would. Then we hit F, and get to
 C from there. But now it it may be in a different position with respect
 to D!

Right, the numbers need to always stay the same as you add more commits
over time.  If walking a given graph assigns a given set of generation
numbers, walking any subgraph should assign all the same generation
numbers to the common nodes.

 I suspect your problem statement may simply assume a linear history,
 which makes this all much simpler. But we are not likely to add a
 feature to git that will break badly once you have a non-linear history. :)

Not assuming a linear history, but assuming a linear changelog file. :)

 I think in the linear case that a generation number _would_ be correct,
 and it is a useful concept by itself. So that may be the best thing to
 add.

Sounds good to me.

- Josh Triplett
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Ævar Arnfjörð Bjarmason
On Tue, Jan 20, 2015 at 10:20 PM, Jeff King p...@peff.net wrote:
 On Tue, Jan 20, 2015 at 09:45:46PM +0100, Ævar Arnfjörð Bjarmason wrote:

 What's happened here is that:

  1. You've authored your commit in ISO-8859-1
  2. Git itself has no place for the encoding of the author name in the
 commit object format

 Is (2) right? The encoding header in a commit object should apply not
 just to the commit message, but also to the author (and committer) name.

 I think the real problem is simply that it defaults to UTF-8, but he is
 giving it iso-8859-1 characters. Setting i18n.commitEncoding should fix
 it.

True, I forgot about that setting.

 -Peff

 PS If you try experimenting with this, you may fall afoul of 08a94a1
(commit/commit-tree: correct latin1 to utf-8, 2012-06-28), which will
silently correct Latin1 characters into UTF-8 (when the commit
message is expected to be in UTF-8, of course). So it actually
_should_ just work under modern gits, but only for Latin1.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] parse_color: fix return value for numeric color values 0-8

2015-01-20 Thread Jeff King
On Tue, Jan 20, 2015 at 10:49:32PM +0100, Ævar Arnfjörð Bjarmason wrote:

 I've had this in my .gitconfig since 2010 which was broken by Jeff's
 v2.1.3-24-g695d95d:
 
 ;; Don't be so invasive about coloring ^M when I'm editing files
 that
 ;; are supposed to have \r\n.
 [color diff]
whitespace = 0

 [...]
 Maybe breaking this is OK (but I can't find what the replacement is),
 but the config or the the changelog doesn't mention breaking existing
 config settings.

Eek. Definitely an unintended regression. The fix is below. Thanks for
reporting (and especially for catching during the -rc period!).

You should not need it, but for reference, using 0 is the same as
black (both in old git and new).

-- 8 --
When commit 695d95d refactored the color parsing, it missed
a return 0 when parsing literal numbers 0-8 (which
represent basic ANSI colors), leading us to report these
colors as an error.

Signed-off-by: Jeff King p...@peff.net
---
 color.c  | 1 +
 t/t4026-color.sh | 4 
 2 files changed, 5 insertions(+)

diff --git a/color.c b/color.c
index 809b359..9027352 100644
--- a/color.c
+++ b/color.c
@@ -112,6 +112,7 @@ static int parse_color(struct color *out, const char *name, 
int len)
} else if (val  8) {
out-type = COLOR_ANSI;
out-value = val;
+   return 0;
} else if (val  256) {
out-type = COLOR_256;
out-value = val;
diff --git a/t/t4026-color.sh b/t/t4026-color.sh
index 267c43b..4d20fea 100755
--- a/t/t4026-color.sh
+++ b/t/t4026-color.sh
@@ -60,6 +60,10 @@ test_expect_success 'absurdly long color specification' '
  [1;2;4;5;7;22;24;25;27;38;2;255;255;255;48;2;255;255;255m
 '
 
+test_expect_success '0-7 are aliases for basic ANSI color names' '
+   color 0 7 [30;47m
+'
+
 test_expect_success '256 colors' '
color 254 bold 255 [1;38;5;254;48;5;255m
 '
-- 
2.2.1.425.g441bb3c

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] show-branch: fix indentation of usage string

2015-01-20 Thread Junio C Hamano
Ralf Thielow ralf.thie...@gmail.com writes:

 Noticed-by: Jean-Noël Avila jn.av...@free.fr
 Signed-off-by: Ralf Thielow ralf.thie...@gmail.com
 ---
 Jiang Xin worldhello@gmail.com wrote:
 2015-01-18 23:53 GMT+08:00 Jean-Noël AVILA jn.av...@free.fr:
 Yes, it's wrong to using mixed tabs and spaces in the message. It comes
 from commit v2.0.5-5-g9990273, and it should be fixed.

 This also breaks the indentation of the command output.

  builtin/show-branch.c | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)

 diff --git a/builtin/show-branch.c b/builtin/show-branch.c
 index 691eeda..365228a 100644
 --- a/builtin/show-branch.c
 +++ b/builtin/show-branch.c
 @@ -7,9 +7,9 @@
  
  static const char* show_branch_usage[] = {
  N_(git show-branch [-a|--all] [-r|--remotes] [--topo-order | 
 --date-order]\n
 -   [--current] [--color[=when] | --no-color] 
 [--sparse]\n
 -   [--more=n | --list | --independent | 
 --merge-base]\n
 -   [--no-name | --sha1-name] [--topics] [(rev | 
 glob)...]),
 +[--current] [--color[=when] | --no-color] 
 [--sparse]\n
 +[--more=n | --list | --independent | --merge-base]\n
 +[--no-name | --sha1-name] [--topics] [(rev | 
 glob)...]),
  N_(git show-branch (-g|--reflog)[=n[,base]] [--list] [ref]),
  NULL
  };

Thanks.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Jeff King
On Tue, Jan 20, 2015 at 09:45:46PM +0100, Ævar Arnfjörð Bjarmason wrote:

 What's happened here is that:
 
  1. You've authored your commit in ISO-8859-1
  2. Git itself has no place for the encoding of the author name in the
 commit object format

Is (2) right? The encoding header in a commit object should apply not
just to the commit message, but also to the author (and committer) name.

I think the real problem is simply that it defaults to UTF-8, but he is
giving it iso-8859-1 characters. Setting i18n.commitEncoding should fix
it.

-Peff

PS If you try experimenting with this, you may fall afoul of 08a94a1
   (commit/commit-tree: correct latin1 to utf-8, 2012-06-28), which will
   silently correct Latin1 characters into UTF-8 (when the commit
   message is expected to be in UTF-8, of course). So it actually
   _should_ just work under modern gits, but only for Latin1.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Nico Williams
On Tue, Jan 20, 2015 at 10:38:40PM +0100, Noralf Trønnes wrote:
 Yes:
 $ echo Noralf Trønnes | xxd
 000: 4e6f 7261 6c66 2054 72f8 6e6e 6573 0aNoralf Tr.nnes.
 
 Is there a command I can run that shows that I'm using ISO-8859-1 ?
 I need something to google with, my previous search only gave locale
 stuff, which seems fine.

The locale(1) command tells you what your locale is set to, but it
doesn't say anything about your input method -- it only tells you what
your shell and commands started from it expect for input and what they
should produce for output.

The input method will generally be part of your windowing environment,
for which you'll have to search how to check/configure your OS
(sometimes it can be set on a per-window basis, sometimes it's a global
setting).

Even if the windowing environment is set to UTF-8, your terminal
emulator might be set to ISO-8859-something, so check the terminal
emulator (e.g., rxvt, Terminator, GNOME Terminal, PuTTY, ...).

Finally, check what stty(1) says (e.g., on Linux it should show that
iutf8 is enabled) (this is mostly so that when you backspace in cooked
mode the line discipline knows how many bytes to drop from the buffer).

Nico
-- 
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Git messes up 'ø' character

2015-01-20 Thread Noralf Trønnes

Den 20.01.2015 21:45, skrev Ævar Arnfjörð Bjarmason:

On Tue, Jan 20, 2015 at 9:17 PM, Noralf Trønnes no...@tronnes.org wrote:

Den 20.01.2015 21:07, skrev Torsten Bögershausen:

On 2015-01-20 20.46, Noralf Trønnes wrote:
could it be that your ø is not encoded as UTF-8,
but in ISO-8859-15 (or so)


$ git log -1
commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
Author: Noralf TrF8nnes no...@tronnes.org

What does
git config -l | grep Noralf | xxd
say ?


$ git config -l | grep Noralf | xxd
000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
010: 2054 72f8 6e6e 6573 0aTr.nnes.

$ file ~/.gitconfig
/home/pi/.gitconfig: ISO-8859 text

What's happened here is that:

  1. You've authored your commit in ISO-8859-1
  2. Git itself has no place for the encoding of the author name in the
commit object format
  3. git-format-patch has a --compose-encoding which I think would sort
this out if you set it to ISO-8859-1, but it defaults to UTF-8
  4. Your patch is actually a ISO-8859-1 byte sequence, but is
advertised as UTF-8
  5. You end up with a screwed-up commit

You could work around this, but I suggest just joining the 21st
century and working exclusively in UTF-8, it makes things much easier,
speaking as someone with 3x more non-ASCII characters their his name
than you :)



Ok, then the question is: How do I switch to UTF-8?

To me it seems I'm already using it:
$ locale charmap
UTF-8

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [ANNOUNCE] Git v2.3.0-rc0

2015-01-20 Thread Ævar Arnfjörð Bjarmason
On Tue, Jan 13, 2015 at 12:57 AM, Junio C Hamano gits...@pobox.com wrote:
 An early preview release Git v2.3.0-rc0 is now available for
 testing at the usual places.
[...]
 Jeff King (38):
[...]
   parse_color: refactor color storage
[...]

I've had this in my .gitconfig since 2010 which was broken by Jeff's
v2.1.3-24-g695d95d:

;; Don't be so invasive about coloring ^M when I'm editing files
that
;; are supposed to have \r\n.
[color diff]
   whitespace = 0

To test this replace \n with \r\n in a file. Before this patch you could do:

git -c color.diff.whitespace=0 show

And just get:

[red]-[/red]
[green]+[/green]

As opposed to:

git -c color.diff.whitespace=1 show

Which gives you:

[red]-
[green]+[/green][red]^M[/red]

Now that just produces:

error: invalid color value: 0
fatal: bad config variable 'color.diff.whitespace' in file
'/home/avar/.gitconfig' at line 16

Maybe breaking this is OK (but I can't find what the replacement is),
but the config or the the changelog doesn't mention breaking existing
config settings.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Pretty format specifier for commit count?

2015-01-20 Thread Jeff King
On Mon, Jan 19, 2015 at 05:17:25PM -0800, Josh Triplett wrote:

  Can you be a bit more specific about the type count that you are after?
  git describe counts commits since the most recent tag (possibly within
  a specific subset of all tags). Is that your desired format?
 
 That might work, since the repository in question has no tags; I'd
 actually like commits since root commit.

That's basically a generation number. But I'm not sure if that's really
what you want; in a non-linear history it's not unique (two children of
commit X are both X+1). It sounds like you really just want commits
counting up from the root, and with side branches to have their own
unique numbers. So something like:

   C
  /
  A--B--D

  A=1
  B=2
  C=3
  D=4

except the last two are assigned arbitrarily. You need some rules for
linearizing the commits.

Git's default output order is deterministic when walking backwards
through history from a specific set of starting points. We keep a queue
of commits to visit, sorted by timestamp, with ties in timestamps broken
by whichever was added first (so two parents of a merge get the first
parent added first, then the second). E.g. (and remember we're walking
backwards from the tip here, but you could do the backwards walk and
then reverse it, and start numbering from the other end):

   C--E
  /\
  A--B--D---F

If we start at F, we might visit F, E, D, C, B, A. Or maybe C before D,
but only if its commit timestamp is newer (and if they tie, we
definitely visit D first, because it will have been queued first).

But that's not deterministic as you add more starting points (either new
ref tips, or just new merges we have to cross). For example, imagine
this:

 G--H
/\
   C--E   \
  /\   \
  A--B--D---F---I

If we start at I, then we might visit H and G first, meaning we learn
about C much earlier than we otherwise would. Then we hit F, and get to
C from there. But now it it may be in a different position with respect
to D!

I suspect your problem statement may simply assume a linear history,
which makes this all much simpler. But we are not likely to add a
feature to git that will break badly once you have a non-linear history. :)

I think in the linear case that a generation number _would_ be correct,
and it is a useful concept by itself. So that may be the best thing to
add.

-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3] remote-curl: fall back to Basic auth if Negotiate fails

2015-01-20 Thread Junio C Hamano
Dan Langille (dalangil) dalan...@cisco.com writes:

 I did not test this patch.  Is that holding up a commit?

I am hoping that you rebuilt the Git you use with this patch by the
time you wrote the message I am responding to and have been using it
for your daily Git needs ;-)

I believe it is queued on the 'next' branch so that others like you
who need the change can verify the improvements, and others unlike
you who do not need the change can make sure the change does not
cause unintended consequences.

Thanks.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] parse_color: fix return value for numeric color values 0-8

2015-01-20 Thread Junio C Hamano
Jeff King p...@peff.net writes:

 Thanks; somebody should have caught this before we applied and
 merged to 'master', but the process obviously did not work well.

 I am not too surprised. The use of numeric values for colors was
 completely undocumented, and we did not have any test coverage for it. I
 did not even know it existed until I started refactoring the function,
 and wondered what was going on (though I did try to preserve it once I
 found it).

I was specifically thinking about the failure of eyeball test.
I often do git am followed by git show -U20 to check the change
in context, which _should_ have caught the lack of return there.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] update-ref: Handle large transactions properly

2015-01-20 Thread Stefan Beller
Test if we can do arbitrary large transactions. Currently this is a known
bug that we cannot do large transactions, so document it at least in the
test suite.

Signed-off-by: Stefan Beller sbel...@google.com
---
 t/t1400-update-ref.sh | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 6805b9e..a7dd1ff 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -1065,4 +1065,31 @@ test_expect_success 'stdin -z delete refs works with 
packed and loose refs' '
test_must_fail git rev-parse --verify -q $c
 '
 
+run_with_limited_open_files () {
+   (ulimit -n 64  $@)
+}
+
+test_lazy_prereq ULIMIT 'run_with_limited_open_files true'
+
+test_expect_failure ULIMIT 'large transaction creating branches does not burst 
open file limit' '
+(
+   for i in $(seq 65)
+   echo create refs/heads/$i HEAD
+   done large_input 
+   git update-ref --stdin large_input 
+   git rev-parse --verify -q refs/heads/65
+)
+'
+
+test_expect_failure ULIMIT 'large transaction deleting branches does not burst 
open file limit' '
+(
+   for i in $(seq 33)
+   do
+   echo delete refs/heads/$i HEAD
+   done large_input 
+   git update-ref --stdin large_input 
+   test_must_fail git rev-parse --verify -q refs/heads/65
+)
+'
+
 test_done
-- 
2.2.1.62.g3f15098

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] update-ref: Handle large transactions properly

2015-01-20 Thread Stefan Beller
On Tue, Jan 20, 2015 at 5:20 PM, Stefan Beller sbel...@google.com wrote:
 Test if we can do arbitrary large transactions. Currently this is a known
 bug that we cannot do large transactions, so document it at least in the
 test suite.

 Signed-off-by: Stefan Beller sbel...@google.com
 ---
  t/t1400-update-ref.sh | 27 +++
  1 file changed, 27 insertions(+)

 diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
 index 6805b9e..a7dd1ff 100755
 --- a/t/t1400-update-ref.sh
 +++ b/t/t1400-update-ref.sh
 @@ -1065,4 +1065,31 @@ test_expect_success 'stdin -z delete refs works with 
 packed and loose refs' '
 test_must_fail git rev-parse --verify -q $c
  '

 +run_with_limited_open_files () {
 +   (ulimit -n 64  $@)
 +}
 +
 +test_lazy_prereq ULIMIT 'run_with_limited_open_files true'
 +
 +test_expect_failure ULIMIT 'large transaction creating branches does not 
 burst open file limit' '
 +(
 +   for i in $(seq 65)
 +   echo create refs/heads/$i HEAD
 +   done large_input 
 +   git update-ref --stdin large_input 
 +   git rev-parse --verify -q refs/heads/65
 +)
 +'
 +
 +test_expect_failure ULIMIT 'large transaction deleting branches does not 
 burst open file limit' '
 +(
 +   for i in $(seq 33)

This should be 65 of course. :/

 +   do
 +   echo delete refs/heads/$i HEAD
 +   done large_input 
 +   git update-ref --stdin large_input 
 +   test_must_fail git rev-parse --verify -q refs/heads/65
 +)
 +'
 +
  test_done
 --
 2.2.1.62.g3f15098

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[ANNOUNCE] Git v2.3.0-rc1

2015-01-20 Thread Junio C Hamano
A release candidate Git v2.3.0-rc1 is now available for testing
at the usual places.

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.3.0-rc1'
tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://code.google.com/p/git-core/
  url = git://git.sourceforge.jp/gitroot/git-core/git.git
  url = git://git-core.git.sourceforge.net/gitroot/git-core/git-core
  url = https://github.com/gitster/git

Git v2.3 Release Notes (draft)
==

Updates since v2.2
--

Ports

 * Recent gcc toolchain on Cygwin started throwing compilation warning,
   which has been squelched.

 * A few updates to build on platforms that lack tv_nsec,
   clock_gettime, CLOCK_MONOTONIC and HMAC_CTX_cleanup (e.g. older
   RHEL) have been added.


UI, Workflows  Features

 * It was cumbersome to use GIT_SSH mechanism when the user wanted
   to pass an extra set of arguments to the underlying ssh.  A new
   environment variable GIT_SSH_COMMAND can be used for this.

 * A request to store an empty note via git notes meant to remove
   note from the object but with --allow-empty we will store a
   (surprise!)  note that is empty.

 * git interpret-trailers learned to properly handle the
   Conflicts: block at the end.

 * git am learned --message-id option to copy the message ID of
   the incoming e-mail to the log message of resulting commit.

 * git clone --reference=over there learned the --dissociate
   option to go with it; it borrows objects from the reference object
   store while cloning only to reduce network traffic and then
   dissociates the resulting clone from the reference by performing
   local copies of borrowed objects.

 * git send-email learned --transfer-encoding option to force a
   non-fault Content-Transfer-Encoding header (e.g. base64).

 * git send-email normally identifies itself via X-Mailer: header in
   the message it sends out.  A new command line flag --no-xmailer
   allows the user to squelch the header.

 * git push into a repository with a working tree normally refuses
   to modify the branch that is checked out.  The command learned to
   optionally do an equivalent of git reset --hard only when there
   is no change to the working tree and the index instead, which would
   be useful to deploy by pushing into a repository.

 * git new-workdir (in contrib/) can be used to populate an empty
   and existing directory now.

 * Credential helpers are asked in turn until one of them give
   positive response, which is cumbersome to turn off when you need to
   run Git in an automated setting.  The credential helper interface
   learned to allow a helper to say stop, don't ask other helpers.
   Also GIT_TERMINAL_PROMPT environment can be set to false to disable
   our built-in prompt mechanism for passwords.

 * git branch -d (delete) and git branch -m (move) learned to
   honor -f (force) flag; unlike many other subcommands, the way to
   force these have been with separate -D/-M options, which was
   inconsistent.

 * diff-highlight filter (in contrib/) allows its color output to be
   customized via configuration variables.

 * git imap-send learned to take -v (verbose) and -q (quiet)
   command line options.

 * git remote add $name $URL is now allowed when url.$URL.insteadOf
   is already defined.

 * git imap-send now can be built to use cURL library to talk to
   IMAP servers (if the library is recent enough, of course).
   This allows you to use authenticate method other than CRAM-MD5,
   among other things.

 * git imap-send now allows GIT_CURL_VERBOSE environment variable to
   control the verbosity when talking via the cURL library.

 * The prompt script (in contrib/) learned to optionally hide prompt
   when in an ignored directory by setting GIT_PS1_HIDE_IF_PWD_IGNORED
   shell variable.


Performance, Internal Implementation, Development Support etc.

 * Earlier we made rev-list --object-edge more aggressively list the
   objects at the edge commits, in order to reduce number of objects 
   fetched into a shallow repository, but the change affected cases
   other than fetching into a shallow repository and made it
   unusably slow (e.g. fetching into a normal repository should not
   have to suffer the overhead from extra processing).  Limit it to a
   more specific case by introducing --objects-edge-aggressive, a new
   option to rev-list.

 * Squelched useless compiler warnings on Mac OS X regarding the
   crypto API.

 * The procedure to generate unicode table has been simplified.

 * Some filesystems assign filemodes in a strange way, fooling then
   automatic filemode trustability check done during a new
   repository creation.  The initialization codepath has been hardened
   against this issue.

 * The codepath in git remote update --prune to drop many refs 

Re: Git messes up 'ø' character

2015-01-20 Thread Noralf Trønnes

Den 20.01.2015 23:18, skrev Nico Williams:

On Tue, Jan 20, 2015 at 10:38:40PM +0100, Noralf Trønnes wrote:

Yes:
$ echo Noralf Trønnes | xxd
000: 4e6f 7261 6c66 2054 72f8 6e6e 6573 0aNoralf Tr.nnes.

Is there a command I can run that shows that I'm using ISO-8859-1 ?
I need something to google with, my previous search only gave locale
stuff, which seems fine.

The locale(1) command tells you what your locale is set to, but it
doesn't say anything about your input method -- it only tells you what
your shell and commands started from it expect for input and what they
should produce for output.

The input method will generally be part of your windowing environment,
for which you'll have to search how to check/configure your OS
(sometimes it can be set on a per-window basis, sometimes it's a global
setting).

Even if the windowing environment is set to UTF-8, your terminal
emulator might be set to ISO-8859-something, so check the terminal
emulator (e.g., rxvt, Terminator, GNOME Terminal, PuTTY, ...).


I use putty which was set to ISO-8859-1. Changing this to UTF-8 gave me 
the correct result:

$ echo Noralf Trønnes | xxd
000: 4e6f 7261 6c66 2054 72c3 b86e 6e65 730a  Noralf Tr..nnes.

Thank you all for helping me!

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] l10n: correct indentation of show-branch usage

2015-01-20 Thread Jiang Xin
An indentation error was found right after we started l10n round 2, and
commit d6589d1 (show-branch: fix indentation of usage string) and this
update would fix it.

Signed-off-by: Jiang Xin worldhello@gmail.com
---
This patch is based on master branch of git://github.com/git-l10n/git-po

 po/de.po| 18 +-
 po/fr.po| 18 +-
 po/git.pot  | 10 +-
 po/sv.po| 38 +++---
 po/vi.po| 18 +-
 po/zh_CN.po | 24 +++-
 6 files changed, 62 insertions(+), 64 deletions(-)

diff --git a/po/de.po b/po/de.po
index 0b93b0f..b2d4639 100644
--- a/po/de.po
+++ b/po/de.po
@@ -7,8 +7,8 @@ msgid 
 msgstr 
 Project-Id-Version: Git\n
 Report-Msgid-Bugs-To: Git Mailing List git@vger.kernel.org\n
-POT-Creation-Date: 2015-01-18 11:24+0800\n
-PO-Revision-Date: 2014-11-20 10:19+0800\n
+POT-Creation-Date: 2015-01-21 14:21+0800\n
+PO-Revision-Date: 2015-01-21 15:01+0800\n
 Last-Translator: Ralf Thielow ralf.thie...@gmail.com\n
 Language-Team: German \n
 Language: de\n
@@ -326,7 +326,7 @@ msgstr kann '%s' nicht erstellen
 msgid index-pack died
 msgstr Erstellung der Paketindexdatei abgebrochen
 
-#: color.c:259
+#: color.c:260
 #, c-format
 msgid invalid color value: %.*s
 msgstr Ungültiger Farbwert: %.*s
@@ -9679,14 +9679,14 @@ msgstr Ausgabe mit Zeilenumbrüchen
 #: builtin/show-branch.c:9
 msgid 
 git show-branch [-a|--all] [-r|--remotes] [--topo-order | --date-order]\n
-\t\t   [--current] [--color[=when] | --no-color] [--sparse]\n
-\t\t   [--more=n | --list | --independent | --merge-base]\n
-\t   [--no-name | --sha1-name] [--topics] [(rev | glob)...]
+\t\t[--current] [--color[=when] | --no-color] [--sparse]\n
+\t\t[--more=n | --list | --independent | --merge-base]\n
+\t\t[--no-name | --sha1-name] [--topics] [(rev | glob)...]
 msgstr 
 git show-branch [-a|--all] [-r|--remotes] [--topo-order | --date-order]\n
-\t\t   [--current] [--color[=Wann] | --no-color] [--sparse]\n
-\t\t   [--more=n | --list | --independent | --merge-base]\n
-\t   [--no-name | --sha1-name] [--topics] [(Commit | glob)...]
+\t\t[--current] [--color[=Wann] | --no-color] [--sparse]\n
+\t\t[--more=n | --list | --independent | --merge-base]\n
+\t\t[--no-name | --sha1-name] [--topics] [(Commit | glob)...]
 
 #: builtin/show-branch.c:13
 msgid git show-branch (-g|--reflog)[=n[,base]] [--list] [ref]
diff --git a/po/fr.po b/po/fr.po
index d1b3397..3235879 100644
--- a/po/fr.po
+++ b/po/fr.po
@@ -73,8 +73,8 @@ msgid 
 msgstr 
 Project-Id-Version: git\n
 Report-Msgid-Bugs-To: Git Mailing List git@vger.kernel.org\n
-POT-Creation-Date: 2015-01-18 11:24+0800\n
-PO-Revision-Date: 2015-01-18 17:01+0100\n
+POT-Creation-Date: 2015-01-21 14:21+0800\n
+PO-Revision-Date: 2015-01-21 14:57+0800\n
 Last-Translator: Jean-Noël Avila jn.av...@free.fr\n
 Language-Team: Jean-Noël Avila jn.av...@free.fr\n
 Language: fr\n
@@ -394,7 +394,7 @@ msgstr impossible de créer '%s'
 msgid index-pack died
 msgstr l'index de groupe a disparu
 
-#: color.c:259
+#: color.c:260
 #, c-format
 msgid invalid color value: %.*s
 msgstr Valeur invalide de couleur : %.*s
@@ -9675,14 +9675,14 @@ msgstr Couper les lignes
 #: builtin/show-branch.c:9
 msgid 
 git show-branch [-a|--all] [-r|--remotes] [--topo-order | --date-order]\n
-\t\t   [--current] [--color[=when] | --no-color] [--sparse]\n
-\t\t   [--more=n | --list | --independent | --merge-base]\n
-\t   [--no-name | --sha1-name] [--topics] [(rev | glob)...]
+\t\t[--current] [--color[=when] | --no-color] [--sparse]\n
+\t\t[--more=n | --list | --independent | --merge-base]\n
+\t\t[--no-name | --sha1-name] [--topics] [(rev | glob)...]
 msgstr 
 git show-branch [-a|--all] [-r|--remotes] [--topo-order | --date-order]\n
-[--current] [--color[=quand] | --no-color] [--sparse]\n
-[--more=n | --list | --independent | --merge-base]\n
-[--no-name | --sha1-name] [--topics] [(rév | glob)...]
+\t\t[--current] [--color[=quand] | --no-color] [--sparse]\n
+\t\t[--more=n | --list | --independent | --merge-base]\n
+\t\t[--no-name | --sha1-name] [--topics] [(rév | glob)...]
 
 #: builtin/show-branch.c:13
 msgid git show-branch (-g|--reflog)[=n[,base]] [--list] [ref]
diff --git a/po/git.pot b/po/git.pot
index 923d617..91fa5a1 100644
--- a/po/git.pot
+++ b/po/git.pot
@@ -8,7 +8,7 @@ msgid 
 msgstr 
 Project-Id-Version: PACKAGE VERSION\n
 Report-Msgid-Bugs-To: Git Mailing List git@vger.kernel.org\n
-POT-Creation-Date: 2015-01-18 11:24+0800\n
+POT-Creation-Date: 2015-01-21 14:21+0800\n
 PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n
 Last-Translator: FULL NAME EMAIL@ADDRESS\n
 Language-Team: LANGUAGE l...@li.org\n
@@ -305,7 +305,7 @@ msgstr 
 msgid index-pack died
 msgstr 
 
-#: color.c:259
+#: color.c:260
 #, c-format
 msgid invalid color value: %.*s
 msgstr 
@@ -9015,9 +9015,9 @@ msgstr 
 #: builtin/show-branch.c:9
 msgid 
 git show-branch [-a|--all] [-r|--remotes] [--topo-order | 

Re: Git messes up 'ø' character

2015-01-20 Thread Ævar Arnfjörð Bjarmason
On Tue, Jan 20, 2015 at 9:17 PM, Noralf Trønnes no...@tronnes.org wrote:
 Den 20.01.2015 21:07, skrev Torsten Bögershausen:

 On 2015-01-20 20.46, Noralf Trønnes wrote:
 could it be that your ø is not encoded as UTF-8,
 but in ISO-8859-15 (or so)

 $ git log -1
 commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
 Author: Noralf TrF8nnes no...@tronnes.org

 What does
 git config -l | grep Noralf | xxd
 say ?

 $ git config -l | grep Noralf | xxd
 000: 7573 6572 2e6e 616d 653d 4e6f 7261 6c66  user.name=Noralf
 010: 2054 72f8 6e6e 6573 0aTr.nnes.

 $ file ~/.gitconfig
 /home/pi/.gitconfig: ISO-8859 text

What's happened here is that:

 1. You've authored your commit in ISO-8859-1
 2. Git itself has no place for the encoding of the author name in the
commit object format
 3. git-format-patch has a --compose-encoding which I think would sort
this out if you set it to ISO-8859-1, but it defaults to UTF-8
 4. Your patch is actually a ISO-8859-1 byte sequence, but is
advertised as UTF-8
 5. You end up with a screwed-up commit

You could work around this, but I suggest just joining the 21st
century and working exclusively in UTF-8, it makes things much easier,
speaking as someone with 3x more non-ASCII characters their his name
than you :)
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


bash completion for git branch/checkout/etc doesn't escape metacharacters

2015-01-20 Thread Kevin Stenerson
The bash completion provided with git doesn't escape or quote parens
in tag or branch names.
Alternatively, it doesn't reject branch or tag names with parenthesis
as an invalid name (as it does for names with spaces or tabs)
Bash treats parentheses in unquoted words as a word separator.

When encountering this problem I was using git version 2.2.2 and bash
version 4.3.11

Steps to reproduce with bash:

mkdir foo
cd foo
git init foo
touch bar
git add bar
git commit -m Went to a bar

Then for tags

git tag 'v1.0.0(215)'

Or branches

git branch 'mybranch(myuser)'

And the actual bash completion

git checkout   # Use completion here -- it won't escape the parenthesis
git checkout master
git branch -d   # Use completion here -- it won't escape the parenthesis

Similarily, when typing a name with an escape character such as
mytag\(, it won't complete the name from that point.

This appears to be true for other bash meta characters as well such as
'', '', '|', ';', ''

---
Kevin Stenerson
ke...@reflexionhealth.com
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] move MAXDEPTH definition to the cache.h

2015-01-20 Thread Torsten Bögershausen
 --- a/cache.h
 +++ b/cache.h
 @@ -1010,6 +1010,7 @@ extern int read_ref(const char *refname, unsigned char 
 *sha1);
   * Caps and underscores refers to the special refs, such as HEAD,
   * FETCH_HEAD and friends, that all live outside of the refs/ directory.
   */

What happened to the comment line ?
Should it go away or better stay ?

When we move the definition to a common file,
it is not 100% clear what MAXDEPTH is about,
and the comment is even more important, I think:

 +/* We allow recursive symbolic links. Only within reason, though. */
 +#define MAXDEPTH 5

On the other hand, if we find a better name for that definition,
we may skip the comment.

A first suggestion may be
#define MAXDEPTH_FOR_SYMLINKS 5
but other variants are possible.


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/24] nd/untracked-cache update

2015-01-20 Thread Torsten Bögershausen
On 2015-01-20 14.03, Nguyễn Thái Ngọc Duy wrote:
 Sorry for this really late update. This fixes bugs in extension
 writing code (10/24), support using the same cache from different
 hosts (23/24), and adds a new bug to point the user to untracked cache from
 'git status -uno' (new patch 24/24)
 
 Diff from 'pu'
 
Thanks for working on this, as I think it is a major improvement.

Do you have a commit on a public repo ?
pu + your serious, or master + V3 + this delta ?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] show-branch: fix indentation of usage string

2015-01-20 Thread Ralf Thielow
Noticed-by: Jean-Noël Avila jn.av...@free.fr
Signed-off-by: Ralf Thielow ralf.thie...@gmail.com
---
Jiang Xin worldhello@gmail.com wrote:
 2015-01-18 23:53 GMT+08:00 Jean-Noël AVILA jn.av...@free.fr:
 Yes, it's wrong to using mixed tabs and spaces in the message. It comes
 from commit v2.0.5-5-g9990273, and it should be fixed.

This also breaks the indentation of the command output.

 builtin/show-branch.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index 691eeda..365228a 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -7,9 +7,9 @@
 
 static const char* show_branch_usage[] = {
 N_(git show-branch [-a|--all] [-r|--remotes] [--topo-order | 
--date-order]\n
- [--current] [--color[=when] | --no-color] 
[--sparse]\n
- [--more=n | --list | --independent | 
--merge-base]\n
- [--no-name | --sha1-name] [--topics] [(rev | 
glob)...]),
+  [--current] [--color[=when] | --no-color] 
[--sparse]\n
+  [--more=n | --list | --independent | --merge-base]\n
+  [--no-name | --sha1-name] [--topics] [(rev | 
glob)...]),
 N_(git show-branch (-g|--reflog)[=n[,base]] [--list] [ref]),
 NULL
 };
-- 
2.3.0.rc0.211.g05e7197

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Git messes up 'ø' character

2015-01-20 Thread Noralf Trønnes

I can't get my name: Noralf Trønnes, to come out correctly when I
format and send a patch. The 'ø' becomes a question mark when received
in my email client.

This is the head of the patch file generated by git format-patch:

From b2a4f6abdb097c4dc092b56995a2af8e42fbea79 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Noralf=20Tr=F8nnes?= no...@tronnes.org
Date: Tue, 20 Jan 2015 18:34:47 +0100
Subject: [PATCH] staging: fbtft: remove ARCH_BCM2708 optimization
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

ARCH_BCM2708 is not present in mainline so remove optimization.

Signed-off-by: Noralf Trønnes no...@tronnes.org
---

$ git log -1
commit b2a4f6abdb097c4dc092b56995a2af8e42fbea79
Author: Noralf TrF8nnes no...@tronnes.org
Date:   Tue Jan 20 18:34:47 2015 +0100

staging: fbtft: remove ARCH_BCM2708 optimization

ARCH_BCM2708 is not present in mainline so remove optimization.

Signed-off-by: Noralf TrF8nnes no...@tronnes.org

$ git send-email --to no...@tronnes.org 
0001-staging-fbtft-remove-ARCH_BCM2708-optimization.patch

0001-staging-fbtft-remove-ARCH_BCM2708-optimization.patch
(mbox) Adding cc: =?UTF-8?q?Noralf=20Tr=F8nnes?= no...@tronnes.org 
from line 'From: =?UTF-8?q?Noralf=20Tr=F8nnes?= no...@tronnes.org'
(body) Adding cc: Noralf Trønnes no...@tronnes.org from line 
'Signed-off-by: Noralf Trønnes no...@tronnes.org'


From: =?UTF-8?q?Noralf=20Tr=F8nnes?= no...@tronnes.org
To: no...@tronnes.org
Subject: [PATCH] staging: fbtft: remove ARCH_BCM2708 optimization
Date: Tue, 20 Jan 2015 20:25:24 +0100
Message-Id: 1421781924-3066-1-git-send-email-no...@tronnes.org
X-Mailer: git-send-email 2.2.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The Cc list above has been expanded by additional
addresses found in the patch commit message. By default
send-email prompts before sending whenever this occurs.
This behavior is controlled by the sendemail.confirm
configuration setting.

For additional information, run 'git send-email --help'.
To retain the current behavior, but squelch this message,
run 'git config --global sendemail.confirm auto'.

Send this email? ([y]es|[n]o|[q]uit|[a]ll): y
OK. Log says:
Server: smtp.ebnett.no
MAIL FROM:no...@tronnes.org
RCPT TO:no...@tronnes.org
From: =?UTF-8?q?Noralf=20Tr=F8nnes?= no...@tronnes.org
To: no...@tronnes.org
Subject: [PATCH] staging: fbtft: remove ARCH_BCM2708 optimization
Date: Tue, 20 Jan 2015 20:44:15 +0100
Message-Id: 1421783055-3117-1-git-send-email-no...@tronnes.org
X-Mailer: git-send-email 2.2.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Result: 250 2.0.0 Ok: queued as B49BB8014D



Setup:

Ubuntu server
$ cat /etc/issue
Ubuntu 12.04.3 LTS \n \l

$ git --version
git version 2.2.2

$ git config -l
user.name=Noralf Trønnes
user.email=no...@tronnes.org
core.editor=nano
alias.serve=daemon --verbose --export-all --base-path=/home/pi --reuseaddr
sendemail.smtpserver=smtp.ebnett.no
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
remote.origin.url=git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git
branch.master.remote=origin
branch.master.merge=refs/heads/master
branch.staging-testing.remote=origin
branch.staging-testing.merge=refs/heads/staging-testing


Regards,
Noralf Trønnes

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html