Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Eric Sunshine]

On Sun, Nov 1, 2015 at 1:32 AM,   wrote:
> Some implementations of SHA_Updates have inherent limits
> on the max chunk size. SHA1_MAX_BLOCK_SIZE can be defined
> to set the max chunk size supported, if required.  This is
> enabled for OSX CommonCrypto library and set to 1GiB.
>
> Signed-off-by: Atousa Pahlevan Duprat 
> ---
> diff --git a/compat/sha1_chunked.c b/compat/sha1_chunked.c
> new file mode 100644
> index 000..bf62b1b
> --- /dev/null
> +++ b/compat/sha1_chunked.c
> @@ -0,0 +1,21 @@
> +#include "cache.h"
> +
> +#ifdef SHA1_MAX_BLOCK_SIZE

This file is only compiled when SHA1_MAX_BLOCK_SIZE is defined, so
does this #ifdef serve a purpose?

> +int git_SHA1_Update(SHA_CTX *c, const void *data, size_t len)
> +{
> +   size_t nr;
> +   size_t total = 0;
> +   char *cdata = (char*)data;

Nit: This could be 'const char *'.

> +   while (len > 0) {
> +   nr = len;
> +   if (nr > SHA1_MAX_BLOCK_SIZE)
> +   nr = SHA1_MAX_BLOCK_SIZE;
> +   SHA1_Update(c, cdata, nr);
> +   total += nr;
> +   cdata += nr;
> +   len -= nr;
> +   }
> +   return total;
> +}
> +#endif
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 18/26] refs: move transaction functions into common code

[Michael Haggerty]

On 10/28/2015 03:14 AM, David Turner wrote:
> The common ref code will build up a ref transaction.  Backends will
> then commit it.  So the transaction creation and update functions should
> be in the common code.  We also need to move the ref structs into
> the common code so that alternate backends can access them.
> 
> Later, we will modify struct ref_update to support alternate backends.

I would prefer that this and later patches *not* add declarations to the
public API in refs.h for functions and data that are only meant to be
used by other reference backends.

So I'm working on a modified version of your series that declares such
functions in refs-internal.h [1] instead. I hope to submit it tomorrow.

Actually, I have half a mind to move all of the refs-related files to a
subdirectory, like

refs.h
refs/refs.c
refs/refs-internal.h
refs/refs-be-files.c
refs/refs-be-lmdb.c   <- still to come

What would you think of that?

Michael

[1] We've discussed this idea earlier, using the tentative names
refs-shared.h or refs-common.h.

-- 
Michael Haggerty
mhag...@alum.mit.edu

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Limit the size of the data block passed to SHA1_Update()

[atousa . p]

From: Atousa Pahlevan Duprat 

Some implementations of SHA_Updates have inherent limits
on the max chunk size. SHA1_MAX_BLOCK_SIZE can be defined
to set the max chunk size supported, if required.  This is
enabled for OSX CommonCrypto library and set to 1GiB.

Signed-off-by: Atousa Pahlevan Duprat 
---
 Makefile |  9 +
 cache.h  |  7 ++-
 compat/apple-common-crypto.h |  4 
 compat/sha1_chunked.c| 21 +
 4 files changed, 40 insertions(+), 1 deletion(-)
 create mode 100644 compat/sha1_chunked.c

diff --git a/Makefile b/Makefile
index 04c2231..5955542 100644
--- a/Makefile
+++ b/Makefile
@@ -141,6 +141,10 @@ all::
 # Define PPC_SHA1 environment variable when running make to make use of
 # a bundled SHA1 routine optimized for PowerPC.
 #
+# Define SHA1_MAX_BLOCK_SIZE if your SSH1_Update() implementation can
+# hash only a limited amount of data in one call (e.g. APPLE_COMMON_CRYPTO
+# may want 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined).
+#
 # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).
 #
 # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).
@@ -1346,6 +1350,7 @@ else
 ifdef APPLE_COMMON_CRYPTO
COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL
SHA1_HEADER = 
+   SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
 else
SHA1_HEADER = 
EXTLIBS += $(LIB_4_CRYPTO)
@@ -1353,6 +1358,10 @@ endif
 endif
 endif
 
+ifdef SHA1_MAX_BLOCK_SIZE
+   LIB_OBJS += compat/sha1_chunked.o
+   BASIC_CFLAGS += -DSHA1_MAX_BLOCK_SIZE="$(SHA1_MAX_BLOCK_SIZE)"
+endif
 ifdef NO_PERL_MAKEMAKER
export NO_PERL_MAKEMAKER
 endif
diff --git a/cache.h b/cache.h
index 79066e5..ec84b16 100644
--- a/cache.h
+++ b/cache.h
@@ -14,7 +14,12 @@
 #ifndef git_SHA_CTX
 #define git_SHA_CTXSHA_CTX
 #define git_SHA1_Init  SHA1_Init
-#define git_SHA1_UpdateSHA1_Update
+#ifdef SHA1_MAX_BLOCK_SIZE
+extern int SHA1_Update_Chunked(SHA_CTX *, const void *, size_t);
+#define git_SHA1_Update SHA1_Update_Chunked
+#else
+#define git_SHA1_Update SHA1_Update
+#endif
 #define git_SHA1_Final SHA1_Final
 #endif
 
diff --git a/compat/apple-common-crypto.h b/compat/apple-common-crypto.h
index c8b9b0e..d3fb264 100644
--- a/compat/apple-common-crypto.h
+++ b/compat/apple-common-crypto.h
@@ -16,6 +16,10 @@
 #undef TYPE_BOOL
 #endif
 
+#ifndef SHA1_MAX_BLOCK_SIZE
+#error Using Apple Common Crypto library requires setting SHA1_MAX_BLOCK_SIZE
+#endif
+
 #ifdef APPLE_LION_OR_NEWER
 #define git_CC_error_check(pattern, err) \
do { \
diff --git a/compat/sha1_chunked.c b/compat/sha1_chunked.c
new file mode 100644
index 000..bf62b1b
--- /dev/null
+++ b/compat/sha1_chunked.c
@@ -0,0 +1,21 @@
+#include "cache.h"
+
+#ifdef SHA1_MAX_BLOCK_SIZE
+int git_SHA1_Update(SHA_CTX *c, const void *data, size_t len)
+{
+   size_t nr;
+   size_t total = 0;
+   char *cdata = (char*)data;
+
+   while (len > 0) {
+   nr = len;
+   if (nr > SHA1_MAX_BLOCK_SIZE)
+   nr = SHA1_MAX_BLOCK_SIZE;
+   SHA1_Update(c, cdata, nr);
+   total += nr;
+   cdata += nr;
+   len -= nr;
+   }
+   return total;
+}
+#endif
-- 
2.4.9 (Apple Git-60)

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Allow hideRefs to match refs outside the namespace

[Lukas Fleischer]

On Sun, 01 Nov 2015 at 00:40:39, Lukas Fleischer wrote:
> On Sat, 31 Oct 2015 at 18:31:23, Junio C Hamano wrote:
> > [...]
> > You earlier (re)discovered a good approach to introduce a new
> > feature without breaking settings of existing users when we
> > discussed a "whitelist".  Since setting the configuration to an
> > empty string did not do anything in the old code, an empty string
> > was an invalid and non-working setting.  By taking advantage of that
> > fact, you safely can say "if you start with an empty that would
> > match everything, we'll treat all the others differently from the
> > way we did before" if you wanted to.  I think you can follow the
> > same principle here.  For example, I can imagine that the rule for
> > the "ref-is-hidden" can be updated to:
> > 
> >  * it now takes refname and also the fullname before stripping the
> >namespace;
> > 
> >  * hide patterns that is prefixed with '!' means negative, just as
> >before;
> > 
> >  * (after possibly '!' is stripped), hide patterns that is prefixed
> >with '^', which was invalid before, means check the fullname with
> >namespace prefix, which is a new rule;
> > 
> >  * otherwise, check the refname after stripping the namespace.
> > 
> > Such an update would allow a new feature "we now allow you to write
> > a pattern that determines the match before stripping the namespace
> > prefix" without breaking the existing repositories, no?
> > 
> 
> Yes. If I understood you correctly, this is exactly what I suggested in
> the last paragraph of my previous email (the only difference being that
> I suggested to use "/" as full name indicator instead of "^" but that is
> just an implementation detail). I will look into implementing this if
> that is the way we want to go.
> [...]

There are two more things I noticed.

Firstly, while looking for other callers of ref_is_hidden(), I realized
that send_ref() in upload-pack.c contains these lines of code:

const char *refname_nons = strip_namespace(refname);
struct object_id peeled;

if (mark_our_ref(refname, oid)) 
return 0;   

where mark_our_ref() performs the ref_is_hidden() check on its first
parameter. So, in contrast to receive-pack, we already match the
original full reference (and not the stripped one) against the hideRefs
pattern there. In particular, when using transfer.hideRefs, the same
pattern does different things when receiving and uploading.

Now, this cannot be intended behavior and I do not think this is
something we want to retain when improving that feature. My suggestion
is:

1. Define the (current) semantics of hideRefs pattern. It either needs
   to be defined to match full references or stripped references. Both
   definitions are equivalent when Git namespaces are not used.
   
   It probably makes sense to define hideRefs patterns to match stripped
   references. If anybody relied on the upload-pack behavior of patterns
   matching full references, it may happen that more refs are hidden
   when that behavior is adjusted to match the new hideRefs semantics.
   The administrator would become aware of that change soon if it
   affects anything (i.e. hides things that should not be hidden). But I
   am pretty sure that this behavior isn't currently being relied on
   either way. Both Git namespaces and hideRefs aren't very popular
   features and anybody using that combination would probably have
   noticed the inconsistency and reported a bug earlier.

2. Improve the documentation and describe the hideRefs semantics better.
   Include details on the choice we made in (1).

3. Fix the send_ref() code in either receive-pack or upload-pack,
   depending on which is buggy according to our new definition.

4. Improve hideRefs patterns and allow to match both full references and
   stripped references by using a special indicator as suggested
   earlier.

5. Add a note on the change in behavior to the release notes of the
   release that "breaks backwards compatibility". Putting it in quotes
   because I actually think that we are fixing a bug rather than
   breaking compatibility. But since there was no documentation on the
   correct behavior, the former implementation was, technically, the
   only specification of "correct" behavior that existed at that
   point...

The second thing I noticed is that having syntax for allowing matches
against both full references and stripped references is extremely handy
and desirable, even if we would not have to introduce it for backwards
compatibility. For example, using the syntax Junio described earlier, my
initial use case could be solved by

receive.hideRefs=^refs/
receive.hideRefs=!refs/

which means "Hide all references but do not hide references from 

Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Atousa Duprat]

>> +int git_SHA1_Update(SHA_CTX *c, const void *data, size_t len)
>> +{
>> + size_t nr;
>> + size_t total = 0;
>> + char *cdata = (char*)data;

> I am not sure about the cast
> here, though.  Doesn't the function SHA1_Update() you are going to
> call in the body of the loop take "const void *" as its second
> parameter?  That's how openssl/sha1.h and block-sha1/sha1.h declare
> this function.

Indeed, the declaration needs a const void *; but I need to advance by
a specific number of bytes in each iteration of the loop.  Hence the
cast.

Atousa
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] ls-files: Add eol diagnostics

[Sebastian Schuberth]

On 31.10.2015 11:25, Matthieu Moy wrote:

>> ca:text-no-eol   wt:text-no-eol   t/t5100/empty
>> ca:binarywt:binaryt/test-binary-2.png
>> ca:text-lf   wt:text-lf   t/t5100/rfc2047-info-0007
>> ca:text-lf   wt:text-crlf doit.bat
>> ca:text-crlf-lf  wt:text-crlf-lf  locale/XX.po
> 
> I would spell the first "in" or "idx" (for "index"), not "ca" (for
> "cache"). I think we avoid talking about "the cache" these days even
> though the doc sometimes says "cached in the index" (i.e. use "cache" as
> a verb, not a noun).

Good point, I'd prefer "idx" over ca", too.

However, the commit message says "to check if text files are stored normalized 
in the *repository*", yet the output refers to the index / cache. Is there a 
(potential) difference between line endings in the index and repo? AFAIK there 
is not. Any I find it a bit confusing to refer to the index where, as e.g. for 
a freshly cloned repo the index should be empty, yet you do have specific line 
endings in the repo.

Long story short, how about consistently talking about line endings in the 
repo, and also using "repo" instead of "ca" here?

-- 
Sebastian Schuberth

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Atousa Duprat]

> Didn't we have this same issue with NonStop port about a year ago and
> decided to provision this through the configure script?

I'll be happy to look at it.  Can you point me to the right email
thread or location in the configure.ac file?

Atousa
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] stash: complain about unknown flags

[Vincent Legoll]

Hello,

On Wed, May 20, 2015 at 8:01 PM, Jeff King  wrote:
> The option parser for git-stash stuffs unknown flags into
> the $FLAGS variable, where they can be accessed by the
> individual commands. However, most commands do not even look
> at these extra flags, leading to unexpected results like
> this:
>
>   $ git stash drop --help
>   Dropped refs/stash@{0} (e6cf6d80faf92bb7828f7b60c47fc61c03bd30a1)
>
> We should notice the extra flags and bail. Rather than
> annotate each command to reject a non-empty $FLAGS variable,
> we can notice that "stash show" is the only command that
> actually _wants_ arbitrary flags. So we switch the default
> mode to reject unknown flags, and let stash_show() opt into
> the feature.

Better late than never, that does look good...

-- 
Vincent Legoll
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/1] stash: recognize "--help" for subcommands

[Vincent Legoll]

On Wed, May 20, 2015 at 8:17 PM, Jeff King  wrote:
> On Wed, May 20, 2015 at 02:01:32PM -0400, Jeff King wrote:
>
>> This takes away the immediate pain. We may also want to
>> teach "--help" to the option. I guess we cannot do better
>> than just having it run "git help stash" in all cases (i.e.,
>> we have no way to get the help for a specific subcommand).
>
> That actually turns out to be pretty painless...

Looks OK, but this "[PATCH 2/1]" is fishy...

-- 
Vincent Legoll
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHv2 8/8] clone: allow an explicit argument for parallel submodule clones

[Eric Sunshine]

On Wed, Oct 28, 2015 at 7:21 PM, Stefan Beller  wrote:
> Just pass it along to "git submodule update", which may pick reasonable
> defaults if you don't specify an explicit number.
>
> Signed-off-by: Stefan Beller 
> ---
> @@ -724,8 +723,20 @@ static int checkout(void)
> err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
>sha1_to_hex(sha1), "1", NULL);
>
> -   if (!err && option_recursive)
> -   err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
> +   if (!err && option_recursive) {
> +   struct argv_array args = ARGV_ARRAY_INIT;
> +   argv_array_pushl(, "submodule", "update", "--init", 
> "--recursive", NULL);
> +
> +   if (max_jobs != -1) {
> +   struct strbuf sb = STRBUF_INIT;
> +   strbuf_addf(, "--jobs=%d", max_jobs);
> +   argv_array_push(, sb.buf);
> +   strbuf_release();

The above four lines can be collapsed to:

argv_array_pushf(, "--jobs=%d", max_jobs);

> +   }
> +
> +   err = run_command_v_opt(args.argv, RUN_GIT_CMD);
> +   argv_array_clear();
> +   }
>
> return err;
>  }
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4] checkout: add --progress option

[Edmundo Carmona Antoranz]

Under normal circumstances, and like other git commands,
git checkout will write progress info to stderr if
attached to a terminal. This option allows progress
to be forced even if not using a terminal. Also,
progress can be skipped if using option --no-progress.

Signed-off-by: Edmundo Carmona Antoranz 
---
 Documentation/git-checkout.txt |  6 ++
 builtin/checkout.c | 26 --
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index e269fb1..93ba35a 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -107,6 +107,12 @@ OPTIONS
 --quiet::
Quiet, suppress feedback messages.
 
+--progress::
+   Progress status is reported on the standard error stream
+   by default when it is attached to a terminal, unless -q
+   is specified. This flag forces progress status even if the
+   standard error stream is not directed to a terminal.
+
 -f::
 --force::
When switching branches, proceed even if the index or the
diff --git a/builtin/checkout.c b/builtin/checkout.c
index bc703c0..c3b8e5d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
int overwrite_ignore;
int ignore_skipworktree;
int ignore_other_worktrees;
+   int show_progress;
 
const char *new_branch;
const char *new_branch_force;
@@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
checkout_opts *o,
opts.reset = 1;
opts.merge = 1;
opts.fn = oneway_merge;
-   opts.verbose_update = !o->quiet && isatty(2);
+   opts.verbose_update = o->show_progress;
opts.src_index = _index;
opts.dst_index = _index;
parse_tree(tree);
@@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
*opts,
topts.update = 1;
topts.merge = 1;
topts.gently = opts->merge && old->commit;
-   topts.verbose_update = !opts->quiet && isatty(2);
+   topts.verbose_update = opts->show_progress;
topts.fn = twoway_merge;
if (opts->overwrite_ignore) {
topts.dir = xcalloc(1, sizeof(*topts.dir));
@@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
N_("second guess 'git checkout 
'")),
OPT_BOOL(0, "ignore-other-worktrees", 
_other_worktrees,
 N_("do not check if another worktree is holding the 
given ref")),
+   OPT_BOOL(0, "progress", _progress, N_("force progress 
reporting")),
OPT_END(),
};
 
@@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
memset(, 0, sizeof(new));
opts.overwrite_ignore = 1;
opts.prefix = prefix;
+   opts.show_progress = -1;
 
gitmodules_config();
git_config(git_checkout_config, );
@@ -1172,6 +1175,25 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
argc = parse_options(argc, argv, prefix, options, checkout_usage,
 PARSE_OPT_KEEP_DASHDASH);
 
+   /*
+* Final processing of show_progress
+* - User selected --progress: show progress
+* - user selected --no-progress: skip progress
+* - User didn't specify:
+* (check rules in order till finding the first matching one)
+* - user selected --quiet: skip progress
+* - stderr is connected to a terminal: show progress
+* - fallback: skip progress
+*/
+   if (opts.show_progress < 0) {
+   /* user didn't specify --[no-]progress */
+   if (opts.quiet) {
+   opts.show_progress = 0;
+   } else {
+   opts.show_progress = isatty(2);
+   }
+   }
+
if (conflict_style) {
opts.merge = 1; /* implied */
git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] mailinfo: fix passing wrong address to git_mailinfo_config

[Junio C Hamano]

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Jeff King]

On Sun, Nov 01, 2015 at 12:52:57PM -0500, Eric Sunshine wrote:

> > diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> > index e269fb1..93ba35a 100644
> > --- a/Documentation/git-checkout.txt
> > +++ b/Documentation/git-checkout.txt
> > @@ -107,6 +107,12 @@ OPTIONS
> >  --quiet::
> > Quiet, suppress feedback messages.
> >
> > +--progress::
> > +   Progress status is reported on the standard error stream
> > +   by default when it is attached to a terminal, unless -q
> > +   is specified. This flag forces progress status even if the
> > +   standard error stream is not directed to a terminal.
> 
> What this kind of implies, but neglects to say explicitly, is that the
> logic implemented by this patch also overrides --quiet. It probably
> should say so explicitly.
> 
> I realize that this text was copied from elsewhere (likely from
> git-clone.txt), but git-checkout.txt does try to do a bit better job
> with formatting, so it might be a good idea to quote -q with backticks
> (`-q` or `--quiet`).

I was the one who suggested originally that --progress should override
--quiet[1]. However, that was just based on what I thought was
reasonable. I didn't look at what clone or other commands do.
Consistency between commands is probably more important than any
particular behavior.

[1] To be honest, this is kind of a crazy corner case anyway. It was
more just that it has to do _something_.

> > +   /*
> > +* Final processing of show_progress
> > +* - User selected --progress: show progress
> > +* - user selected --no-progress: skip progress
> > +* - User didn't specify:
> > +* (check rules in order till finding the first matching one)
> > +* - user selected --quiet: skip progress
> > +* - stderr is connected to a terminal: show progress
> > +* - fallback: skip progress
> > +*/
> 
> It takes longer to read and digest this comment block than it does to
> comprehend the actual logic in code, which is pretty clear in its
> current form. Comment blocks which merely repeat easily digested code
> add little, if any, value, so it might be worthwhile to drop the
> comment altogether.

Yeah, I think I agree.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/5] read-cache: add watchman 'WAMA' extension

[Nguyễn Thái Ngọc Duy]

The extension contains a bitmap, one bit for each entry in the
index. If the n-th bit is zero, the n-th entry is considered
unchanged, we can ce_mark_uptodate() it without refreshing. If the bit
is non-zero and we found out the corresponding file is clean after
refresh, we can clear the bit.

The 'skipping refresh' bit is not in this patch yet as we would need
watchman. More details in later patches.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 cache.h  |  4 
 read-cache.c | 71 ++--
 2 files changed, 73 insertions(+), 2 deletions(-)

diff --git a/cache.h b/cache.h
index 9633acc..a05fd31 100644
--- a/cache.h
+++ b/cache.h
@@ -163,6 +163,8 @@ struct cache_entry {
 #define CE_VALID (0x8000)
 #define CE_STAGESHIFT 12
 
+#define CE_NO_WATCH  (0x0001)
+
 /*
  * Range 0x0FFF in ce_flags is divided into
  * two parts: in-memory flags and on-disk ones.
@@ -298,6 +300,7 @@ static inline unsigned int canon_mode(unsigned int mode)
 #define CACHE_TREE_CHANGED (1 << 5)
 #define SPLIT_INDEX_ORDERED(1 << 6)
 #define UNTRACKED_CHANGED  (1 << 7)
+#define WATCHMAN_CHANGED   (1 << 8)
 
 struct split_index;
 struct untracked_cache;
@@ -322,6 +325,7 @@ struct index_state {
struct untracked_cache *untracked;
void *mmap;
size_t mmap_size;
+   char *last_update;
 };
 
 extern struct index_state the_index;
diff --git a/read-cache.c b/read-cache.c
index f609776..893223e 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -19,6 +19,7 @@
 #include "split-index.h"
 #include "utf8.h"
 #include "shm.h"
+#include "ewah/ewok.h"
 
 static struct cache_entry *refresh_cache_entry(struct cache_entry *ce,
   unsigned int options);
@@ -41,11 +42,13 @@ static struct cache_entry *refresh_cache_entry(struct 
cache_entry *ce,
 #define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* "REUC" */
 #define CACHE_EXT_LINK 0x6c696e6b/* "link" */
 #define CACHE_EXT_UNTRACKED 0x554E5452   /* "UNTR" */
+#define CACHE_EXT_WATCHMAN 0x57414D41/* "WAMA" */
 
 /* changes that can be kept in $GIT_DIR/index (basically all extensions) */
 #define EXTMASK (RESOLVE_UNDO_CHANGED | CACHE_TREE_CHANGED | \
 CE_ENTRY_ADDED | CE_ENTRY_REMOVED | CE_ENTRY_CHANGED | \
-SPLIT_INDEX_ORDERED | UNTRACKED_CHANGED)
+SPLIT_INDEX_ORDERED | UNTRACKED_CHANGED | \
+WATCHMAN_CHANGED)
 
 struct index_state the_index;
 static const char *alternate_index_output;
@@ -1224,8 +1227,13 @@ int refresh_index(struct index_state *istate, unsigned 
int flags,
continue;
 
new = refresh_cache_ent(istate, ce, options, _errno, 
);
-   if (new == ce)
+   if (new == ce) {
+   if (ce->ce_flags & CE_NO_WATCH) {
+   ce->ce_flags  &= ~CE_NO_WATCH;
+   istate->cache_changed |= WATCHMAN_CHANGED;
+   }
continue;
+   }
if (!new) {
const char *fmt;
 
@@ -1369,6 +1377,48 @@ static int verify_hdr(const struct cache_header *hdr, 
unsigned long size)
return 0;
 }
 
+static void mark_no_watchman(size_t pos, void *data)
+{
+   struct index_state *istate = data;
+   assert(pos < istate->cache_nr);
+   istate->cache[pos]->ce_flags |= CE_NO_WATCH;
+}
+
+static int read_watchman_ext(struct index_state *istate, const void *data,
+unsigned long sz)
+{
+   struct ewah_bitmap *bitmap;
+   int ret, len;
+
+   if (memchr(data, 0, sz) == NULL)
+   return error("invalid extension");
+   len = strlen(data) + 1;
+   bitmap = ewah_new();
+   ret = ewah_read_mmap(bitmap, (const char *)data + len, sz - len);
+   if (ret != sz - len) {
+   ewah_free(bitmap);
+   return error("fail to parse ewah bitmap");
+   }
+   istate->last_update = xstrdup(data);
+   ewah_each_bit(bitmap, mark_no_watchman, istate);
+   ewah_free(bitmap);
+   return 0;
+}
+
+static void write_watchman_ext(struct strbuf *sb, struct index_state* istate)
+{
+   struct ewah_bitmap *bitmap;
+   int i;
+
+   strbuf_add(sb, istate->last_update, strlen(istate->last_update) + 1);
+   bitmap = ewah_new();
+   for (i = 0; i < istate->cache_nr; i++)
+   if (istate->cache[i]->ce_flags & CE_NO_WATCH)
+   ewah_set(bitmap, i);
+   ewah_serialize_strbuf(bitmap, sb);
+   ewah_free(bitmap);
+}
+
 static int read_index_extension(struct index_state *istate,
const char *ext, void *data, unsigned long sz)
 {
@@ -1386,6 +1436,11 @@ static int read_index_extension(struct index_state 
*istate,
case CACHE_EXT_UNTRACKED:
istate->untracked = read_untracked_extension(data, sz);
  

[PATCH 4/5] index-helper: use watchman to avoid refreshing index with lstat()

[Nguyễn Thái Ngọc Duy]

Watchman is hidden behind index-helper. Before git tries to read the
index from shm, it notifies index-helper with SIGHUP and sleep,
waiting for index-helper to prepare shm. index-helper then contacts
watchman, updates 'WAMA' extension and put it in a separate shm and
wakes git up with SIGHUP.

Git uses this extension to not lstat unchanged entries. Git only trust
'WAMA' extension when it's received from the separate shm, not from
disk. Unmarked entries are "clean". Marked entries are dirty from
watchman point of view. If it finds out some entries are
'watchman-dirty', but are really unchanged (e.g. the file was changed,
then reverted back), then Git will clear the marking in 'WAMA' before
writing it down.

Hiding watchman behind index-helper means you need both daemons. You
can't run watchman alone. Not so good. But on the other hand, 'git'
binary is not linked to watchman/json libraries, which is good for
packaging. Core git package will run fine without watchman-related
packages. If they need watchman, they can install git-index-helper and
dependencies.

Another reason for tying watchman to index-helper is, when used with
untracked cache, we need to keep track of $GIT_WORK_TREE file
listing. That kind of list can be kept in index-helper.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Makefile   |  5 
 cache.h|  2 ++
 index-helper.c | 84 +++---
 read-cache.c   | 43 ++--
 watchman-support.h |  1 -
 5 files changed, 127 insertions(+), 8 deletions(-)

diff --git a/Makefile b/Makefile
index 761acb6..3f5eac8 100644
--- a/Makefile
+++ b/Makefile
@@ -434,6 +434,7 @@ MSGFMT = msgfmt
 CURL_CONFIG = curl-config
 PTHREAD_LIBS = -lpthread
 PTHREAD_CFLAGS =
+WATCHMAN_LIBS =
 GCOV = gcov
 
 export TCL_PATH TCLTK_PATH
@@ -1392,6 +1393,7 @@ endif
 ifdef USE_WATCHMAN
LIB_H += watchman-support.h
LIB_OBJS += watchman-support.o
+   WATCHMAN_LIBS = -lwatchman
BASIC_CFLAGS += -DUSE_WATCHMAN
 endif
 
@@ -2005,6 +2007,9 @@ git-remote-testsvn$X: remote-testsvn.o GIT-LDFLAGS 
$(GITLIBS) $(VCSSVN_LIB)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) 
$(LIBS) \
$(VCSSVN_LIB)
 
+git-index-helper$X: index-helper.o GIT-LDFLAGS $(GITLIBS)
+   $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) 
$(LIBS) $(WATCHMAN_LIBS)
+
 $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)
$(QUIET_LNCP)$(RM) $@ && \
ln $< $@ 2>/dev/null || \
diff --git a/cache.h b/cache.h
index 572299c..c04141b 100644
--- a/cache.h
+++ b/cache.h
@@ -518,6 +518,7 @@ extern int daemonize(int *);
 
 /* Initialize and use the cache information */
 struct lock_file;
+extern int verify_index(const struct index_state *);
 extern int read_index(struct index_state *);
 extern int read_index_preload(struct index_state *, const struct pathspec 
*pathspec);
 extern int do_read_index(struct index_state *istate, const char *path,
@@ -525,6 +526,7 @@ extern int do_read_index(struct index_state *istate, const 
char *path,
 extern int read_index_from(struct index_state *, const char *path);
 extern int is_index_unborn(struct index_state *);
 extern int read_index_unmerged(struct index_state *);
+extern void write_watchman_ext(struct strbuf *sb, struct index_state* istate);
 #define COMMIT_LOCK(1 << 0)
 #define CLOSE_LOCK (1 << 1)
 #define REFRESH_DAEMON (1 << 2)
diff --git a/index-helper.c b/index-helper.c
index cf26da7..421887e 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -5,15 +5,18 @@
 #include "split-index.h"
 #include "shm.h"
 #include "lockfile.h"
+#include "watchman-support.h"
 
 struct shm {
unsigned char sha1[20];
void *shm;
size_t size;
+   pid_t pid;
 };
 
 static struct shm shm_index;
 static struct shm shm_base_index;
+static struct shm shm_watchman;
 static int daemonized, to_verify = 1;
 
 static void release_index_shm(struct shm *is)
@@ -25,10 +28,21 @@ static void release_index_shm(struct shm *is)
is->shm = NULL;
 }
 
+static void release_watchman_shm(struct shm *is)
+{
+   if (!is->shm)
+   return;
+   munmap(is->shm, is->size);
+   git_shm_unlink("git-watchman-%s-%" PRIuMAX,
+  sha1_to_hex(is->sha1), (uintmax_t)is->pid);
+   is->shm = NULL;
+}
+
 static void cleanup_shm(void)
 {
release_index_shm(_index);
release_index_shm(_base_index);
+   release_watchman_shm(_watchman);
 }
 
 static void cleanup(void)
@@ -120,13 +134,15 @@ static void share_the_index(void)
if (the_index.split_index && the_index.split_index->base)
share_index(the_index.split_index->base, _base_index);
share_index(_index, _index);
-   if (to_verify && !verify_shm())
+   if (to_verify && !verify_shm()) {
cleanup_shm();
-   discard_index(_index);
+   discard_index(_index);
+ 

[PATCH 3/5] read-cache: allow index-helper to prepare shm before git reads it

[Nguyễn Thái Ngọc Duy]

If index-helper puts 'W' before pid in $GIT_DIR/index-helper.pid, then
git will sleep for a while, expecting to be waken up by SIGUSR1 when
index-helper has done shm preparation, or after the timeout.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 read-cache.c | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index 893223e..ae33951 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1591,14 +1591,24 @@ static void do_poke(struct strbuf *sb, int 
refresh_cache)
PostMessage(hwnd, refresh_cache ? WM_USER : WM_USER + 1, 0, 0);
 }
 #else
+static void do_nothing(int sig)
+{
+}
+
 static void do_poke(struct strbuf *sb, int refresh_cache)
 {
-   char*start = sb->buf;
+   int  wait  = sb->buf[0] == 'W';
+   char*start = wait ? sb->buf + 1 : sb->buf;
char*end   = NULL;
pid_tpid   = strtoul(start, , 10);
+   int  ret;
if (!end || end != sb->buf + sb->len)
return;
-   kill(pid, refresh_cache ? SIGHUP : SIGUSR1);
+   ret = kill(pid, refresh_cache ? SIGHUP : SIGUSR1);
+   if (!refresh_cache && !ret && wait) {
+   signal(SIGHUP, do_nothing);
+   sleep(1);
+   }
 }
 #endif
 
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/5] update-index: enable/disable watchman support

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 builtin/update-index.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/builtin/update-index.c b/builtin/update-index.c
index 7431938..86aec21 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -903,6 +903,7 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
 {
int newfd, entries, has_errors = 0, line_termination = '\n';
int untracked_cache = -1;
+   int use_watchman = -1;
int read_from_stdin = 0;
int prefix_length = prefix ? strlen(prefix) : 0;
int preferred_index_format = 0;
@@ -998,6 +999,8 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
N_("enable/disable untracked cache")),
OPT_SET_INT(0, "force-untracked-cache", _cache,
N_("enable untracked cache without testing the 
filesystem"), 2),
+   OPT_BOOL(0, "watchman", _watchman,
+   N_("use or not use watchman to reduce refresh cost")),
OPT_END()
};
 
@@ -1127,6 +1130,14 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
the_index.cache_changed |= UNTRACKED_CHANGED;
}
 
+   if (use_watchman > 0) {
+   the_index.last_update= xstrdup("");
+   the_index.cache_changed |= WATCHMAN_CHANGED;
+   } else if (!use_watchman) {
+   the_index.last_update= NULL;
+   the_index.cache_changed |= WATCHMAN_CHANGED;
+   }
+
if (active_cache_changed) {
if (newfd < 0) {
if (refresh_args.flags & REFRESH_QUIET)
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/5] Add watchman support to reduce index refresh cost

[Nguyễn Thái Ngọc Duy]

The previous patch has the logic to clear bits in 'WAMA' bitmap. This
patch has logic to set bits as told by watchman. The missing bit,
_using_ these bits, are not here yet.

A lot of this code is written by David Turner originally, mostly from
[1]. I'm just copying and polishing it a bit.

[1] http://article.gmane.org/gmane.comp.version-control.git/248006

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Makefile |   7 +++
 cache.h  |   1 +
 config.c |   5 +++
 configure.ac |   8 
 environment.c|   3 ++
 watchman-support.c (new) | 108 +++
 watchman-support.h (new) |   8 
 7 files changed, 140 insertions(+)
 create mode 100644 watchman-support.c
 create mode 100644 watchman-support.h

diff --git a/Makefile b/Makefile
index c01cd2e..761acb6 100644
--- a/Makefile
+++ b/Makefile
@@ -1389,6 +1389,12 @@ else
LIB_OBJS += thread-utils.o
 endif
 
+ifdef USE_WATCHMAN
+   LIB_H += watchman-support.h
+   LIB_OBJS += watchman-support.o
+   BASIC_CFLAGS += -DUSE_WATCHMAN
+endif
+
 ifdef HAVE_PATHS_H
BASIC_CFLAGS += -DHAVE_PATHS_H
 endif
@@ -2135,6 +2141,7 @@ GIT-BUILD-OPTIONS: FORCE
@echo NO_PERL=\''$(subst ','\'',$(subst ','\'',$(NO_PERL)))'\' >>$@+
@echo NO_PYTHON=\''$(subst ','\'',$(subst ','\'',$(NO_PYTHON)))'\' >>$@+
@echo NO_UNIX_SOCKETS=\''$(subst ','\'',$(subst 
','\'',$(NO_UNIX_SOCKETS)))'\' >>$@+
+   @echo USE_WATCHMAN=\''$(subst ','\'',$(subst 
','\'',$(USE_WATCHMAN)))'\' >>$@+
 ifdef TEST_OUTPUT_DIRECTORY
@echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst 
','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+
 endif
diff --git a/cache.h b/cache.h
index a05fd31..572299c 100644
--- a/cache.h
+++ b/cache.h
@@ -648,6 +648,7 @@ extern char *git_replace_ref_base;
 
 extern int fsync_object_files;
 extern int core_preload_index;
+extern int core_watchman_sync_timeout;
 extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
diff --git a/config.c b/config.c
index 248a21a..6b63f66 100644
--- a/config.c
+++ b/config.c
@@ -881,6 +881,11 @@ static int git_default_core_config(const char *var, const 
char *value)
return 0;
}
 
+   if (!strcmp(var, "core.watchmansynctimeout")) {
+   core_watchman_sync_timeout = git_config_int(var, value);
+   return 0;
+   }
+
if (!strcmp(var, "core.createobject")) {
if (!strcmp(value, "rename"))
object_creation_mode = OBJECT_CREATION_USES_RENAMES;
diff --git a/configure.ac b/configure.ac
index 76170ad..9772f79 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1092,6 +1092,14 @@ AC_COMPILE_IFELSE([BSD_SYSCTL_SRC],
HAVE_BSD_SYSCTL=])
 GIT_CONF_SUBST([HAVE_BSD_SYSCTL])
 
+#
+# Check for watchman client library
+
+AC_CHECK_LIB([watchman], [watchman_connect],
+   [USE_WATCHMAN=YesPlease],
+   [USE_WATCHMAN=])
+GIT_CONF_SUBST([USE_WATCHMAN])
+
 ## Other checks.
 # Define USE_PIC if you need the main git objects to be built with -fPIC
 # in order to build and link perl/Git.so.  x86-64 seems to need this.
diff --git a/environment.c b/environment.c
index 2da7fe2..84df431 100644
--- a/environment.c
+++ b/environment.c
@@ -87,6 +87,9 @@ int auto_comment_line_char;
 /* Parallel index stat data preload? */
 int core_preload_index = 1;
 
+int core_watchman_sync_timeout = 300;
+
+
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
 static char *work_tree;
diff --git a/watchman-support.c b/watchman-support.c
new file mode 100644
index 000..7f6c0a9
--- /dev/null
+++ b/watchman-support.c
@@ -0,0 +1,108 @@
+#include "cache.h"
+#include "watchman-support.h"
+#include "strbuf.h"
+#include 
+
+static struct watchman_query *make_query(const char *last_update)
+{
+   struct watchman_query *query = watchman_query();
+   watchman_query_set_fields(query, WATCHMAN_FIELD_NAME |
+WATCHMAN_FIELD_EXISTS |
+WATCHMAN_FIELD_NEWER);
+   watchman_query_set_empty_on_fresh(query, 1);
+   query->sync_timeout = core_watchman_sync_timeout;
+   if (*last_update)
+   watchman_query_set_since_oclock(query, last_update);
+   return query;
+}
+
+static struct watchman_query_result* query_watchman(
+   struct index_state *istate, struct watchman_connection *connection,
+   const char *fs_path, const char *last_update)
+{
+   struct watchman_error wm_error;
+   struct watchman_query *query;
+   struct watchman_expression *expr;
+   struct watchman_query_result *result;
+
+   query = make_query(last_update);
+   expr = watchman_true_expression();
+   result = watchman_do_query(connection, fs_path, query, expr, _error);
+   watchman_free_query(query);
+   

[PATCH 0/5] Use watchman to reduce index refresh time

[Nguyễn Thái Ngọc Duy]

This series builds on top of the index-helper series I just sent and
uses watchman to keep track of file changes in order to avoid lstat()
at refresh time. The series can also be found at [1]

When I started this work, watchman did not support Windows yet. It
does now, even if still experimental [2]. So Windows people, please
try it out if you have time.

To put all pieces so far together, we have split-index to reduce index
write time, untracked cache to reduce I/O as well as computation for
.gitignore, index-helper for index read time and this series for
lstat() at refresh time. The remaining piece is killing lstat() from
untracked cache, but right now it's just some idea and incomplete
code.

[1] https://github.com/pclouds/git/commits/refresh-with-watchman
[2] https://github.com/facebook/watchman/issues/19

Nguyễn Thái Ngọc Duy (5):
  read-cache: add watchman 'WAMA' extension
  Add watchman support to reduce index refresh cost
  read-cache: allow index-helper to prepare shm before git reads it
  index-helper: use watchman to avoid refreshing index with lstat()
  update-index: enable/disable watchman support

 Makefile |  12 +
 builtin/update-index.c   |  11 +
 cache.h  |   7 +++
 config.c |   5 ++
 configure.ac |   8 +++
 environment.c|   3 ++
 index-helper.c   |  84 +--
 read-cache.c | 126 ---
 watchman-support.c (new) | 108 
 watchman-support.h (new) |   7 +++
 10 files changed, 361 insertions(+), 10 deletions(-)
 create mode 100644 watchman-support.c
 create mode 100644 watchman-support.h

-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] mailinfo: fix passing wrong address to git_mailinfo_config

[Nguyễn Thái Ngọc Duy]

git_mailinfo_config() expects "struct mailinfo *". But in
setup_mailinfo(), "mi" is already "struct mailinfo *".  would make
it "struct mailinfo **" and git_mailinfo_config() would damage some
other memory when it assigns some value to mi->use_scissors.

This is caught by t4150.20. git_mailinfo_config() breaks
mi->name.alloc and makes strbuf_release() in clear_mailinfo() attempt
to free strbuf_slopbuf.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 mailinfo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mailinfo.c b/mailinfo.c
index e157ca6..f289941 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -1009,7 +1009,7 @@ void setup_mailinfo(struct mailinfo *mi)
mi->header_stage = 1;
mi->use_inbody_headers = 1;
mi->content_top = mi->content;
-   git_config(git_mailinfo_config, );
+   git_config(git_mailinfo_config, mi);
 }
 
 void clear_mailinfo(struct mailinfo *mi)
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] ls-files: Add eol diagnostics

[Junio C Hamano]

Matthieu Moy  writes:

> Torsten Bögershausen  writes:
>
>> ca:text-no-eol   wt:text-no-eol   t/t5100/empty
>> ca:binarywt:binaryt/test-binary-2.png
>> ca:text-lf   wt:text-lf   t/t5100/rfc2047-info-0007
>> ca:text-lf   wt:text-crlf doit.bat
>> ca:text-crlf-lf  wt:text-crlf-lf  locale/XX.po
>
> I would spell the first "in" or "idx" (for "index"), not "ca" (for
> "cache"). I think we avoid talking about "the cache" these days even
> though the doc sometimes says "cached in the index" (i.e. use "cache" as
> a verb, not a noun).

i/ and w/ have been used to denote the "i"ndex and "w"orktree
versions for the past 7 years with diff.mnemonicprefix option,
which you may want to match.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 3/6] Facilitate debugging Git executables in tests with gdb

[Johannes Schindelin]

Hi Victor,

On Sat, 31 Oct 2015, Victor Leschuk wrote:

> > >   +if test -n "$TEST_GDB_GIT"
> > > +then
> > > +   exec gdb -args "${GIT_EXEC_PATH}/@@PROG@@" "$@"
> > Maybe we could make $TEST_GDB_GIT not just a boolean flag? It would be 
> > useful
> > to contain "gdb" executable name. It would allow to set path to GDB when it
> > is not in $PATH, set different debuggers (for example, I usually use cgdb),
> > or even set it to /path/to/gdb_wrapper.sh which could contain different gdb
> > options and tunings.
> 
> > Sure, as long as TEST_GDB_GIT=1 still works. Why don't you make an add-on
> > patch and submit it?
> 
> Sure, I will prepare the patch as soon as this one is included in master.

Excuse my asking: why do you want to wait?

Ciao,
Johannes
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sat, Oct 31, 2015 at 6:45 PM, Eric Sunshine  wrote:
> Patches in 'next' are pretty much set in stone, and those in 'master'
> definitely are, but 'pu' is volatile, so just send the entire patch
> revised, tagged v2 (or v3, etc.), rather than sending a patch to fix
> what is in 'pu', and Junio will replace the previous version with the
> new one.
>

Will do!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/5] wt-status: avoid building bogus branch name with detached HEAD

[Junio C Hamano]

René Scharfe  writes:

> If we're on a detached HEAD then wt_shortstatus_print_tracking() takes
> the string "HEAD (no branch)", translates it, skips the first eleven
> characters and passes the result to branch_get(), which returns a bogus
> result and accesses memory out of bounds in order to produce it.

The fix is correct, but the above explanation looks "not quite" to
me.

That "HEAD (no branch)" thing is in a separate branch_name variable
that is not involved in the actual computation (i.e. call to
branch_get()).

The function gets "HEAD" in s->branch, uses that and skips the first
eleven characters (i.e. beyond the end of that string), lets
branch_get() to return a garbage and likely missing branch, finds
that nobody tracks that, and does the right thing anyway.  If the
garbage past the end of the "HEAD" happens to have a name of an
existing branch, we would get an incorrect result.

Thanks.

> Somehow stat_tracking_info(), which is passed that result, does the
> right thing anyway, i.e. it finds that there is no base.
>
> Avoid the bogus results and memory accesses by checking for HEAD first
> and exiting early in that case.  This fixes t7060 with --valgrind.
>
> Signed-off-by: Rene Scharfe 
> ---
>  t/t7060-wtstatus.sh |  2 +-
>  wt-status.c | 15 +--
>  2 files changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/t/t7060-wtstatus.sh b/t/t7060-wtstatus.sh
> index e6af772..44bf1d8 100755
> --- a/t/t7060-wtstatus.sh
> +++ b/t/t7060-wtstatus.sh
> @@ -213,7 +213,7 @@ EOF
>   git checkout master
>  '
>  
> -test_expect_failure 'status --branch with detached HEAD' '
> +test_expect_success 'status --branch with detached HEAD' '
>   git reset --hard &&
>   git checkout master^0 &&
>   git status --branch --porcelain >actual &&
> diff --git a/wt-status.c b/wt-status.c
> index 083328f..e206cc9 100644
> --- a/wt-status.c
> +++ b/wt-status.c
> @@ -1644,16 +1644,19 @@ static void wt_shortstatus_print_tracking(struct 
> wt_status *s)
>   return;
>   branch_name = s->branch;
>  
> + if (s->is_initial)
> + color_fprintf(s->fp, header_color, _("Initial commit on "));
> +
> + if (!strcmp(s->branch, "HEAD")) {
> + color_fprintf(s->fp, color(WT_STATUS_NOBRANCH, s), "%s",
> +   _("HEAD (no branch)"));
> + goto conclude;
> + }
> +
>   if (starts_with(branch_name, "refs/heads/"))
>   branch_name += 11;
> - else if (!strcmp(branch_name, "HEAD")) {
> - branch_name = _("HEAD (no branch)");
> - branch_color_local = color(WT_STATUS_NOBRANCH, s);
> - }
>  
>   branch = branch_get(s->branch + 11);
> - if (s->is_initial)
> - color_fprintf(s->fp, header_color, _("Initial commit on "));
>  
>   color_fprintf(s->fp, branch_color_local, "%s", branch_name);
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Junio C Hamano]

atous...@gmail.com writes:

> diff --git a/cache.h b/cache.h
> index 79066e5..ec84b16 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -14,7 +14,12 @@
>  #ifndef git_SHA_CTX
>  #define git_SHA_CTX  SHA_CTX
>  #define git_SHA1_InitSHA1_Init
> -#define git_SHA1_Update  SHA1_Update
> +#ifdef SHA1_MAX_BLOCK_SIZE
> +extern int SHA1_Update_Chunked(SHA_CTX *, const void *, size_t);
> +#define git_SHA1_Update SHA1_Update_Chunked
> +#else
> +#define git_SHA1_Update SHA1_Update
> +#endif
>  #define git_SHA1_Final   SHA1_Final
>  #endif

Hmm, I admit that this mess is my creation, but unfortunately it
does not allow us to say:

make SHA1_MAX_BLOCK_SIZE='1024L*1024L*1024L'

when using other SHA-1 implementations (e.g. blk_SHA1_Update()).

Ideas for cleaning it up, anybody?

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5] checkout: add --progress option

[Edmundo Carmona Antoranz]

Oh, man... I'm sorry... I didn't notice them. Let me go over them to
see what we can do.

On Sun, Nov 1, 2015 at 1:13 PM, Eric Sunshine  wrote:
> I'll assume that you simply overlooked the remainder of my v4 review
> comments, so I'll merely provide a reference to them[1] rather than
> repeating myself. If that assumption is incorrect, please do have the
> courtesy to state that you disagree with review comments and to
> explain your position, otherwise reviewers will feel that their
> efforts are wasted.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 0/9] Reduce index load time

[Nguyễn Thái Ngọc Duy]

This is the rebased version since last time [1] with
s/free_index_shm/release_index_shm/ as suggested by David Turner. It
introduces a daemon that can cache index data in memory so that
subsequent git processes can avoid reading (and more importantly,
verifying) the index from disk. Together with split-index it should
keep index I/O cost down to minimum. The series can also be found at
[2].

One of the factors that affected my design was Windows support. We
now have Dscho back, he can evaluate my approach for Windows.

This daemon is the foundation for watchman support later to reduce
refresh time. To be posted shortly after this.

[0] http://mid.gmane.org/1406548995-28549-1-git-send-email-pclo...@gmail.com
[2] http://github.com/pclouds/git/commits/index-helper

Nguyễn Thái Ngọc Duy (9):
  trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics
  read-cache.c: fix constness of verify_hdr()
  read-cache: allow to keep mmap'd memory after reading
  index-helper: new daemon for caching index and related stuff
  trace.c: add GIT_TRACE_INDEX_STATS for index  statistics
  index-helper: add --strict
  daemonize(): set a flag before exiting the main process
  index-helper: add --detach
  index-helper: add Windows support

 .gitignore   |   1 +
 Documentation/git-index-helper.txt (new) |  56 +++
 Documentation/git.txt|   4 +
 Makefile |   9 ++
 builtin/gc.c |   2 +-
 cache.h  |  12 +-
 config.mak.uname |   3 +
 daemon.c |   2 +-
 git-compat-util.h|   1 +
 git.c|   1 +
 index-helper.c (new) | 264 +++
 read-cache.c | 147 -
 setup.c  |   4 +-
 sha1_file.c  |  24 +++
 shm.c (new)  | 163 +++
 shm.h (new)  |  23 +++
 trace.c  |  16 ++
 trace.h  |   1 +
 18 files changed, 721 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/git-index-helper.txt
 create mode 100644 index-helper.c
 create mode 100644 shm.c
 create mode 100644 shm.h

-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/5] wt-status: don't skip a magical number of characters blindly

[Junio C Hamano]

René Scharfe  writes:

> Use the variable branch_name, which already has "refs/heads/" removed,
> instead of blindly advancing in the ->branch string by 11 bytes.  This
> is safer and less magical.
>
> Signed-off-by: Rene Scharfe 
> ---
>  wt-status.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/wt-status.c b/wt-status.c
> index e206cc9..42ea15e 100644
> --- a/wt-status.c
> +++ b/wt-status.c
> @@ -1656,7 +1656,7 @@ static void wt_shortstatus_print_tracking(struct 
> wt_status *s)
>   if (starts_with(branch_name, "refs/heads/"))
>   branch_name += 11;
>  
> - branch = branch_get(s->branch + 11);
> + branch = branch_get(branch_name);

Is this correct?  s->branch is the refname that is l10n independent;
branch_name is the localized variant for human consumption.

>   color_fprintf(s->fp, branch_color_local, "%s", branch_name);
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Eric Sunshine]

On Sun, Nov 1, 2015 at 12:47 PM, Edmundo Carmona Antoranz
 wrote:
> Under normal circumstances, and like other git commands,
> git checkout will write progress info to stderr if
> attached to a terminal. This option allows progress
> to be forced even if not using a terminal. Also,
> progress can be skipped if using option --no-progress.
>
> Signed-off-by: Edmundo Carmona Antoranz 
> ---
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> index e269fb1..93ba35a 100644
> --- a/Documentation/git-checkout.txt
> +++ b/Documentation/git-checkout.txt
> @@ -107,6 +107,12 @@ OPTIONS
>  --quiet::
> Quiet, suppress feedback messages.
>
> +--progress::
> +   Progress status is reported on the standard error stream
> +   by default when it is attached to a terminal, unless -q
> +   is specified. This flag forces progress status even if the
> +   standard error stream is not directed to a terminal.

What this kind of implies, but neglects to say explicitly, is that the
logic implemented by this patch also overrides --quiet. It probably
should say so explicitly.

I realize that this text was copied from elsewhere (likely from
git-clone.txt), but git-checkout.txt does try to do a bit better job
with formatting, so it might be a good idea to quote -q with backticks
(`-q` or `--quiet`).

> diff --git a/builtin/checkout.c b/builtin/checkout.c
> index bc703c0..c3b8e5d 100644
> --- a/builtin/checkout.c
> +++ b/builtin/checkout.c
> @@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const 
> char *prefix)
> memset(, 0, sizeof(new));
> opts.overwrite_ignore = 1;
> opts.prefix = prefix;
> +   opts.show_progress = -1;
>
> gitmodules_config();
> git_config(git_checkout_config, );
> @@ -1172,6 +1175,25 @@ int cmd_checkout(int argc, const char **argv, const 
> char *prefix)
> argc = parse_options(argc, argv, prefix, options, checkout_usage,
>  PARSE_OPT_KEEP_DASHDASH);
>
> +   /*
> +* Final processing of show_progress
> +* - User selected --progress: show progress
> +* - user selected --no-progress: skip progress
> +* - User didn't specify:
> +* (check rules in order till finding the first matching one)
> +* - user selected --quiet: skip progress
> +* - stderr is connected to a terminal: show progress
> +* - fallback: skip progress
> +*/

It takes longer to read and digest this comment block than it does to
comprehend the actual logic in code, which is pretty clear in its
current form. Comment blocks which merely repeat easily digested code
add little, if any, value, so it might be worthwhile to drop the
comment altogether.

> +   if (opts.show_progress < 0) {
> +   /* user didn't specify --[no-]progress */

Here, also, consider dropping the comment which merely repeats what
the code already states clearly.

> +   if (opts.quiet) {
> +   opts.show_progress = 0;
> +   } else {
> +   opts.show_progress = isatty(2);
> +   }

Style: drop unnecessary braces

> +   }
> +
> if (conflict_style) {
> opts.merge = 1; /* implied */
> git_xmerge_config("merge.conflictstyle", conflict_style, 
> NULL);
> --
> 2.6.1
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5] checkout: add --progress option

[Edmundo Carmona Antoranz]

Under normal circumstances, and like other git commands,
git checkout will write progress info to stderr if
attached to a terminal. This option allows progress
to be forced even if not using a terminal. Also,
progress can be skipped if using option --no-progress.

Signed-off-by: Edmundo Carmona Antoranz 
---
 Documentation/git-checkout.txt |  6 ++
 builtin/checkout.c | 25 +++--
 2 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index e269fb1..93ba35a 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -107,6 +107,12 @@ OPTIONS
 --quiet::
Quiet, suppress feedback messages.
 
+--progress::
+   Progress status is reported on the standard error stream
+   by default when it is attached to a terminal, unless -q
+   is specified. This flag forces progress status even if the
+   standard error stream is not directed to a terminal.
+
 -f::
 --force::
When switching branches, proceed even if the index or the
diff --git a/builtin/checkout.c b/builtin/checkout.c
index bc703c0..65b8b90 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
int overwrite_ignore;
int ignore_skipworktree;
int ignore_other_worktrees;
+   int show_progress;
 
const char *new_branch;
const char *new_branch_force;
@@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
checkout_opts *o,
opts.reset = 1;
opts.merge = 1;
opts.fn = oneway_merge;
-   opts.verbose_update = !o->quiet && isatty(2);
+   opts.verbose_update = o->show_progress;
opts.src_index = _index;
opts.dst_index = _index;
parse_tree(tree);
@@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
*opts,
topts.update = 1;
topts.merge = 1;
topts.gently = opts->merge && old->commit;
-   topts.verbose_update = !opts->quiet && isatty(2);
+   topts.verbose_update = opts->show_progress;
topts.fn = twoway_merge;
if (opts->overwrite_ignore) {
topts.dir = xcalloc(1, sizeof(*topts.dir));
@@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
N_("second guess 'git checkout 
'")),
OPT_BOOL(0, "ignore-other-worktrees", 
_other_worktrees,
 N_("do not check if another worktree is holding the 
given ref")),
+   OPT_BOOL(0, "progress", _progress, N_("force progress 
reporting")),
OPT_END(),
};
 
@@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
memset(, 0, sizeof(new));
opts.overwrite_ignore = 1;
opts.prefix = prefix;
+   opts.show_progress = -1;
 
gitmodules_config();
git_config(git_checkout_config, );
@@ -1172,6 +1175,24 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
argc = parse_options(argc, argv, prefix, options, checkout_usage,
 PARSE_OPT_KEEP_DASHDASH);
 
+   /*
+* Final processing of show_progress
+* - User selected --progress: show progress
+* - user selected --no-progress: skip progress
+* - User didn't specify:
+* (check rules in order till finding the first matching one)
+* - user selected --quiet: skip progress
+* - stderr is connected to a terminal: show progress
+* - fallback: skip progress
+*/
+   if (opts.show_progress < 0) {
+   /* user didn't specify --[no-]progress */
+   if (opts.quiet)
+   opts.show_progress = 0;
+   else
+   opts.show_progress = isatty(2);
+   }
+
if (conflict_style) {
opts.merge = 1; /* implied */
git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/5] wt-status: avoid building bogus branch name with detached HEAD

[René Scharfe]

Am 01.11.2015 um 18:50 schrieb Junio C Hamano:

René Scharfe  writes:


If we're on a detached HEAD then wt_shortstatus_print_tracking() takes
the string "HEAD (no branch)", translates it, skips the first eleven
characters and passes the result to branch_get(), which returns a bogus
result and accesses memory out of bounds in order to produce it.


The fix is correct, but the above explanation looks "not quite" to
me.

That "HEAD (no branch)" thing is in a separate branch_name variable
that is not involved in the actual computation (i.e. call to
branch_get()).

The function gets "HEAD" in s->branch, uses that and skips the first
eleven characters (i.e. beyond the end of that string), lets
branch_get() to return a garbage and likely missing branch, finds
that nobody tracks that, and does the right thing anyway.  If the
garbage past the end of the "HEAD" happens to have a name of an
existing branch, we would get an incorrect result.


Ah, yes.  This came from an earlier round which had patch 3 and 4 
reversed, causing the translated string to be passed to branch_get(). 
Thanks for catching the commit message inconsistency!


René
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] ls-files: Add eol diagnostics

[Matthieu Moy]

Sebastian Schuberth  writes:

> However, the commit message says "to check if text files are stored
> normalized in the *repository*", yet the output refers to the index /
> cache. Is there a (potential) difference between line endings in the
> index and repo?

There is when you have staged changes that are not commited yet.

> Any I find it a bit confusing to refer to the index where, as e.g. for
> a freshly cloned repo the index should be empty,

No it is not. The index is a complete snapshot of your working tree.
When you have no uncommited staged changes, the index contains all files
that are in HEAD. Most commands show you _changes_ in the index (wrt
HEAD or wrt the working tree), but the index itself contain all files.

> Long story short, how about consistently talking about line endings in
> the repo, and also using "repo" instead of "ca" here?

I don't think this is a good idea. One typical use-case for the feature
would probably be:

1) wtf, there's something wrong with my line endings, let's fix this.

2) tweak .gitattributes, try to get everything right

3) prepare a commit to apply the new settings to the repository, play
   with "git add", "dos2unix" and friends.

4) check that it's OK

5) "git commit"

At stage 4), you really want to see the content of the index, because
your HEAD is still broken.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] ls-files: Add eol diagnostics

[Matthieu Moy]

Junio C Hamano  writes:

> i/ and w/ have been used to denote the "i"ndex and "w"orktree
> versions for the past 7 years with diff.mnemonicprefix option,
> which you may want to match.

Indeed.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 4/9] index-helper: new daemon for caching index and related stuff

[Nguyễn Thái Ngọc Duy]

Instead of reading the index from disk and worrying about disk
corruption, the index is cached in memory (memory bit-flips happen
too, but hopefully less often). The result is faster read. Read time
is reduced by 70%.

The biggest gain is not having to verify the trailing SHA-1, which
takes lots of time especially on large index files. But this also
opens doors for further optimiztions:

 - we could create an in-memory format that's essentially the memory
   dump of the index to eliminate most of parsing/allocation
   overhead. The mmap'd memory can be used straight away. Experiment
   [1] shows we could reduce read time by 88%.

 - we could cache non-index info such as name hash

The shared memory's name folows the template "git--"
where  is the trailing SHA-1 of the index file.  is
"index" for cached index files (and may be "name-hash" for name-hash
cache). If such shared memory exists, it contains the same index
content as on disk. The content is already validated by the daemon and
git won't validate it again (except comparing the trailing SHA-1s).

Git can poke the daemon to tell it to refresh the index cache, or to
keep it alive some more minutes via UNIX signals. It can't give any
real index data directly to the daemon. Real data goes to disk first,
then the daemon reads and verifies it from there. Poking only happens
for $GIT_DIR/index, not temporary index files.

$GIT_DIR/index-helper.pid contains a reference to daemon process (and
it's pid on *nix). The file's mtime is updated every time it's accessed
(or should be updated often enough). Old index-helper.pid is considered
stale and ignored.

index-helper requires POSIX realtime extension. POSIX shm interface
however is abstracted away so that Windows support can be added later.

On webkit.git with index format v2, duplicating 8 times to 1.4m
entries and 200MB in size:

(vanilla)  0.986986364 s: read_index_from .git/index
(index-helper) 0.267850279 s: read_index_from .git/index

Interestingly with index v4, we get less out of index-helper. It makes
sense as v4 requires more processing after loading the index:

(vanilla)  0.72249 s: read_index_from .git/index
(index-helper) 0.302741500 s: read_index_from .git/index

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 .gitignore   |   1 +
 Documentation/git-index-helper.txt (new) |  44 +
 Makefile |   9 ++
 cache.h  |   3 +
 config.mak.uname |   1 +
 git-compat-util.h|   1 +
 index-helper.c (new) | 162 +++
 read-cache.c | 106 ++--
 shm.c (new)  |  67 +
 shm.h (new)  |  23 +
 10 files changed, 408 insertions(+), 9 deletions(-)
 create mode 100644 Documentation/git-index-helper.txt
 create mode 100644 index-helper.c
 create mode 100644 shm.c
 create mode 100644 shm.h

diff --git a/.gitignore b/.gitignore
index 1c2f832..f36f1d3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -71,6 +71,7 @@
 /git-http-fetch
 /git-http-push
 /git-imap-send
+/git-index-helper
 /git-index-pack
 /git-init
 /git-init-db
diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
new file mode 100644
index 000..9db28cf
--- /dev/null
+++ b/Documentation/git-index-helper.txt
@@ -0,0 +1,44 @@
+git-index-helper(1)
+===
+
+NAME
+
+git-index-helper - A simple cache daemon for speeding up index file access
+
+SYNOPSIS
+
+[verse]
+'git index-helper' [options]
+
+DESCRIPTION
+---
+Keep the index file in memory for faster access. This daemon is per
+repository.
+
+OPTIONS
+---
+
+--exit-after=::
+   Exit if the cached index is not accessed for ``
+   minutes. Specify 0 to wait forever. Default is 10.
+
+NOTES
+-
+On UNIX-like systems, $GIT_DIR/index-helper.pid contains the process
+id of the daemon. At least on Linux, shared memory objects are
+availble via /dev/shm with the name pattern "git--".
+Normally the daemon will clean up shared memory objects when it exits.
+But if it crashes, some objects could remain there and they can be
+safely deleted with "rm" command. The following signals are used to
+control the daemon:
+
+SIGHUP::
+   Reread the index.
+
+SIGUSR1::
+   Let the daemon know the index is to be read. It keeps the
+   daemon alive longer, unless `--exit-after=0` is used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 43ceeb9..c01cd2e 100644
--- a/Makefile
+++ b/Makefile
@@ -363,6 +363,8 @@ all::
 # Define HAVE_BSD_SYSCTL if your platform has a BSD-compatible sysctl function.
 #
 # Define HAVE_GETDELIM if your system has the getdelim() function.
+#
+# Define HAVE_SHM if you platform support shm_* functions in librt.
 
 GIT-VERSION-FILE: FORCE
@$(SHELL_PATH) 

[PATCH v3] checkout: add --progress option

[Edmundo Carmona Antoranz]

Under normal circumstances, and like other git commands,
git checkout will write progress info to stderr if
attached to a terminal. This option allows progress
to be forced even if not using a terminal. Also,
progress can be skipped if using option --no-progress.
---
 Documentation/git-checkout.txt |  6 ++
 builtin/checkout.c | 26 --
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index e269fb1..93ba35a 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -107,6 +107,12 @@ OPTIONS
 --quiet::
Quiet, suppress feedback messages.
 
+--progress::
+   Progress status is reported on the standard error stream
+   by default when it is attached to a terminal, unless -q
+   is specified. This flag forces progress status even if the
+   standard error stream is not directed to a terminal.
+
 -f::
 --force::
When switching branches, proceed even if the index or the
diff --git a/builtin/checkout.c b/builtin/checkout.c
index bc703c0..1bc4efe 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
int overwrite_ignore;
int ignore_skipworktree;
int ignore_other_worktrees;
+   int show_progress;
 
const char *new_branch;
const char *new_branch_force;
@@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
checkout_opts *o,
opts.reset = 1;
opts.merge = 1;
opts.fn = oneway_merge;
-   opts.verbose_update = !o->quiet && isatty(2);
+   opts.verbose_update = o->show_progress;
opts.src_index = _index;
opts.dst_index = _index;
parse_tree(tree);
@@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
*opts,
topts.update = 1;
topts.merge = 1;
topts.gently = opts->merge && old->commit;
-   topts.verbose_update = !opts->quiet && isatty(2);
+   topts.verbose_update = opts->show_progress;
topts.fn = twoway_merge;
if (opts->overwrite_ignore) {
topts.dir = xcalloc(1, sizeof(*topts.dir));
@@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
N_("second guess 'git checkout 
'")),
OPT_BOOL(0, "ignore-other-worktrees", 
_other_worktrees,
 N_("do not check if another worktree is holding the 
given ref")),
+   OPT_BOOL(0, "progress", _progress, N_("force progress 
reporting")),
OPT_END(),
};
 
@@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
memset(, 0, sizeof(new));
opts.overwrite_ignore = 1;
opts.prefix = prefix;
+   opts.show_progress = -1;
 
gitmodules_config();
git_config(git_checkout_config, );
@@ -1172,6 +1175,25 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
argc = parse_options(argc, argv, prefix, options, checkout_usage,
 PARSE_OPT_KEEP_DASHDASH);
 
+   /*
+* Final processing of show_progress
+* - User selected --progress: show progress
+* - user selected --no-progress: skip progress
+* - User didn't specify:
+* (check rules in order till finding the first matching one)
+* - user selected --quiet: skip progress
+* - stderr is connected to a terminal: show progress
+* - fallback: skip progress
+*/
+   if (opts.show_progress < 0) {
+   /* user selected --progress or didn't specify */
+   if (opts.quiet) {
+   opts.show_progress = 0;
+   } else {
+   opts.show_progress = isatty(2);
+   }
+   }
+
if (conflict_style) {
opts.merge = 1; /* implied */
git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/5] wt-status: don't skip a magical number of characters blindly

[Junio C Hamano]

Junio C Hamano  writes:

>> diff --git a/wt-status.c b/wt-status.c
>> index e206cc9..42ea15e 100644
>> --- a/wt-status.c
>> +++ b/wt-status.c
>> @@ -1656,7 +1656,7 @@ static void wt_shortstatus_print_tracking(struct 
>> wt_status *s)
>>  if (starts_with(branch_name, "refs/heads/"))
>>  branch_name += 11;
>>  
>> -branch = branch_get(s->branch + 11);
>> +branch = branch_get(branch_name);
>
> Is this correct?  s->branch is the refname that is l10n independent;
> branch_name is the localized variant for human consumption.

Ahh, and that convention has been changed at patch 3/5.  Now the
only localizable string "HEAD (no branch)" never goes into that
variable thanks to the code reorganization in 3/5, this variable
is used only to give us a shortened refname.

OK, I misread the code.  The result is correct.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Junio C Hamano]

Atousa Duprat  writes:

> Indeed, the declaration needs a const void *; but I need to advance by
> a specific number of bytes in each iteration of the loop.  Hence the
> cast.

Ah, of course you are right.  Silly me.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5] checkout: add --progress option

[Eric Sunshine]

On Sun, Nov 1, 2015 at 1:03 PM, Edmundo Carmona Antoranz
 wrote:
> Under normal circumstances, and like other git commands,
> git checkout will write progress info to stderr if
> attached to a terminal. This option allows progress
> to be forced even if not using a terminal. Also,
> progress can be skipped if using option --no-progress.
>
> Signed-off-by: Edmundo Carmona Antoranz 
> ---
> +   /*
> +* Final processing of show_progress
> +* - User selected --progress: show progress
> +* - user selected --no-progress: skip progress
> +* - User didn't specify:
> +* (check rules in order till finding the first matching one)
> +* - user selected --quiet: skip progress
> +* - stderr is connected to a terminal: show progress
> +* - fallback: skip progress
> +*/
> +   if (opts.show_progress < 0) {
> +   /* user didn't specify --[no-]progress */
> +   if (opts.quiet)
> +   opts.show_progress = 0;
> +   else
> +   opts.show_progress = isatty(2);
> +   }

Style-wise this looks better.

I'll assume that you simply overlooked the remainder of my v4 review
comments, so I'll merely provide a reference to them[1] rather than
repeating myself. If that assumption is incorrect, please do have the
courtesy to state that you disagree with review comments and to
explain your position, otherwise reviewers will feel that their
efforts are wasted.

Thanks.

[1]: http://article.gmane.org/gmane.comp.version-control.git/280641
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 6/9] index-helper: add --strict

[Nguyễn Thái Ngọc Duy]

There's are "holes" in the index-helper approach because the shared
memory is not verified again by git. If $USER is compromised, shared
memory could be modified. But then they can already modify
$GIT_DIR/index. A more realistic risk is some bugs in index-helper
produce corrupt shared memory. --strict is added to avoid that

Strictly speaking there's still a very small gap where corrupt shared
memory could still be read by git: after we write the trailing SHA-1 in
the shared memory (thus signaling "this shm is ready") and before
verify_shm() detects an error.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Documentation/git-index-helper.txt |  9 +++
 cache.h|  1 +
 index-helper.c | 48 ++
 read-cache.c   |  9 ---
 4 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
index 9db28cf..ad40366 100644
--- a/Documentation/git-index-helper.txt
+++ b/Documentation/git-index-helper.txt
@@ -22,6 +22,15 @@ OPTIONS
Exit if the cached index is not accessed for ``
minutes. Specify 0 to wait forever. Default is 10.
 
+--strict::
+--no-strict::
+   Strict mode makes index-helper verify the shared memory after
+   it's created. If the result does not match what's read from
+   $GIT_DIR/index, the shared memory is destroyed. This makes
+   index-helper take more than double the amount of time required
+   for reading an index, but because it will happen in the
+   background, it's not noticable. `--strict` is enabled by default.
+
 NOTES
 -
 On UNIX-like systems, $GIT_DIR/index-helper.pid contains the process
diff --git a/cache.h b/cache.h
index 69c2365..dd3df26 100644
--- a/cache.h
+++ b/cache.h
@@ -314,6 +314,7 @@ struct index_state {
 keep_mmap : 1,
 from_shm : 1,
 to_shm : 1,
+always_verify_trailing_sha1 : 1,
 initialized : 1;
struct hashmap name_hash;
struct hashmap dir_hash;
diff --git a/index-helper.c b/index-helper.c
index cf2971d..1140bc0 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -14,6 +14,7 @@ struct shm {
 
 static struct shm shm_index;
 static struct shm shm_base_index;
+static int to_verify = 1;
 
 static void release_index_shm(struct shm *is)
 {
@@ -69,11 +70,56 @@ static void share_index(struct index_state *istate, struct 
shm *is)
hashcpy((unsigned char *)new_mmap + istate->mmap_size - 20, is->sha1);
 }
 
+static int verify_shm(void)
+{
+   int i;
+   struct index_state istate;
+   memset(, 0, sizeof(istate));
+   istate.always_verify_trailing_sha1 = 1;
+   istate.to_shm = 1;
+   i = read_index();
+   if (i != the_index.cache_nr)
+   goto done;
+   for (; i < the_index.cache_nr; i++) {
+   struct cache_entry *base, *ce;
+   /* namelen is checked separately */
+   const unsigned int ondisk_flags =
+   CE_STAGEMASK | CE_VALID | CE_EXTENDED_FLAGS;
+   unsigned int ce_flags, base_flags, ret;
+   base = the_index.cache[i];
+   ce = istate.cache[i];
+   if (ce->ce_namelen != base->ce_namelen ||
+   strcmp(ce->name, base->name)) {
+   warning("mismatch at entry %d", i);
+   break;
+   }
+   ce_flags = ce->ce_flags;
+   base_flags = base->ce_flags;
+   /* only on-disk flags matter */
+   ce->ce_flags   &= ondisk_flags;
+   base->ce_flags &= ondisk_flags;
+   ret = memcmp(>ce_stat_data, >ce_stat_data,
+offsetof(struct cache_entry, name) -
+offsetof(struct cache_entry, ce_stat_data));
+   ce->ce_flags = ce_flags;
+   base->ce_flags = base_flags;
+   if (ret) {
+   warning("mismatch at entry %d", i);
+   break;
+   }
+   }
+done:
+   discard_index();
+   return i == the_index.cache_nr;
+}
+
 static void share_the_index(void)
 {
if (the_index.split_index && the_index.split_index->base)
share_index(the_index.split_index->base, _base_index);
share_index(_index, _index);
+   if (to_verify && !verify_shm())
+   cleanup_shm();
discard_index(_index);
 }
 
@@ -130,6 +176,8 @@ int main(int argc, char **argv)
struct option options[] = {
OPT_INTEGER(0, "exit-after", _in_minutes,
N_("exit if not used after some minutes")),
+   OPT_BOOL(0, "strict", _verify,
+"verify shared memory after creating"),
OPT_END()
};
 
diff --git a/read-cache.c b/read-cache.c
index 

[PATCH v4 5/9] trace.c: add GIT_TRACE_INDEX_STATS for index statistics

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Documentation/git.txt |  1 +
 cache.h   |  1 +
 read-cache.c  | 16 
 trace.c   |  5 -
 4 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 1086ced..f2078aa 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -1046,6 +1046,7 @@ of clones and fetches.
See 'GIT_TRACE' for available trace output options.
 
 'GIT_TRACE_PACK_STATS'::
+'GIT_TRACE_INDEX_STATS'::
Print various statistics.
 
 'GIT_TRACE_SETUP'::
diff --git a/cache.h b/cache.h
index 30a3a77..69c2365 100644
--- a/cache.h
+++ b/cache.h
@@ -1754,5 +1754,6 @@ int versioncmp(const char *s1, const char *s2);
 void sleep_millisec(int millisec);
 
 void report_pack_stats(struct trace_key *key);
+void report_index_stats(struct trace_key *key);
 
 #endif /* CACHE_H */
diff --git a/read-cache.c b/read-cache.c
index 6c98e98..6ae50c7 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -50,6 +50,10 @@ static struct cache_entry *refresh_cache_entry(struct 
cache_entry *ce,
 struct index_state the_index;
 static const char *alternate_index_output;
 
+static unsigned int nr_read_index;
+static unsigned int nr_read_shm_index;
+static unsigned int nr_write_index;
+
 static void set_index_entry(struct index_state *istate, int nr, struct 
cache_entry *ce)
 {
istate->cache[nr] = ce;
@@ -1592,6 +1596,7 @@ static int try_shm(struct index_state *istate)
istate->mmap = new_mmap;
istate->mmap_size = new_length;
istate->from_shm = 1;
+   nr_read_shm_index++;
return 0;
 }
 
@@ -1689,6 +1694,7 @@ int do_read_index(struct index_state *istate, const char 
*path, int must_exist)
}
if (!istate->keep_mmap)
munmap(mmap, mmap_size);
+   nr_read_index++;
return istate->cache_nr;
 
 unmap:
@@ -2174,6 +2180,7 @@ static int do_write_index(struct index_state *istate, int 
newfd,
return -1;
istate->timestamp.sec = (unsigned int)st.st_mtime;
istate->timestamp.nsec = ST_MTIME_NSEC(st);
+   nr_write_index++;
return 0;
 }
 
@@ -2400,3 +2407,12 @@ void stat_validity_update(struct stat_validity *sv, int 
fd)
fill_stat_data(sv->sd, );
}
 }
+
+void report_index_stats(struct trace_key *key)
+{
+   trace_printf_key(key, "\n"
+"index stats: file reads= %10u\n"
+"index stats: cache reads   = %10u\n"
+"index stats: file writes   = %10u\n",
+nr_read_index, nr_read_shm_index, nr_write_index);
+}
diff --git a/trace.c b/trace.c
index b1d0885..eea1fa8 100644
--- a/trace.c
+++ b/trace.c
@@ -434,14 +434,17 @@ void trace_command_performance(const char **argv)
 }
 
 static struct trace_key trace_pack_stats = TRACE_KEY_INIT(PACK_STATS);
+static struct trace_key trace_index_stats = TRACE_KEY_INIT(INDEX_STATS);
 
 static void print_stats_atexit(void)
 {
report_pack_stats(_pack_stats);
+   report_index_stats(_index_stats);
 }
 
 void trace_stats(void)
 {
-   if (trace_want(_pack_stats))
+   if (trace_want(_pack_stats) ||
+   trace_want(_index_stats))
atexit(print_stats_atexit);
 }
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 7/9] daemonize(): set a flag before exiting the main process

[Nguyễn Thái Ngọc Duy]

This allows signal handlers and atexit functions to realize this
situation and not clean up.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 builtin/gc.c | 2 +-
 cache.h  | 2 +-
 daemon.c | 2 +-
 setup.c  | 4 +++-
 4 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index df3e454..e59c9d2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -369,7 +369,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 * failure to daemonize is ok, we'll continue
 * in foreground
 */
-   daemonized = !daemonize();
+   daemonized = !daemonize(NULL);
}
} else
add_repack_all_option();
diff --git a/cache.h b/cache.h
index dd3df26..9633acc 100644
--- a/cache.h
+++ b/cache.h
@@ -490,7 +490,7 @@ extern int set_git_dir_init(const char *git_dir, const char 
*real_git_dir, int);
 extern int init_db(const char *template_dir, unsigned int flags);
 
 extern void sanitize_stdfds(void);
-extern int daemonize(void);
+extern int daemonize(int *);
 
 #define alloc_nr(x) (((x)+16)*3/2)
 
diff --git a/daemon.c b/daemon.c
index 56679a1..9f9f057 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1364,7 +1364,7 @@ int main(int argc, char **argv)
return execute();
 
if (detach) {
-   if (daemonize())
+   if (daemonize(NULL))
die("--detach not supported on this platform");
} else
sanitize_stdfds();
diff --git a/setup.c b/setup.c
index d343725..968af3d 100644
--- a/setup.c
+++ b/setup.c
@@ -1015,7 +1015,7 @@ void sanitize_stdfds(void)
close(fd);
 }
 
-int daemonize(void)
+int daemonize(int *daemonized)
 {
 #ifdef NO_POSIX_GOODIES
errno = ENOSYS;
@@ -1027,6 +1027,8 @@ int daemonize(void)
case -1:
die_errno("fork failed");
default:
+   if (daemonized)
+   *daemonized = 1;
exit(0);
}
if (setsid() == -1)
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 9/9] index-helper: add Windows support

[Nguyễn Thái Ngọc Duy]

Windows supports shared memory, but the semantics is a bit different
than POSIX shm. The most noticeable thing is there's no way to get the
shared memory's size by the reader, and wrapping fstat to do that
would be hell. So the shm size is added near the end, hidden away from
shm users (storing it in headers would cause more problems with munmap,
storing it as a separate shm is even worse).

PostMessage is used instead of UNIX signals for
notification. Lightweight (at least code-wise) on the client side.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 config.mak.uname |  2 ++
 index-helper.c   | 48 
 read-cache.c | 13 
 shm.c| 96 
 4 files changed, 159 insertions(+)

diff --git a/config.mak.uname b/config.mak.uname
index 3167e36..260fa82 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -391,6 +391,7 @@ ifndef DEBUG
 else
BASIC_CFLAGS += -Zi -MDd
 endif
+   PROGRAM_OBJS += index-helper.o
X = .exe
 endif
 ifeq ($(uname_S),Interix)
@@ -545,6 +546,7 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT))
 else
NO_CURL = YesPlease
 endif
+   PROGRAM_OBJS += index-helper.o
 endif
 ifeq ($(uname_S),QNX)
COMPAT_CFLAGS += -DSA_RESTART=0
diff --git a/index-helper.c b/index-helper.c
index 4dd9656..cf26da7 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -155,6 +155,51 @@ static void loop(const char *pid_file, int idle_in_seconds)
; /* do nothing, all is handled by signal handlers already */
 }
 
+#elif defined(GIT_WINDOWS_NATIVE)
+
+static void loop(const char *pid_file, int idle_in_seconds)
+{
+   HWND hwnd;
+   UINT_PTR timer = 0;
+   MSG msg;
+   HINSTANCE hinst = GetModuleHandle(NULL);
+   WNDCLASS wc;
+
+   /*
+* Emulate UNIX signals by sending WM_USER+x to a
+* window. Register window class and create a new window to
+* catch these messages.
+*/
+   memset(, 0, sizeof(wc));
+   wc.lpfnWndProc   = DefWindowProc;
+   wc.hInstance = hinst;
+   wc.lpszClassName = "git-index-helper";
+   if (!RegisterClass())
+   die_errno(_("could not register new window class"));
+
+   hwnd = CreateWindow("git-index-helper", pid_file,
+   0, 0, 0, 1, 1, NULL, NULL, hinst, NULL);
+   if (!hwnd)
+   die_errno(_("could not register new window"));
+
+   refresh(0);
+   while (1) {
+   timer = SetTimer(hwnd, timer, idle_in_seconds * 1000, NULL);
+   if (!timer)
+   die(_("no timer!"));
+   if (!GetMessage(, hwnd, 0, 0) || msg.message == WM_TIMER)
+   break;
+   switch (msg.message) {
+   case WM_USER:
+   refresh(0);
+   break;
+   default:
+   /* just reset the timer */
+   break;
+   }
+   }
+}
+
 #else
 
 static void loop(const char *pid_file, int idle_in_seconds)
@@ -198,6 +243,9 @@ int main(int argc, char **argv)
fd = hold_lock_file_for_update(,
   git_path("index-helper.pid"),
   LOCK_DIE_ON_ERROR);
+#ifdef GIT_WINDOWS_NATIVE
+   strbuf_addstr(, "HWND");
+#endif
strbuf_addf(, "%" PRIuMAX, (uintmax_t) getpid());
write_in_full(fd, sb.buf, sb.len);
commit_lock_file();
diff --git a/read-cache.c b/read-cache.c
index 3ae2bc1..f609776 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1524,6 +1524,18 @@ static void check_ce_order(struct index_state *istate)
}
 }
 
+#if defined(GIT_WINDOWS_NATIVE)
+static void do_poke(struct strbuf *sb, int refresh_cache)
+{
+   HWND hwnd;
+   if (!starts_with(sb->buf, "HWND"))
+   return;
+   hwnd = FindWindow("git-index-helper", sb->buf);
+   if (!hwnd)
+   return;
+   PostMessage(hwnd, refresh_cache ? WM_USER : WM_USER + 1, 0, 0);
+}
+#else
 static void do_poke(struct strbuf *sb, int refresh_cache)
 {
char*start = sb->buf;
@@ -1533,6 +1545,7 @@ static void do_poke(struct strbuf *sb, int refresh_cache)
return;
kill(pid, refresh_cache ? SIGHUP : SIGUSR1);
 }
+#endif
 
 static void poke_daemon(struct index_state *istate,
const struct stat *st, int refresh_cache)
diff --git a/shm.c b/shm.c
index 4ec1a00..04d8a35 100644
--- a/shm.c
+++ b/shm.c
@@ -52,6 +52,102 @@ void git_shm_unlink(const char *fmt, ...)
shm_unlink(path);
 }
 
+#elif defined(GIT_WINDOWS_NATIVE)
+
+#define SHM_PATH_LEN 82/* a little bit longer than POSIX because of 
"Local\\" */
+
+static ssize_t create_shm_map(int oflag, int perm, ssize_t length,
+ void **mmap, int prot, int flags,
+ const char *path, unsigned long page_size)
+{
+   

[PATCH v4 8/9] index-helper: add --detach

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Documentation/git-index-helper.txt |  3 +++
 index-helper.c | 10 --
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
index ad40366..9ced091 100644
--- a/Documentation/git-index-helper.txt
+++ b/Documentation/git-index-helper.txt
@@ -31,6 +31,9 @@ OPTIONS
for reading an index, but because it will happen in the
background, it's not noticable. `--strict` is enabled by default.
 
+--detach::
+   Detach from the shell.
+
 NOTES
 -
 On UNIX-like systems, $GIT_DIR/index-helper.pid contains the process
diff --git a/index-helper.c b/index-helper.c
index 1140bc0..4dd9656 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -14,7 +14,7 @@ struct shm {
 
 static struct shm shm_index;
 static struct shm shm_base_index;
-static int to_verify = 1;
+static int daemonized, to_verify = 1;
 
 static void release_index_shm(struct shm *is)
 {
@@ -33,6 +33,8 @@ static void cleanup_shm(void)
 
 static void cleanup(void)
 {
+   if (daemonized)
+   return;
unlink(git_path("index-helper.pid"));
cleanup_shm();
 }
@@ -172,12 +174,13 @@ int main(int argc, char **argv)
static struct lock_file lock;
struct strbuf sb = STRBUF_INIT;
const char *prefix;
-   int fd, idle_in_minutes = 10;
+   int fd, idle_in_minutes = 10, detach = 0;
struct option options[] = {
OPT_INTEGER(0, "exit-after", _in_minutes,
N_("exit if not used after some minutes")),
OPT_BOOL(0, "strict", _verify,
 "verify shared memory after creating"),
+   OPT_BOOL(0, "detach", , "detach the process"),
OPT_END()
};
 
@@ -202,6 +205,9 @@ int main(int argc, char **argv)
atexit(cleanup);
sigchain_push_common(cleanup_on_signal);
 
+   if (detach && daemonize())
+   die_errno("unable to detach");
+
if (!idle_in_minutes)
idle_in_minutes = 0x / 60;
loop(sb.buf, idle_in_minutes * 60);
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 3/9] read-cache: allow to keep mmap'd memory after reading

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 cache.h  |  3 +++
 read-cache.c | 13 -
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/cache.h b/cache.h
index 8791dbc..89e9aaf 100644
--- a/cache.h
+++ b/cache.h
@@ -311,11 +311,14 @@ struct index_state {
struct split_index *split_index;
struct cache_time timestamp;
unsigned name_hash_initialized : 1,
+keep_mmap : 1,
 initialized : 1;
struct hashmap name_hash;
struct hashmap dir_hash;
unsigned char sha1[20];
struct untracked_cache *untracked;
+   void *mmap;
+   size_t mmap_size;
 };
 
 extern struct index_state the_index;
diff --git a/read-cache.c b/read-cache.c
index a76c789..7d04108 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1552,6 +1552,10 @@ int do_read_index(struct index_state *istate, const char 
*path, int must_exist)
mmap = xmmap(NULL, mmap_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (mmap == MAP_FAILED)
die_errno("unable to map index file");
+   if (istate->keep_mmap) {
+   istate->mmap = mmap;
+   istate->mmap_size = mmap_size;
+   }
close(fd);
 
hdr = mmap;
@@ -1604,10 +1608,12 @@ int do_read_index(struct index_state *istate, const 
char *path, int must_exist)
src_offset += 8;
src_offset += extsize;
}
-   munmap(mmap, mmap_size);
+   if (!istate->keep_mmap)
+   munmap(mmap, mmap_size);
return istate->cache_nr;
 
 unmap:
+   istate->mmap = NULL;
munmap(mmap, mmap_size);
die("index file corrupt");
 }
@@ -1632,6 +1638,7 @@ int read_index_from(struct index_state *istate, const 
char *path)
discard_index(split_index->base);
else
split_index->base = xcalloc(1, sizeof(*split_index->base));
+   split_index->base->keep_mmap = istate->keep_mmap;
ret = do_read_index(split_index->base,
git_path("sharedindex.%s",
 sha1_to_hex(split_index->base_sha1)), 1);
@@ -1675,6 +1682,10 @@ int discard_index(struct index_state *istate)
free(istate->cache);
istate->cache = NULL;
istate->cache_alloc = 0;
+   if (istate->keep_mmap && istate->mmap) {
+   munmap(istate->mmap, istate->mmap_size);
+   istate->mmap = NULL;
+   }
discard_split_index(istate);
free_untracked_cache(istate->untracked);
istate->untracked = NULL;
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 1/9] trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics

[Nguyễn Thái Ngọc Duy]

trace_stats() is intended for GIT_TRACE_*_STATS variable group and
GIT_TRACE_PACK_STATS is more like an example how new vars can be added.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Documentation/git.txt |  3 +++
 cache.h   |  2 ++
 git.c |  1 +
 sha1_file.c   | 24 
 trace.c   | 13 +
 trace.h   |  1 +
 6 files changed, 44 insertions(+)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 4585103..1086ced 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -1045,6 +1045,9 @@ of clones and fetches.
time of each Git command.
See 'GIT_TRACE' for available trace output options.
 
+'GIT_TRACE_PACK_STATS'::
+   Print various statistics.
+
 'GIT_TRACE_SETUP'::
Enables trace messages printing the .git, working tree and current
working directory after Git has completed its setup phase.
diff --git a/cache.h b/cache.h
index 3ba0b8f..8791dbc 100644
--- a/cache.h
+++ b/cache.h
@@ -1747,4 +1747,6 @@ void stat_validity_update(struct stat_validity *sv, int 
fd);
 int versioncmp(const char *s1, const char *s2);
 void sleep_millisec(int millisec);
 
+void report_pack_stats(struct trace_key *key);
+
 #endif /* CACHE_H */
diff --git a/git.c b/git.c
index 6ed824c..f4018c5 100644
--- a/git.c
+++ b/git.c
@@ -644,6 +644,7 @@ int main(int argc, char **av)
git_setup_gettext();
 
trace_command_performance(argv);
+   trace_stats();
 
/*
 * "git-" is the same as "git ", but we obviously:
diff --git a/sha1_file.c b/sha1_file.c
index c5b31de..1d3508d 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -517,6 +517,7 @@ static unsigned int peak_pack_open_windows;
 static unsigned int pack_open_windows;
 static unsigned int pack_open_fds;
 static unsigned int pack_max_fds;
+static unsigned int pack_access_nr;
 static size_t peak_pack_mapped;
 static size_t pack_mapped;
 struct packed_git *packed_git;
@@ -542,6 +543,28 @@ void pack_report(void)
sz_fmt(pack_mapped), sz_fmt(peak_pack_mapped));
 }
 
+void report_pack_stats(struct trace_key *key)
+{
+   trace_printf_key(key, "\n"
+"pack_report: getpagesize()= %10" SZ_FMT 
"\n"
+"pack_report: core.packedGitWindowSize = %10" SZ_FMT 
"\n"
+"pack_report: core.packedGitLimit  = %10" SZ_FMT 
"\n"
+"pack_report: pack_used_ctr= %10u\n"
+"pack_report: pack_mmap_calls  = %10u\n"
+"pack_report: pack_open_windows= %10u / %10u\n"
+"pack_report: pack_mapped  = "
+"%10" SZ_FMT " / %10" SZ_FMT "\n"
+"pack_report: pack accesss = %10u\n",
+sz_fmt(getpagesize()),
+sz_fmt(packed_git_window_size),
+sz_fmt(packed_git_limit),
+pack_used_ctr,
+pack_mmap_calls,
+pack_open_windows, peak_pack_open_windows,
+sz_fmt(pack_mapped), sz_fmt(peak_pack_mapped),
+pack_access_nr);
+}
+
 /*
  * Open and mmap the index file at path, perform a couple of
  * consistency checks, then record its information to p.  Return 0 on
@@ -2244,6 +2267,7 @@ static void write_pack_access_log(struct packed_git *p, 
off_t obj_offset)
static struct trace_key pack_access = TRACE_KEY_INIT(PACK_ACCESS);
trace_printf_key(_access, "%s %"PRIuMAX"\n",
 p->pack_name, (uintmax_t)obj_offset);
+   pack_access_nr++;
 }
 
 int do_check_packed_object_crc;
diff --git a/trace.c b/trace.c
index 4aeea60..b1d0885 100644
--- a/trace.c
+++ b/trace.c
@@ -432,3 +432,16 @@ void trace_command_performance(const char **argv)
sq_quote_argv(_line, argv, 0);
command_start_time = getnanotime();
 }
+
+static struct trace_key trace_pack_stats = TRACE_KEY_INIT(PACK_STATS);
+
+static void print_stats_atexit(void)
+{
+   report_pack_stats(_pack_stats);
+}
+
+void trace_stats(void)
+{
+   if (trace_want(_pack_stats))
+   atexit(print_stats_atexit);
+}
diff --git a/trace.h b/trace.h
index 179b249..52bda4e 100644
--- a/trace.h
+++ b/trace.h
@@ -19,6 +19,7 @@ extern void trace_disable(struct trace_key *key);
 extern uint64_t getnanotime(void);
 extern void trace_command_performance(const char **argv);
 extern void trace_verbatim(struct trace_key *key, const void *buf, unsigned 
len);
+extern void trace_stats(void);
 
 #ifndef HAVE_VARIADIC_MACROS
 
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4 2/9] read-cache.c: fix constness of verify_hdr()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 read-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/read-cache.c b/read-cache.c
index 84616c8..a76c789 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1345,7 +1345,7 @@ struct ondisk_cache_entry_extended {
ondisk_cache_entry_extended_size(ce_namelen(ce)) : \
ondisk_cache_entry_size(ce_namelen(ce)))
 
-static int verify_hdr(struct cache_header *hdr, unsigned long size)
+static int verify_hdr(const struct cache_header *hdr, unsigned long size)
 {
git_SHA_CTX c;
unsigned char sha1[20];
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sun, Nov 1, 2015 at 11:22 AM, Edmundo Carmona Antoranz
 wrote:
> +   /*
> +* Final processing of show_progress
> +* - User selected --progress: show progress
> +* - user selected --no-progress: skip progress
> +* - User didn't specify:
> +* (check rules in order till finding the first matching one)
> +* - user selected --quiet: skip progress
> +* - stderr is connected to a terminal: show progress
> +* - fallback: skip progress
> +*/
> +   if (opts.show_progress < 0) {
> +   /* user selected --progress or didn't specify */
> +   if (opts.quiet) {
> +   opts.show_progress = 0;
> +   } else {
> +   opts.show_progress = isatty(2);
> +   }
> +   }
> +


/* user selected --progress or didn't specify */
This comment is not correct. Disregard this patch.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sun, Nov 1, 2015 at 11:52 AM, Eric Sunshine  wrote:
>> +   if (opts.quiet) {
>> +   opts.show_progress = 0;
>> +   } else {
>> +   opts.show_progress = isatty(2);
>> +   }
>
> Style: drop unnecessary braces
>

Ok. WIll do!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Eric Sunshine]

On Sun, Nov 1, 2015 at 2:19 PM, Jeff King  wrote:
> On Sun, Nov 01, 2015 at 12:52:57PM -0500, Eric Sunshine wrote:
>> > +--progress::
>> > +   Progress status is reported on the standard error stream
>> > +   by default when it is attached to a terminal, unless -q
>> > +   is specified. This flag forces progress status even if the
>> > +   standard error stream is not directed to a terminal.
>>
>> What this kind of implies, but neglects to say explicitly, is that the
>> logic implemented by this patch also overrides --quiet. It probably
>> should say so explicitly.
>
> I was the one who suggested originally that --progress should override
> --quiet[1]. However, that was just based on what I thought was
> reasonable. I didn't look at what clone or other commands do.
> Consistency between commands is probably more important than any
> particular behavior.

I did check git-clone during my v4 review, and --progress does
override --quiet (see transport.c:transport_set_verbosity()), so your
suggestion to make --progress override --quiet in git-checkout at
least keeps it consistent.

Aside: Specifying both --quiet and --progress with git-clone doesn't
give great results since (it seems) that no flushing is done until the
newline is emitted, thus you only see the end result rather than a
running progress meter.

> [1] To be honest, this is kind of a crazy corner case anyway. It was
> more just that it has to do _something_.

An alternative would be to error out stating that --quiet and
--progress are mutually exclusive.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] http.c: use CURLOPT_RANGE for range requests

[Daniel Stenberg]

On Fri, 30 Oct 2015, Jeff King wrote:

The goal makes sense. Why weren't we using CURLOPT_RANGE before? Did it not 
exist (or otherwise have limitations) in 2005, and if so, when did it become 
usable? Do we need to protect this with an #ifdef for the curl version?


CURLOPT_RANGE existed already in the first libcurl release: version 7.1, 
relased in August 2000.


--

 / daniel.haxx.se
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

git.git as of tonight

[Junio C Hamano]

I've merged a handful of topics to 'next', and hoping that many of
them can be merged to 'master' before I'll go offline for a few
weeks starting on the week #7 of the cycle (starting Nov 9th).

As I hate sending out the whole text back to back (the last one was
issued on the last Friday), here is an abbreviated update for the
"What's cooking" report, highlighting those topics that I want to
see in 'master' by the end of the week #6 (Nov 8th).

Thanks.


* kn/for-each-branch (2015-10-30) 1 commit
  (merged to 'next' on 2015-11-01 at 4249dc9)
 + ref-filter: fallback on alphabetical comparison

 Using the timestamp based criteria in "git branch --sort" did not
 tiebreak branches that point at commits with the same timestamp (or
 the same commit), making the resulting output unstable.

 Will merge to 'master'.


* jc/mailinfo-lib (2015-11-01) 1 commit
  (merged to 'next' on 2015-11-01 at 3ecaa28)
 + mailinfo: fix passing wrong address to git_mailinfo_config

 Hotfix for a topic already in 'master'.

 Will merge to 'master'.


* sb/submodule-parallel-fetch (2015-10-21) 14 commits
  (merged to 'next' on 2015-10-23 at 8f04bbd)
 + run-command: fix missing output from late callbacks
 + test-run-command: increase test coverage
 + test-run-command: test for gracefully aborting
 + run-command: initialize the shutdown flag
 + run-command: clear leftover state from child_process structure
 + run-command: fix early shutdown
  (merged to 'next' on 2015-10-15 at df63590)
 + submodules: allow parallel fetching, add tests and documentation
 + fetch_populated_submodules: use new parallel job processing
 + run-command: add an asynchronous parallel child processor
 + sigchain: add command to pop all common signals
 + strbuf: add strbuf_read_once to read without blocking
 + xread_nonblock: add functionality to read from fds without blocking
 + xread: poll on non blocking fds
 + submodule.c: write "Fetching submodule " to stderr
 (this branch is used by rs/daemon-leak-fix and sb/submodule-parallel-update.)

 Add a framework to spawn a group of processes in parallel, and use
 it to run "git fetch --recurse-submodules" in parallel.

 Will merge to 'master'.


* rs/daemon-leak-fix (2015-10-31) 3 commits
  (merged to 'next' on 2015-11-01 at 9b6c8f9)
 + daemon: plug memory leak
 + run-command: export child_process_clear()
 + run-command: name the cleanup function child_process_clear()
 (this branch is used by sb/submodule-parallel-update; uses 
sb/submodule-parallel-fetch.)

 "git daemon" uses "run_command()" without "finish_command()", so it
 needs to release resources itself, which it forgot to do.

 Will merge to 'master'.


* rs/wt-status-detached-branch-fix (2015-11-01) 5 commits
  (merged to 'next' on 2015-11-01 at cb23615)
 + wt-status: use skip_prefix() to get rid of magic string length constants
 + wt-status: don't skip a magical number of characters blindly
 + wt-status: avoid building bogus branch name with detached HEAD
 + wt-status: exit early using goto in wt_shortstatus_print_tracking()
 + t7060: add test for status --branch on a detached HEAD

 "git status --branch --short" accessed beyond the constant string
 "HEAD", which has been corrected.

 Will merge to 'master'.


* da/difftool (2015-10-29) 1 commit
  (merged to 'next' on 2015-11-01 at 4e5ab33)
 + difftool: ignore symbolic links in use_wt_file

 The code to prepare the working tree side of temporary directory
 for the "dir-diff" feature forgot that symbolic links need not be
 copied (or symlinked) to the temporary area, as the code already
 special cases and overwrites them.  Besides, it was wrong to try
 computing the object name of the target of symbolic link, which may
 not even exist or may be a directory.

 Will merge to 'master'.


* jk/initialization-fix-to-add-submodule-odb (2015-10-28) 1 commit
  (merged to 'next' on 2015-11-01 at da94b97)
 + add_submodule_odb: initialize alt_odb list earlier

 We peek objects from submodule's object store by linking it to the
 list of alternate object databases, but the code to do so forgot to
 correctly initialize the list.

 Will merge to 'master'.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v4] Add git-grep threads param

[Victor Leschuk]

Hello all,

do we have any objections on this patch?

--
Best Regards,
Victor

From: Victor Leschuk [vlesc...@gmail.com]
Sent: Tuesday, October 27, 2015 14:22
To: git@vger.kernel.org
Cc: Victor Leschuk
Subject: [PATCH v4] Add git-grep threads param

Make number of git-grep worker threads a configuration parameter.
According to several tests on systems with different number of CPU cores
the hard-coded number of 8 threads is not optimal for all systems:
tuning this parameter can significantly speed up grep performance.

Signed-off-by: Victor Leschuk 
---
 Documentation/config.txt   |  4 +++
 Documentation/git-grep.txt |  9 ++
 builtin/grep.c | 56 --
 contrib/completion/git-completion.bash |  1 +
 4 files changed, 54 insertions(+), 16 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 391a0c3..1dd2a61 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1447,6 +1447,10 @@ grep.extendedRegexp::
option is ignored when the 'grep.patternType' option is set to a value
other than 'default'.

+grep.threads::
+   Number of grep worker threads, use it to tune up performance on
+   multicore machines. Default value is 8. Set to 0 to disable threading.
+
 gpg.program::
Use this custom program instead of "gpg" found on $PATH when
making or verifying a PGP signature. The program must support the
diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index 4a44d6d..e766596 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -23,6 +23,7 @@ SYNOPSIS
   [--break] [--heading] [-p | --show-function]
   [-A ] [-B ] [-C ]
   [-W | --function-context]
+  [--threads ]
   [-f ] [-e] 
   [--and|--or|--not|(|)|-e ...]
   [ [--[no-]exclude-standard] [--cached | --no-index | --untracked] | 
...]
@@ -53,6 +54,10 @@ grep.extendedRegexp::
option is ignored when the 'grep.patternType' option is set to a value
other than 'default'.

+grep.threads::
+   Number of grep worker threads, use it to tune up performance on
+   multicore machines. Default value is 8. Set to 0 to disable threading.
+
 grep.fullName::
If set to true, enable '--full-name' option by default.

@@ -227,6 +232,10 @@ OPTIONS
effectively showing the whole function in which the match was
found.

+--threads ::
+   Set number of worker threads to . Default is 8.
+   Set to 0 to disable threading.
+
 -f ::
Read patterns from , one per line.

diff --git a/builtin/grep.c b/builtin/grep.c
index d04f440..694553e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -24,11 +24,11 @@ static char const * const grep_usage[] = {
NULL
 };

-static int use_threads = 1;
+#define GREP_NUM_THREADS_DEFAULT 8
+static int num_threads = -1;

 #ifndef NO_PTHREADS
-#define THREADS 8
-static pthread_t threads[THREADS];
+static pthread_t *threads;

 /* We use one producer thread and THREADS consumer
  * threads. The producer adds struct work_items to 'todo' and the
@@ -63,13 +63,13 @@ static pthread_mutex_t grep_mutex;

 static inline void grep_lock(void)
 {
-   if (use_threads)
+   if (num_threads)
pthread_mutex_lock(_mutex);
 }

 static inline void grep_unlock(void)
 {
-   if (use_threads)
+   if (num_threads)
pthread_mutex_unlock(_mutex);
 }

@@ -206,7 +206,8 @@ static void start_threads(struct grep_opt *opt)
strbuf_init([i].out, 0);
}

-   for (i = 0; i < ARRAY_SIZE(threads); i++) {
+   threads = xcalloc(num_threads, sizeof(pthread_t));
+   for (i = 0; i < num_threads; i++) {
int err;
struct grep_opt *o = grep_opt_dup(opt);
o->output = strbuf_out;
@@ -238,12 +239,14 @@ static int wait_all(void)
pthread_cond_broadcast(_add);
grep_unlock();

-   for (i = 0; i < ARRAY_SIZE(threads); i++) {
+   for (i = 0; i < num_threads; i++) {
void *h;
pthread_join(threads[i], );
hit |= (int) (intptr_t) h;
}

+   free(threads);
+
pthread_mutex_destroy(_mutex);
pthread_mutex_destroy(_read_mutex);
pthread_mutex_destroy(_attr_mutex);
@@ -262,10 +265,22 @@ static int wait_all(void)
 }
 #endif

+static int grep_threads_config(const char *var, const char *value, void *cb)
+{
+   if (!strcmp(var, "grep.threads")) {
+   num_threads = git_config_int(var, value);
+   if (num_threads < 0)
+   die("Invalid number of threads specified (%d)", 
num_threads);
+   }
+   return 0;
+}
+
 static int grep_cmd_config(const char *var, const char *value, void *cb)
 {
int st = grep_config(var, value, cb);
-   if 

Re: [PATCH v2] http.c: use CURLOPT_RANGE for range requests

[Jeff King]

On Mon, Nov 02, 2015 at 12:00:42AM +0100, Daniel Stenberg wrote:

> On Fri, 30 Oct 2015, Jeff King wrote:
> 
> >The goal makes sense. Why weren't we using CURLOPT_RANGE before? Did it
> >not exist (or otherwise have limitations) in 2005, and if so, when did it
> >become usable? Do we need to protect this with an #ifdef for the curl
> >version?
> 
> CURLOPT_RANGE existed already in the first libcurl release: version 7.1,
> relased in August 2000.

Ah, thanks. I guess we don't have to worry about that, then.

While I have your attention, Daniel, am I correct in assuming that
performing a second unrelated request with the same CURL object will
need an explicit:

  curl_easy_setopt(curl, CURLOPT_RANGE, NULL);

to avoid using the range twice?

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] ls-files: Add eol diagnostics

[Sebastian Schuberth]

On Sun, Nov 1, 2015 at 7:40 PM, Matthieu Moy
 wrote:

>> Any I find it a bit confusing to refer to the index where, as e.g. for
>> a freshly cloned repo the index should be empty,
>
> No it is not. The index is a complete snapshot of your working tree.
> When you have no uncommited staged changes, the index contains all files
> that are in HEAD. Most commands show you _changes_ in the index (wrt
> HEAD or wrt the working tree), but the index itself contain all files.

Thanks for the info.

> At stage 4), you really want to see the content of the index, because
> your HEAD is still broken.

Ok, I'm convinced. Thanks again!

-- 
Sebastian Schuberth
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v8] checkout: add --progress option

[Edmundo Carmona Antoranz]

Under normal circumstances, and like other git commands,
git checkout will write progress info to stderr if
attached to a terminal. This option allows progress
to be forced even if not using a terminal. Also,
progress can be skipped if using option --no-progress.

Signed-off-by: Edmundo Carmona Antoranz 
---
 Documentation/git-checkout.txt |  6 ++
 builtin/checkout.c | 14 --
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index e269fb1..5e5273e 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -107,6 +107,12 @@ OPTIONS
 --quiet::
Quiet, suppress feedback messages.
 
+--[no-]progress::
+   Progress status is reported on the standard error stream
+   by default when it is attached to a terminal, unless `--quiet`
+   is specified. This flag enables progress reporting even if not
+   attached to a terminal, regardless of `--quiet`.
+
 -f::
 --force::
When switching branches, proceed even if the index or the
diff --git a/builtin/checkout.c b/builtin/checkout.c
index bc703c0..e346f52 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
int overwrite_ignore;
int ignore_skipworktree;
int ignore_other_worktrees;
+   int show_progress;
 
const char *new_branch;
const char *new_branch_force;
@@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
checkout_opts *o,
opts.reset = 1;
opts.merge = 1;
opts.fn = oneway_merge;
-   opts.verbose_update = !o->quiet && isatty(2);
+   opts.verbose_update = o->show_progress;
opts.src_index = _index;
opts.dst_index = _index;
parse_tree(tree);
@@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
*opts,
topts.update = 1;
topts.merge = 1;
topts.gently = opts->merge && old->commit;
-   topts.verbose_update = !opts->quiet && isatty(2);
+   topts.verbose_update = opts->show_progress;
topts.fn = twoway_merge;
if (opts->overwrite_ignore) {
topts.dir = xcalloc(1, sizeof(*topts.dir));
@@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
N_("second guess 'git checkout 
'")),
OPT_BOOL(0, "ignore-other-worktrees", 
_other_worktrees,
 N_("do not check if another worktree is holding the 
given ref")),
+   OPT_BOOL(0, "progress", _progress, N_("force progress 
reporting")),
OPT_END(),
};
 
@@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
memset(, 0, sizeof(new));
opts.overwrite_ignore = 1;
opts.prefix = prefix;
+   opts.show_progress = -1;
 
gitmodules_config();
git_config(git_checkout_config, );
@@ -1172,6 +1175,13 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
argc = parse_options(argc, argv, prefix, options, checkout_usage,
 PARSE_OPT_KEEP_DASHDASH);
 
+   if (opts.show_progress < 0) {
+   if (opts.quiet)
+   opts.show_progress = 0;
+   else
+   opts.show_progress = isatty(2);
+   }
+
if (conflict_style) {
opts.merge = 1; /* implied */
git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

configure: -lpthread doesn't belong in CFLAGS

[Rainer M. Canavan]

Hi,

some linkers, namely the one on IRIX are rather strict concerning the 
order or arguments for symbol resolution, i.e. no libraries listed
before objects or other libraries on the command line are considered
for symbol resolution. That means that -lpthread can't work if it's 
put in CFLAGS, because it will not be considered for resolving 
pthread_key_create in conftest.o. Apparently only $LIBS goes
after conftest.o when the linker is called.

Without the patch below, the POSIX Threads with '-lpthread' fails, 
with the patch it succeeds. I haven't checked if that is relevant
at all, since that check is immediately followed by 
checking for pthread_create in -lpthread... yes


regards,


rainer


--- ../src/git-2.6.2/configure.ac   Fri Oct 16 23:58:26 CEST 2015
+++ configure.acSun Nov 01 23:19:41 CET 2015
@@ -1126,7 +1126,13 @@
   # would then trigger compiler warnings on every single file we compile.
   for opt in "" -mt -pthread -lpthread; do
  old_CFLAGS="$CFLAGS"
- CFLAGS="$opt $CFLAGS"
+ old_LIBS="$LIBS"
+ if test "$(echo $opt | cut -b 1-2)" = -l ; then
+LIBS="$opt $LIBS"
+ else
+CFLAGS="$opt $CFLAGS"
+ fi
+
  AC_MSG_CHECKING([for POSIX Threads with '$opt'])
  AC_LINK_IFELSE([PTHREADTEST_SRC],
[AC_MSG_RESULT([yes])
@@ -1138,6 +1144,7 @@
],
[AC_MSG_RESULT([no])])
   CFLAGS="$old_CFLAGS"
+  LIBS="$old_LIBS"
   done
   if test $threads_found != yes; then
 AC_CHECK_LIB([pthread], [pthread_create],
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Allow hideRefs to match refs outside the namespace

[Junio C Hamano]

Lukas Fleischer  writes:

> Now, this cannot be intended behavior and I do not think this is
> something we want to retain when improving that feature.

Yup, that makes me suspect that namespace support with hiderefs was
done without giving much thought even stronger than before, and the
fact that nobody has brought it up so far suggests it would be much
smaller deal than usual if a fix brings in incompatibilities to
those who use namespaces.

> 1. Define the (current) semantics of hideRefs pattern. It either needs
>to be defined to match full references or stripped references. Both
>definitions are equivalent when Git namespaces are not used.
>
>It probably makes sense to define hideRefs patterns to match stripped
>references.

OK.

> 2. Improve the documentation and describe the hideRefs semantics better.
> 3. Fix the send_ref() code in either receive-pack or upload-pack,
> 4. Improve hideRefs patterns and allow to match both full references and
> 5. Add a note on the change in behavior to the release notes of the

All OK.

> The second thing I noticed is that having syntax for allowing matches
> against both full references and stripped references is extremely handy
> and desirable, even if we would not have to introduce it for backwards
> compatibility. For example, using the syntax Junio described earlier, my
> initial use case could be solved by
>
> receive.hideRefs=^refs/
> receive.hideRefs=!refs/
>
> which means "Hide all references but do not hide references from the
> current namespace." Here, I am assuming that patterns for stripped refs
> never match anything outside the current namespace because those
> patterns become NULL after stripping.

I would instead assume that the presence of ^ (or !^) in front would
signal "do not strip before checking".  !refs/ would mean "after
stripping, does it begin with refs/?  If so then do not hide it".

But that does not change the conclusion.  With ^refs/ that says
"hide everything that matches refs/ before stripping" (i.e. do not
include anything from anywhere), that is overriden by !refs/ that
says "but do not hide anything that matches refs/ after stripping"
(do include everything from my namespace), I'd think that you'd get
your desired behaviour.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v7] checkout: add --progress option

[Edmundo Carmona Antoranz]

Under normal circumstances, and like other git commands,
git checkout will write progress info to stderr if
attached to a terminal. This option allows progress
to be forced even if not using a terminal. Also,
progress can be skipped if using option --no-progress.

Signed-off-by: Edmundo Carmona Antoranz 
---
 Documentation/git-checkout.txt |  6 ++
 builtin/checkout.c | 14 --
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index e269fb1..9973cf6 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -107,6 +107,12 @@ OPTIONS
 --quiet::
Quiet, suppress feedback messages.
 
+--[no-]progress::
+   Progress status is reported on the standard error stream
+   by default when it is attached to a terminal, unless `--quiet`
+   is specified. This flag enables progress reporting even if not
+   attached to a terminal, regardless of `--quite`.
+
 -f::
 --force::
When switching branches, proceed even if the index or the
diff --git a/builtin/checkout.c b/builtin/checkout.c
index bc703c0..e346f52 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
int overwrite_ignore;
int ignore_skipworktree;
int ignore_other_worktrees;
+   int show_progress;
 
const char *new_branch;
const char *new_branch_force;
@@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
checkout_opts *o,
opts.reset = 1;
opts.merge = 1;
opts.fn = oneway_merge;
-   opts.verbose_update = !o->quiet && isatty(2);
+   opts.verbose_update = o->show_progress;
opts.src_index = _index;
opts.dst_index = _index;
parse_tree(tree);
@@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
*opts,
topts.update = 1;
topts.merge = 1;
topts.gently = opts->merge && old->commit;
-   topts.verbose_update = !opts->quiet && isatty(2);
+   topts.verbose_update = opts->show_progress;
topts.fn = twoway_merge;
if (opts->overwrite_ignore) {
topts.dir = xcalloc(1, sizeof(*topts.dir));
@@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
N_("second guess 'git checkout 
'")),
OPT_BOOL(0, "ignore-other-worktrees", 
_other_worktrees,
 N_("do not check if another worktree is holding the 
given ref")),
+   OPT_BOOL(0, "progress", _progress, N_("force progress 
reporting")),
OPT_END(),
};
 
@@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
memset(, 0, sizeof(new));
opts.overwrite_ignore = 1;
opts.prefix = prefix;
+   opts.show_progress = -1;
 
gitmodules_config();
git_config(git_checkout_config, );
@@ -1172,6 +1175,13 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
argc = parse_options(argc, argv, prefix, options, checkout_usage,
 PARSE_OPT_KEEP_DASHDASH);
 
+   if (opts.show_progress < 0) {
+   if (opts.quiet)
+   opts.show_progress = 0;
+   else
+   opts.show_progress = isatty(2);
+   }
+
if (conflict_style) {
opts.merge = 1; /* implied */
git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v6] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sun, Nov 1, 2015 at 3:06 PM, Eric Sunshine  wrote:
>> +--[no-]progress::
>> +   Progress status is reported on the standard error stream
>> +   by default when it is attached to a terminal, unless --quiet
>> +   is specified. This flag enables progress reporting even if not
>> +   attached to a terminal, regardless of -q.
>
> The mix of -q and --quiet is inconsistent and potentially confusing. I
> suspect that your intention was to hint that they are equivalent,
> however, the reader who is not familiar with -q as an alias of --quiet
> may now be forced to look up both options, rather than just one, only
> to discover that they are the same, thus potentially requiring extra
> effort. It probably would be better to consistently use --quiet.
>
> Also, quoting with backticks is recommended: `--quite`
>
> The rest of the patch looks good.

Good! Going for v7.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/4] Add support for matching full refs in hideRefs

[Lukas Fleischer]

In addition to matching stripped refs, one can now add hideRefs patterns
that the full (unstripped) ref is matched against. To distinguish
between stripped and full matches, those new patterns must be prefixed
with a circumflex (^).

This commit also removes support for the undocumented and unintended
hideRefs settings "have" (suppressing all "have" lines) and
"capabilities^{}" (suppressing the capabilities line).

Signed-off-by: Lukas Fleischer 
---
 Documentation/config.txt |  3 ++-
 builtin/receive-pack.c   | 22 --
 refs.c   | 14 +++---
 refs.h   |  2 +-
 upload-pack.c| 13 -
 5 files changed, 38 insertions(+), 16 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 3da97a1..91ed6a5 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2690,7 +2690,8 @@ the prefix `refs/heads/master` is specified in 
`transfer.hideRefs` and the
 current namespace is `foo`, then `refs/namespaces/foo/refs/heads/master` is
 omitted from the advertisements but `refs/heads/master` and
 `refs/namespaces/bar/refs/heads/master` are still advertised as so-called
-"have" lines.
+"have" lines. In order to match refs before stripping, add a `^` in front of
+the ref name. If you combine `!` and `^`, `!` must be specified first.
 
 transfer.unpackLimit::
When `fetch.unpackLimit` or `receive.unpackLimit` are
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index bcb624b..d5e58e0 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -195,9 +195,6 @@ static int receive_pack_config(const char *var, const char 
*value, void *cb)
 
 static void show_ref(const char *path, const unsigned char *sha1)
 {
-   if (ref_is_hidden(path))
-   return;
-
if (sent_capabilities) {
packet_write(1, "%s %s\n", sha1_to_hex(sha1), path);
} else {
@@ -219,9 +216,14 @@ static void show_ref(const char *path, const unsigned char 
*sha1)
}
 }
 
-static int show_ref_cb(const char *path, const struct object_id *oid, int 
flag, void *unused)
+static int show_ref_cb(const char *path_full, const struct object_id *oid,
+  int flag, void *unused)
 {
-   path = strip_namespace(path);
+   const char *path = strip_namespace(path_full);
+
+   if (ref_is_hidden(path, path_full))
+   return 1;
+
/*
 * Advertise refs outside our current namespace as ".have"
 * refs, so that the client can use them to minimize data
@@ -1198,7 +1200,15 @@ static void reject_updates_to_hidden(struct command 
*commands)
struct command *cmd;
 
for (cmd = commands; cmd; cmd = cmd->next) {
-   if (cmd->error_string || !ref_is_hidden(cmd->ref_name))
+   const char *refname = cmd->ref_name;
+   struct strbuf refname_full_buf = STRBUF_INIT;
+   const char *refname_full;
+
+   strbuf_addf(_full_buf, "%s%s", get_git_namespace(),
+   refname);
+   refname_full = strbuf_detach(_full_buf, NULL);
+
+   if (cmd->error_string || !ref_is_hidden(refname, refname_full))
continue;
if (is_null_sha1(cmd->new_sha1))
cmd->error_string = "deny deleting a hidden ref";
diff --git a/refs.c b/refs.c
index 72d96ed..555c9d5 100644
--- a/refs.c
+++ b/refs.c
@@ -321,7 +321,7 @@ int parse_hide_refs_config(const char *var, const char 
*value, const char *secti
return 0;
 }
 
-int ref_is_hidden(const char *refname)
+int ref_is_hidden(const char *refname, const char *refname_full)
 {
int i;
 
@@ -329,6 +329,7 @@ int ref_is_hidden(const char *refname)
return 0;
for (i = hide_refs->nr - 1; i >= 0; i--) {
const char *match = hide_refs->items[i].string;
+   const char *subject;
int neg = 0;
int len;
 
@@ -337,10 +338,17 @@ int ref_is_hidden(const char *refname)
match++;
}
 
-   if (!starts_with(refname, match))
+   if (*match == '^') {
+   subject = refname_full;
+   match++;
+   } else {
+   subject = refname;
+   }
+
+   if (!subject || !starts_with(subject, match))
continue;
len = strlen(match);
-   if (!refname[len] || refname[len] == '/')
+   if (!subject[len] || subject[len] == '/')
return !neg;
}
return 0;
diff --git a/refs.h b/refs.h
index 69fa4df..fbf8e59 100644
--- a/refs.h
+++ b/refs.h
@@ -604,7 +604,7 @@ int update_ref(const char *msg, const char *refname,
 
 extern int parse_hide_refs_config(const char *var, const char *value, const 
char *);
 
-extern int 

[PATCH 1/4] Document the semantics of hideRefs with namespaces

[Lukas Fleischer]

Right now, there is no clear definition of how transfer.hideRefs should
behave when a namespace is set. Explain that hideRefs prefixes match
stripped names in that case. This is how hideRefs patterns are currently
handled in receive-pack.

Signed-off-by: Lukas Fleischer 
---
 Documentation/config.txt | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 1204072..3da97a1 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2684,6 +2684,13 @@ You may also include a `!` in front of the ref name to 
negate the entry,
 explicitly exposing it, even if an earlier entry marked it as hidden.
 If you have multiple hideRefs values, later entries override earlier ones
 (and entries in more-specific config files override less-specific ones).
++
+If a namespace is set, references are stripped before matching. For example, if
+the prefix `refs/heads/master` is specified in `transfer.hideRefs` and the
+current namespace is `foo`, then `refs/namespaces/foo/refs/heads/master` is
+omitted from the advertisements but `refs/heads/master` and
+`refs/namespaces/bar/refs/heads/master` are still advertised as so-called
+"have" lines.
 
 transfer.unpackLimit::
When `fetch.unpackLimit` or `receive.unpackLimit` are
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] t5509: add basic tests for hideRefs

[Lukas Fleischer]

Test whether regular and full hideRefs patterns work as expected when
namespaces are used.

Signed-off-by: Lukas Fleischer 
---
 t/t5509-fetch-push-namespaces.sh | 29 +
 1 file changed, 29 insertions(+)

diff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh
index cc0b31f..a3f1060 100755
--- a/t/t5509-fetch-push-namespaces.sh
+++ b/t/t5509-fetch-push-namespaces.sh
@@ -82,4 +82,33 @@ test_expect_success 'mirroring a repository using a ref 
namespace' '
)
 '
 
+test_expect_success "Hide namespaced refs with transfer.hideRefs" '
+   cd pushee &&
+   test_config transfer.hideRefs refs/tags &&
+   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
+   printf "$commit1\trefs/heads/master\n" >expected &&
+   test_cmp expected actual &&
+   cd ..
+'
+
+test_expect_success "Check that transfer.hideRefs does not match unstripped 
refs" '
+   cd pushee &&
+   test_config transfer.hideRefs "refs/namespaces/namespace/refs/tags" &&
+   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
+   printf "$commit1\trefs/heads/master\n" >expected &&
+   printf "$commit0\trefs/tags/0\n" >>expected &&
+   printf "$commit1\trefs/tags/1\n" >>expected &&
+   test_cmp expected actual &&
+   cd ..
+'
+
+test_expect_success "Hide full refs with transfer.hideRefs" '
+   cd pushee &&
+   test_config transfer.hideRefs "^refs/namespaces/namespace/refs/tags" &&
+   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
+   printf "$commit1\trefs/heads/master\n" >expected &&
+   test_cmp expected actual &&
+   cd ..
+'
+
 test_done
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/4] upload-pack: strip refs before calling ref_is_hidden()

[Lukas Fleischer]

Make hideRefs handling in upload-pack consistent with the behavior
described in the documentation by stripping refs before comparing them
with prefixes in hideRefs.

Signed-off-by: Lukas Fleischer 
---
 upload-pack.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/upload-pack.c b/upload-pack.c
index d0bc3ca..4ca960e 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -692,7 +692,7 @@ static int mark_our_ref(const char *refname, const struct 
object_id *oid)
 {
struct object *o = lookup_unknown_object(oid->hash);
 
-   if (ref_is_hidden(refname)) {
+   if (refname && ref_is_hidden(refname)) {
o->flags |= HIDDEN_REF;
return 1;
}
@@ -703,7 +703,7 @@ static int mark_our_ref(const char *refname, const struct 
object_id *oid)
 static int check_ref(const char *refname, const struct object_id *oid,
 int flag, void *cb_data)
 {
-   mark_our_ref(refname, oid);
+   mark_our_ref(strip_namespace(refname), oid);
return 0;
 }
 
@@ -726,7 +726,7 @@ static int send_ref(const char *refname, const struct 
object_id *oid,
const char *refname_nons = strip_namespace(refname);
struct object_id peeled;
 
-   if (mark_our_ref(refname, oid))
+   if (mark_our_ref(refname_nons, oid))
return 0;
 
if (capabilities) {
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Eric Sunshine]

On Sun, Nov 1, 2015 at 2:35 PM, Edmundo Carmona Antoranz
 wrote:
> On Sun, Nov 1, 2015 at 1:29 PM, Eric Sunshine  wrote:
 > +--progress::
 > +   Progress status is reported on the standard error stream
 > +   by default when it is attached to a terminal, unless -q
 > +   is specified. This flag forces progress status even if the
 > +   standard error stream is not directed to a terminal.
>
> Before I send a new full patch, could you guys tell me if you find this ok?
>
> -q::
> --quiet::
> Quiet, suppress feedback messages. Progress can skip this option.
> Read the information about --progress

IMHO, this sort of corner case probably doesn't deserve being called
out specially. Also, since --progress immediately follows --quiet in
the documentation, it's quite easy to discover --progress and read
about its behavior. Thus, I'd leave the description of --quiet as is.

> --[no-]progress::
> Progress status is reported on the standard error stream
> by default when it is attached to a terminal, unless --quiet
> or -q is specified. This flag forces progress status even if the
> standard error stream is not directed to a terminal and overrides
> the --quiet or -q options.

No need to make the description long by mentioning both -q and
--quiet. (My v4 review suggested only that you quote -q or --quiet
with backticks; not that you mention both.) The reader can easily
discover the alias of -q or --quiet, so mentioning one or the other
should be sufficient. A bit shorter and sweeter, perhaps:

Progress status is reported on the standard error stream by
default when it is attached to a terminal, unless `--quiet` is
specified. This flag enables progress reporting even if not
attached to a terminal, regardless of `--quiet`.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/4] t5509: add basic tests for hideRefs

[Eric Sunshine]

On Sun, Nov 1, 2015 at 2:34 PM, Lukas Fleischer  wrote:
> Test whether regular and full hideRefs patterns work as expected when
> namespaces are used.
>
> Signed-off-by: Lukas Fleischer 
> ---
> diff --git a/t/t5509-fetch-push-namespaces.sh 
> b/t/t5509-fetch-push-namespaces.sh
> @@ -82,4 +82,33 @@ test_expect_success 'mirroring a repository using a ref 
> namespace' '
> )
>  '
>
> +test_expect_success "Hide namespaced refs with transfer.hideRefs" '

None of the other tests in this file capitalize the test description.
These new test descriptions should probably follow suit by beginning
with lowercase. It is also typical to use single quotes for the
description rather than double.

> +   cd pushee &&
> +   test_config transfer.hideRefs refs/tags &&
> +   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
> +   printf "$commit1\trefs/heads/master\n" >expected &&
> +   test_cmp expected actual &&
> +   cd ..

If any of the commands above "cd .." fail, then "cd .." will never be
invoked, thus subsequent tests will fail since they won't be executed
in the expected directory. The typical way to handle this is to place
the "cd foo" and remaining test body in a subshell, and drop "cd .."
altogether. When the subshell exits (via success or failure), the
working directory will be restored automatically.

test_expect_success '...' '
(
cd pushee &&
test_config ... &&
...
)
'

> +'
> +
> +test_expect_success "Check that transfer.hideRefs does not match unstripped 
> refs" '
> +   cd pushee &&
> +   test_config transfer.hideRefs "refs/namespaces/namespace/refs/tags" &&
> +   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
> +   printf "$commit1\trefs/heads/master\n" >expected &&
> +   printf "$commit0\trefs/tags/0\n" >>expected &&
> +   printf "$commit1\trefs/tags/1\n" >>expected &&
> +   test_cmp expected actual &&
> +   cd ..
> +'
> +
> +test_expect_success "Hide full refs with transfer.hideRefs" '
> +   cd pushee &&
> +   test_config transfer.hideRefs "^refs/namespaces/namespace/refs/tags" 
> &&
> +   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
> +   printf "$commit1\trefs/heads/master\n" >expected &&
> +   test_cmp expected actual &&
> +   cd ..
> +'
> +
>  test_done
> --
> 2.6.2
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7] checkout: add --progress option

[Eric Sunshine]

On Sun, Nov 1, 2015 at 4:12 PM, Edmundo Carmona Antoranz
 wrote:
> Under normal circumstances, and like other git commands,
> git checkout will write progress info to stderr if
> attached to a terminal. This option allows progress
> to be forced even if not using a terminal. Also,
> progress can be skipped if using option --no-progress.
>
> Signed-off-by: Edmundo Carmona Antoranz 
> ---
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> @@ -107,6 +107,12 @@ OPTIONS
>  --quiet::
> Quiet, suppress feedback messages.
>
> +--[no-]progress::
> +   Progress status is reported on the standard error stream
> +   by default when it is attached to a terminal, unless `--quiet`
> +   is specified. This flag enables progress reporting even if not
> +   attached to a terminal, regardless of `--quite`.

s/quite/quiet/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/4] Improve hideRefs when used with namespaces

[Lukas Fleischer]

This is a first set of patches improving documentation and behavior of
the transfer.hideRefs feature as discussed in [1]. In particular,
hideRefs is changed to generally match stripped refs by default and
match full refs when prefixed with "^". The documentation is updated
accordingly. Basic tests are added.

[1] http://marc.info/?l=git=144604694223920

Lukas Fleischer (4):
  Document the semantics of hideRefs with namespaces
  upload-pack: strip refs before calling ref_is_hidden()
  Add support for matching full refs in hideRefs
  t5509: add basic tests for hideRefs

 Documentation/config.txt |  8 
 builtin/receive-pack.c   | 22 --
 refs.c   | 14 +++---
 refs.h   |  2 +-
 t/t5509-fetch-push-namespaces.sh | 29 +
 upload-pack.c| 13 -
 6 files changed, 73 insertions(+), 15 deletions(-)

-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sun, Nov 1, 2015 at 1:29 PM, Eric Sunshine  wrote:
>>> > +--progress::
>>> > +   Progress status is reported on the standard error stream
>>> > +   by default when it is attached to a terminal, unless -q
>>> > +   is specified. This flag forces progress status even if the
>>> > +   standard error stream is not directed to a terminal.

Before I send a new full patch, could you guys tell me if you find this ok?

-q::
--quiet::
Quiet, suppress feedback messages. Progress can skip this option.
Read the information about --progress

--[no-]progress::
Progress status is reported on the standard error stream
by default when it is attached to a terminal, unless --quiet
or -q is specified. This flag forces progress status even if the
standard error stream is not directed to a terminal and overrides
the --quiet or -q options.

Thank you all, guys!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6] checkout: add --progress option

[Edmundo Carmona Antoranz]

Under normal circumstances, and like other git commands,
git checkout will write progress info to stderr if
attached to a terminal. This option allows progress
to be forced even if not using a terminal. Also,
progress can be skipped if using option --no-progress.

Signed-off-by: Edmundo Carmona Antoranz 
---
 Documentation/git-checkout.txt |  6 ++
 builtin/checkout.c | 14 --
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index e269fb1..c24c8ee 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -107,6 +107,12 @@ OPTIONS
 --quiet::
Quiet, suppress feedback messages.
 
+--[no-]progress::
+   Progress status is reported on the standard error stream
+   by default when it is attached to a terminal, unless --quiet
+   is specified. This flag enables progress reporting even if not
+   attached to a terminal, regardless of -q.
+
 -f::
 --force::
When switching branches, proceed even if the index or the
diff --git a/builtin/checkout.c b/builtin/checkout.c
index bc703c0..e346f52 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -37,6 +37,7 @@ struct checkout_opts {
int overwrite_ignore;
int ignore_skipworktree;
int ignore_other_worktrees;
+   int show_progress;
 
const char *new_branch;
const char *new_branch_force;
@@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
checkout_opts *o,
opts.reset = 1;
opts.merge = 1;
opts.fn = oneway_merge;
-   opts.verbose_update = !o->quiet && isatty(2);
+   opts.verbose_update = o->show_progress;
opts.src_index = _index;
opts.dst_index = _index;
parse_tree(tree);
@@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
*opts,
topts.update = 1;
topts.merge = 1;
topts.gently = opts->merge && old->commit;
-   topts.verbose_update = !opts->quiet && isatty(2);
+   topts.verbose_update = opts->show_progress;
topts.fn = twoway_merge;
if (opts->overwrite_ignore) {
topts.dir = xcalloc(1, sizeof(*topts.dir));
@@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
N_("second guess 'git checkout 
'")),
OPT_BOOL(0, "ignore-other-worktrees", 
_other_worktrees,
 N_("do not check if another worktree is holding the 
given ref")),
+   OPT_BOOL(0, "progress", _progress, N_("force progress 
reporting")),
OPT_END(),
};
 
@@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
memset(, 0, sizeof(new));
opts.overwrite_ignore = 1;
opts.prefix = prefix;
+   opts.show_progress = -1;
 
gitmodules_config();
git_config(git_checkout_config, );
@@ -1172,6 +1175,13 @@ int cmd_checkout(int argc, const char **argv, const char 
*prefix)
argc = parse_options(argc, argv, prefix, options, checkout_usage,
 PARSE_OPT_KEEP_DASHDASH);
 
+   if (opts.show_progress < 0) {
+   if (opts.quiet)
+   opts.show_progress = 0;
+   else
+   opts.show_progress = isatty(2);
+   }
+
if (conflict_style) {
opts.merge = 1; /* implied */
git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
-- 
2.6.1

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sun, Nov 1, 2015 at 2:15 PM, Eric Sunshine  wrote:
>
> Progress status is reported on the standard error stream by
> default when it is attached to a terminal, unless `--quiet` is
> specified. This flag enables progress reporting even if not
> attached to a terminal, regardless of `--quiet`.

Sounds good to me.

Sending another full patch.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v6] checkout: add --progress option

[Eric Sunshine]

On Sun, Nov 1, 2015 at 3:27 PM, Edmundo Carmona Antoranz
 wrote:
> Under normal circumstances, and like other git commands,
> git checkout will write progress info to stderr if
> attached to a terminal. This option allows progress
> to be forced even if not using a terminal. Also,
> progress can be skipped if using option --no-progress.
>
> Signed-off-by: Edmundo Carmona Antoranz 
> ---
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> @@ -107,6 +107,12 @@ OPTIONS
>  --quiet::
> Quiet, suppress feedback messages.
>
> +--[no-]progress::
> +   Progress status is reported on the standard error stream
> +   by default when it is attached to a terminal, unless --quiet
> +   is specified. This flag enables progress reporting even if not
> +   attached to a terminal, regardless of -q.

The mix of -q and --quiet is inconsistent and potentially confusing. I
suspect that your intention was to hint that they are equivalent,
however, the reader who is not familiar with -q as an alias of --quiet
may now be forced to look up both options, rather than just one, only
to discover that they are the same, thus potentially requiring extra
effort. It probably would be better to consistently use --quiet.

Also, quoting with backticks is recommended: `--quite`

The rest of the patch looks good.

>  -f::
>  --force::
> When switching branches, proceed even if the index or the
> diff --git a/builtin/checkout.c b/builtin/checkout.c
> @@ -37,6 +37,7 @@ struct checkout_opts {
> int overwrite_ignore;
> int ignore_skipworktree;
> int ignore_other_worktrees;
> +   int show_progress;
>
> const char *new_branch;
> const char *new_branch_force;
> @@ -417,7 +418,7 @@ static int reset_tree(struct tree *tree, const struct 
> checkout_opts *o,
> opts.reset = 1;
> opts.merge = 1;
> opts.fn = oneway_merge;
> -   opts.verbose_update = !o->quiet && isatty(2);
> +   opts.verbose_update = o->show_progress;
> opts.src_index = _index;
> opts.dst_index = _index;
> parse_tree(tree);
> @@ -501,7 +502,7 @@ static int merge_working_tree(const struct checkout_opts 
> *opts,
> topts.update = 1;
> topts.merge = 1;
> topts.gently = opts->merge && old->commit;
> -   topts.verbose_update = !opts->quiet && isatty(2);
> +   topts.verbose_update = opts->show_progress;
> topts.fn = twoway_merge;
> if (opts->overwrite_ignore) {
> topts.dir = xcalloc(1, sizeof(*topts.dir));
> @@ -1156,6 +1157,7 @@ int cmd_checkout(int argc, const char **argv, const 
> char *prefix)
> N_("second guess 'git checkout 
> '")),
> OPT_BOOL(0, "ignore-other-worktrees", 
> _other_worktrees,
>  N_("do not check if another worktree is holding the 
> given ref")),
> +   OPT_BOOL(0, "progress", _progress, N_("force 
> progress reporting")),
> OPT_END(),
> };
>
> @@ -1163,6 +1165,7 @@ int cmd_checkout(int argc, const char **argv, const 
> char *prefix)
> memset(, 0, sizeof(new));
> opts.overwrite_ignore = 1;
> opts.prefix = prefix;
> +   opts.show_progress = -1;
>
> gitmodules_config();
> git_config(git_checkout_config, );
> @@ -1172,6 +1175,13 @@ int cmd_checkout(int argc, const char **argv, const 
> char *prefix)
> argc = parse_options(argc, argv, prefix, options, checkout_usage,
>  PARSE_OPT_KEEP_DASHDASH);
>
> +   if (opts.show_progress < 0) {
> +   if (opts.quiet)
> +   opts.show_progress = 0;
> +   else
> +   opts.show_progress = isatty(2);
> +   }
> +
> if (conflict_style) {
> opts.merge = 1; /* implied */
> git_xmerge_config("merge.conflictstyle", conflict_style, 
> NULL);
> --
> 2.6.1
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7] checkout: add --progress option

[Edmundo Carmona Antoranz]

On Sun, Nov 1, 2015 at 3:15 PM, Eric Sunshine  wrote:
>> +--[no-]progress::
>> +   Progress status is reported on the standard error stream
>> +   by default when it is attached to a terminal, unless `--quiet`
>> +   is specified. This flag enables progress reporting even if not
>> +   attached to a terminal, regardless of `--quite`.
>
> s/quite/quiet/

hahaha. So close!!! v8 on the pipeline.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/4] Add support for matching full refs in hideRefs

[Eric Sunshine]

On Sun, Nov 1, 2015 at 2:34 PM, Lukas Fleischer  wrote:
> In addition to matching stripped refs, one can now add hideRefs patterns
> that the full (unstripped) ref is matched against. To distinguish
> between stripped and full matches, those new patterns must be prefixed
> with a circumflex (^).
>
> This commit also removes support for the undocumented and unintended
> hideRefs settings "have" (suppressing all "have" lines) and
> "capabilities^{}" (suppressing the capabilities line).
>
> Signed-off-by: Lukas Fleischer 
> ---
> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> @@ -1198,7 +1200,15 @@ static void reject_updates_to_hidden(struct command 
> *commands)
> struct command *cmd;
>
> for (cmd = commands; cmd; cmd = cmd->next) {
> -   if (cmd->error_string || !ref_is_hidden(cmd->ref_name))

Would it make sense to retain the cmd->error_string check here in
order to avoid doing the extra refname_full construction work below?

if (cmd->error_string)
continue;

> +   const char *refname = cmd->ref_name;
> +   struct strbuf refname_full_buf = STRBUF_INIT;
> +   const char *refname_full;
> +
> +   strbuf_addf(_full_buf, "%s%s", get_git_namespace(),
> +   refname);
> +   refname_full = strbuf_detach(_full_buf, NULL);
> +
> +   if (cmd->error_string || !ref_is_hidden(refname, 
> refname_full))
> continue;

This is leaking refname_full each time through the loop since it never
gets free()d. If you restructure the code like this, it might be
easier to avoid leaks:

for (cmd = ...; ... ; ...) {
if (cmd->error_string)
continue;
strbuf_addf(_full, "%s%s", ...);
if (ref_is_hidden(...)) {
if (is_null_sha1(...))
cmd->error_string = "...";
else
cmd->error_string = "...";
}
strbuf_release(_full);
}

As a micro-optimization, you could also pre-populate the strbuf with
get_git_namespace() outside the loop and remember the length. Then,
each time through the loop, just append cmd->ref_name, do your
processing, and, at the bottom of the loop, set the strbuf back to the
remembered length. (And, you still need to free the strbuf after the
loop.)

> if (is_null_sha1(cmd->new_sha1))
> cmd->error_string = "deny deleting a hidden ref";
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/4] t5509: add basic tests for hideRefs

[Lukas Fleischer]

On Sun, 01 Nov 2015 at 22:13:51, Eric Sunshine wrote:
> On Sun, Nov 1, 2015 at 2:34 PM, Lukas Fleischer  wrote:
> > Test whether regular and full hideRefs patterns work as expected when
> > namespaces are used.
> >
> > Signed-off-by: Lukas Fleischer 
> > ---
> > diff --git a/t/t5509-fetch-push-namespaces.sh 
> > b/t/t5509-fetch-push-namespaces.sh
> > @@ -82,4 +82,33 @@ test_expect_success 'mirroring a repository using a ref 
> > namespace' '
> > )
> >  '
> >
> > +test_expect_success "Hide namespaced refs with transfer.hideRefs" '
> 
> None of the other tests in this file capitalize the test description.
> These new test descriptions should probably follow suit by beginning
> with lowercase. It is also typical to use single quotes for the
> description rather than double.
> 

Good catch. I originally put these tests somewhere else and noticed that
adding them to t5509 is much better since we already set up the whole
namespace infrastructure there. Seems like I forgot to adjust the style.
Fixed that locally.

> > +   cd pushee &&
> > +   test_config transfer.hideRefs refs/tags &&
> > +   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
> > +   printf "$commit1refs/heads/master\n" >expected &&
> > +   test_cmp expected actual &&
> > +   cd ..
> 
> If any of the commands above "cd .." fail, then "cd .." will never be
> invoked, thus subsequent tests will fail since they won't be executed
> in the expected directory. The typical way to handle this is to place
> the "cd foo" and remaining test body in a subshell, and drop "cd .."
> altogether. When the subshell exits (via success or failure), the
> working directory will be restored automatically.
> 
> test_expect_success '...' '
> (
> cd pushee &&
> test_config ... &&
> ...
> )
> '
> [...]

I chose the `cd ..` approach because test_config does not work from a
subshell. However, searching the Git log for "test_config", I found
1a9a23e (t7610: don't use test_config in a subshell, 2015-09-05) and
da568b6 (t7800: don't use test_config in a subshell, 2015-09-05) which
suggest to use the -C switch. The test cases now look like this:

test_expect_success '[...]' '
test_config -C pushee transfer.hideRefs [...] &&
(
cd pushee &&
[...]
)
'

Thanks!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/4] t5509: add basic tests for hideRefs

[Eric Sunshine]

On Mon, Nov 2, 2015 at 1:25 AM, Lukas Fleischer  wrote:
> On Sun, 01 Nov 2015 at 22:13:51, Eric Sunshine wrote:
>> On Sun, Nov 1, 2015 at 2:34 PM, Lukas Fleischer  wrote:
>> > +   cd pushee &&
>> > +   test_config transfer.hideRefs refs/tags &&
>> > +   GIT_NAMESPACE=namespace git ls-remote "ext::git %s ." >actual &&
>> > +   printf "$commit1refs/heads/master\n" >expected &&
>> > +   test_cmp expected actual &&
>> > +   cd ..
>>
>> If any of the commands above "cd .." fail, then "cd .." will never be
>> invoked, thus subsequent tests will fail since they won't be executed
>> in the expected directory. The typical way to handle this is to place
>> the "cd foo" and remaining test body in a subshell, and drop "cd .."
>> altogether. When the subshell exits (via success or failure), the
>> working directory will be restored automatically.
>>
>> test_expect_success '...' '
>> (
>> cd pushee &&
>> test_config ... &&
>> ...
>> )
>> '
>> [...]
>
> I chose the `cd ..` approach because test_config does not work from a
> subshell. However, searching the Git log for "test_config", I found
> 1a9a23e (t7610: don't use test_config in a subshell, 2015-09-05) and
> da568b6 (t7800: don't use test_config in a subshell, 2015-09-05) which
> suggest to use the -C switch. The test cases now look like this:
>
> test_expect_success '[...]' '
> test_config -C pushee transfer.hideRefs [...] &&
> (
> cd pushee &&
> [...]
> )
> '

Yes, that can work, although for these simple cases, it might be more
straightforward to use the git -c option to set the config variable
just for the duration of the one git command. For instance:

test_expect_success 'hide namespaced refs with transfer.hideRefs' '
(
cd pushee &&
GIT_NAMESPACE=namespace \
git -c transfer.hideRefs=refs/tags \
ls-remote "ext::git %s ." >actual &&
printf "$commit1\trefs/heads/master\n" >expected &&
test_cmp expected actual &&
)
'

In fact, these test are so simple, that you don't really need the 'cd'
at all. You could just use -C (along with -c):

test_expect_success 'hide namespaced refs with transfer.hideRefs' '
GIT_NAMESPACE=namespace \
git -C pushee -c transfer.hideRefs=refs/tags \
ls-remote "ext::git %s ." >actual &&
printf "$commit1\trefs/heads/master\n" >expected &&
test_cmp expected actual &&
'
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Cloning of submodule with --depth should follow the branch option

[Mikhail Zabaluev]

Hi,

I have an issue with git 2.4.3 when trying to limit the amount of data
fetched when cloning submodules. The --depth option is useful, but
when .gitmodules specifies a non-default branch, the submodule
repository clone invocation (which is a single-branch clone due to
--depth) fetches master regardless, and then checkout fails due to a
nonexistent tree reference. I think it would be sensible in this case
to pass --branch to the submodule clone command.

Here's a repository to exercise: https://github.com/gi-rust/gir

Best regards,
  Mikhail
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Git potential bug with fork-point

[Stenberg Jim (2)]

Hello,
my name is Jim Stenberg and I'm a developer, albeit not that experienced with 
Git.
Please note that I'm not a native English speaker.
If there is anything unclear or lacking, feel free to ask me on this email 
address.

My problem:
"Git merge-base --fork-point" acts unexpected when I refer to remote branches 
(typically "origin/".)
With unexpected I mean that if I swap the position of the two references that 
the function takes as argument I get different results.
I highly suspect that this isn't a feature but a bug, or maybe I'm using the 
function in a way it wasn't intended to be used.
I don't need you to fix it (swapping the arguments solves it), I just want you 
to be aware of it.

History & procedure:
When  I was working on my automatic build script I came across the oddity that 
"Git merge-base --fork-point" behaved
differently depending on the order in which the two references are passed.
I tried to strip the problem down to the smallest example I could. Such example 
uses:
. A local folder as repository, initialized by "git init --bare"
. A local clone, acquired by "git clone ../my/path/to/repo repoName"
. Populated with a few files and commits.
I first detected the problem on git version 1.9.5.msysgit.0 but could confirm 
the same behavior on git version 2.6.2.windows.1.
I've pasted a log in text form below, the message did not allow for appended 
images. The log should be sufficient to reproduce the problem.
Please observer the additional new-lines in the log and that no errors or fault 
messages are presented.

System configuration:
OS Name:       Microsoft Windows 7 Enterprise
OS Version:        6.1.7601 Service Pack 1 Build 7601
OS Manufacturer:       Microsoft Corporation
OS Configuration:      Member Workstation
OS Build Type:     Multiprocessor Free
System Type:   x64-based PC
Total Physical Memory: 3 977 MB
Available Physical Memory: 841 MB

Log:
C:\Test\Local\Main>git version
git version 2.6.2.windows.1

C:\Test\Local\Main>git plot
* a17df67 (HEAD -> release/15.F, origin/release/15.F) try try again
* 2af2f42 Other solution to fork-point problem, change position of arguments
* 8bb3a5e more bugfixing, in release branch is now also checked against server
* 9db0a3c bugfix, fork-point do not work with origin/* => use only merge-base
* 6489785 Check on origin/release/* instead of release/*
* 12cd1af X -> T
* 2dde79c P -> X && 15.D -> 15.F
* ffefe54 15.F -> 15.D
* 634e81d T->P
* f4003e5 Overhauled the ECU split solution
* 0e1bc85 reverting even more
* c0e1391 incorrectly changed to X release
* faaa7e8 changed to P release
* 51d5588 Finishing touch, + fault codes
* c03f0b4 canges, ready for release?
* 09a7ed6 (origin/master, master) minor bug fixing
* 4bdb033 (tag: 15.D) Added required files

C:\Test\Local\Main>git merge-base --fork-point release/15.F master
09a7ed6294ae18fb6087ca1ee020d544f4efe28d

C:\Test\Local\Main>git merge-base --fork-point master release/15.F
09a7ed6294ae18fb6087ca1ee020d544f4efe28d

C:\Test\Local\Main>git merge-base --fork-point master origin/release/15.F
09a7ed6294ae18fb6087ca1ee020d544f4efe28d

C:\Test\Local\Main>git merge-base --fork-point origin/master origin/release/15.F

09a7ed6294ae18fb6087ca1ee020d544f4efe28d

C:\Test\Local\Main>git merge-base --fork-point origin/release/15.F origin/master


C:\Test\Local\Main>git merge-base --fork-point origin/release/15.F master

C:\Test\Local\Main>

Literature index:
When reading https://git-scm.com/docs/git-merge-base I didn't found any 
information about this particular case.
I didn't find any list of known bugs, hence I can't confirm if it's reported or 
not.

_
Med Vänliga Hälsningar / Best Regards
Jim Stenberg M.Sc.
ECU node owner

Volvo Bus Corporation
Dept CD74230, ARAK3
SE-405 08 Göteborg, Sweden
Phone: +46 31 32 25173
Email: jim.stenber...@volvo.com

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] http.c: use CURLOPT_RANGE for range requests

[Daniel Stenberg]

On Sun, 1 Nov 2015, Jeff King wrote:

While I have your attention, Daniel, am I correct in assuming that 
performing a second unrelated request with the same CURL object will need an 
explicit:


 curl_easy_setopt(curl, CURLOPT_RANGE, NULL);

to avoid using the range twice?


Correct. As the options stick until modified.

--

 / daniel.haxx.se
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

When a file was locked by some program, git will work stupidly

[dayong xie]

To be specific
In my Unity project, there is a native plugin,  and plugin's extension
is .dll, and this plugin is
under git version control, when Unity is running, the plugin file will
be locked.
If i merge another branch, which contains modification of the plugin,
git will report error, looks
like:
error: unable to unlink old 'xxx/xxx.dll' (Permission denied)
This is not bad, however, the unfinished merge action will not revert
by git, a lot of changes
produced in repository.
usually it makes me crazy, even worse, some of my partners are not
good at using git.
Of course, this problem can be avoided by quit Unity, but not every
time we can remember. In
my opinion, git should revert the unfinished action when the error
occurred, not just stop.

-- 
--
Best Regards,
John Xie
--
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/4] Add support for matching full refs in hideRefs

[Lukas Fleischer]

On Sun, 01 Nov 2015 at 21:42:37, Eric Sunshine wrote:
> On Sun, Nov 1, 2015 at 2:34 PM, Lukas Fleischer  wrote:
> > In addition to matching stripped refs, one can now add hideRefs patterns
> > that the full (unstripped) ref is matched against. To distinguish
> > between stripped and full matches, those new patterns must be prefixed
> > with a circumflex (^).
> >
> > This commit also removes support for the undocumented and unintended
> > hideRefs settings "have" (suppressing all "have" lines) and
> > "capabilities^{}" (suppressing the capabilities line).
> >
> > Signed-off-by: Lukas Fleischer 
> > ---
> > diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> > @@ -1198,7 +1200,15 @@ static void reject_updates_to_hidden(struct command 
> > *commands)
> > struct command *cmd;
> >
> > for (cmd = commands; cmd; cmd = cmd->next) {
> > -   if (cmd->error_string || !ref_is_hidden(cmd->ref_name))
> 
> Would it make sense to retain the cmd->error_string check here in
> order to avoid doing the extra refname_full construction work below?
> 
> if (cmd->error_string)
> continue;
> 
> [...]
> This is leaking refname_full each time through the loop since it never
> gets free()d. If you restructure the code like this, it might be
> easier to avoid leaks:
> 
> for (cmd = ...; ... ; ...) {
> if (cmd->error_string)
> continue;
> strbuf_addf(_full, "%s%s", ...);
> if (ref_is_hidden(...)) {
> if (is_null_sha1(...))
> cmd->error_string = "...";
> else
> cmd->error_string = "...";
> }
> strbuf_release(_full);
> }
> 
> As a micro-optimization, you could also pre-populate the strbuf with
> get_git_namespace() outside the loop and remember the length. Then,
> each time through the loop, just append cmd->ref_name, do your
> processing, and, at the bottom of the loop, set the strbuf back to the
> remembered length. (And, you still need to free the strbuf after the
> loop.)
> [...]

Sounds good to me. I changed the code to initialize the strbuf only
once, save the prefix length and reset using strbuf_setlen() in every
loop iteration. Thanks!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html