[ANNOUNCE] Git v2.8.0-rc3

[Junio C Hamano]

A release candidate Git v2.8.0-rc3 is now available for testing
at the usual places.  It is comprised of 498 non-merge commits
since v2.7.0, contributed by 69 people, 21 of which are new faces.

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.8.0-rc3' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = git://git.sourceforge.jp/gitroot/git-core/git.git
  url = git://git-core.git.sourceforge.net/gitroot/git-core/git-core
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.7.0 are as follows.
Welcome to the Git development community!

  마누엘, Adam Dinwoodie, Andrew Wheeler, Changwoo Ryu,
  Christoph Egger, Christoph Hoopmann, Dan Aloni, Dave Ware, David
  A. Wheeler, Dickson Wong, Felipe Gonçalves Assis, GyuYong Jung,
  Jon Griffiths, Kazutoshi Satoda, Lars Vogel, Martin Amdisen,
  Matthew Kraai, Paul Wagland, Rob Mayoff, Romain Picard, and
  Victor Leschuk.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Alexander Kuleshov, Alex Henrie, Audric Schiltknecht, brian
  m. carlson, Carlos Martín Nieto, Christian Couder, David
  A. Greene, David Turner, Dennis Kaarsemaker, Dimitriy Ryazantcev,
  Edmundo Carmona Antoranz, Elia Pinto, Eric Wong, Jacob Keller,
  Jean-Noel Avila, Jeff King, Jiang Xin, Johannes Schindelin,
  Johannes Sixt, John Keeping, Jonathan Nieder, Junio C Hamano,
  Karsten Blees, Karthik Nayak, Knut Franke, Lars Schneider,
  Matthieu Moy, Matt McCutchen, Michael J Gruber, Mike Hommey,
  Nguyễn Thái Ngọc Duy, Øyvind A. Holm, Patrick Steinhardt,
  Pat Thoyts, Peter Krefting, Ralf Thielow, Sebastian Schuberth,
  Shawn O. Pearce, Stefan Beller, Stephen P. Smith, SZEDER Gábor,
  Thomas Ackermann, Thomas Braun, Thomas Gummerer, Tobias Klauser,
  Torsten Bögershausen, Trần Ngọc Quân, and Will Palmer.



Git 2.8 Release Notes (draft)
=

Backward compatibility note
---

The rsync:// transport has been removed.


Updates since v2.7
--

UI, Workflows & Features

 * It turns out "git clone" over rsync transport has been broken when
   the source repository has packed references for a long time, and
   nobody noticed nor complained about it.

 * "push" learned that its "--delete" option can be shortened to
   "-d", just like "branch --delete" and "branch -d" are the same
   thing.

 * "git blame" learned to produce the progress eye-candy when it takes
   too much time before emitting the first line of the result.

 * "git grep" can now be configured (or told from the command line)
   how many threads to use when searching in the working tree files.

 * Some "git notes" operations, e.g. "git log --notes=", should
   be able to read notes from any tree-ish that is shaped like a notes
   tree, but the notes infrastructure required that the argument must
   be a ref under refs/notes/.  Loosen it to require a valid ref only
   when the operation would update the notes (in which case we must
   have a place to store the updated notes tree, iow, a ref).

 * "git grep" by default does not fall back to its "--no-index"
   behaviour outside a directory under Git's control (otherwise the
   user may by mistake end up running a huge recursive search); with a
   new configuration (set in $HOME/.gitconfig--by definition this
   cannot be set in the config file per project), this safety can be
   disabled.

 * "git pull --rebase" has been extended to allow invoking
   "rebase -i".

 * "git p4" learned to cope with the type of a file getting changed.

 * "git format-patch" learned to notice format.outputDirectory
   configuration variable.  This allows "-o " option to be
   omitted on the command line if you always use the same directory in
   your workflow.

 * "interpret-trailers" has been taught to optionally update a file in
   place, instead of always writing the result to the standard output.

 * Many commands that read files that are expected to contain text
   that is generated (or can be edited) by the end user to control
   their behaviour (e.g. "git grep -f ") have been updated
   to be more tolerant to lines that are terminated with CRLF (they
   used to treat such a line to contain payload that ends with CR,
   which is usually not what the users expect).

 * "git notes merge" used to limit the source of the merged notes tree
   to somewhere under refs/notes/ hierarchy, which was too limiting
   when inventing a workflow to exchange notes with remote
   repositories using remote-tracking notes trees (located in e.g.
   refs/remote-notes/ or somesuch).

 * "git ls-files" learned a new "--eol" option to help diagnose
   end-of-line problems.

 * "ls-remote" learned an option to show which branch 

斯游傥为胜

[斯游傥为胜]

你的老朋友邀你来Q群:343257759

Re: [PATCH] pretty-print: de-tabify indented logs to make things line up properly

[Linus Torvalds]

On Wed, Mar 16, 2016 at 2:37 PM, Junio C Hamano  wrote:
>
> What surprised me was that this new expand logic triggered for
> shortlog, actually.  I somehow assumed the caller that called
> de-tabify helper was only called for --pretty=medium.

I guess that would be ok, since shortlog by definition can't have any
issues with multiple lines lining up with each other.

At the same time, it might be a bit odd to show tabs in that first
line differently for the one-line vs multi-line log version. But maybe
it isn't - I think shortlog is the only thing that does that wrapping
anyway, so shortlog is already special.

I think the reason shortlog output gets both the de-tab and the
wrapping is that shortlog_add_commit() just calls pretty_print_commit
with CMIT_FMT_USERFORMAT.

Linus
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 18/19] index-helper: autorun

[Johannes Schindelin]

Hi Duy,

On Fri, 18 Mar 2016, Duy Nguyen wrote:

> > Well, the way I read the code it is possible that:
> >
> > 1. Git process 1 starts, reading the index
> > 2. Git process 2 starts, poking the index-helper
> > 3. The index-helper updates the .pid file (why not set a bit in the shared
> >memory?) with a prefix "W"
> > 4. Git process 2 reads the .pid file and waits for the "W" to go away
> >(what if index-helper is not fast enough to write the "W"?)
> > 5. Git process 1 access the index, happily oblivious that it is being
> >updated and the data is in an inconsistent state
> 
> No, if process 1 reads the index file, then that file will remain
> consistent/unchanged all the time. index-helper is not allowed to
> touch that file at all.
> 
> The process 2 gets the index content from shm (cached by the index
> helper), verifies that it's good (with the signature at the end of the
> shm). If watchman is used, process 2 can also read the list of
> modified files from another shm, combine it with the in-core index,
> then write it down the normal way. Only then process 1 (or process 3)
> can see the new index content from the file.

So how do you deal with releasing the shared memory instances that are
essentially created for every index update?

Ciao,
Dscho
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: bug: sparse config interpretation incorrectness in 2.8.0-rc2

[Duy Nguyen]

On Thu, Mar 17, 2016 at 8:46 PM, Johannes Schindelin
 wrote:
> Unfortunately, this does not help me at all. In the use case I am trying
> to get to work fast, we have tons and tons of directories and need *one*
> file in pretty much *all* of those directories, and exclude most of the
> other files.
>
> To make matters even worse, the list of excluded (or included) files is
> constantly changing.

OK very weird use case to me :) There's something you might try. [1]
can help trimming unrelated patterns, fewer patterns to match for a
dir, faster matching.

I knew it might help speed up sparse checkout (because we can't spread
sparse patterns over many .gitignore files,  we only have one place
for all patterns). I did try at some point. I just don't remember if I
gave up because I didn't think it's worth the effort or I found out
something wrong with it for sparse checkout case. It may or may help
your case, depending on the patterns, of course.

[1] http://article.gmane.org/gmane.comp.version-control.git/217958
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Michela

[Michela Peterson]

Hei,
Nimeni on Michael, olen 26 vuotta vanha, olen vapaa minded, avoin
sydämestä tyttö, tykkään ottaa elämän yhtä helppoa kuin pystyin,
olen yksi niistä harvoista, joka yhä uskoo ystävyydestä, rakkaudesta,
luottamus ja merkkejä, olen hyvin paljon yhden ja valmis mingle.was
selaat sivun ja minä haluan olla ystäväsi jos et mielessä,
toivottavasti ette ota pyyntöni selvänä, ota yhteyttä sähköpostitse
minulle, aion kiitollinen, jos voit lähettää minulle joitakin kuvia
minun yksityinen sähköpostiosoite, odotan kuulla sinusta pian.

Hello,
My name is Michela, i am 26yrs old, i'm a free minded,open hearted girl, i
like to take life as easy as i could, i'm one of the few that still
believes in friendship, love, trust and signs, am very much single and
ready to mingle.was browsing through your page and i will like to be your
friend if you don't mind, i hope you will not take my request for
granted,feel free to email me, i will appreciate it if you can send me
some pics to my private email address, i look forward to hear from you
soon.


--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH] Makefile: allow po generation through po target

[Junio C Hamano]

Michael J Gruber  writes:

> The main Makefile has a "pot" target that recreates the git.pot file of
> strings which are marked for translation.
>
> Add a "po" target that recreates the $(LANGUAGE).po files which contain
> the translations (or stubs).
>
> Signed-off-by: Michael J Gruber 
> ---
>
> Notes:
> This makes it easier to recreate po (and mo) without reading po/README.
> Alternatively, one may think about a Makefile in po/ which does both pot
> and po updates, just like we have makefiles in t/ and Ducumentation/.
> 
> This doesn't give you proper before-after diffs yet, but at least the 
> after
> state.

More seriously, the "before state" does not exist anywhere, because
*.po and *.pot are expected not to be in sync with the source, and
after a code developer runs "make po", because she does not know all
the languages we have *.po for, she has to "git checkout" to erase
the changes made by "make po".

So while your starting discussion (i.e. RFC-ness of this patch) is
very much appreciated, I do not think this is a good change we would
want to base further work on top.  For "before-after-diff" purposes,
the targets for before and after should drop their output in an
untracked build artifact, instead of overwriting tracked files, I
would think.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/2] dir.c: fix dir re-inclusion rules with "NODIR" and "MUSTBEDIR"

[Nguyễn Thái Ngọc Duy]

For NODIR case, the patterns look like this

*  # exclude dir, dir/file1 and dir/file2..
!dir   # ..except that dir and everything inside is re-included..
dir/file2  # ..except (again!) that dir/file2 is excluded
   # ..which means dir/file1 stays included

When we stay at topdir and test "dir", everything is hunky dory, current
code returns "re-include dir" as expected. When we stay in "dir" and
examine "dir/file1", however, match_basename() checks if the base name
component, which is "file1", matches "dir" from the second rule.

This is wrong. We contradict ourselves because earlier we decided to
re-include dir/file1 (from the second rule) when we were at toplevel.
Because the second rule is ignored, we hit the first one and return
"excluded" even though "dir/file1" should be re-included.

In the MUSTBEDIR case, the patterns look like this

*  # exclude dir, dir/file1 and dir/file2..
!dir/  #  ..except that dir and everything inside is re-included..
dir/file2  # ..except (again!) that dir/file2 is excluded
   # ..which means dir/file1 stays included

Again, we're ok at the toplevel, then we enter "dir" and test
"dir/file1". The MUSTBEDIR code tests if the _full_ path (ie. "dir/file1")
is a directory. We want it to test the "dir" part from "dir/file1"
instead.

In both cases, the second decision on "dir/file1" is wrong and
contradicts with the first one on "dir". This is a perfect use case for
"sticky path list" added earlier to solve a different (but quite
similar) problem. So, when we have the right decision the first time, we
mark the path sticky to override the coming wrong decision.

The reason to do this, instead of actually fixing the code to make the
right second decision in the first place, is because it's soo
complicated. There are many combinations to take care of, many
optimizations involved to keep the cost on normal/common case (and
what's being described here is NOT normal) down to minimum. On top of
that, exclude code is already complicated as it is. It's best to not
turn the code upside down. Not until this approach is proved
insufficient.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 This is NOT for 2.8.0! Posted here to give you some context on the
 problem mentioned in the first patch.

 If you actually read the link in 1/2, you'll notice this patch is
 completely different. "soo complicated" is not an exaggeration. I
 found some problem with that old patch, which ended up with a
 combination explosion of cases I would have to cover separately,
 essentially the same thing I encountered in my first try before that
 patch. I finally admitted I could not fit all that in my brain
 anymore.

 dir.c  |  5 
 t/t3007-ls-files-other-negative.sh | 51 ++
 2 files changed, 56 insertions(+)

diff --git a/dir.c b/dir.c
index 2028094..a704e8a 100644
--- a/dir.c
+++ b/dir.c
@@ -1090,6 +1090,11 @@ static struct exclude 
*last_exclude_matching_from_list(const char *pathname,
 x->pattern, x->srcpos);
return NULL;
}
+   } else if (exc->flags & EXC_FLAG_NEGATIVE) {
+   if (*dtype == DT_UNKNOWN)
+   *dtype = get_dtype(NULL, pathname, pathlen);
+   if (*dtype == DT_DIR)
+   add_sticky(exc, pathname, pathlen);
}
 
trace_printf_key(_exclude, "exclude: %.*s vs %s at line %d => 
%s%s\n",
diff --git a/t/t3007-ls-files-other-negative.sh 
b/t/t3007-ls-files-other-negative.sh
index 0797b86..c8f39dd 100755
--- a/t/t3007-ls-files-other-negative.sh
+++ b/t/t3007-ls-files-other-negative.sh
@@ -150,4 +150,55 @@ test_expect_success 'match, literal pathname, nested 
negatives' '
test_cmp tmp/expected tmp/actual
 '
 
+test_expect_success 're-include case, NODIR' '
+   git init re-include-nodir &&
+   (
+   cd re-include-nodir &&
+   mkdir dir &&
+   touch dir/file1 dir/file2 &&
+   cat >.gitignore <<-\EOF &&
+   *
+   !dir
+   dir/file2
+   EOF
+   git ls-files -o --exclude-standard >../actual &&
+   echo dir/file1 >../expected &&
+   test_cmp ../expected ../actual
+   )
+'
+
+test_expect_success 're-include case, MUSTBEDIR with NODIR' '
+   git init re-include-mustbedir &&
+   (
+   cd re-include-mustbedir &&
+   mkdir dir &&
+   touch dir/file1 dir/file2 &&
+   cat >.gitignore <<-\EOF &&
+   *
+   !dir/
+   dir/file2
+   EOF
+   git ls-files -o --exclude-standard >../actual &&
+   echo dir/file1 >../expected &&
+   test_cmp ../expected ../actual
+   )
+'
+
+test_expect_success 're-include case, MUSTBEDIR 

Re: [PATCH/RFC/GSoC 3/3] t0301: test credential-cache support of XDG_RUNTIME_DIR

[Junio C Hamano]

谭俊浩  writes:

> On 17/03/2016 01:24, Junio C Hamano wrote:
>
>> Using ~/.git-credential-cache/credential-cache.sock would not help
>> at all for existing users, but ~/.git-credential-cache/socket would
>> interoperate well with users with existing versions of Git, no?
>>
 Just being curious, and wanting to see the reasoning behind the
 design decision the patch series makes in the log message of one of
 these patches.
>
> I guess it is better to use /tmp or such instead of $HOME/.* so that
> the users home directory won't be flooded by sockets.

The "fallback" being discussed is to see if $XDG can be used (and
use it if so), otherwise see if ~/.git-credential-cache/socket can
be used (and use it if so), otherwise die with a message (see
credential-cache.c).  The order of the falling back may want to be
the other way around, but in either case, the definition of "can be
used" includes "is there already a directory in which we can create
a socket?".

The existing versions have used ~/.git-credential-cache/socket as
the default socket path, so it is reasonable to expect that users
that are already using the feature already have the directory there.

So I do not think there is any "flooded" involved; if the directory
is already there, we can use it to create and use a single socket.
It's not like we'd be creating many random new directories in ~/.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] pretty-print: add --pretty=noexpand

[Junio C Hamano]

It is reasonable for tweak the default output mode for "git log" to
untabify the commit log message, it sometimes may be necessary to
see the output without tab expansion.

Invent a new --pretty option to do this.  Use this to unbreak the
test breakages, where "git shortlog" and output are tested.

Signed-off-by: Junio C Hamano 
---
 Documentation/pretty-formats.txt | 10 ++
 Documentation/pretty-options.txt |  2 +-
 commit.h |  1 +
 pretty.c | 12 +---
 t/t4201-shortlog.sh  |  6 +++---
 5 files changed, 24 insertions(+), 7 deletions(-)

diff --git a/Documentation/pretty-formats.txt b/Documentation/pretty-formats.txt
index 671cebd..173b932 100644
--- a/Documentation/pretty-formats.txt
+++ b/Documentation/pretty-formats.txt
@@ -39,6 +39,16 @@ This is designed to be as compact as possible.
 
  
 
+ 
+
+* 'noexpand'
+
+ commit 
+ Author: 
+ Date:   
+
+ 
+
  
 
 * 'full'
diff --git a/Documentation/pretty-options.txt b/Documentation/pretty-options.txt
index 4b659ac..7032b1a 100644
--- a/Documentation/pretty-options.txt
+++ b/Documentation/pretty-options.txt
@@ -3,7 +3,7 @@
 
Pretty-print the contents of the commit logs in a given format,
where '' can be one of 'oneline', 'short', 'medium',
-   'full', 'fuller', 'email', 'raw', 'format:'
+   'full', 'fuller', 'email', 'raw', 'noexpand', 'format:'
and 'tformat:'.  When '' is none of the above,
and has '%placeholder' in it, it acts as if
'--pretty=tformat:' were given.
diff --git a/commit.h b/commit.h
index 5d58be0..d511c61 100644
--- a/commit.h
+++ b/commit.h
@@ -126,6 +126,7 @@ enum cmit_fmt {
CMIT_FMT_RAW,
CMIT_FMT_MEDIUM,
CMIT_FMT_DEFAULT = CMIT_FMT_MEDIUM,
+   CMIT_FMT_NOEXPAND,
CMIT_FMT_SHORT,
CMIT_FMT_FULL,
CMIT_FMT_FULLER,
diff --git a/pretty.c b/pretty.c
index 717ceed..8b533dc 100644
--- a/pretty.c
+++ b/pretty.c
@@ -89,6 +89,7 @@ static void setup_commit_formats(void)
struct cmt_fmt_map builtin_formats[] = {
{ "raw",CMIT_FMT_RAW,   0 },
{ "medium", CMIT_FMT_MEDIUM,0 },
+   { "noexpand",   CMIT_FMT_NOEXPAND,  0 },
{ "short",  CMIT_FMT_SHORT, 0 },
{ "email",  CMIT_FMT_EMAIL, 0 },
{ "fuller", CMIT_FMT_FULLER,0 },
@@ -1685,11 +1686,16 @@ static void strbuf_add_tabexpand(struct strbuf *sb,
  * the whole line (without the final newline), after
  * de-tabifying.
  */
-static void pp_handle_indent(struct strbuf *sb, int indent,
+static void pp_handle_indent(struct pretty_print_context *pp,
+struct strbuf *sb,
+int indent,
 const char *line, int linelen)
 {
strbuf_addchars(sb, ' ', indent);
-   strbuf_add_tabexpand(sb, line, linelen);
+   if (pp->fmt == CMIT_FMT_MEDIUM)
+   strbuf_add_tabexpand(sb, line, linelen);
+   else
+   strbuf_add(sb, line, linelen);
 }
 
 void pp_remainder(struct pretty_print_context *pp,
@@ -1716,7 +1722,7 @@ void pp_remainder(struct pretty_print_context *pp,
 
strbuf_grow(sb, linelen + indent + 20);
if (indent)
-   pp_handle_indent(sb, indent, line, linelen);
+   pp_handle_indent(pp, sb, indent, line, linelen);
else
strbuf_add(sb, line, linelen);
strbuf_addch(sb, '\n');
diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index 987b708..34a9fed 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -93,7 +93,7 @@ test_expect_success 'output from user-defined format is 
re-wrapped' '
test_cmp expect log.predictable
 '
 
-test_expect_failure !MINGW 'shortlog wrapping' '
+test_expect_success !MINGW 'shortlog wrapping' '
cat >expect <<\EOF &&
 A U Thor (5):
   Test
@@ -114,8 +114,8 @@ EOF
test_cmp expect out
 '
 
-test_expect_failure !MINGW 'shortlog from non-git directory' '
-   git log HEAD >log &&
+test_expect_success !MINGW 'shortlog from non-git directory' '
+   git log --pretty=noexpand HEAD >log &&
GIT_DIR=non-existing git shortlog -w out &&
test_cmp expect out
 '
-- 
2.8.0-rc3-175-g64dcf62

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHV2 1/2] rebase -x: do not die without -i

[Johannes Schindelin]

Hi Stefan,

On Thu, 17 Mar 2016, Stefan Beller wrote:

> git rev-list old...new |
> while read rev
> do
> $command || break
> done

As Junio pointed out, there should be only two, not three dots.

Ciao,
Dscho
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Windows download broken?

[Levente]

It seems that this link is broken:

https://github.com/git-for-windows/git/releases/download/v2.7.3.windows.1/Git-2.7.3-64-bit.exe

Regards,
Levente
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v9 2/2] pull --rebase: add --[no-]autostash flag

[Eric Sunshine]

On Thu, Mar 17, 2016 at 12:49 PM, Mehul Jain  wrote:
> If rebase.autoStash configuration variable is set, there is no way to
> override it for "git pull --rebase" from the command line.
>
> Teach "git pull --rebase" the --[no-]autostash command line flag which
> overrides the current value of rebase.autoStash, if set. As "git rebase"
> understands the --[no-]autostash option, it's just a matter of passing
> the option to underlying "git rebase" when "git pull --rebase" is called.
>
> Signed-off-by: Mehul Jain 
> ---
> previous patches: 
> http://thread.gmane.org/gmane.comp.version-control.git/287709
>
> Changes:
> * Modified documentation

This would be more helpful for reviewers if it went into a bit more
detail; "modified" alone doesn't say much. For instance, "... to keep
the description of --autostash and --no-autostash together rather than
splitting them apart with a tangential comment" or something.

> * "git pull --[no-]autostash" case is handled bit early then before

Likewise, explaining why it's handled a bit earlier would help
reviewers. For instance, "... since getting the error handling case
out of the way early makes the following code easier to reason about"
or something.

Since this is now a patch series rather than a single patch, another
way to help reviewers is to use a cover letter (see git-format-patch
--cover-letter) where you'd explain the changes, and, importantly,
include an interdiff between the previous and current versions.

> diff --git a/builtin/pull.c b/builtin/pull.c
> @@ -86,6 +86,7 @@ static char *opt_commit;
> +static int opt_autostash = -1;
> @@ -801,6 +804,7 @@ static int run_rebase(const unsigned char *curr_head,
> argv_array_pushv(, opt_strategy_opts.argv);
> if (opt_gpg_sign)
> argv_array_push(, opt_gpg_sign);
> +   argv_array_push(, opt_autostash ? "--autostash" : 
> "--no-autostash");

At this point, we know that opt_autostash can't be -1 (thus
incorrectly triggering use of --autostash) because the conditional in
cmd_pull() set it to the value of config_autostash (either 0 or 1) if
the user did not specify it on the command-line. Okay. Makes sense.

Would an assert(opt_autostash != -1) to document this be desirable? (I
don't feel strongly about it, and it's certainly not worth a re-roll.)

> argv_array_push(, "--onto");
> argv_array_push(, sha1_to_hex(merge_head));
> @@ -846,11 +850,17 @@ int cmd_pull(int argc, const char **argv, const char 
> *prefix)
> if (get_sha1("HEAD", orig_head))
> hashclr(orig_head);
>
> +   if (!opt_rebase && opt_autostash != -1)
> +   die(_("--[no-]autostash option is only valid with 
> --rebase."));
> +
> if (opt_rebase) {
> if (is_null_sha1(orig_head) && !is_cache_unborn())
> die(_("Updating an unborn branch with changes added 
> to the index."));
>
> -   if (!config_autostash)
> +   if (opt_autostash == -1)
> +   opt_autostash = config_autostash;
> +
> +   if (!opt_autostash)
> die_on_unclean_work_tree(prefix);
>
> if (get_rebase_fork_point(rebase_fork_point, repo, *refspecs))
> diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh
> index c952d5e..85d9bea 100755
> --- a/t/t5520-pull.sh
> +++ b/t/t5520-pull.sh
> @@ -255,6 +255,45 @@ test_expect_success 'pull --rebase succeeds with dirty 
> working directory and reb
> test "$(cat new_file)" = dirty &&
> test "$(cat file)" = "modified again"
>  '
> +test_expect_success 'pull --rebase: --autostash overrides rebase.autostash' '

Why do titles of some of the new test titles have a ":" after "rebase"
while other don't?

Also, how about normalizing the titles so that the reader knows in
which tests rebase.autostash is 'true' and in which it is 'false'?
Presently, it's difficult to decipher what's being tested based only
on the titles.

Finally, shouldn't you also be testing --autostash and --no-autostash
when rebase.autostash is not set?

> +   test_config rebase.autostash false &&
> +   git reset --hard before-rebase &&
> +   echo dirty >new_file &&
> +   git add new_file &&
> +   git pull --rebase --autostash . copy &&
> +   test_cmp_rev HEAD^ copy &&
> +   test "$(cat new_file)" = dirty &&
> +   test "$(cat file)" = "modified again"
> +'
> +
> +test_expect_success 'pull --rebase --autostash works with rebase.autostash 
> set true' '
> +   test_config rebase.autostash true &&
> +   git reset --hard before-rebase &&
> +   echo dirty >new_file &&
> +   git add new_file &&
> +   git pull --rebase --autostash . copy &&
> +   test_cmp_rev HEAD^ copy &&
> +   test "$(cat new_file)" = dirty &&
> +   test "$(cat file)" = "modified again"
> +'
> +
> +test_expect_success 'pull --rebase: --no-autostash overrides 
> 

Re: [PATCH 2/2] pull --rebase: add --[no-]autostash flag

[Eric Sunshine]

On Wed, Mar 16, 2016 at 1:00 AM, Mehul Jain  wrote:
> On Wed, Mar 16, 2016 at 3:13 AM, Eric Sunshine  
> wrote:
>>> +--autostash::
>>> +--no-autostash::
>>> +   Before starting rebase, stash local modifications away (see
>>> +   linkgit:git-stash.txt[1]) if needed, and apply the stash when
>>> +   done (this option is only valid when "--rebase" is used).
>>> ++
>>> +`--no-autostash` is useful to override the `rebase.autoStash`
>>> +configuration variable (see linkgit:git-config[1]).
>>
>> The last couple sentences seem reversed. It feels odd to have the bit
>> about --rebase dropped dead in the middle of the description of
>> --autostash and --no-autostash. I'd have expected to see --autostash
>> and --no-autostash discussed together, and then, as a follow-on,
>> mention them being valid only with --rebase.
>
> So you are suggesting something like this:
>
> --autostash::
> --no-autostash::
> Before starting rebase, stash local modifications away (see
> linkgit:git-stash.txt[1]) if needed, and apply the stash when
> done. `--no-autostash` is useful to override the `rebase.autoStash`
> configuration variable (see linkgit:git-config[1]).
> +
> This option is only valid when "--rebase" is used.
>
> Can be done and it make more sense to talk about the validity of the
> option in a seperate line.

Yes, like that.

>>> diff --git a/builtin/pull.c b/builtin/pull.c
>>> @@ -851,12 +855,17 @@ int cmd_pull(int argc, const char **argv, const char 
>>> *prefix)
>>> if (is_null_sha1(orig_head) && !is_cache_unborn())
>>> die(_("Updating an unborn branch with changes added 
>>> to the index."));
>>>
>>> -   if (config_autostash)
>>> +   if (opt_autostash == -1)
>>
>> In patch 1/2, this changed from 'if (autostash)' to 'if
>> (config_autostash)'; it's a bit sad that patch 2/2 then has to touch
>> the same code to change it yet again, this time to 'if
>> (opt_autostash)'. Have you tried just keeping the original 'autostash'
>> variable and modifying its value based upon config_autostash and
>> opt_autostash instead? (Not saying that this would be better, but
>> interested in knowing if the result is as clean or cleaner or worse.)
>
> Yes, I tried that. Things looked something like this:
>
> In patch 1/2
> ...
>
> -int autostash = 0;
> +int autostash = config_autoshash;
>
> if (is_null_sha1(orig_head) && !is_cache_unborn())
> die(_("Updating ..."));
>
> -git_config_get_bool("rebase.autostash", );
> if (!autostash)
> die_on_unclean_work_tree(prefix);

The individual diffs look nicer and are less noisy, thus a bit easier to review.

> In patch 2/2
> ...
> int autostash = config_autoshash;
>
> if (is_null_sha1(orig_head) && !is_cache_unborn())
> die(_("Updating ..."));
>
> +if (opt_autostash != -1)
> +autostash = opt_autostash;

I'd probably have placed this conditional just below the line where
'autostash' is declared so that the logic for computing the value of
'autostash' is all in one place.

> if (!autostash)
> die_on_unclean_work_tree(prefix);
> ...
>
> This implementation looks much more cleaner but we are using some
> extra space (autostash) to do the task. If everyone is fine with this
> trade off then I can re-roll a new patch with this method. Comments please.

Using an extra variable isn't a big deal and would be a good idea if
it helped clarify the logic. In this case, the logic isn't
particularly difficult, so I don't feel too strongly about it one way
or the other.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can't git stash after using git add -N

[Junio C Hamano]

I actually think "silently ignore intent-to-add" was a mistake.
We used to error out at write-tree time, which I think is the
right behaviour--"I know I want to have this path, but I cannot
yet decide with what content" is what the user is telling us when
she says "add -N", so until that indecision is resolved, we
shouldn't write out a tree object out of the index.

On Wed, Mar 16, 2016 at 2:05 PM, Jeff King  wrote:
> On Wed, Mar 16, 2016 at 05:02:45AM -0700, Josh Triplett wrote:
>
>> On Tue, Mar 15, 2016 at 09:51:35PM -0700, Junio C Hamano wrote:
>> > Josh Triplett  writes:
>> >
>> > > As far as I can tell, if I run "git add -N" on a file, and then commit
>> > > without adding the file contents, it gets committed as an empty file.
>> >
>> > Is that true?  Git once worked like that in earlier days, but I
>> > think write-tree (hence commit) would simply ignore intent-to-add
>> > entries from its resulting tree.
>>
>> Git 2.7.0 does appear to commit an empty file if I commit after git add
>> -N.
>
> I don't think this is the case:
>
>   git init
>   echo content >file
>   git add -N file
>   git commit -m "empty?"
>
>   git ls-tree HEAD
>   git status
>
> shows that we committed an empty tree. So I see two obvious
> possibilities for improvement:
>
>   1. This commit should have failed without --allow-if-empty. We need to
>  be more careful about intent-to-add entries when figuring out if
>  the commit would be empty or not
>
>   2. I'm not sure if "silently ignore intent-to-add" is the best policy.
>  At least a warning ("hey, what did you want to do with these
>  entries") seems merited, if not aborting the commit entirely. I
>  hesitate on the latter only because perhaps that would mess up
>  somebody's legitimate workflow.
>
> -Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] hashmap API: introduce for_each_hashmap_entry() helper macro

[Alexander Kuleshov]

Hello Junio,

On Thu, Mar 17, 2016 at 12:09 AM, Junio C Hamano  wrote:
> Alexander Kuleshov  writes:
>
>> diff --git a/hashmap.h b/hashmap.h
>> index ab7958a..b8b158c 100644
>> --- a/hashmap.h
>> +++ b/hashmap.h
>> @@ -95,4 +95,11 @@ static inline const char *strintern(const char *string)
>>   return memintern(string, strlen(string));
>>  }
>>
>> +#define for_each_hashmap_entry(map, type)\
>> + struct type *entry; \
>> + struct hashmap_iter iter;   \
>> + \
>> + hashmap_iter_init(map, );  \
>> + while ((entry = hashmap_iter_next()))
>> +
>
> This is an easy way to introduce decl-after-statement, i.e. needs an
> extra pair of {} around the thing.  It also forbids the callers from
> defining "entry" and "iter" as their own identifier outside the
> scope of this macro and use them inside the block that is iterated
> over by shadowing these two variables.
>
> Other than that, it looks like a good concept.  The syntax however
> needs more thought because of the above two issues, I think.

Thanks for feedback. Will fix first issue and think about second.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 18/19] index-helper: autorun

[Duy Nguyen]

On Thu, Mar 17, 2016 at 1:27 AM, Johannes Schindelin
 wrote:
> I am much more concerned about concurrent accesses and the communication
> between the Git processes and the index-helper. Writing to the .pid file
> sounds very fragile to me, in particular when multiple processes can poke
> the index-helper in succession and some readers are unaware that the index
> is being refreshed.

It's not that bad. We should have protection in place to deal with
this and fall back to reading directly from file when things get
suspicious. But I agree that sending UNIX signals (or PostMessage) is
not really good communication.
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] dir.c: fix bug in 'nd/exclusion-regression-fix' topic

[Junio C Hamano]

Nguyễn Thái Ngọc Duy   writes:

> The topic in question introduces "sticky" path list for this purpose:
> given a path 'abc/def', if 'abc' already matches a pattern X, it's added
> to X's sticky path list. When we need to check if 'abc/def' matches
> pattern X and see that 'abc' is already in the list, we conclude right
> away that 'abc/def' also matches X.

It took me two reads to realize that I can get what this paragraph
wants to say if I ignore "given a path 'abc/def'" at the beginning.
In short, if we already know that a directory 'abc' matches a
pattern, the pattern remembers that fact in the list and further
queries about anything in that directory, e.g. 'abc/def', is
answered with "we already know abc matches, so it also does match".

"Sticky" is probably better understood if called "ancestor directory".

> The short reason (*) for sticky path list is to workaround limitations
> of matching code that will return "not match" when we compare
> 'abc/def' and pattern X.

If one asks about 'abc/def' first, you ask "does '*' match
'abc/def'?"--if there is a limitation with the matching code, this
list would not work at all as a workaround.  But we can safely
assume that before asking about 'abc/def' we would always ask about
'abc', so it is OK.  Is that what happens here?

> The bug is in this code. Not only it does "when we need to check if
> 'abc/def' matches...", it does an extra thing: if 'foo/bar' is _not_ in
> the list, return 'not matched' by bypassing all matching code with the
> "continue;" statement. It should let the remaining code decide match
> status instead.

So once a pattern has a non-empty "sticky" list after examining
'abc' and if you ask about 'foo', because it is not in the list and
instead of saying "it is not in the list, so we need to try matching
the pattern against 'foo' to decide if it matches", it incorrectly
says "'foo' does not match".  Is that what happens here?

An abrupt appearance of 'foo/bar' in the description made me read
the above three times and that is why I dropped '/bar' part in the
above.  The correctness of the 'sticky' hack seems to depend on the
assumption that we would always ask about 'foo' before asking about
'foo/bar', so the scenario presented with 'foo/bar' is implausible;
even when asking about 'foo/bar', we would have 'foo' in the list,
no?

The remainder, assuming that I read the above correctly, made sense
to me and justifies what the update code does.

Thanks.

> This bug affects both .gitignore and sparse checkout, but it's reported
> as a sparse checkout bug, so let's examine how it happens. The
> sparse-checkout pattern has two rules
>
> /*
> !one/hideme
>
> and the worktree has three tracked files, one/hideme, one/showme and
> two/showme. What happens is this
>
> * check "one", it matches the first pattern -> positive -> keep
>   examining.
>
> *1* "thanks" to 'nd/exclusion-regression-fix' we detect this pair of
>   patterns, so we put "one" in the sticky list of pattern "/*"
>
> * enter "one", check "one/hideme", it matches the second pattern
>   first (we search from bottom up) -> negative -> excluded
>
> * check "one/showme", it does not match the second pattern.
>
> *2* We then check it against the first pattern and notice the sticky list
>   that includes "one", so we decide right away that "one/showme" is
>   included.
>
> * leave "one", check "two" which does not match the second pattern.
>
> *3* then we check "two" against the first pattern and notice that this
>pattern has a non-empty sticky list, which contains "one", not "two".
>This bug kicks in and bypasses the true matching logic for pattern
>"/*". As a result, we exclude "two/showme".
>
> One may notice that the order of these steps matter. If *3* occurs
> before *1*, then the sticky list at that moment is empty and the bug
> does not kick in. Sparse checkout always examines entries in
> alphabetical order, so "abc/showme" would be examined before "one" and
> not hit this bug!
>
> The last remark is important when we move to .gitignore. We receive the
> list of entries with readdir() and cannot control the order of
> entries. Which means we can't write a test for .gitignore that will
> reliably fail without this fix. Which is why this patch only adds a test
> for sparse checkout, even though the same above steps happen to
> .gitignore.
>
> (*) The problem is known and will be fixed later and described in
> detail then. For this commit, it's sufficient to see the following
> link because the long reason is really long:
>
> http://article.gmane.org/gmane.comp.version-control.git/288479
>
> Reported-by: Durham Goode 
> Signed-off-by: Nguyễn Thái Ngọc Duy 
> ---
>  dir.c|  1 -
>  t/t1011-read-tree-sparse-checkout.sh | 20 
>  2 files changed, 20 insertions(+), 1 deletion(-)
>
> diff --git a/dir.c b/dir.c
> index 69e0be6..77f38a5 100644
> --- a/dir.c
> +++ 

Re: [GIT PULL] GPIO bulk changes for kernel v4.6

[Linus Torvalds]

On Fri, Mar 18, 2016 at 7:32 AM, Johannes Schindelin
 wrote:
>
> On Fri, 18 Mar 2016, Linus Torvalds wrote:
>
>> I thought git didn't merge two branches that have no common base by
>> default, but it seems it will happily do so.
>
> What happened to "The coolest merge EVER!"?
>
> http://thread.gmane.org/gmane.comp.version-control.git/5126/

I'm not complaining about multi-root capability in general - it's
still cool. In the kernel, we have aef8755711a2 ("Merge Btrfs into
fs/btrfs") that does something slightly similar.

It's literally just the fact that "git merge" does it with no extra
flags or checks. I'd like people to have to be aware of what they are
doing when they merge two different projects, not do it by mistake.

So making it conditional on a flag like "--no-common-root" is what I'd look for.

Or just make it about the merge stategy. For example, "subtree" makes
sense exactly for merging one project into a subdirectory of another.
But the default merge shouldn't do it.

I don't think the original "resolve" did it, for example. You can't do
a three-way merge without a base.

Note how that above "coolest merge" definitely wasn't done by just
"git merge gitk". I had to play around with the git internals. Now, it
shouldn't be _that_ hard, but at the same time it really shouldn't be
as easy as "I didn't know, so I merged two independent projects".

   Linus
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Code reorgnization

[Junio C Hamano]

Thomas Adam  writes:

> On 17 March 2016 at 11:11, Duy Nguyen  wrote:
>> Git's top directory is crowded and I think it's agreed that moving
>> test-* to t/helper is a good move. I just wanted to check if we could
>> take this opportunity (after v2.8.0) to move some other files too. I
>> propose the following new subdirs
>
> I wonder whether previous discussions on this still count?  See:
>
> http://marc.info/?l=git=129650572621523=1

If you refer to ancient discussion, especially to a large thread
like that one, please spend a bit more time to summarize it.  It
is between one person spends a bit more time, and all others
independently go there and read.

The essense of the proposal [1] back then was to move all the source
file to src/, rename t/ to testsuite.  And I think [2] is a pretty
good summary of the common feeling back then that explains why the
proposal died out:

Moving everything into src/ and calling it "organized" doesn't
actually accomplish much other than perhaps making the README
file more visible to newbs; things are _still_ a mess, just a
mess with four more letters...

This round is slightly more organized, so many points the old thread
raised would not apply, I suspect.


[References]

*1* http://thread.gmane.org/gmane.comp.version-control.git/165720/focus=165748

*2* http://thread.gmane.org/gmane.comp.version-control.git/165720/focus=166019
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Code reorgnization

[Duy Nguyen]

On Fri, Mar 18, 2016 at 4:49 AM, Junio C Hamano  wrote:
> John Keeping  writes:
>
>> The organisation of the git code shouldn't make a difference since CGit
>> just links with libgit.a, even if it does CGit pulls in git.git as a
>> submodule so it can just fix any problems in the same commit that
>> updates the submodule reference.
>
> I was mostly worried about where Duy and Stefan want to place *.h

*.h stay with their *.c. CFLAGS has two more -Isrc and -Ilib. I don't
expect any #include line changes. Maybe we can start moving stuff to
"lib" soon. Many of them rarely receive changes these days. The
creation of "src" could be more disruptive and can wait until
$(topdir) is once again unbearable.
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHV2 2/2] t3404: cleanup double empty lines between tests

[Stefan Beller]

Signed-off-by: Stefan Beller 
---
 t/t3404-rebase-interactive.sh | 6 --
 1 file changed, 6 deletions(-)

diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index c8cc03d..f932639 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -771,7 +771,6 @@ test_expect_success 'rebase-i history with funny messages' '
test_cmp expect actual
 '
 
-
 test_expect_success 'prepare for rebase -i --exec' '
git checkout master &&
git checkout -b execute &&
@@ -780,7 +779,6 @@ test_expect_success 'prepare for rebase -i --exec' '
test_commit three_exec main.txt three_exec
 '
 
-
 test_expect_success 'running "git rebase -i --exec git show HEAD"' '
set_fake_editor &&
git rebase -i --exec "git show HEAD" HEAD~2 >actual &&
@@ -793,7 +791,6 @@ test_expect_success 'running "git rebase -i --exec git show 
HEAD"' '
test_cmp expected actual
 '
 
-
 test_expect_success 'running "git rebase --exec git show HEAD -i"' '
git reset --hard execute &&
set_fake_editor &&
@@ -807,7 +804,6 @@ test_expect_success 'running "git rebase --exec git show 
HEAD -i"' '
test_cmp expected actual
 '
 
-
 test_expect_success 'running "git rebase -ix git show HEAD"' '
git reset --hard execute &&
set_fake_editor &&
@@ -835,7 +831,6 @@ test_expect_success 'rebase -ix with several ' '
test_cmp expected actual
 '
 
-
 test_expect_success 'rebase -ix with several instances of --exec' '
git reset --hard execute &&
set_fake_editor &&
@@ -850,7 +845,6 @@ test_expect_success 'rebase -ix with several instances of 
--exec' '
test_cmp expected actual
 '
 
-
 test_expect_success 'rebase -ix with --autosquash' '
git reset --hard execute &&
git checkout -b autosquash &&
-- 
2.8.0.rc3.2.ga804a9e

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH] hashmap API: introduce for_each_hashmap_entry() helper macro

[Junio C Hamano]

Alexander Kuleshov  writes:

> diff --git a/hashmap.h b/hashmap.h
> index ab7958a..b8b158c 100644
> --- a/hashmap.h
> +++ b/hashmap.h
> @@ -95,4 +95,11 @@ static inline const char *strintern(const char *string)
>   return memintern(string, strlen(string));
>  }
>  
> +#define for_each_hashmap_entry(map, type)\
> + struct type *entry; \
> + struct hashmap_iter iter;   \
> + \
> + hashmap_iter_init(map, );  \
> + while ((entry = hashmap_iter_next()))
> +

This is an easy way to introduce decl-after-statement, i.e. needs an
extra pair of {} around the thing.  It also forbids the callers from
defining "entry" and "iter" as their own identifier outside the
scope of this macro and use them inside the block that is iterated
over by shadowing these two variables.

Other than that, it looks like a good concept.  The syntax however
needs more thought because of the above two issues, I think.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: parse-options does not recognize "unspecified" behavior

[Jeff King]

On Thu, Mar 17, 2016 at 01:21:49AM +0530, Pranit Bauva wrote:

> I noticed that parse-options does not recognize the variable which is
> set to -1 so as to denote the "unspecified" value.

Right. Like all of the stock parse-options handlers, it does not ever
read or understand the value passed to it by the caller. It only
increments or decrements.

> I did the following changes in builtin/commit.c (in master branch not
> the patch I am working on) :
>  - static int verbose = -1
>  - introduced a printf statement after parsing the options to print
> the value of verbose.
> 
> When I ran `git commit` :
>  I get the output that verbose is set to -1.
> 
> When I ran `git commit -v` :
> I get the output that verbose is set to 0.
> 
> When I ran `git commit -v -v` :
> I get the output that verbose is set to 1.
> 
> When I ran `git commit --no-verbose` :
> I get the out that verbose is set to 0.
> [...]
> It seems that parse-options just increments the value without
> considering the -1 flag to denote "unspecified value".
> 
> Is this a bug?

Not in parse-options, though I think setting verbose to "-1" in the
first place is wrong.

In general, parse-options does not know or care about the default values
that callers assign to variables; it just writes to them based on the
option-type specified by the caller. So the behavior for "commit",
"commit -v", and "commit -v -v" you show are perfectly reasonable.

But the one for "--no-verbose" is wrong. Parse-options has to write some
"reset" value, and it does not know what the initial default was. So it
writes 0. This is the same for options like OPT_SET_INT, and similar for
string options (where we set it to NULL).

So I think the caller choosing "-1" here as the "not set" value is the
bug.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 00/10] Adding RFC3161 time-stamps to tags

[Anton Wuerfel]

Hi all,

this patch series adds RFC3161 time-stamping functionality to git tags.
This enables users to create and verify cryptographic time stamp signatures to
prove that a commit existed at a certain point in time.

The new RFC3161 time-stamping functionality is independent from GPG signatures.
However, GPG signatures include time-stamp signatures and therefore make them
robust against removal or replacement.

The time stamp signature is stored in the git tag object header under the
`timesig` key, which is equivalent to the format for GPG signatures in signed
commits. This facilitates a possible implementation of time-stamped commits in
the future.

We added two new helper commands, because of their dependencies on libcurl and
libcrypto respectively. If one of the libraries is missing, a warning message is
given when one of the new functionalities is used.

We tried to add as much documentation as possible. Especially timestamp-util.c,
which depends on libcrypto, is a bit tough to read.

Some configuration variables have been added, where `ts.tsaurl` is the most
important, which specifies the Time Stamping Autority to use.

For testing the implemented functionality, the Time Stamping Authority
SafeCreative can be used, which is free for non-commercial use:
https://tsa.safecreative.org.
Simply set the config variable ts.tsaurl to this TSA.

Your feedback is welcome.
Kind regards,
Anton Wuerfel
Phillip Raffeck

Anton Wuerfel (10):
Phillip Raffeck (10):
  Add Testcases for time-stamping functionality
  Documentation for time-stamping functionality
  Documentation: Whitespace fix
  Extending internal CURL wrapper for POST upload
  Make PGP macros global
  Add basic RFC3161 functionality
  Add git-http-timestamp helper tool
  Adding time-stamping helper tool
  Add time-stamping functionality to git verify-tag
  Add time-stamping functionality to git tag

 .gitignore   |   2 +
 Documentation/config.txt |  20 ++
 Documentation/git-http-timestamp.txt |  32 ++
 Documentation/git-tag.txt|  32 +-
 Documentation/git-timestamp-util.txt |  52 +++
 Documentation/git-tsa-store.txt  |  68 
 Documentation/git-verify-tag.txt |  28 +-
 Makefile |  15 +
 builtin/tag.c|  55 +++-
 builtin/verify-tag.c |  61 +++-
 command-list.txt |   2 +
 gpg-interface.c  |   3 -
 gpg-interface.h  |   3 +
 http-timestamp.c |  76 +
 http.c   |  30 +-
 http.h   |  17 +-
 remote-curl.c|   2 +-
 rfc3161.c| 219 +
 rfc3161.h|  12 +
 t/t7031-verify-tag.sh|  69 
 timestamp-util.c | 615 +++
 21 files changed, 1380 insertions(+), 33 deletions(-)
 create mode 100644 Documentation/git-http-timestamp.txt
 create mode 100644 Documentation/git-timestamp-util.txt
 create mode 100644 Documentation/git-tsa-store.txt
 create mode 100644 http-timestamp.c
 create mode 100644 rfc3161.c
 create mode 100644 rfc3161.h
 create mode 100755 t/t7031-verify-tag.sh
 create mode 100644 timestamp-util.c

-- 
2.8.0.rc0.62.gfc8aefa.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] rebase -x: do not die without -i

[Junio C Hamano]

Stefan Beller  writes:

> In the later steps of preparing a patch series I do not want to edit the
> patches any more, but just make sure the test suite passes after each
> patch. Currently I would run
>
>   EDITOR=true git rebase -i  -x "make test"

Hmm, I guess that may "work" but it sounds like quite a roundabout
way to "test all commits".  "rebase" is about replaying history to
end up with a set of newly minted commits, and being able to poke at
the state each commit records in the working tree is a side effect.
"rebase -i" may use the same commit object if you didn't actually
make new commit as an optimization, but otherwise, it is like going
through pages of a book, tearing each page to examine it, and
replacing each page with a photocopy of it before going to examine
the next page.  Which makes me feel somewhat dirty X-<.

In other words, that looks like a workaround for not having

$ git for-each-rev -x "$command" old..new

where you can write "sh -c 'git checkout $1 && make test' -" as
your $command.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Windows download broken?

[Johannes Schindelin]

Hi Stefan,

On Wed, 16 Mar 2016, Stefan Beller wrote:

> On Wed, Mar 16, 2016 at 7:06 AM, Levente  wrote:
> > It seems that this link is broken:
> >
> > https://github.com/git-for-windows/git/releases/download/v2.7.3.windows.1/Git-2.7.3-64-bit.exe
> 
> I think Git for Windows is discussed mostly in the GitHub issues.
> Anywhere, cc'ing Johannes, who does Windows releases.

Correct, on both accounts. These days, I read the Git mailing list,
though, so all is good, people can report stuff here (I sometimes even
look at bug reports via Twitter, although they are all lacking too much
detail to be really useful) ;-)

Ciao,
Dscho
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/2] dir.c: fix dir re-inclusion rules with "NODIR" and "MUSTBEDIR"

[Durham Goode]

On 3/17/16 4:49 PM, Junio C Hamano wrote:

Thanks for these 5 patches, two of which need to be discarded ;-).
I think you can pick either one of 1/2, pick the one that says
"non-NULL" (as opposed to "something") in the log message for 2/2.

Durham, does it fix your issues if you apply the 1/2 and 2/2 (but
not 3/2) on top of 2.8-rc?

Duy, how comfortable are you with the idea of including this two in
2.8 final?  We have long passed the final -rc, and while it is
probably OK to prolong the cycle and do another -rc, we cannot keep
going like "oops, there is another thing discovered by somebod new"
forever.

Thanks.
Patches 1+2 fix the repro steps in the report, yes.  But I've found 
another case that produces different results in 2.8 than in 2.7:


Given a repo with files:

dir1/dir2/show/file
dir1/dir2/hide/file

and a sparse-checkout of

/*
/dir1/dir2/show
!/dir1/dir2/

the working copy still contains dir1/dir2/hide/file when run from 
2.8.0-rc2. In git 2.6 and 2.7.3 it does not show up (which is the 
expected behavior, from what I understand of the docs).  Repro script is 
below.  Notice, the 'dir2/' part of the paths is important.  If I drop 
that directory, the issue doesn't repro.



#!/bin/bash

set -x
rm -rf sparse-test
GIT=git
$GIT init sparse-test
cd sparse-test
$GIT config --add core.sparsecheckout true

mkdir -p dir1/dir2/show dir1/dir2/hide
touch dir1/dir2/show/file1
touch dir1/dir2/hide/file2

$GIT add .
$GIT commit -m "initial commit"
$GIT read-tree --reset -u HEAD

mkdir .git/info
cat > .git/info/sparse-checkout 

Re: [PATCH 0/1] http: Fix crash when passing malformed URL

[Jeff King]

On Wed, Mar 16, 2016 at 11:54:06AM +0100, Anton Wuerfel wrote:

> When passing a malformed URL to http_init() in http.c, git dies from a null
> pointer dereference. An example for a malformed URL is http:/git-scm.com (note
> the single slash after the protocol).
> 
> I could not reproduce this error within any functions of git - I just noticed 
> it
> during development of a git extension together with Phillip Raffeck.

You can reproduce it with:

  git clone https::bogus

Normally we recognize "https://; as the signal to send the URL to the
git-remote-https helper, but you can override the helper by specifying
"whatever::", and then the rest of the string will be passed to it.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] merge: refuse to create too cool a merge by default

[Ramsay Jones]

On 18/03/16 20:21, Junio C Hamano wrote:
> While it makes sense to allow merging unrelated histories of two
> projects that started independently into one, in the way "gitk" was
> merged to "git" itself aka "the coolest merge ever", such a merge is
> still an unusual event.  Worse, if somebody creates an independent
> history by starting from a tarball of an established project and
> sends a pull request to the original project, "git merge" however
> happily creates such a merge without any sign of something unusual
> is happening.
> 
> Teach "git merge" to refuse to create such a merge by default,
> unless the user passes a new "--allow-unrelated-histories" option to
> tell it that the user is aware that two unrelated projects are
> merged.
> 
> Because such a "two project merge" is a rare event, a configuration
> option to always allow such a merge is not added.
> 
> We could add the same option to "git pull" and have it passed
> through to underlying "git merge".  I do not have a fundamental
> opposition against such a feature, but this commit does not do so
> and instead leaves it as low-hanging fruit for others, because such
> a "two project merge" would be done after fetching the other project
> into some location in the working tree of an existing project and
> making sure how well they fit together, it is sufficient to allow a
> local merge without such an option pass-through from "git pull" to
> "git merge".  Many tests that are updated by this patch does the
> pass-through manually by turning:
> 
>   git pull something
> 
> into its equivalent:
> 
>   git fetch something &&
> git merge --allow-unrelated-histories FETCH_HEAD
> 
> If somebody is inclined to add such an option, updated tests in this
> change need to be adjusted back to:
> 
>   git pull --allow-unrelated-histories something
> 
> Signed-off-by: Junio C Hamano 
> ---
> 
>  builtin/merge.c | 12 +---
>  t/t3412-rebase-root.sh  |  2 +-
>  t/t5500-fetch-pack.sh   |  6 --
>  t/t6009-rev-list-parent.sh  |  4 +++-
>  t/t6010-merge-base.sh   |  6 --
>  t/t6012-rev-list-simplify.sh|  2 +-
>  t/t6026-merge-attr.sh   |  3 ++-
>  t/t6029-merge-subtree.sh|  2 +-
>  t/t6101-rev-parse-parents.sh|  2 +-
>  t/t9400-git-cvsserver-server.sh |  3 ++-
>  10 files changed, 28 insertions(+), 14 deletions(-)
> 

[snip]

> diff --git a/t/t6010-merge-base.sh b/t/t6010-merge-base.sh
> index 39b3238..e0c5f44 100755
> --- a/t/t6010-merge-base.sh
> +++ b/t/t6010-merge-base.sh
> @@ -215,11 +215,13 @@ test_expect_success 'criss-cross merge-base for 
> octopus-step' '
>   git reset --hard E &&
>   test_commit CC2 &&
>   test_tick &&
> - git merge -s ours CC1 &&
> + # E is a root commit unrelated to MMR root on which CC1 is based
> + git merge -s ours --allow-unrelated-histories CC1 &&
>   test_commit CC-o &&
>   test_commit CCB &&
>   git reset --hard CC1 &&
> - git merge -s ours CC2 &&
> + # E is a root commit unrelated to MMR root on which CC1 is based
> + git merge -s ours --allow-unrelated-histories CC2 &&

I was only skimming this patch, but the above caught my eye - I assume
that the comment should reference CC2 not CC1. yes?

ATB,
Ramsay Jones

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 12/17] unpack-trees: preserve index extensions

[David Turner]

Make git checkout (and other unpack_tree operations) preserve the
untracked cache and watchman status. This is valuable for two reasons:

1. Often, an unpack_tree operation will not touch large parts of the
working tree, and thus most of the untracked cache will continue to be
valid.

2. Even if the untracked cache were entirely invalidated by such an
operation, the user has signaled their intention to have such a cache,
and we don't want to throw it away.

The same logic applies to the watchman state.

Signed-off-by: David Turner 
---
 cache.h   |  1 +
 read-cache.c  |  8 
 t/t7063-status-untracked-cache.sh | 23 +++
 t/test-lib-functions.sh   |  4 
 unpack-trees.c|  1 +
 5 files changed, 37 insertions(+)

diff --git a/cache.h b/cache.h
index 9fa339a..4ae7dd0 100644
--- a/cache.h
+++ b/cache.h
@@ -572,6 +572,7 @@ extern void write_watchman_ext(struct strbuf *sb, struct 
index_state* istate);
 #define REFRESH_DAEMON (1 << 2)
 extern int write_locked_index(struct index_state *, struct lock_file *lock, 
unsigned flags);
 extern int discard_index(struct index_state *);
+extern void move_index_extensions(struct index_state *dst, struct index_state 
*src);
 extern int unmerged_index(const struct index_state *);
 extern int verify_path(const char *path);
 extern int index_dir_exists(struct index_state *istate, const char *name, int 
namelen);
diff --git a/read-cache.c b/read-cache.c
index 8e886d1..c141fec 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2742,3 +2742,11 @@ void stat_validity_update(struct stat_validity *sv, int 
fd)
fill_stat_data(sv->sd, );
}
 }
+
+void move_index_extensions(struct index_state *dst, struct index_state *src)
+{
+   dst->untracked = src->untracked;
+   src->untracked = NULL;
+   dst->last_update = src->last_update;
+   src->last_update = NULL;
+}
diff --git a/t/t7063-status-untracked-cache.sh 
b/t/t7063-status-untracked-cache.sh
index a971884..a2c8535 100755
--- a/t/t7063-status-untracked-cache.sh
+++ b/t/t7063-status-untracked-cache.sh
@@ -646,4 +646,27 @@ test_expect_success 'test ident field is working' '
test_cmp ../expect ../err
 '
 
+test_expect_success 'untracked cache survives a checkout' '
+   git commit --allow-empty -m empty &&
+   test-dump-untracked-cache >../before &&
+   test_when_finished  "git checkout master" &&
+   git checkout -b other_branch &&
+   test-dump-untracked-cache >../after &&
+   test_cmp ../before ../after &&
+   test_commit test &&
+   test-dump-untracked-cache >../before &&
+   git checkout master &&
+   test-dump-untracked-cache >../after &&
+   test_cmp ../before ../after
+'
+
+
+test_expect_success 'untracked cache survives a commit' '
+   test-dump-untracked-cache >../before &&
+   git add done/two &&
+   git commit -m commit &&
+   test-dump-untracked-cache >../after &&
+   test_cmp ../before ../after
+'
+
 test_done
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 8d99eb3..e974b5b 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -186,6 +186,10 @@ test_commit () {
test_tick
fi &&
git commit $signoff -m "$1" &&
+   if [ "$(git config core.bare)" = false ]
+   then
+   git update-index --force-untracked-cache
+   fi
git tag "${4:-$1}"
 }
 
diff --git a/unpack-trees.c b/unpack-trees.c
index 9f55cc2..fc90eb3 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1215,6 +1215,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, 
struct unpack_trees_options
  WRITE_TREE_SILENT |
  WRITE_TREE_REPAIR);
}
+   move_index_extensions(>result, o->dst_index);
discard_index(o->dst_index);
*o->dst_index = o->result;
} else {
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 15/17] index-helper: autorun mode

[David Turner]

Soon, we'll want to automatically start index-helper, so we need
a mode that silently exists if it can't start up (either because
it's not in a git dir, or because another one is already running).

Signed-off-by: David Turner 
---
 index-helper.c  | 29 +++--
 t/t7900-index-helper.sh |  8 
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/index-helper.c b/index-helper.c
index 7362abb..ab96ded 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -368,8 +368,9 @@ done:
 int main(int argc, char **argv)
 {
const char *prefix;
-   int idle_in_seconds = 600, detach = 0, kill = 0;
+   int idle_in_seconds = 600, detach = 0, kill = 0, autorun = 0;
int fd;
+   int nongit;
struct strbuf pipe_path = STRBUF_INIT;
struct option options[] = {
OPT_INTEGER(0, "exit-after", _in_seconds,
@@ -378,6 +379,7 @@ int main(int argc, char **argv)
 "verify shared memory after creating"),
OPT_BOOL(0, "detach", , "detach the process"),
OPT_BOOL(0, "kill", , "request that existing index helper 
processes exit"),
+   OPT_BOOL(0, "autorun", , "this is an automatic run of 
git index-helper, so certain errors can be solved by silently exiting"),
OPT_END()
};
 
@@ -387,7 +389,14 @@ int main(int argc, char **argv)
if (argc == 2 && !strcmp(argv[1], "-h"))
usage_with_options(usage_text, options);
 
-   prefix = setup_git_directory();
+   prefix = setup_git_directory_gently();
+   if (nongit) {
+   if (autorun)
+   exit(0);
+   else
+   die("Not a git repository");
+   }
+
if (parse_options(argc, (const char **)argv, prefix,
  options, usage_text, 0))
die(_("too many arguments"));
@@ -402,10 +411,18 @@ int main(int argc, char **argv)
 
/* check that no other copy is running */
fd = open(pipe_path.buf, O_RDONLY | O_NONBLOCK);
-   if (fd > 0)
-   die(_("Already running"));
-   if (errno != ENXIO && errno != ENOENT)
-   die_errno(_("Unexpected error checking pipe"));
+   if (fd > 0) {
+   if (autorun)
+   return 0;
+   else
+   die(_("Already running"));
+   }
+   if (errno != ENXIO && errno != ENOENT) {
+   if (autorun)
+   return 0;
+   else
+   die_errno(_("Unexpected error checking pipe"));
+   }
 
atexit(cleanup);
sigchain_push_common(cleanup_on_signal);
diff --git a/t/t7900-index-helper.sh b/t/t7900-index-helper.sh
index ce0cc27..6020fe4 100755
--- a/t/t7900-index-helper.sh
+++ b/t/t7900-index-helper.sh
@@ -33,4 +33,12 @@ test_expect_success 'index-helper will not start if already 
running' '
grep "Already running" err
 '
 
+test_expect_success 'index-helper is quiet with --autorun' '
+   test_when_finished "git index-helper --kill" &&
+   git index-helper --kill &&
+   git index-helper --detach &&
+   test -p .git/index-helper.pipe &&
+   git index-helper --autorun
+'
+
 test_done
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 17/17] read-cache: config for waiting for index-helper

[David Turner]

When we poke index-helper, and index-helper is using watchman, we want
to wait for a response (showing that the watchman extension shm has
been prepared).  If it's not using watchman, we don't.

So add a new config, core.waitforindexhelper, with sensible defaults, to
configure this behavior.

Signed-off-by: David Turner 
---
 cache.h   | 1 +
 config.c  | 5 +
 environment.c | 5 +
 read-cache.c  | 5 -
 4 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/cache.h b/cache.h
index 4ae7dd0..f8b8dbf 100644
--- a/cache.h
+++ b/cache.h
@@ -692,6 +692,7 @@ extern char *git_replace_ref_base;
 extern int fsync_object_files;
 extern int core_preload_index;
 extern int core_watchman_sync_timeout;
+extern int wait_for_index_helper;
 extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
diff --git a/config.c b/config.c
index e6dc141..5f1b8bd 100644
--- a/config.c
+++ b/config.c
@@ -887,6 +887,11 @@ static int git_default_core_config(const char *var, const 
char *value)
return 0;
}
 
+   if (!strcmp(var, "core.waitforindexhelper")) {
+   wait_for_index_helper = git_config_bool(var, value);
+   return 0;
+   }
+
if (!strcmp(var, "core.createobject")) {
if (!strcmp(value, "rename"))
object_creation_mode = OBJECT_CREATION_USES_RENAMES;
diff --git a/environment.c b/environment.c
index 35e03c7..c7fb0a9 100644
--- a/environment.c
+++ b/environment.c
@@ -95,6 +95,11 @@ int core_preload_index = 1;
 int ignore_untracked_cache_config;
 
 int core_watchman_sync_timeout = 300;
+#ifdef USE_WATCHMAN
+int wait_for_index_helper = 1;
+#else
+int wait_for_index_helper = 0;
+#endif
 
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
diff --git a/read-cache.c b/read-cache.c
index c141fec..7fd9b2c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1821,7 +1821,10 @@ static int poke_daemon(struct index_state *istate,
if (refresh_cache) {
ret = write_in_full(fd, "refresh", 8) == 8;
} else {
-   ret = poke_and_wait_for_reply(fd);
+   if (wait_for_index_helper)
+   ret = poke_and_wait_for_reply(fd);
+   else
+   ret = write_in_full(fd, "poke", 5) == 5;
}
 
close(fd);
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 14/17] index-helper: don't run if already running

[David Turner]

Signed-off-by: David Turner 
---
 index-helper.c  | 7 +++
 t/t7900-index-helper.sh | 9 +
 2 files changed, 16 insertions(+)

diff --git a/index-helper.c b/index-helper.c
index ffa3c2a..7362abb 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -400,6 +400,13 @@ int main(int argc, char **argv)
return 0;
}
 
+   /* check that no other copy is running */
+   fd = open(pipe_path.buf, O_RDONLY | O_NONBLOCK);
+   if (fd > 0)
+   die(_("Already running"));
+   if (errno != ENXIO && errno != ENOENT)
+   die_errno(_("Unexpected error checking pipe"));
+
atexit(cleanup);
sigchain_push_common(cleanup_on_signal);
 
diff --git a/t/t7900-index-helper.sh b/t/t7900-index-helper.sh
index 14b5a6c..ce0cc27 100755
--- a/t/t7900-index-helper.sh
+++ b/t/t7900-index-helper.sh
@@ -24,4 +24,13 @@ test_expect_success 'index-helper creates usable pipe file 
and can be killed' '
test_path_is_missing .git/index-helper.pipe
 '
 
+test_expect_success 'index-helper will not start if already running' '
+   test_when_finished "git index-helper --kill" &&
+   git index-helper --detach &&
+   test -p .git/index-helper.pipe &&
+   test_must_fail git index-helper 2>err &&
+   test -p .git/index-helper.pipe &&
+   grep "Already running" err
+'
+
 test_done
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 06/17] index-helper: add --detach

[David Turner]

From: Nguyễn Thái Ngọc Duy 

We detach after creating and opening the named pipe, because otherwise
we might return control to the shell before index-helper is ready to
accept commands.  This might lead to flaky tests.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Documentation/git-index-helper.txt |  3 +++
 index-helper.c | 11 +--
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
index 7afc5c9..b858a8d 100644
--- a/Documentation/git-index-helper.txt
+++ b/Documentation/git-index-helper.txt
@@ -31,6 +31,9 @@ OPTIONS
for reading an index, but because it will happen in the
background, it's not noticable. `--strict` is enabled by default.
 
+--detach::
+   Detach from the shell.
+
 NOTES
 -
 $GIT_DIR/index-helper.pipe is a named pipe that the daemon reads
diff --git a/index-helper.c b/index-helper.c
index 8d221e0..a854ed8 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -15,7 +15,7 @@ struct shm {
 
 static struct shm shm_index;
 static struct shm shm_base_index;
-static int to_verify = 1;
+static int daemonized, to_verify = 1;
 
 static void release_index_shm(struct shm *is)
 {
@@ -34,6 +34,8 @@ static void cleanup_shm(void)
 
 static void cleanup(void)
 {
+   if (daemonized)
+   return;
unlink(git_path("index-helper.pipe"));
cleanup_shm();
 }
@@ -229,7 +231,7 @@ static const char * const usage_text[] = {
 int main(int argc, char **argv)
 {
const char *prefix;
-   int idle_in_seconds = 600;
+   int idle_in_seconds = 600, detach = 0;
int fd;
struct strbuf pipe_path = STRBUF_INIT;
struct option options[] = {
@@ -237,6 +239,7 @@ int main(int argc, char **argv)
N_("exit if not used after some seconds")),
OPT_BOOL(0, "strict", _verify,
 "verify shared memory after creating"),
+   OPT_BOOL(0, "detach", , "detach the process"),
OPT_END()
};
 
@@ -258,6 +261,10 @@ int main(int argc, char **argv)
fd = setup_pipe(pipe_path.buf);
if (fd < 0)
die_errno("Could not set up index-helper.pipe");
+
+   if (detach && daemonize())
+   die_errno("unable to detach");
+
loop(fd, idle_in_seconds);
return 0;
 }
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 05/17] daemonize(): set a flag before exiting the main process

[David Turner]

From: Nguyễn Thái Ngọc Duy 

This allows signal handlers and atexit functions to realize this
situation and not clean up.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 builtin/gc.c | 2 +-
 cache.h  | 2 +-
 daemon.c | 2 +-
 setup.c  | 4 +++-
 4 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index c583aad..37180de 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -385,7 +385,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
 * failure to daemonize is ok, we'll continue
 * in foreground
 */
-   daemonized = !daemonize();
+   daemonized = !daemonize(NULL);
}
} else
add_repack_all_option();
diff --git a/cache.h b/cache.h
index 5805962..d386722 100644
--- a/cache.h
+++ b/cache.h
@@ -530,7 +530,7 @@ extern int set_git_dir_init(const char *git_dir, const char 
*real_git_dir, int);
 extern int init_db(const char *template_dir, unsigned int flags);
 
 extern void sanitize_stdfds(void);
-extern int daemonize(void);
+extern int daemonize(int *);
 
 #define alloc_nr(x) (((x)+16)*3/2)
 
diff --git a/daemon.c b/daemon.c
index 8d45c33..a5cf954 100644
--- a/daemon.c
+++ b/daemon.c
@@ -1365,7 +1365,7 @@ int main(int argc, char **argv)
return execute();
 
if (detach) {
-   if (daemonize())
+   if (daemonize(NULL))
die("--detach not supported on this platform");
} else
sanitize_stdfds();
diff --git a/setup.c b/setup.c
index de1a2a7..9adf13f 100644
--- a/setup.c
+++ b/setup.c
@@ -1017,7 +1017,7 @@ void sanitize_stdfds(void)
close(fd);
 }
 
-int daemonize(void)
+int daemonize(int *daemonized)
 {
 #ifdef NO_POSIX_GOODIES
errno = ENOSYS;
@@ -1029,6 +1029,8 @@ int daemonize(void)
case -1:
die_errno("fork failed");
default:
+   if (daemonized)
+   *daemonized = 1;
exit(0);
}
if (setsid() == -1)
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 03/17] index-helper: new daemon for caching index and related stuff

[David Turner]

From: Nguyễn Thái Ngọc Duy 

Instead of reading the index from disk and worrying about disk
corruption, the index is cached in memory (memory bit-flips happen
too, but hopefully less often). The result is faster read. Read time
is reduced by 70%.

The biggest gain is not having to verify the trailing SHA-1, which
takes lots of time especially on large index files. But this also
opens doors for further optimiztions:

 - we could create an in-memory format that's essentially the memory
   dump of the index to eliminate most of parsing/allocation
   overhead. The mmap'd memory can be used straight away. Experiment
   [1] shows we could reduce read time by 88%.

 - we could cache non-index info such as name hash

The shared memory's name folows the template "git--"
where  is the trailing SHA-1 of the index file.  is
"index" for cached index files (and may be "name-hash" for name-hash
cache). If such shared memory exists, it contains the same index
content as on disk. The content is already validated by the daemon and
git won't validate it again (except comparing the trailing SHA-1s).

We keep this daemon's logic as thin as possible. The "brain" stays in
git. So the daemon can read and validate stuff, but that's all it's
allowed to do. It does not add/create new information. It doesn't even
accept direct updates from git.

Git can poke the daemon via named pipes to tell it to refresh the
index cache, or to keep it alive some more minutes. It can't give any
real index data directly to the daemon. Real data goes to disk first,
then the daemon reads and verifies it from there. Poking only happens
for $GIT_DIR/index, not temporary index files.

$GIT_DIR/index-helper.pipe is the named pipe for daemon process. The
daemon reads from the pipe and executes commands.  Commands that need
replies from the daemon will have to open their own pipe, since a
named pipe should only have one reader.  Unix domain sockets don't
have this problem, but are less portable.

index-helper requires POSIX realtime extension. POSIX shm interface
however is abstracted away so that Windows support can be added later.

On webkit.git with index format v2, duplicating 8 times to 1.4m
entries and 200MB in size:

(vanilla)  0.986986364 s: read_index_from .git/index
(index-helper) 0.267850279 s: read_index_from .git/index

Interestingly with index v4, we get less out of index-helper. It makes
sense as v4 requires more processing after loading the index:

(vanilla)  0.72249 s: read_index_from .git/index
(index-helper) 0.302741500 s: read_index_from .git/index

Signed-off-by: Nguyễn Thái Ngọc Duy 
Signed-off-by: David Turner 
---
 .gitignore |   1 +
 Documentation/git-index-helper.txt |  46 
 Makefile   |   9 ++
 cache.h|   3 +
 config.mak.uname   |   1 +
 git-compat-util.h  |   1 +
 index-helper.c | 215 +
 read-cache.c   |  99 +++--
 shm.c  |  67 
 shm.h  |  23 
 t/t7900-index-helper.sh|  18 
 11 files changed, 474 insertions(+), 9 deletions(-)
 create mode 100644 Documentation/git-index-helper.txt
 create mode 100644 index-helper.c
 create mode 100644 shm.c
 create mode 100644 shm.h
 create mode 100755 t/t7900-index-helper.sh

diff --git a/.gitignore b/.gitignore
index 5087ce1..b92f122 100644
--- a/.gitignore
+++ b/.gitignore
@@ -71,6 +71,7 @@
 /git-http-fetch
 /git-http-push
 /git-imap-send
+/git-index-helper
 /git-index-pack
 /git-init
 /git-init-db
diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
new file mode 100644
index 000..59a5abc
--- /dev/null
+++ b/Documentation/git-index-helper.txt
@@ -0,0 +1,46 @@
+git-index-helper(1)
+===
+
+NAME
+
+git-index-helper - A simple cache daemon for speeding up index file access
+
+SYNOPSIS
+
+[verse]
+'git index-helper' [options]
+
+DESCRIPTION
+---
+Keep the index file in memory for faster access. This daemon is per
+repository.
+
+OPTIONS
+---
+
+--exit-after=::
+   Exit if the cached index is not accessed for ``
+   minutes. Specify 0 to wait forever. Default is 10.
+
+NOTES
+-
+$GIT_DIR/index-helper.pipe is a named pipe that the daemon reads
+commands from. At least on Linux, shared memory objects are availble
+via /dev/shm with the name pattern "git--".  Normally
+the daemon will clean up shared memory objects when it exits.  But if
+it crashes, some objects could remain there and they can be safely
+deleted with "rm" command. The following commands are used to control
+the daemon:
+
+"refresh"::
+   Reread the index.
+
+"poke":
+   Let the daemon know the index is to be read. It keeps the
+   daemon alive longer, unless `--exit-after=0` 

[PATCH v2 00/15] index-helper, watchman

[David Turner]

In this version:

I removed the statistics since they're not really a core part of the
series and we can always add them later.

I added a little more testing.

I merged the untracked cache/watchman mashup into the relevant
patches.  Hopefully I didn't screw it up -- I got into the rebase
weeds here and took a while to get out.  I wish I were better at
rebasing.

I moved the index-helper to a named-pipe based IPC mechanism instead
of a signal-based one.  I don't actualy know much about named pipes
other than what I learned while writing this, so someone with clue
should probably take a peek.

The index-helper pid files was kind of a hassle -- there was a
lot of mechanism involved in making sure that they were in fact
pointing to a live index-helper process.  With named pipes, we don't
really need them: if a live index-helper is running, it can simply be
instructed to die, and if it's not, then the pipe can be safely
removed and a new index-helper started.

Because there is no longer a pid file for index-helper, index-helper
has no way to let git know that it should wait for a watchman update
reply when poked.  I worked around this by making the waiting
configurable, and giving a sensible default.  Note that in the
no-watchman case, the wait will be short even if misconfigured,
because the daemon can immediately send an "OK" message.

I updated some commit messages, following Junio and Duy's suggestions.

Johannes Schindelin said that he would work on the Windows side, so
I've dropped the Windows bits (including one patch).  Since I don't
know anything about Windows, it's probably for the best that I'm not
coding for it.

I eliminated the ability to start up an index-helper when there was
already one running because I don't see why you would want to do that,
and because it would lead to multiple readers on the same named pipe,
which has undefined behavior.  Just kill the old one if you're done
with it.

David Turner (7):
  read-cache: invalidate untracked cache data when reading WAMA
  unpack-trees: preserve index extensions
  index-helper: kill mode
  index-helper: don't run if already running
  index-helper: autorun mode
  index-helper: optionally automatically run
  read-cache: config for waiting for index-helper

Nguyễn Thái Ngọc Duy (10):
  read-cache.c: fix constness of verify_hdr()
  read-cache: allow to keep mmap'd memory after reading
  index-helper: new daemon for caching index and related stuff
  index-helper: add --strict
  daemonize(): set a flag before exiting the main process
  index-helper: add --detach
  read-cache: add watchman 'WAMA' extension
  Add watchman support to reduce index refresh cost
  index-helper: use watchman to avoid refreshing index with lstat()
  update-index: enable/disable watchman support

 .gitignore |   1 +
 Documentation/git-index-helper.txt |  64 ++
 Makefile   |  21 ++
 builtin/gc.c   |   2 +-
 builtin/update-index.c |  11 +
 cache.h|  18 +-
 config.c   |  10 +
 config.mak.uname   |   1 +
 configure.ac   |   8 +
 daemon.c   |   2 +-
 dir.c  |  23 +-
 dir.h  |   6 +
 environment.c  |   8 +
 git-compat-util.h  |   1 +
 git.c  |  35 ++-
 index-helper.c | 439 +++
 read-cache.c   | 455 +++--
 setup.c|   4 +-
 shm.c  |  67 ++
 shm.h  |  23 ++
 t/t7063-status-untracked-cache.sh  |  23 ++
 t/t7900-index-helper.sh|  60 +
 t/test-lib-functions.sh|   4 +
 unpack-trees.c |   1 +
 watchman-support.c | 134 +++
 watchman-support.h |   7 +
 26 files changed, 1406 insertions(+), 22 deletions(-)
 create mode 100644 Documentation/git-index-helper.txt
 create mode 100644 index-helper.c
 create mode 100644 shm.c
 create mode 100644 shm.h
 create mode 100755 t/t7900-index-helper.sh
 create mode 100644 watchman-support.c
 create mode 100644 watchman-support.h

-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] merge: refuse to create too cool a merge by default

[David Turner]

On Fri, 2016-03-18 at 13:21 -0700, Junio C Hamano wrote:
>  Many tests that are updated by this patch does the
> pass-through manually by turning:

Nit: Should be many tests ... do

Also, I didn't notice a test that shows that "cool" merges without
allow-unrelated-histories are forbidden.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

BE PART OF DONALD TRUMP CAMPAIGN TEAM FOR PRESIDENT 2016

[izettasvg]

Hello how are you doing today ? Are you still looking for job ? You don't need 
to stress yourself before making money. We have an open opportunity for you to 
be part of our ongoing campaign for DONALD TRUMP FOR PRESIDENTIAL 2016 ? By 
simply putting a decal advertising of ( DONALD TRUMP FOR PRESIDENT 2016 ) where 
ever you want above: Car, Truck, Bike, Apt, Offices, Shop or Anywhere that is 
fitting to place an adverts so that it can attract a lot of peoples. Our 
company is the campaign organization for Trump 2016 Presidential bid.  You will 
be getting paid of $500 weekly. Let us know if you are interested in this 
offer. Kindly get back to me on my email  at ( john.brown9...@gmail.com ) for 
more detail to know how it works ? Note your subject most be I am interested.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] git-log should use the same diff-options as git-show

[Stefan Beller]

On Fri, Mar 18, 2016 at 2:47 PM, Henning Moll  wrote:
> Hi
>
> Recently i stumbled upon an old stash entry. It was clear to me that the
> stash only contained non-indexed worktree changes. So i assumed to get
> insight by doing
>
> $ git log -1 -p stash@{0}
>
> But surprisingly the result was "no patch" (The problem which i was not
> aware at that time was the fact that a stash commit is a merge). So i asked
> a question on stackoverflow (1) an learned that there are different default
> options used depending on the git command used:
>
> $ git show stash@{0}
> $ git diff stash@{0}^..stash@{0}
>
> work with default, but for git-log i need to
>
> $ git log -1 -p --cc stash@{0}
>
> to make it behave the same. This does not seem reasonable to me, though i
> read about commit 1aec791 (2) in git's own repository. What do you think?
>
> Maybe - as a compromise - just show any kind of hint instead of nothing?

Junio did some slight tweaks to --cc and -p,
see 82dee4160cc6d1b0d792c9f07b5803cd42abc610/
and its parent. That should be live in the upcoming 2.8.

>
> Best regards
> Henning
>
> (1) -
> http://stackoverflow.com/questions/36089674/git-log-1-p-stash0-shows-empty-patch
> (2) -
> https://git.kaarsemaker.net/git/commit/1aec7917dc52901c6df301ddc8fea70f5ce0db09/
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] Git v2.7.4 (and updates to older maintenance tracks)

[Torsten Bögershausen]

> Git v2.7.4 Release Notes
> 
> 
> Fixes since v2.7.3
> --
> 
>  * Bugfix patches were backported from the 'master' front to plug heap
>corruption holes, to catch integer overflow in the computation of
>pathname lengths, and to get rid of the name_path API.  Both of
>these would have resulted in writing over an under-allocated buffer
>when formulating pathnames while tree traversal.
> 
> 
> 
> Changes since v2.7.3 are as follows:
> 
> Jeff King (7):
>   add helpers for detecting size_t overflow
>   tree-diff: catch integer overflow in combine_diff_path allocation
>   http-push: stop using name_path
>   show_object_with_name: simplify by using path_name()
>   list-objects: convert name_path to a strbuf
>   list-objects: drop name_path entirely
>   list-objects: pass full pathname to callbacks
> 
If there is a new 2.7.x release, does it make sense to cherry-pick this one:

commit 7b6daf8d2fee1a9866b1d4eddbfaa5dbc42c5dbb
Author: Torsten Bögershausen 
Date:   Sun Feb 28 21:09:44 2016 +0100

config.mak.uname: use clang for Mac OS X 10.6


--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/2] dir.c: fix dir re-inclusion rules with "NODIR" and "MUSTBEDIR"

[Duy Nguyen]

On Sat, Mar 19, 2016 at 1:00 AM, Junio C Hamano  wrote:
> What is the ultimate goal of nd/exclusion-regression-fix topic over
> the behaviour before 2.7.0 and after 2.7.1 (there was a regression
> in 2.7.0 that was reverted in 2.7.1)?  From the cover letter of the
> series:
>
> Take one was bad and reverted in commit 8c72236. Take two provides a
> more complete solution to the pair of rules
>
>   exclude/this
>   !exclude/this/except/this
>
> 3/4 should do a better job at stopping regressions in take 1. 4/4
> provides the solution. I think I have tested (and wrote tests) for all
> the cases I can imagine.
>
> "solution to the pair of rules" hints there are some problem in the
> pair of rules, without stating what it is trying to solve.
>
> Isn't the root cause of the issue that treat_one_path() when
> deciding if it is worth descending into a directory check if the
> directory itself is excluded and returns path_excluded, even if some
> paths inside it may have a countermanding ! entries that would match
> them?  What if we change that part smarter and allow it to descend?

That's the easy part, detecting a pair and continue descending. The
problem is after you descend, exclude engine may return contradicting
results on old patterns. It's the same thing that 3/2 describes (after
you change patterns from "!dir\ndir/file2" to "dir\n!dir/file2").
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v8 1/2] parse-options.c: make OPTION__COUNTUP consider negative values

[Junio C Hamano]

Pranit Bauva  writes:

> The reason to make it consider negative values or more specifically
> "unspecified" values is to give the ability to differentiate between
> once, multiple time or with --no-option.
>
> Eg. :
> initialize verbose = -1
> `git commit` => verbose = -1
> `git commit -v` => verbose = 1
> `git commit -v -v` => verbose = 2
> `git commit --no-verbose` => verbose = 0

A few more things I noticed about this are:

 - Many uses of COUNTUP has now been replaced with BOOL and what
   remains are verbose/quiet/force.

 - This change will not affect existing users of COUNTUP at all, as
   long as they use the initial value of 0 (or more), as there is no
   mechanism to decrement.  The only thing the command line can do
   is to reset it to zero with "--no-foo".

So it seems a safe and sensible change.  Even though I suspect that
the justification can be written more clearly, I am not sure if it
worth the extra effort.

>
> Signed-off-by: Pranit Bauva 
>
> ---
> The discussion about this patch:
> [1] : http://thread.gmane.org/gmane.comp.version-control.git/289027
>
> Previous version of the patch:
> [v1] : http://thread.gmane.org/gmane.comp.version-control.git/289061
>
> Changes introduced:
> Use a different language in commit message to make the change and its
> utility more clear.
> ---
>  Documentation/technical/api-parse-options.txt | 6 --
>  parse-options.c   | 8 +++-
>  2 files changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/technical/api-parse-options.txt 
> b/Documentation/technical/api-parse-options.txt
> index 5f0757d..a908d8a 100644
> --- a/Documentation/technical/api-parse-options.txt
> +++ b/Documentation/technical/api-parse-options.txt
> @@ -144,8 +144,10 @@ There are some macros to easily define options:
>  
>  `OPT_COUNTUP(short, long, _var, description)`::
>   Introduce a count-up option.
> - `int_var` is incremented on each use of `--option`, and
> - reset to zero with `--no-option`.
> + If the `int_var` is negative and `--option` is absent,
> + then it will retain its value. Otherwise it will first set
> + `int_var` to 0 and then increment it on each use of `--option`,
> + and reset to zero with `--no-option`.
>  
>  `OPT_BIT(short, long, _var, description, mask)`::
>   Introduce a boolean option.
> diff --git a/parse-options.c b/parse-options.c
> index 47a9192..86d30bd 100644
> --- a/parse-options.c
> +++ b/parse-options.c
> @@ -110,7 +110,13 @@ static int get_value(struct parse_opt_ctx_t *p,
>   return 0;
>  
>   case OPTION_COUNTUP:
> - *(int *)opt->value = unset ? 0 : *(int *)opt->value + 1;
> + if (unset)
> + *(int *)opt->value = 0;
> + else {
> + if (*(int *)opt->value < 0)
> + *(int *)opt->value = 0;
> + *(int *)opt->value += 1;
> + }
>   return 0;
>  
>   case OPTION_SET_INT:
>
> --
> https://github.com/git/git/pull/213
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] GPIO bulk changes for kernel v4.6

[Linus Torvalds]

On Fri, Mar 18, 2016 at 9:37 AM, Junio C Hamano  wrote:
>
>> I don't think the original "resolve" did it, for example. You can't do
>> a three-way merge without a base.
>
> Yes, and that continues to this day:

Yeah, "octopus" also refuses it cleanly:

common=$(git merge-base --all $SHA1 $MRC) ||
die "Unable to find common commit with $pretty_name"

The code in the recursive merge that allows this to happen is this:

if (merged_common_ancestors == NULL) {
/* if there is no common ancestor, use an empty tree */
struct tree *tree;

tree = lookup_tree(EMPTY_TREE_SHA1_BIN);
merged_common_ancestors = make_virtual_commit(tree, "ancestor");
}

so the "no common ancestors" is just considered to be an empty merge base.

And I do think that's right, and I think it's clever, and it goes back to 2006:

  934d9a24078e merge-recur: if there is no common ancestor, fake empty one

but I think there should be an option there.

> This is a tangent but I wonder if we should say why we refuse to
> the standard error before calling these two "exit"s.

As mentioned, Octopus does.

That said, there's probably no reason to ever use the old three-way
merge, so I'm not even sure it's worth fixing the old
git-merge-resolve.

Linus
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] merge: refuse to create too cool a merge by default

[Junio C Hamano]

While it makes sense to allow merging unrelated histories of two
projects that started independently into one, in the way "gitk" was
merged to "git" itself aka "the coolest merge ever", such a merge is
still an unusual event.  Worse, if somebody creates an independent
history by starting from a tarball of an established project and
sends a pull request to the original project, "git merge" however
happily creates such a merge without any sign of something unusual
is happening.

Teach "git merge" to refuse to create such a merge by default,
unless the user passes a new "--allow-unrelated-histories" option to
tell it that the user is aware that two unrelated projects are
merged.

Because such a "two project merge" is a rare event, a configuration
option to always allow such a merge is not added.

We could add the same option to "git pull" and have it passed
through to underlying "git merge".  I do not have a fundamental
opposition against such a feature, but this commit does not do so
and instead leaves it as low-hanging fruit for others, because such
a "two project merge" would be done after fetching the other project
into some location in the working tree of an existing project and
making sure how well they fit together, it is sufficient to allow a
local merge without such an option pass-through from "git pull" to
"git merge".  Many tests that are updated by this patch does the
pass-through manually by turning:

git pull something

into its equivalent:

git fetch something &&
git merge --allow-unrelated-histories FETCH_HEAD

If somebody is inclined to add such an option, updated tests in this
change need to be adjusted back to:

git pull --allow-unrelated-histories something

Signed-off-by: Junio C Hamano 
---

 builtin/merge.c | 12 +---
 t/t3412-rebase-root.sh  |  2 +-
 t/t5500-fetch-pack.sh   |  6 --
 t/t6009-rev-list-parent.sh  |  4 +++-
 t/t6010-merge-base.sh   |  6 --
 t/t6012-rev-list-simplify.sh|  2 +-
 t/t6026-merge-attr.sh   |  3 ++-
 t/t6029-merge-subtree.sh|  2 +-
 t/t6101-rev-parse-parents.sh|  2 +-
 t/t9400-git-cvsserver-server.sh |  3 ++-
 10 files changed, 28 insertions(+), 14 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 101ffef..e3db41b 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -64,6 +64,7 @@ static int option_renormalize;
 static int verbosity;
 static int allow_rerere_auto;
 static int abort_current_merge;
+static int allow_unrelated_histories;
 static int show_progress = -1;
 static int default_to_upstream = 1;
 static const char *sign_commit;
@@ -221,6 +222,8 @@ static struct option builtin_merge_options[] = {
OPT__VERBOSITY(),
OPT_BOOL(0, "abort", _current_merge,
N_("abort the current in-progress merge")),
+   OPT_BOOL(0, "allow-unrelated-histories", _unrelated_histories,
+N_("allow merging unrelated histories")),
OPT_SET_INT(0, "progress", _progress, N_("force progress 
reporting"), 1),
{ OPTION_STRING, 'S', "gpg-sign", _commit, N_("key-id"),
  N_("GPG sign commit"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
@@ -1397,9 +1400,12 @@ int cmd_merge(int argc, const char **argv, const char 
*prefix)
update_ref("updating ORIG_HEAD", "ORIG_HEAD", 
head_commit->object.oid.hash,
   NULL, 0, UPDATE_REFS_DIE_ON_ERR);
 
-   if (remoteheads && !common)
-   ; /* No common ancestors found. We need a real merge. */
-   else if (!remoteheads ||
+   if (remoteheads && !common) {
+   /* No common ancestors found. */
+   if (!allow_unrelated_histories)
+   die(_("refusing to merge unrelated histories"));
+   /* otherwise, we need a real merge. */
+   } else if (!remoteheads ||
 (!remoteheads->next && !common->next &&
  common->item == remoteheads->item)) {
/*
diff --git a/t/t3412-rebase-root.sh b/t/t3412-rebase-root.sh
index 0b52105..73a39f2 100755
--- a/t/t3412-rebase-root.sh
+++ b/t/t3412-rebase-root.sh
@@ -133,7 +133,7 @@ test_expect_success 'set up second root and merge' '
rm A B C &&
test_commit 6 D &&
git checkout other &&
-   git merge third
+   git merge --allow-unrelated-histories third
 '
 
 cat > expect-third <<'EOF'
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index e5f83bf..4fca623 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -259,7 +259,8 @@ test_expect_success 'clone shallow object count' '
 test_expect_success 'pull in shallow repo with missing merge base' '
(
cd shallow &&
-   test_must_fail git pull --depth 4 .. A
+   git fetch --depth 4 .. A
+   test_must_fail git merge --allow-unrelated-histories FETCH_HEAD
)
 '
 
@@ -279,9 +280,10 @@ test_expect_success 'clone 

RE: 2.8.0 gitignore enhancement not working as expected

[Richard Furness -X (rfurness - ENSOFT LIMITED at Cisco)]

Hi Duy,

That seems to have fixed it :-)

Thanks for your help!

Richard

> -Original Message-
> From: Duy Nguyen [mailto:pclo...@gmail.com]
> Sent: 18 March 2016 14:53
> To: Richard Furness -X (rfurness - ENSOFT LIMITED at Cisco)
> 
> Cc: git@vger.kernel.org
> Subject: Re: 2.8.0 gitignore enhancement not working as expected
> 
> On Fri, Mar 18, 2016 at 9:32 PM, Richard Furness -X (rfurness - ENSOFT
> LIMITED at Cisco)  wrote:
> > Hi Duy,
> >
> > I tried your exact example and it worked correctly. But then I tried adding
> some more files/dirs at the top level and I still see an issue:
> 
> Thank you. Phew.. I bet you hit the same bug we found yesterday (your
> trace suggests so). Can you try this patch [1] just to confirm?
> 
> [1] http://article.gmane.org/gmane.comp.version-control.git/289101
> --
> Duy
N�r��yb�X��ǧv�^�)޺{.n�+ا���ܨ}���Ơz�:+v���zZ+��+zf���h���~i���z��w���?�&�)ߢf

[PATCH] pretty-print: de-tabify indented logs to make things line up properly

[Linus Torvalds]

From: Linus Torvalds 
Date: Wed, 16 Mar 2016 09:15:53 -0700
Subject: [PATCH] pretty-print: de-tabify indented logs to make things line up 
properly

This should all line up:

  Column 1  Column 2
    
  A B
  ABCD  EFGH
  SPACESInstead of Tabs

Even with multi-byte UTF8 characters:

  Column 1  Column 2
    
  Ä B
  åäö   100
  A Møøse   once bit my sister..

Signed-off-by: Linus Torvalds 
---

This seems to work for me, and while there is some cost, it's minimal. 
Doing a "git log > /dev/null" of the current git tree is about 1% slower 
because of the tab-finding. A tree with a lot of tabs in the commit 
messages would be more noticeable, because then you actually end up 
hitting the whole "how wide is this" issue.

(But if the tabs are all at the beginning of a line, you'd still be ok 
and avoid the utf8 width calculations).

Comments?

 pretty.c | 76 ++--
 1 file changed, 74 insertions(+), 2 deletions(-)

diff --git a/pretty.c b/pretty.c
index 92b2870a7eab..0b40457f99f0 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1629,6 +1629,76 @@ void pp_title_line(struct pretty_print_context *pp,
strbuf_release();
 }
 
+static int pp_utf8_width(const char *start, const char *end)
+{
+   int width = 0;
+   size_t remain = end - start;
+
+   while (remain) {
+   int n = utf8_width(, );
+   if (n < 0 || !start)
+   return -1;
+   width += n;
+   }
+   return width;
+}
+
+/*
+ * pp_handle_indent() prints out the intendation, and
+ * perhaps the whole line (without the final newline)
+ *
+ * Why "perhaps"? If there are tabs in the indented line
+ * it will print it out in order to de-tabify the line.
+ *
+ * But if there are no tabs, we just fall back on the
+ * normal "print the whole line".
+ */
+static int pp_handle_indent(struct strbuf *sb, int indent,
+const char *line, int linelen)
+{
+   const char *tab;
+
+   strbuf_addchars(sb, ' ', indent);
+
+   tab = memchr(line, '\t', linelen);
+   if (!tab)
+   return 0;
+
+   do {
+   int width = pp_utf8_width(line, tab);
+
+   /*
+* If it wasn't well-formed utf8, or it
+* had characters with badly defined
+* width (control characters etc), just
+* give up on trying to align things.
+*/
+   if (width < 0)
+   break;
+
+   /* Output the data .. */
+   strbuf_add(sb, line, tab - line);
+
+   /* .. and the de-tabified tab */
+   strbuf_addchars(sb, ' ', 8-(width & 7));
+
+   /* Skip over the printed part .. */
+   linelen -= 1+tab-line;
+   line = tab + 1;
+
+   /* .. and look for the next tab */
+   tab = memchr(line, '\t', linelen);
+   } while (tab);
+
+   /*
+* Print out everything after the last tab without
+* worrying about width - there's nothing more to
+* align.
+*/
+   strbuf_add(sb, line, linelen);
+   return 1;
+}
+
 void pp_remainder(struct pretty_print_context *pp,
  const char **msg_p,
  struct strbuf *sb,
@@ -1652,8 +1722,10 @@ void pp_remainder(struct pretty_print_context *pp,
first = 0;
 
strbuf_grow(sb, linelen + indent + 20);
-   if (indent)
-   strbuf_addchars(sb, ' ', indent);
+   if (indent) {
+   if (pp_handle_indent(sb, indent, line, linelen))
+   linelen = 0;
+   }
strbuf_add(sb, line, linelen);
strbuf_addch(sb, '\n');
}
-- 
2.8.0.rc2
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: parse-options does not recognize "unspecified" behavior

[Jeff King]

On Wed, Mar 16, 2016 at 04:33:03PM -0700, Stefan Beller wrote:

> The way I understand verbosity is this:
> * You can instruct a command to be more verbose if it is supported by
> adding more -v
> * --no-verbose tells the command to be not verbose, i.e. no additional output.
> 
> So my question was, if there is any command, which is verbose by
> default, and --no-verbose would make it "more quiet"? Such a case
> would be a UX bug, as a user would rather expect --quiet instead of
> --no-verbose IMO. Would such a case ever happen, that we'd want to
> give --no-verbose to decrease the amount said by the command?

Ah, I see. I agree that would be a bug, because --no-verbose is not
"more quiet". It is "cancel all previous -v". The right way to spell
that is "--quiet" (usually, see below).

> IIRC some commands use one integer variable to determine
> the amount of output, i.e. --verbose increases that variable, --quiet
> decreases it.
> What happens for example with
> 
>   git commit -v --no-verbose -v -q -q --no-quiet
> 
> In case of commit, the quietness and verbosity is done in 2 variables,
> so these are orthogonal to each other, there are even no internal checks for
> (verbosity > 0 && quietness > 0) at the same time, so it seems to be a valid
> command.

Yes, I think in general, "-v" and "-q" should work as opposites. But
that is not the case with commit, where "-v" and "-q" operate on totally
separate messages. I think that is a UX mistake, and we would not do
it that way if designing from scratch. But we're stuck with it for
historical reasons (I'd probably name "--verbose" as "--show-diff" or
something if writing it today).

Arguably cmd_commit() should be using OPT_BOOL instead of OPT__VERBOSE,
as there is no such thing as "verbose > 1" here. But I don't think there
is any real user-facing consequence of that (however, given Eric's
suggestion, I suspect it would make Pranit's problem just go away, as it
assigns rather than increments; IOW, it does the thing Eric was
suggestion OPT__VERBOSE to do).

> In case of a command where this is tracked in one variable,
> 
>   git  -v --no-verbose -v -q -q --no-quiet
> 
> you'd expect:
> 
>   verbosity++ // because of first -v
>   verbosity = 0 // because of the reset with --no-verbose
>   verbosity++ // because of second -v
>   verbosity-- // because of first -q
>   verbosity-- // because of second -q
>   verbosity = 0 // because of the reset with --no-quiet
> 
> Having typed that, I think my comment was not adding value to
> the discussion, as --no-{verbose/quiet} would just reset it to 0 no matter
> if you track verbosity/quietness in one or two variables internally.

Right, in a command using OPT_VERBOSITY(), that is how it should (and
does) work. I think "commit" is just funny for historical reasons.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: An idea for the "Leftover Bits": rebase -x to imply -i

[Junio C Hamano]

Junio C Hamano  writes:

> The list is my personal collection of "leftover" things, i.e. topics
> that were raised on the list, perhaps already discussed or perhaps
> nobody thought them interesting, that I found when re-reading the
> past list traffic that did not reach a useful conclusion to result
> in a patch (or resolution, a shared understanding, that it is not a
> good idea).  Getting added to the list should not be a goal.
>
> Your message is perhaps the least effective way to add an item to
> the list.  It hasn't been discussed here, nobody seems to have felt
> it is a good idea, and I didn't think it is particularly interesting
> myself (at least not yet).

Having said that, I could use help in maintaining the collection.

A few "characteristics" of that list, that cannot be updated by
anybody but me (because it is just my personal collection after all)
are:

 * I do not have to worry about useless new entries that do not
   align the overall system design getting added by clueless people.

 * Adding new entry after scanning past list traffic and finding a
   still unresolved topic that "died down" is relatively easy.

 * Removing existing entries because the topic was revived on the
   list and completed is _hard_, as that is merely an administrative
   overhead to me.

Moving it to a public wiki would lose the first point, which is a
benefit.

Merely making it public does not guarantee that the third point
(i.e. clean-up) would happen more easily unless we have volunteers.
It is likely that the "leftover bits" list would go stale just like
other people's wikis or bug trackers.

On the other hand, if we do have volunteers who are scanning for
"stale" items in the "leftover bits" list to tell me "that item was
done with commit c0ffee1eaf" and that would help me quite a bit
without being it a public wiki/bug tracker.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHv3 1/2] rebase: decouple --exec from --interactive

[Stefan Beller]

In the later steps of preparing a patch series I do not want to
edit or reorder the patches any more, but just make sure the
test suite passes after each patch and also to fix breakage
right there if some of the steps fail.  I could run

EDITOR=true git rebase -i  -x "make test"

but it would be simpler if it can be spelled like so:

git rebase  -x "make test"

Signed-off-by: Stefan Beller 
---

  Thanks Junio, Johannes for review!
 
 * Reworded the commit message (took your suggestion)
 
 * Diff to v2 in t3404:
test_expect_success 'rebase --exec works without -i ' '
git reset --hard execute &&
rm -rf exec_output &&
-   git rebase --exec "echo a line >>exec_output"  HEAD~2 2>actual 
&&
+   EDITOR="echo >invoked_editor" git rebase --exec "echo a line 
>>exec_output"  HEAD~2 2>actual &&
test_i18ngrep  "Successfully rebased and updated" actual &&
-   test_line_count = 2 exec_output
+   test_line_count = 2 exec_output &&
+   test_path_is_missing invoked_editor
'
  * I just resend this patch instead of the whole series, so do not expect a
[PATCHv3 2/2] nor cover letter 0/2


 Documentation/git-rebase.txt  |  6 +++---
 git-rebase.sh |  7 +--
 t/t3404-rebase-interactive.sh | 13 ++---
 3 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 6ed610a..0387b40 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -391,9 +391,6 @@ idea unless you know what you are doing (see BUGS below).
final history.  will be interpreted as one or more shell
commands.
 +
-This option can only be used with the `--interactive` option
-(see INTERACTIVE MODE below).
-+
 You may execute several commands by either using one instance of `--exec`
 with several commands:
 +
@@ -406,6 +403,9 @@ or by giving more than one `--exec`:
 If `--autosquash` is used, "exec" lines will not be appended for
 the intermediate commits, and will only appear at the end of each
 squash/fixup series.
++
+This uses the `--interactive` machinery internally, but it can be run
+without an explicit `--interactive`.
 
 --root::
Rebase all commits reachable from , instead of
diff --git a/git-rebase.sh b/git-rebase.sh
index cf60c43..0bf41ee 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -248,6 +248,7 @@ do
;;
--exec=*)
cmd="${cmd}exec ${1#--exec=}${LF}"
+   test -z "$interactive_rebase" && interactive_rebase=implied
;;
--interactive)
interactive_rebase=explicit
@@ -348,12 +349,6 @@ do
 done
 test $# -gt 2 && usage
 
-if test -n "$cmd" &&
-   test "$interactive_rebase" != explicit
-then
-   die "$(gettext "The --exec option must be used with the --interactive 
option")"
-fi
-
 if test -n "$action"
 then
test -z "$in_progress" && die "$(gettext "No rebase in progress?")"
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 544f9ad..21b1f95 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -876,16 +876,15 @@ test_expect_success 'rebase -ix with --autosquash' '
test_cmp expected actual
 '
 
-
-test_expect_success 'rebase --exec without -i shows error message' '
+test_expect_success 'rebase --exec works without -i ' '
git reset --hard execute &&
-   set_fake_editor &&
-   test_must_fail git rebase --exec "git show HEAD" HEAD~2 2>actual &&
-   echo "The --exec option must be used with the --interactive option" 
>expected &&
-   test_i18ncmp expected actual
+   rm -rf exec_output &&
+   EDITOR="echo >invoked_editor" git rebase --exec "echo a line 
>>exec_output"  HEAD~2 2>actual &&
+   test_i18ngrep  "Successfully rebased and updated" actual &&
+   test_line_count = 2 exec_output &&
+   test_path_is_missing invoked_editor
 '
 
-
 test_expect_success 'rebase -i --exec without ' '
git reset --hard execute &&
set_fake_editor &&
-- 
2.8.0.rc3.10.gafa8710.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Tabs in commit messages - de-tabify option in strbuf_stripspace()?

[Junio C Hamano]

Linus Torvalds  writes:

> On Wed, Mar 16, 2016 at 7:27 AM, Marc Branchaud  wrote:
>>
>> Could this also help with diff output, where the leading + or - mars the
>> indentation in a similar way?
>
> I don't think that's a good idea at least by default, since then it
> will break things like running diff in emacs capture mode.
>
> So even if you're in a terminal, you can't assume that you can munge
> the output too much.
>
> Of course, if colorization is on, you might as well pretty-print the
> diff by indenting things properly too, since the end result isn't
> going to be used as a _diff_.

I agree that I will never use such an end result as a diff, but I
may still be tempted to cut-and-paste individual lines after '^+'
when resurrecting a WIP topic that does not rebase very well, so I
agree with you that the output shouldn't be munged by default even
though I think it is OK to have an option..


--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 18/19] index-helper: autorun

[Duy Nguyen]

On Thu, Mar 17, 2016 at 9:43 PM, Johannes Schindelin
 wrote:
> Hi Duy,
>
> On Thu, 17 Mar 2016, Duy Nguyen wrote:
>
>> On Thu, Mar 17, 2016 at 1:27 AM, Johannes Schindelin
>>  wrote:
>> > I am much more concerned about concurrent accesses and the communication
>> > between the Git processes and the index-helper. Writing to the .pid file
>> > sounds very fragile to me, in particular when multiple processes can poke
>> > the index-helper in succession and some readers are unaware that the index
>> > is being refreshed.
>>
>> It's not that bad.
>
> Well, the way I read the code it is possible that:
>
> 1. Git process 1 starts, reading the index
> 2. Git process 2 starts, poking the index-helper
> 3. The index-helper updates the .pid file (why not set a bit in the shared
>memory?) with a prefix "W"
> 4. Git process 2 reads the .pid file and waits for the "W" to go away
>(what if index-helper is not fast enough to write the "W"?)
> 5. Git process 1 access the index, happily oblivious that it is being
>updated and the data is in an inconsistent state

No, if process 1 reads the index file, then that file will remain
consistent/unchanged all the time. index-helper is not allowed to
touch that file at all.

The process 2 gets the index content from shm (cached by the index
helper), verifies that it's good (with the signature at the end of the
shm). If watchman is used, process 2 can also read the list of
modified files from another shm, combine it with the in-core index,
then write it down the normal way. Only then process 1 (or process 3)
can see the new index content from the file.

>> We should have protection in place to deal with this and fall back to
>> reading directly from file when things get suspicious.
>
> I really want to prevent that. I know of use cases where the index weighs
> 300MB, and falling back to reading it directly *really* hurts.

For crying out loud, what do you store in that repo? What I have in
mind for all these works are indexes in 10MB range, or maybe 50MB max.

Very unscientifically, git.git index is about 274kb and contains ~3000
entries, so 94 bytes per entry on average. With a 300MB index , the
extrapolated number of entries is about 3 millions! At around 1
million index entries, I think it's time to just use a database as
index.

>> But I agree that sending UNIX signals (or PostMessage) is not really
>> good communication.
>
> Yeah, I really would like two-way communication instead. Named pipes?
> They'd have the advantage that you could use the full path to the index as
> identifier.

Yep.

> The way I read the current code, we would actually create a different
> shared memory every time the index changes because its checksum is part of
> the shared memory's "path"...

Yep. shm objects are "immutable", pretty much like git objects. But
now that I think of it, I don't know how cheap/expensive shm creation
operation is on Windows.
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH/RFC] parse-options.c: make OPTION__COUNTUP consider negative values

[Pranit Bauva]

On Thu, Mar 17, 2016 at 12:58 PM, Eric Sunshine  wrote:
> On Wed, Mar 16, 2016 at 9:50 PM, Jeff King  wrote:
>> On Wed, Mar 16, 2016 at 11:16:58PM +, Pranit Bauva wrote:
>>> The reason to make it consider negative values or more specifically
>>> "unspecified" values is to differentiate between the option passed
>>> once, multiple times or with --no-option. This makes the receiver
>
> This is inaccurate and rather confusing. It's not that an
> "unspecified" value gives you the ability to "differentiate between
> once, multiple time or with --no-option", but rather that it allows
> you to determine wether --option or --no-option was encountered at
> all.

I guess the language wasn't very clear. I will change this.

>>> know what actually happened with the arguments which is particularly
>>> required with option have multiple levels of that option.
>>>
>>> Eg. :
>>> initialize verbose = -1
>>> `git commit` => verbose = -1
>>> `git commit -v` => verbose = 1
>>> `git commit -v -v` => verbose = 1
>>> `git commit --no-verbose` => verbose = 0
>>
>> This second to last example would be 2, right?
>
> Right.

My bad, it should be 2.

>
> I'm not sure that this example block is helpful, though. A clearer
> commit message which does a better job of explaining the reason for
> the change would likely eliminate the need for an example.
>
>> That aside, this patch does mean that one can no longer use
>> OPT_COUNTUP() for negative values (i.e., the caller must start it at
>> either 0 or 1, and it must always go up from there).
>>
>> And we would need to verify that all of the existing callers are OK with
>> this. Did you check that that (not rhetorical; I suspect they are all
>> OK, but somebody needs to check)?
>>
>> We are also changing semantics without changing the interface, which
>> means any topics in flight (that you _cannot_ review, because you have
>> not seen them yet) may be subtly broken. To me that is not an absolute
>> deal-breaker, but something to weigh against the utility of the change.
>
> Indeed, I was envisioning a more conservative approach of having
> OPT__VERBOSE use a custom callback or perhaps introducing a new
> 'flags' value or even (probably too ugly) abusing the 'defval' field
> to specially indicate that it wants the "negative means unspecified"
> behavior; the other consumers of OPT_COUNTUP would not request this
> special behavior. But, as you say, changing the behavior of
> OPT_COUNTUP unconditionally may not be a deal-breaker.
>
> I also realized that Pranit can achieve the desired behavior without
> modifying OPT__VERBOSE at all. Specifically, rather than initializing
> his opt_verbose variable to -1, he can instead initialize it to 1.
> Then:
>
> * if --verbose is seen (one or more times), opt_verbose will be >=2,
> and the real verbosity level will be (opt_verbose - 1)
>
> * if --no-verbose is seen, opt_verbose will be 0
>
> * if neither is seen, then opt_verbose will remain 1
>
> However, I think this approach is far too ugly and non-obvious to
> seriously suggest using it, whereas the change to OPT__VERBOSE is
> easily understood and could prove useful in the future for other
> commands with multiple verbosity levels.
>
>> When looking more carefully at builtin/commit.c for the other thread, it
>> occurred to me that OPT_BOOL might be a better fit for commit's "-v". It
>> really is a boolean "show the diff or not" and thus unlike the other
>> "make me more verbose". And OPT_BOOL already has the behavior you want,
>> I think.
>
> For completeness (for readers of this thread), it was pointed out in
> the other thread[1] that git-commit does indeed recognize multiple
> verbosity levels, so changing it to use OPT_BOOL would be undesirable
> (wrong).
>
> [1]: 
> http://thread.gmane.org/gmane.comp.version-control.git/289027/focus=289074
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 16/17] index-helper: optionally automatically run

[David Turner]

Introduce a new config option, indexhelper.autorun, to automatically
run git index-helper before starting up a builtin git command.  This
enables users to keep index-helper running without manual
intervention.

Signed-off-by: David Turner 
---
 git.c   | 35 ++-
 t/t7900-index-helper.sh | 16 
 2 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/git.c b/git.c
index 6cc0c07..7d27782 100644
--- a/git.c
+++ b/git.c
@@ -521,6 +521,37 @@ static void strip_extension(const char **argv)
 #define strip_extension(cmd)
 #endif
 
+static int want_auto_index_helper(void)
+{
+   int want = -1;
+   const char *value = NULL;
+   const char *key = "indexhelper.autorun";
+
+   if (git_config_key_is_valid(key) &&
+   !git_config_get_value(key, )) {
+   int b = git_config_maybe_bool(key, value);
+   want = b >= 0 ? b : 0;
+   }
+   return want;
+}
+
+static void maybe_run_index_helper(struct cmd_struct *cmd)
+{
+   const char *argv[] = {"git-index-helper", "--detach", "--autorun", 
NULL};
+
+   if (!(cmd->option & NEED_WORK_TREE))
+   return;
+
+   if (want_auto_index_helper() <= 0)
+   return;
+
+   trace_argv_printf(argv, "trace: auto index-helper:");
+
+   if (run_command_v_opt(argv,
+ RUN_SILENT_EXEC_FAILURE | RUN_CLEAN_ON_EXIT))
+   warning(_("You specified indexhelper.autorun, but running 
git-index-helper failed."));
+}
+
 static void handle_builtin(int argc, const char **argv)
 {
const char *cmd;
@@ -536,8 +567,10 @@ static void handle_builtin(int argc, const char **argv)
}
 
builtin = get_builtin(cmd);
-   if (builtin)
+   if (builtin) {
+   maybe_run_index_helper(builtin);
exit(run_builtin(builtin, argc, argv));
+   }
 }
 
 static void execv_dashed_external(const char **argv)
diff --git a/t/t7900-index-helper.sh b/t/t7900-index-helper.sh
index 6020fe4..7fe66b7 100755
--- a/t/t7900-index-helper.sh
+++ b/t/t7900-index-helper.sh
@@ -41,4 +41,20 @@ test_expect_success 'index-helper is quiet with --autorun' '
git index-helper --autorun
 '
 
+test_expect_success 'index-helper autorun works' '
+   rm -f .git/index-helper.pipe &&
+   git status &&
+   test_path_is_missing .git/index-helper.pipe &&
+   test_config indexhelper.autorun true &&
+   git status &&
+   test -p .git/index-helper.pipe &&
+   git status 2>err &&
+   test -p .git/index-helper.pipe &&
+   ! grep -q . err &&
+   git index-helper --kill &&
+   test_config indexhelper.autorun false &&
+   git status &&
+   test_path_is_missing .git/index-helper.pipe
+'
+
 test_done
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 13/17] index-helper: kill mode

[David Turner]

Add a new command (and command-line arg) to allow index-helpers to
exit cleanly.

This is mainly useful for tests.

Signed-off-by: David Turner 
---
 index-helper.c  | 39 +--
 t/t7900-index-helper.sh |  9 +
 2 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/index-helper.c b/index-helper.c
index 445a0ac..ffa3c2a 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -286,6 +286,8 @@ static void loop(int fd, int idle_in_seconds)
 * alive, nothing to do.
 */
;
+   } else if (!strcmp(command.buf, "die")) {
+   break;
} else {
warning("BUG: Bogus command %s", 
command.buf);
}
@@ -338,10 +340,35 @@ static const char * const usage_text[] = {
NULL
 };
 
+static void request_kill(const char *pipe_path)
+{
+   int fd;
+
+   fd = open(pipe_path, O_WRONLY | O_NONBLOCK);
+   if (fd < 0) {
+   warning("No existing pipe; can't send kill message to old 
process");
+   goto done;
+   }
+
+   write_in_full(fd, "die", 4);
+   close(fd);
+
+done:
+   /*
+* The child will try to do this anyway, but we want to be
+* ready to launch a new daemon immediately after this command
+* returns.
+*/
+
+   unlink(pipe_path);
+   return;
+}
+
+
 int main(int argc, char **argv)
 {
const char *prefix;
-   int idle_in_seconds = 600, detach = 0;
+   int idle_in_seconds = 600, detach = 0, kill = 0;
int fd;
struct strbuf pipe_path = STRBUF_INIT;
struct option options[] = {
@@ -350,6 +377,7 @@ int main(int argc, char **argv)
OPT_BOOL(0, "strict", _verify,
 "verify shared memory after creating"),
OPT_BOOL(0, "detach", , "detach the process"),
+   OPT_BOOL(0, "kill", , "request that existing index helper 
processes exit"),
OPT_END()
};
 
@@ -364,10 +392,17 @@ int main(int argc, char **argv)
  options, usage_text, 0))
die(_("too many arguments"));
 
+   strbuf_git_path(_path, "index-helper.pipe");
+   if (kill) {
+   if (detach)
+   die(_("--kill doesn't want any other options"));
+   request_kill(pipe_path.buf);
+   return 0;
+   }
+
atexit(cleanup);
sigchain_push_common(cleanup_on_signal);
 
-   strbuf_git_path(_path, "index-helper.pipe");
fd = setup_pipe(pipe_path.buf);
if (fd < 0)
die_errno("Could not set up index-helper.pipe");
diff --git a/t/t7900-index-helper.sh b/t/t7900-index-helper.sh
index 7126037..14b5a6c 100755
--- a/t/t7900-index-helper.sh
+++ b/t/t7900-index-helper.sh
@@ -15,4 +15,13 @@ test_expect_success 'index-helper smoke test' '
test_path_is_missing .git/index-helper.pipe
 '
 
+test_expect_success 'index-helper creates usable pipe file and can be killed' '
+   test_when_finished "git index-helper --kill" &&
+   test_path_is_missing .git/index-helper.pipe &&
+   git index-helper --detach &&
+   test -p .git/index-helper.pipe &&
+   git index-helper --kill &&
+   test_path_is_missing .git/index-helper.pipe
+'
+
 test_done
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 10/17] index-helper: use watchman to avoid refreshing index with lstat()

[David Turner]

From: Nguyễn Thái Ngọc Duy 

Watchman is hidden behind index-helper. Before git tries to read the
index from shm, it notifies index-helper through the named pipe and
waits for index-helper to prepare shm. index-helper then contacts
watchman, updates 'WAMA' extension and put it in a separate shm and
wakes git up with a write to git's named pipe.

Git uses this extension to not lstat unchanged entries. Git only
trusts the 'WAMA' extension when it's received from the separate shm,
not from disk. Unmarked entries are "clean". Marked entries are dirty
from watchman point of view. If it finds out some entries are
'watchman-dirty', but are really unchanged (e.g. the file was changed,
then reverted back), then Git will clear the marking in 'WAMA' before
writing it down.

Hiding watchman behind index-helper means you need both daemons. You
can't run watchman alone. Not so good. But on the other hand, 'git'
binary is not linked to watchman/json libraries, which is good for
packaging. Core git package will run fine without watchman-related
packages. If they need watchman, they can install git-index-helper and
dependencies.

This also lets us trust anything in the untracked cache that we haven't
marked invalid, saving those stat() calls.

Another reason for tying watchman to index-helper is, when used with
untracked cache, we need to keep track of $GIT_WORK_TREE file
listing. That kind of list can be kept in index-helper.

Signed-off-by: Nguyễn Thái Ngọc Duy  gmail.com>
Signed-off-by: David Turner 
---
 Documentation/git-index-helper.txt |   6 ++
 cache.h|   2 +
 dir.c  |  12 
 index-helper.c | 128 ++---
 read-cache.c   | 141 +++--
 5 files changed, 275 insertions(+), 14 deletions(-)

diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
index b858a8d..7a8042e 100644
--- a/Documentation/git-index-helper.txt
+++ b/Documentation/git-index-helper.txt
@@ -51,6 +51,12 @@ the daemon:
Let the daemon know the index is to be read. It keeps the
daemon alive longer, unless `--exit-after=0` is used.
 
+"poke ":
+   Like "poke", but replies with "OK" by opening the named pipe
+   at  and writing the string.  If watchman is configured,
+   index-helper queries watchman, then prepares a shared memory
+   object with the watchman index extension before replying.
+
 All commands and replies are terminated by a 0 byte.
 
 GIT
diff --git a/cache.h b/cache.h
index 95715fd..9fa339a 100644
--- a/cache.h
+++ b/cache.h
@@ -558,6 +558,7 @@ extern int daemonize(int *);
 
 /* Initialize and use the cache information */
 struct lock_file;
+extern int verify_index(const struct index_state *);
 extern int read_index(struct index_state *);
 extern int read_index_preload(struct index_state *, const struct pathspec 
*pathspec);
 extern int do_read_index(struct index_state *istate, const char *path,
@@ -565,6 +566,7 @@ extern int do_read_index(struct index_state *istate, const 
char *path,
 extern int read_index_from(struct index_state *, const char *path);
 extern int is_index_unborn(struct index_state *);
 extern int read_index_unmerged(struct index_state *);
+extern void write_watchman_ext(struct strbuf *sb, struct index_state* istate);
 #define COMMIT_LOCK(1 << 0)
 #define CLOSE_LOCK (1 << 1)
 #define REFRESH_DAEMON (1 << 2)
diff --git a/dir.c b/dir.c
index 9b659e6..5058b29 100644
--- a/dir.c
+++ b/dir.c
@@ -1726,6 +1726,17 @@ static int valid_cached_dir(struct dir_struct *dir,
if (!untracked)
return 0;
 
+   if (dir->untracked->use_watchman) {
+   /*
+* With watchman, we can trust the untracked cache's
+* valid field.
+*/
+   if (untracked->valid)
+   goto skip_stat;
+   else
+   invalidate_directory(dir->untracked, untracked);
+   }
+
if (stat(path->len ? path->buf : ".", )) {
invalidate_directory(dir->untracked, untracked);
memset(>stat_data, 0, sizeof(untracked->stat_data));
@@ -1739,6 +1750,7 @@ static int valid_cached_dir(struct dir_struct *dir,
return 0;
}
 
+skip_stat:
if (untracked->check_only != !!check_only) {
invalidate_directory(dir->untracked, untracked);
return 0;
diff --git a/index-helper.c b/index-helper.c
index a854ed8..445a0ac 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -6,15 +6,18 @@
 #include "split-index.h"
 #include "shm.h"
 #include "lockfile.h"
+#include "watchman-support.h"
 
 struct shm {
unsigned char sha1[20];
void *shm;
size_t size;
+   pid_t pid;
 };
 
 static struct shm shm_index;
 static struct shm shm_base_index;
+static struct 

[PATCH v2 11/17] update-index: enable/disable watchman support

[David Turner]

From: Nguyễn Thái Ngọc Duy 

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 builtin/update-index.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/builtin/update-index.c b/builtin/update-index.c
index 1c94ca5..7c08e1c 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -914,6 +914,7 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
 {
int newfd, entries, has_errors = 0, nul_term_line = 0;
enum uc_mode untracked_cache = UC_UNSPECIFIED;
+   int use_watchman = -1;
int read_from_stdin = 0;
int prefix_length = prefix ? strlen(prefix) : 0;
int preferred_index_format = 0;
@@ -1012,6 +1013,8 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
N_("test if the filesystem supports untracked 
cache"), UC_TEST),
OPT_SET_INT(0, "force-untracked-cache", _cache,
N_("enable untracked cache without testing the 
filesystem"), UC_FORCE),
+   OPT_BOOL(0, "watchman", _watchman,
+   N_("use or not use watchman to reduce refresh cost")),
OPT_END()
};
 
@@ -1149,6 +1152,14 @@ int cmd_update_index(int argc, const char **argv, const 
char *prefix)
die("Bug: bad untracked_cache value: %d", untracked_cache);
}
 
+   if (use_watchman > 0) {
+   the_index.last_update= xstrdup("");
+   the_index.cache_changed |= WATCHMAN_CHANGED;
+   } else if (!use_watchman) {
+   the_index.last_update= NULL;
+   the_index.cache_changed |= WATCHMAN_CHANGED;
+   }
+
if (active_cache_changed) {
if (newfd < 0) {
if (refresh_args.flags & REFRESH_QUIET)
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] GPIO bulk changes for kernel v4.6

[Johannes Schindelin]

Hi Linus,

On Fri, 18 Mar 2016, Linus Torvalds wrote:

> I thought git didn't merge two branches that have no common base by
> default, but it seems it will happily do so.

What happened to "The coolest merge EVER!"?

http://thread.gmane.org/gmane.comp.version-control.git/5126/

Ciao,
Dscho
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: bug: sparse config interpretation incorrectness in 2.8.0-rc2

[Johannes Schindelin]

Hi Duy,

On Thu, 17 Mar 2016, Duy Nguyen wrote:

> On Thu, Mar 17, 2016 at 8:46 PM, Johannes Schindelin
>  wrote:
> > Unfortunately, this does not help me at all. In the use case I am trying
> > to get to work fast, we have tons and tons of directories and need *one*
> > file in pretty much *all* of those directories, and exclude most of the
> > other files.
> >
> > To make matters even worse, the list of excluded (or included) files is
> > constantly changing.
> 
> OK very weird use case to me :) There's something you might try. [1]
> can help trimming unrelated patterns, fewer patterns to match for a
> dir, faster matching.

Sadly, [1] (exclude: filter out patterns not applicable to the current
directory) would not help at all, because the regular use case really
would use the top-level directory as current one.

So I really need to speed up the sparse machinery from O(m*n) (where m is
the number of entries in the sparse-checkout file and n is the number of
files in the index).

Ciao,
Dscho
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 07/17] read-cache: add watchman 'WAMA' extension

[David Turner]

From: Nguyễn Thái Ngọc Duy 

The extension contains a bitmap, one bit for each entry in the
index. If the n-th bit is zero, the n-th entry is considered
unchanged, we can ce_mark_uptodate() it without refreshing. If the bit
is non-zero and we found out the corresponding file is clean after
refresh, we can clear the bit.

In addition, there's a list of directories in the untracked-cache
to invalidate (because they have new or modified entries).

The 'skipping refresh' bit is not in this patch yet as we would need
watchman. More details in later patches.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 cache.h  |   4 ++
 dir.h|   3 ++
 read-cache.c | 117 ++-
 3 files changed, 122 insertions(+), 2 deletions(-)

diff --git a/cache.h b/cache.h
index d386722..5b10d52 100644
--- a/cache.h
+++ b/cache.h
@@ -182,6 +182,8 @@ struct cache_entry {
 #define CE_VALID (0x8000)
 #define CE_STAGESHIFT 12
 
+#define CE_WATCHMAN_DIRTY  (0x0001)
+
 /*
  * Range 0x0FFF in ce_flags is divided into
  * two parts: in-memory flags and on-disk ones.
@@ -320,6 +322,7 @@ static inline unsigned int canon_mode(unsigned int mode)
 #define CACHE_TREE_CHANGED (1 << 5)
 #define SPLIT_INDEX_ORDERED(1 << 6)
 #define UNTRACKED_CHANGED  (1 << 7)
+#define WATCHMAN_CHANGED   (1 << 8)
 
 struct split_index;
 struct untracked_cache;
@@ -344,6 +347,7 @@ struct index_state {
struct untracked_cache *untracked;
void *mmap;
size_t mmap_size;
+   char *last_update;
 };
 
 extern struct index_state the_index;
diff --git a/dir.h b/dir.h
index 3ec3fb0..3d540de 100644
--- a/dir.h
+++ b/dir.h
@@ -142,6 +142,9 @@ struct untracked_cache {
int gitignore_invalidated;
int dir_invalidated;
int dir_opened;
+   /* watchman invalidation data */
+   unsigned int use_watchman : 1;
+   struct string_list invalid_untracked;
 };
 
 struct dir_struct {
diff --git a/read-cache.c b/read-cache.c
index fe73828..6d5e871 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -19,6 +19,7 @@
 #include "split-index.h"
 #include "utf8.h"
 #include "shm.h"
+#include "ewah/ewok.h"
 
 static struct cache_entry *refresh_cache_entry(struct cache_entry *ce,
   unsigned int options);
@@ -41,11 +42,13 @@ static struct cache_entry *refresh_cache_entry(struct 
cache_entry *ce,
 #define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* "REUC" */
 #define CACHE_EXT_LINK 0x6c696e6b/* "link" */
 #define CACHE_EXT_UNTRACKED 0x554E5452   /* "UNTR" */
+#define CACHE_EXT_WATCHMAN 0x57414D41/* "WAMA" */
 
 /* changes that can be kept in $GIT_DIR/index (basically all extensions) */
 #define EXTMASK (RESOLVE_UNDO_CHANGED | CACHE_TREE_CHANGED | \
 CE_ENTRY_ADDED | CE_ENTRY_REMOVED | CE_ENTRY_CHANGED | \
-SPLIT_INDEX_ORDERED | UNTRACKED_CHANGED)
+SPLIT_INDEX_ORDERED | UNTRACKED_CHANGED | \
+WATCHMAN_CHANGED)
 
 struct index_state the_index;
 static const char *alternate_index_output;
@@ -1220,8 +1223,13 @@ int refresh_index(struct index_state *istate, unsigned 
int flags,
continue;
 
new = refresh_cache_ent(istate, ce, options, _errno, 
);
-   if (new == ce)
+   if (new == ce) {
+   if (ce->ce_flags & CE_WATCHMAN_DIRTY) {
+   ce->ce_flags  &= ~CE_WATCHMAN_DIRTY;
+   istate->cache_changed |= WATCHMAN_CHANGED;
+   }
continue;
+   }
if (!new) {
const char *fmt;
 
@@ -1365,6 +1373,94 @@ static int verify_hdr(const struct cache_header *hdr, 
unsigned long size)
return 0;
 }
 
+static void mark_no_watchman(size_t pos, void *data)
+{
+   struct index_state *istate = data;
+   assert(pos < istate->cache_nr);
+   istate->cache[pos]->ce_flags |= CE_WATCHMAN_DIRTY;
+}
+
+static int read_watchman_ext(struct index_state *istate, const void *data,
+unsigned long sz)
+{
+   struct ewah_bitmap *bitmap;
+   int ret, len;
+   uint32_t bitmap_size;
+   uint32_t untracked_nr;
+
+   if (memchr(data, 0, sz) == NULL)
+   return error("invalid extension");
+
+   len = strlen(data) + 1;
+   memcpy(_size, (const char *)data + len, 4);
+   memcpy(_nr, (const char *)data + len + 4, 4);
+   untracked_nr = ntohl(untracked_nr);
+   bitmap_size = ntohl(bitmap_size);
+
+   bitmap = ewah_new();
+   ret = ewah_read_mmap(bitmap, (const char *)data + len + 8, bitmap_size);
+   if (ret != bitmap_size) {
+   ewah_free(bitmap);
+   return error("failed to parse ewah bitmap reading watchman 
index extension");
+   }
+   istate->last_update = xstrdup(data);
+   

[PATCH v2 02/17] read-cache: allow to keep mmap'd memory after reading

[David Turner]

From: Nguyễn Thái Ngọc Duy 

Later, we will introduce git index-helper to share this memory with
other git processes.

Since the memory will be shared, it will never be unmapped (although
the kernel may of course choose to page it out).

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 cache.h  |  3 +++
 read-cache.c | 13 -
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/cache.h b/cache.h
index b829410..4180e2b 100644
--- a/cache.h
+++ b/cache.h
@@ -333,11 +333,14 @@ struct index_state {
struct split_index *split_index;
struct cache_time timestamp;
unsigned name_hash_initialized : 1,
+keep_mmap : 1,
 initialized : 1;
struct hashmap name_hash;
struct hashmap dir_hash;
unsigned char sha1[20];
struct untracked_cache *untracked;
+   void *mmap;
+   size_t mmap_size;
 };
 
 extern struct index_state the_index;
diff --git a/read-cache.c b/read-cache.c
index 16cc487..7e387e9 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1574,6 +1574,10 @@ int do_read_index(struct index_state *istate, const char 
*path, int must_exist)
mmap = xmmap(NULL, mmap_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (mmap == MAP_FAILED)
die_errno("unable to map index file");
+   if (istate->keep_mmap) {
+   istate->mmap = mmap;
+   istate->mmap_size = mmap_size;
+   }
close(fd);
 
hdr = mmap;
@@ -1626,10 +1630,12 @@ int do_read_index(struct index_state *istate, const 
char *path, int must_exist)
src_offset += 8;
src_offset += extsize;
}
-   munmap(mmap, mmap_size);
+   if (!istate->keep_mmap)
+   munmap(mmap, mmap_size);
return istate->cache_nr;
 
 unmap:
+   istate->mmap = NULL;
munmap(mmap, mmap_size);
die("index file corrupt");
 }
@@ -1655,6 +1661,7 @@ int read_index_from(struct index_state *istate, const 
char *path)
discard_index(split_index->base);
else
split_index->base = xcalloc(1, sizeof(*split_index->base));
+   split_index->base->keep_mmap = istate->keep_mmap;
ret = do_read_index(split_index->base,
git_path("sharedindex.%s",
 sha1_to_hex(split_index->base_sha1)), 1);
@@ -1698,6 +1705,10 @@ int discard_index(struct index_state *istate)
free(istate->cache);
istate->cache = NULL;
istate->cache_alloc = 0;
+   if (istate->keep_mmap && istate->mmap) {
+   munmap(istate->mmap, istate->mmap_size);
+   istate->mmap = NULL;
+   }
discard_split_index(istate);
free_untracked_cache(istate->untracked);
istate->untracked = NULL;
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 04/17] index-helper: add --strict

[David Turner]

From: Nguyễn Thái Ngọc Duy 

There are "holes" in the index-helper approach because the shared
memory is not verified again by git. If $USER is compromised, shared
memory could be modified. But anyone who could do this could already
modify $GIT_DIR/index. A more realistic risk is some bugs in
index-helper that produce corrupt shared memory. --strict is added to
avoid that.

Strictly speaking there's still a very small gap where corrupt shared
memory could still be read by git: after we write the trailing SHA-1 in
the shared memory (thus signaling "this shm is ready") and before
verify_shm() detects an error.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Documentation/git-index-helper.txt |  9 +++
 cache.h|  1 +
 index-helper.c | 48 ++
 read-cache.c   |  9 ---
 4 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-index-helper.txt 
b/Documentation/git-index-helper.txt
index 59a5abc..7afc5c9 100644
--- a/Documentation/git-index-helper.txt
+++ b/Documentation/git-index-helper.txt
@@ -22,6 +22,15 @@ OPTIONS
Exit if the cached index is not accessed for ``
minutes. Specify 0 to wait forever. Default is 10.
 
+--strict::
+--no-strict::
+   Strict mode makes index-helper verify the shared memory after
+   it's created. If the result does not match what's read from
+   $GIT_DIR/index, the shared memory is destroyed. This makes
+   index-helper take more than double the amount of time required
+   for reading an index, but because it will happen in the
+   background, it's not noticable. `--strict` is enabled by default.
+
 NOTES
 -
 $GIT_DIR/index-helper.pipe is a named pipe that the daemon reads
diff --git a/cache.h b/cache.h
index c3c6d85..5805962 100644
--- a/cache.h
+++ b/cache.h
@@ -336,6 +336,7 @@ struct index_state {
 keep_mmap : 1,
 from_shm : 1,
 to_shm : 1,
+always_verify_trailing_sha1 : 1,
 initialized : 1;
struct hashmap name_hash;
struct hashmap dir_hash;
diff --git a/index-helper.c b/index-helper.c
index 962c973..8d221e0 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -15,6 +15,7 @@ struct shm {
 
 static struct shm shm_index;
 static struct shm shm_base_index;
+static int to_verify = 1;
 
 static void release_index_shm(struct shm *is)
 {
@@ -73,11 +74,56 @@ static void share_index(struct index_state *istate, struct 
shm *is)
hashcpy((unsigned char *)new_mmap + istate->mmap_size - 20, is->sha1);
 }
 
+static int verify_shm(void)
+{
+   int i;
+   struct index_state istate;
+   memset(, 0, sizeof(istate));
+   istate.always_verify_trailing_sha1 = 1;
+   istate.to_shm = 1;
+   i = read_index();
+   if (i != the_index.cache_nr)
+   goto done;
+   for (; i < the_index.cache_nr; i++) {
+   struct cache_entry *base, *ce;
+   /* namelen is checked separately */
+   const unsigned int ondisk_flags =
+   CE_STAGEMASK | CE_VALID | CE_EXTENDED_FLAGS;
+   unsigned int ce_flags, base_flags, ret;
+   base = the_index.cache[i];
+   ce = istate.cache[i];
+   if (ce->ce_namelen != base->ce_namelen ||
+   strcmp(ce->name, base->name)) {
+   warning("mismatch at entry %d", i);
+   break;
+   }
+   ce_flags = ce->ce_flags;
+   base_flags = base->ce_flags;
+   /* only on-disk flags matter */
+   ce->ce_flags   &= ondisk_flags;
+   base->ce_flags &= ondisk_flags;
+   ret = memcmp(>ce_stat_data, >ce_stat_data,
+offsetof(struct cache_entry, name) -
+offsetof(struct cache_entry, ce_stat_data));
+   ce->ce_flags = ce_flags;
+   base->ce_flags = base_flags;
+   if (ret) {
+   warning("mismatch at entry %d", i);
+   break;
+   }
+   }
+done:
+   discard_index();
+   return i == the_index.cache_nr;
+}
+
 static void share_the_index(void)
 {
if (the_index.split_index && the_index.split_index->base)
share_index(the_index.split_index->base, _base_index);
share_index(_index, _index);
+   if (to_verify && !verify_shm())
+   cleanup_shm();
discard_index(_index);
 }
 
@@ -189,6 +235,8 @@ int main(int argc, char **argv)
struct option options[] = {
OPT_INTEGER(0, "exit-after", _in_seconds,
N_("exit if not used after some seconds")),
+   OPT_BOOL(0, "strict", _verify,
+"verify shared memory after creating"),
OPT_END()

[PATCH v2 08/17] read-cache: invalidate untracked cache data when reading WAMA

[David Turner]

When reading the watchman extension, invalidate the listed
untracked-cache entries.

We don't clear these entries yet; we can only do that once we know
that they came from a live watchman (rather than from disk).

Signed-off-by: David Turner 
---
 dir.c| 11 +---
 dir.h|  3 ++
 read-cache.c | 89 
 3 files changed, 94 insertions(+), 9 deletions(-)

diff --git a/dir.c b/dir.c
index 69e0be6..9b659e6 100644
--- a/dir.c
+++ b/dir.c
@@ -597,9 +597,9 @@ static void trim_trailing_spaces(char *buf)
  *
  * If "name" has the trailing slash, it'll be excluded in the search.
  */
-static struct untracked_cache_dir *lookup_untracked(struct untracked_cache *uc,
-   struct untracked_cache_dir 
*dir,
-   const char *name, int len)
+struct untracked_cache_dir *lookup_untracked(struct untracked_cache *uc,
+struct untracked_cache_dir *dir,
+const char *name, int len)
 {
int first, last;
struct untracked_cache_dir *d;
@@ -2625,8 +2625,10 @@ static void free_untracked(struct untracked_cache_dir 
*ucd)
 
 void free_untracked_cache(struct untracked_cache *uc)
 {
-   if (uc)
+   if (uc) {
free_untracked(uc->root);
+   string_list_clear(>invalid_untracked, 0);
+   }
free(uc);
 }
 
@@ -2775,6 +2777,7 @@ struct untracked_cache *read_untracked_extension(const 
void *data, unsigned long
return NULL;
 
uc = xcalloc(1, sizeof(*uc));
+   string_list_init(>invalid_untracked, 1);
strbuf_init(>ident, ident_len);
strbuf_add(>ident, ident, ident_len);
load_sha1_stat(>ss_info_exclude, >info_exclude_stat,
diff --git a/dir.h b/dir.h
index 3d540de..8fd3f9e 100644
--- a/dir.h
+++ b/dir.h
@@ -315,4 +315,7 @@ struct untracked_cache *read_untracked_extension(const void 
*data, unsigned long
 void write_untracked_extension(struct strbuf *out, struct untracked_cache 
*untracked);
 void add_untracked_cache(struct index_state *istate);
 void remove_untracked_cache(struct index_state *istate);
+struct untracked_cache_dir *lookup_untracked(struct untracked_cache *uc,
+struct untracked_cache_dir *dir,
+const char *name, int len);
 #endif
diff --git a/read-cache.c b/read-cache.c
index 6d5e871..b4bd15c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1373,11 +1373,76 @@ static int verify_hdr(const struct cache_header *hdr, 
unsigned long size)
return 0;
 }
 
+static struct untracked_cache_dir *find_untracked_cache_dir(
+   struct untracked_cache *uc, struct untracked_cache_dir *ucd,
+   const char *name)
+{
+   int component_len;
+   const char *end;
+   struct untracked_cache_dir *dir;
+
+   if (!*name)
+   return ucd;
+
+   end = strchr(name, '/');
+   if (end)
+   component_len = end - name;
+   else
+   component_len = strlen(name);
+
+   dir = lookup_untracked(uc, ucd, name, component_len);
+   if (dir)
+   return find_untracked_cache_dir(uc, dir, name + component_len + 
1);
+
+   return NULL;
+}
+
 static void mark_no_watchman(size_t pos, void *data)
 {
struct index_state *istate = data;
+   struct cache_entry *ce = istate->cache[pos];
+   struct strbuf sb = STRBUF_INIT;
+   char *c;
+   struct untracked_cache_dir *dir;
+
assert(pos < istate->cache_nr);
-   istate->cache[pos]->ce_flags |= CE_WATCHMAN_DIRTY;
+   ce->ce_flags |= CE_WATCHMAN_DIRTY;
+
+   if (!istate->untracked || !istate->untracked->root)
+   return;
+
+   strbuf_add(, ce->name, ce_namelen(ce));
+
+   for (c = sb.buf + sb.len - 1; c > sb.buf; c--) {
+   if (*c == '/') {
+   strbuf_setlen(, c - sb.buf);
+   break;
+   }
+   }
+
+   if (c == sb.buf) {
+   strbuf_setlen(, 0);
+   }
+
+   dir = find_untracked_cache_dir(istate->untracked,
+  istate->untracked->root, sb.buf);
+   if (dir)
+   dir->valid = 0;
+
+   strbuf_release();
+}
+
+static int mark_untracked_invalid(struct string_list_item *item, void *uc)
+{
+   struct untracked_cache *untracked = uc;
+   struct untracked_cache_dir *dir;
+
+   dir = find_untracked_cache_dir(untracked, untracked->root,
+  item->string);
+   if (dir)
+   dir->valid = 0;
+
+   return 0;
 }
 
 static int read_watchman_ext(struct index_state *istate, const void *data,
@@ -1407,10 +1472,24 @@ static int read_watchman_ext(struct index_state 
*istate, const void *data,
ewah_each_bit(bitmap, 

[PATCH v2 09/17] Add watchman support to reduce index refresh cost

[David Turner]

From: Nguyễn Thái Ngọc Duy 

The previous patch has the logic to clear bits in 'WAMA' bitmap. This
patch has logic to set bits as told by watchman. The missing bit,
_using_ these bits, are not here yet.

A lot of this code is written by David Turner originally, mostly from
[1]. I'm just copying and polishing it a bit.

[1] http://article.gmane.org/gmane.comp.version-control.git/248006

Signed-off-by: David Turner 
Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 Makefile   |  12 +
 cache.h|   1 +
 config.c   |   5 ++
 configure.ac   |   8 
 environment.c  |   3 ++
 watchman-support.c | 134 +
 watchman-support.h |   7 +++
 7 files changed, 170 insertions(+)
 create mode 100644 watchman-support.c
 create mode 100644 watchman-support.h

diff --git a/Makefile b/Makefile
index 2d72771..8bf705b 100644
--- a/Makefile
+++ b/Makefile
@@ -453,6 +453,7 @@ MSGFMT = msgfmt
 CURL_CONFIG = curl-config
 PTHREAD_LIBS = -lpthread
 PTHREAD_CFLAGS =
+WATCHMAN_LIBS =
 GCOV = gcov
 
 export TCL_PATH TCLTK_PATH
@@ -1419,6 +1420,13 @@ else
LIB_OBJS += thread-utils.o
 endif
 
+ifdef USE_WATCHMAN
+   LIB_H += watchman-support.h
+   LIB_OBJS += watchman-support.o
+   WATCHMAN_LIBS = -lwatchman
+   BASIC_CFLAGS += -DUSE_WATCHMAN
+endif
+
 ifdef HAVE_PATHS_H
BASIC_CFLAGS += -DHAVE_PATHS_H
 endif
@@ -2030,6 +2038,9 @@ git-remote-testsvn$X: remote-testsvn.o GIT-LDFLAGS 
$(GITLIBS) $(VCSSVN_LIB)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) 
$(LIBS) \
$(VCSSVN_LIB)
 
+git-index-helper$X: index-helper.o GIT-LDFLAGS $(GITLIBS)
+   $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) 
$(LIBS) $(WATCHMAN_LIBS)
+
 $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)
$(QUIET_LNCP)$(RM) $@ && \
ln $< $@ 2>/dev/null || \
@@ -2168,6 +2179,7 @@ GIT-BUILD-OPTIONS: FORCE
@echo NO_PERL=\''$(subst ','\'',$(subst ','\'',$(NO_PERL)))'\' >>$@+
@echo NO_PYTHON=\''$(subst ','\'',$(subst ','\'',$(NO_PYTHON)))'\' >>$@+
@echo NO_UNIX_SOCKETS=\''$(subst ','\'',$(subst 
','\'',$(NO_UNIX_SOCKETS)))'\' >>$@+
+   @echo USE_WATCHMAN=\''$(subst ','\'',$(subst 
','\'',$(USE_WATCHMAN)))'\' >>$@+
 ifdef TEST_OUTPUT_DIRECTORY
@echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst 
','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+
 endif
diff --git a/cache.h b/cache.h
index 5b10d52..95715fd 100644
--- a/cache.h
+++ b/cache.h
@@ -688,6 +688,7 @@ extern char *git_replace_ref_base;
 
 extern int fsync_object_files;
 extern int core_preload_index;
+extern int core_watchman_sync_timeout;
 extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
 extern int protect_hfs;
diff --git a/config.c b/config.c
index 9ba40bc..e6dc141 100644
--- a/config.c
+++ b/config.c
@@ -882,6 +882,11 @@ static int git_default_core_config(const char *var, const 
char *value)
return 0;
}
 
+   if (!strcmp(var, "core.watchmansynctimeout")) {
+   core_watchman_sync_timeout = git_config_int(var, value);
+   return 0;
+   }
+
if (!strcmp(var, "core.createobject")) {
if (!strcmp(value, "rename"))
object_creation_mode = OBJECT_CREATION_USES_RENAMES;
diff --git a/configure.ac b/configure.ac
index 0cd9f46..334d63b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1099,6 +1099,14 @@ AC_COMPILE_IFELSE([BSD_SYSCTL_SRC],
HAVE_BSD_SYSCTL=])
 GIT_CONF_SUBST([HAVE_BSD_SYSCTL])
 
+#
+# Check for watchman client library
+
+AC_CHECK_LIB([watchman], [watchman_connect],
+   [USE_WATCHMAN=YesPlease],
+   [USE_WATCHMAN=])
+GIT_CONF_SUBST([USE_WATCHMAN])
+
 ## Other checks.
 # Define USE_PIC if you need the main git objects to be built with -fPIC
 # in order to build and link perl/Git.so.  x86-64 seems to need this.
diff --git a/environment.c b/environment.c
index 6dec9d0..35e03c7 100644
--- a/environment.c
+++ b/environment.c
@@ -94,6 +94,9 @@ int core_preload_index = 1;
  */
 int ignore_untracked_cache_config;
 
+int core_watchman_sync_timeout = 300;
+
+
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
 static char *work_tree;
diff --git a/watchman-support.c b/watchman-support.c
new file mode 100644
index 000..b7302b9
--- /dev/null
+++ b/watchman-support.c
@@ -0,0 +1,134 @@
+#include "cache.h"
+#include "watchman-support.h"
+#include "strbuf.h"
+#include "dir.h"
+#include 
+
+static struct watchman_query *make_query(const char *last_update)
+{
+   struct watchman_query *query = watchman_query();
+   watchman_query_set_fields(query, WATCHMAN_FIELD_NAME |
+WATCHMAN_FIELD_EXISTS |
+WATCHMAN_FIELD_NEWER);
+   watchman_query_set_empty_on_fresh(query, 1);
+   

[PATCH v2 01/17] read-cache.c: fix constness of verify_hdr()

[David Turner]

From: Nguyễn Thái Ngọc Duy 

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 read-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/read-cache.c b/read-cache.c
index d9fb78b..16cc487 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1345,7 +1345,7 @@ struct ondisk_cache_entry_extended {
ondisk_cache_entry_extended_size(ce_namelen(ce)) : \
ondisk_cache_entry_size(ce_namelen(ce)))
 
-static int verify_hdr(struct cache_header *hdr, unsigned long size)
+static int verify_hdr(const struct cache_header *hdr, unsigned long size)
 {
git_SHA_CTX c;
unsigned char sha1[20];
-- 
2.4.2.767.g62658d5-twtrsrc

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] merge: refuse to create too cool a merge by default

[Junio C Hamano]

David Turner  writes:

> Also, I didn't notice a test that shows that "cool" merges without
> allow-unrelated-histories are forbidden.

Yeah, because I didn't write one in the version that was sent out,
which has been corrected in the one that will be on 'pu'.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html