Re: [PATCH] git-gui--askpass: generalize the window title

[Sebastian Schuberth]

On 01.02.2016 13:11, Sebastian Schuberth wrote:


git-gui--askpass is not only used for SSH authentication, but also for
HTTPS. In that context it is confusing to have a window title of
"OpenSSH". So generalize the title so that it also says which parent
process, i.e. Git, requires authentication.

Signed-off-by: Sebastian Schuberth 


I haven't seen this being picked up so far. Any comments?

--
Sebastian Schuberth
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: GSoC 2016: applications open, deadline = Fri, 19/2

[Lars Schneider]

On 12 Feb 2016, at 08:10, Matthieu Moy  wrote:

> Christian Couder  writes:
> 
>> Hi,
>> 
>> On Wed, Feb 10, 2016 at 10:31 AM, Matthieu Moy
>>  wrote:
>>> 
>>> So, the first question is: are there volunteers to be GSoC mentors this
>>> year?
>> 
>> I can co-mentor this year too, with you or someone else.
>> With you I think it will work out even if you have less time than last year.
> 
> So, that makes it 4 possible co-mentors, i.e. 2 potential slots. Not
> much, but it starts looking like last year ... ;-).
> 
> Peff, would you be willing to co-admin with me (that would be cool, you
> are the one with most experience here and you know the SFC stuff for
> payment)? Are there any other co-admin volunteer?

I don't know what level of Git development knowledge and what amount of time
is necessary but I would be available as junior co-mentor :-)

Cheers,
Lars--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: GSoC 2016: applications open, deadline = Fri, 19/2

[Matthieu Moy]

Lars Schneider  writes:

> I don't know what level of Git development knowledge and what amount of time
> is necessary but I would be available as junior co-mentor :-)

AFAICT, you don't have much experience with Git's codebase itself (if I
don't count git-p4 as "Git itself"), but you've already been involved in
typical reviewing cycles (just the discussions on Travis-CI were a good
example), and that is something at least as important as knowing the
codebase well. It's up to you to decide whether you feel experienced
enough, but I think you are welcome as a co-mentor!

As a mentor, to me, the most important things are:

* Give advice on how to interact with the Git community. Students can be
  shy, and then repeating "you should post more to the mailing-list" can
  be useful. They sometimes make mistakes, and explaining off-list
  "there's nothing wrong with what you did, but the custom here is
  to ..." can help.

* Give advice on how to get useful code merged. My usual advice is:
  "don't be too ambitious", which translates to "git this part done,
  reviewed and possibly merged, you'll work on the bells and whistles
  later".

* Avoid overloading the list with reviews. Getting your own GSoC
  tee-shirt and letting the list do the work is unfair ;-). Off-list
  reviews are good to eliminate straightforwards issues, and then
  mentors should actively participate to the on-list review. That is
  probably what takes most time.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Sebastian Schuberth]

On 04.02.2016 11:34, Sebastian Schuberth wrote:


This avoids output like

 warning: ignoring broken ref refs/remotes/origin/HEAD

while completing branch names.

Signed-off-by: Sebastian Schuberth 


The discussion got a bit off the point with the "short" vs. "strip=2" 
stuff, but I guess the patch itself if good to apply?


--
Sebastian Schuberth

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Implement https public key pinning

[Thomas Gummerer]

On 02/12, Christoph Egger wrote:
> Daniel Stenberg  writes:
> > On Thu, 11 Feb 2016, Christoph Egger wrote:
> >> +#if LIBCURL_VERSION_NUM >= 0x074400
> >
> > That should probably be 0x072c00 ...
>
> This is, of course, right.
>
> I used 7.44 / 0x072c00 as base because it has robust support for this
> feature (including the sha256// variant). One could lower that depending
> on the compromises one is willing to take FWIW
>
>   Added in 7.39.0 for OpenSSL, GnuTLS and GSKit. Added in 7.43.0 for NSS
>   and wolfSSL/CyaSSL. Added for mbedtls in 7.47.0, sha256 support added
>   in 7.44.0 for OpenSSL, GnuTLS, NSS and wolfSSL/CyaSSL. Other SSL
>   backends not supported.
>
> Also some people suggested that git should fail if this option is
> requested in the config but not supported by the libcurl version instead
> of falling back to just not pin the key. I'm undecided about that.

This seems to have been suggested off list (or at least I can't find
the message).  FWIW I do agree with failing or as a bare minimum
warning the user if the config option is set, but not supported by the
libcurl version.  Otherwise we risk giving the user a false sense of
security when the option is set, which is arguably worse than not
having the security option at all.

>   Christoph
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 2/1] support -4 and -6 switches for remote operations

[Eric Wong]

Eric Wong  wrote:
> (no rush to review this, unlikely to be around the next few days)

Ping on v2.  I will be online the next few days and likely
working on some other git stuff anyways :)
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: GSoC 2016: applications open, deadline = Fri, 19/2

[Jeff King]

On Fri, Feb 12, 2016 at 08:10:34AM +0100, Matthieu Moy wrote:

> So, that makes it 4 possible co-mentors, i.e. 2 potential slots. Not
> much, but it starts looking like last year ... ;-).
> 
> Peff, would you be willing to co-admin with me (that would be cool, you
> are the one with most experience here and you know the SFC stuff for
> payment)? Are there any other co-admin volunteer?

Yes, I'm willing to co-admin (though I'm also happy to step aside for
somebody else if they would like to do it).

The biggest task there is getting the application together. I went
through the account creation steps at the site (which is different this
year), and the application questions are:

 - Why does your org want to participate in Google Summer of Code?

 - How many potential mentors have agreed to mentor this year?

 - How will you keep mentors engaged with their students?

 - How will you help your students stay on schedule to complete their projects?

 - How will you get your students involved in your community during GSoC?

 - How will you keep students involved with your community after GSoC?

 - Has your org been accepted as a mentoring org in Google Summer of Code 
before?

 - Are you part of a foundation/umbrella organization?

 - What year was your project started? 

I think we can pull most of these answers from previous-year
applications, but I haven't looked yet. In years past we collaborated on
the answers via the git.github.io site, and I pasted them in place.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: GSoC 2016: applications open, deadline = Fri, 19/2

[Jeff King]

On Fri, Feb 12, 2016 at 08:04:46AM -0500, Jeff King wrote:

> The biggest task there is getting the application together. I went
> through the account creation steps at the site (which is different this
> year), and the application questions are:
> [...]

We also need to fill out our profile. Those items are listed below.

Matthieu, I listed you as a co-mentor, so I think it will have sent you
an email to join the application I started (hopefully you didn't do the
exact same thing already... :) ).

 - Website URL

 - Tagline

 - Logo

 - Primary Open Source License

 - Organization Category

 - Technology Tags

 - Topic Tags

 - Ideas List

 - Short Description

 - Long Description

 - Application Instructions

 - Proposal Tags

 - IRC Channel, Mailing List, or Email 

The big thing there is the ideas list.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 14/21] refs: always handle non-normal refs in files backend

[Michael Haggerty]

On 02/05/2016 08:44 PM, David Turner wrote:
> Always handle non-normal (per-worktree or pseudo) refs in the files
> backend instead of alternate backends.
> 
> Sometimes a ref transaction will update both a per-worktree ref and a
> normal ref.  For instance, an ordinary commit might update
> refs/heads/master and HEAD (or at least HEAD's reflog).
> 
> Updates to normal refs continue to go through the chosen backend.
> 
> Updates to non-normal refs are moved to a separate files backend
> transaction.
> 
> Signed-off-by: David Turner 
> ---
>  refs.c | 81 
> --
>  1 file changed, 79 insertions(+), 2 deletions(-)
> 
> diff --git a/refs.c b/refs.c
> index 227c018..18ba356 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -9,6 +9,11 @@
>  #include "object.h"
>  #include "tag.h"
>  
> +static const char split_transaction_fail_warning[] = N_(
> + "A ref transaction was split across two refs backends.  Part of the "
> + "transaction succeeded, but then the update to the per-worktree refs "
> + "failed.  Your repository may be in an inconsistent state.");
> +
>  /*
>   * We always have a files backend and it is the default.
>   */
> @@ -791,6 +796,13 @@ void ref_transaction_free(struct ref_transaction 
> *transaction)
>   free(transaction);
>  }
>  
> +static void add_update_obj(struct ref_transaction *transaction,
> +struct ref_update *update)
> +{
> + ALLOC_GROW(transaction->updates, transaction->nr + 1, 
> transaction->alloc);
> + transaction->updates[transaction->nr++] = update;
> +}
> +
>  static struct ref_update *add_update(struct ref_transaction *transaction,
>const char *refname)
>  {
> @@ -798,8 +810,7 @@ static struct ref_update *add_update(struct 
> ref_transaction *transaction,
>   struct ref_update *update = xcalloc(1, sizeof(*update) + len);
>  
>   memcpy((char *)update->refname, refname, len); /* includes NUL */
> - ALLOC_GROW(transaction->updates, transaction->nr + 1, 
> transaction->alloc);
> - transaction->updates[transaction->nr++] = update;
> + add_update_obj(transaction, update);
>   return update;
>  }
>  
> @@ -1217,11 +1228,38 @@ static int dereference_symrefs(struct ref_transaction 
> *transaction,
>   return 0;
>  }
>  
> +/*
> + * Move all non-normal ref updates into a specially-created
> + * files-backend transaction
> + */
> +static int move_abnormal_ref_updates(struct ref_transaction *transaction,
> +  struct ref_transaction *files_transaction,
> +  struct strbuf *err)
> +{
> + int i;
> +
> + for (i = 0; i < transaction->nr; i++) {
> + int last;
> + struct ref_update *update = transaction->updates[i];
> +
> + if (ref_type(update->refname) == REF_TYPE_NORMAL)
> + continue;
> +
> + last = --transaction->nr;
> + transaction->updates[i] = transaction->updates[last];
> + add_update_obj(files_transaction, update);
> + }
> +
> + return 0;
> +}
> +

I think this function is incorrect. The update that was previously at
transaction->updates[i] never gets checked for abnormality. You could
fix that and also avoid gratuitously changing the order of the updates
like this:

static int move_abnormal_ref_updates(struct ref_transaction *transaction,
 struct ref_transaction *files_transaction,
 struct strbuf *err)
{
int i, normal_nr = 0;

for (i = 0; i < transaction->nr; i++) {
struct ref_update *update = transaction->updates[i];

if (ref_type(update->refname) == REF_TYPE_NORMAL)
transaction->updates[normal_nr++] = update;
else
add_update_obj(files_transaction, update);
}

transaction->nr = normal_nr;
return 0;
}

Another alternative would be to set

update->flags |= REF_ABNORMAL

for the abnormal references and *leave them* in the original transaction
while also adding them to files_transactions. Then teach non-files
backends to skip over updates with REF_ABNORMAL.

The reason I thought of this is that long-term I'd like to make it
possible for some reference updates to fail without aborting the
transaction. To implement that, we would want a way for
ref_transaction_commit() to report back to its caller *which* updates
failed, and an obvious way to do that would be to store the
reference-specific errors in struct ref_update. If you leave the
abnormal ref_updates in the main transaction, then my hoped-for change
would be easier.

But that's a separate and hypothetical idea, so you don't have to let it
influence how you implement this patch.

>  int ref_transaction_commit(struct ref_transaction *transaction,
>  

[PATCH] Disown ssh+git and git+ssh

[Carlos Martín Nieto]

These were silly from the beginning, but we have to support them for
compatibility. That doesn't mean we have to show them in the
documentation. These were already left out of the main list, but a
reference in the main manpage was left, so remove that.

Also add a note to discourage their use if anybody goes looking for them
in the source code.
---
 Documentation/git.txt | 2 +-
 connect.c | 4 
 transport.c   | 4 
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index d987ad2..2f90635 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -1122,7 +1122,7 @@ of clones and fetches.
connection (or proxy, if configured)
 
  - `ssh`: git over ssh (including `host:path` syntax,
-   `git+ssh://`, etc).
+   `ssh://`, etc).
 
  - `rsync`: git over rsync
 
diff --git a/connect.c b/connect.c
index fd7ffe1..4f96424 100644
--- a/connect.c
+++ b/connect.c
@@ -267,6 +267,10 @@ static enum protocol get_protocol(const char *name)
return PROTO_SSH;
if (!strcmp(name, "git"))
return PROTO_GIT;
+   /*
+* These ssh schemes remain supported for compat but are
+* undocumented and their use is discouraged
+*/
if (!strcmp(name, "git+ssh"))
return PROTO_SSH;
if (!strcmp(name, "ssh+git"))
diff --git a/transport.c b/transport.c
index 9ae7184..f5ae707 100644
--- a/transport.c
+++ b/transport.c
@@ -1002,6 +1002,10 @@ struct transport *transport_get(struct remote *remote, 
const char *url)
|| starts_with(url, "file://")
|| starts_with(url, "git://")
|| starts_with(url, "ssh://")
+   /*
+* These ssh schemes remain supported for compat but are
+* undocumented and their use is discouraged
+*/
|| starts_with(url, "git+ssh://")
|| starts_with(url, "ssh+git://")) {
/*
-- 
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Post-receive hook for "git pull"

[Stefan Monnier]

>> I have a system here where it can be quite common to have thousands of
>> branches in the remote repository, and where I'd like to update some
>> local state according to the appearance of new branches (or updates of
>> pre-existing ones).
>> Currently, I use a "git for-each-ref" after pulling and then check (for
>> each one of those refs) if an update is warranted, but this can get slow
>> with that many branches.  Is there some way to get something like the
>> post-receive hook to be run for "git pull", so that the script gets told
>> directly which (remote tracking) branches have been modified/created?

> I do not think there is.  But you could easily script along the
> lines of...

> #!/bin/sh
> git for-each-ref | sort >prestate
> git pull "$@"
> git for-each-ref | sort >poststate
> comm -12 prestate poststate

Right, it kinda works, but it can break down in case of
concurrent operations.

I really wish there was a way to get something like the post-receive
hook to be called everytime new commits are added, regardless if it's
due to a push, a pull, a commit, a fast-import, ...


Stefan

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 13/21] refs: resolve symbolic refs first

[Michael Haggerty]

On 02/05/2016 08:44 PM, David Turner wrote:
> Before committing ref updates, split symbolic ref updates into two
> parts: an update to the underlying ref, and a log-only update to the
> symbolic ref.  This ensures that both references are locked correctly
> while their reflogs are updated.
> 
> It is still possible to confuse git by concurrent updates, since the
> splitting of symbolic refs does not happen under lock. So a symbolic ref
> could be replaced by a plain ref in the middle of this operation, which
> would lead to reflog discontinuities and missed old-ref checks.

This patch is doing too much at once for my little brain to follow.

My first hangup is the change to setting RESOLVE_REF_NO_RECURSE
unconditionally in lock_ref_sha1_basic(). I count five callers of that
function and see no justification for why the change is OK in the
context of each caller. Here are some thoughts:

* The call from files_create_symref() sets REF_NODEREF, so it is
unaffected by this change.

* The call from files_transaction_commit() is preceded by a call to
dereference_symrefs(), which I assume effectively replaces the need for
RESOLVE_REF_NO_RECURSE.

* There are two calls from files_rename_ref(). Why is it OK to do
without RESOLVE_REF_NO_RECURSE there?

  * For the oldrefname call, I suppose the justification is the "(flag &
REF_ISSYMREF)" check earlier in the function. (But does this introduce a
significant TOCTOU race?)

  * For the newrefname call, I suppose it's because the code a little
higher up tries to delete any existing reference with that name. It
looks to me like the old code was slightly broken: if newrefname was an
unborn symbolic reference, then: read_ref_full() would fail;
delete_ref() would be skipped; lock_ref_sha1_basic() would lock the
*referred-to* reference; the referred-to reference would be overwritten
instead of newrefname. So it could be that here REF_NODEREF indirectly
fixes a bug?

* The last call, from files_reflog_expire(), is also questionable before
your patch. If refname is a symref, then the function is expiring the
reflog of the symref. But (before this patch) it locks not the symref
but its referent. This was discussed in some length before on the
mailing list [1], and the conclusion was that the current behavior is
wrong, but for backwards compatibility reasons it would be safest to
change it to locking *both* the symref and its referent.

If possible, it would be better to split this patch up into several: the
first few would each add the REF_NODEREF flag at one callsite, with a
careful justification of why that is OK. Once all the callsites (except
the one in files_transaction_commit()) have been changed, then the last
patch could add the dereference_symrefs() machinery and change the last
callsite.

(I'm not certain that those steps are actually doable independently,
given that REF_NODEREF has other effects besides setting
RESOLVE_REF_NO_RECURSE.)

I'm not just being pedantic here. The patch as written is really too big
to review effectively.

Michael

[1]
http://thread.gmane.org/gmane.comp.version-control.git/263552/focus=263555

-- 
Michael Haggerty
mhag...@alum.mit.edu

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[ANNOUNCE] Git for Windows 2.7.1(2)

[Johannes Schindelin]

Dear Git users,

It is my pleasure to announce that Git for Windows 2.7.1(2) is available from:

https://git-for-windows.github.io/

Changes since Git for Windows v2.7.1 (February 6th 2016)

New Features

  ??? The context menu items in the explorer now show icons.

Bug Fixes

  ??? A bug was fixed where worktrees would forget their location e.g.
after an interactive rebase.
  ??? Thanks to Eric Lawrence and Martijn Laan, our installer sports a
better way to look for system files now.

Filename | SHA-256
 | ---
Git-2.7.1.2-64-bit.exe | 
956417448441e267a0ca601f47a2cd346e2d09589673060ae25d66628b2abc82
Git-2.7.1.2-32-bit.exe | 
be10e98b8c53f92648f8d1c85c7bad4c433aeedcb9d4719588b75030983cd76d
PortableGit-2.7.1.2-64-bit.7z.exe | 
92310dec60b0210acbfb31a0165f2628757b8413efcbcea1cfc95b2984a974dd
PortableGit-2.7.1.2-32-bit.7z.exe | 
7bf8da2b46f40c98d18a5f9c36e5b90df258936f43a0d230d0f29e0d729e3283
Git-2.7.1.2-64-bit.tar.bz2 | 
b3170fc127889f8fa603bc8546c61528869a0a6d2c19bc03de9d96dca2443881
Git-2.7.1.2-32-bit.tar.bz2 | 
4c897917682685194ed9d71fdfafbed721488fe22ecd8f11bfefffe3fec9169a

Ciao,
Johannes

Re: [PATCH v4 20/21] refs: add LMDB refs storage backend

[Michael Haggerty]

On 02/05/2016 08:44 PM, David Turner wrote:
> Add a database backend for refs using LMDB.  This backend runs git
> for-each-ref about 30% faster than the files backend with fully-packed
> refs on a repo with ~120k refs.  It's also about 4x faster than using
> fully-unpacked refs.  In addition, and perhaps more importantly, it
> avoids case-conflict issues on OS X.
> 
> LMDB has a few features that make it suitable for usage in git:
> 

0. It is licensed under the OpenLDAP Public License, which is apparently
GPL compatible [1].

[1] http://www.gnu.org/licenses/license-list.en.html#newOpenLDAP

> 1. It is relatively lightweight; it requires only one header file, and
> the library code takes under 64k at runtime.
> 
> 2. It is well-tested: it's been used in OpenLDAP for years.
> 
> 3. It's very fast.  LMDB's benchmarks show that it is among
> the fastest key-value stores.
> 
> 4. It has a relatively simple concurrency story; readers don't
> block writers and writers don't block readers.
> 
> Ronnie Sahlberg's original version of this patchset used tdb.  The
> major disadvantage of tdb is that tdb is hard to build on OS X.  It's
> also not in homebrew.  So lmdb seemed simpler.
> 
> To test this backend's correctness, I hacked test-lib.sh and
> test-lib-functions.sh to run all tests under the refs backend. Dozens
> of tests use manual ref/reflog reading/writing, or create submodules
> without passing --ref-storage to git init.  If those tests are
> changed to use the update-ref machinery or test-refs-lmdb-backend (or,
> in the case of packed-refs, corrupt refs, and dumb fetch tests, are
> skipped), the only remaining failing tests are the git-new-workdir
> tests and the gitweb tests.

Would it possible to build your "hack" into the test suite? For example,
perhaps one could implement an option that requests tests to use LMDB
wherever possible and skip the tests that are not LMDB-compatible. Over
time, we could try to increase the fraction of tests that are
LMDB-compatible by (for example) using `git update-ref` and `git
rev-parse` rather than reading/writing reference files via the
filesystem directly.

Otherwise, I'm worried that LMDB support will bitrot quickly because it
will be too much of a nuisance to exercise the code.

> Signed-off-by: David Turner 
> ---
>  .gitignore |1 +
>  Documentation/config.txt   |7 +
>  Documentation/git-clone.txt|3 +-
>  Documentation/git-init.txt |2 +-
>  Documentation/technical/refs-lmdb-backend.txt  |   52 +
>  Documentation/technical/repository-version.txt |5 +
>  Makefile   |   12 +
>  configure.ac   |   33 +
>  contrib/workdir/git-new-workdir|3 +
>  refs.c |3 +
>  refs.h |2 +
>  refs/lmdb-backend.c| 2052 
> 
>  setup.c|   11 +-
>  test-refs-lmdb-backend.c   |   64 +
>  transport.c|7 +-
>  15 files changed, 2250 insertions(+), 7 deletions(-)
>  create mode 100644 Documentation/technical/refs-lmdb-backend.txt
>  create mode 100644 refs/lmdb-backend.c
>  create mode 100644 test-refs-lmdb-backend.c
> 
> diff --git a/.gitignore b/.gitignore
> index 1c2f832..87d45a2 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -199,6 +199,7 @@
>  /test-path-utils
>  /test-prio-queue
>  /test-read-cache
> +/test-refs-lmdb-backend
>  /test-regex
>  /test-revision-walking
>  /test-run-command
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 06d3659..bc222c9 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1153,6 +1153,13 @@ difftool..cmd::
>  difftool.prompt::
>   Prompt before each invocation of the diff tool.
>  
> +extensions.refsStorage::
> + Type of refs storage backend. Default is to use the original
> + files based ref storage.  When set to "lmdb", refs are stored in
> + the lmdb database backend.  This setting reflects the refs
> + backend that is currently in use; it is not possible to change
> + the backend by changing this setting.
> +

Let's give the users a pointer to how they *can* change this setting. Maybe

Type of refs storage backend. Default is to use the original
files based ref storage.  When set to "lmdb", refs are stored in
an LMDB database.  This setting reflects the refs storage
backend that was chosen via the --ref-storage option when the
repository was originally created. It is currently
not possible to change the refs storage backend of an
existing repository.


>  fetch.recurseSubmodules::
>   This option can be either set to a boolean value or to 'on-demand'.
>

`.git` symlink makes `git submodule add` fail

[Vít Novotný]

Hi,

$ uname -a
Linux witiko 4.3.0-1-amd64 #1 SMP Debian 4.3.5-1 (2016-02-06) x86_64 
GNU/Linux
$ git --version
git version 2.7.0

I tend to make `.git` a symlink, when I need to maintain several Git
repositories inside a single directory. This was always transparent to Git, but
problems arose, when working with submodules:

$ mkdir -p .repos/repoA
$ ln -s .repos/repoA .git
$ git init
Initialized empty Git repository in /tmp/tmp.q3hkme6nm4/.repos/repoA/
$ git submodule add https://github.com/witiko/git-parallel
Cloning into 'git-parallel'...
remote: Counting objects: 212, done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 212 (delta 6), reused 1 (delta 1), pack-reused 195
Receiving objects: 100% (212/212), 64.20 KiB | 0 bytes/s, done.
Resolving deltas: 100% (120/120), done.
Checking connectivity... done.
fatal: Could not chdir to '../../../git-parallel': No such file or 
directory
Unable to checkout submodule 'git-parallel'

Is this a bug, or is the ability to symlink `.git` just a happy coincidence?

Best Regards,
Vít
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Git patch shows tab but files doesn't

[Jagan Teki]

Hi,

I edited a file as one space between #define and macro and tab space
between macro and BIT value, similar as below.

#define USE_FSR   BIT(6)
#define SNOR_WR BIT(7)

Once I created the patch looks different as tab space between #define
and macro and 2 tab spaces between macro and BIT value looks like it's
added tab spaces while creating a patch as below.

+#defineUSE_FSR  BIT(6)
+#defineSNOR_WRBIT(7)

Any help how to fix this, was it an issues with vim or do we have any
git-config fixes for this?

-- 
Jagan.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 15/21] init: allow alternate ref strorage to be set for new repos

[Michael Haggerty]

On 02/05/2016 08:44 PM, David Turner wrote:
> git init learns a new argument --ref-storage.  Presently, only
> "files" is supported, but later we will add other storage backends.
> 
> When this argument is used, the repository's extensions.refStorage
> configuration value is set (as well as core.repositoryformatversion),
> and the ref storage backend's initdb function is used to set up the
> ref database.
> 
> Signed-off-by: David Turner 
> Signed-off-by: SZEDER Gábor 
> ---
>  Documentation/git-init-db.txt  |  2 +-
>  Documentation/git-init.txt |  7 +-
>  builtin/init-db.c  | 44 
> +++---
>  cache.h|  2 ++
>  contrib/completion/git-completion.bash |  3 ++-
>  path.c | 29 --
>  refs.c |  8 +++
>  refs.h |  8 +++
>  setup.c| 12 ++
>  t/t0001-init.sh| 25 +++
>  10 files changed, 127 insertions(+), 13 deletions(-)
> 
> [...]
> diff --git a/Documentation/git-init.txt b/Documentation/git-init.txt
> index 8174d27..d2b150f 100644
> --- a/Documentation/git-init.txt
> +++ b/Documentation/git-init.txt
> @@ -12,7 +12,7 @@ SYNOPSIS
>  'git init' [-q | --quiet] [--bare] [--template=]
> [--separate-git-dir ]
> [--shared[=]] [directory]
> -
> +   [--ref-storage=]
>  
>  DESCRIPTION
>  ---
> @@ -113,6 +113,11 @@ does not exist, it will be created.
>  
>  --
>  
> +--ref-storage=::
> +Type of refs storage backend. Default is to use the original "files"
> +storage, which stores ref data in files in .git/refs and
> +.git/packed-refs.
> +

Technically, that should be $GIT_DIR/refs and $GIT_DIR/packed-refs. But
it might be that we are not so picky about the distinction in the user docs.

>  TEMPLATE DIRECTORY
>  --
>  
> diff --git a/builtin/init-db.c b/builtin/init-db.c
> index 801e977..d331ce8 100644
> --- a/builtin/init-db.c
> +++ b/builtin/init-db.c
> @@ -24,6 +24,7 @@ static int init_is_bare_repository = 0;
>  static int init_shared_repository = -1;
>  static const char *init_db_template_dir;
>  static const char *git_link;
> +static char *requested_ref_storage_backend;
>  
>  static void copy_templates_1(struct strbuf *path, struct strbuf *template,
>DIR *dir)
> @@ -179,6 +180,7 @@ static int create_default_files(const char *template_path)
>   int reinit;
>   int filemode;
>   struct strbuf err = STRBUF_INIT;
> + int repo_version = 0;
>  
>   /* Just look for `init.templatedir` */
>   git_config(git_init_db_config, NULL);
> @@ -205,6 +207,32 @@ static int create_default_files(const char 
> *template_path)
>   }
>  
>   /*
> +  * Create the default symlink from ".git/HEAD" to the "master"

I know that you are just moving this comment, but s/symlink/symref/
would make it more up-to-date.

> +  * branch, if it does not exist yet.
> +  */
> [...]
> diff --git a/setup.c b/setup.c
> index 0deb022..1a62277 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -1,5 +1,6 @@
>  #include "cache.h"
>  #include "dir.h"
> +#include "refs.h"
>  #include "string-list.h"
>  
>  static int inside_git_dir = -1;
> @@ -263,6 +264,15 @@ int get_common_dir_noenv(struct strbuf *sb, const char 
> *gitdir)
>   return ret;
>  }
>  
> +int ref_storage_backend_config(const char *var, const char *value, void *ptr)
> +{
> + char **cdata = ptr;
> +
> + if (!strcmp(var, "extensions.refstorage"))
> + *cdata = xstrdup(value);
> + return 0;
> +}
> +
>  /*
>   * Test if it looks like we're at a git directory.
>   * We want to see:
> @@ -390,6 +400,8 @@ static int check_repo_format(const char *var, const char 
> *value, void *cb)
>   ;
>   else if (!strcmp(ext, "preciousobjects"))
>   repository_format_precious_objects = 
> git_config_bool(var, value);
> + else if (!strcmp(ext, "refstorage"))
> + ref_storage_backend = xstrdup(value);
>   else
>   string_list_append(_extensions, ext);
>   }
> diff --git a/t/t0001-init.sh b/t/t0001-init.sh
> index 295aa59..31c8279 100755
> --- a/t/t0001-init.sh
> +++ b/t/t0001-init.sh
> @@ -174,6 +174,31 @@ test_expect_success 'reinit' '
>   test_i18ncmp again/empty again/err2
>  '
>  
> +test_expect_success 'init with bogus storage backend fails' '
> +
> + (
> + mkdir again2 &&
> + cd again2 &&
> + test_must_fail git init --ref-storage=test >out2 2>err2 &&
> + test_i18ngrep "Unknown ref storage backend test" err2
> + )
> +'
> +
> +test_expect_failure 'reinit with changed storage backend fails' '
> +
> + (
> + mkdir again3 &&
> + cd again3 &&
> 

Re: [PATCH v2 2/1] support -4 and -6 switches for remote operations

[Torsten Bögershausen]

On 12.02.16 12:31, Eric Wong wrote:
> Eric Wong  wrote:
>> (no rush to review this, unlikely to be around the next few days)
> Ping on v2.  I will be online the next few days and likely
> working on some other git stuff anyways :)
Patch V2 looks OK for me.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 19/21] refs: add register_ref_storage_backends()

[Michael Haggerty]

On 02/05/2016 08:44 PM, David Turner wrote:
> This new function will register all known ref storage backends... once
> there are any other than the default.  For now, it's a no-op.
> 
> Signed-off-by: David Turner 
> ---
>  builtin/init-db.c |  3 +++
>  config.c  | 25 +
>  refs.c|  8 
>  refs.h|  2 ++
>  4 files changed, 38 insertions(+)
> 
> [...]
> diff --git a/config.c b/config.c
> index b95ac3a..b9ef223 100644
> --- a/config.c
> +++ b/config.c
> @@ -11,6 +11,7 @@
>  #include "strbuf.h"
>  #include "quote.h"
>  #include "hashmap.h"
> +#include "refs.h"
>  #include "string-list.h"
>  #include "utf8.h"
>  
> @@ -1207,6 +1208,30 @@ int git_config_early(config_fn_t fn, void *data, const 
> char *repo_config)
>   }
>  
>   if (repo_config && !access_or_die(repo_config, R_OK, 0)) {
> + char *storage = NULL;
> +
> + /*
> +  * make sure we always read the ref storage config
> +  * from the extensions section on startup
> +  */
> + ret += git_config_from_file(ref_storage_backend_config,
> + repo_config, );
> +
> + register_ref_storage_backends();
> + if (!storage)
> + storage = xstrdup("");
> +
> + if ((!*storage) ||
> + (!strcmp(storage, "files"))) {

Nit: you have some unnecessary parentheses here.

> + /* default backend, nothing to do */
> + free(storage);
> + } else {
> + ref_storage_backend = storage;
> + if (set_ref_storage_backend(ref_storage_backend))
> + die(_("Unknown ref storage backend %s"),
> + ref_storage_backend);
> + }
> +
>   ret += git_config_from_file(fn, repo_config, data);
>   found += 1;
>   }
> diff --git a/refs.c b/refs.c
> index 715a492..e50cca0 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1554,3 +1554,11 @@ done:
>   string_list_clear(_refnames, 0);
>   return ret;
>  }
> +
> +void register_ref_storage_backends(void) {
> + /*
> +  * No need to register the files backend; it's registered by
> +  * default. Add register_ref_storage_backend(ptr-to-backend)
> +  * entries below when you add a new backend.
> +  */

This function must be called every run, right? So why not make it
register the "files" backend explicitly? That would make it obvious
really quick if this function fails to get called in some code path.
Just a thought...

> +}
> [...]

Michael

-- 
Michael Haggerty
mhag...@alum.mit.edu

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] mergetool: reorder vim/gvim buffers in three-way diffs

[Junio C Hamano]

David Aguilar  writes:

> On Thu, Feb 11, 2016 at 08:03:57AM -0800, Junio C Hamano wrote:
>> Michael J Gruber  writes:
>> 
>> >> Does this mean that I should warn in the release notes that some
>> >> existing users might get their expectation broken but we are going
>> >> ahead anyway because we think most people read left to right and
>> >> then top down?  I am OK with saying that--I just wanted to make sure
>> >> we know that it is what we are doing.
>> >
>> > I would claim that anyone who notices the difference in buffer numbering
>> > would be positively surprised.
>> 
>> Thanks. I, being a non-user of vim, was wondering if people who had
>> their own user-defined commands (macros? and possibly short-cut keys
>> to invoke them) built around the old (and odd) numbering need to
>> adjust--in which case we may need to forewarn.
>> 
>> > In any case, the buffer numbering is not the same (it is local remote
>> > base merge) but it doesn't matter in this case because only one window
>> > is displayed, so there is no visual association.
>> 
>> OK, thanks.
>
> Sorry for not noticing this thread earlier.
> The change and the rationale sound good to me.
>
> FWIW,
>
> Acked-by: David Aguilar 

Thanks all.  Will merge to 'next' and then to 'master' shortly.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: `.git` symlink makes `git submodule add` fail

[Vít Novotný]

On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:
> Junio C Hamano  writes:
> 
> > Vít Novotný  writes:
> >
> >> Is this a bug, or is the ability to symlink `.git` just a happy 
> >> coincidence?
> >
> > It has never been supported.
> 
> Oops, hit "send" too early.
> 
> We have support for a "gitdir:" facility that would work even on a
> filesystem that cannot do symlinks (see gitrepository-layout(5)),
> and both the higher-level submodule Porcelain and the more recent
> "worktree" experimental code do use it.
> 

That sounds ideal, thank you for letting me know. Still, I believe that
well-behaved applications should be indifferent to symlinks, unless the
distinction is significant. Would there be any objections, if I
submitted a patch / series of patches that would add proper symlink
support?
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Implement https public key pinning

[Jeff King]

On Fri, Feb 12, 2016 at 11:02:26AM +0100, Thomas Gummerer wrote:

> > Also some people suggested that git should fail if this option is
> > requested in the config but not supported by the libcurl version instead
> > of falling back to just not pin the key. I'm undecided about that.
> 
> This seems to have been suggested off list (or at least I can't find
> the message).  FWIW I do agree with failing or as a bare minimum
> warning the user if the config option is set, but not supported by the
> libcurl version.  Otherwise we risk giving the user a false sense of
> security when the option is set, which is arguably worse than not
> having the security option at all.

We can't do this perfectly, because older versions of git do not yet
know about the option, and will therefore just silently ignore it. And
for consistency there, we usually do the same for features that we know
about but are unsupported.

But I agree for something with security implications like this, we are
better off warning when we know support is not built in. That's not
perfect, but it's better than nothing.

I wonder if there are other options which should get the same treatment.
Most of the obvious ones I could think of (e.g., http.sslverify) do not
need it, because either they always have support built, or they
fail-closed, or both.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Gated Merge?

[Junio C Hamano]

Andrew Ardill  writes:

> What is the benefit in doing this in notes vs having the tests in the
> working tree?

Interesting.  I have never thought of adding this information to the
project history proper---I've viewed this as primarily an aid for
keeping track of topics in-flight by an individual, i.e. something
that the rest of the project do not want to even see.

> Pros:
>
>  - merge-gates can be added after the commit, but will stick with the
> commit if it moves around (as opposed to creating a second commit to
> add the merge-gate to the working tree)

I think that this will not be a problem in practice if we took your
alternative approach to cap the topic with an extra commit that adds
the Merge Gate test; because the workflow this targets treat a topic
as a single unit, the extra commit will be rebased together with the
real commits in the topic.

>  - cross repository standards can be established that allow
> merge-gates to be detected and run automatically (arguably could be
> done with a standardised folder structure too, but that is more
> disruptive)

Yeah, I think that this is very similar to "something that the rest
of the project do not want to even see" problem, if you use an in-tree
approach.

> Cons:
>
>  - difficult to see the current complete set of merge-gates

Yes, we would need to have a quick way to enumerate commits with
these notes in the rev-list output.

>  - difficult to make changes to a number of merge-gates at the same time

Hmm, I am not sure if/how that will be an issue.

>  - poorly defined behaviour when multiple merge-gates overlap in
> functionality. Which gates execute first? What if I reorder the
> commits?

True. With an in-tree approach, you can define the order with the
filenames, for example, and the above will be clearer. Instead, you
would need to worry about name clashes, though.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: `.git` symlink makes `git submodule add` fail

[Jeff King]

On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:

> >> Is this a bug, or is the ability to symlink `.git` just a happy 
> >> coincidence?
> >
> > It has never been supported.
> 
> Oops, hit "send" too early.
> 
> We have support for a "gitdir:" facility that would work even on a
> filesystem that cannot do symlinks (see gitrepository-layout(5)),
> and both the higher-level submodule Porcelain and the more recent
> "worktree" experimental code do use it.

And the way to convince git to make the link for you is with clone's
"--separate-git-dir" option.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git patch shows tab but files doesn't

[Jeff King]

On Fri, Feb 12, 2016 at 09:32:32PM +0530, Jagan Teki wrote:

> Hi,
> 
> I edited a file as one space between #define and macro and tab space
> between macro and BIT value, similar as below.
> 
> #define USE_FSR   BIT(6)
> #define SNOR_WR BIT(7)
> 
> Once I created the patch looks different as tab space between #define
> and macro and 2 tab spaces between macro and BIT value looks like it's
> added tab spaces while creating a patch as below.
> 
> +#defineUSE_FSR  BIT(6)
> +#defineSNOR_WRBIT(7)
> 
> Any help how to fix this, was it an issues with vim or do we have any
> git-config fixes for this?

It's hard to tell from your output, but you can often get tab
realignment on your terminal when viewing a patch, because the tabs are
shifted in by one character (the leading "+"). So if your terminal
is showing (as most do) a tab as "skip to the next tabstop", then:

  1. The size of particular tabs may look quite different if they move
 from an exact tabstop to the next character (or vice versa).

 If the space between your "#define" and the macro name is actually
 a tab, then I'd expect it to exhibit this behavior with a tabstop
 of 8 (because #define is 7 chars).

  2. Spaces don't exhibit this, so two lines which _are_ aligned in the
 source for a specific tabstop may not be in the patch.

Piping the diff (and your file) through "cat -A" can often show more
clearly what's going on.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: `.git` symlink makes `git submodule add` fail

[Junio C Hamano]

Vít Novotný  writes:

> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?

It has never been supported.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: `.git` symlink makes `git submodule add` fail

[Junio C Hamano]

Junio C Hamano  writes:

> Vít Novotný  writes:
>
>> Is this a bug, or is the ability to symlink `.git` just a happy coincidence?
>
> It has never been supported.

Oops, hit "send" too early.

We have support for a "gitdir:" facility that would work even on a
filesystem that cannot do symlinks (see gitrepository-layout(5)),
and both the higher-level submodule Porcelain and the more recent
"worktree" experimental code do use it.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Gated Merge?

[Jeff King]

On Fri, Feb 12, 2016 at 09:44:13AM -0800, Junio C Hamano wrote:

> Andrew Ardill  writes:
> 
> > What is the benefit in doing this in notes vs having the tests in the
> > working tree?
> 
> Interesting.  I have never thought of adding this information to the
> project history proper---I've viewed this as primarily an aid for
> keeping track of topics in-flight by an individual, i.e. something
> that the rest of the project do not want to even see.

After reading Andrew's message, I wondered if these are really any
different than regular tests at all.

Let's say I implement feature X on a topic branch, and I add a
regression test for it. Once that is merged, now we have that regression
test forever[1], and any future topic branches that get merged from
master must pass that test or be rejected.

If I update the interface for foo(), it is the same thing. We do not
write a specific test for it, but we expect that the compiler will catch
any callers of the old foo, because the new tree carries the updated
definition that will not work with them (and we often structure our
interface refactoring exactly to catch such things).

So let's go back to your FORMAT_PRINT example. The topic changes all of
the format-attributes into FORMAT_PRINTF, but we never add a "test" that
says "now there are no more bare format-attributes, and if there are, it
is a regression".

But I don't think this needs to have anything to do with merges in
particular, or rules like "when merging a branch that does not have me
in it". It is about saying "from here on out, the tree state should
match this property, and we can test it by running this script". And
then running "make code-lint-tests" becomes part of the acceptance
testing for a proposed topic merge, just like "make test" is already
(and likewise, people forking _new_ branches from master after the topic
is merged would make sure they do not fail the code-lint tests before
even submitting the topic).

-Peff

[1] Of course it doesn't _have_ to be forever. We sometimes modify or
back out tests as new situations come up, and we could do the same for
these sorts of code-lint tests.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Gated Merge?

[Junio C Hamano]

Jeff King  writes:

> But I don't think this needs to have anything to do with merges in
> particular, or rules like "when merging a branch that does not have me
> in it". It is about saying "from here on out, the tree state should
> match this property, and we can test it by running this script". And
> then running "make code-lint-tests" becomes part of the acceptance
> testing for a proposed topic merge, just like "make test" is already
> (and likewise, people forking _new_ branches from master after the topic
> is merged would make sure they do not fail the code-lint tests before
> even submitting the topic).

That certainly is true, but this strays more and more from my
original motive of implementing an automated evil-merge scheme that
is better than what I currently have.

We try to do our tree-wide refactoring in such a way that it would
break the compilation (by changing function signature) when we can.
Catching with "make test" would certainly generalize it, but the
endgame result I was shooting for was to come up with a solution for
each topic in-flight just once and keep replaying it until it is
merged.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Disown ssh+git and git+ssh

[Jeff King]

On Fri, Feb 12, 2016 at 04:09:37PM +0100, Carlos Martín Nieto wrote:

> These were silly from the beginning, but we have to support them for
> compatibility. That doesn't mean we have to show them in the
> documentation. These were already left out of the main list, but a
> reference in the main manpage was left, so remove that.

Yeah, that reference was added by me to try to be thorough, but I think
mentioning ssh:// (as you do here) accomplishes the same goal in a
better way.

> Also add a note to discourage their use if anybody goes looking for them
> in the source code.

Sounds like a good plan.

> diff --git a/transport.c b/transport.c
> index 9ae7184..f5ae707 100644
> --- a/transport.c
> +++ b/transport.c
> @@ -1002,6 +1002,10 @@ struct transport *transport_get(struct remote *remote, 
> const char *url)
>   || starts_with(url, "file://")
>   || starts_with(url, "git://")
>   || starts_with(url, "ssh://")
> + /*
> +  * These ssh schemes remain supported for compat but are
> +  * undocumented and their use is discouraged
> +  */
>   || starts_with(url, "git+ssh://")
>   || starts_with(url, "ssh+git://")) {
>   /*

Breaking apart an ||-chain with a comment like this is a little odd, but
I think the result is reasonably readable, so it's probably OK.

The rest of the patch is obviously correct. Thanks for following up on
the earlier discussion.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] gitk: update German translation

[Ralf Thielow]

Signed-off-by: Ralf Thielow 
---
Note:
Some translations in gitk may differ from translations in git-core
since gitk uses another glossary which I tried to follow here.

 po/de.po | 79 +++-
 1 file changed, 33 insertions(+), 46 deletions(-)

diff --git a/po/de.po b/po/de.po
index d9ba405..bde749e 100644
--- a/po/de.po
+++ b/po/de.po
@@ -21,15 +21,15 @@ msgstr ""
 msgid "Couldn't get list of unmerged files:"
 msgstr "Liste der nicht zusammengeführten Dateien nicht gefunden:"
 
 #: gitk:212 gitk:2381
 msgid "Color words"
-msgstr ""
+msgstr "Wörter einfärben"
 
 #: gitk:217 gitk:2381 gitk:8220 gitk:8253
 msgid "Markup words"
-msgstr ""
+msgstr "Wörter kennzeichnen"
 
 #: gitk:324
 msgid "Error parsing revisions:"
 msgstr "Fehler beim Laden der Versionen:"
 
@@ -185,11 +185,11 @@ msgstr "Dateien:"
 msgid "adding/removing string:"
 msgstr "Änderungen:"
 
 #: gitk:2304 gitk:4779
 msgid "changing lines matching:"
-msgstr ""
+msgstr "Geänderte Zeilen entsprechen:"
 
 #: gitk:2313 gitk:2315 gitk:4766
 msgid "Exact"
 msgstr "Exakt"
 
@@ -246,11 +246,11 @@ msgstr "Kontextzeilen"
 msgid "Ignore space change"
 msgstr "Leerzeichenänderungen ignorieren"
 
 #: gitk:2378 gitk:2380 gitk:7959 gitk:8206
 msgid "Line diff"
-msgstr ""
+msgstr "Zeilenunterschied"
 
 #: gitk:2445
 msgid "Patch"
 msgstr "Patch"
 
@@ -305,23 +305,20 @@ msgstr "Abkömmling von Lesezeichen und dieser Version 
finden"
 #: gitk:2628
 msgid "Compare with marked commit"
 msgstr "Mit Lesezeichen vergleichen"
 
 #: gitk:2629 gitk:2640
-#, fuzzy
 msgid "Diff this -> marked commit"
-msgstr "Vergleich: diese -> gewählte"
+msgstr "Vergleich: diese -> gewählte Version"
 
 #: gitk:2630 gitk:2641
-#, fuzzy
 msgid "Diff marked commit -> this"
-msgstr "Vergleich: gewählte -> diese"
+msgstr "Vergleich: gewählte -> diese Version"
 
 #: gitk:2631
-#, fuzzy
 msgid "Revert this commit"
-msgstr "Lesezeichen setzen"
+msgstr "Version umkehren"
 
 #: gitk:2647
 msgid "Check out this branch"
 msgstr "Auf diesen Zweig umstellen"
 
@@ -329,11 +326,11 @@ msgstr "Auf diesen Zweig umstellen"
 msgid "Remove this branch"
 msgstr "Zweig löschen"
 
 #: gitk:2649
 msgid "Copy branch name"
-msgstr ""
+msgstr "Zweigname kopieren"
 
 #: gitk:2656
 msgid "Highlight this too"
 msgstr "Diesen auch hervorheben"
 
@@ -349,22 +346,21 @@ msgstr "Externes Diff-Programm"
 msgid "Blame parent commit"
 msgstr "Annotieren der Elternversion"
 
 #: gitk:2660
 msgid "Copy path"
-msgstr ""
+msgstr "Pfad kopieren"
 
 #: gitk:2667
 msgid "Show origin of this line"
 msgstr "Herkunft dieser Zeile anzeigen"
 
 #: gitk:2668
 msgid "Run git gui blame on this line"
 msgstr "Diese Zeile annotieren (»git gui blame«)"
 
 #: gitk:3014
-#, fuzzy
 msgid ""
 "\n"
 "Gitk - a commit viewer for git\n"
 "\n"
 "Copyright © 2005-2014 Paul Mackerras\n"
@@ -372,11 +368,11 @@ msgid ""
 "Use and redistribute under the terms of the GNU General Public License"
 msgstr ""
 "\n"
 "Gitk - eine Visualisierung der Git-Historie\n"
 "\n"
-"Copyright \\u00a9 2005-2010 Paul Mackerras\n"
+"Copyright \\u00a9 2005-2014 Paul Mackerras\n"
 "\n"
 "Benutzung und Weiterverbreitung gemäß den Bedingungen der GNU General Public "
 "License"
 
 #: gitk:3022 gitk:3089 gitk:9857
@@ -395,45 +391,42 @@ msgstr "Gitk-Tastaturbelegung:"
 #, tcl-format
 msgid "<%s-Q>\t\tQuit"
 msgstr "<%s-Q>\t\tBeenden"
 
 #: gitk:3049
-#, fuzzy, tcl-format
+#, tcl-format
 msgid "<%s-W>\t\tClose window"
-msgstr "<%s-F>\t\tSuchen"
+msgstr "<%s-F>\t\tFenster schließen"
 
 #: gitk:3050
 msgid "\t\tMove to first commit"
 msgstr "\t\tZur neuesten Version springen"
 
 #: gitk:3051
 msgid "\t\tMove to last commit"
 msgstr "\t\tZur ältesten Version springen"
 
 #: gitk:3052
-#, fuzzy
 msgid ", p, k\tMove up one commit"
-msgstr ", p, i\tNächste neuere Version"
+msgstr ", p, k\tNächste neuere Version"
 
 #: gitk:3053
-#, fuzzy
 msgid ", n, j\tMove down one commit"
-msgstr ", n, k\tNächste ältere Version"
+msgstr ", n, j\tNächste ältere Version"
 
 #: gitk:3054
-#, fuzzy
 msgid ", z, h\tGo back in history list"
-msgstr ", z, j\tEine Version zurückgehen"
+msgstr ", z, h\tEine Version zurückgehen"
 
 #: gitk:3055
 msgid ", x, l\tGo forward in history list"
 msgstr ", x, l\tEine Version weitergehen"
 
 #: gitk:3056
 #, tcl-format
 msgid "<%s-n>\tGo to n-th parent of current commit in history list"
-msgstr ""
+msgstr "<%s-n>\tZu n-ter Elternversion in Versionshistorie springen"
 
 #: gitk:3057
 msgid "\tMove up one page in commit list"
 msgstr "\tEine Seite nach oben blättern"
 
@@ -512,13 +505,12 @@ msgstr "<%s-G>\t\tWeitersuchen"
 #: gitk:3074
 msgid "\tMove to next find hit"
 msgstr "\tWeitersuchen"
 
 #: gitk:3075
-#, fuzzy
 msgid "g\t\tGo to commit"
-msgstr "\t\tZur ältesten Version springen"
+msgstr "g\t\tZu Version springen"
 
 #: gitk:3076
 msgid "/\t\tFocus the search box"
 msgstr "/\t\tTastaturfokus ins Suchfeld"
 
@@ -671,13 +663,12 @@ msgstr "Versionsbeschreibung:"
 #: gitk:4085
 msgid "Matches 

Re: [PATCH] gitk: update German translation

[Stefan Beller]

2016-02-12 10:40 GMT-08:00 Ralf Thielow :
> Signed-off-by: Ralf Thielow 
> ---
> Note:
> Some translations in gitk may differ from translations in git-core
> since gitk uses another glossary which I tried to follow here.
>
>  po/de.po | 79 
> +++-
>  1 file changed, 33 insertions(+), 46 deletions(-)
>
> diff --git a/po/de.po b/po/de.po
> index d9ba405..bde749e 100644
> --- a/po/de.po
> +++ b/po/de.po
> @@ -21,15 +21,15 @@ msgstr ""
>  msgid "Couldn't get list of unmerged files:"
>  msgstr "Liste der nicht zusammengeführten Dateien nicht gefunden:"
>
>  #: gitk:212 gitk:2381
>  msgid "Color words"
> -msgstr ""
> +msgstr "Wörter einfärben"
>
>  #: gitk:217 gitk:2381 gitk:8220 gitk:8253
>  msgid "Markup words"
> -msgstr ""
> +msgstr "Wörter kennzeichnen"
>
>  #: gitk:324
>  msgid "Error parsing revisions:"
>  msgstr "Fehler beim Laden der Versionen:"

"Laden" (loading) is not the most correct translation for parsing,
but maybe good for the user. What about one of "analysieren, (zer)gliedern,
parsen, bestimmen" ?

Thanks,
Stefan
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] gitk: update German translation

[Ralf Thielow]

2016-02-12 19:56 GMT+01:00 Stefan Beller :
> 2016-02-12 10:40 GMT-08:00 Ralf Thielow :
>>  #: gitk:324
>>  msgid "Error parsing revisions:"
>>  msgstr "Fehler beim Laden der Versionen:"
>
> "Laden" (loading) is not the most correct translation for parsing,
> but maybe good for the user. What about one of "analysieren, (zer)gliedern,
> parsen, bestimmen" ?
>

This message comes from a function named "parseviewrevs", so I guess the
function tries to parse the revs gitk wants to present the user.
So I think "Laden der Versionen" is OK for this kinda low-level POV.

Thanks
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Gated Merge?

[Jeff King]

On Fri, Feb 12, 2016 at 11:06:04AM -0800, Junio C Hamano wrote:

> Jeff King  writes:
> 
> > But I don't think this needs to have anything to do with merges in
> > particular, or rules like "when merging a branch that does not have me
> > in it". It is about saying "from here on out, the tree state should
> > match this property, and we can test it by running this script". And
> > then running "make code-lint-tests" becomes part of the acceptance
> > testing for a proposed topic merge, just like "make test" is already
> > (and likewise, people forking _new_ branches from master after the topic
> > is merged would make sure they do not fail the code-lint tests before
> > even submitting the topic).
> 
> That certainly is true, but this strays more and more from my
> original motive of implementing an automated evil-merge scheme that
> is better than what I currently have.
> 
> We try to do our tree-wide refactoring in such a way that it would
> break the compilation (by changing function signature) when we can.
> Catching with "make test" would certainly generalize it, but the
> endgame result I was shooting for was to come up with a solution for
> each topic in-flight just once and keep replaying it until it is
> merged.

Yeah, that makes sense. I do repeated merges myself, and it is a pain.
I do not usually use your Reintegrate scripts, though when I have done
so, they work pretty well. And I don't think you're going to come up
with anything much better for the general case.

Let's split the problem into "detect a problem" from "apply the fix for
a topic".

You do not want to stop detecting once the original topic is merged. The
new rule is "we spell this as FORMAT_PRINTF" and you want to detect it
in simultaneous topics, _and_ in future topics. So if it is worth
checking in an auomated way, that should go into the tree.

So once we've detected a problem, how do we fix it?

For the specific case of FORMAT_PRINTF, you showed a possible "extra
processing" snippet to fix the problem in an automated way. But if we
realize that these gated attributes are really just another form of
test, it should be obvious that we cannot do so automatically in the
general case. You do not expect to fix a failed "make test" on a merge
in an automated way; there are too many possible resolutions. The best
we can do is detect the problem, create a patch, and then apply that
patch as appropriate.

So I'm somewhat doubtful that it would be worth the effort to create
infrastructure for "automate this fix" that is anything except "apply
this patch and tell me if we now pass the test".

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Junio C Hamano]

Jeff King  writes:

> On Thu, Feb 04, 2016 at 11:34:59AM +0100, Sebastian Schuberth wrote:
>
>> This avoids output like
>> 
>> warning: ignoring broken ref refs/remotes/origin/HEAD
>> 
>> while completing branch names.
>
> Hmm. I feel like this case (HEAD points to a branch, then `fetch
> --prune` deletes it) came up recently and we discussed quieting that
> warning. But now I cannot seem to find it.
>
> Anyway, I this is a reasonable workaround. Errors from bash completion
> scripts are almost always going to be useless and get in the way of
> reading your own prompt.

I think that is absolutely the right stance to take, but then I
wonder if it is a sensible execution to sprinkle 2>/dev/null
everywhere.

For example, couldn't we do something like this instead?

This is just for illustration and does not remove all 2>/dev/null
and replace them with a single redirection that covers the entire
shell function body, but something along this line smells a lot more
pleasant.  I dunno.

 contrib/completion/git-completion.bash | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/contrib/completion/git-completion.bash 
b/contrib/completion/git-completion.bash
index ba4137d..637c42d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -47,14 +47,14 @@ __gitdir ()
elif [ -d .git ]; then
echo .git
else
-   git rev-parse --git-dir 2>/dev/null
+   git rev-parse --git-dir
fi
elif [ -d "$1/.git" ]; then
echo "$1/.git"
else
echo "$1"
fi
-}
+} 2>/dev/null
 
 # The following function is based on code from:
 #
@@ -320,7 +320,7 @@ __git_heads ()
refs/heads
return
fi
-}
+} 2>/dev/null
 
 __git_tags ()
 {
@@ -330,7 +330,7 @@ __git_tags ()
refs/tags
return
fi
-}
+} 2>/dev/null
 
 # __git_refs accepts 0, 1 (to pass to __gitdir), or 2 arguments
 # presence of 2nd argument means use the guess heuristic employed
@@ -389,7 +389,7 @@ __git_refs ()
"refs/remotes/$dir/" 2>/dev/null | sed -e "s#^$dir/##"
;;
esac
-}
+} 2>/dev/null
 
 # __git_refs2 requires 1 argument (to pass to __git_refs)
 __git_refs2 ()
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

[Junio C Hamano]

Duy Nguyen  writes:

> On Thu, Feb 11, 2016 at 4:01 AM, Jonathan Nieder  wrote:
>> 
>>
>> I really like this design.  I'm tempted to implement it (since it
>> lacks a bunch of the downsides of clone.bundle).
>
> A bit disappointed we still do not address resumable fetch.

As you already said, the initial "clone" and subsequent incremental
"fetch" are orthogonal issues.

Because the proposed update to "clone" has much larger payoff than
the proposed change to "fetch", i.e.

 * The amount of data that gets transferred is much larger, hence
   the chance of network timing out in a poor network environment is
   much higher, need for resuming much larger.

 * Not only the approach makes "clone" resumable and helping
   clients, it helps the server offload bulk transfer out to CDN.

and it has much smaller damage to the existing code, i.e.

 * We do not have to pessimize the packing process, only to discard
   the bulk of bytes that were generated, like the proposed approach
   for "fetch".

 * The area new code is needed is well isolated and the switch to
   new protocol happens very early in the exchange without sharing
   code to existing codepath; these properties make it less risky to
   introduce regression.

it is simply a good taste to give "clone" higher priority between
the two.

So I do not think there is much reason to feel disappointed.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: `.git` symlink makes `git submodule add` fail

[Vít Novotný]

On Fri, Feb 12, 2016 at 01:27:33PM -0500, Jeff King wrote:
> On Fri, Feb 12, 2016 at 10:19:38AM -0800, Junio C Hamano wrote:
> 
> > >> Is this a bug, or is the ability to symlink `.git` just a happy 
> > >> coincidence?
> > >
> > > It has never been supported.
> > 
> > Oops, hit "send" too early.
> > 
> > We have support for a "gitdir:" facility that would work even on a
> > filesystem that cannot do symlinks (see gitrepository-layout(5)),
> > and both the higher-level submodule Porcelain and the more recent
> > "worktree" experimental code do use it.
> 
> And the way to convince git to make the link for you is with clone's
> "--separate-git-dir" option.
> 
> -Peff

What's curious is that this doesn't really work either, so the issue
doesn't seem to be the lack of symlink support, but rather the lack of
willingness on the part of Git to resolve a path recursively.

The following works flawlessly:

  $ mkdir repos
  $ git init
  $ mv .git repos/repoA
  $ printf 'gitdir: repos/repoA\n' >.git
  $ git submodule add https://github.com/witiko/git-parallel

There is, however, a minor pain, which makes the repository unportable:

  $ cat git-parallel/.git
  gitdir: /tmp/foobar/repos/repoA/modules/git-parallel

When I change the gitdir symlink to a relative path, everything works
fine as long as I don't make Git go over two gitdir symlinks:

  $ sed -i 's#^/tmp/foobar#..#' git-parallel/.git
  $ git status
  [works]
  $ sed -i 's#repos/repoA#.git#' git-parallel/.git
  $ git status
  fatal: Not a git repository: git-parallel/../.git/modules/git-parallel

Clearly, Git resolves the first gitdir symlink:

  git-parallel/.git => git-parallel/../.git/modules/git-parallel

but refuses to resolve recursively:

  git-parallel/../.git/modules/git-parallel =>
  git-parallel/../repos/repoA/modules/git-parallel
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Junio C Hamano]

Jeff King  writes:

> I agree it's a lot more pleasant, assuming there are no cases where we
> would want to pass through an error. But I really cannot think of one.
> Even explosive "woah, your git repo is totally corrupted" messages
> probably should be suppressed in the prompt.
>
>> @@ -320,7 +320,7 @@ __git_heads ()
>>  refs/heads
>>  return
>>  fi
>> -}
>> +} 2>/dev/null
>
> Today I learned about yet another fun corner of POSIX shell.

I may have learned about this soon after I started learning bash,
but I admit that this was the first time I found a practical use
case for it ;-).



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHv10 5/7] git submodule update: have a dedicated helper for cloning

[Stefan Beller]

On Thu, Feb 11, 2016 at 6:03 PM, Stefan Beller  wrote:
> This introduces a new helper function in git submodule--helper
> which takes care of cloning all submodules, which we want to
> parallelize eventually.
>
> Some tests (such as empty URL, update_mode=none) are required in the
> helper to make the decision for cloning. These checks have been
> moved into the C function as well (no need to repeat them in the
> shell script).
>
> Signed-off-by: Stefan Beller 
> ---
>  builtin/submodule--helper.c | 236 
> 
>  git-submodule.sh|  45 +++--
>  2 files changed, 247 insertions(+), 34 deletions(-)
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index f4c3eff..e865fca 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -255,6 +255,241 @@ static int module_clone(int argc, const char **argv, 
> const char *prefix)
> return 0;
>  }
>
> +static int git_submodule_config(const char *var, const char *value, void *cb)
> +{
> +   return submodule_config(var, value, cb);
> +}

This is the thing I forgot to address.
Next reroll will inline submodule_config directly.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[SZEDER Gábor]

Quoting Junio C Hamano :


Jeff King  writes:


On Thu, Feb 04, 2016 at 11:34:59AM +0100, Sebastian Schuberth wrote:


This avoids output like

warning: ignoring broken ref refs/remotes/origin/HEAD

while completing branch names.


Hmm. I feel like this case (HEAD points to a branch, then `fetch
--prune` deletes it) came up recently and we discussed quieting that
warning. But now I cannot seem to find it.

Anyway, I this is a reasonable workaround. Errors from bash completion
scripts are almost always going to be useless and get in the way of
reading your own prompt.


I think that is absolutely the right stance to take, but then I
wonder if it is a sensible execution to sprinkle 2>/dev/null
everywhere.

For example, couldn't we do something like this instead?

This is just for illustration and does not remove all 2>/dev/null
and replace them with a single redirection that covers the entire
shell function body, but something along this line smells a lot more
pleasant.  I dunno.


Please no :)

First, we don't have to redirect stderr of every completion function,
it's sufficient to do so only for the two "main" entry point functions
__git_main() and __gitk_main().

But:

 * It would swallow even those errors that we are interested in,
   e.g. (note the missing quotes around $foo):

  $ func () { if [ $foo = y ] ; then echo "foo is y" ; fi ; }
  $ foo=
  $ func 2>/dev/null
  $ func
  bash: [: =: unary operator expected

   Something like this should not happen, it's a bug in the
   completion script that should be fixed, and we should get a bug
   report.

 * I often find myself tracing/debugging the completion script
   through stderr by scattering

  echo >&2 "foo: '$foo'"

   and the like all over the place.  If completion functions' stderr
   were redirected, then I would have to disable that redirection
   first to be able do this kind of poor man's tracing.

 * I have a WIP patch series that deals with errors from git
   commands.
   It's a mixed bag of __gitdir()-related cleanups, fixes and
   optimizations, which factors out all git executions into a
   __git() wrapper function and redirects stderr only in that
   function, thereby eliminating most of the 2>/dev/null
   redirections in the completion script.
   It still needs some work to iron out a wrinkle or two around
   corner cases, though.




 contrib/completion/git-completion.bash | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/contrib/completion/git-completion.bash  
b/contrib/completion/git-completion.bash

index ba4137d..637c42d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -47,14 +47,14 @@ __gitdir ()
elif [ -d .git ]; then
echo .git
else
-   git rev-parse --git-dir 2>/dev/null
+   git rev-parse --git-dir
fi
elif [ -d "$1/.git" ]; then
echo "$1/.git"
else
echo "$1"
fi
-}
+} 2>/dev/null

 # The following function is based on code from:
 #
@@ -320,7 +320,7 @@ __git_heads ()
refs/heads
return
fi
-}
+} 2>/dev/null

 __git_tags ()
 {
@@ -330,7 +330,7 @@ __git_tags ()
refs/tags
return
fi
-}
+} 2>/dev/null

 # __git_refs accepts 0, 1 (to pass to __gitdir), or 2 arguments
 # presence of 2nd argument means use the guess heuristic employed
@@ -389,7 +389,7 @@ __git_refs ()
"refs/remotes/$dir/" 2>/dev/null | sed -e "s#^$dir/##"
;;
esac
-}
+} 2>/dev/null

 # __git_refs2 requires 1 argument (to pass to __git_refs)
 __git_refs2 ()



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

git svn dcommit doesn't support --username option for file:/// urls

[Tim Ringenbach]

Hi,

'git svn dcommit' doesn't seem to honor the --username argument when
my svn repository url is a file:/// url.  It doesn't complain either,
it just seems to silently ignore the option. My dcommits show up as
the user I'm logged in as. The only way I found to change that is to
'sudo' to some other user.

The actual 'svn' command does support --username with 'svn commit'.

What I'm actually up to, is trying to make a svn to git mirror
bi-directional. Right now, I have a cron job that 'git svn fetch's and
'git push origin's with some configs setup so that it does what I
want.

I was experimenting with writing some scripting to go in the other
direction, and my first step was seeing if I could commit to svn as
any user.  It seems like I should be able to and that git-svn just
doesn't support it.

(BTW, I'm aware there's a lot of pitfalls I'll have to work around,
and that I'll need to be very careful with verifying that the most
recent 'git-svn-id:' matches the branch and revision I expect to be
committing to, and that bad things will happen if I mess it up.)

Thanks,
Tim
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[SZEDER Gábor]

Quoting Johannes Schindelin :


Hi Peff,

On Thu, 4 Feb 2016, Jeff King wrote:


> diff --git a/contrib/completion/git-completion.bash  
b/contrib/completion/git-completion.bash

> index 15ebba5..7c0549d 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -317,7 +317,7 @@ __git_heads ()
>local dir="$(__gitdir)"
>if [ -d "$dir" ]; then
>git --git-dir="$dir" for-each-ref --format='%(refname:short)' \
> -  refs/heads
> +  refs/heads 2>/dev/null
>return

Not really related to your topic, but digging into it caused me to read
b7dd2d2 (for-each-ref: Do not lookup objects when they will not be used,
2009-05-27), which is about making sure for-each-ref is very fast in
completion.

It looks like %(refname:short) is actually kind of expensive:


Yep, this was reported on the Git for Windows bug tracker, too:

https://github.com/git-for-windows/git/issues/524


$ time git for-each-ref --format='%(refname)' refs/tags  >/dev/null

real0m0.004s
user0m0.000s
sys 0m0.004s

$ time git for-each-ref --format='%(refname:short)' refs/tags >/dev/null

real0m0.009s
user0m0.004s
sys 0m0.004s


And the timings in the ticket I mentioned above are not pretty small:
0.055s vs 1.341s


It's ironic that 'refname:short' came about because it was faster than
'refname' plus removing 'refs/{heads,tags,remotes}/' in a shell loop, at
least on Linux.

However, 'refname:short' performs a lot more stat() calls per ref to check
ambiguity, especially since many ref races got fixed.  In a repo with a
single master branch:

  $ strace git for-each-ref --format='%(refname)' refs/heads/master  
2>&1 |grep 'stat("\.git.*\(master\|packed-refs\)'

  stat(".git/refs/heads/master", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0
  lstat(".git/refs/heads/master", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0

  $ strace git for-each-ref --format='%(refname:short)'  
refs/heads/master 2>&1 |grep 'stat("\.git.*\(master\|packed-refs\)'

  stat(".git/refs/heads/master", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0
  lstat(".git/refs/heads/master", {st_mode=S_IFREG|0644, st_size=41, ...}) = 0
  lstat(".git/master", 0x7fff6dac9610)= -1 ENOENT (No such file  
or directory)
  stat(".git/packed-refs", 0x7fff6dac9460) = -1 ENOENT (No such file  
or directory)
  lstat(".git/refs/master", 0x7fff6dac9610) = -1 ENOENT (No such file  
or directory)
  stat(".git/packed-refs", 0x7fff6dac9460) = -1 ENOENT (No such file  
or directory)
  lstat(".git/refs/tags/master", 0x7fff6dac9610) = -1 ENOENT (No such  
file or directory)
  stat(".git/packed-refs", 0x7fff6dac9460) = -1 ENOENT (No such file  
or directory)
  lstat(".git/refs/remotes/master", 0x7fff6dac9610) = -1 ENOENT (No  
such file or directory)
  stat(".git/packed-refs", 0x7fff6dac9460) = -1 ENOENT (No such file  
or directory)
  lstat(".git/refs/remotes/master/HEAD", 0x7fff6dac9610) = -1 ENOENT  
(No such file or directory)
  stat(".git/packed-refs", 0x7fff6dac9460) = -1 ENOENT (No such file  
or directory)


Since stat()s were never a strong side of Windows, I'm afraid 'refname:short'
fired backwards and made things much slower over there.  Ouch.

I think in this case we should opt for performance instead of correctness,
and use Peff's 'refname:strip=2'.  Ambiguous refs will only hurt you, if,
well, your repo actually has ambiguous refs AND you happen to want to do
something with one of those refs.  I suspect that's rather uncommon, and
even then you could simply rename one of those refs.  OTOH, as shown in
the ticket, you don't need that many refs to make refs completion
unacceptably slow on Windows, and it will bite every time you attempt to
complete a ref.

Now, if 'git for-each-ref' could understand '**' globbing, not just
fnmatch...



The upcoming refname:strip does much better:

$ time git for-each-ref --format='%(refname:strip=2)' refs/tags >/dev/null

real0m0.004s
user0m0.000s
sys 0m0.004s


This is funny: after reading the commit message at
https://github.com/git/git/commit/0571979b it eludes me why strip=2 should
be so much faster than short...

Ciao,
Dscho



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

financní služby

[financní služby]

Požádat o pujcku ve výši 3% odpovedi na tento e-mail pro více info
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Jeff King]

On Fri, Feb 12, 2016 at 10:40:48PM +0100, SZEDER Gábor wrote:

>  * It would swallow even those errors that we are interested in,
>e.g. (note the missing quotes around $foo):
> [...]
>  * I often find myself tracing/debugging the completion script
>through stderr by scattering
> 
>   echo >&2 "foo: '$foo'"

One alternative to deal with these would be to add a flag to
conditionally turn off stderr, and then leave it on during normal
operation and disable it (letting everything through, including whatever
random cruft git commands produce) for debugging.

But...

>  * I have a WIP patch series that deals with errors from git
>commands.

I'm happy to wait and see what this patch looks like (and generally
happy to defer to you on maintenance issues for completion, as you are
much more likely than me to be the one fixing things later on :) ).

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Jeff King]

On Fri, Feb 12, 2016 at 12:00:43PM -0800, Junio C Hamano wrote:

> > Anyway, I this is a reasonable workaround. Errors from bash completion
> > scripts are almost always going to be useless and get in the way of
> > reading your own prompt.
> 
> I think that is absolutely the right stance to take, but then I
> wonder if it is a sensible execution to sprinkle 2>/dev/null
> everywhere.
> 
> For example, couldn't we do something like this instead?
> 
> This is just for illustration and does not remove all 2>/dev/null
> and replace them with a single redirection that covers the entire
> shell function body, but something along this line smells a lot more
> pleasant.  I dunno.

I agree it's a lot more pleasant, assuming there are no cases where we
would want to pass through an error. But I really cannot think of one.
Even explosive "woah, your git repo is totally corrupted" messages
probably should be suppressed in the prompt.

> @@ -320,7 +320,7 @@ __git_heads ()
>   refs/heads
>   return
>   fi
> -}
> +} 2>/dev/null

Today I learned about yet another fun corner of POSIX shell.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] submodule: port init from shell to C

[Stefan Beller]

By having the `init` functionality in C, we can reference it easier
from other parts in the code.

Signed-off-by: Stefan Beller 
---
 builtin/submodule--helper.c | 107 
 git-submodule.sh|  39 +---
 submodule.c |  21 +
 submodule.h |   1 +
 4 files changed, 130 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d1e9118..30e623a 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -214,6 +214,112 @@ static int resolve_relative_url_test(int argc, const char 
**argv, const char *pr
return 0;
 }
 
+static void init_submodule(const char *path, const char *prefix, int quiet)
+{
+   const struct submodule *sub;
+   struct strbuf sb = STRBUF_INIT;
+   char *url = NULL;
+   const char *upd = NULL;
+   char *cwd = xgetcwd();
+   const char *displaypath = relative_path(cwd, prefix, );
+
+   /* Only loads from .gitmodules, no overlay with .git/config */
+   gitmodules_config();
+
+   sub = submodule_from_path(null_sha1, path);
+
+   /*
+* Copy url setting when it is not set yet.
+* To look up the url in .git/config, we must not fall back to
+* .gitmodules, so look it up directly.
+*/
+   strbuf_reset();
+   strbuf_addf(, "submodule.%s.url", sub->name);
+   if (git_config_get_string(sb.buf, )) {
+   url = xstrdup(sub->url);
+
+   if (!url)
+   die(_("No url found for submodule path '%s' in 
.gitmodules"),
+   displaypath);
+
+   /* Possibly a url relative to parent */
+   if (starts_with_dot_dot_slash(url) ||
+   starts_with_dot_slash(url)) {
+   char *remoteurl;
+   char *remote = get_default_remote();
+   struct strbuf remotesb = STRBUF_INIT;
+   strbuf_addf(, "remote.%s.url", remote);
+   free(remote);
+
+   if (git_config_get_string(remotesb.buf, ))
+   /*
+* The repository is its own
+* authoritative upstream
+*/
+   remoteurl = xgetcwd();
+   url = relative_url(remoteurl, url, NULL);
+   strbuf_release();
+   free(remoteurl);
+   }
+
+   if (git_config_set(sb.buf, url))
+   die(_("Failed to register url for submodule path '%s'"),
+   displaypath);
+   if (!quiet)
+   printf(_("Submodule '%s' (%s) registered for path 
'%s'\n"),
+   sub->name, url, displaypath);
+   }
+
+   /* Copy "update" setting when it is not set yet */
+   strbuf_reset();
+   strbuf_addf(, "submodule.%s.update", sub->name);
+   if (git_config_get_string_const(sb.buf, ) &&
+   sub->update_strategy.type != SM_UPDATE_UNSPECIFIED) {
+   if (sub->update_strategy.type == SM_UPDATE_COMMAND) {
+   fprintf(stderr, _("warning: command update mode 
suggested for submodule '%s'\n"),
+   sub->name);
+   upd = "none";
+   } else
+   upd = 
submodule_strategy_to_string(>update_strategy);
+
+   if (git_config_set(sb.buf, upd))
+   die(_("Failed to register update mode for submodule 
path '%s'"), displaypath);
+   }
+   strbuf_release();
+   free(cwd);
+   free(url);
+}
+
+static int module_init(int argc, const char **argv, const char *prefix)
+{
+   int quiet = 0;
+   int i;
+
+   struct option module_init_options[] = {
+   OPT_STRING(0, "prefix", ,
+  N_("path"),
+  N_("alternative anchor for relative paths")),
+   OPT__QUIET(, "Suppress output for initialzing a 
submodule"),
+   OPT_END()
+   };
+
+   const char *const git_submodule_helper_usage[] = {
+   N_("git submodule--helper init []"),
+   NULL
+   };
+
+   argc = parse_options(argc, argv, prefix, module_init_options,
+git_submodule_helper_usage, 0);
+
+   if (argc == 0)
+   die(_("Pass at least one submodule"));
+
+   for (i = 0; i < argc; i++)
+   init_submodule(argv[i], prefix, quiet);
+
+   return 0;
+}
+
 struct module_list {
const struct cache_entry **entries;
int alloc, nr;
@@ -709,6 +815,7 @@ static struct cmd_struct commands[] = {
{"update-clone", update_clone},
{"resolve-relative-url", resolve_relative_url},

[PATCH 0/2] Port `git submodule init` from shell to C

[Stefan Beller]

This applies on top of the just sent series which is going to replace
sb/submodule-parallel-update, replacing sb/submodule-init.

* Using enums for the submodule update strategy now.

Thanks,
Stefan

Stefan Beller (2):
  submodule: port resolve_relative_url from shell to C
  submodule: port init from shell to C

 builtin/submodule--helper.c | 315 +++-
 git-submodule.sh| 118 +
 submodule.c |  21 +++
 submodule.h |   1 +
 t/t0060-path-utils.sh   |  42 ++
 5 files changed, 382 insertions(+), 115 deletions(-)

-- 
2.7.1.292.g18a4ced.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] submodule: port resolve_relative_url from shell to C

[Stefan Beller]

Later on we want to automatically call `git submodule init` from
other commands, such that the users don't have to initialize the
submodule themselves.  As these other commands are written in C
already, we'd need the init functionality in C, too.  The
`resolve_relative_url` function is a large part of that init
functionality, so start by porting this function to C.

To create the tests in t0060, the function `resolve_relative_url`
was temporarily enhanced to write all inputs and output to disk
when running the test suite. The added tests in this patch are
a small selection thereof.

Signed-off-by: Stefan Beller 
Signed-off-by: Junio C Hamano 
---
 builtin/submodule--helper.c | 208 +++-
 git-submodule.sh|  81 +
 t/t0060-path-utils.sh   |  42 +
 3 files changed, 253 insertions(+), 78 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 65bdc14..d1e9118 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -9,6 +9,210 @@
 #include "submodule-config.h"
 #include "string-list.h"
 #include "run-command.h"
+#include "remote.h"
+#include "refs.h"
+#include "connect.h"
+
+static char *get_default_remote(void)
+{
+   char *dest = NULL, *ret;
+   unsigned char sha1[20];
+   int flag = 0;
+   struct strbuf sb = STRBUF_INIT;
+   const char *refname = resolve_ref_unsafe("HEAD", 0, sha1, );
+
+   if (!refname)
+   die("No such ref: HEAD");
+
+   /* detached HEAD */
+   if (!strcmp(refname, "HEAD"))
+   return xstrdup("origin");
+
+   if (!skip_prefix(refname, "refs/heads/", ))
+   die(_("Expecting a full ref name, got %s"), refname);
+
+   strbuf_addf(, "branch.%s.remote", refname);
+   if (git_config_get_string(sb.buf, ))
+   ret = xstrdup("origin");
+   else
+   ret = dest;
+
+   strbuf_release();
+   return ret;
+}
+
+static int starts_with_dot_slash(const char *str)
+{
+   return str[0] == '.' && is_dir_sep(str[1]);
+}
+
+static int starts_with_dot_dot_slash(const char *str)
+{
+   return str[0] == '.' && str[1] == '.' && is_dir_sep(str[2]);
+}
+
+/*
+ * Returns 1 if it was the last chop before ':'.
+ */
+static int chop_last_dir(char **remoteurl, int is_relative)
+{
+   char *rfind = find_last_dir_sep(*remoteurl);
+   if (rfind) {
+   *rfind = '\0';
+   return 0;
+   }
+
+   rfind = strrchr(*remoteurl, ':');
+   if (rfind) {
+   *rfind = '\0';
+   return 1;
+   }
+
+   if (is_relative || !strcmp(".", *remoteurl))
+   die(_("cannot strip one component off url '%s'"),
+   *remoteurl);
+
+   free(*remoteurl);
+   *remoteurl = xstrdup(".");
+   return 0;
+}
+
+/*
+ * The `url` argument is the URL that navigates to the submodule origin
+ * repo. When relative, this URL is relative to the superproject origin
+ * URL repo. The `up_path` argument, if specified, is the relative
+ * path that navigates from the submodule working tree to the superproject
+ * working tree. Returns the origin URL of the submodule.
+ *
+ * Return either an absolute URL or filesystem path (if the superproject
+ * origin URL is an absolute URL or filesystem path, respectively) or a
+ * relative file system path (if the superproject origin URL is a relative
+ * file system path).
+ *
+ * When the output is a relative file system path, the path is either
+ * relative to the submodule working tree, if up_path is specified, or to
+ * the superproject working tree otherwise.
+ *
+ * NEEDSWORK: This works incorrectly on the domain and protocol part.
+ * remote_url  url  outcome  correct
+ * http://a.com/b  ../c http://a.com/c   yes
+ * http://a.com/b  ../../c  http://c no (domain should be kept)
+ * http://a.com/b  ../../../c   http:/c  no
+ * http://a.com/b  ../../../../chttp:c   no
+ * http://a.com/b  ../../../../../c.:c   no
+ */
+static char *relative_url(const char *remote_url,
+   const char *url,
+   const char *up_path)
+{
+   int is_relative = 0;
+   int colonsep = 0;
+   char *out;
+   char *remoteurl = xstrdup(remote_url);
+   struct strbuf sb = STRBUF_INIT;
+   size_t len = strlen(remoteurl);
+
+   if (is_dir_sep(remoteurl[len]))
+   remoteurl[len] = '\0';
+
+   if (!url_is_local_not_ssh(remoteurl) || is_absolute_path(remoteurl))
+   is_relative = 0;
+   else {
+   is_relative = 1;
+   /*
+* Prepend a './' to ensure all relative
+* remoteurls start with './' or '../'
+*/
+   if (!starts_with_dot_slash(remoteurl) &&
+   

[PATCHv11 3/7] fetching submodules: respect `submodule.fetchJobs` config option

[Stefan Beller]

This allows to configure fetching and updating in parallel
without having the command line option.

This moved the responsibility to determine how many parallel processes
to start from builtin/fetch to submodule.c as we need a way to communicate
"The user did not specify the number of parallel processes in the command
line options" in the builtin fetch. The submodule code takes care of
the precedence (CLI > config > default).

Signed-off-by: Stefan Beller 
---
 Documentation/config.txt|  6 ++
 builtin/fetch.c |  2 +-
 submodule.c | 16 +++-
 submodule.h |  2 ++
 t/t5526-fetch-submodules.sh | 14 ++
 5 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 27f02be..eb9ae85 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2715,6 +2715,12 @@ submodule..ignore::
"--ignore-submodules" option. The 'git submodule' commands are not
affected by this setting.
 
+submodule.fetchJobs::
+   Specifies how many submodules are fetched/cloned at the same time.
+   A positive integer allows up to that number of submodules fetched
+   in parallel. A value of 0 will give some reasonable default.
+   If unset, it defaults to 1.
+
 tag.sort::
This variable controls the sort ordering of tags when displayed by
linkgit:git-tag[1]. Without the "--sort=" option provided, the
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 8e74213..91f0862 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */
 static int all, append, dry_run, force, keep, multiple, update_head_ok, 
verbosity;
 static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow;
-static int max_children = 1;
+static int max_children = -1;
 static const char *depth;
 static const char *upload_pack;
 static struct strbuf default_rla = STRBUF_INIT;
diff --git a/submodule.c b/submodule.c
index 8fd4512..263cb2a 100644
--- a/submodule.c
+++ b/submodule.c
@@ -15,6 +15,7 @@
 #include "thread-utils.h"
 
 static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
+static int parallel_jobs = 1;
 static struct string_list changed_submodule_paths;
 static int initialized_fetch_ref_tips;
 static struct sha1_array ref_tips_before_fetch;
@@ -169,7 +170,12 @@ void set_diffopt_flags_from_submodule_config(struct 
diff_options *diffopt,
 
 int submodule_config(const char *var, const char *value, void *cb)
 {
-   if (starts_with(var, "submodule."))
+   if (!strcmp(var, "submodule.fetchjobs")) {
+   parallel_jobs = git_config_int(var, value);
+   if (parallel_jobs < 0)
+   die(_("negative values not allowed for 
submodule.fetchJobs"));
+   return 0;
+   } else if (starts_with(var, "submodule."))
return parse_submodule_config_option(var, value);
else if (!strcmp(var, "fetch.recursesubmodules")) {
config_fetch_recurse_submodules = 
parse_fetch_recurse_submodules_arg(var, value);
@@ -770,6 +776,9 @@ int fetch_populated_submodules(const struct argv_array 
*options,
argv_array_push(, "--recurse-submodules-default");
/* default value, "--submodule-prefix" and its value are added later */
 
+   if (max_parallel_jobs < 0)
+   max_parallel_jobs = parallel_jobs;
+
calculate_changed_submodule_paths();
run_processes_parallel(max_parallel_jobs,
   get_next_submodule,
@@ -1116,3 +1125,8 @@ void connect_work_tree_and_git_dir(const char *work_tree, 
const char *git_dir)
strbuf_release(_path);
free((void *)real_work_tree);
 }
+
+int parallel_submodules(void)
+{
+   return parallel_jobs;
+}
diff --git a/submodule.h b/submodule.h
index 9a86124..7ef3775 100644
--- a/submodule.h
+++ b/submodule.h
@@ -27,6 +27,7 @@ struct submodule_update_strategy {
enum submodule_update_type type;
const char *command;
 };
+#define SUBMODULE_UPDATE_STRATEGY_INIT {SM_UPDATE_UNSPECIFIED, NULL}
 
 int is_staging_gitmodules_ok(void);
 int update_path_in_gitmodules(const char *oldpath, const char *newpath);
@@ -58,5 +59,6 @@ int find_unpushed_submodules(unsigned char new_sha1[20], 
const char *remotes_nam
struct string_list *needs_pushing);
 int push_unpushed_submodules(unsigned char new_sha1[20], const char 
*remotes_name);
 void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
+int parallel_submodules(void);
 
 #endif
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 1241146..954d0e4 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -471,4 +471,18 @@ test_expect_success "don't fetch submodule when newly 
recorded commits are alrea
test_i18ncmp 

[PATCHv11 6/7] submodule update: expose parallelism to the user

[Stefan Beller]

Expose possible parallelism either via the "--jobs" CLI parameter or
the "submodule.fetchJobs" setting.

By having the variable initialized to -1, we make sure 0 can be passed
into the parallel processing machine, which will then pick as many parallel
workers as there are CPUs.

Signed-off-by: Stefan Beller 
---
 Documentation/git-submodule.txt |  7 ++-
 builtin/submodule--helper.c | 16 
 git-submodule.sh|  9 +
 t/t7406-submodule-update.sh | 12 
 4 files changed, 39 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 1572f05..13adebf 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -16,7 +16,7 @@ SYNOPSIS
 'git submodule' [--quiet] deinit [-f|--force] [--] ...
 'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
  [-f|--force] [--rebase|--merge] [--reference ]
- [--depth ] [--recursive] [--] [...]
+ [--depth ] [--recursive] [--jobs ] [--] [...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ]
  [commit] [--] [...]
 'git submodule' [--quiet] foreach [--recursive] 
@@ -377,6 +377,11 @@ for linkgit:git-clone[1]'s `--reference` and `--shared` 
options carefully.
clone with a history truncated to the specified number of revisions.
See linkgit:git-clone[1]
 
+-j ::
+--jobs ::
+   This option is only valid for the update command.
+   Clone new submodules in parallel with as many jobs.
+   Defaults to the `submodule.fetchJobs` option.
 
 ...::
Paths to submodule(s). When specified this will restrict the command
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7629a41..65bdc14 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -424,6 +424,7 @@ static int update_clone_task_finished(int result,
 static int update_clone(int argc, const char **argv, const char *prefix)
 {
const char *update = NULL;
+   int max_jobs = -1;
struct string_list_item *item;
struct submodule_update_clone pp = SUBMODULE_UPDATE_CLONE_INIT;
 
@@ -444,6 +445,8 @@ static int update_clone(int argc, const char **argv, const 
char *prefix)
OPT_STRING(0, "depth", , "",
   N_("Create a shallow clone truncated to the "
  "specified number of revisions")),
+   OPT_INTEGER('j', "jobs", _jobs,
+   N_("parallel jobs")),
OPT__QUIET(, N_("do't print cloning progress")),
OPT_END()
};
@@ -469,10 +472,15 @@ static int update_clone(int argc, const char **argv, 
const char *prefix)
gitmodules_config();
/* Overlay the parsed .gitmodules file with .git/config */
git_config(submodule_config, NULL);
-   run_processes_parallel(1, update_clone_get_next_task,
- update_clone_start_failure,
- update_clone_task_finished,
- );
+
+   if (max_jobs < 0)
+   max_jobs = parallel_submodules();
+
+   run_processes_parallel(max_jobs,
+  update_clone_get_next_task,
+  update_clone_start_failure,
+  update_clone_task_finished,
+  );
 
if (pp.print_unmatched) {
printf("#unmatched\n");
diff --git a/git-submodule.sh b/git-submodule.sh
index 9f554fb..10c5af9 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -645,6 +645,14 @@ cmd_update()
--depth=*)
depth=$1
;;
+   -j|--jobs)
+   case "$2" in '') usage ;; esac
+   jobs="--jobs=$2"
+   shift
+   ;;
+   --jobs=*)
+   jobs=$1
+   ;;
--)
shift
break
@@ -670,6 +678,7 @@ cmd_update()
${update:+--update "$update"} \
${reference:+--reference "$reference"} \
${depth:+--depth "$depth"} \
+   ${jobs:+$jobs} \
"$@" | {
err=
while read mode sha1 stage just_cloned sm_path
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 68ea31d..815b4d1 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -774,4 +774,16 @@ test_expect_success 'submodule update --recursive drops 
module name before recur
 test_i18ngrep "Submodule path .deeper/submodule/subsubmodule.: checked 
out" actual
)
 '
+
+test_expect_success 'submodule update can be run in parallel' '
+   (cd super2 &&
+GIT_TRACE=$(pwd)/trace.out git submodule 

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Duy Nguyen]

On Sat, Feb 13, 2016 at 6:46 AM, Jeff King  wrote:
> On Sat, Feb 13, 2016 at 12:43:00AM +0100, SZEDER Gábor wrote:
>
>>
>> Quoting SZEDER Gábor :
>>
>> >Now, if 'git for-each-ref' could understand '**' globbing, not just
>> >fnmatch...
>>
>> Oh, look, though the manpage says:
>>
>>   ...
>>   If one or more patterns are given, only refs are shown that match
>>   against at least one pattern, either using fnmatch(3) or literally,
>
> Yeah, we might want to update that. Wildmatch is basically fnmatch()
> compatible, but it understands "**" (which I _think_ is the reason we
> picked it up in the first place).

The second reason is consistent behavior across platforms.

> I think we dropped it into place by
> default because "**" is otherwise meaningless for fnmatch.
>
> I don't think there are any other differences between the two, but Duy
> probably knows offhand.

Nope. I think that's the only difference, feature-wise, between
fnmatch and wildmatch.

> It looks like we mention fnmatch() in a few places in the documentation,
> and AFAIK each of these is now outdated.
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Jeff King]

On Sat, Feb 13, 2016 at 12:21:22AM +0100, SZEDER Gábor wrote:

> I think in this case we should opt for performance instead of correctness,
> and use Peff's 'refname:strip=2'.  Ambiguous refs will only hurt you, if,
> well, your repo actually has ambiguous refs AND you happen to want to do
> something with one of those refs.  I suspect that's rather uncommon, and
> even then you could simply rename one of those refs.  OTOH, as shown in
> the ticket, you don't need that many refs to make refs completion
> unacceptably slow on Windows, and it will bite every time you attempt to
> complete a ref.

I'm not even sure that this is a correctness tradeoff at all. For
example, in the function __git_heads(), we are asking for-each-ref to
tell us about everything under refs/heads/. If you have a refs/heads/foo
and refs/tags/foo, we don't care; we are trying to print the unqualified
branch names. And in fact having refname:short print "heads/foo" in this
case may be actively wrong. For instance, in _git_branch(), you cannot
use the resulting completion of "heads/foo", as that command wants
unqualified names in "refs/heads/", and you do not have
"refs/heads/heads/foo".

So I think switching to :strip is an improvement in both correctness
_and_ performance.

> Now, if 'git for-each-ref' could understand '**' globbing, not just
> fnmatch...

I think it does already, since 4917e1e (Makefile: promote wildmatch to
be the default fnmatch implementation, 2013-05-30).

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[SZEDER Gábor]

Quoting SZEDER Gábor :


Now, if 'git for-each-ref' could understand '**' globbing, not just
fnmatch...


Oh, look, though the manpage says:

  ...
  If one or more patterns are given, only refs are shown that match
  against at least one pattern, either using fnmatch(3) or literally,

'git for-each-ref' does in fact understand double asterisks:

  $ git for-each-ref --format='%(refname)' '**/master'
  refs/heads/master
  refs/remotes/github/master
  refs/remotes/origin/master
  $ git for-each-ref --format='%(refname)' 'refs/heads/b/**'
  refs/heads/b/r/a/n/c/h

Great, this combined with refname:strip=2 and 3 might open up some
more optimization possibilities...
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Junio C Hamano]

Jeff King  writes:

> On Fri, Feb 12, 2016 at 10:40:48PM +0100, SZEDER Gábor wrote:
>
>>  * It would swallow even those errors that we are interested in,
>>e.g. (note the missing quotes around $foo):
>> [...]
>>  * I often find myself tracing/debugging the completion script
>>through stderr by scattering
>> 
>>   echo >&2 "foo: '$foo'"
>
> One alternative to deal with these would be to add a flag to
> conditionally turn off stderr, and then leave it on during normal
> operation and disable it (letting everything through, including whatever
> random cruft git commands produce) for debugging.
>
> But...
>
>>  * I have a WIP patch series that deals with errors from git
>>commands.
>
> I'm happy to wait and see what this patch looks like (and generally
> happy to defer to you on maintenance issues for completion, as you are
> much more likely than me to be the one fixing things later on :) ).
>
> -Peff

Likewise on both counts.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Interim "What's cooking"

[Eric Wong]

Junio C Hamano  wrote:
> * ew/connect-verbose (2016-01-28) 1 commit
>   (merged to 'next' on 2016-02-03 at ceac37e)
>  + pass transport verbosity down to git_connect

Btw, I posted v2 of this with tests added to t/t5570-git-daemon.sh

  http://mid.gmane.org/20160130085056.ga20...@dcvr.yhbt.net

Can you replace it with my v2 or would you prefer a standalone
patch for just the test?  Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

What's cooking in git.git (Feb 2016, #04; Fri, 12)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.  The ones marked with '.' do not appear in any of
the integration branches, but I am still holding onto them.

You can find the changes described here in the integration branches of the
repositories listed at

http://git-blame.blogspot.com/p/git-public-repositories.html

--
[New Topics]

* ep/format-printf (2016-02-11) 22 commits
 - wt-status.h: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - utf8.h: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - transport-helper.c: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - trace.h: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - strbuf.h: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - remote.c: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - pkt-line.h: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - merge-recursive.c: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - imap-send.c: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - http-backend.c: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - fsck.c: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - daemon.c: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - config.c: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - color.h: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - cache.h: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - builtin/upload-archive.c: use the FORMAT_PRINTF macro to declare the gcc 
function attribute 'format printf'
 - builtin/update-index.c: use the FORMAT_PRINTF macro to declare the gcc 
function attribute 'format printf'
 - builtin/receive-pack.c: use the FORMAT_PRINTF macro to declare the gcc 
function attribute 'format printf'
 - builtin/index-pack.c: use the FORMAT_PRINTF macro to declare the gcc 
function attribute 'format printf'
 - argv-array.h: use the FORMAT_PRINTF macro to declare the gcc function 
attribute 'format printf'
 - advice.h: use the FORMAT_PRINTF macro to declare the gcc function attribute 
'format printf'
 - git-compat-util.h: add the FORMAT_PRINTF macro

 Undecided.


* ew/force-ipv4 (2016-02-12) 1 commit
 - connect & http: support -4 and -6 switches for remote operations

 "git fetch" and friends that make network connections can now be
 told to only use ipv4 (or ipv6).

 Will merge to 'next'.


* jk/lose-name-path (2016-02-12) 5 commits
 - list-objects: pass full pathname to callbacks
 - list-objects: drop name_path entirely
 - list-objects: convert name_path to a strbuf
 - show_object_with_name: simplify by using path_name()
 - http-push: stop using name_path

 The "name_path" API was an attempt to reduce the need to construct
 the full path out of a series of path components while walking a
 tree hierarchy, but over time made less efficient because the path
 needs to be flattened, e.g. to be compared with another path that
 is already flat.  Remove the API and rewrite its users to simplify
 overall code complexity.

 Will merge to 'next'.

--
[Graduated to "master"]

* aw/push-force-with-lease-reporting (2016-02-01) 1 commit
  (merged to 'next' on 2016-02-03 at facd28f)
 + push: fix ref status reporting for --force-with-lease

 "git push --force-with-lease" has been taught to report if the push
 needed to force (or fast-forwarded).


* cc/untracked (2016-01-27) 11 commits
  (merged to 'next' on 2016-02-01 at 8203631)
 + t7063: add tests for core.untrackedCache
 + test-dump-untracked-cache: don't modify the untracked cache
 + config: add core.untrackedCache
 + dir: simplify untracked cache "ident" field
 + dir: add remove_untracked_cache()
 + dir: add {new,add}_untracked_cache()
 + update-index: move 'uc' var declaration
 + update-index: add untracked cache notifications
 + update-index: add --test-untracked-cache
 + update-index: use enum for untracked cache options
 + dir: free untracked cache when removing it

 Update the untracked cache subsystem and change its primary UI from
 "git update-index" to "git config".


* ew/connect-verbose (2016-01-28) 1 commit
  (merged to 'next' on 2016-02-03 at ceac37e)
 + pass transport verbosity down to git_connect

 There were a few "now I am doing this thing" progress messages in
 the TCP connection code that can be triggered by setting a verbose
 option internally in the code, but "git fetch -v" and friends never
 passed the 

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[SZEDER Gábor]

Quoting Jeff King :


On Sat, Feb 13, 2016 at 12:21:22AM +0100, SZEDER Gábor wrote:


I think in this case we should opt for performance instead of correctness,
and use Peff's 'refname:strip=2'.  Ambiguous refs will only hurt you, if,
well, your repo actually has ambiguous refs AND you happen to want to do
something with one of those refs.  I suspect that's rather uncommon, and
even then you could simply rename one of those refs.  OTOH, as shown in
the ticket, you don't need that many refs to make refs completion
unacceptably slow on Windows, and it will bite every time you attempt to
complete a ref.


I'm not even sure that this is a correctness tradeoff at all. For
example, in the function __git_heads(), we are asking for-each-ref to
tell us about everything under refs/heads/. If you have a refs/heads/foo
and refs/tags/foo, we don't care; we are trying to print the unqualified
branch names. And in fact having refname:short print "heads/foo" in this
case may be actively wrong. For instance, in _git_branch(), you cannot
use the resulting completion of "heads/foo", as that command wants
unqualified names in "refs/heads/", and you do not have
"refs/heads/heads/foo".

So I think switching to :strip is an improvement in both correctness
_and_ performance.


Right.  I was more worried about __git_refs(), because it asks for
everything under refs/heads/, refs/tags/ and refs/remotes/, and its
output is used in a lot more places and fed to a lot more commands than
the output of __git_heads() (or __git_tags(), for that matter).  But I
thought that a branch-tag ambiguity would cause git to error out
complaining, just like in the case of ref-path ambiguity.  Successfully
avoiding ambiguous refs for many years, I wasn't aware that 'git
rev-parse' doesn't barf, but only warns and resolves the ambiguity in
favor of the tag.



Now, if 'git for-each-ref' could understand '**' globbing, not just
fnmatch...


I think it does already, since 4917e1e (Makefile: promote wildmatch to
be the default fnmatch implementation, 2013-05-30).


Things are looking up!

A single 'master' branch and 10 remotes with 10k remote branches each,
i.e. a total of 11 refs, all packed.  To uniquely complete
'master ' after 'git checkout m' on Linux in current git.git, i.e.
with 'refname:short':

  $ cur=m ; time __gitcomp_nl "$(__git_refs '' 1)"

  real  0m7.641s
  user  0m5.888s
  sys   0m1.832s

Using 'refname:strip=2' for both 'git for-each-ref' in __git_refs():

  $ cur=m ; time __gitcomp_nl "$(__git_refs '' 1)"

  real  0m2.848s
  user  0m2.308s
  sys   0m0.596s

Quick'n'dirty PoC using 'refname:strip', '**' globbing and a few more
tricks to let 'git for-each-ref' do the filtering instead of the
shell loop behind __gitcomp_nl():

  $ cur=m ; time IFS=$'\n' COMPREPLY=( $(__git_refs_PoC '' 1) )

  real  0m0.247s
  user  0m0.208s
  sys   0m0.032s

Not bad for a Friday night, huh? :)
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 20/21] refs: add LMDB refs storage backend

[David Turner]

On Fri, 2016-02-12 at 18:01 +0100, Michael Haggerty wrote:
> On 02/05/2016 08:44 PM, David Turner wrote:
> > Add a database backend for refs using LMDB.  This backend runs git
> > for-each-ref about 30% faster than the files backend with fully
> > -packed
> > refs on a repo with ~120k refs.  It's also about 4x faster than
> > using
> > fully-unpacked refs.  In addition, and perhaps more importantly, it
> > avoids case-conflict issues on OS X.
> > 
> > LMDB has a few features that make it suitable for usage in git:
> > 
> 
> 0. It is licensed under the OpenLDAP Public License, which is
> apparently
> GPL compatible [1].
> 
> [1] http://www.gnu.org/licenses/license-list.en.html#newOpenLDAP

Oh, yeah, I checked that before choosing it and then forgot all about
it. 

> > 1. It is relatively lightweight; it requires only one header file,
> > and
> > the library code takes under 64k at runtime.
> > 
> > 2. It is well-tested: it's been used in OpenLDAP for years.
> > 
> > 3. It's very fast.  LMDB's benchmarks show that it is among
> > the fastest key-value stores.
> > 
> > 4. It has a relatively simple concurrency story; readers don't
> > block writers and writers don't block readers.
> > 
> > Ronnie Sahlberg's original version of this patchset used tdb.  The
> > major disadvantage of tdb is that tdb is hard to build on OS X. 
> >  It's
> > also not in homebrew.  So lmdb seemed simpler.
> > 
> > To test this backend's correctness, I hacked test-lib.sh and
> > test-lib-functions.sh to run all tests under the refs backend.
> > Dozens
> > of tests use manual ref/reflog reading/writing, or create
> > submodules
> > without passing --ref-storage to git init.  If those tests are
> > changed to use the update-ref machinery or test-refs-lmdb-backend
> > (or,
> > in the case of packed-refs, corrupt refs, and dumb fetch tests, are
> > skipped), the only remaining failing tests are the git-new-workdir
> > tests and the gitweb tests.
> 
> Would it possible to build your "hack" into the test suite? For
> example,
> perhaps one could implement an option that requests tests to use LMDB
> wherever possible and skip the tests that are not LMDB-compatible.
> Over
> time, we could try to increase the fraction of tests that are
> LMDB-compatible by (for example) using `git update-ref` and `git
> rev-parse` rather than reading/writing reference files via the
> filesystem directly.
> 
> Otherwise, I'm worried that LMDB support will bitrot quickly because
> it
> will be too much of a nuisance to exercise the code.

I can add that.  It requires minor changes to a large number of tests,
but that's OK.

> > Signed-off-by: David Turner 
> > ---
> >  .gitignore |1 +
> >  Documentation/config.txt   |7 +
> >  Documentation/git-clone.txt|3 +-
> >  Documentation/git-init.txt |2 +-
> >  Documentation/technical/refs-lmdb-backend.txt  |   52 +
> >  Documentation/technical/repository-version.txt |5 +
> >  Makefile   |   12 +
> >  configure.ac   |   33 +
> >  contrib/workdir/git-new-workdir|3 +
> >  refs.c |3 +
> >  refs.h |2 +
> >  refs/lmdb-backend.c| 2052
> > 
> >  setup.c|   11 +-
> >  test-refs-lmdb-backend.c   |   64 +
> >  transport.c|7 +-
> >  15 files changed, 2250 insertions(+), 7 deletions(-)
> >  create mode 100644 Documentation/technical/refs-lmdb-backend.txt
> >  create mode 100644 refs/lmdb-backend.c
> >  create mode 100644 test-refs-lmdb-backend.c
> > 
> > diff --git a/.gitignore b/.gitignore
> > index 1c2f832..87d45a2 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -199,6 +199,7 @@
> >  /test-path-utils
> >  /test-prio-queue
> >  /test-read-cache
> > +/test-refs-lmdb-backend
> >  /test-regex
> >  /test-revision-walking
> >  /test-run-command
> > diff --git a/Documentation/config.txt b/Documentation/config.txt
> > index 06d3659..bc222c9 100644
> > --- a/Documentation/config.txt
> > +++ b/Documentation/config.txt
> > @@ -1153,6 +1153,13 @@ difftool..cmd::
> >  difftool.prompt::
> > Prompt before each invocation of the diff tool.
> >  
> > +extensions.refsStorage::
> > +   Type of refs storage backend. Default is to use the
> > original
> > +   files based ref storage.  When set to "lmdb", refs are
> > stored in
> > +   the lmdb database backend.  This setting reflects the refs
> > +   backend that is currently in use; it is not possible to
> > change
> > +   the backend by changing this setting.
> > +
> 
> Let's give the users a pointer to how they *can* change this setting.
> Maybe
> 
>   Type of refs storage backend. Default is to use the original
>   

[PATCHv11 7/7] clone: allow an explicit argument for parallel submodule clones

[Stefan Beller]

Just pass it along to "git submodule update", which may pick reasonable
defaults if you don't specify an explicit number.

Signed-off-by: Stefan Beller 
---
 Documentation/git-clone.txt |  6 +-
 builtin/clone.c | 19 +--
 t/t7406-submodule-update.sh | 15 +++
 3 files changed, 33 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 789b668..cb7966a 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,7 +14,7 @@ SYNOPSIS
  [-o ] [-b ] [-u ] [--reference ]
  [--dissociate] [--separate-git-dir ]
  [--depth ] [--[no-]single-branch]
- [--recursive | --recurse-submodules] [--] 
+ [--recursive | --recurse-submodules] [--jobs ] [--] 
  []
 
 DESCRIPTION
@@ -220,6 +220,10 @@ objects from the source repository into a pack in the 
cloned repository.
The result is Git repository can be separated from working
tree.
 
+-j ::
+--jobs ::
+   The number of submodules fetched at the same time.
+   Defaults to the `submodule.fetchJobs` option.
 
 ::
The (possibly remote) repository to clone from.  See the
diff --git a/builtin/clone.c b/builtin/clone.c
index bcba080..191f522 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -50,6 +50,7 @@ static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
+static int max_jobs = -1;
 
 static struct option builtin_clone_options[] = {
OPT__VERBOSITY(_verbosity),
@@ -72,6 +73,8 @@ static struct option builtin_clone_options[] = {
N_("initialize submodules in the clone")),
OPT_BOOL(0, "recurse-submodules", _recursive,
N_("initialize submodules in the clone")),
+   OPT_INTEGER('j', "jobs", _jobs,
+   N_("number of submodules cloned in parallel")),
OPT_STRING(0, "template", _template, N_("template-directory"),
   N_("directory from which templates will be used")),
OPT_STRING_LIST(0, "reference", _reference, N_("repo"),
@@ -95,10 +98,6 @@ static struct option builtin_clone_options[] = {
OPT_END()
 };
 
-static const char *argv_submodule[] = {
-   "submodule", "update", "--init", "--recursive", NULL
-};
-
 static const char *get_repo_path_1(struct strbuf *path, int *is_bundle)
 {
static char *suffix[] = { "/.git", "", ".git/.git", ".git" };
@@ -727,8 +726,16 @@ static int checkout(void)
err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
   sha1_to_hex(sha1), "1", NULL);
 
-   if (!err && option_recursive)
-   err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+   if (!err && option_recursive) {
+   struct argv_array args = ARGV_ARRAY_INIT;
+   argv_array_pushl(, "submodule", "update", "--init", 
"--recursive", NULL);
+
+   if (max_jobs != -1)
+   argv_array_pushf(, "--jobs=%d", max_jobs);
+
+   err = run_command_v_opt(args.argv, RUN_GIT_CMD);
+   argv_array_clear();
+   }
 
return err;
 }
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 815b4d1..0791df7 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -786,4 +786,19 @@ test_expect_success 'submodule update can be run in 
parallel' '
 grep "9 tasks" trace.out
)
 '
+
+test_expect_success 'git clone passes the parallel jobs config on to 
submodules' '
+   test_when_finished "rm -rf super4" &&
+   GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 7 . 
super4 &&
+   grep "7 tasks" trace.out &&
+   rm -rf super4 &&
+   git config --global submodule.fetchJobs 8 &&
+   GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules . super4 &&
+   grep "8 tasks" trace.out &&
+   rm -rf super4 &&
+   GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 9 . 
super4 &&
+   grep "9 tasks" trace.out &&
+   rm -rf super4
+'
+
 test_done
-- 
2.7.1.292.g18a4ced.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHv11 5/7] git submodule update: have a dedicated helper for cloning

[Stefan Beller]

This introduces a new helper function in git submodule--helper
which takes care of cloning all submodules, which we want to
parallelize eventually.

Some tests (such as empty URL, update_mode=none) are required in the
helper to make the decision for cloning. These checks have been
moved into the C function as well (no need to repeat them in the
shell script).

Signed-off-by: Stefan Beller 
---
 builtin/submodule--helper.c | 231 
 git-submodule.sh|  45 +++--
 2 files changed, 242 insertions(+), 34 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f4c3eff..7629a41 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -255,6 +255,236 @@ static int module_clone(int argc, const char **argv, 
const char *prefix)
return 0;
 }
 
+struct submodule_update_clone {
+   /* states */
+   int count;
+   int print_unmatched;
+   /* configuration */
+   int quiet;
+   const char *reference;
+   const char *depth;
+   const char *recursive_prefix;
+   const char *prefix;
+   struct module_list list;
+   struct string_list projectlines;
+   struct submodule_update_strategy update;
+   struct pathspec pathspec;
+};
+#define SUBMODULE_UPDATE_CLONE_INIT {0, 0, 0, NULL, NULL, NULL, NULL, 
MODULE_LIST_INIT, STRING_LIST_INIT_DUP, SUBMODULE_UPDATE_STRATEGY_INIT}
+
+static int update_clone_inspect_next_task(struct child_process *cp,
+ struct strbuf *err,
+ struct submodule_update_clone *pp,
+ void **pp_task_cb,
+ const struct cache_entry *ce)
+{
+   const struct submodule *sub = NULL;
+   struct strbuf displaypath_sb = STRBUF_INIT;
+   struct strbuf sb = STRBUF_INIT;
+   const char *displaypath = NULL;
+   char *url = NULL;
+   int needs_cloning = 0;
+
+   if (ce_stage(ce)) {
+   if (pp->recursive_prefix)
+   strbuf_addf(err, "Skipping unmerged submodule %s/%s\n",
+   pp->recursive_prefix, ce->name);
+   else
+   strbuf_addf(err, "Skipping unmerged submodule %s\n",
+   ce->name);
+   goto cleanup;
+   }
+
+   sub = submodule_from_path(null_sha1, ce->name);
+
+   if (pp->recursive_prefix)
+   displaypath = relative_path(pp->recursive_prefix,
+   ce->name, _sb);
+   else
+   displaypath = ce->name;
+
+   if (pp->update.type == SM_UPDATE_NONE ||
+   (pp->update.type == SM_UPDATE_UNSPECIFIED &&
+sub->update_strategy.type == SM_UPDATE_NONE)) {
+   strbuf_addf(err, "Skipping submodule '%s'\n",
+   displaypath);
+   goto cleanup;
+   }
+
+   /*
+* Looking up the url in .git/config.
+* We must not fall back to .gitmodules as we only want
+* to process configured submodules.
+*/
+   strbuf_reset();
+   strbuf_addf(, "submodule.%s.url", sub->name);
+   git_config_get_string(sb.buf, );
+   if (!url) {
+   /*
+* Only mention uninitialized submodules when its
+* path have been specified
+*/
+   if (pp->pathspec.nr)
+   strbuf_addf(err, _("Submodule path '%s' not 
initialized\n"
+   "Maybe you want to use 'update --init'?"),
+   displaypath);
+   goto cleanup;
+   }
+
+   strbuf_reset();
+   strbuf_addf(, "%s/.git", ce->name);
+   needs_cloning = !file_exists(sb.buf);
+
+   strbuf_reset();
+   strbuf_addf(, "%06o %s %d %d\t%s\n", ce->ce_mode,
+   sha1_to_hex(ce->sha1), ce_stage(ce),
+   needs_cloning, ce->name);
+   string_list_append(>projectlines, sb.buf);
+
+   if (needs_cloning) {
+   cp->git_cmd = 1;
+   cp->no_stdin = 1;
+   cp->stdout_to_stderr = 1;
+   cp->err = -1;
+   argv_array_push(>args, "submodule--helper");
+   argv_array_push(>args, "clone");
+   if (pp->quiet)
+   argv_array_push(>args, "--quiet");
+
+   if (pp->prefix)
+   argv_array_pushl(>args, "--prefix", pp->prefix, 
NULL);
+
+   argv_array_pushl(>args, "--path", sub->path, NULL);
+   argv_array_pushl(>args, "--name", sub->name, NULL);
+   argv_array_pushl(>args, "--url", strdup(url), NULL);
+   if (pp->reference)
+   argv_array_push(>args, pp->reference);
+   if (pp->depth)
+   

[PATCHv11 1/7] submodule-config: keep update strategy around

[Stefan Beller]

Currently submodule..update is only handled by git-submodule.sh.
C code will start to need to make use of that value as more of the
functionality of git-submodule.sh moves into library code in C.

Add the update field to 'struct submodule' and populate it so it can
be read as sm->update or from sm->update_command.

Signed-off-by: Stefan Beller 
---
 submodule-config.c | 12 
 submodule-config.h |  2 ++
 submodule.c| 19 +++
 submodule.h| 16 
 4 files changed, 49 insertions(+)

diff --git a/submodule-config.c b/submodule-config.c
index fe8ceab..254067b 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -194,6 +194,8 @@ static struct submodule *lookup_or_create_by_name(struct 
submodule_cache *cache,
 
submodule->path = NULL;
submodule->url = NULL;
+   submodule->update_strategy.type = SM_UPDATE_UNSPECIFIED;
+   submodule->update_strategy.command = NULL;
submodule->fetch_recurse = RECURSE_SUBMODULES_NONE;
submodule->ignore = NULL;
 
@@ -340,6 +342,16 @@ static int parse_config(const char *var, const char 
*value, void *data)
free((void *) submodule->url);
submodule->url = xstrdup(value);
}
+   } else if (!strcmp(item.buf, "update")) {
+   if (!value)
+   ret = config_error_nonbool(var);
+   else if (!me->overwrite &&
+submodule->update_strategy.type != 
SM_UPDATE_UNSPECIFIED)
+   warn_multiple_config(me->commit_sha1, submodule->name,
+"update");
+   else if (parse_submodule_update_strategy(value,
+>update_strategy) < 0)
+   die(_("invalid value for %s"), var);
}
 
strbuf_release();
diff --git a/submodule-config.h b/submodule-config.h
index 9bfa65a..e4857f5 100644
--- a/submodule-config.h
+++ b/submodule-config.h
@@ -2,6 +2,7 @@
 #define SUBMODULE_CONFIG_CACHE_H
 
 #include "hashmap.h"
+#include "submodule.h"
 #include "strbuf.h"
 
 /*
@@ -14,6 +15,7 @@ struct submodule {
const char *url;
int fetch_recurse;
const char *ignore;
+   struct submodule_update_strategy update_strategy;
/* the sha1 blob id of the responsible .gitmodules file */
unsigned char gitmodules_sha1[20];
 };
diff --git a/submodule.c b/submodule.c
index b83939c..8fd4512 100644
--- a/submodule.c
+++ b/submodule.c
@@ -210,6 +210,25 @@ void gitmodules_config(void)
}
 }
 
+int parse_submodule_update_strategy(const char *value,
+   struct submodule_update_strategy *dst)
+{
+   dst->command = NULL;
+   if (!strcmp(value, "none"))
+   dst->type = SM_UPDATE_NONE;
+   else if (!strcmp(value, "checkout"))
+   dst->type = SM_UPDATE_CHECKOUT;
+   else if (!strcmp(value, "rebase"))
+   dst->type = SM_UPDATE_REBASE;
+   else if (!strcmp(value, "merge"))
+   dst->type = SM_UPDATE_MERGE;
+   else if (skip_prefix(value, "!", >command))
+   dst->type = SM_UPDATE_COMMAND;
+   else
+   return -1;
+   return 0;
+}
+
 void handle_ignore_submodules_arg(struct diff_options *diffopt,
  const char *arg)
 {
diff --git a/submodule.h b/submodule.h
index e06eaa5..9a86124 100644
--- a/submodule.h
+++ b/submodule.h
@@ -14,6 +14,20 @@ enum {
RECURSE_SUBMODULES_ON = 2
 };
 
+enum submodule_update_type {
+   SM_UPDATE_UNSPECIFIED = 0,
+   SM_UPDATE_CHECKOUT,
+   SM_UPDATE_REBASE,
+   SM_UPDATE_MERGE,
+   SM_UPDATE_NONE,
+   SM_UPDATE_COMMAND
+};
+
+struct submodule_update_strategy {
+   enum submodule_update_type type;
+   const char *command;
+};
+
 int is_staging_gitmodules_ok(void);
 int update_path_in_gitmodules(const char *oldpath, const char *newpath);
 int remove_path_from_gitmodules(const char *path);
@@ -22,6 +36,8 @@ void set_diffopt_flags_from_submodule_config(struct 
diff_options *diffopt,
const char *path);
 int submodule_config(const char *var, const char *value, void *cb);
 void gitmodules_config(void);
+int parse_submodule_update_strategy(const char *value,
+   struct submodule_update_strategy *dst);
 void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 void show_submodule_summary(FILE *f, const char *path,
const char *line_prefix,
-- 
2.7.1.292.g18a4ced.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHv11 4/7] submodule update: direct error message to stderr

[Stefan Beller]

Reroute the error message for specified but initialized submodules
to stderr instead of stdout.

Signed-off-by: Stefan Beller 
---
 git-submodule.sh   | 4 ++--
 t/t7400-submodule-basic.sh | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 9bc5c5f..9ee86d4 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -693,7 +693,7 @@ cmd_update()
 
if test "$update_module" = "none"
then
-   echo "Skipping submodule '$displaypath'"
+   echo >&2 "Skipping submodule '$displaypath'"
continue
fi
 
@@ -702,7 +702,7 @@ cmd_update()
# Only mention uninitialized submodules when its
# path have been specified
test "$#" != "0" &&
-   say "$(eval_gettext "Submodule path '\$displaypath' not 
initialized
+   say >&2 "$(eval_gettext "Submodule path '\$displaypath' 
not initialized
 Maybe you want to use 'update --init'?")"
continue
fi
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 540771c..5991e3c 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -462,7 +462,7 @@ test_expect_success 'update --init' '
git config --remove-section submodule.example &&
test_must_fail git config submodule.example.url &&
 
-   git submodule update init > update.out &&
+   git submodule update init 2> update.out &&
cat update.out &&
test_i18ngrep "not initialized" update.out &&
test_must_fail git rev-parse --resolve-git-dir init/.git &&
@@ -480,7 +480,7 @@ test_expect_success 'update --init from subdirectory' '
mkdir -p sub &&
(
cd sub &&
-   git submodule update ../init >update.out &&
+   git submodule update ../init 2>update.out &&
cat update.out &&
test_i18ngrep "not initialized" update.out &&
test_must_fail git rev-parse --resolve-git-dir ../init/.git &&
-- 
2.7.1.292.g18a4ced.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHv11 0/7] Expose submodule parallelism to the user

[Stefan Beller]

> This replaces origin/sb/submodule-parallel-update
> and is based on origin/master
>
> * broke out the patch for redirecting errors to stderr in "submodule update"
>   (Thanks Jonathan, Jacob)
> * use git_config_int and manually check for less than 0.
>   (Thanks Junio)
> * use an enum consistently now for submodule update strategy
>   (Thanks Junio)
> * fixed the funny indentation

* removed the indirect call to submodule_config, but made it directly
  (Thanks Jonathan)

> 
> I haven't looked at how this integrates with Davids refs backend, I'll take a
> look at the merge tomorrow
>
> Sorry for the long turn around time,
> Thanks,
> Stefan


Stefan Beller (7):
  submodule-config: keep update strategy around
  submodule-config: drop check against NULL
  fetching submodules: respect `submodule.fetchJobs` config option
  submodule update: direct error message to stderr
  git submodule update: have a dedicated helper for cloning
  submodule update: expose parallelism to the user
  clone: allow an explicit argument for parallel submodule clones

 Documentation/config.txt|   6 +
 Documentation/git-clone.txt |   6 +-
 Documentation/git-submodule.txt |   7 +-
 builtin/clone.c |  19 +++-
 builtin/fetch.c |   2 +-
 builtin/submodule--helper.c | 239 
 git-submodule.sh|  54 -
 submodule-config.c  |  18 ++-
 submodule-config.h  |   2 +
 submodule.c |  35 +-
 submodule.h |  18 +++
 t/t5526-fetch-submodules.sh |  14 +++
 t/t7400-submodule-basic.sh  |   4 +-
 t/t7406-submodule-update.sh |  27 +
 14 files changed, 402 insertions(+), 49 deletions(-)

-- 
2.7.1.292.g18a4ced.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCHv11 2/7] submodule-config: drop check against NULL

[Stefan Beller]

Adhere to the common coding style of Git and not check explicitly
for NULL throughout the file. There are still other occurrences in the
code base but that is usually inside of conditions with side effects.

Signed-off-by: Stefan Beller 
---
 submodule-config.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/submodule-config.c b/submodule-config.c
index 254067b..afd294b 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -295,7 +295,7 @@ static int parse_config(const char *var, const char *value, 
void *data)
if (!strcmp(item.buf, "path")) {
if (!value)
ret = config_error_nonbool(var);
-   else if (!me->overwrite && submodule->path != NULL)
+   else if (!me->overwrite && submodule->path)
warn_multiple_config(me->commit_sha1, submodule->name,
"path");
else {
@@ -319,7 +319,7 @@ static int parse_config(const char *var, const char *value, 
void *data)
} else if (!strcmp(item.buf, "ignore")) {
if (!value)
ret = config_error_nonbool(var);
-   else if (!me->overwrite && submodule->ignore != NULL)
+   else if (!me->overwrite && submodule->ignore)
warn_multiple_config(me->commit_sha1, submodule->name,
"ignore");
else if (strcmp(value, "untracked") &&
@@ -335,7 +335,7 @@ static int parse_config(const char *var, const char *value, 
void *data)
} else if (!strcmp(item.buf, "url")) {
if (!value) {
ret = config_error_nonbool(var);
-   } else if (!me->overwrite && submodule->url != NULL) {
+   } else if (!me->overwrite && submodule->url) {
warn_multiple_config(me->commit_sha1, submodule->name,
"url");
} else {
-- 
2.7.1.292.g18a4ced.dirty

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-completion.bash: always swallow error output of for-each-ref

[Jeff King]

On Sat, Feb 13, 2016 at 12:43:00AM +0100, SZEDER Gábor wrote:

> 
> Quoting SZEDER Gábor :
> 
> >Now, if 'git for-each-ref' could understand '**' globbing, not just
> >fnmatch...
> 
> Oh, look, though the manpage says:
> 
>   ...
>   If one or more patterns are given, only refs are shown that match
>   against at least one pattern, either using fnmatch(3) or literally,

Yeah, we might want to update that. Wildmatch is basically fnmatch()
compatible, but it understands "**" (which I _think_ is the reason we
picked it up in the first place). I think we dropped it into place by
default because "**" is otherwise meaningless for fnmatch.

I don't think there are any other differences between the two, but Duy
probably knows offhand.

It looks like we mention fnmatch() in a few places in the documentation,
and AFAIK each of these is now outdated.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

[Blake Burkhart]

On Wed, Feb 10, 2016 at 3:49 PM, Jeff King  wrote:
>> 2. Servers that support resumable clone include a "resumable"
>> capability in the advertisement.
>
> Because the magic happens in the git protocol, that would mean this does
> not have to be limited to git-over-http. It could be "resumable="
> to point the client anywhere (the same server over a different protocol,
> another server, etc).

I'd like to call this out as a possible security issue before it gets
implemented. Allowing the server to instruct the client what protocol
to use is a security risk. This sounds like a fine feature, just do it
carefully.

I reported a similar issue was discussed off list which eventually
became CVE-2015-7545. Basically, git-submodule allowed a repository to
specify any protocol via .gitmodules, causing the client to fetch an
arbitrary URL using a protocol of the attacker's choice. Sadly, the
existence of git-remote-ext allows easily executing arbitrary shell
commands if the server can tell the client to use it. Furthermore,
it's possible the client has some insecure or sensitive custom git
remote helpers installed.

To address this GIT_ALLOW_PROTOCOL was introduced, and git-submodule
now uses it as of 33cfccb. This environment variable specifies a
default whitelist of protocols. Whoever implements this should
probably make use of GIT_ALLOW_PROTOCOL to limit resumable clones to
the same default whitelist that git-submodule now uses.

-- 
Blake Burkhart
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html