Re: [PATCH 0/5] Use watchman to reduce index refresh time

[Paolo Ciarrocchi]

On Sun, Nov 1, 2015 at 2:55 PM, Nguyễn Thái Ngọc Duy  wrote:

Hi Duy,

> This series builds on top of the index-helper series I just sent and
> uses watchman to keep track of file changes in order to avoid lstat()
> at refresh time. The series can also be found at [1]
>
> When I started this work, watchman did not support Windows yet. It
> does now, even if still experimental [2]. So Windows people, please
> try it out if you have time.
>
> To put all pieces so far together, we have split-index to reduce index
> write time, untracked cache to reduce I/O as well as computation for
> .gitignore, index-helper for index read time and this series for
> lstat() at refresh time. The remaining piece is killing lstat() from
> untracked cache, but right now it's just some idea and incomplete
> code.

Did you manage to measure the speedup introduced by this series?

Ciao,
-- 
Paolo
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

git log --author=me

[Harry Jeffery]

Hi,

I've written a patch that allows `me` to be used as shorthand for 
$(user.name) or $(user.email) in the `--author` and `--commiter` fields.


The purpose being to make finding your own commits quicker and easier:
git log --author=me

Is this a change that would be accepted if submitted? The only thing 
it's currently missing is unit tests.


Regards,
Harry
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: When a file was locked by some program, git will work stupidly

[Mr Guillaume Seren]

Quoting dayong xie (2015-11-02 05:56:55)
> To be specific
> In my Unity project, there is a native plugin,  and plugin's extension
> is .dll, and this plugin is
> under git version control, when Unity is running, the plugin file will
> be locked.
> If i merge another branch, which contains modification of the plugin,
> git will report error, looks
> like:
> error: unable to unlink old 'xxx/xxx.dll' (Permission denied)
> This is not bad, however, the unfinished merge action will not revert
> by git, a lot of changes
> produced in repository.
> usually it makes me crazy, even worse, some of my partners are not
> good at using git.
> Of course, this problem can be avoided by quit Unity, but not every
> time we can remember. In
> my opinion, git should revert the unfinished action when the error
> occurred, not just stop.
> 
> -- 
> --
> Best Regards,
> John Xie
> --
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



| Mr Guillaume Seren
|
| website: http://www.guillaumeseren.com/
| github: https://github.com/GuillaumeSeren

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] Add git-grep threads param

[Junio C Hamano]

Victor Leschuk  writes:

> do we have any objections on this patch?

The question you should be asking is "do we have any support".

It is not like the default for any series is to be included; it is
quite the opposite.  "Is this worth having in our tree?" is the
question we all ask ourselves.

Also I think you should have CC'ed those who gave review comments on
this topic in the earlier rounds, if you wanted to receive answers
to that question (whichever one it is ;-).

Having said that, I didn't immediately spot anything objectionable.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: When a file was locked by some program, git will work stupidly

[Randall S. Becker]

On November-01-15 11:57 PM dayong xie wrote:
>To be specific
>In my Unity project, there is a native plugin,  and plugin's extension is 
>.dll, >and this plugin is under git version control, when Unity is running, 
>the plugin >file will be locked.
>If i merge another branch, which contains modification of the plugin, git will 
>>report error, looks
>like:
>error: unable to unlink old 'xxx/xxx.dll' (Permission denied) This is not 
>>bad, however, the unfinished merge action will not revert by git, a lot of 
>>changes produced in repository.
>usually it makes me crazy, even worse, some of my partners are not good at 
>>using git.
>Of course, this problem can be avoided by quit Unity, but not every time we 
>can >remember. In my opinion, git should revert the unfinished action when the 
>error >occurred, not just stop.

What version of Unity (or Unity Pro) are you using? Is this experienced with 
the Commit in MonoDevelop/Visual Studio or are you using a separate git client? 

I have found similar issues in some versions of Unity and MonoDevelop or Visual 
Studio not saving all files, especially the project files until you have fully 
exited - nothing to do with git, but your git commits may not contain complete 
images of your change.

When I use git with Unity, I either have the source committed through 
MonoDevelop (no issues) if my changes are source-only, or if I have assets and 
project changes, then I do exit completely so that I am sure Unity flushes 
everything to disk and I can get a single atomic commit with all the Unity and 
C# bits using SourceTree or gitk.

OTOH I'm not sure you really should be storing build products out of 
MonoDevelop or Visual Studio in your git repository. If the DLL can be rebuild 
automatically on the client - usual answer is yes - then let it. Handle the 
release build separately - at least in a separate branch that does not involve 
having the Unity editor open to get in the way.

In my environment, I have added *.dll (and other stuff) to .gitignore so that I 
do not track dll changes - they get built on demand instead. There are 
recommended ways of using git in the Unity forums.

Cheers,
Randall

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Anonymous commit (empty Author and Committer)

[Lennart Sorensen]

On Sat, Oct 31, 2015 at 08:24:06AM +0300, Andrei Borzenkov wrote:
> If we decide to fix this commit it is better done now, while it is the last
> one. It is annoying but do you have suggestion how it can be done
> differently?

Well the only options are:

1) Leave it alone
2) Break the tree of anyone that already pulled.  (Posting instructions on
how to fix it would at least be nice then).

Not sure how many people regularly pull the tree.

-- 
Len Sorensen
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 25/26] refs: break out ref conflict checks

[Michael Haggerty]

On 10/28/2015 03:14 AM, David Turner wrote:
> Create new function verify_no_descendants, to hold one of the ref
> conflict checks used in verify_refname_available.  Multiple backends
> will need this function, so it goes in the common code.
> 
> rename_ref_available also moves to the common code, because alternate
> backends might need it and it has no files-backend-specific code.

It is preferred that log messages be written in the imperative voice:

... so move it to the common code.

Also move rename_ref_available ...

> Signed-off-by: David Turner 
> Signed-off-by: Junio C Hamano 
> Signed-off-by: Michael Haggerty 
> ---
>  refs-be-files.c | 49 -
>  refs.c  | 38 ++
>  refs.h  | 15 +++
>  3 files changed, 61 insertions(+), 41 deletions(-)
> 
> diff --git a/refs-be-files.c b/refs-be-files.c
> index 0eabea9..97a5983 100644
> --- a/refs-be-files.c
> +++ b/refs-be-files.c
> @@ -729,6 +729,7 @@ static int verify_refname_available_dir(const char 
> *refname,
>   struct strbuf *err)
>  {
>   const char *slash;
> + const char *extra_refname;
>   int pos;
>   struct strbuf dirname = STRBUF_INIT;
>   int ret = -1;
> @@ -834,33 +835,15 @@ static int verify_refname_available_dir(const char 
> *refname,
>   }
>   }
>  
> - if (extras) {
> - /*
> -  * Check for entries in extras that start with
> -  * "$refname/". We do that by looking for the place
> -  * where "$refname/" would be inserted in extras. If
> -  * there is an entry at that position that starts with
> -  * "$refname/" and is not in skip, then we have a
> -  * conflict.
> -  */

In your version, the above comment is replaced with

/* Look for the place where dirname would be inserted in extras. */

which is pretty obvious, I think, from the following line. It's
debatable whether the long-winded old comment was worthwhile, but I
think the new comment is clearly not. I suggest either restoring a
longer explanation or eliminating the comment altogether.

> - for (pos = string_list_find_insert_index(extras, dirname.buf, 
> 0);
> -  pos < extras->nr; pos++) {
> - const char *extra_refname = extras->items[pos].string;
> -
> - if (!starts_with(extra_refname, dirname.buf))
> - break;
> -
> - if (!skip || !string_list_has_string(skip, 
> extra_refname)) {
> - strbuf_addf(err, "cannot process '%s' and '%s' 
> at the same time",
> - refname, extra_refname);
> - goto cleanup;
> - }
> - }
> + extra_refname = find_descendant_ref(dirname.buf, extras, skip);
> + if (extra_refname) {
> + strbuf_addf(err,
> + "cannot process '%s' and '%s' at the same time",
> + refname, extra_refname);
> + } else {
> + ret = 0;
>   }

Project practice is to leave out unnecessary braces.

>  
> - /* No conflicts were found */
> - ret = 0;
> -
>  cleanup:
>   strbuf_release();
>   return ret;
> @@ -2460,22 +2443,6 @@ out:
> [...]
> diff --git a/refs.c b/refs.c
> index 056c172..5d8b6ea 100644
> --- a/refs.c
> +++ b/refs.c
> [...]
> diff --git a/refs.h b/refs.h
> index f97a2e4..88fea3e 100644
> --- a/refs.h
> +++ b/refs.h
> [...]
> @@ -623,6 +625,19 @@ int files_log_ref_write(const char *refname, const 
> unsigned char *old_sha1,
>   const unsigned char *new_sha1, const char *msg,
>   int flags, struct strbuf *err);
>  
> +/*
> + * Check for entries in extras that are within the specified
> + * directory, where dirname is a reference directory name including
> + * the trailing slash (e.g., "refs/heads/master/"). Ignore any
> + * conflicting references that are found in skip. If there is a
> + * conflicting reference, return its name.
> + *
> + * extras and skip must be sorted lists of reference names. skip can
> + * be NULL; extras cannot.

The last sentence is incorrect; the function can handle extras == NULL.

> + */
> +const char *find_descendant_ref(const char *dirname,
> + const struct string_list *extras,
> + const struct string_list *skip);
>  
>  enum expire_reflog_flags {
>   EXPIRE_REFLOGS_DRY_RUN = 1 << 0,
> 

Michael

-- 
Michael Haggerty
mhag...@alum.mit.edu

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git log --author=me

[Junio C Hamano]

Andreas Schwab  writes:

> Harry Jeffery  writes:
>
>> The purpose being to make finding your own commits quicker and easier:
>> git log --author=me
>
> Since --author does a regexp search, this would most likely break
> someone's searches.  Better add a new option for that functionality.

I do not think it would hurt too much to add "--mine" to look for
"--author" that matches you, but on the other hand, I am not sure
its value outweighs the additional noise in the manual page.

It certainly does not help "I have done this much to this project in
the time period, and you can look at the output of this command
yourself to verify my claim.  Type 'git shortlog --mine' and then
promote me this season, please!" use case ;-)

So...
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

announcing git-svnsync (server-side GIT↔SVN sync)

[Anthony Baire]

Hi,
today we are announcing the first release of git-svnsync

http://git-svnsync.gforge.inria.fr/

Git-svnsync is a bi-directional server-side synchronisation tool between 
a git and a subversion repository.


It is based on hooks and it is designed to allow a smooth transition of 
projects from a subversion repository to a git repository. Git-svnsync 
guarantees that any branch update ('svn commit' or 'git push') is 
applied atomically in both repositories, thus providing a seamless 
experience to the developers.



At the moment the tool is minimalist (syncs only one branch), but quite 
useful in projects where everybody commits in trunk and it is 100% free. 
We have used it internally since 2013 without any major issue and hope 
it will be helpful for the community.


Best Regards
--
Anthony Baire
IRISA/University of Rennes 1
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git log --author=me

[Duy Nguyen]

On Mon, Nov 2, 2015 at 2:27 PM, Harry Jeffery  wrote:
> Hi,
>
> I've written a patch that allows `me` to be used as shorthand for
> $(user.name) or $(user.email) in the `--author` and `--commiter` fields.
>
> The purpose being to make finding your own commits quicker and easier:
> git log --author=me
>

It would be even cooler if it accepts mail aliases, then you can
define "me" to your address and also have shortcuts to a few of your
best friends. Though as Andreas pointed out --author is not a good fit
because it accepts regex, and you can't use --mine either, so more
work..
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git filter-branch and merging of signed tags

[Michael J Gruber]

Uwe Kleine-König venit, vidit, dixit 02.11.2015 09:04:
> Hello,
> 
> Consider I want to rewrite a commit that is a merge of a signed tag. In
> my case that's 064ca93f7ab927c2752d86fc5727716e340d737c that currently
> sits in linux-next:
> 
>   ~/gsrc/linux$ git version
>   git version 2.6.1
>   ~/gsrc/linux$ git checkout 064ca93f7ab927c2752d86fc5727716e340d737c
>   ~/gsrc/linux$ git cat-file commit HEAD
>   tree 7086bbd51d06ebd0e3fcca70380d0a066efce6ef
>   parent e44a01fa0e0aec2f3a21dadb9c030d14831d4258
>   parent f719a0d6a8541b383c506aaa4b4fa6b4109669f4
>   author Arnd Bergmann  1444139956 +0200
>   committer Arnd Bergmann  1444139956 +0200
>   mergetag object f719a0d6a8541b383c506aaa4b4fa6b4109669f4
>type commit
>tag efm32-for-4.4-rc1
>tagger Uwe Kleine-König  1443549594 +0200
>
>efm32 cleanups for 4.4-rc1
>
>These are just two followup cleanups for commits that are in v3.17-rc1 
> and
>waited in my private tree for application since that time.
>-BEGIN PGP SIGNATURE-
>
>iQEcBAABCgAGBQJWCtIwAAoJEMH8FHityuwJFPYH/2LJpi0yUhI01s0D9U5rZgjl
>8mLP70utAMaqXiGqP/O0ndH+za5TrrUTwDJE5jhAHru1Q6q62Ld2BfOR5+K8zlrP
>4UDGBDj3ePJgEtbSsJYJIRT+AKque2GONbeGpJh2arPrNamWChKIz8HoGSgwdRlx
>TvQuVFEa6C3ujP0ATM8x54f0vR4bYm9WmBS522DwZQY4JpGZC005A3Quw6fmFxPJ
>vJRo1fE1d2OZ3Cuxb15m8PLtZw6sbMdMwgyirAFGrJsN9u/V6BzsSzH0s8suT7IP
>WIohnmYtFSUfkQjaGrMl2wIpJSsIzettiXf9GrQDXSGZsQ4k7sW3U+qrcZ7ZUR4=
>=ngv1
>-END PGP SIGNATURE-
> 
>   Merge tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux 
> into next/cleanup
> 
>   Merge "efm32 cleanups for 4.4-rc1" from Uwe Kleine-Koenig:
> 
>   These are just two followup cleanups for commits that are in v3.17-rc1 
> and
>   waited in my private tree for application since that time.
> 
>   * tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux:
> ARM: efm32: switch to vendor,device compatible strings
> ARM: efm32: switch to properly namespaced location property
> 
> OK, to fix my name in the commit log I do:
> 
>git filter-branch -f --msg-filter "sed 's/Kleine-Koenig/$(echo 
> Kleine-K+APY-nig | iconv -f utf-7 -t utf-8)/'" HEAD ^HEAD^ ^HEAD^2
> 
> This destroys the mergetag, as now I have:
> 
>   ~/gsrc/linux$ git cat-file commit HEAD
>   tree 7086bbd51d06ebd0e3fcca70380d0a066efce6ef
>   parent e44a01fa0e0aec2f3a21dadb9c030d14831d4258
>   parent f719a0d6a8541b383c506aaa4b4fa6b4109669f4
>   author Arnd Bergmann  1444139956 +0200
>   committer Arnd Bergmann  1444139956 +0200
> 
>efm32 cleanups for 4.4-rc1
>
>These are just two followup cleanups for commits that are in v3.17-rc1 
> and
>waited in my private tree for application since that time.
>-BEGIN PGP SIGNATURE-
>
>iQEcBAABCgAGBQJWCtIwAAoJEMH8FHityuwJFPYH/2LJpi0yUhI01s0D9U5rZgjl
>8mLP70utAMaqXiGqP/O0ndH+za5TrrUTwDJE5jhAHru1Q6q62Ld2BfOR5+K8zlrP
>4UDGBDj3ePJgEtbSsJYJIRT+AKque2GONbeGpJh2arPrNamWChKIz8HoGSgwdRlx
>TvQuVFEa6C3ujP0ATM8x54f0vR4bYm9WmBS522DwZQY4JpGZC005A3Quw6fmFxPJ
>vJRo1fE1d2OZ3Cuxb15m8PLtZw6sbMdMwgyirAFGrJsN9u/V6BzsSzH0s8suT7IP
>WIohnmYtFSUfkQjaGrMl2wIpJSsIzettiXf9GrQDXSGZsQ4k7sW3U+qrcZ7ZUR4=
>=ngv1
>-END PGP SIGNATURE-
> 
>   Merge tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux 
> into next/cleanup
> 
>   Merge "efm32 cleanups for 4.4-rc1" from Uwe Kleine-König:
> 
>   These are just two followup cleanups for commits that are in v3.17-rc1 
> and
>   waited in my private tree for application since that time.
> 
>   * tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux:
> ARM: efm32: switch to vendor,device compatible strings
> ARM: efm32: switch to properly namespaced location property
> 
>   ~/gsrc/linux$ diff -u <(git cat-file commit HEAD) <(git cat-file commit 
> 064ca93f7ab927c2752d86fc5727716e340d737c)
>   --- /dev/fd/63  2015-11-02 09:02:17.310373539 +0100
>   +++ /dev/fd/62  2015-11-02 09:02:17.310373539 +0100
>   @@ -3,7 +3,11 @@
>parent f719a0d6a8541b383c506aaa4b4fa6b4109669f4
>author Arnd Bergmann  1444139956 +0200
>committer Arnd Bergmann  1444139956 +0200
>   -
>   +mergetag object f719a0d6a8541b383c506aaa4b4fa6b4109669f4
>   + type commit
>   + tag efm32-for-4.4-rc1
>   + tagger Uwe Kleine-König  1443549594 +0200
>   + 
> efm32 cleanups for 4.4-rc1
> 
> These are just two followup cleanups for commits that are in 
> v3.17-rc1 and
>   @@ -21,7 +25,7 @@
>
>Merge tag 'efm32-for-4.4-rc1' of 
> 

Re: Anonymous commit (empty Author and Committer)

[Junio C Hamano]

"Lennart Sorensen"  writes:

> On Sat, Oct 31, 2015 at 08:24:06AM +0300, Andrei Borzenkov wrote:
>> If we decide to fix this commit it is better done now, while it is the last
>> one. It is annoying but do you have suggestion how it can be done
>> differently?
>
> Well the only options are:
>
> 1) Leave it alone
> 2) Break the tree of anyone that already pulled.  (Posting instructions on
> how to fix it would at least be nice then).
>
> Not sure how many people regularly pull the tree.

Is the discussion still require attention by Git folks for their
help?  If not, could you take this back to savannah specific mailing
list?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git log --author=me

[Andreas Schwab]

Harry Jeffery  writes:

> The purpose being to make finding your own commits quicker and easier:
> git log --author=me

Since --author does a regexp search, this would most likely break
someone's searches.  Better add a new option for that functionality.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git log --author=me

[Matthieu Moy]

Junio C Hamano  writes:

> Andreas Schwab  writes:
>
>> Harry Jeffery  writes:
>>
>>> The purpose being to make finding your own commits quicker and easier:
>>> git log --author=me
>>
>> Since --author does a regexp search, this would most likely break
>> someone's searches.  Better add a new option for that functionality.
>
> I do not think it would hurt too much to add "--mine" to look for
> "--author" that matches you, but on the other hand, I am not sure
> its value outweighs the additional noise in the manual page.

I have a few aliases to congratulate myself for being such an awesome
contributor from time to time ;-):

[alias]
mymaster = log --oneline --author=matthieu@imag.fr origin/master

It did require me to write these lines in my config file, but I think
that the trouble of writing them is smaller than the extra noise in the
manual page and possibly the output of `git log -h`. Having the
shorthand "--author=me" would be better than "git log --mine" as far as
cluttering the doc is concerned IMHO, but it's also likely to break
someone's search.

BTW, the output of `git log -h` is disappointing:

$ git log -h
usage: git log [] [] [[--] ...]
   or: git show [] ...

-q, --quiet   suppress diff output
--source  show source
--use-mailmap Use mail map file
--decorate[=...]  decorate options
-L  Process line range n,m in file, counting from 1

There are far more options than this...

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 07/25] refs: move the hidden refs functions to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

Move the hidden refs functions to refs/refs.c since these functions do
not contain any backend specific code.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 51 ---
 refs/refs.c  | 51 +++
 2 files changed, 51 insertions(+), 51 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 8eba347..8f1e486 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -4219,57 +4219,6 @@ char *shorten_unambiguous_ref(const char *refname, int 
strict)
return xstrdup(refname);
 }
 
-static struct string_list *hide_refs;
-
-int parse_hide_refs_config(const char *var, const char *value, const char 
*section)
-{
-   if (!strcmp("transfer.hiderefs", var) ||
-   /* NEEDSWORK: use parse_config_key() once both are merged */
-   (starts_with(var, section) && var[strlen(section)] == '.' &&
-!strcmp(var + strlen(section), ".hiderefs"))) {
-   char *ref;
-   int len;
-
-   if (!value)
-   return config_error_nonbool(var);
-   ref = xstrdup(value);
-   len = strlen(ref);
-   while (len && ref[len - 1] == '/')
-   ref[--len] = '\0';
-   if (!hide_refs) {
-   hide_refs = xcalloc(1, sizeof(*hide_refs));
-   hide_refs->strdup_strings = 1;
-   }
-   string_list_append(hide_refs, ref);
-   }
-   return 0;
-}
-
-int ref_is_hidden(const char *refname)
-{
-   int i;
-
-   if (!hide_refs)
-   return 0;
-   for (i = hide_refs->nr - 1; i >= 0; i--) {
-   const char *match = hide_refs->items[i].string;
-   int neg = 0;
-   int len;
-
-   if (*match == '!') {
-   neg = 1;
-   match++;
-   }
-
-   if (!starts_with(refname, match))
-   continue;
-   len = strlen(match);
-   if (!refname[len] || refname[len] == '/')
-   return !neg;
-   }
-   return 0;
-}
-
 struct expire_reflog_cb {
unsigned int flags;
reflog_expiry_should_prune_fn *should_prune_fn;
diff --git a/refs/refs.c b/refs/refs.c
index 99221a3..0620ba3 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -291,3 +291,54 @@ int read_ref_at(const char *refname, unsigned int flags, 
unsigned long at_time,
 
return 1;
 }
+
+static struct string_list *hide_refs;
+
+int parse_hide_refs_config(const char *var, const char *value, const char 
*section)
+{
+   if (!strcmp("transfer.hiderefs", var) ||
+   /* NEEDSWORK: use parse_config_key() once both are merged */
+   (starts_with(var, section) && var[strlen(section)] == '.' &&
+!strcmp(var + strlen(section), ".hiderefs"))) {
+   char *ref;
+   int len;
+
+   if (!value)
+   return config_error_nonbool(var);
+   ref = xstrdup(value);
+   len = strlen(ref);
+   while (len && ref[len - 1] == '/')
+   ref[--len] = '\0';
+   if (!hide_refs) {
+   hide_refs = xcalloc(1, sizeof(*hide_refs));
+   hide_refs->strdup_strings = 1;
+   }
+   string_list_append(hide_refs, ref);
+   }
+   return 0;
+}
+
+int ref_is_hidden(const char *refname)
+{
+   int i;
+
+   if (!hide_refs)
+   return 0;
+   for (i = hide_refs->nr - 1; i >= 0; i--) {
+   const char *match = hide_refs->items[i].string;
+   int neg = 0;
+   int len;
+
+   if (*match == '!') {
+   neg = 1;
+   match++;
+   }
+
+   if (!starts_with(refname, match))
+   continue;
+   len = strlen(match);
+   if (!refname[len] || refname[len] == '/')
+   return !neg;
+   }
+   return 0;
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 22/25] initdb: make safe_create_dir public

[Michael Haggerty]

From: David Turner 

Soon we will want to create initdb functions for ref backends, and
code from initdb that calls this function needs to move into the files
backend. So this function needs to be public.

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 builtin/init-db.c | 12 
 cache.h   |  8 
 path.c| 12 
 3 files changed, 20 insertions(+), 12 deletions(-)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index f59f407..07229d6 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -24,18 +24,6 @@ static int init_shared_repository = -1;
 static const char *init_db_template_dir;
 static const char *git_link;
 
-static void safe_create_dir(const char *dir, int share)
-{
-   if (mkdir(dir, 0777) < 0) {
-   if (errno != EEXIST) {
-   perror(dir);
-   exit(1);
-   }
-   }
-   else if (share && adjust_shared_perm(dir))
-   die(_("Could not make %s writable by group"), dir);
-}
-
 static void copy_templates_1(struct strbuf *path, struct strbuf *template,
 DIR *dir)
 {
diff --git a/cache.h b/cache.h
index 9a905a8..0f9808c 100644
--- a/cache.h
+++ b/cache.h
@@ -1737,4 +1737,12 @@ void stat_validity_update(struct stat_validity *sv, int 
fd);
 int versioncmp(const char *s1, const char *s2);
 void sleep_millisec(int millisec);
 
+/*
+ * Create a directory and (if share is nonzero) adjust its permissions
+ * according to the shared_repository setting. Only use this for
+ * directories under $GIT_DIR.  Don't use it for working tree
+ * directories.
+ */
+void safe_create_dir(const char *dir, int share);
+
 #endif /* CACHE_H */
diff --git a/path.c b/path.c
index 212695a..9e0283c 100644
--- a/path.c
+++ b/path.c
@@ -723,6 +723,18 @@ int adjust_shared_perm(const char *path)
return 0;
 }
 
+void safe_create_dir(const char *dir, int share)
+{
+   if (mkdir(dir, 0777) < 0) {
+   if (errno != EEXIST) {
+   perror(dir);
+   exit(1);
+   }
+   }
+   else if (share && adjust_shared_perm(dir))
+   die(_("Could not make %s writable by group"), dir);
+}
+
 static int have_same_root(const char *path1, const char *path2)
 {
int is_abs1, is_abs2;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 17/25] refs: move transaction functions to the common code

[Michael Haggerty]

From: David Turner 

The common ref code will build up a ref transaction. Backends will
then commit it. So the transaction creation and update functions
should be in the common code.

The backends will also need access to some constants and structs that,
until now, were defined within refs.c and treated as private. So move
those into a new header file, refs/refs-internal.h, that is intended
for internal use only within the refs module.

Later, we will modify struct ref_update to support alternate backends.

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 200 +--
 refs/refs-internal.h | 102 ++
 refs/refs.c  | 110 +++-
 3 files changed, 212 insertions(+), 200 deletions(-)
 create mode 100644 refs/refs-internal.h

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 728e646..1719720 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1,5 +1,5 @@
 #include "../cache.h"
-#include "../refs.h"
+#include "refs-internal.h"
 #include "../lockfile.h"
 #include "../object.h"
 #include "../tag.h"
@@ -13,41 +13,6 @@ struct ref_lock {
struct object_id old_oid;
 };
 
-/*
- * Flag passed to lock_ref_sha1_basic() telling it to tolerate broken
- * refs (i.e., because the reference is about to be deleted anyway).
- */
-#define REF_DELETING   0x02
-
-/*
- * Used as a flag in ref_update::flags when a loose ref is being
- * pruned.
- */
-#define REF_ISPRUNING  0x04
-
-/*
- * Used as a flag in ref_update::flags when the reference should be
- * updated to new_sha1.
- */
-#define REF_HAVE_NEW   0x08
-
-/*
- * Used as a flag in ref_update::flags when old_sha1 should be
- * checked.
- */
-#define REF_HAVE_OLD   0x10
-
-/*
- * Used as a flag in ref_update::flags when the lockfile needs to be
- * committed.
- */
-#define REF_NEEDS_COMMIT 0x20
-
-/*
- * 0x40 is REF_FORCE_CREATE_REFLOG, so skip it if you're adding a
- * value to ref_update::flags
- */
-
 struct ref_entry;
 
 /*
@@ -3289,169 +3254,6 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
return retval;
 }
 
-/**
- * Information needed for a single ref update. Set new_sha1 to the new
- * value or to null_sha1 to delete the ref. To check the old value
- * while the ref is locked, set (flags & REF_HAVE_OLD) and set
- * old_sha1 to the old value, or to null_sha1 to ensure the ref does
- * not exist before update.
- */
-struct ref_update {
-   /*
-* If (flags & REF_HAVE_NEW), set the reference to this value:
-*/
-   unsigned char new_sha1[20];
-   /*
-* If (flags & REF_HAVE_OLD), check that the reference
-* previously had this value:
-*/
-   unsigned char old_sha1[20];
-   /*
-* One or more of REF_HAVE_NEW, REF_HAVE_OLD, REF_NODEREF,
-* REF_DELETING, and REF_ISPRUNING:
-*/
-   unsigned int flags;
-   struct ref_lock *lock;
-   int type;
-   char *msg;
-   const char refname[FLEX_ARRAY];
-};
-
-/*
- * Transaction states.
- * OPEN:   The transaction is in a valid state and can accept new updates.
- * An OPEN transaction can be committed.
- * CLOSED: A closed transaction is no longer active and no other operations
- * than free can be used on it in this state.
- * A transaction can either become closed by successfully committing
- * an active transaction or if there is a failure while building
- * the transaction thus rendering it failed/inactive.
- */
-enum ref_transaction_state {
-   REF_TRANSACTION_OPEN   = 0,
-   REF_TRANSACTION_CLOSED = 1
-};
-
-/*
- * Data structure for holding a reference transaction, which can
- * consist of checks and updates to multiple references, carried out
- * as atomically as possible.  This structure is opaque to callers.
- */
-struct ref_transaction {
-   struct ref_update **updates;
-   size_t alloc;
-   size_t nr;
-   enum ref_transaction_state state;
-};
-
-struct ref_transaction *ref_transaction_begin(struct strbuf *err)
-{
-   assert(err);
-
-   return xcalloc(1, sizeof(struct ref_transaction));
-}
-
-void ref_transaction_free(struct ref_transaction *transaction)
-{
-   int i;
-
-   if (!transaction)
-   return;
-
-   for (i = 0; i < transaction->nr; i++) {
-   free(transaction->updates[i]->msg);
-   free(transaction->updates[i]);
-   }
-   free(transaction->updates);
-   free(transaction);
-}
-
-static struct ref_update *add_update(struct ref_transaction *transaction,
-const char *refname)
-{
-   size_t len = strlen(refname) + 1;
-   struct ref_update *update = xcalloc(1, sizeof(*update) + len);
-
-   memcpy((char *)update->refname, refname, len); /* includes 

[PATCH v6 10/25] refs: move read_ref, read_ref_full and ref_exists to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

These functions do not depend on the backend implementation so move
them to the common code.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 18 --
 refs/refs.c  | 18 ++
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 9fb77c2..6940d54 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1803,24 +1803,6 @@ struct ref_filter {
void *cb_data;
 };
 
-int read_ref_full(const char *refname, int resolve_flags, unsigned char *sha1, 
int *flags)
-{
-   if (resolve_ref_unsafe(refname, resolve_flags, sha1, flags))
-   return 0;
-   return -1;
-}
-
-int read_ref(const char *refname, unsigned char *sha1)
-{
-   return read_ref_full(refname, RESOLVE_REF_READING, sha1, NULL);
-}
-
-int ref_exists(const char *refname)
-{
-   unsigned char sha1[20];
-   return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
-}
-
 static int filter_refs(const char *refname, const struct object_id *oid,
   int flags, void *data)
 {
diff --git a/refs/refs.c b/refs/refs.c
index 6d9ea7c..ae55d85 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -597,3 +597,21 @@ void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, 
const struct string_li
data.msg_fmt = msg_fmt;
for_each_rawref(warn_if_dangling_symref, );
 }
+
+int read_ref_full(const char *refname, int resolve_flags, unsigned char *sha1, 
int *flags)
+{
+   if (resolve_ref_unsafe(refname, resolve_flags, sha1, flags))
+   return 0;
+   return -1;
+}
+
+int read_ref(const char *refname, unsigned char *sha1)
+{
+   return read_ref_full(refname, RESOLVE_REF_READING, sha1, NULL);
+}
+
+int ref_exists(const char *refname)
+{
+   unsigned char sha1[20];
+   return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 05/25] refs: move delete_pseudoref and delete_ref to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

Move delete_pseudoref() and delete_ref() to refs/refs.c since these
functions do not contain any backend specific code. We can't move
delete_refs yet because it depends on the files-backend-specific
repack_without_refs.

Signed-off-by: David Turner 
Signed-off-by: Ronnie Sahlberg 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 56 
 refs/refs.c  | 56 
 2 files changed, 56 insertions(+), 56 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e8c592a..6d502ac 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2892,62 +2892,6 @@ static int delete_ref_loose(struct ref_lock *lock, int 
flag, struct strbuf *err)
return 0;
 }
 
-static int delete_pseudoref(const char *pseudoref, const unsigned char 
*old_sha1)
-{
-   static struct lock_file lock;
-   const char *filename;
-
-   filename = git_path("%s", pseudoref);
-
-   if (old_sha1 && !is_null_sha1(old_sha1)) {
-   int fd;
-   unsigned char actual_old_sha1[20];
-
-   fd = hold_lock_file_for_update(, filename,
-  LOCK_DIE_ON_ERROR);
-   if (fd < 0)
-   die_errno(_("Could not open '%s' for writing"), 
filename);
-   if (read_ref(pseudoref, actual_old_sha1))
-   die("could not read ref '%s'", pseudoref);
-   if (hashcmp(actual_old_sha1, old_sha1)) {
-   warning("Unexpected sha1 when deleting %s", pseudoref);
-   rollback_lock_file();
-   return -1;
-   }
-
-   unlink(filename);
-   rollback_lock_file();
-   } else {
-   unlink(filename);
-   }
-
-   return 0;
-}
-
-int delete_ref(const char *refname, const unsigned char *old_sha1,
-  unsigned int flags)
-{
-   struct ref_transaction *transaction;
-   struct strbuf err = STRBUF_INIT;
-
-   if (ref_type(refname) == REF_TYPE_PSEUDOREF)
-   return delete_pseudoref(refname, old_sha1);
-
-   transaction = ref_transaction_begin();
-   if (!transaction ||
-   ref_transaction_delete(transaction, refname, old_sha1,
-  flags, NULL, ) ||
-   ref_transaction_commit(transaction, )) {
-   error("%s", err.buf);
-   ref_transaction_free(transaction);
-   strbuf_release();
-   return 1;
-   }
-   ref_transaction_free(transaction);
-   strbuf_release();
-   return 0;
-}
-
 int delete_refs(struct string_list *refnames)
 {
struct strbuf err = STRBUF_INIT;
diff --git a/refs/refs.c b/refs/refs.c
index a63a58d..96d5c6d 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -117,3 +117,59 @@ int update_ref(const char *msg, const char *refname,
ref_transaction_free(t);
return 0;
 }
+
+static int delete_pseudoref(const char *pseudoref, const unsigned char 
*old_sha1)
+{
+   static struct lock_file lock;
+   const char *filename;
+
+   filename = git_path("%s", pseudoref);
+
+   if (old_sha1 && !is_null_sha1(old_sha1)) {
+   int fd;
+   unsigned char actual_old_sha1[20];
+
+   fd = hold_lock_file_for_update(, filename,
+  LOCK_DIE_ON_ERROR);
+   if (fd < 0)
+   die_errno(_("Could not open '%s' for writing"), 
filename);
+   if (read_ref(pseudoref, actual_old_sha1))
+   die("could not read ref '%s'", pseudoref);
+   if (hashcmp(actual_old_sha1, old_sha1)) {
+   warning("Unexpected sha1 when deleting %s", pseudoref);
+   rollback_lock_file();
+   return -1;
+   }
+
+   unlink(filename);
+   rollback_lock_file();
+   } else {
+   unlink(filename);
+   }
+
+   return 0;
+}
+
+int delete_ref(const char *refname, const unsigned char *old_sha1,
+  unsigned int flags)
+{
+   struct ref_transaction *transaction;
+   struct strbuf err = STRBUF_INIT;
+
+   if (ref_type(refname) == REF_TYPE_PSEUDOREF)
+   return delete_pseudoref(refname, old_sha1);
+
+   transaction = ref_transaction_begin();
+   if (!transaction ||
+   ref_transaction_delete(transaction, refname, old_sha1,
+  flags, NULL, ) ||
+   ref_transaction_commit(transaction, )) {
+   error("%s", err.buf);
+   ref_transaction_free(transaction);
+   strbuf_release();
+   return 1;

[PATCH v6 09/25] refs: move warn_if_dangling_symref* to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

These functions do not use any backend specific code so move them to
the common code.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 52 
 refs/refs.c  | 52 
 2 files changed, 52 insertions(+), 52 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 8809595..9fb77c2 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1956,58 +1956,6 @@ int peel_ref(const char *refname, unsigned char *sha1)
return peel_object(base, sha1);
 }
 
-struct warn_if_dangling_data {
-   FILE *fp;
-   const char *refname;
-   const struct string_list *refnames;
-   const char *msg_fmt;
-};
-
-static int warn_if_dangling_symref(const char *refname, const struct object_id 
*oid,
-  int flags, void *cb_data)
-{
-   struct warn_if_dangling_data *d = cb_data;
-   const char *resolves_to;
-   struct object_id junk;
-
-   if (!(flags & REF_ISSYMREF))
-   return 0;
-
-   resolves_to = resolve_ref_unsafe(refname, 0, junk.hash, NULL);
-   if (!resolves_to
-   || (d->refname
-   ? strcmp(resolves_to, d->refname)
-   : !string_list_has_string(d->refnames, resolves_to))) {
-   return 0;
-   }
-
-   fprintf(d->fp, d->msg_fmt, refname);
-   fputc('\n', d->fp);
-   return 0;
-}
-
-void warn_dangling_symref(FILE *fp, const char *msg_fmt, const char *refname)
-{
-   struct warn_if_dangling_data data;
-
-   data.fp = fp;
-   data.refname = refname;
-   data.refnames = NULL;
-   data.msg_fmt = msg_fmt;
-   for_each_rawref(warn_if_dangling_symref, );
-}
-
-void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct 
string_list *refnames)
-{
-   struct warn_if_dangling_data data;
-
-   data.fp = fp;
-   data.refname = NULL;
-   data.refnames = refnames;
-   data.msg_fmt = msg_fmt;
-   for_each_rawref(warn_if_dangling_symref, );
-}
-
 /*
  * Call fn for each reference in the specified ref_cache, omitting
  * references not in the containing_dir of base.  fn is called for all
diff --git a/refs/refs.c b/refs/refs.c
index 40d7831..6d9ea7c 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -545,3 +545,55 @@ char *shorten_unambiguous_ref(const char *refname, int 
strict)
free(short_name);
return xstrdup(refname);
 }
+
+struct warn_if_dangling_data {
+   FILE *fp;
+   const char *refname;
+   const struct string_list *refnames;
+   const char *msg_fmt;
+};
+
+static int warn_if_dangling_symref(const char *refname, const struct object_id 
*oid,
+  int flags, void *cb_data)
+{
+   struct warn_if_dangling_data *d = cb_data;
+   const char *resolves_to;
+   struct object_id junk;
+
+   if (!(flags & REF_ISSYMREF))
+   return 0;
+
+   resolves_to = resolve_ref_unsafe(refname, 0, junk.hash, NULL);
+   if (!resolves_to
+   || (d->refname
+   ? strcmp(resolves_to, d->refname)
+   : !string_list_has_string(d->refnames, resolves_to))) {
+   return 0;
+   }
+
+   fprintf(d->fp, d->msg_fmt, refname);
+   fputc('\n', d->fp);
+   return 0;
+}
+
+void warn_dangling_symref(FILE *fp, const char *msg_fmt, const char *refname)
+{
+   struct warn_if_dangling_data data;
+
+   data.fp = fp;
+   data.refname = refname;
+   data.refnames = NULL;
+   data.msg_fmt = msg_fmt;
+   for_each_rawref(warn_if_dangling_symref, );
+}
+
+void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct 
string_list *refnames)
+{
+   struct warn_if_dangling_data data;
+
+   data.fp = fp;
+   data.refname = NULL;
+   data.refnames = refnames;
+   data.msg_fmt = msg_fmt;
+   for_each_rawref(warn_if_dangling_symref, );
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 23/25] files_log_ref_write: new function

[Michael Haggerty]

From: David Turner 

Because HEAD and stash are per-worktree, every refs backend needs to
go through the files backend to write these refs.

So create a new function, files_log_ref_write, and add it to
refs/refs-internal.h. Later, we will use this to handle reflog updates
for per-worktree symbolic refs (HEAD).

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 8 
 refs/refs-internal.h | 4 
 2 files changed, 12 insertions(+)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7dc68b8..20e2924 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2733,6 +2733,14 @@ static int log_ref_write(const char *refname, const 
unsigned char *old_sha1,
 const unsigned char *new_sha1, const char *msg,
 int flags, struct strbuf *err)
 {
+   return files_log_ref_write(refname, old_sha1, new_sha1, msg, flags,
+  err);
+}
+
+int files_log_ref_write(const char *refname, const unsigned char *old_sha1,
+   const unsigned char *new_sha1, const char *msg,
+   int flags, struct strbuf *err)
+{
struct strbuf sb = STRBUF_INIT;
int ret = log_ref_write_1(refname, old_sha1, new_sha1, msg, , flags,
  err);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index d854d9e..5f447d4 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -157,4 +157,8 @@ extern int should_autocreate_reflog(const char *refname);
  */
 int copy_reflog_msg(char *buf, const char *msg);
 
+int files_log_ref_write(const char *refname, const unsigned char *old_sha1,
+   const unsigned char *new_sha1, const char *msg,
+   int flags, struct strbuf *err);
+
 #endif /* REFS_REFS_INTERNAL_H */
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Watchman/inotify support and other ways to speed up git status

[Christian Couder]

On Tue, Nov 3, 2015 at 6:45 AM, Duy Nguyen  wrote:
> On Mon, Nov 2, 2015 at 9:56 PM, David Turner  wrote:
>> On Thu, 2015-10-29 at 09:10 +0100, Christian Couder wrote:
>>> > We're using Watchman at Twitter.  A week or two ago posted a dump of our
>>> > code to github, but I would advise waiting a day or two to use it, as
>>> > I'm about to pull a large number of bugfixes into it (I'll update this
>>> > thread and provide a link once I do so).
>>>
>>> Great, I will have a look at it then!
>>
>> Here's the most recent version:
>>
>> https://github.com/dturner-tw/git/tree/dturner/watchman
>
> Christian, the index-helper/watchman series are posted because you
> showed interest in this area. I'm not rerolling to address David's
> comments on the series for now.

Ok no problem. Thanks a lot to you and David for posting your rebased series!

> Take your time evaluate the two
> approaches, then you can pick one (and let me know if you want me to
> hand my series over, very glad to do so).

Yeah, I will do that, thanks again!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git.git as of tonight

[Johannes Sixt]

Am 03.11.2015 um 00:06 schrieb Stefan Beller:

On Mon, Nov 2, 2015 at 1:15 PM, Johannes Sixt  wrote:

run-command.c: In function 'set_nonblocking':
run-command.c:1011: error: 'F_GETFL' undeclared (first use in this function)
run-command.c:1011: error: (Each undeclared identifier is reported only once
run-command.c:1011: error: for each function it appears in.)
run-command.c:1015: error: 'F_SETFL' undeclared (first use in this function)
run-command.c:1015: error: 'O_NONBLOCK' undeclared (first use in this function)
make: *** [run-command.o] Error 1


Going by a quick search http://stackoverflow.com/a/22756664
I'd hope we only need to modify the set_nonblocking function using #ifdefs ?


Unfortunately, the solutions outlined in that post do not work for us. 
On Windows, the FIONBIO option is only available for sockets. 
"Overlapped IO" can be used only when the file descriptor is set in this 
mode right from the beginning (by open/CreateFile), and if it were so, 
it would have to be used in a totally different way than in Posix code.


My findings so far are negative. The only short-term and mid-term 
solution I see so far is to opt-out from the framework during build-time.


-- Hannes

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 04/25] refs: move update_ref to refs/refs.c

[Michael Haggerty]

From: Ronnie Sahlberg 

Move update_ref() to refs/refs.c since this function does not contain
any backend specific code. Move the ref classifier functions and
write_pseudoref as well, since update_ref depends on them.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 117 +--
 refs/refs.c  | 116 ++
 2 files changed, 117 insertions(+), 116 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index dc39b36..e8c592a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2683,8 +2683,6 @@ struct pack_refs_cb_data {
struct ref_to_prune *ref_to_prune;
 };
 
-static int is_per_worktree_ref(const char *refname);
-
 /*
  * An each_ref_entry_fn that is run over loose references only.  If
  * the loose reference can be packed, add an entry in the packed ref
@@ -2699,7 +2697,7 @@ static int pack_if_possible_fn(struct ref_entry *entry, 
void *cb_data)
int is_tag_ref = starts_with(entry->name, "refs/tags/");
 
/* Do not pack per-worktree refs: */
-   if (is_per_worktree_ref(entry->name))
+   if (ref_type(entry->name) == REF_TYPE_PER_WORKTREE)
return 0;
 
/* ALWAYS pack tags */
@@ -2894,77 +2892,6 @@ static int delete_ref_loose(struct ref_lock *lock, int 
flag, struct strbuf *err)
return 0;
 }
 
-static int is_per_worktree_ref(const char *refname)
-{
-   return !strcmp(refname, "HEAD") ||
-   starts_with(refname, "refs/bisect/");
-}
-
-static int is_pseudoref_syntax(const char *refname)
-{
-   const char *c;
-
-   for (c = refname; *c; c++) {
-   if (!isupper(*c) && *c != '-' && *c != '_')
-   return 0;
-   }
-
-   return 1;
-}
-
-enum ref_type ref_type(const char *refname)
-{
-   if (is_per_worktree_ref(refname))
-   return REF_TYPE_PER_WORKTREE;
-   if (is_pseudoref_syntax(refname))
-   return REF_TYPE_PSEUDOREF;
-   return REF_TYPE_NORMAL;
-}
-
-static int write_pseudoref(const char *pseudoref, const unsigned char *sha1,
-  const unsigned char *old_sha1, struct strbuf *err)
-{
-   const char *filename;
-   int fd;
-   static struct lock_file lock;
-   struct strbuf buf = STRBUF_INIT;
-   int ret = -1;
-
-   strbuf_addf(, "%s\n", sha1_to_hex(sha1));
-
-   filename = git_path("%s", pseudoref);
-   fd = hold_lock_file_for_update(, filename, LOCK_DIE_ON_ERROR);
-   if (fd < 0) {
-   strbuf_addf(err, "Could not open '%s' for writing: %s",
-   filename, strerror(errno));
-   return -1;
-   }
-
-   if (old_sha1) {
-   unsigned char actual_old_sha1[20];
-
-   if (read_ref(pseudoref, actual_old_sha1))
-   die("could not read ref '%s'", pseudoref);
-   if (hashcmp(actual_old_sha1, old_sha1)) {
-   strbuf_addf(err, "Unexpected sha1 when writing %s", 
pseudoref);
-   rollback_lock_file();
-   goto done;
-   }
-   }
-
-   if (write_in_full(fd, buf.buf, buf.len) != buf.len) {
-   strbuf_addf(err, "Could not write to '%s'", filename);
-   rollback_lock_file();
-   goto done;
-   }
-
-   commit_lock_file();
-   ret = 0;
-done:
-   strbuf_release();
-   return ret;
-}
-
 static int delete_pseudoref(const char *pseudoref, const unsigned char 
*old_sha1)
 {
static struct lock_file lock;
@@ -4105,48 +4032,6 @@ int ref_transaction_verify(struct ref_transaction 
*transaction,
  flags, NULL, err);
 }
 
-int update_ref(const char *msg, const char *refname,
-  const unsigned char *new_sha1, const unsigned char *old_sha1,
-  unsigned int flags, enum action_on_err onerr)
-{
-   struct ref_transaction *t = NULL;
-   struct strbuf err = STRBUF_INIT;
-   int ret = 0;
-
-   if (ref_type(refname) == REF_TYPE_PSEUDOREF) {
-   ret = write_pseudoref(refname, new_sha1, old_sha1, );
-   } else {
-   t = ref_transaction_begin();
-   if (!t ||
-   ref_transaction_update(t, refname, new_sha1, old_sha1,
-  flags, msg, ) ||
-   ref_transaction_commit(t, )) {
-   ret = 1;
-   ref_transaction_free(t);
-   }
-   }
-   if (ret) {
-   const char *str = "update_ref failed for ref '%s': %s";
-
-   switch (onerr) {
-   case UPDATE_REFS_MSG_ON_ERR:
-   

[PATCH v6 19/25] refs: move copy_msg to the common code

[Michael Haggerty]

From: David Turner 

Rename copy_msg to copy_reflog_msg and add it to refs/refs-internal.h.

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 28 +---
 refs/refs-internal.h |  7 +++
 refs/refs.c  | 21 +
 3 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3a4aff6..4807b32 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2665,32 +2665,6 @@ static int commit_ref(struct ref_lock *lock)
return 0;
 }
 
-/*
- * copy the reflog message msg to buf, which has been allocated sufficiently
- * large, while cleaning up the whitespaces.  Especially, convert LF to space,
- * because reflog file is one line per entry.
- */
-static int copy_msg(char *buf, const char *msg)
-{
-   char *cp = buf;
-   char c;
-   int wasspace = 1;
-
-   *cp++ = '\t';
-   while ((c = *msg++)) {
-   if (wasspace && isspace(c))
-   continue;
-   wasspace = isspace(c);
-   if (wasspace)
-   c = ' ';
-   *cp++ = c;
-   }
-   while (buf < cp && isspace(cp[-1]))
-   cp--;
-   *cp++ = '\n';
-   return cp - buf;
-}
-
 static int should_autocreate_reflog(const char *refname)
 {
if (!log_all_ref_updates)
@@ -2774,7 +2748,7 @@ static int log_ref_write_fd(int fd, const unsigned char 
*old_sha1,
sha1_to_hex(new_sha1),
committer);
if (msglen)
-   len += copy_msg(logrec + len - 1, msg) - 1;
+   len += copy_reflog_msg(logrec + len - 1, msg) - 1;
 
written = len <= maxlen ? write_in_full(fd, logrec, len) : -1;
free(logrec);
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 5ae084b..a1655e3 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -114,4 +114,11 @@ struct ref_transaction {
enum ref_transaction_state state;
 };
 
+/*
+ * Copy the reflog message msg to buf, which has been allocated sufficiently
+ * large, while cleaning up the whitespaces.  Especially, convert LF to space,
+ * because reflog file is one line per entry.
+ */
+int copy_reflog_msg(char *buf, const char *msg);
+
 #endif /* REFS_REFS_INTERNAL_H */
diff --git a/refs/refs.c b/refs/refs.c
index 31f1b19..6bf790b 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -827,6 +827,27 @@ int for_each_remote_ref_submodule(const char *submodule, 
each_ref_fn fn, void *c
return for_each_ref_in_submodule(submodule, "refs/remotes/", fn, 
cb_data);
 }
 
+int copy_reflog_msg(char *buf, const char *msg)
+{
+   char *cp = buf;
+   char c;
+   int wasspace = 1;
+
+   *cp++ = '\t';
+   while ((c = *msg++)) {
+   if (wasspace && isspace(c))
+   continue;
+   wasspace = isspace(c);
+   if (wasspace)
+   c = ' ';
+   *cp++ = c;
+   }
+   while (buf < cp && isspace(cp[-1]))
+   cp--;
+   *cp++ = '\n';
+   return cp - buf;
+}
+
 int head_ref_namespaced(each_ref_fn fn, void *cb_data)
 {
struct strbuf buf = STRBUF_INIT;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 21/25] refs: move should_autocreate_reflog to common code

[Michael Haggerty]

From: David Turner 

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 10 --
 refs/refs-internal.h |  2 ++
 refs/refs.c  | 10 ++
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 2073004..7dc68b8 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2612,16 +2612,6 @@ static int commit_ref(struct ref_lock *lock)
return 0;
 }
 
-static int should_autocreate_reflog(const char *refname)
-{
-   if (!log_all_ref_updates)
-   return 0;
-   return starts_with(refname, "refs/heads/") ||
-   starts_with(refname, "refs/remotes/") ||
-   starts_with(refname, "refs/notes/") ||
-   !strcmp(refname, "HEAD");
-}
-
 /*
  * Create a reflog for a ref.  If force_create = 0, the reflog will
  * only be created for certain refs (those for which
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 5b4176e..d854d9e 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -148,6 +148,8 @@ struct ref_transaction {
enum ref_transaction_state state;
 };
 
+extern int should_autocreate_reflog(const char *refname);
+
 /*
  * Copy the reflog message msg to buf, which has been allocated sufficiently
  * large, while cleaning up the whitespaces.  Especially, convert LF to space,
diff --git a/refs/refs.c b/refs/refs.c
index 5d95804..25d4e22 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -625,6 +625,16 @@ char *resolve_refdup(const char *refname, int 
resolve_flags,
  sha1, flags));
 }
 
+int should_autocreate_reflog(const char *refname)
+{
+   if (!log_all_ref_updates)
+   return 0;
+   return starts_with(refname, "refs/heads/") ||
+   starts_with(refname, "refs/remotes/") ||
+   starts_with(refname, "refs/notes/") ||
+   !strcmp(refname, "HEAD");
+}
+
 /*
  * How to handle various characters in refnames:
  * 0: An acceptable character for refs
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 20/25] refs: move peel_object to the common code

[Michael Haggerty]

From: David Turner 

This function does not contain any backend specific code so move it to
the common code.

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 53 
 refs/refs-internal.h | 34 +
 refs/refs.c  | 23 +++
 3 files changed, 57 insertions(+), 53 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 4807b32..2073004 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1612,59 +1612,6 @@ const char *resolve_ref_unsafe(const char *refname, int 
resolve_flags,
return ret;
 }
 
-enum peel_status {
-   /* object was peeled successfully: */
-   PEEL_PEELED = 0,
-
-   /*
-* object cannot be peeled because the named object (or an
-* object referred to by a tag in the peel chain), does not
-* exist.
-*/
-   PEEL_INVALID = -1,
-
-   /* object cannot be peeled because it is not a tag: */
-   PEEL_NON_TAG = -2,
-
-   /* ref_entry contains no peeled value because it is a symref: */
-   PEEL_IS_SYMREF = -3,
-
-   /*
-* ref_entry cannot be peeled because it is broken (i.e., the
-* symbolic reference cannot even be resolved to an object
-* name):
-*/
-   PEEL_BROKEN = -4
-};
-
-/*
- * Peel the named object; i.e., if the object is a tag, resolve the
- * tag recursively until a non-tag is found.  If successful, store the
- * result to sha1 and return PEEL_PEELED.  If the object is not a tag
- * or is not valid, return PEEL_NON_TAG or PEEL_INVALID, respectively,
- * and leave sha1 unchanged.
- */
-static enum peel_status peel_object(const unsigned char *name, unsigned char 
*sha1)
-{
-   struct object *o = lookup_unknown_object(name);
-
-   if (o->type == OBJ_NONE) {
-   int type = sha1_object_info(name, NULL);
-   if (type < 0 || !object_as_type(o, type, 0))
-   return PEEL_INVALID;
-   }
-
-   if (o->type != OBJ_TAG)
-   return PEEL_NON_TAG;
-
-   o = deref_tag_noverify(o);
-   if (!o)
-   return PEEL_INVALID;
-
-   hashcpy(sha1, o->sha1);
-   return PEEL_PEELED;
-}
-
 /*
  * Peel the entry (if possible) and return its new peel_status.  If
  * repeel is true, re-peel the entry even if there is an old peeled
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index a1655e3..5b4176e 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -59,6 +59,40 @@ int refname_is_safe(const char *refname);
  * value to ref_update::flags
  */
 
+enum peel_status {
+   /* object was peeled successfully: */
+   PEEL_PEELED = 0,
+
+   /*
+* object cannot be peeled because the named object (or an
+* object referred to by a tag in the peel chain), does not
+* exist.
+*/
+   PEEL_INVALID = -1,
+
+   /* object cannot be peeled because it is not a tag: */
+   PEEL_NON_TAG = -2,
+
+   /* ref_entry contains no peeled value because it is a symref: */
+   PEEL_IS_SYMREF = -3,
+
+   /*
+* ref_entry cannot be peeled because it is broken (i.e., the
+* symbolic reference cannot even be resolved to an object
+* name):
+*/
+   PEEL_BROKEN = -4
+};
+
+/*
+ * Peel the named object; i.e., if the object is a tag, resolve the
+ * tag recursively until a non-tag is found.  If successful, store the
+ * result to sha1 and return PEEL_PEELED.  If the object is not a tag
+ * or is not valid, return PEEL_NON_TAG or PEEL_INVALID, respectively,
+ * and leave sha1 unchanged.
+ */
+enum peel_status peel_object(const unsigned char *name, unsigned char *sha1);
+
 /**
  * Information needed for a single ref update. Set new_sha1 to the new
  * value or to null_sha1 to delete the ref. To check the old value
diff --git a/refs/refs.c b/refs/refs.c
index 6bf790b..5d95804 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -4,6 +4,8 @@
 #include "../cache.h"
 #include "refs-internal.h"
 #include "../lockfile.h"
+#include "../object.h"
+#include "../tag.h"
 
 static int is_per_worktree_ref(const char *refname)
 {
@@ -994,3 +996,24 @@ int refname_is_safe(const char *refname)
}
return 1;
 }
+
+enum peel_status peel_object(const unsigned char *name, unsigned char *sha1)
+{
+   struct object *o = lookup_unknown_object(name);
+
+   if (o->type == OBJ_NONE) {
+   int type = sha1_object_info(name, NULL);
+   if (type < 0 || !object_as_type(o, type, 0))
+   return PEEL_INVALID;
+   }
+
+   if (o->type != OBJ_TAG)
+   return PEEL_NON_TAG;
+
+   o = deref_tag_noverify(o);
+   if (!o)
+   return PEEL_INVALID;
+
+   hashcpy(sha1, o->sha1);
+   

[PATCH v6 06/25] refs: move read_ref_at to the common refs file

[Michael Haggerty]

From: Ronnie Sahlberg 

Move read_ref_at() to refs/refs.c since this function does not contain
any backend specific code.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 118 ---
 refs/refs.c  | 118 +++
 2 files changed, 118 insertions(+), 118 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6d502ac..8eba347 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3469,124 +3469,6 @@ int create_symref(const char *ref_target, const char 
*refs_heads_master,
return 0;
 }
 
-struct read_ref_at_cb {
-   const char *refname;
-   unsigned long at_time;
-   int cnt;
-   int reccnt;
-   unsigned char *sha1;
-   int found_it;
-
-   unsigned char osha1[20];
-   unsigned char nsha1[20];
-   int tz;
-   unsigned long date;
-   char **msg;
-   unsigned long *cutoff_time;
-   int *cutoff_tz;
-   int *cutoff_cnt;
-};
-
-static int read_ref_at_ent(unsigned char *osha1, unsigned char *nsha1,
-   const char *email, unsigned long timestamp, int tz,
-   const char *message, void *cb_data)
-{
-   struct read_ref_at_cb *cb = cb_data;
-
-   cb->reccnt++;
-   cb->tz = tz;
-   cb->date = timestamp;
-
-   if (timestamp <= cb->at_time || cb->cnt == 0) {
-   if (cb->msg)
-   *cb->msg = xstrdup(message);
-   if (cb->cutoff_time)
-   *cb->cutoff_time = timestamp;
-   if (cb->cutoff_tz)
-   *cb->cutoff_tz = tz;
-   if (cb->cutoff_cnt)
-   *cb->cutoff_cnt = cb->reccnt - 1;
-   /*
-* we have not yet updated cb->[n|o]sha1 so they still
-* hold the values for the previous record.
-*/
-   if (!is_null_sha1(cb->osha1)) {
-   hashcpy(cb->sha1, nsha1);
-   if (hashcmp(cb->osha1, nsha1))
-   warning("Log for ref %s has gap after %s.",
-   cb->refname, show_date(cb->date, 
cb->tz, DATE_MODE(RFC2822)));
-   }
-   else if (cb->date == cb->at_time)
-   hashcpy(cb->sha1, nsha1);
-   else if (hashcmp(nsha1, cb->sha1))
-   warning("Log for ref %s unexpectedly ended on %s.",
-   cb->refname, show_date(cb->date, cb->tz,
-  DATE_MODE(RFC2822)));
-   hashcpy(cb->osha1, osha1);
-   hashcpy(cb->nsha1, nsha1);
-   cb->found_it = 1;
-   return 1;
-   }
-   hashcpy(cb->osha1, osha1);
-   hashcpy(cb->nsha1, nsha1);
-   if (cb->cnt > 0)
-   cb->cnt--;
-   return 0;
-}
-
-static int read_ref_at_ent_oldest(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp,
- int tz, const char *message, void *cb_data)
-{
-   struct read_ref_at_cb *cb = cb_data;
-
-   if (cb->msg)
-   *cb->msg = xstrdup(message);
-   if (cb->cutoff_time)
-   *cb->cutoff_time = timestamp;
-   if (cb->cutoff_tz)
-   *cb->cutoff_tz = tz;
-   if (cb->cutoff_cnt)
-   *cb->cutoff_cnt = cb->reccnt;
-   hashcpy(cb->sha1, osha1);
-   if (is_null_sha1(cb->sha1))
-   hashcpy(cb->sha1, nsha1);
-   /* We just want the first entry */
-   return 1;
-}
-
-int read_ref_at(const char *refname, unsigned int flags, unsigned long 
at_time, int cnt,
-   unsigned char *sha1, char **msg,
-   unsigned long *cutoff_time, int *cutoff_tz, int *cutoff_cnt)
-{
-   struct read_ref_at_cb cb;
-
-   memset(, 0, sizeof(cb));
-   cb.refname = refname;
-   cb.at_time = at_time;
-   cb.cnt = cnt;
-   cb.msg = msg;
-   cb.cutoff_time = cutoff_time;
-   cb.cutoff_tz = cutoff_tz;
-   cb.cutoff_cnt = cutoff_cnt;
-   cb.sha1 = sha1;
-
-   for_each_reflog_ent_reverse(refname, read_ref_at_ent, );
-
-   if (!cb.reccnt) {
-   if (flags & GET_SHA1_QUIETLY)
-   exit(128);
-   else
-   die("Log for %s is empty.", refname);
-   }
-   if (cb.found_it)
-   return 0;
-
-   for_each_reflog_ent(refname, read_ref_at_ent_oldest, );
-
-   return 1;
-}
-
 int reflog_exists(const char *refname)
 {
struct stat st;
diff --git a/refs/refs.c b/refs/refs.c
index 96d5c6d..99221a3 100644
--- a/refs/refs.c
+++ 

[PATCH v6 03/25] refs: add a new file, refs/refs.c, to hold common refs code

[Michael Haggerty]

From: Ronnie Sahlberg 

Create a new file, refs/refs.c, that will be used to hold all the refs
code that is backend agnostic and can be used across all backends.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 Makefile| 1 +
 refs/refs.c | 3 +++
 2 files changed, 4 insertions(+)
 create mode 100644 refs/refs.c

diff --git a/Makefile b/Makefile
index 1a63383..be45704 100644
--- a/Makefile
+++ b/Makefile
@@ -764,6 +764,7 @@ LIB_OBJS += read-cache.o
 LIB_OBJS += reflog-walk.o
 LIB_OBJS += ref-filter.o
 LIB_OBJS += refs/files-backend.o
+LIB_OBJS += refs/refs.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
 LIB_OBJS += rerere.o
diff --git a/refs/refs.c b/refs/refs.c
new file mode 100644
index 000..77492ff
--- /dev/null
+++ b/refs/refs.c
@@ -0,0 +1,3 @@
+/*
+ * Common refs code for all backends.
+ */
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 00/25] refs backend pre-vtable

[Michael Haggerty]

This is a proposed revision of David Turner's refs-backend-pre-vtable
v5 [1]. It incorporates all of the changes I suggested when reviewing
that series, plus it moves the refs-related code to a subdirectory and
creates a new header file refs/refs-internal.h for declarations meant
to be used within the refs module but not by other code. I've retained
the authors of the original patches because most of what I've done is
just shift things around and adjust some includes and comments
accordingly.

This branch proposes the following file layout for the refs module:

refs.h
refs/refs.c
refs/refs-internal.h
refs/files-backend.c
refs/lmdb-backend.c   <- suggested name for the upcoming LMDB code

I changed the name of "refs-be-files.c" to "refs/files-backend.c"
because otherwise the "refs" would have been redundant with the
subdirectory name, and now there is room in the filename to spell out
"backend".

Summary of changes in my branch vs. your v5:

* Rename refs-be-files.c to refs/files-backend.c
* Rename refs.c to refs/refs.c
* Introduce a new header file, refs/refs-internal.h, to hold
  declarations that are needed by other refs backends but shouldn't
  be used by non-refs code. The end result is that refs.h is almost
  unchanged by this series.
* Adjust commit messages for the new file locations.
* Instead of making verify_refname_available() public, move it to
  refs/refs-internal.h. (I think this function will only be needed
  by other refs backends and not by other code.) This change
  required your patch 03/26 to be moved later in the series.
* Add a missing LF in your patch 18/26.
* Touch up the log message and comments in your patch 25/26.
* Drop patch 26/26, Peff's "introduce "extensions" form of
  core.repositoryformatversion", as that patch is already in master.

These patches are also available as the `refs-backend-pre-vtable`
branch in my GitHub repo [2].

I don't want to hijack this patch series or anything, but it seemed
simpler to express my suggestions as code. Feel free to pick and
choose what you like and submit a v7 if that is your preference.

Michael

[1] http://thread.gmane.org/gmane.comp.version-control.git/280325
[2] https://github.com/mhagger/git

David Turner (9):
  refs: make is_branch public
  refs: move transaction functions to the common code
  refs: move refname_is_safe to the common code
  refs: move copy_msg to the common code
  refs: move peel_object to the common code
  refs: move should_autocreate_reflog to common code
  initdb: make safe_create_dir public
  files_log_ref_write: new function
  refs: break out ref conflict checks

Ronnie Sahlberg (16):
  refs/files-backend.c: new file, renamed from refs.c
  refs: add a new file, refs/refs.c, to hold common refs code
  refs: move update_ref to refs/refs.c
  refs: move delete_pseudoref and delete_ref to the common code
  refs: move read_ref_at to the common refs file
  refs: move the hidden refs functions to the common code
  refs: move dwim and friend functions to the common refs code
  refs: move warn_if_dangling_symref* to the common code
  refs: move read_ref, read_ref_full and ref_exists to the common code
  refs: move resolve_refdup to common
  refs: move check_refname_format to the common code
  refs: move is_branch to the common code
  refs: move prettify_refname to the common code
  refs: move ref iterators to the common code
  refs: move head_ref_namespaced to the common code
  refs: create a shared version of verify_refname_available

 Makefile   |5 +-
 builtin/init-db.c  |   12 -
 cache.h|8 +
 path.c |   12 +
 refs.h |2 +
 refs.c => refs/files-backend.c | 1291 ++--
 refs/refs-internal.h   |  200 +++
 refs/refs.c| 1073 +
 8 files changed, 1348 insertions(+), 1255 deletions(-)
 rename refs.c => refs/files-backend.c (75%)
 create mode 100644 refs/refs-internal.h
 create mode 100644 refs/refs.c

-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 14/25] refs: move prettify_refname to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 9 -
 refs/refs.c  | 9 +
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 468758a..89410a7 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2064,15 +2064,6 @@ int for_each_rawref(each_ref_fn fn, void *cb_data)
   DO_FOR_EACH_INCLUDE_BROKEN, cb_data);
 }
 
-const char *prettify_refname(const char *name)
-{
-   return name + (
-   starts_with(name, "refs/heads/") ? 11 :
-   starts_with(name, "refs/tags/") ? 10 :
-   starts_with(name, "refs/remotes/") ? 13 :
-   0);
-}
-
 static void unlock_ref(struct ref_lock *lock)
 {
/* Do not free lock->lk -- atexit() still looks at them */
diff --git a/refs/refs.c b/refs/refs.c
index fb0e5df..3dd8e91 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -736,3 +736,12 @@ int is_branch(const char *refname)
 {
return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
 }
+
+const char *prettify_refname(const char *name)
+{
+   return name + (
+   starts_with(name, "refs/heads/") ? 11 :
+   starts_with(name, "refs/tags/") ? 10 :
+   starts_with(name, "refs/remotes/") ? 13 :
+   0);
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 01/25] refs: make is_branch public

[Michael Haggerty]

From: David Turner 

is_branch was already non-static, but this patch declares it in the
header.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/refs.h b/refs.h
index 6d30c98..39b8edc 100644
--- a/refs.h
+++ b/refs.h
@@ -217,6 +217,8 @@ extern void warn_dangling_symrefs(FILE *fp, const char 
*msg_fmt, const struct st
  */
 int pack_refs(unsigned int flags);
 
+int is_branch(const char *refname);
+
 /*
  * Flags controlling ref_transaction_update(), ref_transaction_create(), etc.
  * REF_NODEREF: act on the ref directly, instead of dereferencing
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 12/25] refs: move check_refname_format to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

This function does not contain any backend specific code so move it to
the common code.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 109 ---
 refs/refs.c  | 109 +++
 2 files changed, 109 insertions(+), 109 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index f84fa9e..545075f 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -14,27 +14,6 @@ struct ref_lock {
 };
 
 /*
- * How to handle various characters in refnames:
- * 0: An acceptable character for refs
- * 1: End-of-component
- * 2: ., look for a preceding . to reject .. in refs
- * 3: {, look for a preceding @ to reject @{ in refs
- * 4: A bad character: ASCII control characters, and
- *":", "?", "[", "\", "^", "~", SP, or TAB
- * 5: *, reject unless REFNAME_REFSPEC_PATTERN is set
- */
-static unsigned char refname_disposition[256] = {
-   1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-   4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
-   4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 2, 1,
-   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 4,
-   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 4, 0,
-   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4
-};
-
-/*
  * Flag passed to lock_ref_sha1_basic() telling it to tolerate broken
  * refs (i.e., because the reference is about to be deleted anyway).
  */
@@ -69,94 +48,6 @@ static unsigned char refname_disposition[256] = {
  * value to ref_update::flags
  */
 
-/*
- * Try to read one refname component from the front of refname.
- * Return the length of the component found, or -1 if the component is
- * not legal.  It is legal if it is something reasonable to have under
- * ".git/refs/"; We do not like it if:
- *
- * - any path component of it begins with ".", or
- * - it has double dots "..", or
- * - it has ASCII control characters, or
- * - it has ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
- * - it has "*" anywhere unless REFNAME_REFSPEC_PATTERN is set, or
- * - it ends with a "/", or
- * - it ends with ".lock", or
- * - it contains a "@{" portion
- */
-static int check_refname_component(const char *refname, int *flags)
-{
-   const char *cp;
-   char last = '\0';
-
-   for (cp = refname; ; cp++) {
-   int ch = *cp & 255;
-   unsigned char disp = refname_disposition[ch];
-   switch (disp) {
-   case 1:
-   goto out;
-   case 2:
-   if (last == '.')
-   return -1; /* Refname contains "..". */
-   break;
-   case 3:
-   if (last == '@')
-   return -1; /* Refname contains "@{". */
-   break;
-   case 4:
-   return -1;
-   case 5:
-   if (!(*flags & REFNAME_REFSPEC_PATTERN))
-   return -1; /* refspec can't be a pattern */
-
-   /*
-* Unset the pattern flag so that we only accept
-* a single asterisk for one side of refspec.
-*/
-   *flags &= ~ REFNAME_REFSPEC_PATTERN;
-   break;
-   }
-   last = ch;
-   }
-out:
-   if (cp == refname)
-   return 0; /* Component has zero length. */
-   if (refname[0] == '.')
-   return -1; /* Component starts with '.'. */
-   if (cp - refname >= LOCK_SUFFIX_LEN &&
-   !memcmp(cp - LOCK_SUFFIX_LEN, LOCK_SUFFIX, LOCK_SUFFIX_LEN))
-   return -1; /* Refname ends with ".lock". */
-   return cp - refname;
-}
-
-int check_refname_format(const char *refname, int flags)
-{
-   int component_len, component_count = 0;
-
-   if (!strcmp(refname, "@"))
-   /* Refname is a single character '@'. */
-   return -1;
-
-   while (1) {
-   /* We are at the start of a path component. */
-   component_len = check_refname_component(refname, );
-   if (component_len <= 0)
-   return -1;
-
-   component_count++;
-   if (refname[component_len] == '\0')
-   break;
-   /* Skip to next component. */
-   refname += component_len + 1;
-   }
-
-   if (refname[component_len - 1] == '.')
-   return -1; /* Refname ends with '.'. */
-   if (!(flags & 

[PATCH v6 02/25] refs/files-backend.c: new file, renamed from refs.c

[Michael Haggerty]

From: Ronnie Sahlberg 

We are about to split up the reference code into multiple source
files, to separate the generic code from the code that is specific to
the current files-based backend. To keep things organized, we will
move the reference-related code to a subdirectory, "refs/". Over time,
more files will appear in this subdirectory.

As the first step, rename refs.c to refs/files-backend.c. This file
will eventually contain only the code specific to the files-based
backend. In subsequent commits we will move backend-agnostic code to a
new refs/refs.c file that will hold the common refs code that is
shared across backends.

The reason for first moving all the code to refs/files-backend.c and
then moving the backend agnostic code back to refs/refs.c instead of
moving refs.c to refs/refs.c first is because the code that will
eventually remain in this new files-backend.c file is so entangled
that it would then be very difficult to break the split up into small
independent patches/chunks.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 Makefile   |  4 ++--
 refs.c => refs/files-backend.c | 14 +++---
 2 files changed, 9 insertions(+), 9 deletions(-)
 rename refs.c => refs/files-backend.c (99%)

diff --git a/Makefile b/Makefile
index 0d9f5dd..1a63383 100644
--- a/Makefile
+++ b/Makefile
@@ -762,8 +762,8 @@ LIB_OBJS += quote.o
 LIB_OBJS += reachable.o
 LIB_OBJS += read-cache.o
 LIB_OBJS += reflog-walk.o
-LIB_OBJS += refs.o
 LIB_OBJS += ref-filter.o
+LIB_OBJS += refs/files-backend.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace_object.o
 LIB_OBJS += rerere.o
@@ -2412,7 +2412,7 @@ profile-clean:
$(RM) $(addsuffix *.gcno,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
 
 clean: profile-clean coverage-clean
-   $(RM) *.o *.res block-sha1/*.o ppc/*.o compat/*.o compat/*/*.o
+   $(RM) *.o *.res refs/*.o block-sha1/*.o ppc/*.o compat/*.o compat/*/*.o
$(RM) xdiff/*.o vcs-svn/*.o ewah/*.o builtin/*.o
$(RM) $(LIB_FILE) $(XDIFF_LIB) $(VCSSVN_LIB)
$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
diff --git a/refs.c b/refs/files-backend.c
similarity index 99%
rename from refs.c
rename to refs/files-backend.c
index 132eff5..dc39b36 100644
--- a/refs.c
+++ b/refs/files-backend.c
@@ -1,10 +1,10 @@
-#include "cache.h"
-#include "lockfile.h"
-#include "refs.h"
-#include "object.h"
-#include "tag.h"
-#include "dir.h"
-#include "string-list.h"
+#include "../cache.h"
+#include "../refs.h"
+#include "../lockfile.h"
+#include "../object.h"
+#include "../tag.h"
+#include "../dir.h"
+#include "../string-list.h"
 
 struct ref_lock {
char *ref_name;
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 25/25] refs: break out ref conflict checks

[Michael Haggerty]

From: David Turner 

Create new function verify_no_descendants, to hold one of the ref
conflict checks used in verify_refname_available. Multiple backends
will need this function, so move it to the common code.

Also move rename_ref_available to the common code, because alternate
backends might need it and it has no files-backend-specific code.

Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 49 +++--
 refs/refs-internal.h | 16 
 refs/refs.c  | 44 
 3 files changed, 67 insertions(+), 42 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index e0ae769..baa13bb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -729,6 +729,7 @@ static int verify_refname_available_dir(const char *refname,
struct strbuf *err)
 {
const char *slash;
+   const char *extra_refname;
int pos;
struct strbuf dirname = STRBUF_INIT;
int ret = -1;
@@ -834,32 +835,12 @@ static int verify_refname_available_dir(const char 
*refname,
}
}
 
-   if (extras) {
-   /*
-* Check for entries in extras that start with
-* "$refname/". We do that by looking for the place
-* where "$refname/" would be inserted in extras. If
-* there is an entry at that position that starts with
-* "$refname/" and is not in skip, then we have a
-* conflict.
-*/
-   for (pos = string_list_find_insert_index(extras, dirname.buf, 
0);
-pos < extras->nr; pos++) {
-   const char *extra_refname = extras->items[pos].string;
-
-   if (!starts_with(extra_refname, dirname.buf))
-   break;
-
-   if (!skip || !string_list_has_string(skip, 
extra_refname)) {
-   strbuf_addf(err, "cannot process '%s' and '%s' 
at the same time",
-   refname, extra_refname);
-   goto cleanup;
-   }
-   }
-   }
-
-   /* No conflicts were found */
-   ret = 0;
+   extra_refname = find_descendant_ref(dirname.buf, extras, skip);
+   if (extra_refname)
+   strbuf_addf(err, "cannot process '%s' and '%s' at the same 
time",
+   refname, extra_refname);
+   else
+   ret = 0;
 
 cleanup:
strbuf_release();
@@ -2461,22 +2442,6 @@ out:
return ret;
 }
 
-static int rename_ref_available(const char *oldname, const char *newname)
-{
-   struct string_list skip = STRING_LIST_INIT_NODUP;
-   struct strbuf err = STRBUF_INIT;
-   int ret;
-
-   string_list_insert(, oldname);
-   ret = !verify_refname_available(newname, NULL, , );
-   if (!ret)
-   error("%s", err.buf);
-
-   string_list_clear(, 0);
-   strbuf_release();
-   return ret;
-}
-
 static int write_ref_to_lockfile(struct ref_lock *lock,
 const unsigned char *sha1, struct strbuf *err);
 static int commit_ref_update(struct ref_lock *lock,
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f6a79b9..500b708 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -25,6 +25,22 @@
 int refname_is_safe(const char *refname);
 
 /*
+ * Check for entries in extras that are within the specified
+ * directory, where dirname is a reference directory name including
+ * the trailing slash (e.g., "refs/heads/foo/"). Ignore any
+ * conflicting references that are found in skip. If there is a
+ * conflicting reference, return its name.
+ *
+ * extras and skip must be sorted lists of reference names. Either one
+ * can be NULL, signifying the empty list.
+ */
+const char *find_descendant_ref(const char *dirname,
+   const struct string_list *extras,
+   const struct string_list *skip);
+
+int rename_ref_available(const char *oldname, const char *newname);
+
+/*
  * Flag passed to lock_ref_sha1_basic() telling it to tolerate broken
  * refs (i.e., because the reference is about to be deleted anyway).
  */
diff --git a/refs/refs.c b/refs/refs.c
index 25d4e22..1c1c6cb 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -1027,3 +1027,47 @@ enum peel_status peel_object(const unsigned char *name, 
unsigned char *sha1)
hashcpy(sha1, o->sha1);
return PEEL_PEELED;
 }
+
+const char *find_descendant_ref(const char *dirname,
+   const struct string_list *extras,
+   const struct string_list *skip)
+{
+   int pos;

[PATCH v6 15/25] refs: move ref iterators to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: Junio C Hamano 
Signed-off-by: David Turner 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 81 
 refs/refs.c  | 81 
 2 files changed, 81 insertions(+), 81 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 89410a7..53f69f3 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1680,23 +1680,6 @@ const char *resolve_ref_unsafe(const char *refname, int 
resolve_flags,
return ret;
 }
 
-/* The argument to filter_refs */
-struct ref_filter {
-   const char *pattern;
-   each_ref_fn *fn;
-   void *cb_data;
-};
-
-static int filter_refs(const char *refname, const struct object_id *oid,
-  int flags, void *data)
-{
-   struct ref_filter *filter = (struct ref_filter *)data;
-
-   if (wildmatch(filter->pattern, refname, 0, NULL))
-   return 0;
-   return filter->fn(refname, oid, flags, filter->cb_data);
-}
-
 enum peel_status {
/* object was peeled successfully: */
PEEL_PEELED = 0,
@@ -1963,36 +1946,6 @@ int for_each_ref_in_submodule(const char *submodule, 
const char *prefix,
return do_for_each_ref(get_ref_cache(submodule), prefix, fn, 
strlen(prefix), 0, cb_data);
 }
 
-int for_each_tag_ref(each_ref_fn fn, void *cb_data)
-{
-   return for_each_ref_in("refs/tags/", fn, cb_data);
-}
-
-int for_each_tag_ref_submodule(const char *submodule, each_ref_fn fn, void 
*cb_data)
-{
-   return for_each_ref_in_submodule(submodule, "refs/tags/", fn, cb_data);
-}
-
-int for_each_branch_ref(each_ref_fn fn, void *cb_data)
-{
-   return for_each_ref_in("refs/heads/", fn, cb_data);
-}
-
-int for_each_branch_ref_submodule(const char *submodule, each_ref_fn fn, void 
*cb_data)
-{
-   return for_each_ref_in_submodule(submodule, "refs/heads/", fn, cb_data);
-}
-
-int for_each_remote_ref(each_ref_fn fn, void *cb_data)
-{
-   return for_each_ref_in("refs/remotes/", fn, cb_data);
-}
-
-int for_each_remote_ref_submodule(const char *submodule, each_ref_fn fn, void 
*cb_data)
-{
-   return for_each_ref_in_submodule(submodule, "refs/remotes/", fn, 
cb_data);
-}
-
 int for_each_replace_ref(each_ref_fn fn, void *cb_data)
 {
return do_for_each_ref(_cache, git_replace_ref_base, fn,
@@ -2024,40 +1977,6 @@ int for_each_namespaced_ref(each_ref_fn fn, void 
*cb_data)
return ret;
 }
 
-int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
-   const char *prefix, void *cb_data)
-{
-   struct strbuf real_pattern = STRBUF_INIT;
-   struct ref_filter filter;
-   int ret;
-
-   if (!prefix && !starts_with(pattern, "refs/"))
-   strbuf_addstr(_pattern, "refs/");
-   else if (prefix)
-   strbuf_addstr(_pattern, prefix);
-   strbuf_addstr(_pattern, pattern);
-
-   if (!has_glob_specials(pattern)) {
-   /* Append implied '/' '*' if not present. */
-   strbuf_complete(_pattern, '/');
-   /* No need to check for '*', there is none. */
-   strbuf_addch(_pattern, '*');
-   }
-
-   filter.pattern = real_pattern.buf;
-   filter.fn = fn;
-   filter.cb_data = cb_data;
-   ret = for_each_ref(filter_refs, );
-
-   strbuf_release(_pattern);
-   return ret;
-}
-
-int for_each_glob_ref(each_ref_fn fn, const char *pattern, void *cb_data)
-{
-   return for_each_glob_ref_in(fn, pattern, NULL, cb_data);
-}
-
 int for_each_rawref(each_ref_fn fn, void *cb_data)
 {
return do_for_each_ref(_cache, "", fn, 0,
diff --git a/refs/refs.c b/refs/refs.c
index 3dd8e91..9644df6 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -745,3 +745,84 @@ const char *prettify_refname(const char *name)
starts_with(name, "refs/remotes/") ? 13 :
0);
 }
+
+/* The argument to filter_refs */
+struct ref_filter {
+   const char *pattern;
+   each_ref_fn *fn;
+   void *cb_data;
+};
+
+static int filter_refs(const char *refname, const struct object_id *oid, int 
flags,
+  void *data)
+{
+   struct ref_filter *filter = (struct ref_filter *)data;
+
+   if (wildmatch(filter->pattern, refname, 0, NULL))
+   return 0;
+   return filter->fn(refname, oid, flags, filter->cb_data);
+}
+
+int for_each_glob_ref_in(each_ref_fn fn, const char *pattern,
+   const char *prefix, void *cb_data)
+{
+   struct strbuf real_pattern = STRBUF_INIT;
+   struct ref_filter filter;
+   int ret;
+
+   if (!prefix && !starts_with(pattern, "refs/"))
+   strbuf_addstr(_pattern, "refs/");
+   else if (prefix)
+   strbuf_addstr(_pattern, prefix);
+   strbuf_addstr(_pattern, pattern);
+
+   

[PATCH v6 13/25] refs: move is_branch to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 5 -
 refs/refs.c  | 5 +
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 545075f..468758a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3002,11 +3002,6 @@ static int log_ref_write(const char *refname, const 
unsigned char *old_sha1,
return ret;
 }
 
-int is_branch(const char *refname)
-{
-   return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
-}
-
 /*
  * Write sha1 into the open lockfile, then close the lockfile. On
  * errors, rollback the lockfile, fill in *err and
diff --git a/refs/refs.c b/refs/refs.c
index bc7e159..fb0e5df 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -731,3 +731,8 @@ int check_refname_format(const char *refname, int flags)
return -1; /* Refname has only one component. */
return 0;
 }
+
+int is_branch(const char *refname)
+{
+   return !strcmp(refname, "HEAD") || starts_with(refname, "refs/heads/");
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 16/25] refs: move head_ref_namespaced to the common code

[Michael Haggerty]

From: Ronnie Sahlberg 

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 15 ---
 refs/refs.c  | 15 +++
 2 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 53f69f3..728e646 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1952,21 +1952,6 @@ int for_each_replace_ref(each_ref_fn fn, void *cb_data)
   strlen(git_replace_ref_base), 0, cb_data);
 }
 
-int head_ref_namespaced(each_ref_fn fn, void *cb_data)
-{
-   struct strbuf buf = STRBUF_INIT;
-   int ret = 0;
-   struct object_id oid;
-   int flag;
-
-   strbuf_addf(, "%sHEAD", get_git_namespace());
-   if (!read_ref_full(buf.buf, RESOLVE_REF_READING, oid.hash, ))
-   ret = fn(buf.buf, , flag, cb_data);
-   strbuf_release();
-
-   return ret;
-}
-
 int for_each_namespaced_ref(each_ref_fn fn, void *cb_data)
 {
struct strbuf buf = STRBUF_INIT;
diff --git a/refs/refs.c b/refs/refs.c
index 9644df6..6af5fd6 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -826,3 +826,18 @@ int for_each_remote_ref_submodule(const char *submodule, 
each_ref_fn fn, void *c
 {
return for_each_ref_in_submodule(submodule, "refs/remotes/", fn, 
cb_data);
 }
+
+int head_ref_namespaced(each_ref_fn fn, void *cb_data)
+{
+   struct strbuf buf = STRBUF_INIT;
+   int ret = 0;
+   struct object_id oid;
+   int flag;
+
+   strbuf_addf(, "%sHEAD", get_git_namespace());
+   if (!read_ref_full(buf.buf, RESOLVE_REF_READING, oid.hash, ))
+   ret = fn(buf.buf, , flag, cb_data);
+   strbuf_release();
+
+   return ret;
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 18/25] refs: move refname_is_safe to the common code

[Michael Haggerty]

From: David Turner 

This function does not contain any backend specific code, so move it
to the common code. This function might be used by other refs
backends.

While we are doing so, improve the comment's grammar and clarify a
safety rule.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 33 -
 refs/refs-internal.h | 15 +++
 refs/refs.c  | 24 
 3 files changed, 39 insertions(+), 33 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 1719720..3a4aff6 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -196,39 +196,6 @@ static struct ref_dir *get_ref_dir(struct ref_entry *entry)
return dir;
 }
 
-/*
- * Check if a refname is safe.
- * For refs that start with "refs/" we consider it safe as long they do
- * not try to resolve to outside of refs/.
- *
- * For all other refs we only consider them safe iff they only contain
- * upper case characters and '_' (like "HEAD" AND "MERGE_HEAD", and not like
- * "config").
- */
-static int refname_is_safe(const char *refname)
-{
-   if (starts_with(refname, "refs/")) {
-   char *buf;
-   int result;
-
-   buf = xmalloc(strlen(refname) + 1);
-   /*
-* Does the refname try to escape refs/?
-* For example: refs/foo/../bar is safe but refs/foo/../../bar
-* is not.
-*/
-   result = !normalize_path_copy(buf, refname + strlen("refs/"));
-   free(buf);
-   return result;
-   }
-   while (*refname) {
-   if (!isupper(*refname) && *refname != '_')
-   return 0;
-   refname++;
-   }
-   return 1;
-}
-
 static struct ref_entry *create_ref_entry(const char *refname,
  const unsigned char *sha1, int flag,
  int check_name)
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 94f5dde..5ae084b 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -10,6 +10,21 @@
 #include "../refs.h"
 
 /*
+ * Return true iff refname is minimally safe. "Safe" here means that
+ * deleting a loose reference by this name will not do any damage, for
+ * example by causing a file that is not a reference to be deleted.
+ * This function does not check that the reference name is legal; for
+ * that, use check_refname_format().
+ *
+ * We consider a refname that starts with "refs/" to be safe as long
+ * as any ".." components that it might contain do not escape "refs/".
+ * Names that do not start with "refs/" are considered safe iff they
+ * consist entirely of upper case characters and '_' (like "HEAD" and
+ * "MERGE_HEAD" but not "config" or "FOO/BAR").
+ */
+int refname_is_safe(const char *refname);
+
+/*
  * Flag passed to lock_ref_sha1_basic() telling it to tolerate broken
  * refs (i.e., because the reference is about to be deleted anyway).
  */
diff --git a/refs/refs.c b/refs/refs.c
index 0d0579f..31f1b19 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -949,3 +949,27 @@ int ref_transaction_verify(struct ref_transaction 
*transaction,
  NULL, old_sha1,
  flags, NULL, err);
 }
+
+int refname_is_safe(const char *refname)
+{
+   if (starts_with(refname, "refs/")) {
+   char *buf;
+   int result;
+
+   buf = xmalloc(strlen(refname) + 1);
+   /*
+* Does the refname try to escape refs/?
+* For example: refs/foo/../bar is safe but refs/foo/../../bar
+* is not.
+*/
+   result = !normalize_path_copy(buf, refname + strlen("refs/"));
+   free(buf);
+   return result;
+   }
+   while (*refname) {
+   if (!isupper(*refname) && *refname != '_')
+   return 0;
+   refname++;
+   }
+   return 1;
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v6 11/25] refs: move resolve_refdup to common

[Michael Haggerty]

From: Ronnie Sahlberg 

This function can be shared across all refs backends so move it
to the common code.

Signed-off-by: Ronnie Sahlberg 
Signed-off-by: David Turner 
Signed-off-by: Junio C Hamano 
Signed-off-by: Michael Haggerty 
---
 refs/files-backend.c | 7 ---
 refs/refs.c  | 7 +++
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6940d54..f84fa9e 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1789,13 +1789,6 @@ const char *resolve_ref_unsafe(const char *refname, int 
resolve_flags,
return ret;
 }
 
-char *resolve_refdup(const char *refname, int resolve_flags,
-unsigned char *sha1, int *flags)
-{
-   return xstrdup_or_null(resolve_ref_unsafe(refname, resolve_flags,
- sha1, flags));
-}
-
 /* The argument to filter_refs */
 struct ref_filter {
const char *pattern;
diff --git a/refs/refs.c b/refs/refs.c
index ae55d85..56dee05 100644
--- a/refs/refs.c
+++ b/refs/refs.c
@@ -615,3 +615,10 @@ int ref_exists(const char *refname)
unsigned char sha1[20];
return !!resolve_ref_unsafe(refname, RESOLVE_REF_READING, sha1, NULL);
 }
+
+char *resolve_refdup(const char *refname, int resolve_flags,
+unsigned char *sha1, int *flags)
+{
+   return xstrdup_or_null(resolve_ref_unsafe(refname, resolve_flags,
+ sha1, flags));
+}
-- 
2.6.2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] setup: do not create $X/gitdir unnecessarily when accessing git file $X

[Jeff King]

On Mon, Nov 02, 2015 at 08:08:26PM +0100, Nguyễn Thái Ngọc Duy wrote:

> $X/gitdir is created, or refreshed, in order to keep a linked worktree
> from being pruned. But while git file is used as the foundation for
> linked worktrees, it's used for other purposes as well and we should
> not create $X/gitdir in those cases.
> 
> Tighten the check. Only update an existing file, which is an
> indication this is a linked worktree.

Hrm. I think this fixes the immediate problem, but it seems odd for us
to rely on "does the file exist"[1].

We trigger this code unconditionally from read_gitfile_gently(). But
.git files are a general-purpose mechanism.  Shouldn't we be doing this
only if we suspect we are working with a linked working tree directory
in the first place?

Or we do not know at all, because we are operating in the linked dir,
and seeing the presence of the "gitdir" file is the only way we say "ah,
it turns out we are linked, so we should take the opportunity to do some
maintenance"?

If the latter, then I guess this is the only way to do it. It does seem
a bit strange to me that an otherwise read-only operation (reading the
file) might involve writing[2].

-Peff

[1] You check only !stat(), so it is not really "does it exist", but
"can we stat it". I think that is OK, because this is an
opportunistic update, and failing the stat should just mean we don't
do the update.

[2] I suspect this code should use write_file_gently(). What happens if
I have a read-only linked checkout?
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] setup: do not create $X/gitdir unnecessarily when accessing git file $X

[Nguyễn Thái Ngọc Duy]

$X/gitdir is created, or refreshed, in order to keep a linked worktree
from being pruned. But while git file is used as the foundation for
linked worktrees, it's used for other purposes as well and we should
not create $X/gitdir in those cases.

Tighten the check. Only update an existing file, which is an
indication this is a linked worktree.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 setup.c| 2 +-
 t/t0002-gitfile.sh | 7 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index d343725..b30d923 100644
--- a/setup.c
+++ b/setup.c
@@ -440,7 +440,7 @@ static void update_linked_gitdir(const char *gitfile, const 
char *gitdir)
struct stat st;
 
strbuf_addf(, "%s/gitdir", gitdir);
-   if (stat(path.buf, ) || st.st_mtime + 24 * 3600 < time(NULL))
+   if (!stat(path.buf, ) && st.st_mtime + 24 * 3600 < time(NULL))
write_file(path.buf, "%s", gitfile);
strbuf_release();
 }
diff --git a/t/t0002-gitfile.sh b/t/t0002-gitfile.sh
index 9670e8c..b1b59f2 100755
--- a/t/t0002-gitfile.sh
+++ b/t/t0002-gitfile.sh
@@ -99,6 +99,13 @@ test_expect_success 'check rev-list' '
test "$SHA" = "$(git rev-list HEAD)"
 '
 
+test_expect_success '$REAL/gitdir is not created on ordinary git file' '
+   echo "gitdir: $REAL" >expected &&
+   test_cmp expected .git &&
+   git status &&
+   ! test -f "$REAL"/gitdir
+'
+
 test_expect_success 'setup_git_dir twice in subdir' '
git init sgd &&
(
-- 
2.2.0.513.g477eb31

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/5] Use watchman to reduce index refresh time

[Duy Nguyen]

On Mon, Nov 2, 2015 at 3:54 PM, Paolo Ciarrocchi
 wrote:
> On Sun, Nov 1, 2015 at 2:55 PM, Nguyễn Thái Ngọc Duy  
> wrote:
>
> Hi Duy,
>
>> This series builds on top of the index-helper series I just sent and
>> uses watchman to keep track of file changes in order to avoid lstat()
>> at refresh time. The series can also be found at [1]
>>
>> When I started this work, watchman did not support Windows yet. It
>> does now, even if still experimental [2]. So Windows people, please
>> try it out if you have time.
>>
>> To put all pieces so far together, we have split-index to reduce index
>> write time, untracked cache to reduce I/O as well as computation for
>> .gitignore, index-helper for index read time and this series for
>> lstat() at refresh time. The remaining piece is killing lstat() from
>> untracked cache, but right now it's just some idea and incomplete
>> code.
>
> Did you manage to measure the speedup introduced by this series?

It was from last year. I may have measured it but because I didn't
save it in the commit message, it was lost anyway. Installing watchman
and measuring with webkit.git soon..
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3] http.c: use CURLOPT_RANGE for range requests

[Jeff King]

On Mon, Nov 02, 2015 at 02:36:26PM -0500, David Turner wrote:

> A HTTP server is permitted to return a non-range response to a HTTP
> range request (and Apache httpd in fact does this in some cases).
> While libcurl knows how to correctly handle this (by skipping bytes
> before and after the requested range), it only turns on this handling
> if it is aware that a range request is being made.  By manually
> setting the range header instead of using CURLOPT_RANGE, we were
> hiding the fact that this was a range request from libcurl.  This
> could cause corruption.
> 
> Signed-off-by: David Turner 
> ---
> 
> This version breaks the range option formatting/setting out to a
> helper function, as suggested by Junio and Jeff.
> 
> In addition, it clears the range option when curl slots are cleared
> before reuse, also as suggested

Thanks, this looks much nicer to me.

A few minor comments:

> +static void http_opt_request_remainder(CURL *curl, ssize_t lo)

I notice you used ssize_t here. If we are going to deal with large files, I
would think off_t would make more sense (i.e., to allow >2GB on a 32-bit
system).

But much worse than that, the value we are passing typically comes from
ftell(), which only returns a long. So we're truncated anyway in that
case.

I certainly don't think we are making anything _worse_ here; the problem
is in the existing code. But I don't think ssize_t is making anything
better (it's generally the same size as a long anyway). So I think I'd
prefer one of the following:

  1. Leave it as "long". At least then we are matching ftell(), which is
 clear (and works fine on 64-bit machines).

  2. Use off_t here instead. It doesn't fix the problem, but at least
 fixes our one component, so it's working towards a better solution
 in the long run.

  3. Detect and complain when we overflow the long. Hopefully ftell()
 returns -1 on a 32-bit system when the file is larger than 2GB, so
 this Just Works already, and we don't create a broken output.

  4. Fix all of the callers. I suspect this would involve calling
 fstat(fileno(fh)) to get a real off_t.

Options (3) and (4) are obviously more work, and I don't necessarily
expect you to do them. But I think I'd prefer (2) to what you have now.
Using off_t has an issue with being unsigned, but...

> +{
> + char buf[128];
> + int len = 0;
> +
> + if (lo >= 0)
> + len += xsnprintf(buf + len, 128 - len, "%"PRIiMAX,
> +  (intmax_t)lo);
> + len += xsnprintf(buf + len, 128 - len, "-");

I think we could just drop this "lo >= 0". Now that there is no "hi"
(which I think is fine), there's no reason to call the function at all
if you do not have a limit.

Also, we should prefer "sizeof(buf)" to repeating the "128", as the two
getting out of sync would be disastrous. So altogether something like:


  static void http_opt_request_remainder(CURL *curl, off_t pos)
  {
char buf[128];
xsnprintf(buf, sizeof(buf), "%"PRIuMAX", (uintmax_t)pos);
curl_easy_setopt(curl, CURLOPT_RANGE, buf);
  }

would be enough, I think.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3] http.c: use CURLOPT_RANGE for range requests

[David Turner]

A HTTP server is permitted to return a non-range response to a HTTP
range request (and Apache httpd in fact does this in some cases).
While libcurl knows how to correctly handle this (by skipping bytes
before and after the requested range), it only turns on this handling
if it is aware that a range request is being made.  By manually
setting the range header instead of using CURLOPT_RANGE, we were
hiding the fact that this was a range request from libcurl.  This
could cause corruption.

Signed-off-by: David Turner 
---

This version breaks the range option formatting/setting out to a
helper function, as suggested by Junio and Jeff.

In addition, it clears the range option when curl slots are cleared
before reuse, also as suggested

---
 http.c | 39 ++-
 http.h |  1 -
 2 files changed, 18 insertions(+), 22 deletions(-)

diff --git a/http.c b/http.c
index 6b89dea..eea5836 100644
--- a/http.c
+++ b/http.c
@@ -30,7 +30,6 @@ static CURL *curl_default;
 #endif
 
 #define PREV_BUF_SIZE 4096
-#define RANGE_HEADER_SIZE 30
 
 char curl_errorstr[CURL_ERROR_SIZE];
 
@@ -692,6 +691,7 @@ struct active_request_slot *get_active_slot(void)
curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 0);
curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(slot->curl, CURLOPT_FAILONERROR, 1);
+   curl_easy_setopt(slot->curl, CURLOPT_RANGE, NULL);
 #ifdef LIBCURL_CAN_HANDLE_AUTH_ANY
curl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, http_auth_methods);
 #endif
@@ -1184,6 +1184,19 @@ static const char *get_accept_language(void)
return cached_accept_language;
 }
 
+static void http_opt_request_remainder(CURL *curl, ssize_t lo)
+{
+   char buf[128];
+   int len = 0;
+
+   if (lo >= 0)
+   len += xsnprintf(buf + len, 128 - len, "%"PRIiMAX,
+(intmax_t)lo);
+   len += xsnprintf(buf + len, 128 - len, "-");
+
+   curl_easy_setopt(curl, CURLOPT_RANGE, buf);
+}
+
 /* http_request() targets */
 #define HTTP_REQUEST_STRBUF0
 #define HTTP_REQUEST_FILE  1
@@ -1212,11 +1225,8 @@ static int http_request(const char *url,
long posn = ftell(result);
curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
 fwrite);
-   if (posn > 0) {
-   strbuf_addf(, "Range: bytes=%ld-", posn);
-   headers = curl_slist_append(headers, buf.buf);
-   strbuf_reset();
-   }
+   if (posn > 0)
+   http_opt_request_remainder(slot->curl, posn);
} else
curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
 fwrite_buffer);
@@ -1526,10 +1536,6 @@ void release_http_pack_request(struct http_pack_request 
*preq)
fclose(preq->packfile);
preq->packfile = NULL;
}
-   if (preq->range_header != NULL) {
-   curl_slist_free_all(preq->range_header);
-   preq->range_header = NULL;
-   }
preq->slot = NULL;
free(preq->url);
free(preq);
@@ -1593,7 +1599,6 @@ struct http_pack_request *new_http_pack_request(
struct packed_git *target, const char *base_url)
 {
long prev_posn = 0;
-   char range[RANGE_HEADER_SIZE];
struct strbuf buf = STRBUF_INIT;
struct http_pack_request *preq;
 
@@ -1631,10 +1636,7 @@ struct http_pack_request *new_http_pack_request(
fprintf(stderr,
"Resuming fetch of pack %s at byte %ld\n",
sha1_to_hex(target->sha1), prev_posn);
-   xsnprintf(range, sizeof(range), "Range: bytes=%ld-", prev_posn);
-   preq->range_header = curl_slist_append(NULL, range);
-   curl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER,
-   preq->range_header);
+   http_opt_request_remainder(preq->slot->curl, prev_posn);
}
 
return preq;
@@ -1684,8 +1686,6 @@ struct http_object_request *new_http_object_request(const 
char *base_url,
char prev_buf[PREV_BUF_SIZE];
ssize_t prev_read = 0;
long prev_posn = 0;
-   char range[RANGE_HEADER_SIZE];
-   struct curl_slist *range_header = NULL;
struct http_object_request *freq;
 
freq = xcalloc(1, sizeof(*freq));
@@ -1791,10 +1791,7 @@ struct http_object_request 
*new_http_object_request(const char *base_url,
fprintf(stderr,
"Resuming fetch of object %s at byte %ld\n",
hex, prev_posn);
-   xsnprintf(range, sizeof(range), "Range: bytes=%ld-", prev_posn);
-   range_header = curl_slist_append(range_header, 

Cant download Git for Mac

[MH]

I am trying to download Git and when I try it says “downloading” but it isn’t/

There is an option to download from source forge but I have been told that is 
not a good idea.

Any reason Git is not downloading from git.scm/downloads?

Thank you.

MH--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] setup: do not create $X/gitdir unnecessarily when accessing git file $X

[Eric Sunshine]

On Mon, Nov 2, 2015 at 2:08 PM, Nguyễn Thái Ngọc Duy  wrote:
> $X/gitdir is created, or refreshed, in order to keep a linked worktree
> from being pruned. But while git file is used as the foundation for
> linked worktrees, it's used for other purposes as well and we should
> not create $X/gitdir in those cases.
>
> Tighten the check. Only update an existing file, which is an
> indication this is a linked worktree.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy 
> ---
> diff --git a/t/t0002-gitfile.sh b/t/t0002-gitfile.sh
> @@ -99,6 +99,13 @@ test_expect_success 'check rev-list' '
> +test_expect_success '$REAL/gitdir is not created on ordinary git file' '
> +   echo "gitdir: $REAL" >expected &&
> +   test_cmp expected .git &&
> +   git status &&
> +   ! test -f "$REAL"/gitdir

Minor: test_path_is_missing() might convey the intention a bit more clearly.

> +'
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] Limit the size of the data block passed to SHA1_Update()

[atousa . p]

From: Atousa Pahlevan Duprat 

Some implementations of SHA_Updates have inherent limits
on the max chunk size. SHA1_MAX_BLOCK_SIZE can be defined
to set the max chunk size supported, if required.  This is
enabled for OSX CommonCrypto library and set to 1GiB.

Signed-off-by: Atousa Pahlevan Duprat 
---
 Makefile | 16 +++-
 block-sha1/sha1.h|  2 +-
 cache.h  | 17 +
 compat/apple-common-crypto.h |  4 
 compat/sha1_chunked.c| 19 +++
 5 files changed, 52 insertions(+), 6 deletions(-)
 create mode 100644 compat/sha1_chunked.c

diff --git a/Makefile b/Makefile
index 04c2231..1b098cc 100644
--- a/Makefile
+++ b/Makefile
@@ -136,11 +136,15 @@ all::
 # to provide your own OpenSSL library, for example from MacPorts.
 #
 # Define BLK_SHA1 environment variable to make use of the bundled
-# optimized C SHA1 routine.
+# optimized C SHA1 routine.  This implies NO_APPLE_COMMON_CRYPTO.
 #
 # Define PPC_SHA1 environment variable when running make to make use of
 # a bundled SHA1 routine optimized for PowerPC.
 #
+# Define SHA1_MAX_BLOCK_SIZE if your SSH1_Update() implementation can
+# hash only a limited amount of data in one call (e.g. APPLE_COMMON_CRYPTO
+# may want 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined).
+#
 # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).
 #
 # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).
@@ -986,6 +990,10 @@ ifeq (no,$(USE_PARENS_AROUND_GETTEXT_N))
 endif
 endif
 
+ifdef BLK_SHA1
+   NO_APPLE_COMMON_CRYPTO=1
+endif
+
 ifeq ($(uname_S),Darwin)
ifndef NO_FINK
ifeq ($(shell test -d /sw/lib && echo y),y)
@@ -1346,6 +1354,8 @@ else
 ifdef APPLE_COMMON_CRYPTO
COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL
SHA1_HEADER = 
+   # Apple CommonCrypto requires chunking
+   SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
 else
SHA1_HEADER = 
EXTLIBS += $(LIB_4_CRYPTO)
@@ -1353,6 +1363,10 @@ endif
 endif
 endif
 
+ifdef SHA1_MAX_BLOCK_SIZE
+   LIB_OBJS += compat/sha1_chunked.o
+   BASIC_CFLAGS += -DSHA1_MAX_BLOCK_SIZE="$(SHA1_MAX_BLOCK_SIZE)"
+endif
 ifdef NO_PERL_MAKEMAKER
export NO_PERL_MAKEMAKER
 endif
diff --git a/block-sha1/sha1.h b/block-sha1/sha1.h
index b864df6..d085412 100644
--- a/block-sha1/sha1.h
+++ b/block-sha1/sha1.h
@@ -18,5 +18,5 @@ void blk_SHA1_Final(unsigned char hashout[20], blk_SHA_CTX 
*ctx);
 
 #define git_SHA_CTXblk_SHA_CTX
 #define git_SHA1_Init  blk_SHA1_Init
-#define git_SHA1_Updateblk_SHA1_Update
+#define platform_SHA1_Update   blk_SHA1_Update
 #define git_SHA1_Final blk_SHA1_Final
diff --git a/cache.h b/cache.h
index 79066e5..a501652 100644
--- a/cache.h
+++ b/cache.h
@@ -10,12 +10,21 @@
 #include "trace.h"
 #include "string-list.h"
 
+// platform's underlying implementation of SHA1
 #include SHA1_HEADER
 #ifndef git_SHA_CTX
-#define git_SHA_CTXSHA_CTX
-#define git_SHA1_Init  SHA1_Init
-#define git_SHA1_UpdateSHA1_Update
-#define git_SHA1_Final SHA1_Final
+#define git_SHA_CTXSHA_CTX
+#define git_SHA1_Init  SHA1_Init
+#define platform_SHA1_Update   SHA1_Update
+#define git_SHA1_Final SHA1_Final
+#endif
+
+// choose whether chunked implementation or not
+#ifdef SHA1_MAX_BLOCK_SIZE
+int git_SHA1_Update_Chunked(SHA_CTX *c, const void *data, size_t len);
+#define git_SHA1_Update   git_SHA1_Update_Chunked
+#else
+#define git_SHA1_Update   platform_SHA1_Update
 #endif
 
 #include 
diff --git a/compat/apple-common-crypto.h b/compat/apple-common-crypto.h
index c8b9b0e..d3fb264 100644
--- a/compat/apple-common-crypto.h
+++ b/compat/apple-common-crypto.h
@@ -16,6 +16,10 @@
 #undef TYPE_BOOL
 #endif
 
+#ifndef SHA1_MAX_BLOCK_SIZE
+#error Using Apple Common Crypto library requires setting SHA1_MAX_BLOCK_SIZE
+#endif
+
 #ifdef APPLE_LION_OR_NEWER
 #define git_CC_error_check(pattern, err) \
do { \
diff --git a/compat/sha1_chunked.c b/compat/sha1_chunked.c
new file mode 100644
index 000..61f67de
--- /dev/null
+++ b/compat/sha1_chunked.c
@@ -0,0 +1,19 @@
+#include "cache.h"
+
+int git_SHA1_Update_Chunked(SHA_CTX *c, const void *data, size_t len)
+{
+   size_t nr;
+   size_t total = 0;
+   const char *cdata = (const char*)data;
+
+   while (len > 0) {
+   nr = len;
+   if (nr > SHA1_MAX_BLOCK_SIZE)
+   nr = SHA1_MAX_BLOCK_SIZE;
+   platform_SHA1_Update(c, cdata, nr);
+   total += nr;
+   cdata += nr;
+   len -= nr;
+   }
+   return total;
+}
-- 
2.4.9 (Apple Git-60)

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 00/26] refs backend pre-vtable

[Michael Haggerty]

On 10/28/2015 03:14 AM, David Turner wrote:
> This version of the series makes some minor changes from v4:
> 
> refs.c: move refname_is_safe to the common code: clarify a comment
> initdb: move safe_create_dir into common code: add a caveat on safe_create_dir
> refs.c: move update_ref to refs.c: update commit message to include 
> write_pseudoref
> refs.c: move delete_pseudoref and delete_ref to the common code: remove 
> spurious
> added newline
> refs: make files_log_ref_write functions public: reword commit message
> refs.c: move ref iterators to the common code: added my DCO because this 
> patch changed a bunch since Ronnie wrote it

Thanks for your re-roll. I think this series is almost ready. I already
sent a couple of minor comments about it, plus one bigger one: wouldn't
it be better to have a header file for declarations intended to be used
internally by the various refs backends but not be exposed to non-refs
code? I also think we agreed that the refs code could be moved to a
refs/ subdirectory.

So I revised your patch series, including a suggested new file naming
scheme and addressing my other comments. I will submit that version
shortly as v6.

Otherwise it all looks good. I've checked everything rather carefully
and didn't find any problems.

Thanks for your patience!

Michael

[1] https://github.com/mhagger/git

-- 
Michael Haggerty
mhag...@alum.mit.edu

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Watchman/inotify support and other ways to speed up git status

[Duy Nguyen]

On Mon, Nov 2, 2015 at 9:56 PM, David Turner  wrote:
> On Thu, 2015-10-29 at 09:10 +0100, Christian Couder wrote:
>> > We're using Watchman at Twitter.  A week or two ago posted a dump of our
>> > code to github, but I would advise waiting a day or two to use it, as
>> > I'm about to pull a large number of bugfixes into it (I'll update this
>> > thread and provide a link once I do so).
>>
>> Great, I will have a look at it then!
>
> Here's the most recent version:
>
> https://github.com/dturner-tw/git/tree/dturner/watchman

Christian, the index-helper/watchman series are posted because you
showed interest in this area. I'm not rerolling to address David's
comments on the series for now. Take your time evaluate the two
approaches, then you can pick one (and let me know if you want me to
hand my series over, very glad to do so).
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] setup: do not create $X/gitdir unnecessarily when accessing git file $X

[Duy Nguyen]

(resend)

On Mon, Nov 2, 2015 at 9:51 PM, Junio C Hamano  wrote:
> Jeff King  writes:
>
>> [2] I suspect this code should use write_file_gently(). What happens if
>> I have a read-only linked checkout?

I can't hide anything from you guys can I? :) My first attempt was
move this update logic back to setup_..._gentle where it should
belong, but it got complicated because read_file_gently was buried too
deep and there was no easy way to get the information out.

I can try again, or..

>
> Or you may not be the owner of the repository, you think you are
> doing a read-only operation, and you silently end up creating a file
> that cannot be written by the repository owner?
>
> Honestly, I think this whole "just in case the user moved without
> telling us, we sneakily fix things without telling the user" should
> just go away.  This is not the first incidence of a tool trying to
> be overly clever and pretend to know better than the end user biting
> us, is it?

The whole prune strategy is a bit messy trying to cover all cases
while still keeping out of the user's way. Perhaps if we implement
"git worktree mv", or even "worktree fixup" so the user can do it
manually (back when the prune strategy commit was implemented, there
was no git-worktree), then we don't need this magic any more.

So, which way to go?

-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

git filter-branch and merging of signed tags

[Uwe Kleine-König]

Hello,

Consider I want to rewrite a commit that is a merge of a signed tag. In
my case that's 064ca93f7ab927c2752d86fc5727716e340d737c that currently
sits in linux-next:

~/gsrc/linux$ git version
git version 2.6.1
~/gsrc/linux$ git checkout 064ca93f7ab927c2752d86fc5727716e340d737c
~/gsrc/linux$ git cat-file commit HEAD
tree 7086bbd51d06ebd0e3fcca70380d0a066efce6ef
parent e44a01fa0e0aec2f3a21dadb9c030d14831d4258
parent f719a0d6a8541b383c506aaa4b4fa6b4109669f4
author Arnd Bergmann  1444139956 +0200
committer Arnd Bergmann  1444139956 +0200
mergetag object f719a0d6a8541b383c506aaa4b4fa6b4109669f4
 type commit
 tag efm32-for-4.4-rc1
 tagger Uwe Kleine-König  1443549594 +0200
 
 efm32 cleanups for 4.4-rc1
 
 These are just two followup cleanups for commits that are in v3.17-rc1 
and
 waited in my private tree for application since that time.
 -BEGIN PGP SIGNATURE-
 
 iQEcBAABCgAGBQJWCtIwAAoJEMH8FHityuwJFPYH/2LJpi0yUhI01s0D9U5rZgjl
 8mLP70utAMaqXiGqP/O0ndH+za5TrrUTwDJE5jhAHru1Q6q62Ld2BfOR5+K8zlrP
 4UDGBDj3ePJgEtbSsJYJIRT+AKque2GONbeGpJh2arPrNamWChKIz8HoGSgwdRlx
 TvQuVFEa6C3ujP0ATM8x54f0vR4bYm9WmBS522DwZQY4JpGZC005A3Quw6fmFxPJ
 vJRo1fE1d2OZ3Cuxb15m8PLtZw6sbMdMwgyirAFGrJsN9u/V6BzsSzH0s8suT7IP
 WIohnmYtFSUfkQjaGrMl2wIpJSsIzettiXf9GrQDXSGZsQ4k7sW3U+qrcZ7ZUR4=
 =ngv1
 -END PGP SIGNATURE-

Merge tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux 
into next/cleanup

Merge "efm32 cleanups for 4.4-rc1" from Uwe Kleine-Koenig:

These are just two followup cleanups for commits that are in v3.17-rc1 
and
waited in my private tree for application since that time.

* tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux:
  ARM: efm32: switch to vendor,device compatible strings
  ARM: efm32: switch to properly namespaced location property

OK, to fix my name in the commit log I do:

 git filter-branch -f --msg-filter "sed 's/Kleine-Koenig/$(echo 
Kleine-K+APY-nig | iconv -f utf-7 -t utf-8)/'" HEAD ^HEAD^ ^HEAD^2

This destroys the mergetag, as now I have:

~/gsrc/linux$ git cat-file commit HEAD
tree 7086bbd51d06ebd0e3fcca70380d0a066efce6ef
parent e44a01fa0e0aec2f3a21dadb9c030d14831d4258
parent f719a0d6a8541b383c506aaa4b4fa6b4109669f4
author Arnd Bergmann  1444139956 +0200
committer Arnd Bergmann  1444139956 +0200

 efm32 cleanups for 4.4-rc1
 
 These are just two followup cleanups for commits that are in v3.17-rc1 
and
 waited in my private tree for application since that time.
 -BEGIN PGP SIGNATURE-
 
 iQEcBAABCgAGBQJWCtIwAAoJEMH8FHityuwJFPYH/2LJpi0yUhI01s0D9U5rZgjl
 8mLP70utAMaqXiGqP/O0ndH+za5TrrUTwDJE5jhAHru1Q6q62Ld2BfOR5+K8zlrP
 4UDGBDj3ePJgEtbSsJYJIRT+AKque2GONbeGpJh2arPrNamWChKIz8HoGSgwdRlx
 TvQuVFEa6C3ujP0ATM8x54f0vR4bYm9WmBS522DwZQY4JpGZC005A3Quw6fmFxPJ
 vJRo1fE1d2OZ3Cuxb15m8PLtZw6sbMdMwgyirAFGrJsN9u/V6BzsSzH0s8suT7IP
 WIohnmYtFSUfkQjaGrMl2wIpJSsIzettiXf9GrQDXSGZsQ4k7sW3U+qrcZ7ZUR4=
 =ngv1
 -END PGP SIGNATURE-

Merge tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux 
into next/cleanup

Merge "efm32 cleanups for 4.4-rc1" from Uwe Kleine-König:

These are just two followup cleanups for commits that are in v3.17-rc1 
and
waited in my private tree for application since that time.

* tag 'efm32-for-4.4-rc1' of git://git.pengutronix.de/git/ukl/linux:
  ARM: efm32: switch to vendor,device compatible strings
  ARM: efm32: switch to properly namespaced location property

~/gsrc/linux$ diff -u <(git cat-file commit HEAD) <(git cat-file commit 
064ca93f7ab927c2752d86fc5727716e340d737c)
--- /dev/fd/63  2015-11-02 09:02:17.310373539 +0100
+++ /dev/fd/62  2015-11-02 09:02:17.310373539 +0100
@@ -3,7 +3,11 @@
 parent f719a0d6a8541b383c506aaa4b4fa6b4109669f4
 author Arnd Bergmann  1444139956 +0200
 committer Arnd Bergmann  1444139956 +0200
-
+mergetag object f719a0d6a8541b383c506aaa4b4fa6b4109669f4
+ type commit
+ tag efm32-for-4.4-rc1
+ tagger Uwe Kleine-König  1443549594 +0200
+ 
  efm32 cleanups for 4.4-rc1
  
  These are just two followup cleanups for commits that are in 
v3.17-rc1 and
@@ -21,7 +25,7 @@
 
 Merge tag 'efm32-for-4.4-rc1' of 
git://git.pengutronix.de/git/ukl/linux into next/cleanup
 
-Merge "efm32 cleanups for 4.4-rc1" from Uwe Kleine-König:
  

Re: configure: -lpthread doesn't belong in CFLAGS

[Matthieu Moy]

"Rainer M. Canavan"  writes:

> Hi,

Hi,

Thanks for the patch. However, it will need a bit more work to be
integrated into git.git.

Please, read
https://github.com/git/git/blob/master/Documentation/SubmittingPatches

The body of your email should end up being the commit message. It is not
(yet) properly written as such. Also, read about Developer's Certificate
of Origin in the document above.

>for opt in "" -mt -pthread -lpthread; do
>   old_CFLAGS="$CFLAGS"
> - CFLAGS="$opt $CFLAGS"
> + old_LIBS="$LIBS"
> + if test "$(echo $opt | cut -b 1-2)" = -l ; then

Don't use "echo" on string that may begin with - (different versions of
echo will have different behaviors). One option is to use "printf" which
is more robust. Another one, more elegant IMHO in this case is to use a
case:

case "$opt" in
   -l*)
  # ...
  ;;
   *)
  # ...
  ;;
esac

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/2]

[Knut Franke]

Changes in the third iteration:

* don't break support for curl < 7.10.7

* fix some memory leaks

* explicitly set anyauth as fallback for unsupported proyx authmethod setting,
  and tell the user what we did

* clean up usage of curl version #ifdefs

* fix more code formatting / style / naming issues

* add in-code comment explaining the proxy variable uppercase/lowercase mess

-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Dr. Arno Steitz
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git filter-branch and merging of signed tags

[Uwe Kleine-König]

Hello Michael,

On Mon, Nov 02, 2015 at 05:21:03PM +0100, Michael J Gruber wrote:
> Uwe Kleine-König venit, vidit, dixit 02.11.2015 09:04:
> > ~/gsrc/linux$ git version
> > git version 2.6.1
> 
> Does your git include
> a5a4b3f (filter-branch: remove multi-line headers in msg filter,
> 2015-10-08)? That takes care of the incomplete header tag parsing.
No, it does not. Thanks for the hint.

> Should be in master by now, I think.
> 
> Keeping the mergetag signature would require some extra logic in
> filter-branch.
that I'd like to have :-). It seems I have to dig into that myself. (If
someone is quicker than me, I promise not to be angry :-)

Best regards
Uwe

-- 
Pengutronix e.K.   | Uwe Kleine-König|
Industrial Linux Solutions | http://www.pengutronix.de/  |
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] http: allow selection of proxy authentication method

[Knut Franke]

CURLAUTH_ANY does not work with proxies which answer unauthenticated requests
with a 307 redirect to an error page instead of a 407 listing supported
authentication methods. Therefore, allow the authentication method to be set
using the environment variable GIT_HTTP_PROXY_AUTHMETHOD or configuration
variables http.proxyAuthmethod and remote..proxyAuthmethod (in analogy
to http.proxy and remote..proxy).

The following values are supported:

* anyauth (default)
* basic
* digest
* negotiate
* ntlm

Signed-off-by: Knut Franke 
Reviewed-by: Junio C Hamano 
Reviewed-by: Eric Sunshine 
---
 Documentation/config.txt | 26 +
 http.c   | 72 ++--
 remote.c |  3 ++
 remote.h |  1 +
 4 files changed, 99 insertions(+), 3 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 391a0c3..6f29893 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1597,6 +1597,27 @@ http.proxy::
`curl(1)`).  This can be overridden on a per-remote basis; see
remote..proxy
 
+http.proxyAuthMethod::
+   Set the method with which to authenticate against the HTTP proxy. This 
only
+   takes effect if the configured proxy URI contains a user name part 
(i.e. is
+   of the form 'user@host' or 'user@host:port'). This can be overridden on 
a
+   per-remote basis; see `remote..proxyAuthMethod`. Both can be
+   overridden by the 'GIT_HTTP_PROXY_AUTHMETHOD' environment variable.
+   Possible values are:
++
+--
+* `anyauth` - Automatically pick a suitable authentication method. It is
+  assumed that the proxy answers an unauthenticated request with a 407
+  status code and one or more Proxy-authenticate headers with supported
+  authentication methods. This is the default.
+* `basic` - HTTP Basic authentication
+* `digest` - HTTP Digest authentication; this prevents the password from being
+  transmitted to the proxy in clear text
+* `negotiate` - GSS-Negotiate authentication (compare the --negotiate option
+  of `curl(1)`)
+* `ntlm` - NTLM authentication (compare the --ntlm option of `curl(1)`)
+--
+
 http.cookieFile::
File containing previously stored cookie lines which should be used
in the Git http session, if they match the server. The file format
@@ -2390,6 +2411,11 @@ remote..proxy::
the proxy to use for that remote.  Set to the empty string to
disable proxying for that remote.
 
+remote..proxyAuthMethod::
+   For remotes that require curl (http, https and ftp), the method to use 
for
+   authenticating against the proxy in use (probably set in
+   `remote..proxy`). See `http.proxyAuthMethod`.
+
 remote..fetch::
The default set of "refspec" for linkgit:git-fetch[1]. See
linkgit:git-fetch[1].
diff --git a/http.c b/http.c
index 7da76ed..1172819 100644
--- a/http.c
+++ b/http.c
@@ -63,6 +63,24 @@ static long curl_low_speed_limit = -1;
 static long curl_low_speed_time = -1;
 static int curl_ftp_no_epsv;
 static const char *curl_http_proxy;
+static const char *http_proxy_authmethod = NULL;
+static struct {
+   const char *name;
+   long curlauth_param;
+} proxy_authmethods[] = {
+   { "basic", CURLAUTH_BASIC },
+   { "digest", CURLAUTH_DIGEST },
+   { "negotiate", CURLAUTH_GSSNEGOTIATE },
+   { "ntlm", CURLAUTH_NTLM },
+#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY
+   { "anyauth", CURLAUTH_ANY },
+#endif
+   /*
+* CURLAUTH_DIGEST_IE has no corresponding command-line option in
+* curl(1) and is not included in CURLAUTH_ANY, so we leave it out
+* here, too
+*/
+};
 static const char *curl_cookie_file;
 static int curl_save_cookies;
 struct credential http_auth = CREDENTIAL_INIT;
@@ -257,6 +275,9 @@ static int http_options(const char *var, const char *value, 
void *cb)
if (!strcmp("http.proxy", var))
return git_config_string(_http_proxy, var, value);
 
+   if (!strcmp("http.proxyauthmethod", var))
+   return git_config_string(_proxy_authmethod, var, value);
+
if (!strcmp("http.cookiefile", var))
return git_config_string(_cookie_file, var, value);
if (!strcmp("http.savecookies", var)) {
@@ -305,6 +326,42 @@ static void init_curl_http_auth(CURL *result)
 #endif
 }
 
+/* assumes *var is either NULL or free-able */
+static void env_override(const char **var, const char *envname)
+{
+   const char *val = getenv(envname);
+   if (val) {
+   if (*var)
+   free((void*)*var);
+   *var = xstrdup(val);
+   }
+}
+
+static void init_curl_proxy_auth(CURL *result)
+{
+   env_override(_proxy_authmethod, "GIT_HTTP_PROXY_AUTHMETHOD");
+
+#if LIBCURL_VERSION_NUM >= 0x070a07 /* CURLOPT_PROXYAUTH and CURLAUTH_ANY */
+   if (http_proxy_authmethod) {
+ 

[PATCH 2/2] http: use credential API to handle proxy authentication

[Knut Franke]

Currently, the only way to pass proxy credentials to curl is by including them
in the proxy URL. Usually, this means they will end up on disk unencrypted, one
way or another (by inclusion in ~/.gitconfig, shell profile or history). Since
proxy authentication often uses a domain user, credentials can be security
sensitive; therefore, a safer way of passing credentials is desirable.

If the configured proxy contains a username but not a password, query the
credential API for one. Also, make sure we approve/reject proxy credentials
properly.

For consistency reasons, add parsing of http_proxy/https_proxy/all_proxy
environment variables, which would otherwise be evaluated as a fallback by curl.
Without this, we would have different semantics for git configuration and
environment variables.

Signed-off-by: Knut Franke 
Reviewed-by: Junio C Hamano 
Reviewed-by: Eric Sunshine 
---
 http.c | 76 --
 http.h |  1 +
 2 files changed, 75 insertions(+), 2 deletions(-)

diff --git a/http.c b/http.c
index 1172819..5708c7a 100644
--- a/http.c
+++ b/http.c
@@ -62,7 +62,7 @@ static const char *ssl_cainfo;
 static long curl_low_speed_limit = -1;
 static long curl_low_speed_time = -1;
 static int curl_ftp_no_epsv;
-static const char *curl_http_proxy;
+static const char *curl_http_proxy = NULL;
 static const char *http_proxy_authmethod = NULL;
 static struct {
const char *name;
@@ -81,6 +81,8 @@ static struct {
 * here, too
 */
 };
+static struct credential proxy_auth = CREDENTIAL_INIT;
+static const char *curl_proxyuserpwd = NULL;
 static const char *curl_cookie_file;
 static int curl_save_cookies;
 struct credential http_auth = CREDENTIAL_INIT;
@@ -178,6 +180,9 @@ static void finish_active_slot(struct active_request_slot 
*slot)
 #else
slot->results->auth_avail = 0;
 #endif
+
+   curl_easy_getinfo(slot->curl, CURLINFO_HTTP_CONNECTCODE,
+   >results->http_connectcode);
}
 
/* Run callback if appropriate */
@@ -339,6 +344,24 @@ static void env_override(const char **var, const char 
*envname)
 
 static void init_curl_proxy_auth(CURL *result)
 {
+   if (proxy_auth.username) {
+   if (!proxy_auth.password)
+   credential_fill(_auth);
+#if LIBCURL_VERSION_NUM >= 0x071301
+   curl_easy_setopt(result, CURLOPT_PROXYUSERNAME,
+   proxy_auth.username);
+   curl_easy_setopt(result, CURLOPT_PROXYPASSWORD,
+   proxy_auth.password);
+#else
+   struct strbuf s = STRBUF_INIT;
+   strbuf_addstr_urlencode(, proxy_auth.username, 1);
+   strbuf_addch(, ':');
+   strbuf_addstr_urlencode(, proxy_auth.password, 1);
+   curl_proxyuserpwd = strbuf_detach(, NULL);
+   curl_easy_setopt(result, CURLOPT_PROXYUSERPWD, 
curl_proxyuserpwd);
+#endif
+   }
+
env_override(_proxy_authmethod, "GIT_HTTP_PROXY_AUTHMETHOD");
 
 #if LIBCURL_VERSION_NUM >= 0x070a07 /* CURLOPT_PROXYAUTH and CURLAUTH_ANY */
@@ -520,8 +543,42 @@ static CURL *get_curl_handle(void)
curl_easy_setopt(result, CURLOPT_USE_SSL, CURLUSESSL_TRY);
 #endif
 
+   /*
+* CURL also examines these variables as a fallback; but we need to 
query
+* them here in order to decide whether to prompt for missing password 
(cf.
+* init_curl_proxy_auth()).
+*
+* Unlike many other common environment variables, these are 
historically
+* lowercase only. It appears that CURL did not know this and 
implemented
+* only uppercase variants, which was later corrected to take both - 
with
+* the exception of http_proxy, which is lowercase only also in CURL. As
+* the lowercase versions are the historical quasi-standard, they take
+* precedence here, as in CURL.
+*/
+   if (!curl_http_proxy) {
+   if (!strcmp(http_auth.protocol, "https")) {
+   env_override(_http_proxy, "HTTPS_PROXY");
+   env_override(_http_proxy, "https_proxy");
+   } else {
+   env_override(_http_proxy, "http_proxy");
+   }
+   if (!curl_http_proxy) {
+   env_override(_http_proxy, "ALL_PROXY");
+   env_override(_http_proxy, "all_proxy");
+   }
+   }
+
if (curl_http_proxy) {
-   curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
+   if (strstr(curl_http_proxy, "://"))
+   credential_from_url(_auth, curl_http_proxy);
+   else {
+   struct strbuf url = STRBUF_INIT;
+   strbuf_addf(, "http://%s;, curl_http_proxy);
+   credential_from_url(_auth, url.buf);

Contact me

[Richard Sun]

Hello,
My name is Mr. Richard Sun from Hong Kong. I want you to be my partner
in a business project. Contact me back via my private e-mail address for more
details;
ricadt...@ymail.com
Thank you
Richard Sun.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Savannah-users] Anonymous commit (empty Author and Committer)

[Kaz Kylheku]

On 02.11.2015 07:11, Lennart Sorensen wrote:

On Sat, Oct 31, 2015 at 08:24:06AM +0300, Andrei Borzenkov wrote:
If we decide to fix this commit it is better done now, while it is the 
last

one. It is annoying but do you have suggestion how it can be done
differently?


Well the only options are:

1) Leave it alone
2) Break the tree of anyone that already pulled.  (Posting instructions 
on

how to fix it would at least be nice then).

Not sure how many people regularly pull the tree.


This is silly FUD. There is no ever any "broken tree".

Firstly, people who have no local changes do not experience any ill 
effect.

They just have to do

  $ git fetch

  # oops, upstream and local branch have diverged
  # but I have absolutely no local work to preserve;
  # just "warp" HEAD to the new one:

  $ git reset --hard origin/master

  # done!

Those who have outstanding local work never have any expectation of 
non-breakage!

*All* fetches are "non-fast-forward" with regard to unpublished
local work, because they occur to a prior version of the work, 
underneath your

changes.

In *all* cases you can do this:

  # Fast-forward or not, I don't care!
  $ git fetch

  # Take my 13 unpublished commits (or whatever number), rebase them 
over the

  # latest upstream master, and make the result my local master branch:

  $ git rebase HEAD~13 --onto origin/master



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] setup: do not create $X/gitdir unnecessarily when accessing git file $X

[Junio C Hamano]

Jeff King  writes:

> [2] I suspect this code should use write_file_gently(). What happens if
> I have a read-only linked checkout?

Or you may not be the owner of the repository, you think you are
doing a read-only operation, and you silently end up creating a file
that cannot be written by the repository owner?

Honestly, I think this whole "just in case the user moved without
telling us, we sneakily fix things without telling the user" should
just go away.  This is not the first incidence of a tool trying to
be overly clever and pretend to know better than the end user biting
us, is it?
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Atousa Duprat]

In the Makefile there is the following:

ifdef BLK_SHA1
SHA1_HEADER = "block-sha1/sha1.h"
LIB_OBJS += block-sha1/sha1.o
else
ifdef PPC_SHA1
SHA1_HEADER = "ppc/sha1.h"
LIB_OBJS += ppc/sha1.o ppc/sha1ppc.o
else
ifdef APPLE_COMMON_CRYPTO
COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL
SHA1_HEADER = 
SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
else
SHA1_HEADER = 
EXTLIBS += $(LIB_4_CRYPTO)
endif

which seems to imply that BLK_SHA1 and APPLE_COMMON_CRYPTO are
mutually exclusive?

On Sun, Nov 1, 2015 at 10:37 AM, Junio C Hamano  wrote:
> atous...@gmail.com writes:
>
>> diff --git a/cache.h b/cache.h
>> index 79066e5..ec84b16 100644
>> --- a/cache.h
>> +++ b/cache.h
>> @@ -14,7 +14,12 @@
>>  #ifndef git_SHA_CTX
>>  #define git_SHA_CTX  SHA_CTX
>>  #define git_SHA1_InitSHA1_Init
>> -#define git_SHA1_Update  SHA1_Update
>> +#ifdef SHA1_MAX_BLOCK_SIZE
>> +extern int SHA1_Update_Chunked(SHA_CTX *, const void *, size_t);
>> +#define git_SHA1_Update SHA1_Update_Chunked
>> +#else
>> +#define git_SHA1_Update SHA1_Update
>> +#endif
>>  #define git_SHA1_Final   SHA1_Final
>>  #endif
>
> Hmm, I admit that this mess is my creation, but unfortunately it
> does not allow us to say:
>
> make SHA1_MAX_BLOCK_SIZE='1024L*1024L*1024L'
>
> when using other SHA-1 implementations (e.g. blk_SHA1_Update()).
>
> Ideas for cleaning it up, anybody?
>



-- 
Atousa Pahlevan, PhD
M.Math. University of Waterloo, Canada
Ph.D. Department of Computer Science, University of Victoria, Canada
Voice: 415-341-6206
Email: apahle...@ieee.org
Website: www.apahlevan.org
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Watchman/inotify support and other ways to speed up git status

[David Turner]

On Thu, 2015-10-29 at 09:10 +0100, Christian Couder wrote:
> > We're using Watchman at Twitter.  A week or two ago posted a dump of our
> > code to github, but I would advise waiting a day or two to use it, as
> > I'm about to pull a large number of bugfixes into it (I'll update this
> > thread and provide a link once I do so).
> 
> Great, I will have a look at it then!

Here's the most recent version:

https://github.com/dturner-tw/git/tree/dturner/watchman



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/5] Add watchman support to reduce index refresh cost

[David Turner]

On Sun, 2015-11-01 at 14:55 +0100, Nguyễn Thái Ngọc Duy wrote:
> The previous patch has the logic to clear bits in 'WAMA' bitmap. This
> patch has logic to set bits as told by watchman. The missing bit,
> _using_ these bits, are not here yet.
> 
> A lot of this code is written by David Turner originally, mostly from
> [1]. I'm just copying and polishing it a bit.
>
> [1] http://article.gmane.org/gmane.comp.version-control.git/248006

Our code has evolved somewhat from there[1].  It looks like you've
incorporated some of these updates.  But I wanted to call out one thing
in particular that it doesn't look like this patch handles: there's some
real ugliness on at least OSX around case-changing renames. 

You probably won't be able to use our code directly as-is, but we'll
want to do something about this situation.  This is thicket of TOCTOU
issues.  For instance, watchman learns that a file called FOO has
changed, but by the time it goes to the filesystem to look up the
canonical case, FOO has been renamed to foo already (or renamed and then
deleted).  

I won't say for sure that your code is insufficient as I haven't yet
tried it out, but we should ensure this case is handled before this is
merged.

[1] https://github.com/dturner-tw/git/tree/dturner/watchman

> +
> + pos = index_name_pos(istate, wm->name, strlen(wm->name));

This is the bit where case matters.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3] http.c: use CURLOPT_RANGE for range requests

[Jeff King]

On Mon, Nov 02, 2015 at 10:50:10PM +0100, Andreas Schwab wrote:

> Jeff King  writes:
> 
> >   4. Fix all of the callers. I suspect this would involve calling
> >  fstat(fileno(fh)) to get a real off_t.
> 
> You can also use ftello which returns off_t.

Thanks, I forgot about that function. That would probably be the sanest
path forward.

I think it is as simple as this (on top of David's patch):

-- >8 --
Subject: [PATCH] http: use off_t to store partial file size

When we try to resume transfer of a partially-downloaded
object or pack, we fopen() the existing file for append,
then use ftell() to get the current position. We use a
"long", which can hold only 2GB on a 32-bit system, even
though packfiles may be larger than that.

Let's switch to using off_t, which should hold any file size
our system is capable of storing. We need to use ftello() to
get the off_t. This is in POSIX and hopefully available
everywhere; if not, we should be able to wrap it by falling
back to ftell(), which would presumably return "-1" on such
a large file (and we would simply skip resuming in that case).

Signed-off-by: Jeff King 
---
 http.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/http.c b/http.c
index 5f36846..42f29ce 100644
--- a/http.c
+++ b/http.c
@@ -1205,7 +1205,7 @@ static int http_request(const char *url,
curl_easy_setopt(slot->curl, CURLOPT_FILE, result);
 
if (target == HTTP_REQUEST_FILE) {
-   long posn = ftell(result);
+   off_t posn = ftello(result);
curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
 fwrite);
if (posn > 0)
@@ -1581,7 +1581,7 @@ int finish_http_pack_request(struct http_pack_request 
*preq)
 struct http_pack_request *new_http_pack_request(
struct packed_git *target, const char *base_url)
 {
-   long prev_posn = 0;
+   off_t prev_posn = 0;
struct strbuf buf = STRBUF_INIT;
struct http_pack_request *preq;
 
@@ -1613,7 +1613,7 @@ struct http_pack_request *new_http_pack_request(
 * If there is data present from a previous transfer attempt,
 * resume where it left off
 */
-   prev_posn = ftell(preq->packfile);
+   prev_posn = ftello(preq->packfile);
if (prev_posn>0) {
if (http_is_verbose)
fprintf(stderr,
@@ -1668,7 +1668,7 @@ struct http_object_request *new_http_object_request(const 
char *base_url,
int prevlocal;
char prev_buf[PREV_BUF_SIZE];
ssize_t prev_read = 0;
-   long prev_posn = 0;
+   off_t prev_posn = 0;
struct http_object_request *freq;
 
freq = xcalloc(1, sizeof(*freq));
-- 
2.6.2.627.g377c1c5

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4 4/9] index-helper: new daemon for caching index and related stuff

[David Turner]

On Sun, 2015-11-01 at 14:42 +0100, Nguyễn Thái Ngọc Duy wrote:
> + memcpy(new_mmap, istate->mmap, istate->mmap_size - 20);
> +
> + /*
> +  * The trailing hash must be written last after everything is
> +  * written. It's the indication that the shared memory is now
> +  * ready.
> +  */
> + hashcpy((unsigned char *)new_mmap + istate->mmap_size - 20, is->sha1);
> +}

You need a memory barrier here.  Otherwise, compilers may reorder these
statements.

> +#define SHM_PATH_LEN 72  /* we don't create very long paths.. */
> +
> +ssize_t git_shm_map(int oflag, int perm, ssize_t length, void **mmap,
> + int prot, int flags, const char *fmt, ...)
> +{
> + va_list ap;
> + char path[SHM_PATH_LEN];
> + int fd;
> +
> + path[0] = '/';
> + va_start(ap, fmt);
> + vsprintf(path + 1, fmt, ap);
> + va_end(ap);

This would be safer with vsnprintf.


--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 18/26] refs: move transaction functions into common code

[David Turner]

On Sun, 2015-11-01 at 09:17 +0100, Michael Haggerty wrote:
> On 10/28/2015 03:14 AM, David Turner wrote:
> > The common ref code will build up a ref transaction.  Backends will
> > then commit it.  So the transaction creation and update functions should
> > be in the common code.  We also need to move the ref structs into
> > the common code so that alternate backends can access them.
> > 
> > Later, we will modify struct ref_update to support alternate backends.
> 
> I would prefer that this and later patches *not* add declarations to the
> public API in refs.h for functions and data that are only meant to be
> used by other reference backends.
> 
> So I'm working on a modified version of your series that declares such
> functions in refs-internal.h [1] instead. I hope to submit it tomorrow.

OK, I will fix up your other two issues and then wait for that series.

> Actually, I have half a mind to move all of the refs-related files to a
> subdirectory, like
> 
> refs.h
> refs/refs.c
> refs/refs-internal.h
> refs/refs-be-files.c
> refs/refs-be-lmdb.c   <- still to come
> 
> What would you think of that?

Since we're moving stuff around anyway, we might as well introduce a
little more hierarchy.  

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] setup: do not create $X/gitdir unnecessarily when accessing git file $X

[Jeff King]

On Mon, Nov 02, 2015 at 12:51:16PM -0800, Junio C Hamano wrote:

> Jeff King  writes:
> 
> > [2] I suspect this code should use write_file_gently(). What happens if
> > I have a read-only linked checkout?
> 
> Or you may not be the owner of the repository, you think you are
> doing a read-only operation, and you silently end up creating a file
> that cannot be written by the repository owner?
> 
> Honestly, I think this whole "just in case the user moved without
> telling us, we sneakily fix things without telling the user" should
> just go away.  This is not the first incidence of a tool trying to
> be overly clever and pretend to know better than the end user biting
> us, is it?

I have to admit, that was my gut feeling, too, but I do not know enough
about the problem it is solving to say whether it is a good tradeoff.
Unfortunately 23af91d102e1efaff33b77ab7746356835a3d600 did not have much
discussion. I didn't dig into the mailing list, though. I was hoping Duy
could summarize it. :)

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4] http.c: use CURLOPT_RANGE for range requests

[David Turner]

A HTTP server is permitted to return a non-range response to a HTTP
range request (and Apache httpd in fact does this in some cases).
While libcurl knows how to correctly handle this (by skipping bytes
before and after the requested range), it only turns on this handling
if it is aware that a range request is being made.  By manually
setting the range header instead of using CURLOPT_RANGE, we were
hiding the fact that this was a range request from libcurl.  This
could cause corruption.

Signed-off-by: David Turner 
---

This one incorporates Jeff's suggestions about off_t.  It also
simplifies by removing the possiblity of a missing low-end of a range;
the entire point of this function is to add a range HEADER and a
range of - is nugatory.

---
 http.c | 33 -
 http.h |  1 -
 2 files changed, 12 insertions(+), 22 deletions(-)

diff --git a/http.c b/http.c
index 6b89dea..f9a0dc5 100644
--- a/http.c
+++ b/http.c
@@ -30,7 +30,6 @@ static CURL *curl_default;
 #endif
 
 #define PREV_BUF_SIZE 4096
-#define RANGE_HEADER_SIZE 30
 
 char curl_errorstr[CURL_ERROR_SIZE];
 
@@ -692,6 +691,7 @@ struct active_request_slot *get_active_slot(void)
curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 0);
curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(slot->curl, CURLOPT_FAILONERROR, 1);
+   curl_easy_setopt(slot->curl, CURLOPT_RANGE, NULL);
 #ifdef LIBCURL_CAN_HANDLE_AUTH_ANY
curl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, http_auth_methods);
 #endif
@@ -1184,6 +1184,13 @@ static const char *get_accept_language(void)
return cached_accept_language;
 }
 
+static void http_opt_request_remainder(CURL *curl, off_t pos)
+{
+   char buf[128];
+   xsnprintf(buf, sizeof(buf), "%"PRIuMAX"-", (uintmax_t)pos);
+   curl_easy_setopt(curl, CURLOPT_RANGE, buf);
+}
+
 /* http_request() targets */
 #define HTTP_REQUEST_STRBUF0
 #define HTTP_REQUEST_FILE  1
@@ -1212,11 +1219,8 @@ static int http_request(const char *url,
long posn = ftell(result);
curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
 fwrite);
-   if (posn > 0) {
-   strbuf_addf(, "Range: bytes=%ld-", posn);
-   headers = curl_slist_append(headers, buf.buf);
-   strbuf_reset();
-   }
+   if (posn > 0)
+   http_opt_request_remainder(slot->curl, posn);
} else
curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION,
 fwrite_buffer);
@@ -1526,10 +1530,6 @@ void release_http_pack_request(struct http_pack_request 
*preq)
fclose(preq->packfile);
preq->packfile = NULL;
}
-   if (preq->range_header != NULL) {
-   curl_slist_free_all(preq->range_header);
-   preq->range_header = NULL;
-   }
preq->slot = NULL;
free(preq->url);
free(preq);
@@ -1593,7 +1593,6 @@ struct http_pack_request *new_http_pack_request(
struct packed_git *target, const char *base_url)
 {
long prev_posn = 0;
-   char range[RANGE_HEADER_SIZE];
struct strbuf buf = STRBUF_INIT;
struct http_pack_request *preq;
 
@@ -1631,10 +1630,7 @@ struct http_pack_request *new_http_pack_request(
fprintf(stderr,
"Resuming fetch of pack %s at byte %ld\n",
sha1_to_hex(target->sha1), prev_posn);
-   xsnprintf(range, sizeof(range), "Range: bytes=%ld-", prev_posn);
-   preq->range_header = curl_slist_append(NULL, range);
-   curl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER,
-   preq->range_header);
+   http_opt_request_remainder(preq->slot->curl, prev_posn);
}
 
return preq;
@@ -1684,8 +1680,6 @@ struct http_object_request *new_http_object_request(const 
char *base_url,
char prev_buf[PREV_BUF_SIZE];
ssize_t prev_read = 0;
long prev_posn = 0;
-   char range[RANGE_HEADER_SIZE];
-   struct curl_slist *range_header = NULL;
struct http_object_request *freq;
 
freq = xcalloc(1, sizeof(*freq));
@@ -1791,10 +1785,7 @@ struct http_object_request 
*new_http_object_request(const char *base_url,
fprintf(stderr,
"Resuming fetch of object %s at byte %ld\n",
hex, prev_posn);
-   xsnprintf(range, sizeof(range), "Range: bytes=%ld-", prev_posn);
-   range_header = curl_slist_append(range_header, range);
-   curl_easy_setopt(freq->slot->curl,
-CURLOPT_HTTPHEADER, range_header);
+   

Re: [PATCH v3] http.c: use CURLOPT_RANGE for range requests

[Andreas Schwab]

Jeff King  writes:

>   4. Fix all of the callers. I suspect this would involve calling
>  fstat(fileno(fh)) to get a real off_t.

You can also use ftello which returns off_t.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] http.c: use CURLOPT_RANGE for range requests

[Jeff King]

On Mon, Nov 02, 2015 at 04:39:58PM -0500, David Turner wrote:

> A HTTP server is permitted to return a non-range response to a HTTP
> range request (and Apache httpd in fact does this in some cases).
> While libcurl knows how to correctly handle this (by skipping bytes
> before and after the requested range), it only turns on this handling
> if it is aware that a range request is being made.  By manually
> setting the range header instead of using CURLOPT_RANGE, we were
> hiding the fact that this was a range request from libcurl.  This
> could cause corruption.
> 
> Signed-off-by: David Turner 
> ---
> 
> This one incorporates Jeff's suggestions about off_t.  It also
> simplifies by removing the possiblity of a missing low-end of a range;
> the entire point of this function is to add a range HEADER and a
> range of - is nugatory.

Thanks, looks good to me. And I learned a new vocabulary word. :)

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git.git as of tonight

[Junio C Hamano]

Johannes Sixt  writes:

> Am 02.11.2015 um 03:58 schrieb Junio C Hamano:
>> * sb/submodule-parallel-fetch (2015-10-21) 14 commits
>>(merged to 'next' on 2015-10-23 at 8f04bbd)
>>   + run-command: fix missing output from late callbacks
>>...
>>   + submodule.c: write "Fetching submodule " to stderr
>>   (this branch is used by rs/daemon-leak-fix and 
>> sb/submodule-parallel-update.)
>> 
>>   Add a framework to spawn a group of processes in parallel, and use
>>   it to run "git fetch --recurse-submodules" in parallel.
>> 
>>   Will merge to 'master'.
>
> Please don't, yet. This series does not build on Windows:

The only reason the series is listed here is because the cycle is
still young and I was hoping that any fallout will be addressed by
the time we tag -rc0; if the extent of required fixups is too great,
that obviously would not work well.

I'll try to see if I can untangle rs/daemon-leak-fix topic so that
it does not depend on this thing and have it graduate separately.

Thanks for stopping me.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git.git as of tonight

[Johannes Sixt]

Am 02.11.2015 um 03:58 schrieb Junio C Hamano:
> * sb/submodule-parallel-fetch (2015-10-21) 14 commits
>(merged to 'next' on 2015-10-23 at 8f04bbd)
>   + run-command: fix missing output from late callbacks
>   + test-run-command: increase test coverage
>   + test-run-command: test for gracefully aborting
>   + run-command: initialize the shutdown flag
>   + run-command: clear leftover state from child_process structure
>   + run-command: fix early shutdown
>(merged to 'next' on 2015-10-15 at df63590)
>   + submodules: allow parallel fetching, add tests and documentation
>   + fetch_populated_submodules: use new parallel job processing
>   + run-command: add an asynchronous parallel child processor
>   + sigchain: add command to pop all common signals
>   + strbuf: add strbuf_read_once to read without blocking
>   + xread_nonblock: add functionality to read from fds without blocking
>   + xread: poll on non blocking fds
>   + submodule.c: write "Fetching submodule " to stderr
>   (this branch is used by rs/daemon-leak-fix and 
> sb/submodule-parallel-update.)
> 
>   Add a framework to spawn a group of processes in parallel, and use
>   it to run "git fetch --recurse-submodules" in parallel.
> 
>   Will merge to 'master'.

Please don't, yet. This series does not build on Windows:

run-command.c: In function 'set_nonblocking':
run-command.c:1011: error: 'F_GETFL' undeclared (first use in this function)
run-command.c:1011: error: (Each undeclared identifier is reported only once
run-command.c:1011: error: for each function it appears in.)
run-command.c:1015: error: 'F_SETFL' undeclared (first use in this function)
run-command.c:1015: error: 'O_NONBLOCK' undeclared (first use in this function)
make: *** [run-command.o] Error 1

I have to investigate whether we can have some sort of Posixy
non-blocking IO on Windows or whether we have to opt-out from this
parallel-process facility. Any help from Windows experts would be
appreciated.

-- Hannes

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Limit the size of the data block passed to SHA1_Update()

[Junio C Hamano]

Atousa Duprat  writes:

> On Sun, Nov 1, 2015 at 10:37 AM, Junio C Hamano  wrote:
>>
>> Hmm, I admit that this mess is my creation, but unfortunately it
>> does not allow us to say:
>>
>> make SHA1_MAX_BLOCK_SIZE='1024L*1024L*1024L'
>>
>> when using other SHA-1 implementations (e.g. blk_SHA1_Update()).
>>
>> Ideas for cleaning it up, anybody?
>>
> In the Makefile there is the following:
>
> ifdef BLK_SHA1
> SHA1_HEADER = "block-sha1/sha1.h"
> LIB_OBJS += block-sha1/sha1.o
> else
> ifdef PPC_SHA1
> SHA1_HEADER = "ppc/sha1.h"
> LIB_OBJS += ppc/sha1.o ppc/sha1ppc.o
> else
> ifdef APPLE_COMMON_CRYPTO
> COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL
> SHA1_HEADER = 
> SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
> else
> SHA1_HEADER = 
> EXTLIBS += $(LIB_4_CRYPTO)
> endif
>
> which seems to imply that BLK_SHA1 and APPLE_COMMON_CRYPTO are
> mutually exclusive?

Yes, you are correct that these two cannot be used at the same time.
In general (not limited to BLK_SHA1 and APPLE_COMMON_CRYPTO) you can
pick only _one_ underlying SHA-1 implementation to use with the
system.

If you use APPLE_COMMON_CRYPTO, you may have to use the Chunked
thing, because of APPLE_COMMON_CRYPTO implementation's limitation.

But the above two facts taken together does not have to imply that
you are forbidden from choosing to use Chunked thing if you are
using BLK_SHA1 or OpenSSL's SHA-1 implementation.  If only for
making sure that the Chunked wrapper passes compilation test, for
trying it out to see how well it works, or just for satisfying
curiosity, it would be nice if we allowed such a combination.

The original arrangement of macro was:

 * The user code uses git_SHA1_Update()

 * cache.h renames git_SHA1_Update() to refer to the underlying
   SHA1_Update() function, either from OpenSSL or AppleCommonCrypto,
   or block-sha1/sha1.h renames git_SHA1_Update() to refer to our
   implementation blk_SHA1_Update().

What we want with Chunked is:

 * The user code uses git_SHA1_Update(); we must not change this, as
   there are many existing calls.

 * We want git_SHA1_Update() to call the Chunked thing when
   SHA1_MAX_BLOCK_SIZE is set.

 * The Chunked thing must delegate the actual hashing to underlying
   SHA1_Update(), either from OpenSSL or AppleCommonCrypto.  If we
   are using BLK_SHA1, we want the Chunked thing to instead call
   blk_SHA1_Update().

I do not seem to be able to find a way to do this with the current
two-level indirection.  If we added another level, we can.

* In cache.h, define platform_SHA1_Update() to refer to
  SHA1_Update() from the platform (unless block-sha1/ is used).
  git_SHA1_Update() in the user code may directly call it, or it may
  go to the Chunked thing.

  #ifndef git_SHA1_CTX
  #define platform_SHA1_Update  SHA1_Update
  #endif

  #ifdef SHA1_MAX_BLOCK_SIZE
  #define git_SHA1_Update   git_SHA1_Update_Chunked
  #else
  #define git_SHA1_Update   platform_SHA1_Update
  #endif

* In block-sha1/sha1.h, redirect platform_SHA1_Update() to
  blk_SHA1_Update().

  #define platform_SHA1_Update  blk_SHA1_Update

* In compat/sha1_chunked.c, implement the Chunked thing in terms of
  the platform_SHA1_Update():

  git_SHA1_Update_Chunked(...)
  {
...
while (...) {
platform_SHA1_Update(...);
}
  }


I am not sure if the above is worth it, but I suspect the damage is
localized enough that this may be OK.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/5] read-cache: add watchman 'WAMA' extension

[David Turner]

On Sun, 2015-11-01 at 14:55 +0100, Nguyễn Thái Ngọc Duy wrote:
>
>+#define CE_NO_WATCH  (0x0001)

This name seems very confusing to me.  CE_NO_WATCHMAN_STAT?
CE_UNKNOWN_TO_WATCHMAN?  

(one reason it may seem more confusing to me than to others is that
Twitter's code has a concept of files that we don't watch at all e.g.
Intellij's .idea dir).  

> @@ -322,6 +325,7 @@ struct index_state {
>   struct untracked_cache *untracked;
>   void *mmap;
>   size_t mmap_size;
> + char *last_update;

Might be worth a comment explaining what this is.


--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git.git as of tonight

[Stefan Beller]

On Mon, Nov 2, 2015 at 1:15 PM, Johannes Sixt  wrote:
> Am 02.11.2015 um 03:58 schrieb Junio C Hamano:
>> * sb/submodule-parallel-fetch (2015-10-21) 14 commits
>>(merged to 'next' on 2015-10-23 at 8f04bbd)
>>   + run-command: fix missing output from late callbacks
>>   + test-run-command: increase test coverage
>>   + test-run-command: test for gracefully aborting
>>   + run-command: initialize the shutdown flag
>>   + run-command: clear leftover state from child_process structure
>>   + run-command: fix early shutdown
>>(merged to 'next' on 2015-10-15 at df63590)
>>   + submodules: allow parallel fetching, add tests and documentation
>>   + fetch_populated_submodules: use new parallel job processing
>>   + run-command: add an asynchronous parallel child processor
>>   + sigchain: add command to pop all common signals
>>   + strbuf: add strbuf_read_once to read without blocking
>>   + xread_nonblock: add functionality to read from fds without blocking
>>   + xread: poll on non blocking fds
>>   + submodule.c: write "Fetching submodule " to stderr
>>   (this branch is used by rs/daemon-leak-fix and 
>> sb/submodule-parallel-update.)
>>
>>   Add a framework to spawn a group of processes in parallel, and use
>>   it to run "git fetch --recurse-submodules" in parallel.
>>
>>   Will merge to 'master'.
>
> Please don't, yet. This series does not build on Windows:
>
> run-command.c: In function 'set_nonblocking':
> run-command.c:1011: error: 'F_GETFL' undeclared (first use in this function)
> run-command.c:1011: error: (Each undeclared identifier is reported only once
> run-command.c:1011: error: for each function it appears in.)
> run-command.c:1015: error: 'F_SETFL' undeclared (first use in this function)
> run-command.c:1015: error: 'O_NONBLOCK' undeclared (first use in this 
> function)
> make: *** [run-command.o] Error 1

Going by a quick search http://stackoverflow.com/a/22756664
I'd hope we only need to modify the set_nonblocking function using #ifdefs ?

>
> I have to investigate whether we can have some sort of Posixy
> non-blocking IO on Windows or whether we have to opt-out from this
> parallel-process facility. Any help from Windows experts would be
> appreciated.
>
> -- Hannes
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3] http.c: use CURLOPT_RANGE for range requests

[Jeff King]

On Mon, Nov 02, 2015 at 05:41:24PM -0500, David Turner wrote:

> > Let's switch to using off_t, which should hold any file size
> > our system is capable of storing. We need to use ftello() to
> > get the off_t. This is in POSIX and hopefully available
> > everywhere; if not, we should be able to wrap it by falling
> > back to ftell(), which would presumably return "-1" on such
> > a large file (and we would simply skip resuming in that case).
> 
> It would skip resuming, but would still maybe write to the end of the
> existing file, right?  So I think we would need to seek to the beginning
> of the file in that case.

Oh, you're right. That's no worse than the current behavior, but it's
definitely not ideal. I'd leave that for a follow-on patch that actually
implements such an ftello wrapper.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3] http.c: use CURLOPT_RANGE for range requests

[David Turner]

On Mon, 2015-11-02 at 17:10 -0500, Jeff King wrote:
> On Mon, Nov 02, 2015 at 10:50:10PM +0100, Andreas Schwab wrote:
> 
> > Jeff King  writes:
> > 
> > >   4. Fix all of the callers. I suspect this would involve calling
> > >  fstat(fileno(fh)) to get a real off_t.
> > 
> > You can also use ftello which returns off_t.
> 
> Thanks, I forgot about that function. That would probably be the sanest
> path forward.
> 
> I think it is as simple as this (on top of David's patch):
> 
> -- >8 --
> Subject: [PATCH] http: use off_t to store partial file size
> 
> When we try to resume transfer of a partially-downloaded
> object or pack, we fopen() the existing file for append,
> then use ftell() to get the current position. We use a
> "long", which can hold only 2GB on a 32-bit system, even
> though packfiles may be larger than that.
> 
> Let's switch to using off_t, which should hold any file size
> our system is capable of storing. We need to use ftello() to
> get the off_t. This is in POSIX and hopefully available
> everywhere; if not, we should be able to wrap it by falling
> back to ftell(), which would presumably return "-1" on such
> a large file (and we would simply skip resuming in that case).

It would skip resuming, but would still maybe write to the end of the
existing file, right?  So I think we would need to seek to the beginning
of the file in that case.

(other than that, lgtm)

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] http: use credential API to handle proxy authentication

[Junio C Hamano]

Knut Franke  writes:

> Currently, the only way to pass proxy credentials to curl is by including them
> in the proxy URL. Usually, this means they will end up on disk unencrypted, 
> one
> way or another (by inclusion in ~/.gitconfig, shell profile or history). Since
> proxy authentication often uses a domain user, credentials can be security
> sensitive; therefore, a safer way of passing credentials is desirable.
>
> If the configured proxy contains a username but not a password, query the
> credential API for one. Also, make sure we approve/reject proxy credentials
> properly.
>
> For consistency reasons, add parsing of http_proxy/https_proxy/all_proxy
> environment variables, which would otherwise be evaluated as a fallback by 
> curl.
> Without this, we would have different semantics for git configuration and
> environment variables.
>
> Signed-off-by: Knut Franke 
> Reviewed-by: Junio C Hamano 
> Reviewed-by: Eric Sunshine 

As 1/2, I never reviewed this version yet.

> ---
>  http.c | 76 
> --
>  http.h |  1 +
>  2 files changed, 75 insertions(+), 2 deletions(-)
>
> diff --git a/http.c b/http.c
> index 1172819..5708c7a 100644
> --- a/http.c
> +++ b/http.c
> @@ -62,7 +62,7 @@ static const char *ssl_cainfo;
>  static long curl_low_speed_limit = -1;
>  static long curl_low_speed_time = -1;
>  static int curl_ftp_no_epsv;
> -static const char *curl_http_proxy;
> +static const char *curl_http_proxy = NULL;
>  static const char *http_proxy_authmethod = NULL;

We do not do unnecessary initialization of file-scope globals to 0
or NULL.  The existing definition of curl_http_proxy is correct;
the one for http_proxy_authmethod needs to be changed to match.

>  static void init_curl_proxy_auth(CURL *result)
>  {
> + if (proxy_auth.username) {
> + if (!proxy_auth.password)
> + credential_fill(_auth);
> +#if LIBCURL_VERSION_NUM >= 0x071301
> + curl_easy_setopt(result, CURLOPT_PROXYUSERNAME,
> + proxy_auth.username);
> + curl_easy_setopt(result, CURLOPT_PROXYPASSWORD,
> + proxy_auth.password);
> +#else
> + struct strbuf s = STRBUF_INIT;
> + strbuf_addstr_urlencode(, proxy_auth.username, 1);
> + strbuf_addch(, ':');
> + strbuf_addstr_urlencode(, proxy_auth.password, 1);
> + curl_proxyuserpwd = strbuf_detach(, NULL);
> + curl_easy_setopt(result, CURLOPT_PROXYUSERPWD, 
> curl_proxyuserpwd);
> +#endif

I think #else clause of this thing would introduce decl-after-stmt
compilation error.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Strange diff-index output

[Ch'Gans]

Hi there,

We're using a script to verify that what we are building is clean
against our git repository, for this we're using "git rev-parse
--short HEAD" to get the current hash and "git diff-index --quiet
HEAD" to check for local modification.
This script works fine on developer's machines, but doesn't on our CI
system (Atlassian Bamboo/Stash).

HEAD points to refs/heads/FIX-XYZ and refs/heads/FIX-XYZ exists.

On our CI, the output of diff-index gives something like:
:100644 100644 abcdef 00 M file1

Our problem is very likely related with how Bamboo works and/or how we
use it, but to investigate this issue further, I first would like to
understand the meaning of the above output.

>From the man page of diff-index:
- 100644 is the mode of "src"
- 100644 is the mode of dst
- abcdef is the hash of src
- 00 is the hash of dst
- M stands for "Modified"
- file1 is the filename being considered.

>From my understanding, src here means the file system and dst the git
object. So the above should mean "file1" is new and out of sync with
the index, which doesn't make sense since "file1" is definitely in the
repository. The weird stuff is that diff-index report the same problem
on every single file no just a few!

Notes:
 - Something I've noticed with Bamboo is that the cloned repository
doesn't have any remotes. Although I'm not sure if it is relevant
here.
 - I have tried with "shallow copy" Bamboo option enabled or not and I
get the same problems
 - I have tried "git update-index", but still get the same results
 - git log still gives me the full log history (but no remote heads)
 - git status reports nothing else than a couple of generated files
 - git status HEAD reports that the working directory is clean
 - git diff-index --cached reports nothing

Does any of you have ever come across this kind of problem? Could
anyone shed some light on what's going on with our clone? Any
documentation point out or hints appreciated.

Thanks,
Chris
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] http: allow selection of proxy authentication method

[Junio C Hamano]

Knut Franke  writes:

> CURLAUTH_ANY does not work with proxies which answer unauthenticated requests
> with a 307 redirect to an error page instead of a 407 listing supported
> authentication methods. Therefore, allow the authentication method to be set
> using the environment variable GIT_HTTP_PROXY_AUTHMETHOD or configuration
> variables http.proxyAuthmethod and remote..proxyAuthmethod (in analogy
> to http.proxy and remote..proxy).
>
> The following values are supported:
>
> * anyauth (default)
> * basic
> * digest
> * negotiate
> * ntlm
>
> Signed-off-by: Knut Franke 


> Reviewed-by: Junio C Hamano 
> Reviewed-by: Eric Sunshine 

Please add these only when you are doing the final submission,
sending the same version reviewed by these people after they said
the patch(es) look good.  To credit others for helping you to polish
your patch, Helped-by: would be more appropriate.

> @@ -305,6 +326,42 @@ static void init_curl_http_auth(CURL *result)
>  #endif
>  }
>  
> +/* assumes *var is either NULL or free-able */
> +static void env_override(const char **var, const char *envname)
> +{
> + const char *val = getenv(envname);
> + if (val) {
> + if (*var)
> + free((void*)*var);

Just
free((void *)*var);

would be more idiomatic (freeing NULL is not a crime but a norm).
Also as you did elsewhere, have a space between void and the
asterisk.

> +static void init_curl_proxy_auth(CURL *result)
> +{
> + env_override(_proxy_authmethod, "GIT_HTTP_PROXY_AUTHMETHOD");

Shouldn't this also be part of the #if/#endif?

> +
> +#if LIBCURL_VERSION_NUM >= 0x070a07 /* CURLOPT_PROXYAUTH and CURLAUTH_ANY */
> + if (http_proxy_authmethod) {
> + int i;
> + for (i = 0; i < ARRAY_SIZE(proxy_authmethods); i++) {
> + if (!strcmp(http_proxy_authmethod, 
> proxy_authmethods[i].name)) {
> + curl_easy_setopt(result, CURLOPT_PROXYAUTH,
> + 
> proxy_authmethods[i].curlauth_param);
> + break;
> + }
> + }
> + if (i == ARRAY_SIZE(proxy_authmethods)) {
> + warning("unsupported proxy authentication method %s: 
> using anyauth",
> + http_proxy_authmethod);
> + curl_easy_setopt(result, CURLOPT_PROXYAUTH, 
> CURLAUTH_ANY);
> + }
> + }
> + else
> + curl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);
> +#endif
> +}
> +
>  static int has_cert_password(void)
>  {
>   if (ssl_cert == NULL || ssl_cert_password_required != 1)
> @@ -466,9 +523,7 @@ static CURL *get_curl_handle(void)
>   if (curl_http_proxy) {
>   curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
>   }
> -#if LIBCURL_VERSION_NUM >= 0x070a07
> - curl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);
> -#endif
> + init_curl_proxy_auth(result);
>  
>   set_curl_keepalive(result);
>  
> @@ -509,6 +564,12 @@ void http_init(struct remote *remote, const char *url, 
> int proactive_auth)
>   if (remote && remote->http_proxy)
>   curl_http_proxy = xstrdup(remote->http_proxy);
>  
> + if (remote && remote->http_proxy_authmethod) {
> + if (http_proxy_authmethod)
> + free((void*)http_proxy_authmethod);

Just
free((void *)http_proxy_authmethod);

without NULL-ness check.

But this makes me wonder if env_override() was a good abstraction.

That is, with this helper:

/* existing value in *var must be freeable */
static void var_override(const char **var, char *value)
{
if (value) {
free((void *)(*var));
var = xstrdup(value);
}
}

the beginning of the init_proxy_auth() would become:

static void init_curl_proxy_auth(CURL *result)
{
#if LIBCURL_VERSION_NUM >= 0x070a07 /* CURLOPT_PROXYAUTH and 
CURLAUTH_ANY */
var_override(_proxy_authmethod, 
getenv("GIT_HTTP_PROXY_AUTHMETHOD"));
...

and this code would be:

if (remote)
var_override(_proxy_authmethod, 
remote->http_proxy_authmethod);

which might be even cleaner.

> + http_proxy_authmethod = xstrdup(remote->http_proxy_authmethod);
> + }
> +
>   pragma_header = curl_slist_append(pragma_header, "Pragma: no-cache");
>   no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
>  
> @@ -607,6 +668,11 @@ void http_cleanup(void)
>   curl_http_proxy = NULL;
>   }
>  
> + if (http_proxy_authmethod) {
> + free((void *)http_proxy_authmethod);
> + http_proxy_authmethod = NULL;
> + }

No need for NULL-ness check here, either.

Re: Cant download Git for Mac

[Xue Fuqiao]

On Tue, Nov 3, 2015 at 3:18 AM, MH  wrote:
> I am trying to download Git and when I try it says “downloading” but it isn’t/

Yes, I can reproduce on Firefox 41.0.2 and Google Chrome 46.0.2490.80.

> There is an option to download from source forge but I have been told that is 
> not a good idea.

Why isn't it a good idea?  IMHO they are the same.  See:

* 
https://github.com/git/git-scm.com/blob/master/app/controllers/downloads_controller.rb
* https://github.com/git/git-scm.com/blob/master/lib/tasks/downloads.rake
* 
https://github.com/git/git-scm.com/blob/master/app/views/downloads/installers/index.html.haml

> Any reason Git is not downloading from git.scm/downloads?

I don't know.  But this mailing list is not the right place to report
issues of git-scm.com.  Please report it here:
https://github.com/git/git-scm.com/issues/new

Thank you.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Strange diff-index output

[David Turner]

On Tue, 2015-11-03 at 12:59 +1300, Ch'Gans wrote:
> Hi there,
> 
> We're using a script to verify that what we are building is clean
> against our git repository, for this we're using "git rev-parse
> --short HEAD" to get the current hash and "git diff-index --quiet
> HEAD" to check for local modification.

See this note from the git diff-index man page:
   Note
   As with other commands of this type, git diff-index does not
   actually look at the contents of the file at all. So maybe
   kernel/sched.c hasn’t actually changed, and it’s just that
you
   touched it. In either case, it’s a note that you need to git
   update-index it to make the index be in sync.

You can either add an update-index to your script, or use git status
--porcelain.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Strange diff-index output

[Ch'Gans]

On 3 November 2015 at 13:27, David Turner  wrote:
> On Tue, 2015-11-03 at 12:59 +1300, Ch'Gans wrote:
>> Hi there,
>>
>> We're using a script to verify that what we are building is clean
>> against our git repository, for this we're using "git rev-parse
>> --short HEAD" to get the current hash and "git diff-index --quiet
>> HEAD" to check for local modification.
>
> See this note from the git diff-index man page:
>Note
>As with other commands of this type, git diff-index does not
>actually look at the contents of the file at all. So maybe
>kernel/sched.c hasn’t actually changed, and it’s just that
> you
>touched it. In either case, it’s a note that you need to git
>update-index it to make the index be in sync.
>
> You can either add an update-index to your script, or use git status
> --porcelain.

Hi David,

I first tried "git update-index" but it didn't work. However "git
update-index --refresh" seems to fix our problem.
I didn't get why "--refresh" is needed thought, I'm really not
familiar with the caching aspect of git.
Anyway, I think that this fix is what I need right now.

Thanks,
Chris

>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/4] diff-highlight: add `less -r` to cmd in README

[Junio C Hamano]

Jonathan Lebon  writes:

> As it is, the suggested command for trying out diff-highlight will just
> dump the whole git log output to the terminal. Let's pipe it through
> `less` so users aren't surprised on the first try.

That justifies the "less" part but not your choice of "-r".

I am assuming that you are telling "less" not to show the ANSI
"color" escape sequences using the caret notation with "-r", which
is a very natural and sensible thing to do when using `highlight`.

But if that is the case, you don't want "-r" (raw control chars for
everything).  You would want to say "-R", I think.

Other than that, looks like a sensible thing to do to me.

Thanks.


> Signed-off-by: Jonathan Lebon 
> ---
>  contrib/diff-highlight/README | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/contrib/diff-highlight/README b/contrib/diff-highlight/README
> index 836b97a..bbbfdda 100644
> --- a/contrib/diff-highlight/README
> +++ b/contrib/diff-highlight/README
> @@ -44,9 +44,9 @@ Use
>  
>  You can try out the diff-highlight program with:
>  
> --
> -git log -p --color | /path/to/diff-highlight
> --
> +--
> +git log -p --color | /path/to/diff-highlight | less -r
> +--
>  
>  If you want to use it all the time, drop it in your $PATH and put the
>  following in your git configuration:
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/4] diff-highlight: add `less -r` to cmd in README

[Jonathan Lebon]

On Mon, Nov 2, 2015 at 9:41 PM, Junio C Hamano  wrote:
>
> Jonathan Lebon  writes:
>
> > As it is, the suggested command for trying out diff-highlight will just
> > dump the whole git log output to the terminal. Let's pipe it through
> > `less` so users aren't surprised on the first try.
>
> That justifies the "less" part but not your choice of "-r".
>
> I am assuming that you are telling "less" not to show the ANSI
> "color" escape sequences using the caret notation with "-r", which
> is a very natural and sensible thing to do when using `highlight`.
>
> But if that is the case, you don't want "-r" (raw control chars for
> everything).  You would want to say "-R", I think.

Ahh thanks, that makes sense. I will update this for v2 tomorrow.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] diff-highlight: add maxhunksize config option

[Jonathan Lebon]

As the size of the hunk gets bigger, it becomes harder to jump back and
forth between the removed and added lines, and highlighting becomes less
beneficial. We add a new config option called

diff-highlight.maxhunksize

which controls the maximum size of the hunk allowed for which
highlighting is still performed. The default value is set to 20.

Signed-off-by: Jonathan Lebon 
---
 contrib/diff-highlight/README | 20 
 contrib/diff-highlight/diff-highlight | 16 
 2 files changed, 36 insertions(+)

diff --git a/contrib/diff-highlight/README b/contrib/diff-highlight/README
index 885ff2f..ed12be1 100644
--- a/contrib/diff-highlight/README
+++ b/contrib/diff-highlight/README
@@ -89,6 +89,26 @@ newHighlight = "black #aaffaa"
 -
 
 
+Max Hunk Config
+---
+
+By default, diff-highlight will not do any highlighting if either the
+number of removed or added lines is greater than 20. This is because as
+the hunk gets bigger, it becomes harder to jump back and forth between
+the removed and added lines, and highlighting becomes less beneficial.
+
+You can change this default by setting the "diff-highlight.maxhunksize"
+configuration.
+
+Example:
+
+-
+# Increase the maximum diff-highlight to 30
+[diff-highlight]
+maxhunksize = 30
+-
+
+
 Bugs
 
 
diff --git a/contrib/diff-highlight/diff-highlight 
b/contrib/diff-highlight/diff-highlight
index 46556fc..a005146 100755
--- a/contrib/diff-highlight/diff-highlight
+++ b/contrib/diff-highlight/diff-highlight
@@ -17,6 +17,8 @@ my @NEW_HIGHLIGHT = (
color_config('color.diff-highlight.newreset', $OLD_HIGHLIGHT[2])
 );
 
+my $MAX_HUNK_SIZE = config('diff-highlight.maxhunksize', 20);
+
 my $RESET = "\x1b[m";
 my $COLOR = qr/\x1b\[[0-9;]*m/;
 my $BORING = qr/$COLOR|\s/;
@@ -79,6 +81,13 @@ sub color_config {
return length($s) ? $s : $default;
 }
 
+# Also handle our own fallback here to be independent.
+sub config {
+   my ($key, $default) = @_;
+   my $s = `git config --get $key 2>/dev/null`;
+   return length($s) ? $s : $default;
+}
+
 sub show_hunk {
my ($a, $b) = @_;
 
@@ -88,6 +97,13 @@ sub show_hunk {
return;
}
 
+   # Skip highlighting if the hunk gets bigger than the user configured
+   # limit.
+   if (@$a > $MAX_HUNK_SIZE || @$b > $MAX_HUNK_SIZE) {
+   print @$a, @$b;
+   return;
+   }
+
my @queue;
match_and_highlight_pairs($a, 0, scalar @$a, $b, 0, scalar @$b, 
\@queue);
print @queue;
-- 
2.6.0

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/4] diff-highlight: add `less -r` to cmd in README

[Jonathan Lebon]

As it is, the suggested command for trying out diff-highlight will just
dump the whole git log output to the terminal. Let's pipe it through
`less` so users aren't surprised on the first try.

Signed-off-by: Jonathan Lebon 
---
 contrib/diff-highlight/README | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/contrib/diff-highlight/README b/contrib/diff-highlight/README
index 836b97a..bbbfdda 100644
--- a/contrib/diff-highlight/README
+++ b/contrib/diff-highlight/README
@@ -44,9 +44,9 @@ Use
 
 You can try out the diff-highlight program with:
 
--
-git log -p --color | /path/to/diff-highlight
--
+--
+git log -p --color | /path/to/diff-highlight | less -r
+--
 
 If you want to use it all the time, drop it in your $PATH and put the
 following in your git configuration:
-- 
2.6.0

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/4] diff-highlight: factor out prefix/suffix functions

[Jonathan Lebon]

In preparation for the next patch, we factor out the functions for
finding the common prefix and suffix between two lines.

Signed-off-by: Jonathan Lebon 
---
 contrib/diff-highlight/diff-highlight | 98 ---
 1 file changed, 56 insertions(+), 42 deletions(-)

diff --git a/contrib/diff-highlight/diff-highlight 
b/contrib/diff-highlight/diff-highlight
index ffefc31..a332f86 100755
--- a/contrib/diff-highlight/diff-highlight
+++ b/contrib/diff-highlight/diff-highlight
@@ -110,48 +110,8 @@ sub highlight_pair {
my @a = split_line(shift);
my @b = split_line(shift);
 
-   # Find common prefix, taking care to skip any ansi
-   # color codes.
-   my $seen_plusminus;
-   my ($pa, $pb) = (0, 0);
-   while ($pa < @a && $pb < @b) {
-   if ($a[$pa] =~ /$COLOR/) {
-   $pa++;
-   }
-   elsif ($b[$pb] =~ /$COLOR/) {
-   $pb++;
-   }
-   elsif ($a[$pa] eq $b[$pb]) {
-   $pa++;
-   $pb++;
-   }
-   elsif (!$seen_plusminus && $a[$pa] eq '-' && $b[$pb] eq '+') {
-   $seen_plusminus = 1;
-   $pa++;
-   $pb++;
-   }
-   else {
-   last;
-   }
-   }
-
-   # Find common suffix, ignoring colors.
-   my ($sa, $sb) = ($#a, $#b);
-   while ($sa >= $pa && $sb >= $pb) {
-   if ($a[$sa] =~ /$COLOR/) {
-   $sa--;
-   }
-   elsif ($b[$sb] =~ /$COLOR/) {
-   $sb--;
-   }
-   elsif ($a[$sa] eq $b[$sb]) {
-   $sa--;
-   $sb--;
-   }
-   else {
-   last;
-   }
-   }
+   my ($pa, $pb) = find_common_prefix(\@a, \@b);
+   my ($sa, $sb) = find_common_suffix(\@a, $pa, \@b, $pb);
 
if (is_pair_interesting(\@a, $pa, $sa, \@b, $pb, $sb)) {
return highlight_line(\@a, $pa, $sa, \@OLD_HIGHLIGHT),
@@ -173,6 +133,60 @@ sub split_line {
split /($COLOR+)/;
 }
 
+sub find_common_prefix {
+   my ($a, $b) = @_;
+
+   # Take care to skip any ansi color codes.
+   my $seen_plusminus;
+   my ($pa, $pb) = (0, 0);
+   while ($pa < @$a && $pb < @$b) {
+   if ($a->[$pa] =~ /$COLOR/) {
+   $pa++;
+   }
+   elsif ($b->[$pb] =~ /$COLOR/) {
+   $pb++;
+   }
+   elsif ($a->[$pa] eq $b->[$pb]) {
+   $pa++;
+   $pb++;
+   }
+   elsif (!$seen_plusminus && $a->[$pa] eq '-' && $b->[$pb] eq 
'+') {
+   $seen_plusminus = 1;
+   $pa++;
+   $pb++;
+   }
+   else {
+   last;
+   }
+   }
+
+   return $pa, $pb;
+}
+
+sub find_common_suffix {
+   my ($a, $pa, $b, $pb) = @_;
+
+   # Take care to skip any ansi color codes.
+   my ($sa, $sb) = ($#$a, $#$b);
+   while ($sa >= $pa && $sb >= $pb) {
+   if ($a->[$sa] =~ /$COLOR/) {
+   $sa--;
+   }
+   elsif ($b->[$sb] =~ /$COLOR/) {
+   $sb--;
+   }
+   elsif ($a->[$sa] eq $b->[$sb]) {
+   $sa--;
+   $sb--;
+   }
+   else {
+   last;
+   }
+   }
+
+   return $sa, $sb;
+}
+
 sub highlight_line {
my ($line, $prefix, $suffix, $theme) = @_;
 
-- 
2.6.0

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/4] diff-highlight: match up lines before highlighting

[Jonathan Lebon]

As mentioned in the README, one of the current limitations of
diff-highlight is that it only calculates highlights when the hunk
contains the same number of removed lines as added lines.

A further limitation upon this is that diff-highlight assumes that the
first line removed matches the first line added, similarly with the
second, the third, etc... As was demonstrated in the "Bugs" section of
the README, this poses limitations since that assumption does not always
give the best result.

With this patch, we eliminate those limitations by trying to match up
the removed and added lines before highlighting them. This is done using
a recursive algorithm.

Note that I did not bother with some common optimizations such as
memoization since the usual number of removed/added lines in a single
hunk are small. In practice, I have not felt any lag at all during
paging.

Signed-off-by: Jonathan Lebon 
---
 contrib/diff-highlight/README | 61 +
 contrib/diff-highlight/diff-highlight | 83 +--
 2 files changed, 70 insertions(+), 74 deletions(-)

diff --git a/contrib/diff-highlight/README b/contrib/diff-highlight/README
index bbbfdda..885ff2f 100644
--- a/contrib/diff-highlight/README
+++ b/contrib/diff-highlight/README
@@ -14,17 +14,7 @@ Instead, this script post-processes the line-oriented diff, 
finds pairs
 of lines, and highlights the differing segments.  It's currently very
 simple and stupid about doing these tasks. In particular:
 
-  1. It will only highlight hunks in which the number of removed and
- added lines is the same, and it will pair lines within the hunk by
- position (so the first removed line is compared to the first added
- line, and so forth). This is simple and tends to work well in
- practice. More complex changes don't highlight well, so we tend to
- exclude them due to the "same number of removed and added lines"
- restriction. Or even if we do try to highlight them, they end up
- not highlighting because of our "don't highlight if the whole line
- would be highlighted" rule.
-
-  2. It will find the common prefix and suffix of two lines, and
+  1. It will find the common prefix and suffix of two lines, and
  consider everything in the middle to be "different". It could
  instead do a real diff of the characters between the two lines and
  find common subsequences. However, the point of the highlight is to
@@ -142,52 +132,3 @@ heuristics.
 -
 
which is less readable than the current output.
-
-2. The multi-line matching assumes that lines in the pre- and post-image
-   match by position. This is often the case, but can be fooled when a
-   line is removed from the top and a new one added at the bottom (or
-   vice versa). Unless the lines in the middle are also changed, diffs
-   will show this as two hunks, and it will not get highlighted at all
-   (which is good). But if the lines in the middle are changed, the
-   highlighting can be misleading. Here's a pathological case:
-
--
--one
--two
--three
--four
-+two 2
-+three 3
-+four 4
-+five 5
--
-
-   which gets highlighted as:
-
--
--one
--t-{wo}
--three
--f-{our}
-+two 2
-+t+{hree 3}
-+four 4
-+f+{ive 5}
--
-
-   because it matches "two" to "three 3", and so forth. It would be
-   nicer as:
-
--
--one
--two
--three
--four
-+two +{2}
-+three +{3}
-+four +{4}
-+five 5
--
-
-   which would probably involve pre-matching the lines into pairs
-   according to some heuristic.
diff --git a/contrib/diff-highlight/diff-highlight 
b/contrib/diff-highlight/diff-highlight
index a332f86..46556fc 100755
--- a/contrib/diff-highlight/diff-highlight
+++ b/contrib/diff-highlight/diff-highlight
@@ -88,24 +88,79 @@ sub show_hunk {
return;
}
 
-   # If we have mismatched numbers of lines on each side, we could try to
-   # be clever and match up similar lines. But for now we are simple and
-   # stupid, and only handle multi-line hunks that remove and add the same
-   # number of lines.
-   if (@$a != @$b) {
-   print @$a, @$b;
-   return;
-   }
-
my @queue;
-   for (my $i = 0; $i < @$a; $i++) {
-   my ($rm, $add) = highlight_pair($a->[$i], $b->[$i]);
-   print $rm;
-   push @queue, $add;
-   }
+   match_and_highlight_pairs($a, 0, scalar @$a, $b, 0, scalar @$b, 
\@queue);
print @queue;
 }
 
+# Here, we try to be clever and match up similar lines. I.e. we try to
+# find which lines in the `rem` lines (array a) became which other lines
+# 

[PATCH 0/4] diff-highlight: make a few improvements

[Jonathan Lebon]

These patches bring a few improvements to the contrib/diff-highlight
Perl script. The major improvement is done in patch 3/4, which improves
diff-highlighting accuracy by implementing a recursive line matching
algorithm.

Please note that I have limited experience with Perl, so there may be
better ways to do things. (Let me know if that is the case!)

Jonathan Lebon (4):
  diff-highlight: add `less -r` to cmd in README
  diff-highlight: factor out prefix/suffix functions
  diff-highlight: match up lines before highlighting
  diff-highlight: add maxhunksize config option

 contrib/diff-highlight/README |  87 +---
 contrib/diff-highlight/diff-highlight | 189 --
 2 files changed, 161 insertions(+), 115 deletions(-)

-- 
2.6.0

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html