[PATCH v2] git-config support for diff.relative setting

2014-12-20 Thread kelson
By default, git-diff shows changes and pathnames relative to the 
repository root. Setting the diff.relative config option to true shows 
pathnames relative to the current directory and excludes changes outside 
this directory (identical to git diff --relative).


Signed-off-by: Brandon Phillips kel...@shysecurity.com
---
 Documentation/diff-config.txt |  6 ++
 diff.c|  8 
 t/t4045-diff-relative.sh  | 21 +
 3 files changed, 35 insertions(+)

diff --git a/Documentation/diff-config.txt b/Documentation/diff-config.txt
index b001779..10f183f 100644
--- a/Documentation/diff-config.txt
+++ b/Documentation/diff-config.txt
@@ -182,3 +182,9 @@ diff.algorithm::
low-occurrence common elements.
 --
 +
+
+diff.relative::
+   By default, linkgit:git-diff[1] shows changes and pathnames
+   relative to the repository root. Setting this variable to
+   `true` shows pathnames relative to the current directory and
+   excludes changes outside this directory.
diff --git a/diff.c b/diff.c
index d1bd534..03697a9 100644
--- a/diff.c
+++ b/diff.c
@@ -223,6 +223,14 @@ int git_diff_ui_config(const char *var, const char
*value, void *cb)
return 0;
}

+   if (!strcmp(var, diff.relative)) {
+   if (git_config_bool(var, value))
+   DIFF_OPT_SET(default_diff_options, RELATIVE_NAME);
+   else
+   DIFF_OPT_CLR(default_diff_options, RELATIVE_NAME);
+   return 0;
+   }
+
if (git_color_config(var, value, cb)  0)
return -1;

diff --git a/t/t4045-diff-relative.sh b/t/t4045-diff-relative.sh
index 3950f50..8c8fe0b 100755
--- a/t/t4045-diff-relative.sh
+++ b/t/t4045-diff-relative.sh
@@ -29,6 +29,23 @@ test_expect_success -p $* 
 
 }

+check_config() {
+expect=$1; shift
+cat expected EOF
+diff --git a/$expect b/$expect
+new file mode 100644
+index 000..25c05ef
+--- /dev/null
 b/$expect
+@@ -0,0 +1 @@
++other content
+EOF
+test_expect_success git-config diff.relative=true in $1 
+   (cd $1; git -c diff.relative=true diff -p HEAD^ ../actual) 
+   test_cmp expected actual
+
+}
+
 check_numstat() {
 expect=$1; shift
 cat expected EOF
@@ -69,5 +86,9 @@ for type in diff numstat stat raw; do
check_$type file2 --relative=subdir
check_$type dir/file2 --relative=sub
 done
+for type in config; do
+   check_$type file2 subdir/
+   check_$type file2 subdir
+done

 test_done
--
1.9.1


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


git update-ref --stdin : too many open files

2014-12-20 Thread Loic Dachary
Hi,

Steps to reproduce:

$ git --version
git version 1.9.1
$ wc -l /tmp/1
9090 /tmp/1
$ head /tmp/1
delete refs/pull/1/head
create refs/heads/pull/1 86b715f346e52920ca7c9dfe65424eb9946ebd61
delete refs/pull/1/merge
create refs/merges/1 c0633abdc5311354c9729374e0ba25c97a89f69e
...
$ ulimit -n
1024
$ git update-ref --stdin  /tmp/1
fatal: Unable to create 
'/home/gitmirror/repositories/Ceph/ceph/refs/heads/pull/1917.lock': Too many 
open files
$ head -20 /tmp/1 | git update-ref --stdin
$ echo $?
0

The workaround is to increase ulimit -n

git update-ref --stdin should probably close some files.

Cheers
-- 
Loïc Dachary, Artisan Logiciel Libre



signature.asc
Description: OpenPGP digital signature


Re: bug patch: exit codes from internal commands are handled incorrectly

2014-12-20 Thread Kenneth Lorber

On Dec 18, 2014, at 2:18 PM, Junio C Hamano gits...@pobox.com wrote:

 Kenneth Lorber k...@his.com writes:
 
 Bug: exit codes from (at least) internal commands are handled incorrectly.
 E.g. git-merge-file, docs say:
   The exit value of this program is negative on error, and the number of
   conflicts otherwise. If the merge was clean, the exit value is 0.
 But only 8 bits get carried through exit, so 256 conflicts gives
 exit(0), which means the merge was clean.
 
 Wouldn't any cmd_foo() that returns negative to main() be buggy?

Absolutely.

 Your change sweeps such problems under the rug, which is not a
 healthy thing to do.

True.  But as I don’t know the codebase it seemed the safe thing to do
at least to prove the bug existed before filing a bug report.

 Expecting that the exit code can signal small positive integers and
 other generic kinds of failures is a losing proposition.  I think it
 is a better fix to update cmd_merge_file() to return 1 (when ret is
 positive), 0 (when ret is zero) or 128 (when ret is negative), or
 something simple like that, and update the documentation to match
 that, without touching git.c::main().

I agree.

It does leave open the question of whether this bit of information
is of any use and should/could be reported out in another way.  It’s not
to me, so I won’t be proposing anything to address that.

 Among the in-tree users, I notice git-cvsserver.perl is already
 using the command incorrectly.  It does this:
 
my $return = system(git, merge-file, $file_local, $file_old, 
 $file_new);
$return = 8;
 
cleanupTmpDir();
 
if ( $return == 0 )
{
$log-info(Merged successfully);
...
}
elsif ( $return == 1 )
{
$log-info(Merged with conflicts);
...
}
else
{
$log-warn(Merge failed);
next;
}
 
 which assumes $return == 1 is special half-good, which is not
 consistent with the documented interface.  It will incorrectly
 say Merge failed when there are two or more conflicts.
 
 And with such a 1 or 0 or -1 change, you will fix that caller as
 well ;-)

Yay! :-)

Thanks,
Keni

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2] git-config support for diff.relative setting

2014-12-20 Thread Philip Oakley

From: kel...@shysecurity.com
By default, git-diff shows changes and pathnames relative to the 
repository root. Setting the diff.relative config option to true 
shows pathnames relative to the current directory and excludes changes 
outside this directory (identical to git diff --relative).
Shouldn't this `(identical to git diff --relative)` also be included 
in the documentation change? It would truly clarify the intenbt for the 
reader.




Signed-off-by: Brandon Phillips kel...@shysecurity.com
---
 Documentation/diff-config.txt |  6 ++
 diff.c|  8 
 t/t4045-diff-relative.sh  | 21 +
 3 files changed, 35 insertions(+)

diff --git a/Documentation/diff-config.txt 
b/Documentation/diff-config.txt

index b001779..10f183f 100644
--- a/Documentation/diff-config.txt
+++ b/Documentation/diff-config.txt
@@ -182,3 +182,9 @@ diff.algorithm::
 low-occurrence common elements.
 --
 +
+
+diff.relative::
+ By default, linkgit:git-diff[1] shows changes and pathnames
+ relative to the repository root. Setting this variable to
+ `true` shows pathnames relative to the current directory and
+ excludes changes outside this directory.

   Insert `(identical to git diff --relative)` here?


diff --git a/diff.c b/diff.c
index d1bd534..03697a9 100644
--- a/diff.c
+++ b/diff.c
@@ -223,6 +223,14 @@ int git_diff_ui_config(const char *var, const 
char

*value, void *cb)
 return 0;
 }

+ if (!strcmp(var, diff.relative)) {
+ if (git_config_bool(var, value))
+ DIFF_OPT_SET(default_diff_options, RELATIVE_NAME);
+ else
+ DIFF_OPT_CLR(default_diff_options, RELATIVE_NAME);
+ return 0;
+ }
+
 if (git_color_config(var, value, cb)  0)
 return -1;

diff --git a/t/t4045-diff-relative.sh b/t/t4045-diff-relative.sh
index 3950f50..8c8fe0b 100755
--- a/t/t4045-diff-relative.sh
+++ b/t/t4045-diff-relative.sh
@@ -29,6 +29,23 @@ test_expect_success -p $* 
 
 }

+check_config() {
+expect=$1; shift
+cat expected EOF
+diff --git a/$expect b/$expect
+new file mode 100644
+index 000..25c05ef
+--- /dev/null
 b/$expect
+@@ -0,0 +1 @@
++other content
+EOF
+test_expect_success git-config diff.relative=true in $1 
+ (cd $1; git -c diff.relative=true diff -p HEAD^ ../actual) 
+ test_cmp expected actual
+
+}
+
 check_numstat() {
 expect=$1; shift
 cat expected EOF
@@ -69,5 +86,9 @@ for type in diff numstat stat raw; do
 check_$type file2 --relative=subdir
 check_$type dir/file2 --relative=sub
 done
+for type in config; do
+ check_$type file2 subdir/
+ check_$type file2 subdir
+done

 test_done
--
Philip 


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] list-objects: mark fewer commits as edges for non-shallow clones

2014-12-20 Thread brian m. carlson
On Thu, Dec 11, 2014 at 11:26:47AM -0800, Junio C Hamano wrote:
 The right approach would be more like allocating one more bit in
 struct rev_info (call that edge_hint_aggressive), give a new option
 --objects-edge-aggressive, and do something like
 
   if (thin) {
   use_internal_rev_list = 1;
   argv_array_push(rp, is_repository_shallow()
   ? --objects-edge-aggressive
 : --objects-edge);
   }
 
 in this codepath?  I'd actually suggest is_repository_shallow()
 detection to happen one level even higher (i.e. make decision at the
 caller of pack-objects) and decide to pass either --thin or
 --thin-aggressive, so that we can make sure that the damage caused
 by fbd4a70 to be limited only to fetches into shallow repository
 with stronger confidence.

Sorry it's taken me so long to get back to this.  Real life keeps
getting in the way.

I think adding --objects-edge-aggressive is the best way forward here
and then applying the patch above.  If we add --thin-aggressive, we push
the problem to a higher level, which will require more code changes and
make the performance regression continue to affect external remote
helpers, which we don't want.  We know what the right decision is here,
so let's just do it.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187


signature.asc
Description: Digital signature


Re: [PATCH] remote: allow adding remote w same name as alias

2014-12-20 Thread Anastas Dancha
On Fri, Dec 19, 2014 at 11:30 AM, Michael J Gruber
g...@drmicha.warpmail.net wrote:
 Anastas Dancha schrieb am 19.12.2014 um 16:44:
 Hello Johannes,

 On Fri, Dec 19, 2014 at 4:37 AM, Johannes Schindelin
 johannes.schinde...@gmx.de wrote:
 [...]
 There is one bit left to clarify: let me guess, you have a $HOME/.gitconfig
 like this:

 [url anas...@company.com]
 insteadOf = backup
 pushInsteadOf = backup

 and then you want to add the backup remote in a Git working directory
 like this:

 git remote add backup me@my-laptop.local

 but my suggested fix will still disallow this because the URL does not
 match the url.anas...@company.com.insteadOf?

 Ciao,
 Johannes

 Precisely that. In fact, it will not work even if you do any of these:

 git remote add backup anas...@company.com
 git remote add backup anyth...@anyhost.com
 git remote add backup backup

 The original / current code and your suggested fix - both exhibit
 similar behaviour with the use cases I've described above.

 Thanks,
 Anastas


 OK, I'll repeat it again: We cannot allow that.

 git push can take a url or a remote as a parameter. Which one is it if
 you have a remote and a url (alias) with the same name?

 Michael

I suppose it could be either. A string can be checked if there is a
remote with such name first, and if not, it could be attempted to be
used as a url.
In fact, my current workaround involves the following:

git remote add tempname backup:product/repo.git
git remote rename tempname backup
git push backup

This seems to work just fine right now with no code changes.
Michael, could you explain in more detail what will break in **git
push backup** if code changes are made to eliminate the necessity of
the add/remote workaround.

Regards,
Anastas
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] list-objects: mark fewer commits as edges for non-shallow clones

2014-12-20 Thread brian m. carlson
On Thu, Dec 11, 2014 at 05:51:54PM +0700, Duy Nguyen wrote:
 I'm glad it's now working better for you. Out of curiosity, have you
 enabled pack bitmap on this repo? I would expect it to reduce time
 some (or a lot) more, or we would find something to optimize further.

The client performs much, much worse with pack bitmaps.  The server has
bitmaps enabled, but it doesn't have any patches for this issue applied
to it.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187


signature.asc
Description: Digital signature


[PATCH v2 2/3] rev-list: add an option to mark fewer edges as uninteresting

2014-12-20 Thread brian m. carlson
In commit fbd4a70 (list-objects: mark more commits as edges in
mark_edges_uninteresting - 2013-08-16), we marked an increasing number
of edges uninteresting.  This change, and the subsequent change to make
this conditional on --objects-edge, are used by --thin to make much
smaller packs for shallow clones.

Unfortunately, they cause a significant performance regression when
pushing non-shallow clones with lots of refs (23.322 seconds vs.
4.785 seconds with 22400 refs).  Add an option to git rev-list,
--objects-edge-aggressive, that preserves this more aggressive behavior,
while leaving --objects-edge to provide more performant behavior.

Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
---
 Documentation/git-rev-list.txt | 3 ++-
 Documentation/rev-list-options.txt | 4 
 list-objects.c | 4 ++--
 revision.c | 6 ++
 revision.h | 1 +
 5 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt
index fd7f8b5..5b11922 100644
--- a/Documentation/git-rev-list.txt
+++ b/Documentation/git-rev-list.txt
@@ -46,7 +46,8 @@ SYNOPSIS
 [ \--extended-regexp | -E ]
 [ \--fixed-strings | -F ]
 [ \--date=(local|relative|default|iso|iso-strict|rfc|short) ]
-[ [\--objects | \--objects-edge] [ \--unpacked ] ]
+[ [ \--objects | \--objects-edge | \--objects-edge-aggressive ]
+  [ \--unpacked ] ]
 [ \--pretty | \--header ]
 [ \--bisect ]
 [ \--bisect-vars ]
diff --git a/Documentation/rev-list-options.txt 
b/Documentation/rev-list-options.txt
index 2277fcb..8cb6f92 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -657,6 +657,10 @@ These options are mostly targeted for packing of Git 
repositories.
objects in deltified form based on objects contained in these
excluded commits to reduce network traffic.
 
+--objects-edge-aggressive::
+   Similar to `--objects-edge`, but it tries harder to find excluded
+   commits at the cost of increased time.
+
 --unpacked::
Only useful with `--objects`; print the object IDs that are not
in packs.
diff --git a/list-objects.c b/list-objects.c
index 2910bec..2a139b6 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -157,7 +157,7 @@ void mark_edges_uninteresting(struct rev_info *revs, 
show_edge_fn show_edge)
 
if (commit-object.flags  UNINTERESTING) {
mark_tree_uninteresting(commit-tree);
-   if (revs-edge_hint  !(commit-object.flags  SHOWN)) 
{
+   if (revs-edge_hint_aggressive  
!(commit-object.flags  SHOWN)) {
commit-object.flags |= SHOWN;
show_edge(commit);
}
@@ -165,7 +165,7 @@ void mark_edges_uninteresting(struct rev_info *revs, 
show_edge_fn show_edge)
}
mark_edge_parents_uninteresting(commit, revs, show_edge);
}
-   if (revs-edge_hint) {
+   if (revs-edge_hint_aggressive) {
for (i = 0; i  revs-cmdline.nr; i++) {
struct object *obj = revs-cmdline.rev[i].item;
struct commit *commit = (struct commit *)obj;
diff --git a/revision.c b/revision.c
index 75dda92..753dd2f 100644
--- a/revision.c
+++ b/revision.c
@@ -1853,6 +1853,12 @@ static int handle_revision_opt(struct rev_info *revs, 
int argc, const char **arg
revs-tree_objects = 1;
revs-blob_objects = 1;
revs-edge_hint = 1;
+   } else if (!strcmp(arg, --objects-edge-aggressive)) {
+   revs-tag_objects = 1;
+   revs-tree_objects = 1;
+   revs-blob_objects = 1;
+   revs-edge_hint = 1;
+   revs-edge_hint_aggressive = 1;
} else if (!strcmp(arg, --verify-objects)) {
revs-tag_objects = 1;
revs-tree_objects = 1;
diff --git a/revision.h b/revision.h
index 9cb5adc..033a244 100644
--- a/revision.h
+++ b/revision.h
@@ -93,6 +93,7 @@ struct rev_info {
blob_objects:1,
verify_objects:1,
edge_hint:1,
+   edge_hint_aggressive:1,
limited:1,
unpacked:1,
boundary:2,
-- 
2.2.1.209.g41e5f3a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v2 0/3] Improve push performance with lots of refs

2014-12-20 Thread brian m. carlson
This series contains patches to address a significant push performance
regression in repositories with large amounts of refs.  It avoids
performing expensive edge marking unless the repository is shallow.

The first patch in the series is a fix for a minor typo I discovered
when editing the documentation.  The second patch implements git
rev-list --objects-edge-aggressive, and the final patch ensures it's
used for shallow repos only.  As the final patch was suggested by Junio,
it will need his sign-off.

I considered Junio's suggestion for a --thin-aggressive, but felt that
it was better to have the fix localized to a single location to improve
maintainability and that --thin-aggressive would be uncommonly used
outside of automatic invocations.

Also, as I understand it, the goal of thin packs is to improve
performance by sending fewer objects.  In most cases, the benefit of the
decreased time spent marking edges push times will dwarf the increase in
time it takes for a slightly larger pack to go over the wire, so it
doesn't make sense to make this a tunable.

The original fix was suggested by Duy Nguyen.

brian m. carlson (3):
  Documentation: add missing article in rev-list-options.txt
  rev-list: add an option to mark fewer edges as uninteresting
  pack-objects: use --objects-edge-aggressive only for shallow repos

 Documentation/git-rev-list.txt | 3 ++-
 Documentation/rev-list-options.txt | 7 ++-
 builtin/pack-objects.c | 4 +++-
 list-objects.c | 4 ++--
 revision.c | 6 ++
 revision.h | 1 +
 6 files changed, 20 insertions(+), 5 deletions(-)

-- 
2.2.1.209.g41e5f3a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v2 3/3] pack-objects: use --objects-edge-aggressive only for shallow repos

2014-12-20 Thread brian m. carlson
Using --objects-edge-aggressive is important for shallow repos, as it
can result in a much smaller pack (in some cases, 10% of the size).
However, it performs poorly on large non-shallow repositories with many
refs.  Since shallow repositories are less likely to have many refs (due
to having less history), the smaller pack size is advantageous there.
Adjust pack-objects to use --objects-edge-aggressive only for shallow
repositories.

Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
---
 Documentation/rev-list-options.txt | 3 ++-
 builtin/pack-objects.c | 4 +++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/rev-list-options.txt 
b/Documentation/rev-list-options.txt
index 8cb6f92..2984f40 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -659,7 +659,8 @@ These options are mostly targeted for packing of Git 
repositories.
 
 --objects-edge-aggressive::
Similar to `--objects-edge`, but it tries harder to find excluded
-   commits at the cost of increased time.
+   commits at the cost of increased time.  This is used instead of
+   `--objects-edge` to build ``thin'' packs for shallow repositories.
 
 --unpacked::
Only useful with `--objects`; print the object IDs that are not
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 3f9f5c7..f3ba861 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2711,7 +2711,9 @@ int cmd_pack_objects(int argc, const char **argv, const 
char *prefix)
argv_array_push(rp, pack-objects);
if (thin) {
use_internal_rev_list = 1;
-   argv_array_push(rp, --objects-edge);
+   argv_array_push(rp, is_repository_shallow()
+   ? --objects-edge-aggressive
+   : --objects-edge);
} else
argv_array_push(rp, --objects);
 
-- 
2.2.1.209.g41e5f3a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v2 1/3] Documentation: add missing article in rev-list-options.txt

2014-12-20 Thread brian m. carlson
Add the missing article a.

Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
---
 Documentation/rev-list-options.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/rev-list-options.txt 
b/Documentation/rev-list-options.txt
index afccfdc..2277fcb 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -653,7 +653,7 @@ These options are mostly targeted for packing of Git 
repositories.
 --objects-edge::
Similar to `--objects`, but also print the IDs of excluded
commits prefixed with a ``-'' character.  This is used by
-   linkgit:git-pack-objects[1] to build ``thin'' pack, which records
+   linkgit:git-pack-objects[1] to build a ``thin'' pack, which records
objects in deltified form based on objects contained in these
excluded commits to reduce network traffic.
 
-- 
2.2.1.209.g41e5f3a

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html