Re: How do I regularly clean up .git/objects/pack/tmp_* files?

[Jeff King]

On Sat, Jun 03, 2017 at 04:53:20PM -0400, Dun Peal wrote:

> I try to follow the best practice of regular repo maintenance, which
> afaik consists of running `git gc` every week or so.
> 
> So I thought I had a well-maintained repository, and was quite
> surprised to discover some very large .git/objects/pack/tmp_* files,
> which apparently are entirely needless and should be cleaned up, but
> `git gc` doesn't clean them.
> 
> Is there another command I should regularly run, instead or in
> addition to `git gc`, to keep my repo well-maintained, and
> specifically, to remove such unnecessary files from my .git directory?

They're kept with the same time-based grace period that is used for
unreachable objects, which defaults to the rather conservative "2
weeks". Try "git gc --prune=1.hour.ago".

-Peff

Investment Interest & Offer

[Seydou Thieba]

Re: [PATCH] perf: work around the tested repo having an index.lock

[Junio C Hamano]

Jeff King  writes:

> But I think a more compelling case is that there may be an ongoing
> operation in the original repo (e.g., say you are in the middle of
> writing a commit message) when we do a blind copy of the filesystem
> contents. You might racily pick up a lockfile.
>
> Should we find and delete all *.lock files in the copied directory? That
> would get ref locks, etc. Half-formed object files are OK. Technically
> if you want to get an uncorrupted repository you'd also want to copy
> refs before objects (in case somebody makes a new object and updates a
> ref while you're copying).

Or "git branch -m A B" is in progress.

I think it all depends on what your "threat" model is ;-).  Do we
assume that many users are "time-sharing" a box and a repository?
If not, i.e. if you are the sole user of a box and a repository on
it, such a concurrent access to make the result of git-unaware copy
problematic will not be in index.lock (after all you are now doing
the perf thing, not editing a commit log message in the repository
used for testing Git), but will be in ref locks (somebody else
pushing into the repository you are *not* currently using from
sideways).

Re: [WIP/PATCH 7/6] perf: add a performance test for core.fsmonitor

[Junio C Hamano]

Ævar Arnfjörð Bjarmason   writes:

> This is WIP code for the reasons explained in the setup comments,
> unfortunately the perf code doesn't easily allow you to run different
> setup code for different versions you're testing. This test will stop
> working if the fsmonitor is merged into the master branch.
> ...
> +
> + # Relies on core.fsmonitor not being merged into master. Needs
> + # better per-test ways to disable it if it gets merged.
> + git config core.fsmonitor true &&

Will stop working and relies on not merged can be read but I cannot
read "why" explained, and I cannot quite guess what the reason is.

If the code to read the configuration is not there, setting this
would not have any effect.  If the code is there, setting this would
have effect (either talking fsmonitor helps or it hurts).

And I do not think we'd ever see a version of Git that always relies
on talking to fsmonitor, i.e. "git config core.fsmonitor false" is not
a way to disable it, so...

Puzzled.

Re: [PATCH] perf: work around the tested repo having an index.lock

[Junio C Hamano]

Junio C Hamano  writes:

> Perhaps run "fsck" and "status" immediately after copying to make
> sure they succeed, or something like that?

Hmph, this is me half-being silly, as this "copying an existing one"
is meant for testing Git with a large repository, and having to run
fsck may add meaningful overhead in addition to the actual copying
in the set-up procedure.  I do not offhand think of a good solution.

Re: [PATCH] test-lib: add ability to cap the runtime of tests

[Junio C Hamano]

Ævar Arnfjörð Bjarmason   writes:

> Speeding up the test suite by simply cataloging and skipping tests
> that take longer than N seconds is a hassle to maintain, and entirely
> skips some tests which would be nice to at least partially run,
> e.g. instead of entirely skipping t3404-rebase-interactive.sh we can
> run it for N seconds and get at least some "git rebase -i" test
> coverage in a fast test run.

I'd be more supportive to the former approach in the longer run for
two reasons.

Is it even safe to stop a test in the middle?  Won't we leave
leftover server processes, for example?

I see start_httpd at least sets up "trap" to call stop_httpd
when the shell exits, so HTTP testing via lib-httpd.sh may be
safe.  I do not know about other network-y tests, though.

Granted, when a test fails, we already have the same problem, but
then we'd go in and investigate, and the first thing we notice would
be that the old leftover server instance is holding onto the port to
prevent the attempt to re-run the test from running, which then we'd
kill.  But with this option, the user is not even made aware of
tests being killed in the middle.

> While running with a timeout of 10 seconds cuts the runtime in half,
> over 92% of the tests are still run. The test coverage is higher than
> that number indicates, just taking into account the many similar tests
> t0027-auto-crlf.sh runs brings it up to 95%.

I certainly understand that but in the longer term, I'd prefer the
approach to call out an overly large test.  That will hopefully
motivate us to split it (or speed up the thing) to help folks on
many-core machines.

I am afraid that the proposed change will disincentivize that by
sweeping the problematic ones under the rug.  Perhaps you can
collect what tests are terminated in the middle because they run for
too long and show the list of them at the end, or something?

Also, I thought that it was a no-no to say "to_skil=all" with
skipped-reason in the middle of a test when the test is run under
prove?

Oh, by the way, is "date +%s" even portable?  I thought not.

Re: [PATCH] perf: work around the tested repo having an index.lock

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  writes:

>> My feeling exactly.  Diagnosing and failing upfront saying "well you
>> made a copy but it is not suitable for testing" sounds more sensible
>> at lesat to me.
>
> This change makes the repo suitable for testing when it wasn't before.

Perhaps "not suitable" was a bit too vague.

The copy you made is not in a consistent state that is good for
testing.  This change may declare that it is now in a consistent
state, but removal of a single *.lock file does not make it so.  We
do not know what other transient inconsistency the resulting copy
has; it is inherent to git-unaware copy---that is why we discouraged
and removed rsync transport after all.

> Yes, there are cases where there are other issues than index.lock
> preventing testing the repo, but I don't see why there shouldn't be a
> partial solution that solves a very common case in lieu of a perfect
> solution.

As long as the partial solution makes sure that the case it
addressed was the only breakage, I'd be happy to see that it leaves
other kinds of inconsistencies "too rare to bother fixing".  I however
feel dirty if the punting is "we won't even bother diagnosing and
assume that *.lock is the only thing we care about".

Perhaps run "fsck" and "status" immediately after copying to make
sure they succeed, or something like that?

Re: [WIP v2 2/2] pack-objects: support --blob-max-bytes

[Junio C Hamano]

Jeff King  writes:

> I do also wonder if the two features would need to be separated for a
> GVFS-style solution. If you're not just avoiding large blobs but trying
> to get a narrow clone, you don't want the .git files from the
> uninteresting parts of the tree. You want to get no blobs at all, and
> then fault them in as they become relevant due to user action.

Thanks; I didn't think of this one while reviewing the overall design,
but I agree that this is something important to think about.

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

[Junio C Hamano]

Jeff King  writes:

> ...
> That explains why you want to pass "-q" in the other hunk; to
> countermand the explicit --progress here. But if we separate the two as
> I mentioned above, you'd want logic more like:
>
>   if test -t 2 && test "$GIT_QUIET" != "t"
>   git_format_patch_opt="$git_format_patch_opt --progress"
>   fi

This looks like a sensible line of thought.  Kevin, do you want to
try this again with the "explicit --progress option" approach?  I
think the gist of the v1 review is that everybody agrees that the
end goal of showing a progress eye-candy from a long-running
"format-patch" phase in a "rebase" is a good idea, but it is dubious
"format-patch" should show it in cases other than when --progress is
explicitly asked for.

Thanks.

issue: "git subtree split" vs "git svn fetch" consequences

[Andry]

Hi, git folks!

I think i've found something about inconsistency between "git subtree split" 
and "git svn fetch".

First of all, "git subtree split" ignores commits with 0 changed files, when 
the "git svn fetch" is not.
For example, it could be an svn commit where only properties has changed, but 
no file changes.

Second, seems "git subtree split" and "git svn fetch" produces different commit 
hashes for the same commits.

Experiment:

For instance, we have SVN repo: repoA with project in subdirectory, say repoA/B.

If try to make "git svn clone repoA A.git" and "git svn clone repoA/B B.git", 
then you will get totally different commit's hash list:

>cd A.git
>git log --pretty=oneline master
 (HEAD -> master, git-svn) A1
 B1
 A2
 B2
...
 AN
 BN

>cd ../B.git
>git log --pretty=oneline master
 (HEAD -> master, git-svn) B1
 B2
...
 BN

,where  not equal to the same commit , where k=1..N.

I beleave git does create a commit hash using eigher distance to the root, or 
previous commit hash in a commit chain.
Anyway, whatever it does this all leads to unavailability to synchronize such 
working copies which has been made through the "git svn clone repoA"+"git 
subtree split --prefix=B"+"git clone repoB.git"
with the SVN repository referenced by next call to the "git svn init repoA/B".

In another words, say we already have a git repository made from "git subtree 
split" - repoB.git.

Now, this set of commands looks resonable:

>git clone repoB.git B.git
Cloning into 'B.git'...
remote: Counting objects: 931, done.
remote: Compressing objects: 100% (395/395), done.
remote: Total 931 (delta 517), reused 931 (delta 517), pack-reused 0
Receiving objects: 100% (931/931), 10.15 MiB | 475.00 KiB/s, done.
Resolving deltas: 100% (517/517), done.

>cd B.git

>git log --pretty=online --grep=git-svn-id: master
 (HEAD -> master, git-svn) B1
 B2
...
 BN

>git svn init repoA/B

>git update-ref refs/remotes/git-svn 

(This one is important here, otherwise the next "git svn fetch" will retake 
whole svn repository from 0, which is might be too long if there is hundreds of 
revisions.)

(take a notice, the problem goes from here, see details below)

>git svn fetch
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map. ...
r4 = 
r6 = 
r12 = 
...
rM-1 = 
rM = 
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.

rNEW =  (refs/remotes/git-svn)

>git svn rebase
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/git-svn.

>git push origin master
Counting objects: 3, done.
Delta compression using up to 6 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 375 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To repoB.git
   ..  master -> master

And it seems is all ok, but for one exception. The repoB.git is not the subtree 
splitted repository.
This is repository directly cloned from the SVN repoA/B just before rNEW commit 
has made into the SVN repository which is made in turn after the clone.

The real output is this:

>git svn fetch
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map. ...
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map. ...
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map. ...
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.
Index mismatch:  != 
rereading 

W: -empty_dir: blabla1
W: -empty_dir: blabla2
...
W: -empty_dir: blablaL
Last fetched revision of refs/remotes/git-svn was r92, but we are about to 
fetch: r4!

I googled the whole internet inside out, but there is just no any solution for 
it.
It's broken and nothing about it.

The only solution i see here is just throw "git subtree split" as a hindering 
garbage for the sake of git-svn synchronization.

[ANNOUNCE] git-cinnabar 0.5.0b1

[Mike Hommey]

Hi,

Git-cinnabar is a git remote helper to interact with mercurial
repositories. It allows to clone, pull and push from/to mercurial remote
repositories, using git.

Code on https://github.com/glandium/git-cinnabar
This release on
https://github.com/glandium/git-cinnabar/releases/tag/0.5.0b1

What's new since 0.4.0?

- git-cinnabar-helper is now mandatory. You can either download one with
  `git cinnabar download` on supported platforms or build one with `make
  helper`.
- Metadata changes require to run `git cinnabar fsck`.
- Mercurial tags are consolidated in a separate (fake) repository. See
  the README file.
- Updated git to 2.13.0 for git-cinnabar-helper.
- Improved memory consumption and performance.
- Improved experimental support for pushing merges.
- Experimental support for clonebundles
  (https://www.mercurial-scm.org/wiki/ClonebundlesExtension).
- Removed support for the .git/hgrc file for mercurial specific
  configuration.
- Support any version of Git (was previously limited to 1.8.5 minimum)

Mike

[PATCH] test-lib: add ability to cap the runtime of tests

[Ævar Arnfjörð Bjarmason]

Add a GIT_TEST_TIMEOUT environment variable which optimistically sets
an approximate upper limit on how long any one test is allowed to
run.

Once the timeout is exceeded all remaining tests are skipped, no
attempt is made to stop a long running test in-progress, or deal with
the edge case of the epoch changing the epoch from under us by
e.g. ntpd.

On my machine median runtime for a test is around 150ms, but 8 tests
take more than 10 seconds to run, with t3404-rebase-interactive.sh
taking 18 seconds.

On a machine with a large number of cores these long-tail tests become
the limiting factor in how long it takes to run the entire test suite,
even if it's run with "prove --state=slow,save". This is because the
first long-running tests started at the very beginning will still be
running by the time the rest of the test suite finishes.

Speeding up the test suite by simply cataloging and skipping tests
that take longer than N seconds is a hassle to maintain, and entirely
skips some tests which would be nice to at least partially run,
e.g. instead of entirely skipping t3404-rebase-interactive.sh we can
run it for N seconds and get at least some "git rebase -i" test
coverage in a fast test run.

On a 56 core Xeon E5-2690 v4 @ 2.60GHz the runtime for the test suite
is cut in half with GIT_TEST_TIMEOUT=10 under prove -j56
t[0-9]*.sh. Approximate time to run all the tests with various
GIT_TEST_TIMEOUT settings[1]:

N/A 30s
20  20s
10  15s
5   12s
1   12s

Setting a timeout lower than 5-10 seconds or so starts to reach
diminishing returns, e.g. t7063-status-untracked-cache.sh always takes
at least 6 seconds to run since it's blocking on a single
"update-index --test-untracked-cache" command. So there's room for
improvement, but this simple facility gives us most of the benefits.

The number of tests on the aforementioned machine which are run with
the various timeouts[2]:

N/A 16261
20  16037
10  14972
5   13952
1   8409

While running with a timeout of 10 seconds cuts the runtime in half,
over 92% of the tests are still run. The test coverage is higher than
that number indicates, just taking into account the many similar tests
t0027-auto-crlf.sh runs brings it up to 95%.

1. for t in '' 20 10; do printf "%s\t" $t && (time GIT_TEST_TIMEOUT=$t prove 
-j$(parallel --number-of-cores) --state=slow,save t[0-9]*.sh) 2>&1 | grep ^real 
| grep -E -o '[0-9].*'; done
2. for t in '' 20 10 5 1; do printf "%s\t" $t && (time GIT_TEST_TIMEOUT=$t 
prove -j$(parallel --number-of-cores) --state=slow,save -v t[0-9]*.sh) 2>&1 | 
grep -e ^real -e '^1\.\.' | sed 's/^1\.\.//' | awk '{s+=$1} END {print s}'; done

Signed-off-by: Ævar Arnfjörð Bjarmason 
---
 t/test-lib.sh | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4936725c67..0353e73873 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -15,6 +15,13 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see http://www.gnu.org/licenses/ .
 
+# If we have a set max runtime record the startup time before anything
+# else is done.
+if test -n "$GIT_TEST_TIMEOUT"
+then
+   TEST_STARTUP_TIME=$(date +%s)
+fi
+
 # Test the binaries we have just built.  The tests are kept in
 # t/ subdirectory and are run in 'trash directory' subdirectory.
 if test -z "$TEST_DIRECTORY"
@@ -689,11 +696,22 @@ test_skip () {
to_skip=t
skipped_reason="--run"
fi
+   if test -n "$GIT_TEST_TIMEOUT" &&
+   test "$(($(date +%s) - $TEST_STARTUP_TIME))" -ge 
"$GIT_TEST_TIMEOUT"
+   then
+   to_skip=all
+   skipped_reason="Exceeded GIT_TEST_TIMEOUT in runtime"
+   fi
 
case "$to_skip" in
-   t)
+   all|t)
say_color skip >&3 "skipping test: $@"
say_color skip "ok $test_count # skip $1 ($skipped_reason)"
+   if test "$to_skip" = all
+   then
+   skip_all="$skipped_reason"
+   test_done
+   fi
: true
;;
*)
-- 
2.13.0.506.g27d5fe0cd

How do I regularly clean up .git/objects/pack/tmp_* files?

[Dun Peal]

Hi there,

I try to follow the best practice of regular repo maintenance, which
afaik consists of running `git gc` every week or so.

So I thought I had a well-maintained repository, and was quite
surprised to discover some very large .git/objects/pack/tmp_* files,
which apparently are entirely needless and should be cleaned up, but
`git gc` doesn't clean them.

Is there another command I should regularly run, instead or in
addition to `git gc`, to keep my repo well-maintained, and
specifically, to remove such unnecessary files from my .git directory?

Thanks, D.

Fast Approval in 1-2 Hours.

[adetriyasa]

We provide PERSONAL and BUSINESS assistance No Deposit when you apply Loan with 
us!! ,
To Ease Your Financial Burden.
-Easy Apply
-Flexible Package
-Fast Approval in 1-2 Hours.
Email: quickfinac...@qq.com

Fast Approval in 1-2 Hours.

[adetriyasa]

We provide PERSONAL and BUSINESS assistance No Deposit when you apply Loan with 
us!! ,
To Ease Your Financial Burden.
-Easy Apply
-Flexible Package
-Fast Approval in 1-2 Hours.
Email: quickfinac...@qq.com

Re: [WIP/PATCH 7/6] perf: add a performance test for core.fsmonitor

[Ævar Arnfjörð Bjarmason]

On Fri, Jun 2, 2017 at 11:44 PM, David Turner  wrote:
> BTW, a medium-sized (~250k files across 40k dirs) synthetic repo is available 
> over bittorrent at:
> http://bitmover.com/2015-04-03-1M-git-bare.tar.bz2.torrent

Can you seed this please? I've been trying to download that since
yesterday, still at 0%.

Fast Approval in 1-2 Hours.

[adetriyasa]

We provide PERSONAL and BUSINESS assistance No Deposit when you apply Loan with 
us!! ,
To Ease Your Financial Burden.
-Easy Apply
-Flexible Package
-Fast Approval in 1-2 Hours.
Email: quickfinac...@qq.com

Re: [PATCH] git-p4: remove obsolete version check

[Ævar Arnfjörð Bjarmason]

On Sat, Jun 3, 2017 at 3:31 PM, Jakub Wilk  wrote:
> The file is syntactically correct only in Python >= 2.6, so the
> version check never does anything.

[CC-ing Eric who added that check]

Your commit message doesn't give an example of this, but with e.g.
python 2.0 you get:

  File "git-p4.py", line 469
yield pattern
^
SyntaxError: invalid syntax

I checked the various other python files that had similar warnings,
they all work correctly with python 2.0.

One workaround to keep this would be to make git-p4.py import some
library to do all its work, and use some subset of python syntax to
just load and defer to that library. That works for me when I change
it like that locally. Alternatively, does Python have something like
Perl's BEGIN {} blocks where you can execute code right there before
the file has finished parsing?

Or we could just remove this, just wanted to note the above since I
dug into it, and the commit message light on details.



> Signed-off-by: Jakub Wilk 
> ---
>  git-p4.py | 4 
>  1 file changed, 4 deletions(-)
>
> diff --git a/git-p4.py b/git-p4.py
> index 8d151da91..4278cd9d4 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -8,10 +8,6 @@
>  # License: MIT 
>  #
>  import sys
> -if sys.hexversion < 0x0204:
> -# The limiter is the subprocess module
> -sys.stderr.write("git-p4: requires Python 2.4 or later.\n")
> -sys.exit(1)
>  import os
>  import optparse
>  import marshal
> --
> 2.13.0.506.g27d5fe0cd
>

Re: [PATCH] perf: work around the tested repo having an index.lock

[Ævar Arnfjörð Bjarmason]

On Sat, Jun 3, 2017 at 1:52 AM, Junio C Hamano  wrote:
> Jeff King  writes:
>
>> But I think a more compelling case is that there may be an ongoing
>> operation in the original repo (e.g., say you are in the middle of
>> writing a commit message) when we do a blind copy of the filesystem
>> contents. You might racily pick up a lockfile.
>>
>> Should we find and delete all *.lock files in the copied directory? That
>> would get ref locks, etc. Half-formed object files are OK. Technically
>> if you want to get an uncorrupted repository you'd also want to copy
>> refs before objects (in case somebody makes a new object and updates a
>> ref while you're copying).
>>
>> I don't know how careful it's worth being. I don't really _object_ to
>> this patch exactly, but it does seem like it's picking up one random
>> case (that presumably you hit) and ignoring all of the related cases.
>
> My feeling exactly.  Diagnosing and failing upfront saying "well you
> made a copy but it is not suitable for testing" sounds more sensible
> at lesat to me.

This change makes the repo suitable for testing when it wasn't before.

Yes, there are cases where there are other issues than index.lock
preventing testing the repo, but I don't see why there shouldn't be a
partial solution that solves a very common case in lieu of a perfect
solution.

Re: [PATCH] strbuf: let strbuf_addftime handle %z and %Z itself

[René Scharfe]

Am 03.06.2017 um 15:13 schrieb Ulrich Mueller:

On Sat, 3 Jun 2017, René Scharfe wrote:



+   case 'Z':
+   strbuf_addstr(_fmt, tz_name);


Is it guaranteed that tz_name cannot contain a percent sign itself?


Currently yes, because the only caller passes an empty string.

The fact that tz_name is subject to expansion by strftime could be
mentioned explicitly in strbuf.h.  I'm not sure if that's a desirable
property, but it allows callers to expand %z internally and %Z using
strftime.

René

[PATCH] git-p4: remove obsolete version check

[Jakub Wilk]

The file is syntactically correct only in Python >= 2.6, so the
version check never does anything.

Signed-off-by: Jakub Wilk 
---
 git-p4.py | 4 
 1 file changed, 4 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 8d151da91..4278cd9d4 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -8,10 +8,6 @@
 # License: MIT 
 #
 import sys
-if sys.hexversion < 0x0204:
-# The limiter is the subprocess module
-sys.stderr.write("git-p4: requires Python 2.4 or later.\n")
-sys.exit(1)
 import os
 import optparse
 import marshal
-- 
2.13.0.506.g27d5fe0cd

Re: [PATCH] submodule foreach: correct $sm_path in nested submodules from a dir

[Ramsay Jones]

On 03/06/17 01:37, Stefan Beller wrote:
> When running 'git submodule foreach' from a subdirectory of your
> repository, nested submodules get a bogus value for $sm_path:
> For a submodule 'sub' that contains a nested submodule 'nested',
> running 'git -C dir submodule foreach echo $path' would report
> path='../nested' for the nested submodule. The first part '../' is
> derived from the logic computing the relative path from $pwd to the
> root of the superproject. The second part is the submodule path inside
> the submodule. This value is of little use and is hard to document.
> 
> There are two different possible solutions that have more value:
> (a) The path value is documented as the path from the toplevel of the
> superproject to the mount point of the submodule.
> In this case we would want to have path='sub/nested'.
> 
> (b) As Ramsay noticed the documented value is wrong. For the non-nested
> case the path is equal to the relative path from $pwd to the
> submodules working directory. When following this model,
> the expected value would be path='../sub/nested'.
> 
> The behavior for (b) was introduced in 091a6eb0fe (submodule: drop the

Ah, so prior to 091a6eb0fe the documentation of $path was actually
correct - you could not run the command from a sub-directory, so
$path _was_ the 'name of the submodule directory relative to the
superproject'. (The fact that git-ls-files worked from a subdirectory
is not in the least relevant - it never was!) ;-)

> top-level requirement, 2013-06-16) the intent for $path seemed to be
> relative to $cwd to the submodule worktree, but that did not work for
> nested submodules, as the intermittent submodules were not included in

intermediate?

Hmm, so nobody noticed the change in behaviour (and, of course, that
the documentation hadn't been updated to match) since 2013!

> the path.
> 
> If we were to fix the meaning of the $path using (a) such that "path"
> is "the path from the toplevel of the superproject to the mount point
> of the submodule", we would break any existing submodule user that runs
> foreach from non-root of the superproject as the non-nested submodule
> '../sub' would change its path to 'sub'.
> 
> If we would fix the meaning of the $path using (b), such that "path"
> is "the relative path from $pwd to the submodule", then we would break
> any user that uses nested submodules (even from the root directory) as
> the 'nested' would become 'sub/nested'.
> 
> Both groups can be found in the wild.  The author has no data if one group
> outweighs the other by large margin, and offending each one seems equally
> bad at first.  However in the authors imagination it is better to go with
> (a) as running from a sub directory sounds like it is carried out
> by a human rather than by some automation task.  With a human on
> the keyboard the feedback loop is short and the changed behavior can be
> adapted to quickly unlike some automation that can break silently.

I do not use submodules (I have absolutely no interest in them, except
in a general wish to improve git sense). So, I have no idea what kind
of impact either change will have. However, FWIW, I agree with your
reasoning here. (Not that I actually get a vote, but I vote for a!)

> To ameliorate the situation, perform these changes
> * Document 'sm_path' instead of 'path'.
>   As using a variable '$path' may be harmful to users due to
>   capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
>   use $path variable in eval_gettext string, 2012-04-17). Adjust
>   the documentation to advocate for using $sm_path,  which contains
>   the same value. We still make the 'path' variable available,
>   though not documented.
> 
> * Clarify the 'toplevel' variable documentation.
>   It does not contain the topmost superproject as the author assumed,
>   but the direct superproject, such that $toplevel/$sm_path is the
>   actual absolute path of the submodule.
> 
> * The variable '$displaypath' was accessible but undocumented.
>   Rename it '$displaypath' to '$dpath'. Document what it contains.
>   Users that are broken by the behavior change of 'sm_path' introduced
>   in this commit, can switch from '$path' to '$dpath'.
> 
> Discussed-with: Ramsay Jones 
> Signed-off-by: Prathamesh Chavan 
> Signed-off-by: Stefan Beller 
> ---
> 

  Documentation/git-submodule.txt | 32 ++--
>  git-submodule.sh|  7 +++
>  t/t7407-submodule-foreach.sh| 39 ---
>  3 files changed, 57 insertions(+), 21 deletions(-)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index 74bc6200d5..52e3ef1325 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -218,20 +218,24 @@ information too.
>  
>  foreach [--recursive] ::
>   Evaluates an 

Re: [PATCH] strbuf: let strbuf_addftime handle %z and %Z itself

[Ulrich Mueller]

> On Sat, 3 Jun 2017, René Scharfe wrote:

> + case 'Z':
> + strbuf_addstr(_fmt, tz_name);

Is it guaranteed that tz_name cannot contain a percent sign itself?

Ulrich

[PATCH] strbuf: let strbuf_addftime handle %z and %Z itself

[René Scharfe]

There is no portable way to pass timezone information to strftime.  Add
parameters for timezone offset and name to strbuf_addftime and let it
handle the timezone-related format specifiers %z and %Z internally.
Callers can opt out by passing NULL as timezone name.

Use an empty string as timezone name in show_date (the only current
caller) for now because we only have the timezone offset in non-local
mode.  POSIX allows %Z to resolve to nothing in case of missing info.

Helped-by: Ulrich Mueller 
Helped-by: Jeff King 
Signed-off-by: Rene Scharfe 
---
Duplicates strbuf_expand to a certain extent, but not too badly, I
think.  Leaves the door open for letting strftime handle the local
case.

 date.c |  2 +-
 strbuf.c   | 42 ++
 strbuf.h   | 11 ---
 t/t6006-rev-list-format.sh | 12 
 4 files changed, 59 insertions(+), 8 deletions(-)

diff --git a/date.c b/date.c
index 63fa99685e..5580577334 100644
--- a/date.c
+++ b/date.c
@@ -246,7 +246,7 @@ const char *show_date(timestamp_t time, int tz, const 
struct date_mode *mode)
month_names[tm->tm_mon], tm->tm_year + 1900,
tm->tm_hour, tm->tm_min, tm->tm_sec, tz);
else if (mode->type == DATE_STRFTIME)
-   strbuf_addftime(, mode->strftime_fmt, tm);
+   strbuf_addftime(, mode->strftime_fmt, tm, tz, "");
else
strbuf_addf(, "%.3s %.3s %d %02d:%02d:%02d %d%c%+05d",
weekday_names[tm->tm_wday],
diff --git a/strbuf.c b/strbuf.c
index 00457940cf..b880107370 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -785,14 +785,47 @@ char *xstrfmt(const char *fmt, ...)
return ret;
 }
 
-void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm)
+void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct tm *tm,
+int tz_offset, const char *tz_name)
 {
+   struct strbuf munged_fmt = STRBUF_INIT;
size_t hint = 128;
size_t len;
 
if (!*fmt)
return;
 
+   /*
+* There is no portable way to pass timezone information to
+* strftime, so we handle %z and %Z here.
+*/
+   if (tz_name) {
+   for (;;) {
+   const char *percent = strchrnul(fmt, '%');
+   strbuf_add(_fmt, fmt, percent - fmt);
+   if (!*percent)
+   break;
+   fmt = percent + 1;
+   switch (*fmt) {
+   case '%':
+   strbuf_addstr(_fmt, "%%");
+   fmt++;
+   break;
+   case 'z':
+   strbuf_addf(_fmt, "%+05d", tz_offset);
+   fmt++;
+   break;
+   case 'Z':
+   strbuf_addstr(_fmt, tz_name);
+   fmt++;
+   break;
+   default:
+   strbuf_addch(_fmt, '%');
+   }
+   }
+   fmt = munged_fmt.buf;
+   }
+
strbuf_grow(sb, hint);
len = strftime(sb->buf + sb->len, sb->alloc - sb->len, fmt, tm);
 
@@ -804,17 +837,18 @@ void strbuf_addftime(struct strbuf *sb, const char *fmt, 
const struct tm *tm)
 * output contains at least one character, and then drop the 
extra
 * character before returning.
 */
-   struct strbuf munged_fmt = STRBUF_INIT;
-   strbuf_addf(_fmt, "%s ", fmt);
+   if (fmt != munged_fmt.buf)
+   strbuf_addstr(_fmt, fmt);
+   strbuf_addch(_fmt, ' ');
while (!len) {
hint *= 2;
strbuf_grow(sb, hint);
len = strftime(sb->buf + sb->len, sb->alloc - sb->len,
   munged_fmt.buf, tm);
}
-   strbuf_release(_fmt);
len--; /* drop munged space */
}
+   strbuf_release(_fmt);
strbuf_setlen(sb, sb->len + len);
 }
 
diff --git a/strbuf.h b/strbuf.h
index 80047b1bb7..39d5836abd 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -339,9 +339,14 @@ __attribute__((format (printf,2,0)))
 extern void strbuf_vaddf(struct strbuf *sb, const char *fmt, va_list ap);
 
 /**
- * Add the time specified by `tm`, as formatted by `strftime`.
- */
-extern void strbuf_addftime(struct strbuf *sb, const char *fmt, const struct 
tm *tm);
+ * Add the time specified by `tm`, as formatted by `strftime`.  `tz_offset`
+ * and `tz_name` are used to expand %z and %Z internally, unless `tz_name`
+ * is NULL.  `tz_offset` is in 

[PATCH 1/2] add setup step to filter-branch

[Andreas Heiduk]

A specific `--setup` step in `git filter-branch` makes it much easier
to define the initial values of variables used in the real filters.
Also sourcing/defining utility functions here instead of
`--env-filter` improves performance and minimizes clogging the output
in case of errors.

Signed-off-by: Andreas Heiduk 
---
 Documentation/git-filter-branch.txt | 16 +++-
 git-filter-branch.sh| 18 +-
 2 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-filter-branch.txt 
b/Documentation/git-filter-branch.txt
index 6e4bb0220..45c849d8c 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -8,11 +8,11 @@ git-filter-branch - Rewrite branches
 SYNOPSIS
 
 [verse]
-'git filter-branch' [--env-filter ] [--tree-filter ]
-   [--index-filter ] [--parent-filter ]
-   [--msg-filter ] [--commit-filter ]
-   [--tag-name-filter ] [--subdirectory-filter ]
-   [--prune-empty]
+'git filter-branch' [--setup ] [--env-filter ]
+   [--tree-filter ] [--index-filter ]
+   [--parent-filter ] [--msg-filter ]
+   [--commit-filter ] [--tag-name-filter ]
+   [--subdirectory-filter ] [--prune-empty]
[--original ] [-d ] [-f | --force]
[--] [...]
 
@@ -82,6 +82,12 @@ multiple commits.
 OPTIONS
 ---
 
+--setup ::
+   This is not a real filter executed for each commit but a one
+   time setup just before the loop. Therefore no commit-specific
+   variables are defined yet.  Functions or variables defined here
+   can be used or modified in the following filter steps.
+
 --env-filter ::
This filter may be used if you only need to modify the environment
in which the commit will be performed.  Specifically, you might
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index aafaf708d..2758ae5eb 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -81,11 +81,12 @@ set_ident () {
finish_ident COMMITTER
 }
 
-USAGE="[--env-filter ] [--tree-filter ]
-   [--index-filter ] [--parent-filter ]
-   [--msg-filter ] [--commit-filter ]
-   [--tag-name-filter ] [--subdirectory-filter ]
-   [--original ] [-d ] [-f | --force]
+USAGE="[--setup ] [--env-filter ]
+   [--tree-filter ] [--index-filter ]
+   [--parent-filter ] [--msg-filter ]
+   [--commit-filter ] [--tag-name-filter ]
+   [--subdirectory-filter ] [--original ]
+   [-d ] [-f | --force]
[...]"
 
 OPTIONS_SPEC=
@@ -96,6 +97,7 @@ if [ "$(is_bare_repository)" = false ]; then
 fi
 
 tempdir=.git-rewrite
+filter_setup=
 filter_env=
 filter_tree=
 filter_index=
@@ -148,6 +150,9 @@ do
-d)
tempdir="$OPTARG"
;;
+   --setup)
+   filter_setup="$OPTARG"
+   ;;
--env-filter)
filter_env="$OPTARG"
;;
@@ -317,6 +322,9 @@ else
need_index=
 fi
 
+eval "$filter_setup" < /dev/null ||
+   die "filter setup failed: $filter_setup"
+
 while read commit parents; do
git_filter_branch__commit_count=$(($git_filter_branch__commit_count+1))
 
-- 
2.13.0

[PATCH 2/2] add [--] to usage of filter-branch

[Andreas Heiduk]

Signed-off-by: Andreas Heiduk 
---
 Documentation/git-filter-branch.txt | 3 ++-
 git-filter-branch.sh| 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-filter-branch.txt 
b/Documentation/git-filter-branch.txt
index 45c849d8c..1efdda804 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -86,7 +86,8 @@ OPTIONS
This is not a real filter executed for each commit but a one
time setup just before the loop. Therefore no commit-specific
variables are defined yet.  Functions or variables defined here
-   can be used or modified in the following filter steps.
+   can be used or modified in the following filter steps except
+   the commit filter, for technical reasons.
 
 --env-filter ::
This filter may be used if you only need to modify the environment
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 2758ae5eb..3a74602ef 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -87,7 +87,7 @@ USAGE="[--setup ] [--env-filter ]
[--commit-filter ] [--tag-name-filter ]
[--subdirectory-filter ] [--original ]
[-d ] [-f | --force]
-   [...]"
+   [--] [...]"
 
 OPTIONS_SPEC=
 . git-sh-setup
-- 
2.13.0

Git Error Message

[christienne shultz]

Hello,
I have installed the Git software and upon trying to update a
repository by the "open recent repository" link and am unable to
access the menu-or anything-from there.  Instead a window pops up with
the following error message:

"\usage C:\Program Files\Git\mingw64/libexec/git-core\git-GUI[
{blame|browser|citool}]

Can you please tell me what this error message means and how to fix
the issue?  Thank you!