Re: git silently ignores include directive with single quotes

[Paul Smith]

On Sat, 2018-09-08 at 13:13 -0700, Stas Bekman wrote:
> I remind that the original problem came from a simple command:
> 
>  git config --local include.path '../.gitconfig'
> 
> Which on linux removed the quotes and all was fine, and on windows
> the same command kept the quotes and the user was tearing his hair
> out trying to understand why the custom config was ignored.

I'm quite sure that the user was not using the Git Bash shell when they
entered this command, but instead using command.com or powershell or
some variant of that.

If you use Git Bash as your shell then quotes will be handled like any
POSIX shell and the above will do what (we all) expect.

If you use command.com and powershell and use single quotes, then the
single quotes will be put into the config file as you observed, because
those shells don't deal with single quotes.

You could use double-quotes, which ARE handled by command.com and
powershell; in that case they would be stripped out and would not
appear in the config.


If we were designing from scratch maybe using something like GNU make's
"include" vs "sinclude" (silent include--this is another name for the
already mentioned "-include") would work; maybe "path" and "spath" or
something.  But to make it work right you really want the default
behavior to be "warn if the file is not found" and have the special
behavior be "quiet if the file is not found" otherwise it doesn't
really help beginners to avoid errors.  And that's a backward-
compatibility problem.  To my mind, adding extra "check this" options
isn't very useful either: these kinds of warnings need to be on by
default to be effective.  The beginners, who need them, aren't going to
remember to add extra options to enable more checking.

What I personally think would be more useful would be some sort of
"verbose parsing" option to git config, that would parse the
configuration just as a normal Git command would and show diagnostic
output as the entire config is parsed: for each action line the config
file name and line number, and the operation performed (and any message
about it) would be printed.  This could be useful in a variety of
situations, for instance to discover conflicts between local, global,
and system configuration, easily see where settings are coming from,
etc.

And as part of this output, when an include file was not present or we
didn't have permissions or whatever, an appropriate error message would
be generated.

Re: [ANNOUNCE] Git v2.19.0-rc0

[Paul Smith]

On Tue, 2018-08-21 at 23:03 -0400, Jeff King wrote:
>  static inline int hashcmp(const unsigned char *sha1, const unsigned
> char *sha2)
>  {
> +   assert(the_hash_algo->rawsz == 20);
> return memcmp(sha1, sha2, the_hash_algo->rawsz);
>  }

I'm not familiar with Git code, but for most environments assert() is a
macro which is compiled out when built for "release mode" (whatever
that might mean).  If that's the case for Git too, then relying on
assert() to provide a side-effect (even an optimizer hint side-effect)
won't work and this will actually get slower when built for "release
mode".

Just a thought...

Re: Help with "fatal: unable to read ...." error during GC?

[Paul Smith]

On Wed, 2018-08-08 at 14:24 -0400, Jeff King wrote:
> If so, can you try running it under gdb and getting a stack trace?
> Something like:
> 
>   gdb git
>   [and then inside gdb...]
>   set args pack-objects --all --reflog --indexed-objects foobreak die
>   run
>   bt
> 
> That might give us a clue where the broken object reference is coming
> from.

Oh no.  I messed up :(.

I rebuilt Git 2.18.0 without optimization to try to get more debug
information.  Unfortunately I didn't think to create a backup of my
problematic .git directory.

When I ran the above command under the debugger using the non-optimized 
version of Git... it worked!  That fixed the problem so that now when I
run "git gc" using the original optimized version I no longer see the
issue there either.

So... clearly something is wrong but because I was dumb and didn't make
a backup I can no longer reproduce the problem :(.  On the other hand,
my repository is no longer throwing errors so that's good.

I do still have these warnings and no amount of git gc/git fsck/etc.
has reduced them in any way:

$ git gc
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
Enumerating objects: 506556, done.
Counting objects: 100% (506556/506556), done.
Delta compression using up to 8 threads.
Compressing objects: 100% (101199/101199), done.
Writing objects: 100% (506556/506556), done.
Total 506556 (delta 358957), reused 506556 (delta 358957)
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
Checking connectivity: 506556, done.

I've run git gc --prune=all then git fsck reports only these dangling
commits:

dangling commit cef0678a5e0765506e3fac41286696fd37a9b1e9
dangling commit 1729195f021a1b95ea8ca10b9c32e76bf2257e67
dangling commit 08385b9731291607a8c6d4bf10272002d8f31e1f
dangling commit c4ddfb2139eeb5a3c132dbfc84cc6e27fdeb46d1
dangling commit 1df8ebcc1cd5f59dd224ce1f3ba39f24370cf4e7

(this is down from probably 50 or so "dangling ..." commits, blobs, and
trees before).

Re: Help with "fatal: unable to read ...." error during GC?

[Paul Smith]

On Wed, 2018-08-08 at 14:24 -0400, Jeff King wrote:
> If so, can you try running it under gdb and getting a stack trace?
> Something like:
> 
>   gdb git
>   [and then inside gdb...]
>   set args pack-objects --all --reflog --indexed-objects foobreak die
>   run
>   bt
> 
> That might give us a clue where the broken object reference is coming

Here we go.  I can rebuild with -Og or -O0 if more detailed debugging
is needed; most everything appears to be optimized out:

  ...
Compressing objects: 100% (10/10), done.
Writing objects:  54% (274416/508176)   
Thread 1 "git" hit Breakpoint 1, die (err=err@entry=0x5a373a "unable to read 
%s") at usage.c:119
119 {
(gdb) bt
#0  die (err=err@entry=0x5a373a "unable to read %s") at usage.c:119
#1  0x004563f3 in get_delta (entry=) at 
builtin/pack-objects.c:143
#2  write_no_reuse_object () at builtin/pack-objects.c:308
#3  0x00456592 in write_reuse_object (usable_delta=, 
limit=, entry=, f=) at 
builtin/pack-objects.c:516
#4  write_object (write_offset=, entry=0x7fffc9a8d940, 
f=0x198fb70) at builtin/pack-objects.c:518
#5  write_one () at builtin/pack-objects.c:576
#6  0x004592f0 in write_pack_file () at builtin/pack-objects.c:849
#7  cmd_pack_objects (argc=, argv=, 
prefix=) at builtin/pack-objects.c:3354
#8  0x00404f06 in run_builtin (argv=, argc=, p=) at git.c:417
#9  handle_builtin (argc=, argv=) at git.c:632
#10 0x00405f21 in run_argv (argv=0x7fffe210, argcp=0x7fffe21c) 
at git.c:761
#11 cmd_main (argc=, argc@entry=6, argv=, 
argv@entry=0x7fffe448) at git.c:761
#12 0x00404b15 in main (argc=6, argv=0x7fffe448) at common-main.c:45

Re: Help with "fatal: unable to read ...." error during GC?

[Paul Smith]

On Wed, 2018-08-08 at 14:24 -0400, Jeff King wrote:
> Let's narrow it down first and make sure we're dying where I expect.
> Can
> you try:
> 
>   GIT_TRACE=1 git gc
> 
> and confirm the program running when the fatal error is produced?
> 
> From what you've shown it's going to be git-repack, but what I'm not
> clear on is whether it is repack itself that is complaining, or the
> pack-objects process it spawns. I'd guess the latter.

You are correct:

15:27:24.264161 git.c:415   trace: built-in: git pack-
objects --keep-true-parents --honor-pack-keep --non-empty --all --
reflog --indexed-objects --unpack-unreachable=2.weeks.ago --local --
delta-base-offset .git/objects/pack/.tmp-17617-pack

> If so, can you try running it under gdb and getting a stack trace?

I would... but I discovered all my Git binaries are stripped to the max
and no symbols available.

I'll do a quick rebuild with some debug info and get back to you.

Thanks for the pointers!

Re: Help with "fatal: unable to read ...." error during GC?

[Paul Smith]

On Wed, 2018-08-08 at 12:06 -0400, Jeff King wrote:
> I'd have expected fsck to find it, too. However, looking at the code,
> I'm not convinced that fsck is actually considering detached worktree
> heads properly, either. Try:
> 
>   git rev-list --all --reflog --objects >/dev/null
> 
> which I know checks worktrees correctly. I'd expect that to fail.
> 
> If it does, then we need to narrow down which worktree is corrupt.
> Perhaps something like:
> 
>   git worktree list |
>   while read worktree head junk; do
> git rev-list --objects $head >/dev/null ||
> echo "$worktree seems corrupt"
>   done

Thanks for the note!  Unhappily for me none of these operations seem to
find any actionable problems...

$ git rev-list --all --reflog --objects >/dev/null
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
warning: reflog of 'HEAD' references pruned commits
$ echo $?
0

$ git worktree list | while read wt head junk; do \
  git rev-list --objects $head >/dev/null || echo "$wt seems corrupt"; \
  done
$

Just to be sure I updated the loop above to echo $wt and $head and they
were correct.  I also re-ran git gc after the above and still got the
original error output so it didn't magically fix itself :).

Help with "fatal: unable to read ...." error during GC?

[Paul Smith]

I recently upgraded from Git 2.9.2 to 2.18.0 (note, I have no
particular reason to believe this is related just passing info).  I'm
running on Linux (64bit Ubuntu 18.04.1 but I've compiled Git myself
from source, I'm not using the distro version).

I have a local repository I've been using for about two years (the
.git/description file, which I don't use, has a TLM of July 31, 2016),
with lots of worktrees being created/pruned/etc. during that time.

Note I'm doing all these operations in the 'main' repository, not in
any of the worktrees.

Yesterday, when I tried to fetch from my upstream I got a notification
about GC needed.  Then GC failed with these errors (HEAD is set to
master which is the same as origin/master):

  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  warning: reflog of 'HEAD' references pruned commits
  fatal: unable to read c104b8fb3631b5c54695206b2f73310c023c9963
  error: failed to run repack

I ran a git fsck --full which showed me a lot of dangling commits and
blobs, but no errors, no broken link messages, etc.

I ran git reflog expire --all --stale-fix but no change.

I can't find that SHA anywhere: I looked in .git/objects, etc.  I also
can't find any problems with my repo; obviously I haven't checked
everything but I can show the git log back to the initial commit, all
my stashes look fine, all my worktrees seem to be OK (git status etc.
work fine in all of them).

But whenever I pull etc. Git wants to run gc and I get this set of
errors again.  FWIW other repos created from the same remote don't show
any issues so it appears to be just this local copy of the repo.

I've seen many SO and blog posts about issues like this but all were
concentrating on recovering things and I don't even know if I've lost
anything... and anyway the operations they suggest don't work for me
because nothing can access that SHA; I just get "bad object".

Any ideas on what to look at next?

I would hate to have to throw this setup away since it has 23 stashes
and 25 worktrees in various states that would be annoying to have to
recreate... 

Re: Git 2.18: RUNTIME_PREFIX... is it working?

[Paul Smith]

On Fri, 2018-07-06 at 09:18 -0400, Daniel Jacques wrote:
> I forewent autoconf because I was concerned that the option was too
> obscure and the configuration too nuanced to be worth adding via
> flag, as RUNTIME_PREFIX requires some degree of path alignment and is
> fairly special-case. If you prefer autoconf, though, it sounds like a
> good thing to add, and I'm happy that you are finding the feature
> useful!

Well, far from obscure, I actually think that RUNTIME_PREFIX should be
the default behavior on all platforms.  In fact speaking for myself, I
see no value at all in the hardcoded path behavior and it could be
removed and RUNTIME_PREFIX be the only option and that would be fine
with me.

The only possible advantage I can see to the current default that you
can copy the Git binary alone somewhere else, but that's of very little
value IMO: you could instead create a symbolic link or a two-line shell
script wrapper if you wanted to have "git" available outside of its
normal relation to the rest of the installation for some reason.

Thanks for making this work in any event, Daniel!

Re: Git 2.18: RUNTIME_PREFIX... is it working?

[Paul Smith]

On Wed, 2018-07-04 at 13:22 +0200, Johannes Schindelin wrote:
> > Basically what happens is that I run configure with
> > --prefix=/my/install/path --with-gitconfig=etc/gitconfig
> > --with-gitattributes=etc/gitattributes.
> > 
> > Then I run make with RUNTIME_PREFIX=YesPlease.
> 
> Ah. In Git for Windows, we do not use configure. I *think* this
> points to an incompatibility of the RUNTIME_PREFIX feature with our
> autoconf support, and this is a grand opportunity for you to step in
> and help.
> 
> Essentially, what you will want to do is to implement a new configure
> option --with-runtime-prefix that then prevents the autoconf script
> from munging the relative paths in the way it does.

FYI I was able to get this to work by overriding variables on the make
command line, like this:

  make ... RUNTIME_PREFIX=YesPlease \
  gitexecdir=libexec/git-core \
  template_dir=share/git-core/templates \
  sysconfdir=etc

I agree a new autoconf option would be much simpler to use.  I'll think
about it as I happen to have some some experience in these areas ;) ...
but time is limited of course :).

Re: RUNTIME_PREFIX references in gitconfig variable paths

[Paul Smith]

On Wed, 2018-07-04 at 13:26 +0200, Johannes Schindelin wrote:
> On Wed, 4 Jul 2018, Paul Smith wrote:
> 
> > One thing I wanted to do was provide a default ca-bundle.crt file
> > along with my local build of Git.  I need my installation to be
> > relocatable and I'm using RUNTIME_PREFIX with Git 2.18.0 (on
> > GNU/Linux).
> 
> Understandable. We do this all the time in Git for Windows. Our
> config entry has this form:
> 
> [http]
> sslCAinfo = /ssl/certs/ca-bundle.crt
> 
> and in the RUNTIME_PREFIX mode, this will be made relative to the
> runtime prefix. It is my understanding that bf9acba (http: treat
> config options sslCAPath and sslCAInfo as paths, 2015-11-23) makes
> this work.

Hm.  Unless I'm missing something this doesn't happen (and indeed, it
does not work for me; with:

  [http]
  sslcainfo = /etc/ca-bundle.crt

I get:

  fatal: unable to access 'https://github.com/myrepo.git/': error
setting certificate verify locations:
CAfile: /etc/ca-bundle.crt
CApath: none

although it works if I use a fully-qualified pathname, and using strace
I find the process never attempted to access any other path for ca-
bundle.crt).

In http.c we see how this path is treated in http_options():

if (!strcmp("http.sslcainfo", var))
return git_config_pathname(_cainfo, var, value);

I can't tell exactly how this function is invoked, but the result
(ssl_cainfo) is used here without further modification:

curl_easy_setopt(result, CURLOPT_CAINFO, ssl_cainfo);

In config.c we find get_config_pathname() which does this:

*dest = expand_user_path(value, 0);

In path.c we find expand_user_path() which does this:

if (path == NULL)
goto return_null;
if (path[0] == '~') {
...
}
strbuf_addstr(_path, to_copy);
return strbuf_detach(_path, NULL);

I don't see any reference to system_prefix(), system_path(), etc. which
would be needed to RUNTIME_PREFIX-ize things.

RUNTIME_PREFIX references in gitconfig variable paths

[Paul Smith]

One thing I wanted to do was provide a default ca-bundle.crt file along
with my local build of Git.  I need my installation to be relocatable
and I'm using RUNTIME_PREFIX with Git 2.18.0 (on GNU/Linux).

I can provide a system gitconfig file with a setting for http.sslCAInfo
but the problem is I can't create a relocatable path here so I don't
know how to set it:

  $ cat $prefix/etc/gitconfig
  [http]
  sslCAInfo = /etc/ca-bundle.crt

What do I use for  above since I want it to be relocatable? 
Basically I want this to be in the same directory as the relocatable
sysconfdir (I don't actually care much but that seems like a good
place).

Is there some way to create a reference to a path relative to the
installation directory?

For example "~" is accepted as the users $HOME path; is there some
syntax which refers to the Git installation directory?

If not this seems like something that would be very useful.


I can use a wrapper script and set GIT_SSL_CAINFO, but that will also
override any user's setting of http.sslCAInfo in their local gitconfig
which I don't really want.

Git 2.18: RUNTIME_PREFIX... is it working?

[Paul Smith]

I was excited to see the RUNTIME_PREFIX for POSIX systems patchset go
by earlier this year.  Although I didn't see any mention of it being
included in the 2.18.0 release notes, it does appear that it was merged
in for this release.

Has anyone else tried to get it working?  It doesn't appear to be
working properly for me so I'm not sure if I'm supposed to be doing
something different... I didn't see any documentation on it.

Basically what happens is that I run configure with
--prefix=/my/install/path --with-gitconfig=etc/gitconfig
--with-gitattributes=etc/gitattributes.

Then I run make with RUNTIME_PREFIX=YesPlease.

When I look in the makefile, I see that the make variable gitexecdir is
initially properly set to libexec/git-core which is what I expect.

However, later in the makefile we include the config.mak.autogen file,
which was generated from config.mk.in by configure.  In the .in file we
have this:

 gitexecdir = @libexecdir@/git-core

After configure gets done with it, this becomes:

 gitexecdir = ${prefix}/libexec/git-core

which is a fully-qualified path.  This means that exec-cmd.c is
compiled with -DGIT_EXEC_PATH="/my/install/path/libexec/git-core" which
effectively disables RUNTIME_PREFIX, as the exec-cmd.c:system_prefix()
function always returns FALLBACK_RUNTIME_PREFIX since GIT_EXEC_PATH is
not a suffix of executable_dirname (once the install location has been
moved).

I suppose we need to pass more configure options to reset paths; is
there information somewhere on exactly which ones should be overridden?
 For example if I try to pass configure --libexecdir=libexec to solve
the above issue, I get an error from configure:

 configure: error: expected an absolute directory name for --libexecdir: libexec

Any info on how this is supposed to work, is welcome!

Git 2.16.2: Build failure linking static libcurl / static SSL

[Paul Smith]

I'm compiling Git with my own static libcurl and my own static
LibreSSL.  They live in two different locations.

Building Curl with a pointer to my LibreSSL works fine, and compiling
Git works fine: the correct -I options are added to the compile line
when I configure with --with-openssl=/path/to/libressl/dist

However, linking fails to find the crypto and ssl libraries, because
the OPENSSLDIR lib directory is not added to the link line with -L. 
The link line in question is from Makefile:

  git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(CURL_LIBCURL) $(LIBS)
  git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)

The OpenSSL libraries are included in CURL_LIBCURL, but the -L flags
are not:

ifdef NEEDS_SSL_WITH_CURL
CURL_LIBCURL += -lssl
ifdef NEEDS_CRYPTO_WITH_SSL
CURL_LIBCURL += -lcrypto
endif
endif

This section needs to add in the OPENSSL_LINK variable, or maybe it has
to go directly in the git-http-fetch/push recipe, I'm not sure which is
appropriate.  There seems to be a lot of different variables that have
similar content, that maybe should be aligned (OPENSSL_LIBSSL,
OPENSSL_LINK, LIB_4_CRYPTO, CURL_LIBCURL, etc.)

But, the -L is definitely missing:

  gcc -g -O2 -I. -DGIT_HOST_CPU="\"x86_64\"" -DHAVE_ALLOCA_H 
-I/work/src/git/Linux-Release-make/dist/include 
-I/work/src/libressl/Linux-Release-make/dist/include -DNO_GETTEXT -DSHA1_DC 
-DSHA1DC_NO_STANDARD_INCLUDES -DSHA1DC_INIT_SAFE_HASH_DEFAULT=0 
-DSHA1DC_CUSTOM_INCLUDE_SHA1_C="\"cache.h\"" 
-DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C="\"git-compat-util.h\"" -pthread 
-DHAVE_PATHS_H -DHAVE_STRINGS_H -DHAVE_DEV_TTY -DHAVE_CLOCK_GETTIME 
-DHAVE_CLOCK_MONOTONIC -DHAVE_GETDELIM  -DFREAD_READS_DIRECTORIES -DNO_STRLCPY 
-DSHELL_PATH='"/bin/sh"' -DPAGER_ENV='"LESS=FRX LV=-c"' -o git-http-fetch   
http.o http-walker.o http-fetch.o common-main.o \
-L/work/src/git/Linux-Release-make/dist/lib 
-L/work/src/git/Linux-Release-make/dist/lib -lcurl -lssl -lcrypto -lidn 
libgit.a xdiff/lib.a  -lz -pthread -lrt
  ld: error: cannot find -lssl
  ld: error: cannot find -lcrypto

Note how the -I.../libressl/.../include option is present, but the
-L.../libressl/.../lib option is missing.

Git "branch properties"-- thoughts?

[Paul Smith]

Hi all.  I'm wondering if anyone has any thoughts about the best, or
even any, way to have "branch properties": some extra information which
is attached to a branch (that is, the actual branch name not the commit
it currently points to).

My requirements are that the information needs to be pushed to the
server, so that lets out branch descriptions for example.  Ideally the
information would also be easily updated and available in all clones
during normal fetch operations but this isn't a hard requirement: I
could script it.

My immediate desire is to find a way to mark a branch as "frozen", that
will control which pushes are allowed.  I use gitolite on my server and
I can easily write a server hook that will handle the checking,
rejecting, etc.  I already have such an infrastructure.  What I need is
a way to know which branches are in that state, so my hook can see that
and DTRT.  There are other "branch properties" I could envision, too,
but don't have a real need right now.

Of course I could embed the frozen state into the gitolite repository
configuration.  Indeed, I have already implemented "locks" for obsolete
branches.  But "frozen" is a more ephemeral state and requiring access
to the gitolite repository to manage it is just not what I want; it's a
separate repository so the state is not visible, requires privileges I
really don't want to hand out to everyone, and is generally difficult. 
I want some users to be able to manage frozen branches relatively
easily, and all users to be able to see the state of which branches are
frozen, etc.

So then I thought about creating a "frozen" tag, like "frozen/v1.0" or
something.  This is slightly weird because it is applied to a commit,
which is not really right, but whatever: it's just a marker so I would
just be checking to see if it exists or not.  The other problem is that
Git tags are not intended to be transient/moveable.  While you CAN
delete them and move them, when someone pulls the repository they won't
get that update by default.  Since the hook is server-side the fact
that the local repository has the wrong information doesn't matter for
behavior, but it's confusing for people.  So, it's not ideal.

I thought about creating a branch, like "frozen/v1.0", rather than a
tag.  I don't need a branch here, and no one would push to that branch
(I'd have to disallow that in my hooks), and the commit associated with
the branch would not be relevant most likely.  I would only check to
see if the branch existed, or not.  Branches are nice because creating
and deleting them is handled automatically (if you use prune
consistently, which we do because we have tons of transient branches).

Then I looked into using notes, and they look interesting, but they're
associated with a specific commit as well and I don't want that: a
frozen branch can still have new commits pushed to it they just have
meet certain criteria.  This makes them hard to translate into a branch
name.

So far, using a special branch name seems the most "reasonable".  But,
I wonder if I missed some cool aspect if Git that would work better, or
if anyone else has other suggestions.

Cheers!

Re: "git branch" issue in 2.16.1

[Paul Smith]

On Tue, 2018-02-06 at 11:49 -0800, Jason Racey wrote:
> After upgrading git from 2.16.0 to 2.16.1 (via Homebrew - I’m on
> macOS) I noticed that the “git branch” command appears to display the
> branch listing in something similar to a vi editor - though not quite
> the same. I don’t know the technical term for this state. You can’t
> actually edit the output of the command, but you’re in a state where
> you have to type “q” to exit and then the list disappears. It’s very
> inconvenient and it doesn’t seem like it was by design. I’m using zsh
> in iTerm2 if that helps. Thanks.

I think you mean that you're in the pager (less(1), most likely). 
Many/most Git commands that can generate a large amount of output (git
log, git diff, git show, etc.) will automatically send the output to a
pager so you can scroll through it easily.


The man page for git branch says:

  CONFIGURATION
   pager.branch is only respected when listing branches, i.e., when --list
   is used or implied. The default is to use a pager. See git-config(1).

So, if you never want to use the pager for git branch output you can
configure the pager.branch option to set it always off.

Or you can use "git branch | cat" so that stdout is not a terminal :).

Re: Can't squash merge with merge.ff set to false

[Paul Smith]

On Fri, 2018-01-05 at 12:12 -0800, Bryan Turner wrote:
> On Fri, Jan 5, 2018 at 11:59 AM, Robert Dailey  
> wrote:
> > Not sure if this is intended or a bug, but with the following
> > configuration:
> > 
> > $ git config --global merge.ff false
> > 
> > I am not able to merge my topic branch into master with squash
> > option:
> > 
> > $ git checkout master
> > $ git merge --squash topic
> > fatal: You cannot combine --squash with --no-ff.
> > 
> > I'm not sure why a non-fast-forward merge would prevent a squash
> > merge, since by its very nature a squashed merge is not a fast
> > forward merge (or maybe it is if you only have one commit).

Hah!  I was just thinking of checking the latest Git RC I built
yesterday to see if this pet peeve of mine has been fixed yet.  I guess
not!

> The easiest way to move forward is probably to pass "--ff" on the
> command line to override the config, when you're using "--squash".

That's what we always have to do.  Very annoying; we use squash-merge
extensively but also want to require ff merge by default.

> As for why the two aren't allowed together, my assumption would be
> because if you're only squashing a single commit "--squash" and that
> commit is fast-forward from the target, a new commit is not created
> and instead the target branch is fast-forwarded. With "--no-ff", it's
> questionable what "--squash" should do in that case. Fast-forward
> anyway? Rewrite the commit simply to get new committer details and
> SHA-1?

If it only failed when you were squash-merging a single commit that was
also fast-forwardable, I guess that would be one thing.  But even if I
have multiple commits and I want to squash-merge them, which clearly is
a separate operation giving different results, I get this error.

I don't think Git should try to be clever here (if that's what it's
doing--I always assumed it was just a missing configuration case in the
error check).  If I asked for a squash-merge then Git should give me a
squash merge.

So in answer to your question, --squash should give me a squash merge
and the setting of --ff / --no-ff should be completely ignored, as it's
irrelevant.

My $0.02.

Re: [ANNOUNCE] Git v2.16.0-rc0

[Paul Smith]

On Thu, 2018-01-04 at 20:18 +, Thomas Gummerer wrote:
> On 12/29, Paul Smith wrote:
> > On Thu, 2017-12-28 at 20:30 -0800, Junio C Hamano wrote:
> > >   * The way "git worktree add" determines what branch to create
> > > from where and checkout in the new worktree has been updated a
> > > bit.
> > 
> > Does this include the enhancements published a few weeks ago to
> > allow worktrees to be created directly from remote branches without
> > first checking out the branch locally? I'm really looking forward
> > to that change...
> 
> Yes, this release will include that.  It would be awesome if you
> could test the rc, now is the best time to scream if something about
> it is not as you'd expect :)

OK, I pulled and built this locally, and indeed the straightforward
"git worktree add  " works just as I'd hoped!  Nice!

I'll play with it more and let you know if I hit issues.

Re: [ANNOUNCE] Git v2.16.0-rc0

[Paul Smith]

On Thu, 2017-12-28 at 20:30 -0800, Junio C Hamano wrote:
>  * The way "git worktree add" determines what branch to create from
>where and checkout in the new worktree has been updated a bit.

Does this include the enhancements published a few weeks ago to allow
worktrees to be created directly from remote branches without first
checking out the branch locally? I'm really looking forward to that
change...

Thanks!

Re: Bring together merge and rebase

[Paul Smith]

On Tue, 2017-12-26 at 12:44 -0700, Carl Baldwin wrote:
> > Sure, it could be opt in, be a new format etc. But you haven't
> > explained why you think a feature like this would need to rely on
> > an entirely new parent structure and side-DAG, as opposed to just
> > the more minor changes I'm pointing out above, and which I think
> > will give you what you need from a UX level.
> 
> I have not wrapped my head around it enough to convince myself that
> it gives what I'm after. Let me spend a little more time with it to
> get a feel for it.

As someone working in an environment where we do a lot of rebasing and
very little merging, I read these proposals with interest.  I'm not
convinced that we would switch to using a "replaces"-type feature, but
I'm pretty sure that the "null-merge and rebase" trick described
previously would not be something we're interested in using.

Although "git log" doesn't follow these merges (unless requested), all
the graphical tools that are used to display history WOULD show all
those branches.  In a "replaces"-type environment I think the point is
that we would not want to see them (certainly not by default) as they
would be used mainly for deeper spelunking, but since they just seem
like normal merges I don't see any way to turn them off.

If "replaces" was a separate capability then it could be treated
differently by history browsing tools, and shown or not shown as
desired.  For example, a commit that had a "replaces" element could be
selected somehow and you could expand that set of commits that were
replaced, or something like that.

Re: [PATCH v4 4/4] worktree: make add dwim

[Paul Smith]

On Sat, 2017-11-25 at 20:06 +, Thomas Gummerer wrote:
> This part is getting done in 3/4, and is definitely going to work
> without an additional flag, so this is (hopefully) soon going to work
> just as you want :)

Yay!  Thanks!

Re: [PATCH v4 4/4] worktree: make add dwim

[Paul Smith]

On Sat, 2017-11-25 at 17:50 +, Thomas Gummerer wrote:
> This would be the output in the new version:
> 
>  $ git worktree add ../bla
>  Branch 'bla' set up to track remote branch 'bla' from 'origin'.
>  Preparing ../bla (identifier bla)
>  HEAD is now at 4aade43 bla
> 
> vs. the output without the changed behaviour:
> 
>  $ git worktree add ../bla
>  Preparing ../bla (identifier bla)
>  HEAD is now at 0f215c9 initial import
> 
> Of course that assumes that it's used directly, not in scripts, and
> that users will actually read the output of the command when they
> invoke it.  Maybe these are not safe assumptions to make though, and
> we'd rather not have this on by default then.  As I mentioned
> previously I would prefer having this as default, but I'm happy to
> hide this behaviour behind a flag if we want to be more careful about
> introducing this.  Dunno?

Speaking as a simple user, I find the current behavior of Git worktree
add very frustrating; I am constantly wanting to create worktrees for
other peoples' branches so I can look at the code there without messing
up my workspace, and it's really inconvenient to do that now.

Also, the current special handling of the directory name as a putative
branch name is not helpful for me because many of the branches I need
to examine use "/" as their separator.  I don't begrudge making that
feature more "DWIM" for those that can use it, but hopefully some help
is forthcoming for those who can't.

For example, I need to create a local worktree for the remote rel/1.0
branch... what do I do?

What I want to work is this:

git worktree add ../1.0 rel/1.0

and have it create a worktree at ../1.0, then do the equivalent of "git
checkout rel/1.0" which includes setting up to track the remote branch.
 But of course this doesn't work at all; I get:

fatal: invalid reference: rel/1.0

Personally I would think it odd to have to add an extra flag to get
what I would expect would be "normal" behavior (checkout).

But maybe that's just me.

Re: bash script to pull in branch B the changes from parent branch A

[Paul Smith]

On Wed, 2017-11-22 at 00:19 +0800, Laetitia Pfaender wrote:
> cd repo-in-branchB
> git branch --set-upstream-to=origin/branchB
> git pull
> 
> git branch --set-upstream-to=origin/branchA
> git pull
> git branch --set-
> upstream-to=origin/branchB
> 
> It does exactly what I want but, as I have
> many children branches B to update, I would like to prompt my
> username and password only once and then makes the script use them in
> all following git requests.

It would be nice if you explained in words exactly what it is you want
to do.

This seems like a lot more work than necessary.  A "git pull" consists
of two steps: first a "git fetch" which is the part that actually goes
out to the remote and pulls all the new content, and then a merge
operation to the remote's version of the current branch.

The "git fetch" is all that needs credentials, and it pulls the entire
contents of the repo including all branches, so you only need to do it
once.

Is there some reason why you can't do the following:

  cd repo
  git fetch (requires you to enter username/password)
  git merge origin/branchB
  git merge origin/branchA

and just continue to merge for each different branch (without re-
running git fetch)?

>  I came to the conclusion that I needed to update my script as
> follow:
> echo “pull from branchB"
> git pull https://$username:$password@g
> ithub.com/myrepo.git heads/branchB
> echo “pull from parent branchA"
> git
> pull https://$username:$passw...@github.com/myrepo.git heads/branchA
>  I came to the conclusion that I needed to update my script as
> follow:
> echo “pull from branchB"
> git pull https://$username:$password@g
> ithub.com/myrepo.git heads/branchB
> echo “pull from parent branchA"
> git
> pull https://$username:$passw...@github.com/myrepo.git heads/branchA

Don't know how well this works as I don't use HTTPS remotes very much. 
But note, this will make your username AND password for your GitHub
account visible to anyone one the system who happens to run "ps" while
your pull command is running.

Re: "git rm" seems to do recursive removal even without "-r"

[Paul Smith]

On Tue, 2017-10-10 at 04:36 -0400, Robert P. J. Day wrote:
>   ah, now *that* is a compelling rationale that justifies the
> underlying weirdness. but it still doesn't explain the different
> behaviour between:
> 
>   $ git rm -n 'Makefile*'
>   $ git rm -n '*Makefile'

I explained that behavior in the email up-thread from this reply:

> Globbing in "git rm" matches on the FULL PATH, not just the file name. 
> So, if you have a list of Makefiles in your repository like:
> 
>   Makefile
>   foo/Makefile
>   bar/Makefile
> 
> Then 'Makefile*' only matches the first one, since 'Makefile*' doesn't
> match 'foo/Makefile' or 'bar/Makefile'.
>
> If you you worry that '*Makefile' will match things you don't want to
> match, you'll have to use:
> 
>   git rm -n Makefile '*/Makefile'

Re: "git rm" seems to do recursive removal even without "-r"

[Paul Smith]

On Sat, 2017-10-07 at 17:55 -0400, Robert P. J. Day wrote:
> On Sat, 7 Oct 2017, Paul Smith wrote:
> > On Sat, 2017-10-07 at 15:43 -0400, Robert P. J. Day wrote:
> > > it's been a long week, so take this in the spirit in which it is
> > > intended ... i think the "git rm" command and its man page should be
> > > printed out, run through a paper shredder, then set on fire. i can't
> > > remember the last time i saw such a thoroughly badly-designed,
> > > badly-documented and non-intuitive utility.
> >
> > "git rm" works the same way that the UNIX rm command has worked, for
> > 40+ years now.  Very simple, very well designed, and very intuitive
> > (IMO).
> >
> > The major difference is the ability to handle globbing patterns,
> > which UNIX rm doesn't do.  Maybe the way this is implemented is a
> > little confusing, although I just read the man page and it seemed
> > pretty clear to me.
> 
>   um, wrong.

I don't know what part of my comment here you consider "wrong".  I've
re-read it and I believe everything I said is correct.

> > If you don't use glob patterns (or more specifically if you let the
> > shell handle glob patterns, which is how I always do it) then there
> > is really nothing bizarre about "git rm".  Maybe you could be more
> > precise in your criticism.
> 
> ok, fine, let me explain why this command is a nightmarish
> monstrosity. as i now understand, if i use an escaped wildcard
> pattern, "git rm" will *automatically* (with no further guidance from
> me, and no warning), operate recursively. so if, in the kernel source
> tree, i ran:
> 
>   $ git rm \*.c

Yes, this is what I said: "IF YOU DON'T USE GLOB PATTERNS (or more
specifically if you let the shell handle glob patterns ...) then there
is nothing bizarre about "git rm"" (emphasis added).

In this example you ARE sending glob patterns to Git, because you're
escaping them from the shell.  Hence, you might consider the behavior
bizarre, at least until you grok it fully.

If you want to avoid this, simply use normal shell globbing and don't
ask Git to do the globbing for you.  Then it behaves exactly as normal
UNIX rm, including with the '-r' option, and is very simple.

> so if i wanted git to remove, say, all files named "Makefile*" from
> everywhere in the linux kernel source tree, the (dry run) command
> would be:
> 
>   $ git rm -n Makefile\*
> 
> is that it? let's try that:
> 
>   $ git rm -n Makefile\*
>   rm 'Makefile'
>   $
> 
> oops.

Globbing in "git rm" matches on the FULL PATH, not just the file name. 
So, if you have a list of Makefiles in your repository like:

  Makefile
  foo/Makefile
  bar/Makefile

Then 'Makefile*' only matches the first one, since 'Makefile*' doesn't
match 'foo/Makefile' or 'bar/Makefile'.

If you you worry that '*Makefile' will match things you don't want to
match, you'll have to use:

  git rm -n Makefile '*/Makefile'

Personally I don't use Git's magical globbing capabilities, and use "git
rm" as if it were UNIX rm.  So in your request above I'd use:

   git rm $(find . -name Makefile)

which I find simpler.

Re: "git rm" seems to do recursive removal even without "-r"

[Paul Smith]

On Sat, 2017-10-07 at 15:43 -0400, Robert P. J. Day wrote:
>   it's been a long week, so take this in the spirit in which it is
> intended ... i think the "git rm" command and its man page should be
> printed out, run through a paper shredder, then set on fire. i can't
> remember the last time i saw such a thoroughly badly-designed,
> badly-documented and non-intuitive utility.

"git rm" works the same way that the UNIX rm command has worked, for 40+
years now.  Very simple, very well designed, and very intuitive (IMO).

The major difference is the ability to handle globbing patterns, which
UNIX rm doesn't do.  Maybe the way this is implemented is a little
confusing, although I just read the man page and it seemed pretty clear
to me.

If you don't use glob patterns (or more specifically if you let the
shell handle glob patterns, which is how I always do it) then there is
really nothing bizarre about "git rm".  Maybe you could be more precise
in your criticism.

Re: Git Bash Bug

[Paul Smith]

On Fri, 2017-07-14 at 22:33 +0200, Johannes Schindelin wrote:
> > You absolutely have to have /bin and /usr/bin on your PATH,
> 
> As Kavita talks about Git Bash, it is probably Git for Windows, for
> which /bin should not be in the PATH but /mingw64/bin or /mingw32/bin
> (depending on the architecture).

I did check this with my Git for Windows installation before posting. 
Mine is older (2.7.0) though so maybe things have changed:

  pds@build-win MINGW64 ~
  $ type -a ls
  ls is aliased to `ls -F --color=auto --show-control-chars'
  ls is /usr/bin/ls
  ls is /bin/ls
  ls is /usr/bin/ls
  ls is /usr/bin/ls

Clearly I have a lot of duplicates on my PATH now that I notice :)

I have /mingw64/bin on my PATH as well but looking there it has git,
gettext, edit, a bunch of DLL's, etc. but it doesn't contain ls or
other coreutils programs.

Cheers!

Re: Git Bash Bug

[Paul Smith]

On Fri, 2017-07-14 at 09:59 -0500, Kavita Desai wrote:
> What does "echo $PATH" show?
> /c/Users/Kavita/

Well, there you go.  That's clearly wrong.

You absolutely have to have /bin and /usr/bin on your PATH, _at least_
if you want to be able to run standard UNIX tools.  And most likely
you'll have a ton of other directories on your PATH as well.

I would investigate the shell configuration files etc. and see where
you've messed up resetting your PATH variable.

> What does "type -a ls" show?
> ls is aliased to `ls -F --color=auto --show-control-chars'

Yep.  There is no ls binary found.

Re: Git Bash Bug

[Paul Smith]

On Fri, 2017-07-14 at 09:34 -0500, Kavita Desai wrote:
> Sorry for not being specific. What I meant by not working was that the
> bash commands are not found.
> Here is an example
> 
> $ ls
> bash: ls: command not found

The most obvious issue is your PATH is wrong.

What does "echo $PATH" show?

What does "type -a ls" show?

Re: Git Strange behaviour with remote deleted branch

[Paul Smith]

On Mon, 2017-06-19 at 23:33 +0200, Reda Lyazidi wrote:
> with git I noticed when I removed a remote branch with git push origin 
> --delete 
> in my clone when I used git branch -a I don't the deleted branch
> but my colleagues still see it.
> 
> I tried with two clones in my PC, with the first one delete branch and 
> the other still sees it
> despite git pull .

Normally Git will not remove a remote's branch refs on a fetch (or
pull) operation.  This is a safety measure, since you might still have
a use for that branch (for example to run diff against or similar).

In order to remove branches that have been deleted in your remote, you
must use the --prune (or -p) option to git fetch or git pull:

  git pull -p

will get rid of them.

Re: Minor missing features in worktree compare to new-workdir

[Paul Smith]

On Wed, 2017-06-14 at 08:31 -0400, Yichao Yu wrote:
> 1. the branch name in new-workdir has the same behavior as checkout,
> i.e. when it matches a remote branch name a local branch tracking that
> remote branch will be created and checked out. worktree gives an error
> in this case. This is very useful for fetching someone else' feature
> branch into a different work dir for testing.

I agree; this behavior of worktree is frustrating.  It's a very common
use-case to aid in code reviews etc. and it's not easy to explain to
people what they need to do to make this work.

> 2. worktree doesn't seem to support multiple worktree on the same
> branch.

I think this is a very good thing about worktrees as opposed to
workdirs.

In a situation where I may want multiple worktrees pointing to the same
commit I just check out the SHA rather than the branch: you can have as
many branches set to the same SHA (detached HEAD) as you like, and
there's no concern about dirty workspaces.  This latter can actually be
a really big problem (suppose the workdir contained some modified files
then you update another workdir with the same branch... it's not easy to
figure out what happened here!)

Re: git stash filename - stashing single files.

[Paul Smith]

On Wed, 2016-12-14 at 09:44 -0500, Jeff King wrote:
> Personally, I think this is a pretty terrible interface. Besides the
> fact that I have never written a stash message in all my years of using
> git, it's totally inconsistent with the rest of git (which would use
> "-m" for the message, and treat arguments as pathspecs).

I agree that this is a terrible (and inconsistent) interface and I would
love to see it changed.

I do use messages on stashes all the time (because I can never remember
their context a week later without a hint--just looking at the diff is
not enough for me) but I would welcome this change.  Definitely the
first step would be introducing the "-m" option so people and tools
could begin the switch to using it.

Re: [PATCH v1 1/2] config.mak.in: set NO_OPENSSL and APPLE_COMMON_CRYPTO for macOS >10.11

[Paul Smith]

On Mon, 2016-11-07 at 12:46 -0500, Jeff King wrote:
> Specifically I wanted to make sure that
> 
>   FOO = bar
>   FOO =
>   ifdef FOO
>   ... something ...
>   endif
> 
> works as if FOO had never been set in the first place. Which it seems
> to, at least in GNU make (and that is the only one we support, for
> other reasons).

Yes, it will work.  Confusingly, "ifdef" actually tests whether the
variable has a non-empty value, not whether it's defined:

> The 'ifdef' form takes the _name_ of a variable as its argument, not
> a reference to a variable.  The value of that variable has a non-
> empty value, the TEXT-IF-TRUE is effective; otherwise, the TEXT-IF-
> FALSE, if any, is effective

*sigh* History...

Re: [PATCH v1 1/2] config.mak.in: set NO_OPENSSL and APPLE_COMMON_CRYPTO for macOS >10.11

[Paul Smith]

On Mon, 2016-11-07 at 12:26 -0500, Jeff King wrote:
> I have in the back of my mind a fear that it is harder to unset a
> make variable than it is to override it with a new value (which is
> what you'd want to do here to turn openssl back on),

It depends on what you mean by "unset".

If you mean it as per the shell "unset" command, where the variable is
completely forgotten as if it never was set at all, that's tricky.  You
have to use the "undefine" special command which was introduced in GNU
make 3.82 (released in 2010).

But if you just want to set the variable to the empty string, using
"FOO=" works fine for that in all versions of make (GNU and otherwise)
and using all the normal rules (command line override, etc.)

It's not easy to distinguish between a variable that is empty and one
that is actually not defined, in make, so it's a difference without a
distinction in almost all situations.

Re: git merge deletes my changes

[Paul Smith]

On Mon, 2016-10-10 at 10:19 +, Eduard Egorov wrote:
> # ~/gitbuild/git-2.10.1/git merge -s subtree --squash ceph_ansible
> 
> Can somebody confirm this please? Doesn't "merge -s subtree" really
> merges branches?

I think possibly you're not fully understanding what the --squash flag
does... that's what's causing your issue here, not the "-s" option.

A squash merge takes the commits that would be merged from the origin
branch and squashes them into a single patch and applies them to the
current branch as a new commit... but this new commit is not a merge
commit (that is, when you look at it with "git show" etc. the commit
will have only one parent, not two--or more--parents like a normal merge
commit).

Basically, it's syntactic sugar for a diff plus patch operation plus
some Git goodness wrapped around it to make it easier to use.

But ultimately once you're done, Git has no idea that this new commit
has any relationship whatsoever to the origin branch.  So the next time
you merge, Git doesn't know that there was a previous merge and it will
try to merge everything from scratch rather than starting at the
previous common merge point.

So either you'll have to use a normal, non-squash merge, or else you'll
have to tell Git by hand what the previous common merge point was (as
Jeff King's excellent email suggests).  Or else, you'll have to live
with this behavior.

Re: git 2.9.2: is RUNTIME_PREFIX supposed to work?

[Paul Smith]

On Mon, 2016-09-26 at 14:57 -0700, Junio C Hamano wrote:
> On Mon, Sep 26, 2016 at 2:32 PM, Paul Smith <p...@mad-scientist.net> wrote:
> > 
> > Hi all.  I'm trying to create a relocatable installation of Git 2.9.2,
> > so I can copy it anywhere and it continues to run without any problem.
> > This is on GNU/Linux systems, FWIW.
> 
> I had an impression that the setting was only to support MS Windows.

Hm.  You may be right.  If so that's too bad, because a relocatable Git
is very handy even on UNIX systems.  Is there a reason for invoking the
subcommands by providing the plain command ("fetch", "merge-base") as
argv[0], rather than giving the fully-qualified path to a Git command?

git 2.9.2: is RUNTIME_PREFIX supposed to work?

[Paul Smith]

Hi all.  I'm trying to create a relocatable installation of Git 2.9.2,
so I can copy it anywhere and it continues to run without any problem.
This is on GNU/Linux systems, FWIW.

Looking through the code (for some other reason) I discovered the
RUNTIME_PREFIX setting which appears to attempt to set up the system
paths based on a prefix determined from the directory containing the
git command.  That looks like exactly what I want.

If I set RUNTIME_PREFIX=YesPlease and gitexecdir=libexec/git-core on
the make invocation, it appears to do the right thing when I invoke
.../libexec/git-core/git, even if I move it around.  Cool!

Except, when it doesn't.  And when it doesn't is all the situations
where Git runs subcommands: for example, "git pull" which wants to
invoke fetch and merge-base commands.

When RUNTIME_PREFIX is defined, it's a requirement that the argv[0] for
the process be a fully-qualified pathname: if it's not, then git will
assert at exec_cmd.c:23 in system_path():

assert(argv0_path);

The argv0_path variable is set based on argv[0] passed in to main().
When I invoke top-level Git commands, that value is a fully-qualified
path just as you'd expect.  However, when Git itself invokes
subcommands it does so in a weird way where argv[0] is the command it
wants to invoke, using the magical execv() facility that lets you
invoke a command while providing a different value as argv[0].

For example my core dump from the assert of the merge-base shows:

  (gdb) p argv[0]
  $2 = 0x7fffd70c338e "merge-base"
  (gdb) p argv[1]
  $3 = 0x7fffd70c3399 "--fork-point"
  (gdb) p argv[2]
  $4 = 0x7fffd70c33a6 "refs/remotes/origin/master"
  (gdb) p argv[3]
  $5 = 0x7fffd70c33c1 "master"

Looking at builtin/pull.c I see get_rebase_fork_point() calls
capture_command() with this Git command, which calls start_command(),
which calls execv_git_command(), which calls sane_execvp(), which
invokes "git" as the filename but with argv[0] as "merge-base":

        sane_execvp("git", (char **)nargv.argv);
           ...calls sane_execvp():
        if (!execvp(file, argv))
return 0; /* cannot happen ;-) */

This causes the assert.

So, my question is: is this a bug in RUNTIME_PREFIX support?  Or is
RUNTIME_PREFIX no longer supported, or maybe not supported at all on
UNIX-type operating systems?

Cheers!

Building Git with HTTPS support: avoiding libcurl?

[Paul Smith]

I'm trying to build Git (2.6.4) on GNU/Linux, but without any
requirements (other than basic libc etc.) on the local system.  This
works fine except for one thing: git-remote-https.

In order to build this I need to have libcurl, but libcurl is a MONSTER
library with an enormous number of prerequisites (see below).

Just wondering if anyone has considered an alternative to libcurl; maybe
I'm wrong but it seems to me that HTTPS support for Git would require
only a tiny fraction of the libcurl features and maybe there's an
alternative available which would be more targeted?

I realize this is not a short-term thing in that there won't be an API
compatible library that can just be dropped in.  This is more a forward
-looking question.  For now I'm looking to see if I can rebuild libcurl
myself without most of these dependencies such as Kerberos, LDAP, etc.


$ ldd /usr/lib/x86_64-linux-gnu/libcurl.so.4
linux-vdso.so.1 =>  (0x7fff37d81000)
libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 
(0x7f682b921000)
librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 
(0x7f682b704000)
libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 
(0x7f682b49a000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 
(0x7f682b058000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 
(0x7f682ae0e000)
liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 
(0x7f682abfe000)
libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 
(0x7f682a9ac000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7f682a792000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 
(0x7f682a573000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f682a1a9000)
libgnutls-deb0.so.28 => /usr/lib/x86_64-linux-gnu/libgnutls-deb0.so.28 
(0x7f6829e8d000)
libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 
(0x7f6829c59000)
libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 
(0x7f6829a23000)
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 
(0x7f68297a3000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f682959e000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 
(0x7f68292cc000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 
(0x7f682909d000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 
(0x7f6828e98000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 
(0x7f6828c8d000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 
(0x7f6828a71000)
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 
(0x7f6828855000)
libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 
(0x7f6828615000)
/lib64/ld-linux-x86-64.so.2 (0x559b03259000)
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 
(0x7f68283b)
libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 
(0x7f682819c000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 
(0x7f6827f98000)
libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 
(0x7f6827d8e000)
libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 
(0x7f6827b04000)
libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 
(0x7f6827861000)
libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 
(0x7f682762d000)
libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 
(0x7f6827418000)
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 
(0x7f682721)
libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 
(0x7f6826fe6000)
libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 
(0x7f6826dd7000)
libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 
(0x7f6826b8c000)
libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 
(0x7f68268be000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 
(0x7f6826686000)
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Building Git with HTTPS support: avoiding libcurl?

[Paul Smith]

On Tue, 2015-12-22 at 09:08 -0800, Dave Borowitz wrote:
> Well, IIUC one of the reasons for Git's fork-everything strategy is to
> avoid having to dynamically link the core git binary against large
> libraries like libcurl, so the dependency size has been taken into
> account at least in that sense.

Sure, and it's great that I still get most Git even if I don't have
libcurl.

But without support for cloning https remotes there's a significant hole
in Git functionality...

I grok that Git doesn't want to re-invent the wheel and that libcurl is
convenient.  I just wonder if anyone knows of another wheel, that
doesn't come attached to an entire tractor-trailer, that could be used
instead :).
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: add windows compatibility

[Paul Smith]

On Tue, 2015-05-26 at 11:53 +0200, Johannes Schindelin wrote:
 The biggest problem with `mklink` is that it is only supported on
 Windows Vista and later, while I really like to keep Windows XP
 support in Git for Windows.

No, the biggest problem with mklink is that you have to have
administrative privileges to use it... from wikipedia:

http://en.wikipedia.org/wiki/NTFS_symbolic_link

 The default security settings in Windows Vista/Windows 7 disallow
 non-elevated administrators and all non-administrators from creating
 symbolic links. This behavior can be changed running secpol.msc the
 Local Security Policy management console (under: Security Settings
 \Local Policies\User Rights Assignment\Create symbolic links). It can
 be worked around by starting cmd.exe with Run as administrator option
 or the runas command.

Except even that is not so simple, as various StackOverflow questions
and answers will show (I have to run so I can't look them up now).  I
did try to get this to work a year or so ago, and although I'm in no way
a Windows person (so maybe someone else would have better luck) I
couldn't get it to work.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

profile-fast is failing in my Git 2.2.1 build from tar in a Git repo

[Paul Smith]

Is anyone aware of this?  It seems that profile-fast fails when invoked
from a downloaded tarball, if you are in a Git repository when you
unpack it.

So, for example, I have:

  $ cd $HOME/src
  $ git status
  On branch master
  Your branch is up-to-date with 'origin/master'.

(this is NOT the Git source repo, this is my personal repo containing
scripts used to build various source packages)

Then I download git-2.2.1.tar.xz, unpack it, configure, and run make
profile-fast.

After it builds everything it starts trying to run the tests, and the
t/perf/run script fails (I added some -x flags in the scripts):

  === Running 9 tests in this tree ===
  cp: cannot stat 
'/home/psmith/src/git-2.2.1/t/..//home/psmith/src/.git/objects': No such file 
or directory
  error: failed to copy repository '/home/psmith/src/git-2.2.1/t/..' to 
'/home/psmith/src/git-2.2.1/t/perf/trash directory.p-perf-lib-sanity'

The problem is in the t/perf/perf-lib.sh:test_perf_create_repo_from()
function, where we see this:

  repo=$1
  source=$2
  source_git=$source/$(cd $source  git rev-parse --git-dir)

The function is invoked as:

  test_perf_create_repo_from '/home/psmith/src/git-2.2.1/t/perf/trash 
directory.p5302-pack-index' /home/psmith/src/git-2.2.1/t/..

but this doesn't work, because $source is not a Git repository; it's
the root of the unpacked tarball.  But when we run git rev-parse in
it, it finds the parent Git directory ($HOME/git) and uses that, so the
value of source_git becomes:

  /home/psmith/src/git-2.2.1/t/..//home/psmith/src/.git/objects

which is obviously invalid.  I think you want source_git to be set
something like this:

  source_git=$(cd $source  git rev-parse --git-dir || echo $source)

instead.  There are other ways to do this of course, but this worked for
me... basically we want to use either git rev-parse OR $source but not
both.  I think...?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Wed, 2014-11-26 at 15:16 -0800, Junio C Hamano wrote:
 Paul Smith p...@mad-scientist.net writes:
 
  This is what happens for a file:
 
  $ rm -f foo
 
  $ touch foo
 
  $ ./src/git/contrib/workdir/git-new-workdir src/git foo master
  mkdir: cannot create directory ‘foo’: Not a directory
  unable to create new workdir foo!
 
 ;-)  That comes from mkdir || fail which is indeed sufficient.

Hi Junio;

Is this all set to be applied or is there more to do?

Cheers!

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Wed, 2014-11-26 at 15:16 -0800, Junio C Hamano wrote:
  $ ./src/git/contrib/workdir/git-new-workdir src/git foo master
  mkdir: cannot create directory ‘foo’: Not a directory
  unable to create new workdir foo!
 
 ;-)  That comes from mkdir || fail which is indeed sufficient.

Right.  Often I find it simpler/clearer to let the underlying commands
give the errors: they use perror() and can often provide more specific
error messages than my script can, unless I spend a lot of effort trying
to determine exactly what the problem is (permissions, disk space, bad
symlink, existing file, whatever).

Should I respin this with the \$new_workdir\ - '$new_workdir' change
(I actually prefer the latter myself but the former was used somewhere
so I kept it)?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Why does git merge --squash fail when merge.ff is set to only?

[Paul Smith]

In our development process we always want to do rebase and only rarely
create merge commits, and so we have a recommendation to set the
merge.ff configuration attribute to only.

This is great, however it appears to break git merge --squash (which we
also use constantly).

If I'm squash-merging from a branch which is not based on HEAD (cannot
use fast-forwarding) then I get the extremely cryptic error:

  $ git checkout master
  $ git pull
  $ git merge --squash my-branch
  fatal: Not possible to fast-forward, aborting.

I couldn't even understand what this error meant for a while: what was
being fast-forwarded here anyway?  It took me a lot of thought to
realize it was related to the merge.ff config setting.

Eventually I figured it out, and now have this workaround:

  $ git merge --ff --squash my-branch

But, shouldn't we consider this a bug?  I don't see any reason why
--squash should pay attention to the ff config setting, or command line
flags either for that matter.  IMHO when you add the --squash flag, the
ff options/config should be ignored.

Or, am I missing some subtle issue here?

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

Allow new workdirs to be created in an empty directory (similar to git
clone).  Provide more error checking and clean up on failure.

Signed-off-by: Paul Smith p...@mad-scientist.net
---

Hopefully this doesn't contain unwanted stylistic changes.  There's a
kind of gross thing about the behavior here: because we cd much earlier
than in my previous version we could actually trap and rm -rf while
our current working directory is inside the removed location (inside
$cleandir).  On a traditional UNIX filesystem this is fine, of course,
but it's possible that some filesystems don't like this in some way.  In
Windows for example this will definitely fail (but new-workdir doesn't
support Windows anyway).  The only options are (a) don't worry about it
(that's what this patch does), (b) put the cd after the trap reset
again, (c) move the trap reset earlier and don't clean up if we get
failures after we cd, or (d) cd again inside the cleanup function.
Of these, (d) is the most complex and hence my least favorite.

 contrib/workdir/git-new-workdir | 53 +
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..d1d7f32 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -10,6 +10,10 @@ die () {
exit 128
 }
 
+failed () {
+   die unable to create new workdir \$new_workdir\!
+}
+
 if test $# -lt 2 || test $# -gt 3
 then
usage $0 repository new_workdir [branch]
@@ -35,7 +39,7 @@ esac
 
 # don't link to a configured bare repository
 isbare=$(git --git-dir=$git_dir config --bool --get core.bare)
-if test ztrue = z$isbare
+if test ztrue = z$isbare
 then
die \$git_dir\ has core.bare set to true, \
 remove from \$git_dir/config\ to use $0
@@ -48,35 +52,54 @@ then
a complete repository.
 fi
 
-# don't recreate a workdir over an existing repository
-if test -e $new_workdir
+# make sure the links in the workdir have full paths to the original repo
+git_dir=$(cd $git_dir  pwd) || exit 1
+
+# don't recreate a workdir over an existing directory, unless it's empty
+if test -d $new_workdir
 then
-   die destination directory '$new_workdir' already exists.
+   if test $(ls -a1 $new_workdir/. | wc -l) -ne 2
+   then
+   die destination directory '$new_workdir' is not empty.
+   fi
+   cleandir=$new_workdir/.git
+else
+   cleandir=$new_workdir
 fi
 
-# make sure the links use full paths
-git_dir=$(cd $git_dir; pwd)
+mkdir -p $new_workdir/.git || failed
+cleandir=$(cd $cleandir  pwd) || failed
 
-# create the workdir
-mkdir -p $new_workdir/.git || die unable to create \$new_workdir\!
+cleanup () {
+   rm -rf $cleandir
+}
+siglist=0 1 2 15
+trap cleanup $siglist
 
 # create the links to the original repo.  explicitly exclude index, HEAD and
 # logs/HEAD from the list since they are purely related to the current working
 # directory, and should not be shared.
 for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache 
svn
 do
+   # create a containing directory if needed
case $x in
*/*)
-   mkdir -p $(dirname $new_workdir/.git/$x)
+   mkdir -p $new_workdir/.git/${x%/*}
;;
esac
-   ln -s $git_dir/$x $new_workdir/.git/$x
+
+   ln -s $git_dir/$x $new_workdir/.git/$x || failed
 done
 
-# now setup the workdir
-cd $new_workdir
+# commands below this are run in the context of the new workdir
+cd $new_workdir || failed
+
 # copy the HEAD from the original repository as a default branch
-cp $git_dir/HEAD .git/HEAD
-# checkout the branch (either the same as HEAD from the original repository, or
-# the one that was asked for)
+cp $git_dir/HEAD .git/HEAD || failed
+
+# the workdir is set up.  if the checkout fails, the user can fix it.
+trap - $siglist
+
+# checkout the branch (either the same as HEAD from the original repository,
+# or the one that was asked for)
 git checkout -f $branch
-- 
1.8.5.3



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v4] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Wed, 2014-11-26 at 13:55 -0800, Junio C Hamano wrote:
 The comment in the original is somewhat misleading, but test -e
 was test -e and not test -d to stop when an existing file was
 given by mistake as $new_workdir, I think.  I do not know what
 happens in the new code in that case.

I did test that.  I have a little set of tests with a no directory,
empty directory, non-empty directory, plus various permissions issues
(existing directory without write privs, no write privs to the parent
directory), and also if the new directory name is a file, a symlink
pointing to something, a symlink pointing to nothing, etc.

This is what happens for a file:

$ rm -f foo

$ touch foo

$ ./src/git/contrib/workdir/git-new-workdir src/git foo master
mkdir: cannot create directory ‘foo’: Not a directory
unable to create new workdir foo!


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Thu, 2014-11-20 at 09:13 -0800, Junio C Hamano wrote:
 Paul Smith p...@mad-scientist.net writes:
  +# don't recreate a workdir over an existing directory, unless it's empty
  +if test -d $new_workdir
   then
  -   die destination directory '$new_workdir' already exists.
  +   if test $(ls -a1 $new_workdir/. | wc -l) -ne 2
 
 You used to quote this as
 
   if test $(ls -a1 $new_workdir/. | wc -l) -ne 2
 
 and I think that made a lot more sense than the new quoting.

Just personal script coding style.  I tend to quote just variables, in
case I want to add globbing or whatever later.  I've run into people who
think that $foo/bar needs quotes because it concatenates two strings,
but you don't need to quote $foo by itself; quoting just the variable
helps avoid that impression.

But this doesn't matter much to me.

 Case arms come at the same indentation level as case/esac.

OK.

 Why these changes?  I thought you are making sure $cleandir is
 absolute so that you do not have to do this and can just cd into
 the new working tree, the same way the user would do once it is set
 up.

I made cleandir absolute mainly because you asked me to :-).  I didn't
realize that the implication behind that request was that I'd put back
the original cd as well.

I personally distrust and avoid raw cd in shell scripts.  It's
action-at-a-distance and can cause all sorts of problems, sometimes
disastrous ones.  If I need to change directories I localize it, e.g.:

  (cd $somedir  do some command)

so it's clear that it's in effect only for that command.

I understand that in this particular situation the cd is right before
the end so this doesn't hurt (as the script stands today).  If you
prefer the cd explicitly I'll add it back.  You're the one that needs
to be happy with it :-).

What if I move the cd down to right before the checkout command, and use
$new_workdir in the copy of HEAD?  At least it won't be in effect until
after the workdir is completely set up.  I would feel better about it
then :-).

  -# now setup the workdir
  -cd $new_workdir
   # copy the HEAD from the original repository as a default branch
  -cp $git_dir/HEAD .git/HEAD
  -# checkout the branch (either the same as HEAD from the original 
  repository, or
  -# the one that was asked for)
  -git checkout -f $branch

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Wed, 2014-11-19 at 09:32 -0800, Junio C Hamano wrote:
 Paul Smith p...@mad-scientist.net writes:
 I took a look at this again, and I do not agree with one design
 decision it makes, namely:
 
  I split the creation of the directories from the symlinks: see the new
  loop above.  This allows us to avoid the icky dirname stuff.
 
 which forces those who maintain the script to make sure that these
 two loops
 kept consistent with each other.  If you forget to add frotz to the
 upper loop when adding frotz/nitfol to the latter, you are breaking
 it.

Yes, but this mistake won't live past even a single attempt to run the
script, since it will immediately cause a fatal error.  So this doesn't
bother me.

 I find it much more icky than computing what is necessary on the fly.

OK, I'll go back to the previous way.

 Aversion to turning $cleandir to an absolute path?  Why?

In some situations switching to absolute can cause unexpected behaviors,
where relative paths are no longer where you expect them etc.  However
in this case it's not a problem: I'll change it.

 These uses of --git-dir/--work-tree look somewhat funny.  You want
 to say I want to run checkout in that $new_workdir, so say it in a
 more direct way, i.e.
 
 git -C $new_workdir checkout -f $branch

Good idea.

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

Allow new workdirs to be created in an empty directory (similar to git
clone).  Provide more error checking and clean up on failure.

Signed-off-by: Paul Smith p...@mad-scientist.net
---
 contrib/workdir/git-new-workdir | 54 +++--
 1 file changed, 36 insertions(+), 18 deletions(-)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..7334720 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -10,6 +10,10 @@ die () {
exit 128
 }
 
+failed () {
+   die unable to create new workdir \$new_workdir\!
+}
+
 if test $# -lt 2 || test $# -gt 3
 then
usage $0 repository new_workdir [branch]
@@ -35,7 +39,7 @@ esac
 
 # don't link to a configured bare repository
 isbare=$(git --git-dir=$git_dir config --bool --get core.bare)
-if test ztrue = z$isbare
+if test ztrue = z$isbare
 then
die \$git_dir\ has core.bare set to true, \
 remove from \$git_dir/config\ to use $0
@@ -48,35 +52,49 @@ then
a complete repository.
 fi
 
-# don't recreate a workdir over an existing repository
-if test -e $new_workdir
+# make sure the links in the workdir have full paths to the original repo
+git_dir=$(cd $git_dir  pwd) || exit 1
+
+# don't recreate a workdir over an existing directory, unless it's empty
+if test -d $new_workdir
 then
-   die destination directory '$new_workdir' already exists.
+   if test $(ls -a1 $new_workdir/. | wc -l) -ne 2
+   then
+   die destination directory '$new_workdir' is not empty.
+   fi
+   cleandir=$new_workdir/.git
+else
+   cleandir=$new_workdir
 fi
 
-# make sure the links use full paths
-git_dir=$(cd $git_dir; pwd)
+mkdir -p $new_workdir/.git || failed
+cleandir=$(cd $cleandir  pwd) || failed
 
-# create the workdir
-mkdir -p $new_workdir/.git || die unable to create \$new_workdir\!
+cleanup () {
+   rm -rf $cleandir
+}
+siglist=0 1 2 15
+trap cleanup $siglist
 
 # create the links to the original repo.  explicitly exclude index, HEAD and
 # logs/HEAD from the list since they are purely related to the current working
 # directory, and should not be shared.
 for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache 
svn
 do
+   # Create a containing directory if needed
case $x in
-   */*)
-   mkdir -p $(dirname $new_workdir/.git/$x)
-   ;;
+   */*) mkdir -p $new_workdir/.git/${x%/*} ;;
esac
-   ln -s $git_dir/$x $new_workdir/.git/$x
+
+   ln -s $git_dir/$x $new_workdir/.git/$x || failed
 done
 
-# now setup the workdir
-cd $new_workdir
 # copy the HEAD from the original repository as a default branch
-cp $git_dir/HEAD .git/HEAD
-# checkout the branch (either the same as HEAD from the original repository, or
-# the one that was asked for)
-git checkout -f $branch
+cp $git_dir/HEAD $new_workdir/.git/HEAD || failed
+
+# the workdir is set up.  if the checkout fails, the user can fix it.
+trap - $siglist
+
+# checkout the branch (either the same as HEAD from the original repository,
+# or the one that was asked for)
+git -C $new_workdir checkout -f $branch
-- 
1.8.5.3



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

Junio C Hamano gits...@pobox.com writes:
 Paul Smith p...@mad-scientist.net writes:

 No need to resend only to correct the above, even though there may
 be comments on the patch itself from me or others that may make you
 want to reroll this patch, in which case I'd like to see these nits
 gone.

I'll fix in the next iteration.

 -# don't recreate a workdir over an existing repository
 -if test -e $new_workdir
 +# make sure the links use full paths
 +git_dir=$(cd $git_dir; pwd)

 With this change, the comment gets much harder to understand.  What
 links?  would be the reaction from those who are reading the patch.

I just moved this line; I don't think it had much more context
beforehand, but I'll definitely rewrite the comment to be more clear.

 +if test $(ls -a1 $new_workdir/. | wc -l) -ne 2

 I wonder if this check is portable for all platforms we care about,
 but that is OK, as it should be so for the ones I think of and care
 about ;-)

Do you mean . and .. representing an empty directory?  That will
work on any system where /bin/sh works, for sure.

Or do you mean using ls and wc?  I can easily avoid this.

Recall that new-workdir itself only works on systems that support
symlinks; this limits its portability.  For example it doesn't work on
Windows (unfortunately).  I spent the better part of a day a few months
ago playing with the various symlink-ish capabilities of Windows NTFS
and couldn't find any reliable way to make this work (although I have
virtually no Windows fu).

 The script chdirs around; did you turn $new_workdir into an absolute
 path already, or given a relative $new_workdir this is attempting to
 remove a hierarchy that is different from what you created?

Good point, I'll fix this up

 +}
 +siglist=0 1 2 15
 +trap cleanup $siglist
  
 -# create the workdir
 -mkdir -p $new_workdir/.git || die unable to create \$new_workdir\!
 +# create embedded directories
 +for x in logs
 +do
 +mkdir -p $new_workdir/.git/$x || failed
 +done
  
  # create the links to the original repo.  explicitly exclude index, HEAD and
  # logs/HEAD from the list since they are purely related to the current
 working
  # directory, and should not be shared.
  for x in config refs logs/refs objects info hooks packed-refs remotes
 rr-cache svn
  do
 -case $x in
 -*/*)
 -mkdir -p $(dirname $new_workdir/.git/$x)
 -;;
 -esac

 What's this removal about?  If $new_workdir/.git/logs does not
 exist, would ln -s $there/logs/refs $new_workdir/.git/logs/refs
 succeed without first creating $new_workdir/.git/logs directory?

I split the creation of the directories from the symlinks: see the new
loop above.  This allows us to avoid the icky dirname stuff.

New iteration will follow shortly.

Cheers!

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

Allow new workdirs to be created in an empty directory (similar to git
clone).  Provide more error checking and clean up on failure.

Signed-off-by: Paul Smith p...@mad-scientist.net
---

Getting rid of ls/wc was not as simple as I'd hoped, due to glob
pathname expansion (can't rely on nullglob e.g.)  If you want this let
me know; it will require some yucky code to do the whole thing in native
shell.  Since new-workdir only works on systems with ln -s I think we
can feel confident requiring ls and wc as well.

 contrib/workdir/git-new-workdir | 55 +++--
 1 file changed, 36 insertions(+), 19 deletions(-)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..aef632d 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -10,6 +10,10 @@ die () {
exit 128
 }
 
+failed () {
+   die unable to create new workdir \$new_workdir\!
+}
+
 if test $# -lt 2 || test $# -gt 3
 then
usage $0 repository new_workdir [branch]
@@ -48,35 +52,48 @@ then
a complete repository.
 fi
 
-# don't recreate a workdir over an existing repository
-if test -e $new_workdir
+# make sure the links in the workdir have full paths to the original repo
+git_dir=$(cd $git_dir; pwd)
+
+# don't recreate a workdir over an existing directory, unless it's empty
+if test -d $new_workdir
 then
-   die destination directory '$new_workdir' already exists.
+   if test $(ls -a1 $new_workdir/. | wc -l) -ne 2
+   then
+   die destination directory '$new_workdir' is not empty.
+   fi
+   cleandir=$new_workdir/.git
+else
+   mkdir -p $new_workdir || failed
+   cleandir=$new_workdir
 fi
 
-# make sure the links use full paths
-git_dir=$(cd $git_dir; pwd)
+cleanup () {
+   rm -rf $cleandir
+}
+siglist=0 1 2 15
+trap cleanup $siglist
 
-# create the workdir
-mkdir -p $new_workdir/.git || die unable to create \$new_workdir\!
+# create embedded directories
+for x in logs
+do
+   mkdir -p $new_workdir/.git/$x || failed
+done
 
 # create the links to the original repo.  explicitly exclude index, HEAD and
 # logs/HEAD from the list since they are purely related to the current working
 # directory, and should not be shared.
 for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache 
svn
 do
-   case $x in
-   */*)
-   mkdir -p $(dirname $new_workdir/.git/$x)
-   ;;
-   esac
-   ln -s $git_dir/$x $new_workdir/.git/$x
+   ln -s $git_dir/$x $new_workdir/.git/$x || failed
 done
 
-# now setup the workdir
-cd $new_workdir
 # copy the HEAD from the original repository as a default branch
-cp $git_dir/HEAD .git/HEAD
-# checkout the branch (either the same as HEAD from the original repository, or
-# the one that was asked for)
-git checkout -f $branch
+cp $git_dir/HEAD $new_workdir/.git/HEAD || failed
+
+# the workdir is set up.  if the checkout fails, the user can fix it.
+trap - $siglist
+
+# checkout the branch (either the same as HEAD from the original repository,
+# or the one that was asked for)
+git --git-dir=$new_workdir/.git --work-tree=$new_workdir checkout -f 
$branch
-- 
1.8.5.3


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Tue, 2014-11-18 at 12:15 -0800, Junio C Hamano wrote:
 Paul Smith p...@mad-scientist.net writes:
 
  Getting rid of ls/wc was not as simple as I'd hoped,
 
 I didn't say ls/wc was not portable.  Assuming that a directory for
 which the output from ls -a has two entries is empty may be.

Yes, I didn't get your email before I sent the latest patch.  Sorry
about that.

On Tue, 2014-11-18 at 11:32 -0800, Junio C Hamano wrote:
 Even on network mounts from esoteric filesystems and such?  When
 
 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html
 
 mentions the -A option, it says:
 
 -A
 Write out all directory entries, including those whose names
 begin with a period ( '.' ) but excluding the entries dot
 and dot-dot (if they exist).
 
 The if they exist part suggests, at least to me, that it is valid
 for a POSIX filesystem to lack these two (I suspect that one may be
 able to find a more definitive answer from other parts of the POSIX
 but I didn't bother).

Hm.  Well, POSIX clearly reserves . and .. to be special and
requires that directories containing only those values are considered
empty (rmdir(2) says so).  The definitions section contains special
wording for dot and dot-dot.

Looking at
http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap04.html

Each directory has exactly one parent directory which is
represented by the name dot-dot in the first directory.

which implies that every directory must have ...  This is in the
Rationale, I realize.

There are also various mentions to using . as a synonym for the
current directory.

I can't find a clear statement that both are required and that ls -a
must show them.  I've used a wide range of UNIX-en and filesystems for
30 years or so and never seen one that didn't provide them.  It seems
like it would break quite a few scripts, at least.  Even virtual
filesystems like ClearCase's MVFS provide . and ...

If you want to allow for this possibility I can do so but it would be a
bit crufty.  Personally I think it would be overkill but you're the
boss: let me know :-).

Cheers!

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Tue, 2014-11-18 at 12:58 -0800, Junio C Hamano wrote:
 Doesn't the description of the -A option I quoted upthread hint a
 simpler and clearer solution?  I.e. test $(ls -A | wc -l) = 0?

Yes, but unfortunately for us the -A flag was added to POSIX Issue 7.
It's not present in the previous version of POSIX, Issue 6:

http://pubs.opengroup.org/onlinepubs/009695399/utilities/ls.html

It came from the BSD world, so it might not be available on older
SysV-derived systems (AIX, HP-UX, even Solaris... I don't have access to
these anymore so I can't say).  Ultimately it's probably more portable
to assume ls -a always prints . and .. than to assume ls -A is
supported.

Cheers!

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

On Tue, 2014-11-18 at 14:51 -0800, Junio C Hamano wrote:
 OK, thanks for digging. Let's go with this version, then.

Thanks for your attention, Junio!

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] git-new-workdir: Don't fail if the target directory is empty

[Paul Smith]

From 545c0d526eaa41f9306b567275a7d53799987482 Mon Sep 17 00:00:00 2001
From: Paul Smith p...@mad-scientist.net
Date: Fri, 14 Nov 2014 17:11:19 -0500
Subject: [PATCH] git-new-workdir: Don't fail if the target directory is empty

Also provide more error checking and clean up on failure.

Signed-off-by: Paul Smith p...@mad-scientist.net
---

Thanks Junio.  I've reworked the change so it will automatically succeed
if the directory does not exists or exists but is empty, and fail
otherwise, which as far as I can tell is the behavior git clone uses
as well.  I removed the -f flag as no longer needed.  I also added some
cleanup that is performed if the new-workdir operation fails for any
reason so you don't get partly-constructed workdirs.  I also added more
error checking so that we immediately stop if any step fails.  Some may
suggest set -e but that flag can be tricky... I preferred to make the
failure explicit.

 contrib/workdir/git-new-workdir | 54 +
 1 file changed, 39 insertions(+), 15 deletions(-)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..c402000 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -10,6 +10,10 @@ die () {
exit 128
 }
 
+failed () {
+   die unable to create new workdir \$new_workdir\!
+}
+
 if test $# -lt 2 || test $# -gt 3
 then
usage $0 repository new_workdir [branch]
@@ -48,35 +52,55 @@ then
a complete repository.
 fi
 
-# don't recreate a workdir over an existing repository
-if test -e $new_workdir
+# make sure the links use full paths
+git_dir=$(cd $git_dir; pwd)
+
+# don't recreate a workdir over an existing directory unless it's empty
+if test -d $new_workdir
 then
-   die destination directory '$new_workdir' already exists.
+   if test $(ls -a1 $new_workdir/. | wc -l) -ne 2
+   then
+   die destination directory '$new_workdir' is not empty.
+   fi
+   was_existing=true
+else
+   mkdir -p $new_workdir || failed
+   was_existing=false
 fi
 
-# make sure the links use full paths
-git_dir=$(cd $git_dir; pwd)
+cleanup () {
+   if $was_existing
+   then
+   rm -rf $new_workdir/* $new_workdir/.[!.] $new_workdir/.??*
+   else
+   rm -rf $new_workdir
+   fi
+}
+siglist=0 1 2 15
+trap cleanup $siglist
 
-# create the workdir
-mkdir -p $new_workdir/.git || die unable to create \$new_workdir\!
+# create embedded directories
+for x in logs
+do
+   mkdir -p $new_workdir/.git/$x || failed
+done
 
 # create the links to the original repo.  explicitly exclude index, HEAD and
 # logs/HEAD from the list since they are purely related to the current working
 # directory, and should not be shared.
 for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache 
svn
 do
-   case $x in
-   */*)
-   mkdir -p $(dirname $new_workdir/.git/$x)
-   ;;
-   esac
-   ln -s $git_dir/$x $new_workdir/.git/$x
+   ln -s $git_dir/$x $new_workdir/.git/$x || failed
 done
 
 # now setup the workdir
-cd $new_workdir
+cd $new_workdir || failed
 # copy the HEAD from the original repository as a default branch
-cp $git_dir/HEAD .git/HEAD
+cp $git_dir/HEAD .git/HEAD || failed
+
+# don't delete the new workdir on exit
+trap - $siglist
+
 # checkout the branch (either the same as HEAD from the original repository, or
 # the one that was asked for)
 git checkout -f $branch
-- 
1.8.5.3



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] git-new-workdir: Add -f to force new-workdir in existing directory.

[Paul Smith]

From: Paul Smith p...@mad-scientist.net
Date: Thu, 13 Nov 2014 14:01:34 -0500
Subject: [PATCH] git-new-workdir: Add -f to force new-workdir in existing 
directory.

Signed-off-by: Paul Smith psm...@mad-scientist.net
---

I have an environment I want to use new-workdir for, where the directory
I need to use is pre-created for me and I'm dropped into that directory
and I have no control over this (it's an automated build system).  The
directory is empty but git-new-workdir still is unhappy about it.  I
added a -f flag to allow the user to force git-new-workdir to continue
even if the directory exists.  It still bails if there's a .git
directory already, however.

 contrib/workdir/git-new-workdir | 16 +---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir
index 75e8b25..a4079c1 100755
--- a/contrib/workdir/git-new-workdir
+++ b/contrib/workdir/git-new-workdir
@@ -10,11 +10,17 @@ die () {
exit 128
 }
 
-if test $# -lt 2 || test $# -gt 3
+if test $# -lt 2 || test $# -gt 4
 then
-   usage $0 repository new_workdir [branch]
+   usage $0 [-f] repository new_workdir [branch]
 fi
 
+force=false
+if [ x$1 = x-f ]
+then
+force=true
+shift
+fi
 orig_git=$1
 new_workdir=$2
 branch=$3
@@ -51,7 +57,11 @@ fi
 # don't recreate a workdir over an existing repository
 if test -e $new_workdir
 then
-   die destination directory '$new_workdir' already exists.
+   $force || die destination directory '$new_workdir' already exists.
+   if test -e $new_workdir/.git
+   then
+   die destination directory '$new_workdir/.git' already exists.
+   fi
 fi
 
 # make sure the links use full paths
-- 
2.1.3



--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC/PATCH] git-completion.bash: remove bashism to fix ZSH compatibility

[Paul Smith]

On Mon, 2013-03-11 at 11:09 -0700, Junio C Hamano wrote:
 So strictly speaking there is no reason for an extra subshell here,
 but writing this in the way the patch does makes our intention
 crystal clear, I think.

If you're concerned about the extra processing of the new shell you can
use {} instead of ():

{
test -n ${CDPATH+set}  unset CDPATH
# NOTE: $2 is not quoted in order to support multiple options
cd $1  git ls-files --exclude-standard $2
} 2/dev/null

Zsh does support this properly in my testing.  It's only redirection of
an entire function body, as the original, that is working differently in
zsh and bash.


--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html