Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Stefan Beller]

On Fri, Nov 3, 2017 at 2:32 AM, Kaartic Sivaraam
 wrote:
> On Monday 23 October 2017 11:07 PM, Stefan Beller wrote:
>>
>> Exactly. By memory I mean volatile RAM (as opposed to
>> memory on a spinning disk).
>>
>> Using GIT_TEST_OPTS has had some issues (I remember vaguely
>> there was an inconsistency between the output of `make test` and prove),
>> so I put my entire working tree on a tmpfs, I run roughly this script
>> after booting my computer:
>>
>>sudo mount -t tmpfs -o size=16g tmpfs /u
>>mkdir /u/git
>>echo "gitdir:
>> /usr/local/google/home/sbeller/OSS/git/.git/worktrees/git"
>>>
>>> /u/git/.git
>>
>>git -C /u/git checkout -f HEAD
>>
>>cat >DEVELOPER=1
>>DEVELOPERS=1
>>CFLAGS += -g -O2
>>CFLAGS += -DFLEX_ARRAY=2048
>>#CFLAGS += -Wno-unused-value
>>EOF
>
>
> Did I thank you for a good explanation? If not, thanks that was interesting
> and enlightening.
>
>> The test suite (excluding t9*) runs in less than 50 seconds on the ram
>> disk.
>>

Just tested again, I meant to say 70s instead of 50s.


> BTW, this is what I call _way way_ faster. Unfortunately due to the limited
> configuration of my system, the test suite has following timing
>
> real3m14.482s
> user2m10.556s
> sys 1m12.328s

This sounds to me as if it is running with just one thread
(because sys+user = real); I usually run with

cd t
prove --jobs 25 t[0-8][0-9]*.sh

The multithreading can be seen in the timing as well
real 1m9.913s
user 1m50.796s
sys 0m54.092s
as user > real already.

If you run tests via 'make test' or 'cd t && make', you can also give
a --jobs 
to make it faster. I have no good answer for how many, but I have the impression
overloading the system is no big deal. (I only have a few cores in this machine,
4 or 6, but still run with --jobs 25; 'git grep sleep' returns a
couple of lines,
and such threads sleeping definitely don't need a CPU)

Thanks,
Stefan

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Kaartic Sivaraam]

On Monday 23 October 2017 11:07 PM, Stefan Beller wrote:

Exactly. By memory I mean volatile RAM (as opposed to
memory on a spinning disk).

Using GIT_TEST_OPTS has had some issues (I remember vaguely
there was an inconsistency between the output of `make test` and prove),
so I put my entire working tree on a tmpfs, I run roughly this script
after booting my computer:

   sudo mount -t tmpfs -o size=16g tmpfs /u
   mkdir /u/git
   echo "gitdir:
/usr/local/google/home/sbeller/OSS/git/.git/worktrees/git"

/u/git/.git

   git -C /u/git checkout -f HEAD

   cat 

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Stefan Beller]

On Sun, Oct 22, 2017 at 5:36 PM, Jeff King  wrote:
> On Sun, Oct 22, 2017 at 07:54:57PM +0530, Kaartic Sivaraam wrote:
>
>> On Fri, 2017-10-20 at 13:45 -0700, Stefan Beller wrote:
>> >
>> > The git-test from Michael sounds intriguing. Initially I put off using
>> > it as I had my main working dir (or rather test dir) on a spinning
>> > disk, still. Now I test in memory only, which is a lot faster, so I could
>> > try if git-test can keep up with my local commit pace.
>> >
>>
>> Excuse my ignorance but I don't get your statement correctly. What do
>> you mean by "I test in memory only"? How do you do that?
>
> I assume it means pointing "--root" at a tmpfs or ram disk in
> GIT_TEST_OPTS. That makes the test suite run _way_ faster.

Exactly. By memory I mean volatile RAM (as opposed to
memory on a spinning disk).

Using GIT_TEST_OPTS has had some issues (I remember vaguely
there was an inconsistency between the output of `make test` and prove),
so I put my entire working tree on a tmpfs, I run roughly this script
after booting my computer:

  sudo mount -t tmpfs -o size=16g tmpfs /u
  mkdir /u/git
  echo "gitdir:
/usr/local/google/home/sbeller/OSS/git/.git/worktrees/git"
>/u/git/.git
  git -C /u/git checkout -f HEAD

  cat 
> -Peff

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Jeff King]

On Sun, Oct 22, 2017 at 07:54:57PM +0530, Kaartic Sivaraam wrote:

> On Fri, 2017-10-20 at 13:45 -0700, Stefan Beller wrote:
> > 
> > The git-test from Michael sounds intriguing. Initially I put off using
> > it as I had my main working dir (or rather test dir) on a spinning
> > disk, still. Now I test in memory only, which is a lot faster, so I could
> > try if git-test can keep up with my local commit pace.
> > 
> 
> Excuse my ignorance but I don't get your statement correctly. What do
> you mean by "I test in memory only"? How do you do that?

I assume it means pointing "--root" at a tmpfs or ram disk in
GIT_TEST_OPTS. That makes the test suite run _way_ faster.

-Peff

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Kaartic Sivaraam]

On Fri, 2017-10-20 at 13:45 -0700, Stefan Beller wrote:
> 
> The git-test from Michael sounds intriguing. Initially I put off using
> it as I had my main working dir (or rather test dir) on a spinning
> disk, still. Now I test in memory only, which is a lot faster, so I could
> try if git-test can keep up with my local commit pace.
> 

Excuse my ignorance but I don't get your statement correctly. What do
you mean by "I test in memory only"? How do you do that?

---
Kaartic

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Stefan Beller]

On Thu, Oct 19, 2017 at 11:04 PM, Jeff King  wrote:
> On Thu, Oct 19, 2017 at 10:27:28PM -0700, Stefan Beller wrote:
>
>> > If my analysis above is correct, then it's already fixed. You just had
>> > leftover corruption.
>>
>> Well fetching yesterday worked and the commit in question is from
>> 8/23, the merge  8a044c7f1d56cef657be342e40de0795d688e882
>> occurred 9/18, so I suspect there is something else at play.
>> (I do not remember having a gc between yesterday and today.
>> Though maybe one in the background?)
>
> Even a gc between yesterday and today should have used the new code,
> which would have been safe. So yeah, maybe it is something else
> entirely.

Oh, yeah.

>
>> I am curious how you can have a worktree owned by multiple
>> repositories [1] (?).
>
> Sorry, I forgot my footnote. I saw this with my "ci" script:
>
>   https://github.com/peff/git/blob/7905ff395adecdd2bb7ab045a24223dfb103e0e9/ci
>
> I check out the contents of my "meta" branch as "Meta", and it contains
> that script. It basically just waits for ref updates, then walks over
> all the commits and runs "make test" on them in the background (caching
> the results, thanks to the git-test[1] script). So I kick off "Meta/ci"
> in a terminal and forget about it, and magically it builds my commits in
> the background as I work.
>
> It operates in a worktree inside the Meta directory (Meta/tmp-ci), so as
> not to disturb what I'm doing. So far so good.
>
> But I actually have _two_ clones of Git on my system. One on which I do
> most of my work, and then the other which has the fork we use in
> production at GitHub. I symlink the Meta directory from the first into
> the latter, which means they both see the same worktree directory. And
> somehow running "Meta/ci" in the second corrupted things.
>
> I can get some funniness now, but I think it's mostly caused by the
> script being confused about the worktree existing but not having access
> to our branches. That's not a corruption, just a confusion. I _think_ I
> had a bogus HEAD in the worktree at one point, but I may be
> mis-remembering. I can't seem to trigger it now.

Thanks for these insights.

I played around with Meta a bit, but I did not feel it would enhance
my workflow enough as I am not involved with any maintainance
of git using git.

The git-test from Michael sounds intriguing. Initially I put off using
it as I had my main working dir (or rather test dir) on a spinning
disk, still. Now I test in memory only, which is a lot faster, so I could
try if git-test can keep up with my local commit pace.

Thanks,
Stefan

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Jeff King]

On Thu, Oct 19, 2017 at 10:27:28PM -0700, Stefan Beller wrote:

> > If my analysis above is correct, then it's already fixed. You just had
> > leftover corruption.
> 
> Well fetching yesterday worked and the commit in question is from
> 8/23, the merge  8a044c7f1d56cef657be342e40de0795d688e882
> occurred 9/18, so I suspect there is something else at play.
> (I do not remember having a gc between yesterday and today.
> Though maybe one in the background?)

Even a gc between yesterday and today should have used the new code,
which would have been safe. So yeah, maybe it is something else
entirely.

> I am curious how you can have a worktree owned by multiple
> repositories [1] (?).

Sorry, I forgot my footnote. I saw this with my "ci" script:

  https://github.com/peff/git/blob/7905ff395adecdd2bb7ab045a24223dfb103e0e9/ci

I check out the contents of my "meta" branch as "Meta", and it contains
that script. It basically just waits for ref updates, then walks over
all the commits and runs "make test" on them in the background (caching
the results, thanks to the git-test[1] script). So I kick off "Meta/ci"
in a terminal and forget about it, and magically it builds my commits in
the background as I work.

It operates in a worktree inside the Meta directory (Meta/tmp-ci), so as
not to disturb what I'm doing. So far so good.

But I actually have _two_ clones of Git on my system. One on which I do
most of my work, and then the other which has the fork we use in
production at GitHub. I symlink the Meta directory from the first into
the latter, which means they both see the same worktree directory. And
somehow running "Meta/ci" in the second corrupted things.

I can get some funniness now, but I think it's mostly caused by the
script being confused about the worktree existing but not having access
to our branches. That's not a corruption, just a confusion. I _think_ I
had a bogus HEAD in the worktree at one point, but I may be
mis-remembering. I can't seem to trigger it now.

-Peff

[1] https://github.com/mhagger/git-test

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Stefan Beller]

On Thu, Oct 19, 2017 at 8:16 PM, Jeff King  wrote:
> On Thu, Oct 19, 2017 at 06:43:57PM -0700, Stefan Beller wrote:

>
> So that's the right immediate workaround (and hoping there was nothing
> valuable in that worktree!).

How would I know? ;)

Without the commit available I do not even know if the code there
is well reasoned about. (My commit messages usually tell my future
self how much thought I put into the code)

>
>> Any idea which direction we need to heading for a real fix?
>
> If my analysis above is correct, then it's already fixed. You just had
> leftover corruption.

Well fetching yesterday worked and the commit in question is from
8/23, the merge  8a044c7f1d56cef657be342e40de0795d688e882
occurred 9/18, so I suspect there is something else at play.
(I do not remember having a gc between yesterday and today.
Though maybe one in the background?)


I am curious how you can have a worktree owned by multiple
repositories [1] (?).

Thanks,
Stefan

Re: "Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Jeff King]

On Thu, Oct 19, 2017 at 06:43:57PM -0700, Stefan Beller wrote:

> So I ran "git fetch --all" inside git.git that you are all familiar with.
> All fetches failed with a similar error as:
> Fetching kernelorg
> fatal: bad object HEAD
> error: https://kernel.googlesource.com/pub/scm/git/git did not send
> all necessary objects
> 
> error: Could not fetch kernelorg

I ran into this a while back with worktrees, too. In my case I had a
worktree which was "owned" by multiple repositories[1]. One would write
the HEAD, but the other did not necessarily have those objects.

I don't quite recall how I ended up there. Doing "git worktree add"
multiple times does seem to complain about the directory already
existing. My case did involve relative symlinks in both repositories, so
maybe that confused things. Anyway, I can't seem to replicate it now.

> And here we go, I do use worktrees nowadays!
> $ git worktree list
> /usr/local/google/home/sbeller/OSS/git   117ddefdb4
> (detached HEAD)
> /u/git   d0c39a49cc
> (detached HEAD)
> /usr/local/google/home/sbeller/OSS/git_origin_master b14f27f917
> (detached HEAD)
> /usr/local/google/home/sbeller/OSS/submodule_remote_dot  3d9025bd69
> (detached HEAD)
> 
> $ git show 3d9025bd69
> fatal: ambiguous argument '3d9025bd69': unknown revision or path not
> in the working tree.

So maybe you've ended up in the same situation and somehow "reused"
submodule_remote_dot.

But I think until that commit you mentioned (d0c39a49cc), "gc" in the
parent repo could drop objects accessible only from the worktree. So if
that happened _before_ d0c39a49cc, then at that point your worktree was
corrupted. And now of course further operations will complain.

> ok, so I presume I just delete that working tree to "fix my copy of git"
> sbeller@sbeller:/u/git$ rm -rf
> /usr/local/google/home/sbeller/OSS/submodule_remote_dot
> sbeller@sbeller:/u/git$ git worktree prune
> sbeller@sbeller:/u/git$ git worktree list
> /usr/local/google/home/sbeller/OSS/git117ddefdb4 (detached 
> HEAD)
> /u/gitd0c39a49cc (detached 
> HEAD)
> /usr/local/google/home/sbeller/OSS/git_origin_master  b14f27f917 (detached 
> HEAD)

So that's the right immediate workaround (and hoping there was nothing
valuable in that worktree!).

> Any idea which direction we need to heading for a real fix?

If my analysis above is correct, then it's already fixed. You just had
leftover corruption.

-Peff

"Cannot fetch git.git" (worktrees at fault? or origin/HEAD) ?

[Stefan Beller]

So I ran "git fetch --all" inside git.git that you are all familiar with.
All fetches failed with a similar error as:
Fetching kernelorg
fatal: bad object HEAD
error: https://kernel.googlesource.com/pub/scm/git/git did not send
all necessary objects

error: Could not fetch kernelorg

Working with an older version of Git (2.13.0) gives me

$ GIT_TRACE=/u/trace GIT_TRACE_PACKET=/u/out git fetch git://github.com/git/git
>From git://github.com/git/git
 * branch  HEAD   -> FETCH_HEAD

After fiddling around and some printf debugging,
clean fsck(!)
I could not make sense of it, so bisecting

$ git bisect bad
d0c39a49ccb5dfe7feba4325c3374d99ab123c59 is the first bad commit
commit d0c39a49ccb5dfe7feba4325c3374d99ab123c59
Author: Nguyễn Thái Ngọc Duy 
Date:   Wed Aug 23 19:36:59 2017 +0700

revision.c: --all adds HEAD from all worktrees

Unless single_worktree is set, --all now adds HEAD from all worktrees.

Since reachable.c code does not use setup_revisions(), we need to call
other_head_refs_submodule() explicitly there to have the same effect on
"git prune", so that we won't accidentally delete objects needed by some
other HEADs.

A new FIXME is added because we would need something like

int refs_other_head_refs(struct ref_store *, each_ref_fn, cb_data);

in addition to other_head_refs() to handle it, which might require

int get_submodule_worktrees(const char *submodule, int flags);

It could be a separate topic to reduce the scope of this one.

Signed-off-by: Nguyễn Thái Ngọc Duy 
Signed-off-by: Junio C Hamano 

And here we go, I do use worktrees nowadays!
$ git worktree list
/usr/local/google/home/sbeller/OSS/git   117ddefdb4
(detached HEAD)
/u/git   d0c39a49cc
(detached HEAD)
/usr/local/google/home/sbeller/OSS/git_origin_master b14f27f917
(detached HEAD)
/usr/local/google/home/sbeller/OSS/submodule_remote_dot  3d9025bd69
(detached HEAD)

$ git show 3d9025bd69
fatal: ambiguous argument '3d9025bd69': unknown revision or path not
in the working tree.

ok, so I presume I just delete that working tree to "fix my copy of git"
sbeller@sbeller:/u/git$ rm -rf
/usr/local/google/home/sbeller/OSS/submodule_remote_dot
sbeller@sbeller:/u/git$ git worktree prune
sbeller@sbeller:/u/git$ git worktree list
/usr/local/google/home/sbeller/OSS/git117ddefdb4 (detached HEAD)
/u/gitd0c39a49cc (detached HEAD)
/usr/local/google/home/sbeller/OSS/git_origin_master  b14f27f917 (detached HEAD)
$ git bisect reset
Previous HEAD position was d0c39a49cc... revision.c: --all adds HEAD
from all worktrees
HEAD is now at 660fb3dfa8... Sync with maint
$ make install
$ git fetch --all
# works fine!

Any idea which direction we need to heading for a real fix?