Re: [BUG] git submodule update is not fail safe

2013-01-05 Thread Manlio Perillo 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Il 04/01/2013 22:51, Junio C Hamano ha scritto:
 Manlio Perillo manlio.peri...@gmail.com writes:
 
 $ git submodule update --init
 ...
 Submodule 'roms/vgabios' (git://git.qemu.org/vgabios.git/) registered
 for path 'roms/vgabios'
 fatal: unable to connect to anongit.freedesktop.org:
 anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out

 Unable to fetch in submodule path 'pixman'

 $ git submodule update --init
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'

 The problem is easy to solve: manually remove the pixman directory;
 however IMHO git submodule update should not fail this way since it may
 confuse the user.
 
 Sounds like a reasonable observation.  Jens, Heiko, comments?

I have found another, related problem.

Today I tried to update qemu submodules again, however the command
failed with an obscure error message:

$ git submodule update pixman
fatal: Needed a single revision
Unable to find current revision in submodule path 'pixman'


The pixman submodule is the one that I failed to update in the very begin.
The problem is not with the pixman or qemu repository: if I clone again
qemu (with the --recursive option), all is ok.

The problem is with the private working copy (in .git/modules/pixman)
being corrupted:

$git log
fatal: bad default revision 'HEAD'.

The HEAD file contains ref: refs/heads/master, but the refs/heads
directory is empty.


By the way: since git submodule is a porcelain command, IMHO it should
not show to the user these low level error message; at least it should
give more details.
As an example, in this case it could say something like:

  the local module pixmap seems to be corrupted.
  Run xxx to remove the module and yyy to create it again.

The ideal solution is, for submodule update, to never leave an
incomplete directory; that is: the update command should be atomic.


Regards  Manlio
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlDoMAEACgkQscQJ24LbaUQVugCggdl36Hx5JIW/hd1SVXWv+ths
zpYAnR+93BfDLaFhXEiaQvu/TickmDA0
=2Mnw
-END PGP SIGNATURE-
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] git submodule update is not fail safe

2013-01-05 Thread Jens Lehmann 
Am 04.01.2013 22:51, schrieb Junio C Hamano:
 Manlio Perillo manlio.peri...@gmail.com writes:
 
 $ git submodule update --init
 ...
 Submodule 'roms/vgabios' (git://git.qemu.org/vgabios.git/) registered
 for path 'roms/vgabios'
 fatal: unable to connect to anongit.freedesktop.org:
 anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out

 Unable to fetch in submodule path 'pixman'

 $ git submodule update --init
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'

 The problem is easy to solve: manually remove the pixman directory;
 however IMHO git submodule update should not fail this way since it may
 confuse the user.
 
 Sounds like a reasonable observation.  Jens, Heiko, comments?

The reason seems to be that clone leaves a partial initialized .git
directory in case of connection problems. The next time submodule
update runs it tries to revive the submodule from .git/modules/name
but fails as there are no objects in it.

This looks like a bug in clone to me, as it takes precautions to clean
up if something goes wrong but doesn't do that in this case. But while
glancing over the code I didn't find out what goes wrong here.

If this isn't seen as a bug in clone, we could also remove the
.git/modules/name directory in module_clone() of git-submodule.s
h when the clone fails. Manilo, does the following patch remove the
problems you are seeing (after removing .git/modules/pixman manually)?

diff --git a/git-submodule.sh b/git-submodule.sh
index 2365149..4098702 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -208,7 +208,10 @@ module_clone()
git clone $quiet -n ${reference:+$reference} \
--separate-git-dir $gitdir $url $sm_path
) ||
-   die $(eval_gettext Clone of '\$url' into submodule path 
'\$sm_path' failed)
+   (
+   rm -rf $gitdir 
+   die $(eval_gettext Clone of '\$url' into submodule 
path '\$sm_path' failed)
+   )
fi

# We already are at the root of the work tree but cd_to_toplevel will

--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] git submodule update is not fail safe

2013-01-05 Thread Jens Lehmann 
Am 05.01.2013 14:52, schrieb Manlio Perillo:
 Il 04/01/2013 22:51, Junio C Hamano ha scritto:
 Manlio Perillo manlio.peri...@gmail.com writes:
 
 $ git submodule update --init
 ...
 Submodule 'roms/vgabios' (git://git.qemu.org/vgabios.git/) registered
 for path 'roms/vgabios'
 fatal: unable to connect to anongit.freedesktop.org:
 anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out

 Unable to fetch in submodule path 'pixman'

 $ git submodule update --init
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'

 The problem is easy to solve: manually remove the pixman directory;
 however IMHO git submodule update should not fail this way since it may
 confuse the user.
 
 Sounds like a reasonable observation.  Jens, Heiko, comments?
 
 I have found another, related problem.
 
 Today I tried to update qemu submodules again, however the command
 failed with an obscure error message:
 
 $ git submodule update pixman
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'
 
 
 The pixman submodule is the one that I failed to update in the very begin.
 The problem is not with the pixman or qemu repository: if I clone again
 qemu (with the --recursive option), all is ok.
 
 The problem is with the private working copy (in .git/modules/pixman)
 being corrupted:
 
 $git log
 fatal: bad default revision 'HEAD'.
 
 The HEAD file contains ref: refs/heads/master, but the refs/heads
 directory is empty.

Yep, as I explained in my other email the partially set up
.git/modules/pixman is the reason for the trouble you have.

 By the way: since git submodule is a porcelain command, IMHO it should
 not show to the user these low level error message; at least it should
 give more details.
 As an example, in this case it could say something like:
 
   the local module pixmap seems to be corrupted.
   Run xxx to remove the module and yyy to create it again.
 
 The ideal solution is, for submodule update, to never leave an
 incomplete directory; that is: the update command should be atomic.

I agree that submodule update should not leave an inconsistent state.
In that case you wouldn't see any low level error messages (which I
think is ok if something the porcelain didn't expect to happen occurs,
like it did here).
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] git submodule update is not fail safe

2013-01-05 Thread Manlio Perillo 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Il 05/01/2013 15:01, Jens Lehmann ha scritto:
 [...]
 $ git submodule update --init
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'

 The problem is easy to solve: manually remove the pixman directory;
 however IMHO git submodule update should not fail this way since it may
 confuse the user.

 Sounds like a reasonable observation.  Jens, Heiko, comments?
 
 The reason seems to be that clone leaves a partial initialized .git
 directory in case of connection problems. The next time submodule
 update runs it tries to revive the submodule from .git/modules/name
 but fails as there are no objects in it.
 
 [...]

 If this isn't seen as a bug in clone, we could also remove the
 .git/modules/name directory in module_clone() of git-submodule.s
 h when the clone fails. Manilo,

Its Manlio, not Manilo ;-).

 does the following patch remove the
 problems you are seeing (after removing .git/modules/pixman manually)?
 

I don't think I can test it right now, since the problem can only be
reproduced when git clone fails due to network problems.

Without the patch, if I remove the .git/modules/pixman directory,
`git submodule update --init pixamp` fails:

  Unable to find current revision in submodule path 'pixman'
  fatal: Not a git repository: pixman/../.git/modules/pixman


To reproduce the problem, however, it seems all you need to do is to
send SIGINT signal during `git submodule update` :

  $ git submodule update --init pixman
  Cloning into 'pixman'...
  remote: Counting objects: 10137, done.
  ^C

  $ git submodule update pixman
  remote: Counting objects: 10137, done.
  ^C

  $ git submodule update pixman
  fatal: Needed a single revision
  Unable to find current revision in submodule path 'pixman'


Note that I had to send SIGINT two times, in order to corrupt the module.

I suspect your patch does not fix this (since I don't get the Clone
failed error message).


I also noted that If I send SIGINT before git starts counting remote
objects, I get a different count number:


  $ git submodule update pixman
  Cloning into 'pixman'...
  ^C

  $ git submodule update pixman
  remote: Counting objects: 9757, done.
  ^C

  $ git submodule update pixman
  fatal: Needed a single revision
  Unable to find current revision in submodule path 'pixman'


Note that git is reporting 9757 remote objects, instead of 10137.


P.S.:
sorry for the mail I sent today.
It reported the exact same problem I reported yesterday: this morning I
was rather sure that I got a different error message from submodule
update...



Regards   Manlio
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlDoPZMACgkQscQJ24LbaUTfNQCdFvhSQwGlJZlvOr+TIHHyDFJY
d8AAn0zuHKjBGIcqr8RH/rftHjomvPtM
=48RN
-END PGP SIGNATURE-
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] git submodule update is not fail safe

2013-01-05 Thread Jens Lehmann 
Am 05.01.2013 15:01, schrieb Jens Lehmann:
 Am 04.01.2013 22:51, schrieb Junio C Hamano:
 Manlio Perillo manlio.peri...@gmail.com writes:

 $ git submodule update --init
 ...
 Submodule 'roms/vgabios' (git://git.qemu.org/vgabios.git/) registered
 for path 'roms/vgabios'
 fatal: unable to connect to anongit.freedesktop.org:
 anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out

 Unable to fetch in submodule path 'pixman'

 $ git submodule update --init
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'

 The problem is easy to solve: manually remove the pixman directory;
 however IMHO git submodule update should not fail this way since it may
 confuse the user.

 Sounds like a reasonable observation.  Jens, Heiko, comments?
 
 The reason seems to be that clone leaves a partial initialized .git
 directory in case of connection problems. The next time submodule
 update runs it tries to revive the submodule from .git/modules/name
 but fails as there are no objects in it.
 
 This looks like a bug in clone to me, as it takes precautions to clean
 up if something goes wrong but doesn't do that in this case. But while
 glancing over the code I didn't find out what goes wrong here.

I dug a bit deeper here and found the reason: In remove_junk() of
builtin/clone.c the junk_git_dir points to the gitfile in the
submodules work tree, not the .git/modules/name directory where
clone was told to put the git directory. Will see if I can come up
with a patch soonish ...
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [BUG] git submodule update is not fail safe

2013-01-04 Thread Junio C Hamano 
Manlio Perillo manlio.peri...@gmail.com writes:

 $ git submodule update --init
 ...
 Submodule 'roms/vgabios' (git://git.qemu.org/vgabios.git/) registered
 for path 'roms/vgabios'
 fatal: unable to connect to anongit.freedesktop.org:
 anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out

 Unable to fetch in submodule path 'pixman'

 $ git submodule update --init
 fatal: Needed a single revision
 Unable to find current revision in submodule path 'pixman'

 The problem is easy to solve: manually remove the pixman directory;
 however IMHO git submodule update should not fail this way since it may
 confuse the user.

Sounds like a reasonable observation.  Jens, Heiko, comments?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html