Deadlock between git-remote-http and git fetch-pack
Hi there, While investigating a hung job in our CI system today, I think I found a deadlock in git-remote-http Git version: 2.9.3 Linux (amd64) kernel 4.9.0 Excerpt from the process list: jenkins 27316 0.0 0.0 18508 6024 ?S19:30 0:00 | \_ git -C ../../../arista fetch --unshallow jenkins 27317 0.0 0.0 169608 10916 ?S19:30 0:00 | \_ git-remote-http origin http://gerrit/arista jenkins 27319 0.0 0.0 24160 8260 ?S19:30 0:00 | \_ git fetch-pack --stateless-rpc --stdin --lock-pack --include-tag --thin --no-progress --depth=2147483647 http://gerrit/arista/ Here PID 27319 (git fetch-pack) is stuck reading on stdin, while its parent, PID 27317 (git-remote-http) is stuck reading on its child’s stdout. Nothing has moved for like 2h, it’s deadlocked. > strace -fp 27319 strace: Process 27319 attached read(0, Here FD 0 is a pipe: ~ @8a33a534e2f7> lsof -np 27319 | grep 0r git 27319 jenkins0r FIFO 0,10 0t0 354519158 pipe The writing end of which is owned by the parent process: ~ @8a33a534e2f7> lsof -n 2>/dev/null | fgrep 354519158 git-remot 27317jenkins4w FIFO 0,10 0t0 354519158 pipe git 27319jenkins0r FIFO 0,10 0t0 354519158 pipe And the parent process (git-remote-http) is stuck reading from another FD: > strace -fp 27317 strace: Process 27317 attached read(5, And here FD 5 is another pipe: ~ @8a33a534e2f7> lsof -np 27317 | grep 5r git-remot 27317 jenkins5r FIFO 0,10 0t0 354519159 pipe Which is the child’s stdout: > lsof -n 2>/dev/null | fgrep 354519159 git-remot 27317jenkins5r FIFO 0,10 0t0 354519159 pipe git 27319jenkins1w FIFO 0,10 0t0 354519159 pipe Hence the deadlock. Stack trace in git-remote-http: (gdb) bt #0 0x7f04f1e1363d in read () from target:/lib64/libpthread.so.0 #1 0x562417472d73 in xread () #2 0x562417472f2b in read_in_full () #3 0x562417438a6e in get_packet_data () #4 0x562417439129 in packet_read () #5 0x5624174245e0 in rpc_service () #6 0x562417424f10 in fetch_git () #7 0x5624174233fd in main () Stack trace in git fetch-pack: (gdb) bt #0 0x7fb3ab478620 in __read_nocancel () from target:/lib64/libpthread.so.0 #1 0x55f688827283 in xread () #2 0x55f68882743b in read_in_full () #3 0x55f6887ce35e in get_packet_data () #4 0x55f6887cea19 in packet_read () #5 0x55f6887ceb90 in packet_read_line () #6 0x55f68879dd05 in get_ack () #7 0x55f68879f6b4 in fetch_pack () #8 0x55f688710619 in cmd_fetch_pack () #9 0x55f6886dff7b in handle_builtin () #10 0x55f6886df026 in main () I looked at the diff between v2.9.3 and HEAD on fetch-pack.c and remote-curl.c and didn’t see anything noteworthy in that area of the code, so I presume the bug is still there in master. -- Benoit "tsuna" Sigoure
Re: [PATCH] Undefine strlcpy if needed.
On Sun, Aug 24, 2014 at 4:10 AM, Ramsay Jones ram...@ramsay1.demon.co.uk wrote: Hmm, which version of OS X are we talking about? OS X 10.9.4: $ uname -a Darwin damogran.local 13.3.0 Darwin Kernel Version 13.3.0: Tue Jun 3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64 config.mak.uname contains this: ifeq ($(shell expr $(uname_R) : '[15]\.'),2) NO_STRLCPY = YesPlease What does ./configure put in config.mak.autogen for NO_STRLCPY? NO_STRLCPY= I guess I saw all the warnings because I did just a “git pull —rebase make -j8” without running “make configure ./configure”. -- Benoit tsuna Sigoure -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Undefine strlcpy if needed.
On Sun, Aug 24, 2014 at 12:49 PM, Torsten Bögershausen tbo...@web.de wrote: On 2014-08-24 18.18, Ramsay Jones wrote: On 24/08/14 12:13, tsuna wrote: On Sun, Aug 24, 2014 at 4:10 AM, Ramsay Jones ram...@ramsay1.demon.co.uk wrote: Hmm, which version of OS X are we talking about? OS X 10.9.4: $ uname -a Darwin damogran.local 13.3.0 Darwin Kernel Version 13.3.0: Tue Jun 3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64 Hmm, does 'uname -r' return 13.3.0 or 13.4.0? (or something else!) $ uname -r 13.3.0 config.mak.uname contains this: ifeq ($(shell expr $(uname_R) : '[15]\.'),2) NO_STRLCPY = YesPlease What does ./configure put in config.mak.autogen for NO_STRLCPY? NO_STRLCPY= OK, so I've got to my limit here! ;-) The conditional shown above (from config.mak.uname) should not have set NO_STRLCPY (assuming that 'uname -r' is returning 13.3.0 or 13.4.0). So, unless NO_STRLCPY is being set somewhere else (command-line, environment), this should just work. puzzled. :( I guess I saw all the warnings because I did just a “git pull —rebase make -j8” without running “make configure ./configure”. Yes, but use of configure is supposed to be optional ... Hopefully, someone who actually knows OS X can solve the mystery. ATB, Ramsay Jones I need to admit that I can not reproduce the warning here, uname -r gives 13.3.0 Could it be that something is special on your machine ? Something in the environment ? Not that I can think of, the only non-standard” thing I have installed is Homebrew (http://brew.sh/), but otherwise it’s all the standard OS X stuff and Developer tools. I write code on this machine on a daily basis. Does a fresh clone help ? A fresh clone doesn’t even build :-/ $ git clone git://github.com/git/git.git Cloning into 'git'... remote: Counting objects: 176423, done. remote: Compressing objects: 100% (47201/47201), done. remote: Total 176423 (delta 127349), reused 176233 (delta 127209) Receiving objects: 100% (176423/176423), 64.05 MiB | 6.13 MiB/s, done. Resolving deltas: 100% (127349/127349), done. Checking connectivity... done. $ cd git $ make GIT_VERSION = 2.1.0 * new build flags CC credential-store.o In file included from credential-store.c:1: In file included from ./cache.h:8: ./gettext.h:17:11: fatal error: 'libintl.h' file not found # include libintl.h ^ 1 error generated. make: *** [credential-store.o] Error 1 I need to run configure first: $ make configure GEN configure $ ./configure configure: Setting lib to 'lib' (the default) […] $ make tsuna@damogran /tmp/git $ make * new build flags CC credential-store.o * new link flags CC abspath.o […] Then the build succeeds. $ grep NO_STRLCPY config.mak.autogen NO_STRLCPY= $ which cc /usr/bin/cc $ cc --version Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn) Target: x86_64-apple-darwin13.3.0 Thread model: posix -- Benoit tsuna” Sigoure -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Undefine strlcpy if needed.
On Sun, Aug 24, 2014 at 5:32 PM, Ramsay Jones ram...@ramsay1.demon.co.uk wrote: Again, I don't have access to an OS X system, so I don't know which package provides libintl/gettext, but it seems to be missing on your system. Probably yeah, those libraries don’t seem to be provided in standard with OS X or OS X’s development tools, so maybe the Makefile should also default to having NO_GETTEXT=YesPlease when on OS X. You can avoid the build failure, without running configure, by setting NO_GETTEXT=YesPlease in your config.mak file. I need to run configure first: $ make configure GEN configure $ ./configure configure: Setting lib to 'lib' (the default) […] So, presumably, configure has set NO_GETEXT=YesPlease in your config.mak.autogen file. Yes it did. I don’t mind running configure, but so far Git has compiled fine without doing it. Should we fix the default values of NO_STRLCPY / NO_GETEXT on OS X? -- Benoit tsuna Sigoure -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Fix compilation on OS X.
On Sat, Jul 20, 2013 at 10:53 PM, Junio C Hamano gits...@pobox.com wrote: Actually, it is _wrong_ for us to rely on system header files to define this symbol for us. Declaring extern char **environ is responsibility of the user programs (like us). Actually, that's right. The C99 standard doesn't mention anything about `environ' (only 7.20.4.5 defines `getenv') and POSIX explicitly states the [environ] variable, which must be declared by the user if it is to be used directly (http://pubs.opengroup.org/onlinepubs/9699919799/functions/environ.html) When _GNU_SOURCE is defined glibc header (I think it is unistd.h) seem to define it for us. Perhaps the correct fix is to revert ec535cc2 for everybody, and if MinGW needs such a workaround, do it inside #ifndef MINGW? That sounds right. -- Benoit tsuna Sigoure -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Fix compilation on OS X.
On Sat, Jul 20, 2013 at 12:55 AM, Ramkumar Ramachandra artag...@gmail.com wrote: Benoit Sigoure wrote: diff --git a/compat/unsetenv.c b/compat/unsetenv.c index 4ea1856..addf3dc 100644 --- a/compat/unsetenv.c +++ b/compat/unsetenv.c @@ -1,5 +1,10 @@ #include ../git-compat-util.h +#ifdef __APPLE__ +// On OS X libc headers don't define this symbol. +extern char **environ; +#endif + Shouldn't this go into git-compat-util.h, since there may be other files depending on this variable? I thought about that but there are no other files that use `environ' so I opted for putting it here instead. -- Benoit tsuna Sigoure -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
