Chris Brannon <[email protected]> wrote: > I can reproduce it reliably on both Alpine Linux and the musl variant of > Void Linux. Just clone a mirror of public-inbox.org/meta and try and > convert it: > > $ git clone --mirror https://public-inbox.org/meta > $ public-inbox-convert meta metanew > > If I pass --no-index, the process is successful. > > This is a deadlock of some sort, because during the hang, git cat-file > is blocked on write and other things are blocked on read.
I've requested some packages w/ cfarm admins to test with. Do you know which pipes are which? "lsof -p $PID +E" can help with connectivity checking, as can script/dtas-graph in https://80x24.org/dtas.git if you have Graph::Easy What's curious is all the code paths should be independent of stdio buffering, so I wouldn't think libc differences would matter... I also just did the above meta conversion on FreeBSD w/o problems. > That's as far as I have gotten with debugging it. Some shots in the dark: 1. force read pipe on our end to be non-blocking diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm index 882a9a4a..b239ceb9 100644 --- a/lib/PublicInbox/Git.pm +++ b/lib/PublicInbox/Git.pm @@ -143,6 +143,7 @@ sub _bidi_pipe { fcntl($out_w, 1031, 4096); fcntl($in_r, 1031, 4096) if $batch eq '--batch-check'; } + $in_r->blocking(0); $self->{$out} = $out_w; $self->{$in} = $in_r; } 2. Tweak $PIPE_BUFSIZ and/or MAX_INFLIGHT to smaller values. e.g. diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm index 882a9a4a..ec40edd7 100644 --- a/lib/PublicInbox/Git.pm +++ b/lib/PublicInbox/Git.pm @@ -23,13 +23,12 @@ use Carp qw(croak carp); use Digest::SHA (); use PublicInbox::DS qw(dwaitpid); our @EXPORT_OK = qw(git_unquote git_quote); -our $PIPE_BUFSIZ = 65536; # Linux default +our $PIPE_BUFSIZ = 4096; # Linux default our $in_cleanup; our $RDTIMEO = 60_000; # milliseconds our $async_warn; # true in read-only daemons -use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) / - 65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1 +use constant MAX_INFLIGHT => 4; my %GIT_ESC = ( a => "\a", MAX_INFLIGHT could go down to 1 outside of t/git.t, I think... But my sleep deprived mind isn't finding anything that jumps out...
