Eric Wong <[email protected]> writes: > Do you know which pipes are which? "lsof -p $PID +E" can help > with connectivity checking, as can script/dtas-graph in > https://80x24.org/dtas.git if you have Graph::Easy
Yes. I'm attaching my lsof output and a typescript. The processes of interest here are 4849 public-inbox-convert and 4879 git cat-file. PID 4849's FD 11 is the write end of a pipe, with 4879's stdin as the read end. PID 4849's FD 12 is the read end of a pipe, with 4879's stdout as the write end. At the point of the hang, 4849 is trying to write a SHA1 to FD 11, while 4879 is writing an email message to its stdout. > Some shots in the dark: > > 2. Tweak $PIPE_BUFSIZ and/or MAX_INFLIGHT to smaller values. e.g. > > diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm > index 882a9a4a..ec40edd7 100644 > --- a/lib/PublicInbox/Git.pm > +++ b/lib/PublicInbox/Git.pm > @@ -23,13 +23,12 @@ use Carp qw(croak carp); > use Digest::SHA (); > use PublicInbox::DS qw(dwaitpid); > our @EXPORT_OK = qw(git_unquote git_quote); > -our $PIPE_BUFSIZ = 65536; # Linux default > +our $PIPE_BUFSIZ = 4096; # Linux default > our $in_cleanup; > our $RDTIMEO = 60_000; # milliseconds > our $async_warn; # true in read-only daemons > > -use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) / > - 65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1 > +use constant MAX_INFLIGHT => 4; > > my %GIT_ESC = ( > a => "\a", > This right here seems to have fixed it, when testing locally. PS. Thank you for that lsof command. I've never used lsof in that way; I'll have to add that to my *nix debugging toolbelt. -- Chris
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME screen 4028 chris 11u CHR 5,2 0t0 86 /dev/ptmx ->/dev/pts/7 2214,mksh,0u 2214,mksh,1u 2214,mksh,2u 4849,public-in,0u 4849,public-in,1u 4849,public-in,2u 4874,shard[0],0u 4874,shard[0],1u 4874,shard[0],2u 4879,git,2u public-in 4849 chris cwd DIR 0,42 412 17211 /files/cmb-files/repos public-in 4849 chris rtd DIR 0,26 28 34 / public-in 4849 chris txt REG 0,26 14144 239824 /usr/bin/perl public-in 4849 chris mem REG 0,26 2969912 227085 /usr/lib/libstdc++.so.6.0.30 public-in 4849 chris mem REG 0,26 100328 564900 /usr/lib/libz.so.1.2.13 public-in 4849 chris mem REG 0,26 30760 399826 /usr/lib/libuuid.so.1.3.0 public-in 4849 chris mem REG 0,26 121208 226756 /usr/lib/libgcc_s.so.1 public-in 4849 chris mem REG 0,26 2234600 563318 /usr/lib/libxapian.so.30.12.2 public-in 4849 chris mem REG 0,26 500184 219769 /usr/lib/perl5/vendor_perl/auto/Search/Xapian/Xapian.so public-in 4849 chris mem REG 0,26 132928 239703 /usr/lib/perl5/core_perl/auto/Compress/Raw/Zlib/Zlib.so public-in 4849 chris mem REG 0,26 1351488 347561 /usr/lib/libsqlite3.so.0.8.6 public-in 4849 chris mem REG 0,26 180112 239928 /usr/lib/perl5/vendor_perl/auto/DBD/SQLite/SQLite.so public-in 4849 chris mem REG 0,26 138264 240017 /usr/lib/perl5/vendor_perl/auto/DBI/DBI.so public-in 4849 chris mem REG 0,26 2310 4758 /usr/share/zoneinfo/PST8PDT public-in 4849 chris mem REG 0,26 18168 239686 /usr/lib/perl5/core_perl/auto/MIME/Base64/Base64.so public-in 4849 chris mem REG 0,26 657216 239665 /usr/lib/perl5/core_perl/auto/re/re.so public-in 4849 chris mem REG 0,26 116640 239674 /usr/lib/perl5/core_perl/auto/Storable/Storable.so public-in 4849 chris mem REG 0,26 46920 239702 /usr/lib/perl5/core_perl/auto/Encode/Encode.so public-in 4849 chris mem REG 0,26 39608 218906 /usr/lib/perl5/vendor_perl/auto/Email/Address/XS/XS.so public-in 4849 chris mem REG 0,26 46920 239675 /usr/lib/perl5/core_perl/auto/Socket/Socket.so public-in 4849 chris mem REG 0,26 59128 239687 /usr/lib/perl5/core_perl/auto/List/Util/Util.so public-in 4849 chris mem REG 0,26 46840 238715 /usr/lib/perl5/core_perl/auto/Digest/SHA/SHA.so public-in 4849 chris mem REG 0,26 30456 239671 /usr/lib/perl5/core_perl/auto/Time/HiRes/HiRes.so public-in 4849 chris mem REG 0,26 18168 239744 /usr/lib/perl5/core_perl/auto/Cwd/Cwd.so public-in 4849 chris mem REG 0,26 30536 239695 /usr/lib/perl5/core_perl/auto/File/Glob/Glob.so public-in 4849 chris mem REG 0,26 22264 239689 /usr/lib/perl5/core_perl/auto/IO/IO.so public-in 4849 chris mem REG 0,26 108360 239682 /usr/lib/perl5/core_perl/auto/POSIX/POSIX.so public-in 4849 chris mem REG 0,26 22344 239697 /usr/lib/perl5/core_perl/auto/Fcntl/Fcntl.so public-in 4849 chris mem REG 0,26 3789544 239902 /usr/lib/libperl.so.5.36.0 public-in 4849 chris mem REG 0,26 4251720 228062 /usr/lib/libc.so public-in 4849 chris 0u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u public-in 4849 chris 1u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u public-in 4849 chris 2u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u public-in 4849 chris 3w CHR 1,3 0t0 4 /dev/null public-in 4849 chris 4uW REG 0,42 0 2074958 /files/cmb-files/repos/metanew/inbox.lock public-in 4849 chris 5ur REG 0,42 24576 2074742 /files/cmb-files/repos/metanew/msgmap.sqlite3 public-in 4849 chris 6w FIFO 0,12 0t0 26905684 pipe 4874,shard[0],5r public-in 4849 chris 7r FIFO 0,12 0t0 26905685 pipe 4874,shard[0],8w public-in 4849 chris 8ur REG 0,42 69632 2074960 /files/cmb-files/repos/metanew/xap15/over.sqlite3 public-in 4849 chris 9u REG 0,49 461301 410306 /tmp/PerlIO_LaapMm (deleted) public-in 4849 chris 10u REG 0,42 24576 2074986 /files/cmb-files/repos/metanew/mm_tmp-4849-i60n public-in 4849 chris 11w FIFO 0,12 0t0 26905179 pipe 4879,git,0r public-in 4849 chris 12r FIFO 0,12 0t0 26905180 pipe 4879,git,1w public-in 4849 chris 13u REG 0,42 16928 2074743 /files/cmb-files/repos/metanew/msgmap.sqlite3-journal public-in 4849 chris 14u REG 0,42 66176 2074961 /files/cmb-files/repos/metanew/xap15/over.sqlite3-journal shard[0] 4874 chris 5r FIFO 0,12 0t0 26905684 pipe 4849,public-in,6w shard[0] 4874 chris 8w FIFO 0,12 0t0 26905685 pipe 4849,public-in,7r git 4879 chris 0r FIFO 0,12 0t0 26905179 pipe 4849,public-in,11w git 4879 chris 1w FIFO 0,12 0t0 26905180 pipe 4849,public-in,12r
Script started on 2022-12-21 05:12:15-08:00 [TERM="screen.linux" TTY="/dev/pts/10" COLUMNS="80" LINES="30"] chris@beast:/home/chris $ doas strace -p 4849 doas (chris@beast) password: strace: Process 4849 attached write(11, "4187293ba385ec204df84742b7197f4d"..., 41^Cstrace: Process 4849 detached <detached ...> Interrupt chris@beast:/home/chris $ doas strace -p 4879 strace: Process 4879 attached write(1, "Return-Path: <[email protected]>\nX-Spa"..., 3793^Cstrace: Process 4879 detached <detached ...> Interrupt chris@beast:/home/chris $ exit Script done on 2022-12-21 05:12:43-08:00 [COMMAND_EXIT_CODE="130"]
