Killing a zombie process?

[Paul Goyette]

I'm not sure how I got to this point (but see high-level steps below). 
I have this zombie process:


root27237  0.0  0.0   0  0 pts/2- Z  - 0:00.00 (sh)

Various web resources say "kill the parent" and the zombie child will 
die, too.  But that's probably not a good idea here, since the parent is 
(or at least, appears to be) init (pid==1).


I checked for other potential parents (ie, any process with pts/2 for 
its TTY), and found two shell processes (one was my "login" shell on 
that terminal, and the other was the result of a "su" command).  I 
logged out of both processes, but the zombie remained.


This is the second time this has happened, and both times were when I 
was using pkgsrc's mksandbox to rebuild something.  The sandbox is 
"almost" standard, created with this command:


# mksandbox --src=/build/netbsd-local/src   \
--xsrc=/build/netbsd-local/xsrc \
--rwdirs=/tmp   \
/sandbox

(I added the rwdirs=/tmp so that /sandbox/tmp would be a memory-based 
tmpfs filesystem.)


I wouldn't usually worry too much about the zombie, but it's running 
/bin/sh _from_within_the_sandbox_ and therefore its image/text file owns 
a reference to /sandbox/bin/sh and this reference prevents me from 
properly unmounting the sandbox.


I suppose I could just manually run "umount -f" but I just hate forcing 
an unmount of an in-use file-system.  :)


Suggestions?


+--+--+-+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:   |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com|
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org  |
+--+--+-+

Re: Killing a zombie process?

[Gary Duzan]

In Message ,
   Paul Goyette wrote:

=>I'm not sure how I got to this point (but see high-level steps below). 
=>I have this zombie process:
=>
=>root27237  0.0  0.0   0  0 pts/2- Z  - 0:00.00 (sh)
=>
=>Various web resources say "kill the parent" and the zombie child will 
=>die, too.  But that's probably not a good idea here, since the parent is 
=>(or at least, appears to be) init (pid==1).

   Can you conform with "ps axl"?

=>I checked for other potential parents (ie, any process with pts/2 for 
=>its TTY), and found two shell processes (one was my "login" shell on 
=>that terminal, and the other was the result of a "su" command).  I 
=>logged out of both processes, but the zombie remained.
=>
=>This is the second time this has happened, and both times were when I 
=>was using pkgsrc's mksandbox to rebuild something.  The sandbox is 
=>"almost" standard, created with this command:
=>
=>  # mksandbox --src=/build/netbsd-local/src   \
=>  --xsrc=/build/netbsd-local/xsrc \
=>  --rwdirs=/tmp   \
=>  /sandbox
=>
=>(I added the rwdirs=/tmp so that /sandbox/tmp would be a memory-based 
=>tmpfs filesystem.)
=>
=>I wouldn't usually worry too much about the zombie, but it's running 
=>/bin/sh _from_within_the_sandbox_ and therefore its image/text file owns 
=>a reference to /sandbox/bin/sh and this reference prevents me from 
=>properly unmounting the sandbox.
=>
=>I suppose I could just manually run "umount -f" but I just hate forcing 
=>an unmount of an in-use file-system.  :)
=>
=>Suggestions?

   If init is really its parent, check its "ps axl" output and
check its WCHAN. If it isn't "wait", maybe run "ktruss -p 1" to
get an idea of what it is doing instead of wait*() calls.

Gary Duzan

Re: Killing a zombie process?

[Paul Goyette]

On Thu, 24 Sep 2015, Gary Duzan wrote:


In Message ,
  Paul Goyette wrote:

=>I'm not sure how I got to this point (but see high-level steps below).
=>I have this zombie process:
=>
=>root27237  0.0  0.0   0  0 pts/2- Z  - 0:00.00 (sh)
=>
=>Various web resources say "kill the parent" and the zombie child will
=>die, too.  But that's probably not a good idea here, since the parent is
=>(or at least, appears to be) init (pid==1).

  Can you conform with "ps axl"?


 UID   PID  PPID   CPU PRI NI VSZRSS WCHAN   STAT TTY   TIME COMMAND
   0 0 0 0   0  0   0  15044 -   OKl  ?  1:49.02 [syste
   0 1 0 0  85  0   13092   1336 waitIs   ?  0:00.59 init
...
   0 27237 1 0   0  0   0  0 -   Zpts/2- 0:00.00 (sh)


Yup, my zombie's parent PPID==1




  If init is really its parent, check its "ps axl" output and
check its WCHAN. If it isn't "wait", maybe run "ktruss -p 1" to
get an idea of what it is doing instead of wait*() calls.


See ps output above;  init's WCHAN==wait

So no clue on why it's not getting around to reaping child 27237.


+--+--+-+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:   |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com|
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org  |
+--+--+-+

Re: Killing a zombie process?

[Paul Goyette]

On Thu, 24 Sep 2015, Greg Troxel wrote:



Paul Goyette  writes:


On Thu, 24 Sep 2015, Gary Duzan wrote:
Yup, my zombie's parent PPID==1


  If init is really its parent, check its "ps axl" output and
check its WCHAN. If it isn't "wait", maybe run "ktruss -p 1" to
get an idea of what it is doing instead of wait*() calls.


See ps output above;  init's WCHAN==wait

So no clue on why it's not getting around to reaping child 27237.


I would try sending init a HUP, which should rescan /etc/ttys and not
really do anything.  But it will then call wait(2) again, and if there
was a glitch where init was already in wait and the offending process's
transition to zombie and ppid==1 didn't cause a wakeup, then it may
resolve.


No luck...  I HUPed init, but the zombie is still there.

# kill -HUP 1
# ps axl | grep ' Z '
   0 27237 1 0   0  0   0  0 -   Zpts/2- 0:00.00 (sh)




+--+--+-+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:   |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com|
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org  |
+--+--+-+

Re: Killing a zombie process?

[Greg Troxel]

Paul Goyette  writes:

> On Thu, 24 Sep 2015, Gary Duzan wrote:
> Yup, my zombie's parent PPID==1
>
>>   If init is really its parent, check its "ps axl" output and
>> check its WCHAN. If it isn't "wait", maybe run "ktruss -p 1" to
>> get an idea of what it is doing instead of wait*() calls.
>
> See ps output above;  init's WCHAN==wait
>
> So no clue on why it's not getting around to reaping child 27237.

I would try sending init a HUP, which should rescan /etc/ttys and not
really do anything.  But it will then call wait(2) again, and if there
was a glitch where init was already in wait and the offending process's
transition to zombie and ppid==1 didn't cause a wakeup, then it may
resolve.


pgpCqfQnHuAzu.pgp
Description: PGP signature

Automated report: NetBSD-current/i386 build failure

[NetBSD Test Fixture]

This is an automatically generated notice of a NetBSD-current/i386
build failure.

The failure occurred on babylon5.NetBSD.org, a NetBSD/amd64 host,
using sources from CVS date 2015.09.24.14.58.19.

An extract from the build.sh output follows:

  remove_bkpt(phdl, r_debug_state_sym.st_value, saved);
 ^

/tmp/bracket/build/2015.09.24.14.58.19-i386/src/external/bsd/libproc/tests/../dist/tests/proc_test.c:317:25:
 error: conversion to '__uintptr_t' from 'Elf64_Addr' may alter its value 
[-Werror=conversion]
  set_bkpt(phdl, main_sym.st_value, );
 ^

/tmp/bracket/build/2015.09.24.14.58.19-i386/src/external/bsd/libproc/tests/../dist/tests/proc_test.c:320:28:
 error: conversion to '__uintptr_t' from 'Elf64_Addr' may alter its value 
[-Werror=conversion]
  remove_bkpt(phdl, main_sym.st_value, saved);
^
--- dependall-sbin ---
--- cgdparse.o ---
#   compile  cgdconfig/cgdparse.o
/tmp/bracket/build/2015.09.24.14.58.19-i386/tools/bin/i486--netbsdelf-gcc 
-O2-std=gnu99-Wall -Wstrict-prototypes -Wmissing-prototypes 
-Wpointer-arith -Wno-sign-compare  -Wno-traditional  -Wa,--fatal-warnings 
-Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra 
-Wno-unused-parameter -Wno-sign-compare -Wold-style-definition -Wsign-compare 
-Wformat=2  -Wno-format-zero-length  -Werror
-I/tmp/bracket/build/2015.09.24.14.58.19-i386/src/sbin/cgdconfig -I. 
-DYY_NO_INPUT --sysroot=/tmp/bracket/build/2015.09.24.14.58.19-i386/destdir  -c 
   cgdparse.c
--- dependall-external ---
cc1: all warnings being treated as errors
*** [proc_test.o] Error code 1
nbmake[9]: stopped in 
/tmp/bracket/build/2015.09.24.14.58.19-i386/src/external/bsd/libproc/tests
1 error

The following commits were made between the last successful build and
the failed build:

2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/Makefile,v 1.1
2015.09.24.14.05.35 christos 
src/external/bsd/libproc/dist/Makefile.depend,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/_libproc.h,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/libproc.h,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/proc_bkpt.c,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/proc_create.c,v 
1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/proc_regs.c,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/proc_rtld.c,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/proc_sym.c,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/proc_util.c,v 1.1
2015.09.24.14.05.35 christos src/external/bsd/libproc/dist/tests/Makefile,v 
1.1
2015.09.24.14.05.35 christos 
src/external/bsd/libproc/dist/tests/proc_test.c,v 1.1
2015.09.24.14.05.35 christos 
src/external/bsd/libproc/dist/tests/target_prog.c,v 1.1
2015.09.24.14.12.47 christos src/external/bsd/libproc/Makefile,v 1.1
2015.09.24.14.12.48 christos src/external/bsd/libproc/Makefile.inc,v 1.1
2015.09.24.14.12.48 christos src/external/bsd/libproc/dist/proc_bkpt.c,v 1.2
2015.09.24.14.12.48 christos src/external/bsd/libproc/dist/proc_create.c,v 
1.2
2015.09.24.14.12.48 christos src/external/bsd/libproc/dist/proc_regs.c,v 1.2
2015.09.24.14.12.48 christos src/external/bsd/libproc/dist/proc_rtld.c,v 1.2
2015.09.24.14.12.48 christos src/external/bsd/libproc/dist/proc_sym.c,v 1.2
2015.09.24.14.12.48 christos src/external/bsd/libproc/dist/proc_util.c,v 1.2
2015.09.24.14.12.48 christos 
src/external/bsd/libproc/dist/tests/proc_test.c,v 1.2
2015.09.24.14.12.48 christos 
src/external/bsd/libproc/dist/tests/target_prog.c,v 1.2
2015.09.24.14.12.48 christos src/external/bsd/libproc/lib/Makefile,v 1.1
2015.09.24.14.12.48 christos src/external/bsd/libproc/tests/Makefile,v 1.1
2015.09.24.14.14.37 christos src/external/bsd/librtld_db/dist/Makefile,v 1.1
2015.09.24.14.14.37 christos 
src/external/bsd/librtld_db/dist/Makefile.depend,v 1.1
2015.09.24.14.14.37 christos 
src/external/bsd/librtld_db/dist/librtld_db.3,v 1.1
2015.09.24.14.14.37 christos src/external/bsd/librtld_db/dist/rtld_db.c,v 
1.1
2015.09.24.14.14.37 christos src/external/bsd/librtld_db/dist/rtld_db.h,v 
1.1
2015.09.24.14.15.58 christos src/external/bsd/librtld_db/Makefile,v 1.1
2015.09.24.14.15.58 christos src/external/bsd/librtld_db/dist/rtld_db.c,v 
1.2
2015.09.24.14.15.59 christos src/external/bsd/librtld_db/lib/Makefile,v 1.1
2015.09.24.14.17.20 christos src/external/cddl/osnet/include/libproc.h,v 1.3
2015.09.24.14.18.24 christos 
src/external/cddl/osnet/dist/common/ctf/ctf_create.c,v 1.4
2015.09.24.14.18.24 christos 
src/external/cddl/osnet/dist/common/ctf/ctf_error.c,v 1.2
2015.09.24.14.18.24 christos 
src/external/cddl/osnet/dist/common/ctf/ctf_impl.h,v 1.2

daily CVS update output

[NetBSD source update]

Updating src tree:
P src/distrib/sets/lists/base/md.amd64
P src/distrib/sets/lists/base/shl.mi
P src/distrib/sets/lists/comp/mi
P src/distrib/sets/lists/comp/shl.mi
P src/distrib/sets/lists/debug/md.amd64
P src/distrib/sets/lists/debug/mi
P src/distrib/sets/lists/debug/shl.mi
P src/distrib/sets/lists/man/mi
P src/distrib/sets/lists/tests/mi
P src/distrib/sets/lists/xcomp/md.i386
P src/distrib/sets/lists/xdebug/md.i386
P src/doc/3RDPARTY
P src/doc/CHANGES
P src/etc/mtree/NetBSD.dist.tests
P src/external/bsd/Makefile
U src/external/bsd/libproc/Makefile
U src/external/bsd/libproc/Makefile.inc
U src/external/bsd/libproc/dist/Makefile
U src/external/bsd/libproc/dist/Makefile.depend
U src/external/bsd/libproc/dist/_libproc.h
U src/external/bsd/libproc/dist/libproc.h
U src/external/bsd/libproc/dist/proc_bkpt.c
U src/external/bsd/libproc/dist/proc_create.c
U src/external/bsd/libproc/dist/proc_regs.c
U src/external/bsd/libproc/dist/proc_rtld.c
U src/external/bsd/libproc/dist/proc_sym.c
U src/external/bsd/libproc/dist/proc_util.c
U src/external/bsd/libproc/dist/tests/Makefile
U src/external/bsd/libproc/dist/tests/proc_test.c
U src/external/bsd/libproc/dist/tests/target_prog.c
U src/external/bsd/libproc/lib/Makefile
U src/external/bsd/libproc/tests/Makefile
U src/external/bsd/librtld_db/Makefile
U src/external/bsd/librtld_db/dist/Makefile
U src/external/bsd/librtld_db/dist/Makefile.depend
U src/external/bsd/librtld_db/dist/librtld_db.3
U src/external/bsd/librtld_db/dist/rtld_db.c
U src/external/bsd/librtld_db/dist/rtld_db.h
U src/external/bsd/librtld_db/lib/Makefile
P src/external/cddl/osnet/dev/systrace/systrace.c
P src/external/cddl/osnet/dist/common/ctf/ctf_create.c
P src/external/cddl/osnet/dist/common/ctf/ctf_error.c
P src/external/cddl/osnet/dist/common/ctf/ctf_impl.h
P src/external/cddl/osnet/dist/common/ctf/ctf_labels.c
P src/external/cddl/osnet/dist/common/ctf/ctf_open.c
P src/external/cddl/osnet/dist/common/ctf/ctf_types.c
U src/external/cddl/osnet/dist/lib/libctf/common/ctf.5
P src/external/cddl/osnet/dist/lib/libctf/common/ctf_lib.c
P src/external/cddl/osnet/dist/lib/libctf/common/ctf_subr.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/drti.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_aggregate.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_as.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_cc.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_cg.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_consume.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_decl.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_decl.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_dis.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_dof.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_error.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_errtags.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_grammar.y
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_handle.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_ident.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_impl.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_lex.l
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_link.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_list.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_map.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_module.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_module.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_open.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_options.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_parser.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_parser.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_pid.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_pid.h
U src/external/cddl/osnet/dist/lib/libdtrace/common/dt_pq.c
U src/external/cddl/osnet/dist/lib/libdtrace/common/dt_pq.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_pragma.c
U src/external/cddl/osnet/dist/lib/libdtrace/common/dt_print.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_printf.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_proc.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_proc.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_program.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_provider.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_provider.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_regset.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_regset.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_string.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_string.h
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_subr.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_work.c
P src/external/cddl/osnet/dist/lib/libdtrace/common/dt_xlator.c
P