On Mon, Jan 29, 2018 at 10:42:20AM +0100, Martin Pieuchot wrote:
> Hello Artturi,
>
> On 28/01/18(Sun) 09:08, Artturi Alm wrote:
> > >Synopsis: stuck in netlock
> > >Category: amd64
> > >Environment:
> > System : OpenBSD 6.2
> > Details : OpenBSD 6.2-current (GENERIC.MP) #333: Sun Jan 7
> > 09:13:00 MST 2018
> >
> > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> >
> > Architecture: OpenBSD.amd64
> > Machine : amd64
> > >Description:
> > processes getting stuck w/STATE=netlock, kill has no effect.
> > >How-To-Repeat:
> > using the desktop normally, until trying to restart chrome ends
> > up failing.
>
> What do you mean with "using the desktop normally"? Which applications
> are you using? Which browser plugins? Can you find out the minimum
> setup to reproduce this deadlock?
>
> > I've had this happen to me atleast twice in the last few of weeks.
>
> Do you know how to reproduce it easily?
>
this time i had less than 10tabs open, so i guess it can be narrowed
down even further.
> > At first time i noticed how trying to launch chrome did lock up
> > all the other processes in netlock, and "pkill chrome" did allow
> > the system to recover, i was unable to figure out what was wrong
> > and rebooting did make everything work again, while ie.
> > removing ~/.cache & ~/.config did not.
>
> So the deadlock is related to your chrome usage?
>
now it does feel like so. i'll upgrade tonight.
> > long before running the "ps cl" below, i had already killed all
> > the xterm-windows those processes were in. cwm(1) was unable to
> > kill some of those, but xkill did not.
>
> Well killing process waiting for the 'netlock' won't help. What has to
> be find is which process is holding it. For that we need the full ps
> output, including kernel and userland threads.
> >
> > after exiting X w/ctrl+alt+backspace(iirc?) i didn't get back to
> > $-prompt, and ^T did show xauth stuck in netlock..
> > i guess it's obvious where it was heading; so i got pics of
> > "# reboot -nq" failing because stuck in the fckng netlock -_-
> >
> > i do have ddb.{panic,console,log}=1, but
> > "# sysctl ddb.trigger=1" ==
> > "sysctl: ddb.trigger: Operation not supported by device"
>
> Not having DDB access will limit the debugging experience. Are you sure
> you tried to enter it on your console?
>
so this requires ttyC0, right?
this time it was ifconfig in [netlock], that prevented using ttyC0.
i got there from X by running "virsh shutdown <domain" from the kvm host,
i guess it emulates what pressing actual power button would(acpi?).
> > ?? so i had no option but "virsh reset <domain>"...
>
> Did you try top(1)? What were the kernel processes doing?
see below, if "top -bCHS -d 1 999" should do.
anything else i could do? anyway, thanks in advance:)
-Artturi
load averages: 0.00, 0.02, 0.06 tfort.my.domain 20:04:13
145 threads: 1 running, 139 idle, 5 on processor up 1 day, 11:38
CPU0 states: 0.2% user, 0.0% nice, 0.4% system, 0.3% interrupt, 99.2% idle
CPU1 states: 1.1% user, 0.1% nice, 2.3% system, 0.0% interrupt, 96.5% idle
CPU2 states: 1.3% user, 0.1% nice, 2.5% system, 0.0% interrupt, 96.1% idle
CPU3 states: 0.9% user, 0.2% nice, 2.9% system, 0.0% interrupt, 96.0% idle
CPU4 states: 0.3% user, 0.1% nice, 0.8% system, 0.0% interrupt, 98.8% idle
CPU5 states: 0.4% user, 0.1% nice, 1.2% system, 0.0% interrupt, 98.3% idle
Memory: Real: 285M/1053M act/tot Free: 6876M Cache: 521M Swap: 0K/4336M
PID TID PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
14495 155467 2 0 35M 40M sleep/1 poll 39:05 1.61%
/usr/X11R6/bin/X :0 -auth /home/aalm/.serverauth.KsYBQlXE5t
70058 507112 2 0 9652K 13M sleep/1 select 0:02 0.05% xterm
13394 440936 -22 0 0K 21M idle - 35.3H 0.00% idle0
6862 125212 -22 0 0K 21M onproc/5 - 35.2H 0.00% idle5
43153 547872 -22 0 0K 21M onproc/4 - 35.0H 0.00% idle4
661 212291 -22 0 0K 21M onproc/3 - 34.7H 0.00% idle3
25137 319342 -22 0 0K 21M onproc/1 - 34.4H 0.00% idle1
65690 467656 -22 0 0K 21M idle - 34.4H 0.00% idle2
3067 485689 10 0 12M 23M idle netlock 3:12 0.00% weechat -r
/connect freenode
87817 410790 68 20 0K 21M run/2 - 2:29 0.00% zerothread
14495 421539 2 0 35M 40M sleep/4 poll 1:51 0.00%
/usr/X11R6/bin/X :0 -auth /home/aalm/.serverauth.KsYBQlXE5t
13992 615559 10 -20 888K 2452K idle netlock 0:47 0.00% ntpd: ntp
engine
30357 245010 10 0 0K 21M idle netlock 0:42 0.00% softclock
61217 230818 10 0 0K 21M idle netlock 0:30 0.00% softnet
51008 255493 18 0 0K 21M sleep/1 syncer 0:30 0.00% update
70625 286762 10 0 0K 21M sleep/1 bored 0:28 0.00% systq
94504 451160 2 0 4124K 13M idle select 0:18 0.00% xfe
33315 484574 34 0 141M 102M idle thrslee 0:17 0.00% chrome:
36172 453673 10 0 0K 21M idle usbtsk 0:16 0.00% usbtask
39893 337592 4 0 724K 544K sleep/5 bpf 0:16 0.00% pflogd:
[running] -s 160 -i pflog0 -f /var/log/pflog
53882 315963 2 0 7680K 8864K idle select 0:15 0.00% xterm
5353 101076 -18 0 0K 21M sleep/5 reaper 0:15 0.00% reaper
35731 216145 2 0 2184K 5984K sleep/1 poll 0:14 0.00%
/usr/local/libexec/at-spi2-registryd
98256 602209 10 0 0K 21M sleep/3 bored 0:11 0.00% systqmp
17398 194216 10 0 1072K 3780K idle netlock 0:09 0.00% ntpd:
constraint from 2a00:1450:400f:806::2004
35819 545415 2 0 784K 2432K idle poll 0:06 0.00%
/usr/local/bin/dbus-daemon
--config-file=/usr/local/share/defaults/at-spi2/accessibility.conf --nofork
--print-address 3
95229 161133 2 0 7708K 8960K idle select 0:06 0.00% xterm
7658 251559 2 0 1488K 6320K sleep/2 poll 0:04 0.00% cwm
4561 518683 2 0 5964K 8920K idle select 0:04 0.00% xterm
54407 393915 2 0 3148K 5676K idle poll 0:04 0.00%
/usr/local/libexec/at-spi-bus-launcher
43330 322248 18 0 676K 824K idle pause 0:03 0.00% -ksh
0 100000 -18 0 0K 21M sleep/2 schedul 0:03 0.00% swapper
30418 202727 2 0 8532K 12M idle select 0:01 0.00% xterm
36972 147487 10 0 0K 21M idle usbatsk 0:01 0.00% usbatsk
25835 316204 2 0 1528K 6832K sleep/5 poll 0:01 0.00% xclock
72329 601323 2 0 5952K 10M idle select 0:01 0.00% xterm
7664 528677 2 0 920K 1692K idle kqread 0:01 0.00%
/usr/sbin/syslogd
83389 257121 2 0 7252K 11M sleep/1 select 0:01 0.00% xterm
26949 457506 2 0 2008K 4592K sleep/1 poll 0:01 0.00% top
49903 184590 2 0 1172K 1328K sleep/1 poll 0:01 0.00%
/usr/sbin/cron
89149 330400 2 0 5924K 9300K idle select 0:01 0.00% xterm -e
ssh 10.0.1.2 doas tail -f /var/log/daemon
8397 307475 2 0 1704K 4624K idle poll 0:00 0.00% xconsole
82280 557981 2 0 5928K 8760K idle select 0:00 0.00% xterm
83314 573665 2 0 5932K 8368K idle select 0:00 0.00% xterm
3183 109413 2 0 5920K 8520K idle select 0:00 0.00% xterm -e
ssh 10.0.1.2 doas tail -f /var/log/messages
4293 225199 2 0 5924K 8356K idle select 0:00 0.00% xterm
4617 408637 2 0 1448K 4088K idle select 0:00 0.00% ssh
10.0.1.2 doas tail -f /var/log/daemon
21989 479865 2 0 5940K 8376K idle select 0:00 0.00% xterm -e
ssh 10.0.1.2 doas tail -f /var/log/authlog
46277 558542 2 0 7296K 11M idle select 0:00 0.00% xterm
31523 438959 2 0 604K 528K idle poll 0:00 0.00% dhclient:
vio0 [priv]
87024 617358 2 0 1452K 4084K idle select 0:00 0.00% ssh
10.0.1.2 doas tail -f /var/log/messages
66268 224449 2 0 1444K 4072K idle select 0:00 0.00% ssh
10.0.1.2 doas tail -f /var/log/authlog
33315 262946 28 0 141M 102M idle fsleep 0:00 0.00% chrome:
85141 228595 2 -20 756K 1748K idle poll 0:00 0.00%
/usr/sbin/ntpd
31097 620520 2 0 588K 2384K idle poll 0:00 0.00% ssh-agent -s
33315 123420 28 0 141M 102M idle fsleep 0:00 0.00% chrome:
88786 574966 2 0 604K 1904K idle poll 0:00 0.00%
/usr/local/bin/dbus-daemon --syslog --fork --print-pid 5 --print-address 7
--session
54407 257551 2 0 3148K 5676K idle poll 0:00 0.00%
/usr/local/libexec/at-spi-bus-launcher
33315 443734 -6 0 141M 102M idle viowait 0:00 0.00% chrome:
79398 434603 2 0 1588K 3860K idle kqread 0:00 0.00% smtpd: queue
77921 214889 3 0 680K 812K idle ttyin 0:00 0.00% -ksh
74345 112202 3 0 808K 856K idle ttyin 0:00 0.00% -ksh
33315 313524 2 0 141M 102M idle poll 0:00 0.00% chrome:
64223 226902 18 0 744K 892K idle pause 0:00 0.00% -ksh
20626 492796 2 0 1540K 3856K idle kqread 0:00 0.00% smtpd:
control
3834 486708 18 0 748K 856K sleep/2 pause 0:00 0.00% -ksh
54407 490935 2 0 3148K 5676K idle poll 0:00 0.00%
/usr/local/libexec/at-spi-bus-launcher
87615 611410 2 0 1540K 3992K idle kqread 0:00 0.00% smtpd: pony
express
27809 212832 2 0 2948K 5772K idle poll 0:00 0.00%
/usr/local/libexec/gvfsd
41871 383430 2 0 920K 1348K idle select 0:00 0.00%
/usr/sbin/sshd
33315 585404 28 0 141M 102M idle fsleep 0:00 0.00% chrome:
14495 122264 31 0 35M 40M idle fsleep 0:00 0.00%
/usr/X11R6/bin/X :0 -auth /home/aalm/.serverauth.KsYBQlXE5t
97852 388194 2 0 1388K 3760K idle kqread 0:00 0.00% smtpd:
lookup
59867 310950 10 0 1460K 4152K idle netlock 0:00 0.00% ssh 10.0.1.4
95935 209526 2 0 1268K 3592K idle kqread 0:00 0.00% smtpd:
scheduler
33315 204638 28 0 141M 102M sleep/2 fsleep 0:00 0.00% chrome:
40985 282468 2 0 1264K 3524K idle kqread 0:00 0.00% smtpd:
klondike
26683 403143 2 0 2224K 1280K idle netio 0:00 0.00% X: [priv]
16901 218184 18 0 680K 800K idle pause 0:00 0.00% -ksh
62588 321190 18 0 684K 820K idle pause 0:00 0.00% -ksh
77631 557114 2 0 220K 336K idle netcon 0:00 0.00% nfsd: master
33315 116563 28 0 141M 102M sleep/4 fsleep 0:00 0.00% chrome:
12097 547114 2 0 668K 2316K idle poll 0:00 0.00% ntpd: dns
engine
45861 500739 2 0 140K 156K idle nfsd 0:00 0.00% nfsd: server
63000 119248 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
1959 474643 3 0 316K 1288K idle ttyin 0:00 0.00%
/usr/libexec/getty std.9600 ttyC3
75796 203066 2 0 672K 752K idle poll 0:00 0.00% mountd:
parent
91228 394672 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
73066 389907 2 0 1540K 2264K idle kqread 0:00 0.00%
/usr/sbin/smtpd
33315 539831 28 0 141M 102M sleep/3 fsleep 0:00 0.00% chrome:
33315 236547 28 0 141M 102M idle fsleep 0:00 0.00% chrome:
61229 136255 3 0 816K 908K idle ttyin 0:00 0.00% -ksh
33315 514421 29 0 141M 102M idle fsleep 0:00 0.00% chrome:
27809 260205 2 0 2948K 5772K idle poll 0:00 0.00%
/usr/local/libexec/gvfsd
2846 113818 2 0 680K 612K idle netio 0:00 0.00% pflogd:
[priv]
31957 551446 2 -20 436K 1048K idle poll 0:00 0.00%
/usr/bin/sndiod
39187 119771 2 0 736K 684K idle poll 0:00 0.00% dhclient:
vio0
94324 372170 10 0 432K 1828K idle wait 0:00 0.00% xinit
/home/aalm/.xinitrc -- /usr/X11R6/bin/X :0 -auth
/home/aalm/.serverauth.KsYBQlXE5t
25954 531521 3 0 312K 1276K idle ttyin 0:00 0.00%
/usr/libexec/getty std.9600 ttyC1
77645 555246 3 0 312K 1272K idle ttyin 0:00 0.00%
/usr/libexec/getty std.9600 ttyC2
24372 144495 3 0 680K 840K idle ttyin 0:00 0.00% -ksh
54451 115117 3 0 312K 1272K idle ttyin 0:00 0.00%
/usr/libexec/getty std.9600 ttyC5
45064 261299 18 0 804K 860K idle pause 0:00 0.00% -ksh
33315 317401 2 0 141M 102M idle kqread 0:00 0.00% chrome:
33315 367737 28 0 141M 102M sleep/4 fsleep 0:00 0.00% chrome:
49805 158039 2 0 508K 468K idle poll 0:00 0.00% mountd:
[priv]
27809 450428 2 0 2948K 5772K idle poll 0:00 0.00%
/usr/local/libexec/gvfsd
6307 102098 3 0 684K 816K idle ttyin 0:00 0.00% -ksh
33315 595662 30 0 141M 102M idle fsleep 0:00 0.00% chrome:
1 398685 10 0 388K 452K idle wait 0:00 0.00% /sbin/init
54083 465642 18 0 676K 792K idle pause 0:00 0.00% /bin/sh
/usr/X11R6/bin/startx
35731 409155 2 0 2184K 5984K idle poll 0:00 0.00%
/usr/local/libexec/at-spi2-registryd
51400 304022 -18 0 0K 21M idle pgdaemo 0:00 0.00% pagedaemon
5798 348610 18 0 664K 768K idle pause 0:00 0.00% sh
/home/aalm/.xinitrc
34823 269871 10 0 0K 21M idle bored 0:00 0.00% ttm_swap
4063 460059 3 0 684K 808K idle ttyin 0:00 0.00% -ksh
60668 341706 -13 0 0K 21M idle cleaner 0:00 0.00% cleaner
27861 371189 -18 0 0K 21M idle aiodone 0:00 0.00% aiodoned
24713 509614 2 0 496K 1960K idle netio 0:00 0.00% syslogd:
[priv]
69707 613393 2 0 1036K 1080K idle netio 0:00 0.00% xconsole
59595 265281 2 0 596K 1844K idle select 0:00 0.00% dbus-launch
--sh-syntax --exit-with-session
53526 603738 10 0 432K 1532K idle netlock 0:00 0.00% systat
35731 431557 2 0 2184K 5984K idle poll 0:00 0.00%
/usr/local/libexec/at-spi2-registryd
45074 233314 2 0 140K 156K idle nfsd 0:00 0.00% nfsd: server
81708 418838 2 0 384K 1072K idle poll 0:00 0.00%
/usr/sbin/portmap
58374 516454 2 0 140K 156K idle nfsd 0:00 0.00% nfsd: server
9217 478612 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
7100 505816 2 0 140K 156K idle nfsd 0:00 0.00% nfsd: server
33315 250331 2 0 141M 102M idle poll 0:00 0.00% chrome:
75588 539022 2 0 416K 900K idle poll 0:00 0.00% sndiod:
helper
24254 306793 2 0 140K 156K idle nfsd 0:00 0.00% nfsd: server
20053 327803 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
83191 436025 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
82243 423406 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
54407 602688 2 0 3148K 5676K idle poll 0:00 0.00%
/usr/local/libexec/at-spi-bus-launcher
33315 403281 30 0 141M 102M idle fsleep 0:00 0.00% chrome:
33315 222186 28 0 141M 102M idle fsleep 0:00 0.00% chrome:
33315 117266 30 0 141M 102M idle fsleep 0:00 0.00% chrome:
33315 505611 2 0 141M 102M idle poll 0:00 0.00% chrome:
33315 439073 30 0 141M 102M idle fsleep 0:00 0.00% chrome:
70689 286455 2 0 140K 152K idle nfsd 0:00 0.00% nfsd: server
42722 600020 10 0 0K 21M idle acpi0 0:00 0.00% acpi0
37946 233490 10 0 0K 21M idle bored 0:00 0.00% crynlk
26234 539619 10 0 0K 21M idle bored 0:00 0.00% crypto
93494 221357 28 0 516K 1504K onproc/0 - 0:00 0.00% top -bCHS
-d 1 999
