On Thu, Jun 06, 2019 at 02:55:35PM +0100, Stuart Henderson wrote:
> I'm testing the "pump my sched" and read/write unlock diffs and ran into
> the panic below. Seems more likely that it would be connected with the
> sched diff rather than anything else. I'll build a WITNESS kernel and
> see if I can get more details if it hapens again.
>
Funny thing, I ran into the same panic last night after running this diff
without issue for several days. Mine didn't include a stack trace though.
> ddb{1}> show panic
> kernel diagnostic assertion "__mp_lock_held(&sched_lock, curcpu()) == 0"
> failed
> : file "/sys/kern/kern_lock.c", line 63
> ddb{1}> tr
> db_enter() at db_enter+0x10
> panic() at panic+0x128
> __assert(ffffffff81aebffd,ffffffff81ac7a49,3f,ffffffff81b0e315) at
> __assert+0x2
> e
> _kernel_lock() at _kernel_lock+0xf6
> pageflttrap() at pageflttrap+0x75
> kerntrap(ffff800033ee0ca0) at kerntrap+0x91
> alltraps_kern(6,380,fffbffff,2,ffff80002201aff0,e0) at alltraps_kern+0x7b
> setrunqueue(0,ffff8000342b2ce8,e0) at setrunqueue+0xbc
> setrunnable(ffff8000342b2ce8,e0) at setrunnable+0xa5
> ptsignal(ffff8000342b2ce8,13,0) at ptsignal+0x3c4
> sys_kill(ffff800033c2fcc0,ffff800033ee0ec0,ffff800033ee0f20) at sys_kill+0x1c5
> syscall(ffff800033ee0f90) at syscall+0x389
> Xsyscall(6,7a,0,7a,13,7f7ffffe3e18) at Xsyscall+0x128
> end of kernel
> end trace frame: 0x7f7ffffe3cb0, count: -13
> ddb{1}> ps
> PID TID PPID UID S FLAGS WAIT COMMAND
> 89165 411452 1971 667 3 0x1000b2 poll ping
> 1971 11232 86869 667 3 0x82 piperd check_ping
> 66051 380473 92153 667 3 0x1000b2 poll ping
> 92153 480496 86869 667 3 0x82 piperd check_ping
> 7608 110605 8332 667 3 0x1000b2 poll ping
> 8332 284560 86869 667 3 0x82 piperd check_ping
> 40097 336365 35317 667 3 0x1000b2 poll ping
> 35317 385677 86869 667 3 0x82 piperd check_ping
> 33714 102547 21689 667 3 0x1000b2 poll ping
> 21689 210571 86869 667 3 0x82 piperd check_ping
> 72142 347023 62694 667 3 0x1000b2 poll ping
> 62694 439660 86869 667 3 0x82 piperd check_ping
> 87347 447624 76730 1000 4 0x8000430 xscreensaver
> 37569 319522 76730 1000 2 0x800402 piecewise
> 11841 481452 18848 667 3 0x1000b2 poll ping
> 18848 111230 86869 667 3 0x82 piperd check_ping
> 69292 189646 44902 667 3 0x1000b2 poll ping
> 44902 21064 86869 667 3 0x82 piperd check_ping
> 67905 363250 1761 507 3 0x92 kqread pickup
> 86709 146945 91951 1000 3 0x92 kqread imap
> 36596 227858 91951 666 3 0x92 kqread imap-login
> 90723 180281 91951 1000 3 0x92 kqread imap
> 42867 129499 91951 1000 3 0x92 kqread imap
> 52963 203919 91951 1000 3 0x92 kqread imap
> 61599 167915 91951 666 3 0x92 kqread imap-login
> 54450 154426 91951 666 3 0x92 kqread imap-login
> 54061 103553 91951 666 3 0x92 kqread imap-login
> 39422 362611 85872 715 3 0x90 netcon perl
> 49809 260623 85872 715 3 0x90 lockf perl
> 3824 92473 85872 715 3 0x90 lockf perl
> 30990 398152 85872 715 3 0x90 lockf perl
> 3327 264795 85872 715 3 0x90 lockf perl
> 24406 217143 85872 715 3 0x90 lockf perl
> 52871 404585 85872 715 3 0x90 lockf perl
> 58220 118881 85872 715 3 0x90 lockf perl
> 32898 307331 85872 715 3 0x90 lockf perl
> 50203 396987 85872 715 3 0x90 lockf perl
> 93024 24785 1 1000 3 0x100080 select ssh
> 22261 482889 52840 1000 3 0x100083 poll ssh
> 52840 404967 61470 1000 3 0x10008b pause ksh
> 35796 132597 61470 1000 3 0xb0 netio urxvt
> 61470 485028 72107 1000 3 0xb2 select urxvt
> 72107 196192 1 1000 3 0x10008a pause sh
> 70238 396933 1 1000 3 0x100080 select ssh
> 41840 302658 10362 1000 3 0x100083 poll ssh
> 89638 411799 91951 1000 3 0x92 kqread imap
> 49250 507579 81821 1000 3 0x100083 ttyin ksh
> 37608 523403 91951 666 3 0x92 kqread imap-login
> 75552 67606 33096 1000 3 0x100083 poll mutt
> 33096 492099 81821 1000 3 0x10008b pause ksh
> 81821 270355 1 1000 3 0x100080 kqread tmux
> 24522 341281 91861 1000 3 0x100083 kqread tmux
> 23218 441369 78842 1000 3 0x82 poll firefox
> 23218 430421 78842 1000 3 0x4000082 kqread firefox
> 23218 460884 78842 1000 3 0x4000082 fsleep firefox
> 23218 17619 78842 1000 3 0x4000082 fsleep firefox
> 23218 138796 78842 1000 3 0x4000082 fsleep firefox
> 23218 341276 78842 1000 3 0x4000082 fsleep firefox
> 23218 345261 78842 1000 3 0x4000082 fsleep firefox
> 23218 236988 78842 1000 3 0x4000082 poll firefox
> 23218 360519 78842 1000 3 0x4000082 fsleep firefox
> 23218 268973 78842 1000 3 0x4000082 fsleep firefox
> 23218 488245 78842 1000 3 0x4000082 fsleep firefox
> 23218 149318 78842 1000 3 0x4000082 fsleep firefox
> 23218 481219 78842 1000 3 0x4000082 fsleep firefox
> 23218 62901 78842 1000 3 0x4000082 fsleep firefox
> 23218 483497 78842 1000 3 0x4000082 fsleep firefox
> 23218 11651 78842 1000 3 0x4000082 fsleep firefox
> 23218 408040 78842 1000 3 0x4000082 fsleep firefox
> 23218 408833 78842 1000 3 0x4000082 poll firefox
> 23218 172608 78842 1000 3 0x4000082 poll firefox
> 23218 131952 78842 1000 3 0x4000082 fsleep firefox
> 23218 139895 78842 1000 3 0x4000082 fsleep firefox
> 23218 334905 78842 1000 3 0x4000082 fsleep firefox
> 23218 424793 78842 1000 3 0x4000082 fsleep firefox
> 23218 284539 78842 1000 3 0x4000082 fsleep firefox
> 67288 7820 78842 1000 3 0x82 poll firefox
> 67288 492589 78842 1000 3 0x4000082 kqread firefox
> 67288 220358 78842 1000 3 0x4000082 fsleep firefox
> 67288 78134 78842 1000 3 0x4000082 fsleep firefox
> 67288 309349 78842 1000 3 0x4000082 fsleep firefox
> 67288 87299 78842 1000 3 0x4000082 fsleep firefox
> 67288 319957 78842 1000 3 0x4000082 fsleep firefox
> 67288 114126 78842 1000 3 0x4000082 poll firefox
> 67288 522341 78842 1000 3 0x4000082 fsleep firefox
> 67288 125152 78842 1000 3 0x4000082 fsleep firefox
> 67288 473175 78842 1000 3 0x4000082 fsleep firefox
> 67288 320053 78842 1000 3 0x4000082 fsleep firefox
> 67288 443779 78842 1000 3 0x4000082 fsleep firefox
> 67288 513544 78842 1000 3 0x4000082 fsleep firefox
> 67288 275216 78842 1000 3 0x4000082 fsleep firefox
> 67288 145821 78842 1000 3 0x4000082 fsleep firefox
> 67288 478367 78842 1000 3 0x4000082 fsleep firefox
> 20978 507155 78842 1000 3 0x82 poll firefox
> 20978 163725 78842 1000 3 0x4000082 kqread firefox
> 20978 510914 78842 1000 3 0x4000082 fsleep firefox
> 20978 410725 78842 1000 3 0x4000082 fsleep firefox
> 20978 462243 78842 1000 3 0x4000082 fsleep firefox
> 20978 78498 78842 1000 3 0x4000082 fsleep firefox
> 20978 95376 78842 1000 3 0x4000082 fsleep firefox
> 20978 251934 78842 1000 3 0x4000082 poll firefox
> 20978 33558 78842 1000 3 0x4000082 fsleep firefox
> 20978 157336 78842 1000 3 0x4000082 fsleep firefox
> 20978 231901 78842 1000 3 0x4000082 fsleep firefox
> 20978 288905 78842 1000 3 0x4000082 fsleep firefox
> 20978 302941 78842 1000 3 0x4000082 fsleep firefox
> 20978 110857 78842 1000 3 0x4000082 fsleep firefox
> 20978 144389 78842 1000 3 0x4000082 fsleep firefox
> 20978 160511 78842 1000 3 0x4000082 fsleep firefox
> 20978 310439 78842 1000 3 0x4000082 fsleep firefox
> 20978 79290 78842 1000 3 0x4000082 poll firefox
> 20978 296177 78842 1000 3 0x4000082 poll firefox
> 20978 247200 78842 1000 3 0x4000082 fsleep firefox
> 20978 240753 78842 1000 3 0x4000082 fsleep firefox
> 20978 70839 78842 1000 3 0x4000082 fsleep firefox
> 20978 294619 78842 1000 3 0x4000082 fsleep firefox
> 20978 271864 78842 1000 3 0x4000082 fsleep firefox
> 55886 46899 10215 1000 3 0x82 nanosleep xstatbar
> 87554 468139 70489 1000 3 0x82 nanosleep xstatbar
> 40158 206125 1 1000 3 0x82 poll gvfsd
> 40158 514851 1 1000 3 0x4000082 poll gvfsd
> 40158 11571 1 1000 3 0x4000082 poll gvfsd
> 91861 11859 18512 1000 3 0x10008b pause ksh
> 7116 270625 58791 1000 3 0x82 nanosleep i3status
> 10215 10545 1 1000 3 0x100088 pause sh
> 58791 226088 94550 1000 3 0x10008a pause sh
> 12721 166217 18512 1000 3 0xb0 netio urxvt
> 94550 98706 52031 1000 3 0x82 poll i3bar
> 52031 149769 1 1000 3 0x10008a pause sh
> 78842 127569 20559 1000 3 0x82 poll firefox
> 78842 232480 20559 1000 3 0x4000082 kqread firefox
> 78842 83607 20559 1000 3 0x4000082 fsleep firefox
> 78842 187835 20559 1000 3 0x4000082 fsleep firefox
> 78842 29304 20559 1000 3 0x4000082 fsleep firefox
> 78842 460381 20559 1000 3 0x4000082 fsleep firefox
> 78842 348130 20559 1000 3 0x4000082 fsleep firefox
> 78842 330328 20559 1000 3 0x4000082 fsleep firefox
> 78842 33079 20559 1000 3 0x4000082 poll firefox
> 78842 397871 20559 1000 3 0x4000082 poll firefox
> 78842 287565 20559 1000 3 0x4000082 poll firefox
> 78842 232164 20559 1000 3 0x4000082 fsleep firefox
> 78842 359702 20559 1000 3 0x4000082 fsleep firefox
> 78842 117474 20559 1000 3 0x4000082 fsleep firefox
> 78842 35230 20559 1000 3 0x4000082 fsleep firefox
> 78842 435658 20559 1000 3 0x4000082 fsleep firefox
> 78842 404592 20559 1000 3 0x4000082 fsleep firefox
> 78842 221634 20559 1000 3 0x4000082 fsleep firefox
> 78842 444110 20559 1000 3 0x4000082 fsleep firefox
> 78842 170451 20559 1000 3 0x4000082 fsleep firefox
> 78842 48898 20559 1000 3 0x4000082 fsleep firefox
> 78842 283190 20559 1000 3 0x4000082 fsleep firefox
> 78842 2508 20559 1000 3 0x4000082 fsleep firefox
> 78842 159620 20559 1000 3 0x4000082 fsleep firefox
> 78842 203290 20559 1000 3 0x4000082 fsleep firefox
> 78842 176147 20559 1000 3 0x4000082 fsleep firefox
> 78842 310228 20559 1000 3 0x4000082 fsleep firefox
> 78842 188070 20559 1000 3 0x4000082 fsleep firefox
> 78842 114759 20559 1000 3 0x4000082 fsleep firefox
> 78842 425314 20559 1000 3 0x4000082 fsleep firefox
> 78842 219062 20559 1000 3 0x4000082 fsleep firefox
> 78842 330735 20559 1000 3 0x4000082 fsleep firefox
> 78842 420516 20559 1000 3 0x4000082 fsleep firefox
> 78842 470988 20559 1000 3 0x4000082 fsleep firefox
> 78842 85136 20559 1000 3 0x4000082 fsleep firefox
> 78842 65281 20559 1000 3 0x4000082 fsleep firefox
> 78842 503468 20559 1000 3 0x4000082 fsleep firefox
> 78842 448430 20559 1000 3 0x4000082 fsleep firefox
> 78842 126876 20559 1000 3 0x4000082 fsleep firefox
> 78842 75868 20559 1000 3 0x4000082 fsleep firefox
> 78842 115104 20559 1000 3 0x4000082 fsleep firefox
> 78842 56824 20559 1000 3 0x4000082 fsleep firefox
> 78842 39391 20559 1000 3 0x4000082 fsleep firefox
> 78842 81953 20559 1000 3 0x4000082 fsleep firefox
> 78842 285469 20559 1000 3 0x4000082 fsleep firefox
> 78842 237473 20559 1000 3 0x4000082 fsleep firefox
> 78842 481014 20559 1000 3 0x4000082 fsleep firefox
> 78842 457590 20559 1000 3 0x4000082 fsleep firefox
> 78842 79799 20559 1000 3 0x4000082 fsleep firefox
> 78842 237177 20559 1000 3 0x4000082 fsleep firefox
> 78842 374259 20559 1000 3 0x4000082 fsleep firefox
> 78842 137633 20559 1000 3 0x4000082 fsleep firefox
> 78842 363618 20559 1000 3 0x4000082 fsleep firefox
> 78842 111820 20559 1000 3 0x4000082 fsleep firefox
> 20559 387215 1 1000 3 0x10008a pause sh
> 18512 255033 7113 1000 3 0xb2 select urxvt
> 7113 394344 1 1000 3 0x10008a pause sh
> 74270 185769 1 1000 3 0xb0 select urxvtd
> 8407 86815 1 1000 3 0xb0 netio urxvtd
> 93737 172730 80105 1000 3 0x82 nanosleep redshift
> 70489 426133 80105 1000 3 0x100088 pause sh
> 50303 445891 79878 1000 3 0x100083 ttyin vi
> 10362 222596 72814 1000 3 0x10008b pause ksh
> 71700 106064 1 1000 3 0x80 poll dbus-daemon
> 5659 416615 72814 1000 3 0xb0 netio urxvt
> 38684 122378 79878 1000 3 0xb0 netio urxvt
> 48293 385854 1 1000 3 0x80 select dbus-launch
> 95347 269889 9956 1000 3 0x82 select xsettingsd
> 79878 136903 58823 1000 3 0xb2 select urxvt
> 9956 505805 80105 1000 3 0x100088 pause sh
> *76730 438372 80105 1000 7 0x32 xscreensaver
> 58823 315732 80105 1000 3 0x100088 pause sh
> 72814 511431 80105 1000 3 0xb2 select urxvt
> 42101 427536 80105 1000 3 0x82 poll xcompmgr
> 80105 508248 39236 1000 3 0x100082 poll i3
> 63370 85087 1 1000 3 0x1000b0 poll ssh-agent
> 39236 407272 15693 1000 3 0x10008a pause sh
> 96980 7376 1761 507 3 0x92 kqread tlsmgr
> 91049 183316 91951 518 3 0x92 kqread aggregator
> 13998 411982 10593 732 3 0x200082 select mongod
> 13998 250981 10593 732 3 0x420008a sigwait mongod
> 13998 257829 10593 732 3 0x4200082 fsleep mongod
> 13998 326433 10593 732 3 0x4200082 nanosleep mongod
> 13998 135502 10593 732 3 0x4200082 fsleep mongod
> 13998 216141 10593 732 3 0x4200082 fsleep mongod
> 13998 446853 10593 732 3 0x4200082 fsleep mongod
> 13998 305233 10593 732 3 0x4200082 fsleep mongod
> 13998 427505 10593 732 3 0x4200082 nanosleep mongod
> 13998 393262 10593 732 3 0x4200082 nanosleep mongod
> 13998 277598 10593 732 3 0x4200082 fsleep mongod
> 13998 516787 10593 732 3 0x4200082 nanosleep mongod
> 13998 414313 10593 732 3 0x4200082 fsleep mongod
> 13998 97634 10593 732 3 0x4200082 netio mongod
> 13998 414354 10593 732 3 0x4200082 netio mongod
> 13998 300938 10593 732 3 0x4200082 netio mongod
> 13998 94519 10593 732 3 0x4200082 netio mongod
> 13998 497361 10593 732 3 0x4200082 netio mongod
> 13998 160000 10593 732 3 0x4200082 netio mongod
> 15693 335335 46193 0 3 0x100080 wait xenodm
> 6342 30677 49656 0 3 0x100080 netio Xorg
> 29216 31979 1 0 3 0x100083 ttyin getty
> 64696 14586 1 0 3 0x100083 ttyin getty
> 99541 97264 1 0 3 0x100083 ttyin getty
> 87733 276361 1 0 3 0x100083 ttyin getty
> 59091 516839 1 0 3 0x100083 ttyin getty
> 42622 467906 1 0 3 0x100083 ttyin getty
> 49656 142614 46193 35 3 0x92 poll Xorg
> 49656 115571 46193 35 3 0x4000092 fsleep Xorg
> 49656 291708 46193 35 3 0x4000092 poll Xorg
> 46193 31286 1 0 3 0x88 pause xenodm
> 67474 76987 1 0 3 0x100098 poll cron
> 80788 344477 1 0 3 0x100080 nanosleep sensorsd
> 89498 446171 1 0 3 0x80 kqread apmd
> 95018 318399 55227 503 3 0x80 poll postgres
> 80768 392167 55227 503 3 0x80 poll postgres
> 63517 190729 55227 503 3 0x80 poll postgres
> 59278 421061 55227 503 3 0x80 poll postgres
> 51429 480142 55227 503 3 0x80 poll postgres
> 49201 376746 55227 503 3 0x80 poll postgres
> 55227 473429 1 503 3 0x83 select postgres
> 26145 397477 1 801 3 0x200083 fsleep java
> 26145 391752 1 801 3 0x4200083 fsleep java
> 26145 418595 1 801 3 0x4200083 fsleep java
> 26145 21825 1 801 3 0x4200083 fsleep java
> 26145 197907 1 801 3 0x4200083 fsleep java
> 26145 165160 1 801 3 0x4200083 fsleep java
> 26145 131821 1 801 3 0x4200083 fsleep java
> 26145 194603 1 801 3 0x4200083 fsleep java
> 26145 295686 1 801 3 0x4200083 fsleep java
> 26145 497848 1 801 3 0x4200083 fsleep java
> 26145 191181 1 801 3 0x4200083 fsleep java
> 26145 13568 1 801 3 0x4200083 fsleep java
> 26145 515608 1 801 3 0x4200083 fsleep java
> 26145 329792 1 801 3 0x4200083 fsleep java
> 26145 496793 1 801 3 0x4200083 fsleep java
> 26145 260931 1 801 3 0x4200083 fsleep java
> 26145 155225 1 801 3 0x4200083 fsleep java
> 26145 164092 1 801 3 0x4200083 fsleep java
> 26145 200631 1 801 3 0x4200083 fsleep java
> 26145 381635 1 801 3 0x4200083 fsleep java
> 26145 296379 1 801 3 0x4200083 fsleep java
> 26145 113528 1 801 3 0x4200083 fsleep java
> 26145 164240 1 801 3 0x4200083 poll java
> 26145 284516 1 801 3 0x4200083 poll java
> 26145 131036 1 801 3 0x4200083 fsleep java
> 26145 76142 1 801 3 0x4200083 poll java
> 26145 238943 1 801 3 0x4200083 netcon java
> 26145 67697 1 801 3 0x4200083 fsleep java
> 26145 480817 1 801 3 0x4200083 kqread java
> 26145 283946 1 801 3 0x4200083 fsleep java
> 26145 80611 1 801 3 0x4200083 fsleep java
> 26145 204015 1 801 3 0x4200083 kqread java
> 26145 239171 1 801 3 0x4200083 fsleep java
> 26145 24886 1 801 3 0x4200083 fsleep java
> 26145 119873 1 801 3 0x4200083 fsleep java
> 26145 384374 1 801 3 0x4200083 fsleep java
> 26145 139560 1 801 3 0x4200083 fsleep java
> 26145 88147 1 801 3 0x4200083 kqread java
> 26145 88876 1 801 3 0x4200083 fsleep java
> 26145 244985 1 801 3 0x4200083 fsleep java
> 26145 102152 1 801 3 0x4200083 kqread java
> 26145 411418 1 801 3 0x4200083 fsleep java
> 26145 260651 1 801 3 0x4200083 fsleep java
> 26145 27136 1 801 3 0x4200083 fsleep java
> 26145 269310 1 801 3 0x4200083 fsleep java
> 26145 478498 1 801 3 0x4200083 fsleep java
> 26145 381834 1 801 3 0x4200083 fsleep java
> 26145 243015 1 801 3 0x4200083 fsleep java
> 62369 247814 1 667 3 0x80 nanosleep icinga2
> 62369 87387 1 667 3 0x4000080 fsleep icinga2
> 62369 270428 1 667 3 0x4000080 fsleep icinga2
> 62369 448747 1 667 3 0x4000080 fsleep icinga2
> 62369 316599 1 667 3 0x4000080 fsleep icinga2
> 62369 37111 1 667 3 0x4000080 fsleep icinga2
> 62369 178282 1 667 3 0x4000080 fsleep icinga2
> 62369 330209 1 667 3 0x4000080 fsleep icinga2
> 62369 22330 1 667 3 0x4000080 netcon icinga2
> 62369 486871 1 667 3 0x4000080 poll icinga2
> 62369 118644 1 667 3 0x4000080 poll icinga2
> 62369 106981 1 667 3 0x4000080 fsleep icinga2
> 62369 15092 1 667 3 0x4000080 fsleep icinga2
> 62369 440582 1 667 3 0x4000080 poll icinga2
> 62369 86781 1 667 3 0x4000080 poll icinga2
> 62369 375154 1 667 3 0x4000080 poll icinga2
> 62369 447461 1 667 3 0x4000080 poll icinga2
> 86869 316738 1 667 3 0x81 netio icinga2
> 10593 468953 1 732 3 0x200083 fsleep java
> 10593 76173 1 732 3 0x4200083 fsleep java
> 10593 493028 1 732 3 0x4200083 fsleep java
> 10593 246870 1 732 3 0x4200083 fsleep java
> 10593 410664 1 732 3 0x4200083 fsleep java
> 10593 471835 1 732 3 0x4200083 fsleep java
> 10593 114532 1 732 3 0x4200083 fsleep java
> 10593 363324 1 732 3 0x4200083 fsleep java
> 10593 486554 1 732 3 0x4200083 fsleep java
> 10593 412033 1 732 3 0x4200083 fsleep java
> 10593 93357 1 732 3 0x4200083 fsleep java
> 10593 388671 1 732 3 0x4200083 fsleep java
> 10593 129909 1 732 3 0x4200083 fsleep java
> 10593 223467 1 732 3 0x4200083 fsleep java
> 10593 131581 1 732 3 0x4200083 fsleep java
> 10593 34346 1 732 3 0x4200083 fsleep java
> 10593 125228 1 732 3 0x4200083 kqread java
> 10593 114108 1 732 3 0x4200083 kqread java
> 10593 134958 1 732 3 0x4200083 fsleep java
> 10593 345037 1 732 3 0x4200083 piperd java
> 10593 226673 1 732 3 0x4200083 fsleep java
> 10593 38927 1 732 3 0x4200083 fsleep java
> 10593 467658 1 732 3 0x4200083 wait java
> 10593 19301 1 732 3 0x4200083 fsleep java
> 10593 295156 1 732 3 0x4200083 fsleep java
> 10593 336535 1 732 3 0x4200083 fsleep java
> 10593 163200 1 732 3 0x4200083 fsleep java
> 10593 401653 1 732 3 0x4200083 netio java
> 10593 286975 1 732 3 0x4200083 fsleep java
> 10593 299822 1 732 3 0x4200083 fsleep java
> 10593 488174 1 732 3 0x4200083 fsleep java
> 10593 141957 1 732 3 0x4200083 fsleep java
> 10593 182252 1 732 3 0x4200083 fsleep java
> 10593 366248 1 732 3 0x4200083 netio java
> 10593 343161 1 732 3 0x4200083 fsleep java
> 10593 19133 1 732 3 0x4200083 fsleep java
> 10593 412872 1 732 3 0x4200083 fsleep java
> 10593 203958 1 732 3 0x4200083 fsleep java
> 10593 95723 1 732 3 0x4200083 fsleep java
> 10593 407156 1 732 3 0x4200083 fsleep java
> 10593 442545 1 732 3 0x4200083 fsleep java
> 10593 71646 1 732 3 0x4200083 fsleep java
> 10593 453444 1 732 3 0x4200083 fsleep java
> 10593 428375 1 732 3 0x4200083 fsleep java
> 10593 372137 1 732 3 0x4200083 fsleep java
> 10593 67763 1 732 3 0x4200083 fsleep java
> 10593 401536 1 732 3 0x4200083 fsleep java
> 10593 471358 1 732 3 0x4200083 kqread java
> 10593 347079 1 732 3 0x4200083 kqread java
> 10593 68712 1 732 3 0x4200083 netcon java
> 10593 71463 1 732 3 0x4200083 fsleep java
> 10593 379607 1 732 3 0x4200083 kqread java
> 10593 141621 1 732 3 0x4200083 kqread java
> 10593 464692 1 732 3 0x4200083 kqread java
> 10593 301040 1 732 3 0x4200083 netcon java
> 10593 284563 1 732 3 0x4200083 fsleep java
> 10593 427981 1 732 3 0x4200083 kqread java
> 10593 292478 1 732 3 0x4200083 kqread java
> 10593 265448 1 732 3 0x4200083 kqread java
> 10593 205302 1 732 3 0x4200083 netcon java
> 10593 371495 1 732 3 0x4200083 fsleep java
> 10593 39734 1 732 3 0x4200083 kqread java
> 10593 153264 1 732 3 0x4200083 kqread java
> 10593 467935 1 732 3 0x4200083 netcon java
> 10593 354037 1 732 3 0x4200083 fsleep java
> 71610 473496 91951 0 3 0x92 kqread config
> 83275 321028 91951 518 3 0x92 kqread stats
> 84624 269511 91951 0 3 0x92 kqread log
> 86605 380755 91951 518 3 0x92 kqread anvil
> 13455 140701 91951 0 3 0x92 kqread replicator
> 91951 442568 1 0 3 0x80 kqread dovecot
> 26298 340304 1761 507 3 0x92 kqread qmgr
> 1761 129594 1 0 3 0x90 kqread master
> 85872 135783 60848 715 3 0x90 select perl
> 17286 361456 60848 715 3 0x90 select perl
> 60848 139691 1 715 3 0x90 nanosleep perl
> 62860 147979 25582 720 3 0x90 kqread lldpd
> 25582 116170 1 0 3 0x80 netio lldpd
> 3219 10155 1 0 3 0x80 poll nmbd
> 59649 384381 31761 0 3 0x90 poll smbd
> 46087 460988 31761 0 3 0x80 poll smbd
> 65462 81068 31761 0 3 0x80 poll smbd
> 31761 173021 1 0 3 0x90 poll smbd
> 61768 452664 1 694 3 0x82 poll colord
> 61768 429086 1 694 3 0x4000082 poll colord
> 61768 59767 1 694 3 0x4000082 poll colord
> 8617 157998 1 0 3 0x82 kqread cupsd
> 32433 308869 1 559 3 0x90 kqread radiusd
> 32433 501890 1 559 3 0x4000090 fsleep radiusd
> 32433 213186 1 559 3 0x4000090 fsleep radiusd
> 32433 327956 1 559 3 0x4000090 fsleep radiusd
> 32433 67975 1 559 3 0x4000090 fsleep radiusd
> 32433 416063 1 559 3 0x4000090 fsleep radiusd
> 84055 143355 1 8085 3 0x200080 select python2.7
> 91417 307470 26161 756 3 0x90 kqread rspamd
> 55254 171364 26161 756 3 0x90 kqread rspamd
> 53449 23637 26161 756 3 0x90 kqread rspamd
> 56212 110281 26161 756 3 0x90 kqread rspamd
> 26161 281767 1 0 3 0x80 kqread rspamd
> 84684 209441 1 637 3 0x90 kqread memcached
> 84684 123485 1 637 3 0x4000090 nanosleep memcached
> 84684 81476 1 637 3 0x4000090 kqread memcached
> 84684 310562 1 637 3 0x4000090 kqread memcached
> 84684 244749 1 637 3 0x4000090 kqread memcached
> 84684 55408 1 637 3 0x4000090 kqread memcached
> 84684 387841 1 637 3 0x4000090 fsleep memcached
> 84684 274864 1 637 3 0x4000090 fsleep memcached
> 84684 96749 1 637 3 0x4000090 nanosleep memcached
> 84684 515803 1 637 3 0x4000090 fsleep memcached
> 29260 160742 1 67 3 0x100090 kqread slowcgi
> 46073 393784 1 0 3 0x90 select bacula-fd
> 46073 36157 1 0 3 0x4000090 fsleep bacula-fd
> 10416 512221 89760 502 3 0x93 poll mysqld
> 10416 495719 89760 502 3 0x4000093 fsleep mysqld
> 10416 387626 89760 502 3 0x4000093 fsleep mysqld
> 10416 433053 89760 502 3 0x4000093 fsleep mysqld
> 10416 163043 89760 502 3 0x4000093 fsleep mysqld
> 10416 289517 89760 502 3 0x4000093 fsleep mysqld
> 10416 473878 89760 502 3 0x4000093 fsleep mysqld
> 10416 251152 89760 502 3 0x4000093 fsleep mysqld
> 10416 237194 89760 502 3 0x4000093 fsleep mysqld
> 10416 275029 89760 502 3 0x4000093 fsleep mysqld
> 10416 189873 89760 502 3 0x4000093 fsleep mysqld
> 10416 399272 89760 502 3 0x4000093 fsleep mysqld
> 10416 380672 89760 502 3 0x4000093 fsleep mysqld
> 10416 356706 89760 502 3 0x4000093 fsleep mysqld
> 10416 203501 89760 502 3 0x4000093 fsleep mysqld
> 10416 114031 89760 502 3 0x4000093 fsleep mysqld
> 10416 111565 89760 502 3 0x4000093 fsleep mysqld
> 10416 20237 89760 502 3 0x4000093 fsleep mysqld
> 10416 255350 89760 502 3 0x4000093 fsleep mysqld
> 10416 376596 89760 502 3 0x4000093 nanosleep mysqld
> 10416 418516 89760 502 3 0x4000093 fsleep mysqld
> 10416 91608 89760 502 3 0x4000093 fsleep mysqld
> 10416 345855 89760 502 3 0x4000093 fsleep mysqld
> 10416 270097 89760 502 3 0x4000093 fsleep mysqld
> 10416 290188 89760 502 3 0x4000093 fsleep mysqld
> 10416 340204 89760 502 3 0x4000093 nanosleep mysqld
> 10416 360138 89760 502 3 0x4000093 fsleep mysqld
> 10416 70347 89760 502 3 0x4000093 fsleep mysqld
> 10416 394616 89760 502 3 0x4000093 fsleep mysqld
> 10416 417646 89760 502 3 0x400009b sigwait mysqld
> 10416 516307 89760 502 3 0x4000093 fsleep mysqld
> 10416 190662 89760 502 3 0x4000093 poll mysqld
> 10416 405067 89760 502 3 0x4000093 fsleep mysqld
> 10416 475783 89760 502 3 0x4000093 fsleep mysqld
> 10416 413203 89760 502 3 0x4000093 fsleep mysqld
> 10416 345427 89760 502 3 0x4000093 fsleep mysqld
> 10416 111073 89760 502 3 0x4000093 fsleep mysqld
> 10416 345407 89760 502 3 0x4000093 fsleep mysqld
> 10416 492155 89760 502 3 0x4000093 fsleep mysqld
> 10416 302851 89760 502 3 0x4000093 fsleep mysqld
> 89760 234047 1 0 3 0x10008b pause sh
> 77294 49219 1 725 3 0x80 poll rrdcached
> 77294 8961 1 725 3 0x4000088 sigwait rrdcached
> 77294 450370 1 725 3 0x4000080 fsleep rrdcached
> 77294 388444 1 725 3 0x4000080 fsleep rrdcached
> 77294 259332 1 725 3 0x4000080 fsleep rrdcached
> 77294 497398 1 725 3 0x4000080 fsleep rrdcached
> 77294 488436 1 725 3 0x4000080 fsleep rrdcached
> 45662 373820 49468 67 3 0x90 kqread nginx
> 49468 501291 1 0 3 0x88 pause nginx
> 16813 513451 1 0 3 0x80 kqread php-fpm-7.3
> 56098 417190 1 629 3 0x90 poll avahi-daemon
> 31041 464655 1 572 3 0x90 poll dbus-daemon
> 54770 302885 1 686 3 0x80 kqread redis-server
> 54770 71848 1 686 3 0x4000080 fsleep redis-server
> 54770 15649 1 686 3 0x4000080 fsleep redis-server
> 54770 87202 1 686 3 0x4000080 fsleep redis-server
> 94149 140306 1 99 3 0x100090 poll sndiod
> 35615 402588 1 110 3 0x100090 poll sndiod
> 50794 408451 1 0 3 0x100080 poll rpc.bootparamd
> 42199 149940 1 0 3 0x100080 poll rarpd
> 60153 427621 26628 29 3 0x90 kqread identd
> 26628 286461 1 29 3 0x100090 kqread identd
> 15121 90586 1 79 3 0x100090 kqread tftpd
> 73131 380216 1 67 3 0x100090 kqread slowcgi
> 56463 430582 1 0 3 0x100080 kqread snmpd
> 7969 497415 1 91 3 0x100092 kqread snmpd
> 93165 222336 1 91 3 0x92 kqread snmpd
> 16228 193774 1 0 3 0x80 select sshd
> 67476 192677 0 0 3 0x14280 nfsidl nfsio
> 64988 468812 0 0 3 0x14280 nfsidl nfsio
> 58955 232176 0 0 3 0x14280 nfsidl nfsio
> 40398 162426 0 0 3 0x14280 nfsidl nfsio
> 12703 79783 1 0 3 0x80 poll rpc.statd
> 13873 144507 1 0 3 0x80 poll rpc.lockd
> 3383 337343 81635 0 3 0x80 nfsd nfsd
> 56477 379334 81635 0 3 0x80 nfsd nfsd
> 93219 450 81635 0 3 0x80 nfsd nfsd
> 17347 18118 81635 0 3 0x80 nfsd nfsd
> 18396 349556 81635 0 3 0x80 nfsd nfsd
> 50668 462372 81635 0 3 0x80 nfsd nfsd
> 81635 461771 1 0 3 0x80 netcon nfsd
> 81249 25183 22676 0 3 0x80 poll mountd
> 22676 267756 1 0 3 0x100080 poll mountd
> 70953 50826 1 28 3 0x100090 poll portmap
> 10997 430863 22831 101 3 0x100090 kqread iked
> 13710 136885 22831 101 3 0x100090 kqread iked
> 99035 427680 22831 101 3 0x100090 kqread iked
> 22831 66472 1 0 3 0x100080 kqread iked
> 83083 166385 1 0 3 0x100080 poll ntpd
> 81354 240966 92937 83 3 0x100092 poll ntpd
> 92937 235462 1 83 3 0x100092 poll ntpd
> 73393 299754 57934 53 3 0x90 kqread unbound
> 57934 483333 1 53 3 0x90 kqread unbound
> 60470 170610 26365 74 3 0x100092 bpf pflogd
> 26365 233162 1 0 3 0x80 netio pflogd
> 4598 463062 92844 73 3 0x100090 kqread syslogd
> 92844 489409 1 0 3 0x100082 netio syslogd
> 3011 166860 2594 115 3 0x100092 kqread slaacd
> 40726 517214 2594 115 3 0x100092 kqread slaacd
> 2594 64106 1 0 3 0x100080 kqread slaacd
> 9771 449389 0 0 3 0x14280 schto i915/signal:6
> 17492 513686 0 0 3 0x14280 schto i915/signal:2
> 31762 332091 0 0 3 0x14280 schto i915/signal:1
> 60187 209323 0 0 3 0x14280 schto i915/signal:0
> 4679 166134 0 0 3 0x14200 bored i915-userptr-acq
> 10754 365896 0 0 3 0x14200 bored i915_modeset
> 9709 247994 0 0 3 0x14200 bored i915-dp
> 61388 506334 0 0 3 0x14200 bored i915
> 65153 351589 0 0 3 0x14200 pgzero zerothread
> 7079 137514 0 0 3 0x14200 aiodoned aiodoned
> 27306 209501 0 0 3 0x14200 syncer update
> 39935 44263 0 0 3 0x14200 cleaner cleaner
> 92740 117972 0 0 3 0x14200 reaper reaper
> 13724 103724 0 0 3 0x14200 pgdaemon pagedaemon
> 52351 360332 0 0 3 0x14200 bored crynlk
> 87244 441796 0 0 3 0x14200 bored crypto
> 86571 305705 0 0 3 0x14200 usbtsk usbtask
> 43009 394753 0 0 3 0x14200 usbatsk usbatsk
> 50062 127263 0 0 3 0x14200 bored drmtskl
> 67985 461541 0 0 3 0x14200 bored drmlwq
> 4799 5867 0 0 3 0x14200 bored drmubwq
> 57843 196580 0 0 3 0x14200 bored drmwq
> 18392 114573 0 0 3 0x40014200 acpi0 acpi0
> 41341 364778 0 0 7 0x40014200 idle3
> 26870 273096 0 0 7 0x40014200 idle2
> 57532 103683 0 0 3 0x40014200 idle1
> 2971 183150 0 0 3 0x14200 bored sensors
> 75452 389259 0 0 3 0x14200 bored softnet
> 40559 309668 0 0 3 0x14200 bored systqmp
> 43967 200638 0 0 3 0x14200 bored systq
> 87275 343642 0 0 3 0x40014200 bored softclock
> 10865 126292 0 0 7 0x40014200 idle0
> 32207 247204 0 0 3 0x14200 bored smr
> 1 190282 0 0 3 0x82 wait init
> 0 0 -1 0 3 0x10200 scheduler swapper
> ddb{1}> ps /o
> TID PID UID PRFLAGS PFLAGS CPU COMMAND
> *438372 76730 1000 0x32 0 1K xscreensaver
> ddb{1}> show all locks
> No such command
> ddb{1}> sh all pools
> Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
> ipsec policy 352 8 0 6 4 3 1 1 0 8 0
> arp 64 187 0 165 1 0 1 1 0 8 0
> inpcbpl 280 878081 0 877750 118 91 27 34 0 8 0
> drmreq 600 1234913 0 1234908 213 211 2 4 0 8 1
> drmlut 48 149685 0 149530 5 0 5 5 0 8 0
> drmvma 528 148008 0 147771 34 11 23 32 0 8 0
> drmobj 720 148570 0 148320 47 16 31 44 0 8 0
> plcache 128 40 0 0 2 0 2 2 0 8 0
> rtentry 112 680 0 613 7 4 3 3 0 8 0
> syncache 264 20993 0 20907 98 92 6 10 0 8 0
> sackhl 24 326 0 326 12 12 0 1 0 8 0
> tcpqe 32 30623 0 30623 104 103 1 4 0 8 1
> tcpcb 544 73835 0 73710 56 46 10 17 0 8 0
> rttmr 72 572 0 572 142 142 0 1 0 8 0
> nd6 48 6 0 0 1 0 1 1 0 8 0
> cryptop 344 529264 0 529264 4 3 1 1 0 8 1
> ppxss 1128 1 0 0 1 0 1 1 0 8 0
> pffrag 232 653780 0 653780 394 394 0 1 0 482 0
> pffrnode 88 653780 0 653780 394 394 0 1 0 8 0
> pffrent 40 3917926 0 3917926 394 394 0 1 0 8 0
> pfosfp 40 846 0 423 5 0 5 5 0 8 0
> pfosfpen 112 1428 0 714 21 0 21 21 0 8 0
> pfrke_plain 160 12 0 0 1 0 1 1 0 8 0
> pfrktable 1344 7 0 4 1 0 1 1 0 8 0
> pfstitem 24 382236 0 381868 91 86 5 27 0 8 0
> pfstkey 112 382236 0 381868 1537 1523 14 126 0 8 0
> pfstate 328 382236 0 381868 6660 6628 32 366 0 8 0
> pfrule 1360 34 0 16 3 1 2 3 0 8 0
> art_heap8 4096 3 0 0 3 0 3 3 0 8 0
> art_heap4 256 2965 0 2718 48 32 16 17 0 8 0
> art_table 32 2968 0 2718 5 2 3 3 0 8 0
> art_node 16 680 0 614 1 0 1 1 0 8 0
> semapl 112 8 0 0 1 0 1 1 0 8 0
> shmpl 112 4 0 3 1 0 1 1 0 8 0
> nfsnodepl 496 7249 0 22 905 1 904 904 0 8 0
> dirhash 1024 23885 0 19216 724 140 584 611 0 8 0
> dino2pl 256 2055754 0 1774601 17611 33 17578 17578 0 8 0
> dino1pl 128 180236 0 109652 5122 2256 2866 5122 0 8 0
> ffsino 240 2235990 0 1884253 20728 32 20696 20696 0 8 0
> nchpl 144 7012920 0 7006995 252 32 220 220 0 8 0
> rtmask 32 10 0 7 1 0 1 1 0 8 0
> uvmvnodes 72 370332 0 0 6734 0 6734 6734 0 8 0
> vnodes 200 370332 0 0 19492 0 19492 19492 0 8 0
> namei 1024 96628044 0 96628044 1 0 1 1 0 8 1
> nfsreqpl 96 738673 0 738673 915 914 1 1 0 8 1
> percpumem 32 35 0 0 1 0 1 1 0 8 0
> aesni 576 14 0 12 1 0 1 1 1 8 0
> ehcixfer 264 97 0 93 1 0 1 1 0 8 0
> xhcixfer 240 315 0 303 1 0 1 1 0 8 0
> idrpl 32 177012 0 176694 5 0 5 5 0 8 0
> pfiaddrpl 120 2 0 0 1 0 1 1 0 8 0
> scxspl 192 4929384 0 4929384 2284 2282 2 5 0 8 2
> plimitpl 152 159912 0 159819 8 3 5 7 0 8 0
> sigapl 432 517592 0 517396 384 354 30 35 0 8 0
> futexpl 56 14694491 0 14694274 4 0 4 4 0 8 0
> knotepl 112 365922 0 365203 51 27 24 38 0 8 0
> kqueuepl 104 6550 0 6462 7 4 3 5 0 8 0
> pipepl 112 1510952 0 1510502 605 590 15 22 0 8 0
> fdescpl 424 517593 0 517396 707 679 28 36 0 8 0
> filepl 120 79327912 0 79325730 118 31 87 98 0 8 0
> lockfpl 104 857191 0 857049 6 1 5 6 0 8 0
> lockfspl 48 41515 0 41385 2 0 2 2 0 8 0
> sessionpl 112 180495 0 180406 4 0 4 4 0 8 0
> pgrppl 48 180651 0 180554 3 1 2 2 0 8 0
> ucredpl 96 437674 0 437468 8 1 7 7 0 8 0
> zombiepl 144 517410 0 517396 3 2 1 2 0 8 0
> processpl 840 517629 0 517396 417 383 34 40 0 8 0
> procpl 648 530019 0 529457 117 61 56 59 0 8 0
> sockpl 384 979160 0 978327 171 68 103 141 0 8 0
> mcl64k 65536 708 0 0 12 7 5 6 0 8 0
> mcl16k 16384 8576 0 0 49 43 6 7 0 8 0
> mcl12k 12288 601 0 0 7 2 5 5 0 8 0
> mcl9k 9216 175 0 0 3 0 3 3 0 8 0
> mcl8k 8192 6932 0 0 39 33 6 7 0 8 0
> mcl4k 4096 12651 0 0 113 107 6 8 0 8 0
> mcl2k2 2112 2336 0 0 49 43 6 36 0 8 0
> mcl2k 2048 6794 0 0 54 46 8 10 0 8 0
> mtagpl 80 14657 0 0 2 0 2 2 0 8 0
> mbufpl 256 31433 0 0 99 56 43 64 0 8 0
> bufpl 256 1334050 0 664758 46667 3913 42754 45668 0 8 0
> anonpl 16 158046376 0 157380525 5429 2462 2967 3710 0 1014 0
> amapchunkpl 152 14215671 0 14198376 4297 3506 791 826 0 158 0
> amappl16 192 2801293 0 2779850 10147 9044 1103 1876 0 8 0
> amappl15 184 101097 0 101011 9 4 5 6 0 8 0
> amappl14 176 141822 0 141652 27 16 11 12 0 8 0
> amappl13 168 43897 0 43703 28 19 9 10 0 8 0
> amappl12 160 61786 0 61454 28 12 16 16 0 8 0
> amappl11 152 570798 0 570317 54 28 26 27 0 8 0
> amappl10 144 490830 0 490532 53 38 15 15 0 8 0
> amappl9 136 245604 0 244820 71 38 33 45 0 8 0
> amappl8 128 2878845 0 2875330 418 286 132 162 0 8 0
> amappl7 120 223894 0 223555 41 28 13 15 0 8 0
> amappl6 112 527910 0 527341 245 216 29 31 0 8 0
> amappl5 104 934269 0 933451 46 18 28 34 0 8 0
> amappl4 96 1626127 0 1622055 165 60 105 112 0 8 0
> amappl3 88 1309908 0 1307002 151 55 96 104 0 8 0
> amappl2 80 4653704 0 4641735 612 329 283 319 0 8 0
> amappl1 72 43869732 0 43755370 10530 8330 2200 2356 0 8 0
> amappl 80 3596044 0 3590635 193 60 133 133 0 84 0
> dma4096 4096 2 0 2 1 1 0 1 0 8 0
> dma512 512 2 0 1 1 0 1 1 0 8 0
> dma256 256 20 0 20 1 1 0 1 0 8 0
> dma64 64 13 0 13 1 1 0 1 0 8 0
> dma32 32 18 0 18 1 1 0 1 0 8 0
> dma16 16 38 0 38 1 1 0 1 0 8 0
> aobjpl 64 148426 0 148249 7 0 7 7 0 8 0
> uaddrrnd 24 517593 0 517396 2 0 2 2 0 8 0
> uaddrbest 32 2 0 0 1 0 1 1 0 8 0
> uaddr 24 517593 0 517396 2 0 2 2 0 8 0
> vmmpekpl 168 55562360 0 55561824 33 4 29 29 0 8 0
> vmmpepl 168 249479782 0 249245798 62434 51737 10697 14715 0 357 146
> vmsppl 272 517592 0 517396 76 56 20 21 0 8 0
> pdppl 4096 1035193 0 1034792 1911 1849 62 80 0 8 0
> pvpl 32 583545147 0 582711745 20812 13151 7661 9045 0 265 0
> pmappl 200 517592 0 517396 53 37 16 17 0 8 0
> extentpl 40 114 0 61 1 0 1 1 0 8 0
> phpool 112 88198 0 6396 2400 20 2380 2397 0 8 0
> ddb{1}> sh uvmexp
> Current UVM status:
> pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
> 4040120 VM pages: 738551 active, 38654 inactive, 106847 wired, 624715 free
> (8
> 9117 zero)
> min 10% (25) anon, 10% (25) vnode, 5% (12) vtext
> freemin=134670, free-target=179560, inactive-target=0, wired-max=1346706
> faults=329012883, traps=335369071, intrs=9449122, ctxswitch=106382672
> fpuswit
> ch=0
> softint=16700016, syscalls=771043928, kmapent=12
> fault counts:
> noram=0, noanon=0, noamap=0, pgwait=0, pgrele=0
> ok relocks(total)=309276(309994), anget(retries)=131833939(0),
> amapcopy=755
> 90860
> neighbor anon/obj pg=17400618/233487522, gets(lock/unlock)=69653627/309994
> cases: anon=115988479, anoncow=15845460, obj=54964382, prcopy=14688527,
> prz
> ero=127512389
> daemon and swap counts:
> woke=0, revs=0, scans=0, obscans=0, anscans=0
> busy=0, freed=0, reactivate=0, deactivate=0
> pageouts=0, pending=0, nswget=0
> nswapdev=1
> swpages=1572863, swpginuse=0, swpgonly=0 paging=0
> kernel pointers:
> objs(kern)=0xffffffff81d3b858
> ddb{1}> sh bcst
> Current Buffer Cache status:
> numbufs 669292 busymapped 0, delwri 38
> kvaslots 6553 avail kva slots 6553
> bufpages 2323422, dmapages 147424, dirtypages 304
> pendingreads 0, pendingwrites -1101
> highflips 1221969, highflops 0, dmaflips 19241
> ddb{1}> mach ddbcpu 0
> Stopped at x86_ipi_db+0x12: leave
> x86_ipi_db(ffffffff81cf0ff0) at x86_ipi_db+0x12
> x86_ipi_handler() at x86_ipi_handler+0x80
> Xresume_lapic_ipi(4,ffffffff81cf0ff0,0,0,0,0) at Xresume_lapic_ipi+0x23
> _kernel_lock() at _kernel_lock+0xb1
> Xsoftclock(0,33,1388,0,33,ffffffff81cf16f8) at Xsoftclock+0x1f
> acpicpu_idle() at acpicpu_idle+0x1db
> sched_idle(ffffffff81cf0ff0) at sched_idle+0x235
> end trace frame: 0x0, count: 8
> ddb{0}> sh reg
> rdi 0xffffffff81cf0ff0 cpu_info_full_primary+0x1ff0
> rsi 0
> rbp 0xffff80002229f180
> rbx 0xffffffff81d05a48 ipifunc+0x38
> rdx 0
> rcx 0x7
> rax 0xffffff7f
> r8 0
> r9 0
> r10 0
> r11 0x2dbd03bba6d1ee3d
> r12 0x7
> r13 0x4
> r14 0xffffffff81cf0ff0 cpu_info_full_primary+0x1ff0
> r15 0
> rip 0xffffffff817baee2 x86_ipi_db+0x12
> cs 0x8
> rflags 0x202
> rsp 0xffff80002229f170
> ss 0x10
> x86_ipi_db+0x12: leave
> ddb{0}> mach ddbcpu 1
> Stopped at db_enter+0x10: popq %rbp
> db_enter() at db_enter+0x10
> panic() at panic+0x128
> __assert(ffffffff81aebffd,ffffffff81ac7a49,3f,ffffffff81b0e315) at
> __assert+0x2
> e
> _kernel_lock() at _kernel_lock+0xf6
> pageflttrap() at pageflttrap+0x75
> kerntrap(ffff800033ee0ca0) at kerntrap+0x91
> alltraps_kern(6,380,fffbffff,2,ffff80002201aff0,e0) at alltraps_kern+0x7b
> setrunqueue(0,ffff8000342b2ce8,e0) at setrunqueue+0xbc
> setrunnable(ffff8000342b2ce8,e0) at setrunnable+0xa5
> ptsignal(ffff8000342b2ce8,13,0) at ptsignal+0x3c4
> sys_kill(ffff800033c2fcc0,ffff800033ee0ec0,ffff800033ee0f20) at sys_kill+0x1c5
> syscall(ffff800033ee0f90) at syscall+0x389
> Xsyscall(6,7a,0,7a,13,7f7ffffe3e18) at Xsyscall+0x128
> end of kernel
> end trace frame: 0x7f7ffffe3cb0, count: 2
> ddb{1}> sh reg
> rdi 0xffffffff81cf4168 kprintf_mutex
> rsi 0x5
> rbp 0xffff800033ee0ae0
> rbx 0xffff800033ee0b90
> rdx 0x8000000000000000
> rcx 0x206
> rax 0x1
> r8 0xffff800033ee0aa0
> r9 0xcccccccccccccccd
> r10 0x20644a82dc27587
> r11 0x97c086753b464733
> r12 0x3000000008
> r13 0xffff800033ee0af0
> r14 0x100
> r15 0xffffffff81a9fa3b cmd0646_9_tim_udma+0x1cffc
> rip 0xffffffff817baf10 db_enter+0x10
> cs 0x8
> rflags 0x286
> rsp 0xffff800033ee0ae0
> ss 0x10
> db_enter+0x10: popq %rbp
> ddb{1}> mach ddbcpu 2
> Stopped at x86_ipi_db+0x12: leave
> x86_ipi_db(ffff80002201aff0) at x86_ipi_db+0x12
> x86_ipi_handler() at x86_ipi_handler+0x80
> Xresume_lapic_ipi(0,33,1388,0,33,ffff80002201b6f8) at Xresume_lapic_ipi+0x23
> acpicpu_idle() at acpicpu_idle+0x1db
> sched_idle(ffff80002201aff0) at sched_idle+0x235
> end trace frame: 0x0, count: 10
> ddb{2}> sh reg
> rdi 0xffff80002201aff0
> rsi 0
> rbp 0xffff8000222c95a0
> rbx 0xffffffff81d05a48 ipifunc+0x38
> rdx 0
> rcx 0x7
> rax 0xffffff7f
> r8 0
> r9 0
> r10 0
> r11 0x2dbd03bba6d1ee3d
> r12 0x7
> r13 0
> r14 0xffff80002201aff0
> r15 0
> rip 0xffffffff817baee2 x86_ipi_db+0x12
> cs 0x8
> rflags 0x3202 __ALIGN_SIZE+0x2202
> rsp 0xffff8000222c9590
> ss 0x10
> x86_ipi_db+0x12: leave
> ddb{2}> mach ddbcpu 3
> Stopped at x86_ipi_db+0x12: leave
> x86_ipi_db(ffff800022023ff0) at x86_ipi_db+0x12
> x86_ipi_handler() at x86_ipi_handler+0x80
> Xresume_lapic_ipi(0,33,1388,0,33,ffff8000220246f8) at Xresume_lapic_ipi+0x23
> acpicpu_idle() at acpicpu_idle+0x1db
> sched_idle(ffff800022023ff0) at sched_idle+0x235
> end trace frame: 0x0, count: 10
> ddb{3}> sh reg
> rdi 0xffff800022023ff0
> rsi 0
> rbp 0xffff8000222d14a0
> rbx 0xffffffff81d05a48 ipifunc+0x38
> rdx 0
> rcx 0x7
> rax 0xffffff7f
> r8 0
> r9 0
> r10 0
> r11 0x2dbd03bba6d1ee3d
> r12 0x7
> r13 0
> r14 0xffff800022023ff0
> r15 0
> rip 0xffffffff817baee2 x86_ipi_db+0x12
> cs 0x8
> rflags 0x202
> rsp 0xffff8000222d1490
> ss 0x10
> x86_ipi_db+0x12: leave
> ddb{3}> call cpu_reset
> >> OpenBSD/amd64 BOOT 3.44
> boot>
> booting hd0a:/bsd: 11020056+2515976+331808+0+675840
> [739578+128+884568+623575]=0x10060b0
> entry point at 0xffffffff81001000
> [ using 2248880 bytes of bsd ELF symbol table ]
> Copyright (c) 1982, 1986, 1989, 1991, 1993
> The Regents of the University of California. All rights reserved.
> Copyright (c) 1995-2019 OpenBSD. All rights reserved. https://www.OpenBSD.org
>
> OpenBSD 6.5-current (GENERIC.MP) #2: Wed Jun 5 22:04:40 BST 2019
> [email protected]:/sys/arch/amd64/compile/GENERIC.MP
> real mem = 17067200512 (16276MB)
> avail mem = 16539836416 (15773MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec400 (92 entries)
> bios0: vendor Dell Inc. version "A12" date 05/11/2017
> bios0: Dell Inc. PowerEdge T20
> acpi0 at bios0: rev 2
> acpi0: sleep states S0 S4 S5
> acpi0: tables DSDT FACP APIC FPDT SLIC LPIT SSDT SSDT SSDT HPET SSDT MCFG
> SSDT ASF! DMAR
> acpi0: wakeup devices UAR1(S4) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4)
> PXSX(S4) RP05(S4) PXSX(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S3) EHC2(S3)
> XHC_(S4) HDEF(S4) [...]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.58 MHz, 06-3c-03
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 99MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.15 MHz, 06-3c-03
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.15 MHz, 06-3c-03
> cpu2:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu2: 256KB 64b/line 8-way L2 cache
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> [cpu3: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.15 MHz, 06-3c-03
> cpu3:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu3: 256KB 64b/line 8-way L2 cache
> cpu3: smt 0, core 3, package 0
> ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
> acpihpet0 at acpi0: 14318179 Hz
> acpimcfg0 at acpi0
> acpimcfg0: addr 0xf8000000, bus 0-63
> acpiprt0 at acpi0: bus 0 (PCI0)
> acpiprt1 at acpi0: bus 2 (RP01)
> acpiprt2 at acpi0: bus 3 (RP02)
> acpiprt3 at acpi0: bus 5 (RP05)
> acpiprt4 at acpi0: bus 1 (PEG0)
> acpiprt5 at acpi0: bus -1 (PEG1)
> acpiprt6 at acpi0: bus -1 (PEG2)
> acpiec0 at acpi0: not present
> acpicpu0 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu1 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu2 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu3 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpitz0 at acpi0: critical temperature is 105 degC
> acpitz1 at acpi0: critical temperature is 105 degC
> acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
> acpicmos0 at acpi0
> acpibtn0 at acpi0: PWRB
> "PNP0C14" at acpi0 not configured
> acpivideo0 at acpi0: GFX0
> acpivout0 at acpivideo0: DD1F
> cpu0: using VERW MDS workaround (except on vmm entry)
> cpu0: Enhanced SpeedStep 3392 MHz: speeds: 3201, 3200, 3000, 2900, 2700,
> 2500, 2300, 2200, 2000, 1800, 1700, 1500, 1300, 1100, 1000, 800 MHz
> pci0 at mainbus0 bus 0
> pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
> ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
> pci1 at ppb0 bus 1
> em0 at pci1 dev 0 function 0 "Intel 82572EI" rev 0x06: apic 8 int 16unknown
> -- use `?']
> , address 00:15:17:8e:79:85
> inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics P4600" rev 0x06
> drm0 at inteldrm0
> inteldrm0: msi
> xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x04: msi, xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00
> addr 1
> "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
> puc0 at pci0 dev 22 function 3 "Intel 8 Series KT" rev 0x04: ports: 16 com
> com4 at puc0 port 0 apic 8 int 19: ns16550a, 16 byte fifo
> com4: probed fifo depth: 0 bytes
> em1 at pci0 dev 25 function 0 "Intel I217-LM" rev 0x04: msi, address
> f8:b1:56:ac:32:76
> ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 16
> usb1 at ehci0: USB revision 2.0
> uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
> addr 1
> azalia0 at pci0 dev 27 function 0 "Intel 8 Series HD Audio" rev 0x04: msi
> azalia0: codecs: Realtek/0x0280
> audio0 at azalia0
> ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd4
> pci2 at ppb1 bus 2
> ppb2 at pci0 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd4: msi
> pci3 at ppb2 bus 3
> ppb3 at pci3 dev 0 function 0 "TI XIO2001 PCIE-PCI" rev 0x00
> pci4 at ppb3 bus 4
> ppb4 at pci0 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd4: msi
> pci5 at ppb4 bus 5
> nvme0 at pci5 dev 0 function 0 "Samsung SM961/PM961 NVMe" rev 0x00: msi, NVMe
> 1.2
> nvme0: SAMSUNG MZVLW256HEHP-000L7, firmware 4L7QCXB7, serial S35ENX0J765205
> scsibus1 at nvme0: 1 targets
> sd0 at scsibus1 targ 0 lun 0: <NVMe, SAMSUNG MZVLW256, 4L7Q> SCSI4 0/direct
> fixed
> sd0: 244198MB, 512 bytes/sector, 500118192 sectors
> ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 23
> usb2 at ehci1: USB revision 2.0
> uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
> addr 1
> pcib0 at pci0 dev 31 function 0 "Intel C226 LPC" rev 0x04
> ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x04: msi, AHCI 1.3
> ahci0: port 0: 6.0Gb/s
> scsibus2 at ahci0: 32 targets
> sd1 at scsibus2 targ 0 lun 0: <ATA, Samsung SSD 850, EMT0> SCSI3 0/direct
> fixed naa.5002538d4086e2f8
> sd1: 476940MB, 512 bytes/sector, 976773168 sectors, thin
> ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x04: apic 8 int
> 18
> iic0 at ichiic0
> sdtemp0 at iic0 addr 0x18: se97
> sdtemp1 at iic0 addr 0x19: mcp98243
> sdtemp2 at iic0 addr 0x1a: se97
> sdtemp3 at iic0 addr 0x1b: mcp98243
> spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> spdmem1 at iic0 addr 0x51: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> spdmem2 at iic0 addr 0x52: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> spdmem3 at iic0 addr 0x53: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> isa0 at pcib0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> com0: console
> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> pckbd0 at pckbc0 (kbd slot)
> wskbd0 at pckbd0: console keyboard
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> vmm0 at mainbus0: VMX/EPT
>
>
