[...] > Hi Alexander, > > The good news: I can consistently reproduce the hang problem. But the > bad news is that even with a WITNESS kernel and kern.witness.watch=2 > (or even 3) I don't see any message or kernel panic. > > Any additional suggestion in order to increase debug information or > force the machine to go into ddb?
I tried a MP_LOCKDEBUG kernel and finally got a panic, shown below. Sorry for not including 'show locks' and all the cpus, but my serial console session froze before that. Hope it helps, --Kor pmap_tlb_shootwait: spun out Stopped at db_enter+0x14: popq %rbp ddb{13}> -- ddb{13}> show panic the kernel did not panic -- ddb{13}> ps /o TID PID UID PRFLAGS PFLAGS CPU COMMAND 307801 52176 0 0x2 0x4000000 10 semaphore 410431 54690 0 0x2 0x4000000 15 semaphore 257513 26918 0 0x2 0x4000000 11 semaphore 138054 68532 0 0x2 0x4000000 5 semaphore 203059 2408 0 0x2 0x4000000 9 semaphore 89141 5998 0 0x2 0x4000000 8 semaphore 263439 99498 0 0x2 0x4000000 7 semaphore 477129 15941 0 0x2 0x4000000 12 semaphore 41845 21800 0 0x3 0x4000000 2 nfdump 371022 21800 0 0x3 0x4000000 0 nfdump *391905 66338 0 0x3 0x4000000 13 nfdump 367615 66338 0 0x3 0x4000000 6 nfdump 394180 56727 0 0x3 0x4000000 3 nfdump 46184 33300 0 0x100003 0 1K top -- ddb{13}> trace db_enter() at db_enter+0x14 pmap_page_remove(fffffd8184bdc408) at pmap_page_remove+0x38e uvm_anfree_list(fffffd9c88e2be50,0) at uvm_anfree_list+0x63 amap_wipeout(fffffd9ca390fc88) at amap_wipeout+0x198 uvm_unmap_detach(ffff80005608fcc0,0) at uvm_unmap_detach+0xbe sys_munmap(ffff800055cda540,ffff80005608fdc0,ffff80005608fd40) at sys_munmap+0x185 syscall(ffff80005608fdc0) at syscall+0x5f9 Xsyscall() at Xsyscall+0x128 end of kernel end trace frame: 0x123cf0e2b80, count: -8 -- ddb{13}> machine ddbcpu 1 Stopped at x86_ipi_db+0x16: leave ddb{1}> trace x86_ipi_db(ffff8000552aaff0) at x86_ipi_db+0x16 x86_ipi_handler() at x86_ipi_handler+0x80 Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27 pmap_enter(fffffd9cd8988d00,7103880b6000,1c7767a000,3,33) at pmap_enter+0x703 uvm_fault_upper(ffff800055da4928,ffff800055da4960,ffff800055da4820) at uvm_fault_upper+0x1e0 uvm_fault(fffffd9ca1ab8b58,7103880b6000,2,3) at uvm_fault+0xce uvm_fault_wire(fffffd9ca1ab8b58,7103880b6000,7103880b7000,3) at uvm_fault_wire+0x58 uvm_map_pageable_wire(fffffd9ca1ab8b58,fffffd9c8f522700,fffffd9ca0915ab8,0,3,0) at uvm_map_pageable_wire+0x19d sysctl_vslock(7103880b6d68,18) at sysctl_vslock+0x9b sys_sysctl(ffff800055d61c60,ffff800055da4c80,ffff800055da4c00) at sys_sysctl+0x187 syscall(ffff800055da4c80) at syscall+0x5f9 Xsyscall() at Xsyscall+0x128 end of kernel end trace frame: 0x7103880b6dc0, count: -12 -- [...]