The GENERIC bsd kernel is happy under vmm, but booting a GENERIC.MP kernel
hits a witness panic. I suspect some "one CPU only" optimization is
resulting in the witness code being misinformed.
Here's the boot output in the vmm console. (Yes, the userland is out of
date, but that shouldn't lead to a witness panic either.)
(The weird "show witness" output for scsi_base.c mutexes is because
they're on the stack and need to be unlinked from witness before
returning; that *might* be causing the problem here, but I doubt it. I'm
starting on a diff for that part...)
Philip Guenther
-------
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2018 OpenBSD. All rights reserved. https://www.OpenBSD.org
OpenBSD 6.3-current (GENERIC.MP) #25: Thu Jun 7 16:29:55 PDT 2018
[email protected]:/usr/src/sys-realclean/arch/amd64/compile/GENERIC.MP
real mem = 520093696 (496MB)
avail mem = 485457920 (462MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0
acpi at bios0 not configured
cpu0 at mainbus0: (uniprocessor)
cpu0: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz, 2594.54 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,LONG,LAHF,ABM,3DNOWP,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,RDSEED,ADX,SMAP,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
pvbus0 at mainbus0: OpenBSD
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "OpenBSD VMM Host" rev 0x00
virtio0 at pci0 dev 1 function 0 "Qumranet Virtio RNG" rev 0x00
viornd0 at virtio0
virtio0: irq 3
virtio1 at pci0 dev 2 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk0 at virtio1
scsibus1 at vioblk0: 2 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 4096MB, 512 bytes/sector, 8388608 sectors
virtio1: irq 5
virtio2 at pci0 dev 3 function 0 "OpenBSD VMM Control" rev 0x00
vmmci0 at virtio2
virtio2: irq 6
isa0 at mainbus0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16450, no fifo
com0: console
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (0084d990f4e53393.a) swap on sd0b dump on sd0b
Automatic boot in progress: starting file system checks.
/dev/sd0a (0084d990f4e53393.a): file system is clean; not checking
/dev/sd0e (0084d990f4e53393.e): file system is clean; not checking
/dev/sd0d (0084d990f4e53393.d): file system is clean; not checking
setting tty flags
pfctl: pfctl_rules
pfctl: DIOCXROLLBACK: Invalid argument
pf enabled
starting network
pfctl: pfctl_rules
pfctl: DIOCXROLLBACK: Invalid argument
reordering libraries:panic: acquiring blockable sleep lock with spinlock or
critical section held (kernel_lock) &kernel_lock @
/usr/src/sys-realclean/arch/amd64/amd64/intr.c:525
Stopped at db_enter+0x5: popq %rbp
TID PID UID PRFLAGS PFLAGS CPU COMMAND
*522028 67277 0 0x14000 0x200 0 reaper
db_enter() at db_enter+0x5
panic() at panic+0x138
witness_checkorder(ffffffff81b7c59c,20d,0,ffffffff81cf7ca0,ffff80000002af00) at
witness_checkorder+0xd32
___mp_lock(ffff80000002af00,ffff80000e0eaca0,ffffffff81bdaff0) at ___mp_lock+0x
70
intr_handler(1,ffff80000002ae80) at intr_handler+0x40
Xintr_legacy8_untramp(ffff80000e0ead60,ffffffff81d16c60,c,10,ffff80000e0ead30,f
fffffff814562c0) at Xintr_legacy8_untramp+0x155
Xspllower(0,282,ffffffff818c9e53,1ca9c,ffffff0002570000,10) at Xspllower+0xc
uvm_pmr_freepages(1f12000,ffffff001f75e380) at uvm_pmr_freepages+0x204
pmap_do_remove(ffffff001bd30a18,ffffff001f75f5a0,ffff80000e0ab4d0,ffffffff81053
c20) at pmap_do_remove+0x463
uvm_map_teardown(0) at uvm_map_teardown+0x143
uvmspace_free(ffff80000e0f9148) at uvmspace_free+0x36
uvm_exit(ffff80000e0f9148) at uvm_exit+0x16
reaper() at reaper+0x156
end trace frame: 0x0, count: 2
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports. Insufficient info makes it difficult to find and fix bugs.
ddb{0}>
ddb{0}> show locks
exclusive mutex &uvm.fpageqlock r = 0 (0xffffffff81d1bcc0) locked @ /usr/src/sy
s-realclean/uvm/uvm_pmemrange.c:1124
ddb{0}> show witness
Sleep locks:
sysctllk (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclean/ker
n/kern_sysctl.c:233
&map->lock (type: rwlock, depth: 2) -- last acquired @ /usr/src/sys-realclean/
uvm/uvm_map.c:1936
netlock (type: rwlock, depth: 1) -- last acquired @ /usr/src/sys-realclean/net
inet/igmp.c:609
pools (type: rwlock, depth: 2) -- last acquired @ /usr/src/sys-realclean/kern
/subr_pool.c:474
&ar->ar_lock (type: rwlock, depth: 2) -- last acquired @ /usr/src/sys-realcle
an/net/rtable.c:500
swplk (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclean/uvm/uv
m_swap.c:615
&ip->i_lock (type: rrwlock, depth: 1) -- last acquired @ /usr/src/sys-realclea
n/ufs/ufs/ufs_vnops.c:1559
&map->lock (type: rwlock, depth: 2) -- (already displayed)
&ufsdirhash_mtx (type: rwlock, depth: 2) -- last acquired @ /usr/src/sys-real
clean/ufs/ufs/ufs_dirhash.c:238
&dh->dh_mtx (type: rwlock, depth: 3) -- last acquired @ /usr/src/sys-realcle
an/ufs/ufs/ufs_dirhash.c:355
&dh->dh_mtx (type: rwlock, depth: 3) -- (already displayed)
&mp->mnt_lock (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclea
n/kern/vfs_subr.c:196
&map->lock (type: rwlock, depth: 2) -- (already displayed)
&diskp->dk_lock (type: rwlock, depth: 1) -- last acquired @ /usr/src/sys-realc
lean/kern/subr_disk.c:1280
&ip->i_lock (type: rrwlock, depth: 1) -- (already displayed)
&sc->sc_lock (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclean
/dev/softraid.c:1199
&diskp->dk_lock (type: rwlock, depth: 1) -- (already displayed)
&pp->pr_lock (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclean
/kern/subr_pool.c:583
&route_cb.rcb_lk (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realc
lean/net/rtsock.c:267
ifclonerslk (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclean/
net/if.c:1282
&newfdp->fd_fd.fd_lock (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys
-realclean/kern/kern_descrip.c:712
&map->lock (type: rwlock, depth: 2) -- (already displayed)
&diskp->dk_lock (type: rwlock, depth: 1) -- (already displayed)
&ip->i_lock (type: rrwlock, depth: 1) -- (already displayed)
vfs_stall (type: rwlock, depth: 1) -- last acquired @ /usr/src/sys-realclean/k
ern/vfs_subr.c:1642
ptarrlk (type: rwlock, depth: 1) -- last acquired @ /usr/src/sys-realclean/ker
n/tty_pty.c:168
&uidinfolk (type: rwlock, depth: 0) -- last acquired @ /usr/src/sys-realclean/k
ern/kern_proc.c:130
&kernel_lock (type: kernel_lock, depth: 0) -- last acquired @ /usr/src/sys-real
clean/kern/kern_exit.c:397
Spin locks:
&pp->pr_lock (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/
kern/subr_pool.c:583
&pp->pr_lock (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/
kern/subr_pool.c:804
/usr/src/sys-realclean/netinet/ip_input.c:101 (type: mutex, depth: 0) -- last a
cquired @ /usr/src/sys-realclean/netinet/ip_input.c:1013
/usr/src/sys-realclean/netinet6/frag6.c:57 (type: mutex, depth: 0) -- last acqu
ired @ /usr/src/sys-realclean/netinet6/frag6.c:580
&fp->f_mtx (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/ke
rn/sys_generic.c:360
(null) (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/s
csi_base.c:1338
&sched_lock (type: sched_lock, depth: 3) -- last acquired @ /usr/src/sys-realc
lean/kern/kern_synch.c:444
/usr/src/sys-realclean/kern/kern_timeout.c:92 (type: mutex, depth: 4) -- last
acquired @ /usr/src/sys-realclean/kern/kern_timeout.c:315
(null) (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/s
csi_base.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
Hsv (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/scsi
_base.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
(type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/scsi_ba
se.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
"G`/} (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/s
csi_base.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
6 (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/scsi_b
ase.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
@ (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi/scsi_b
ase.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
P& (type: mutex, depth: 0) -- last acquired @ /usr/src/sys-realclean/scsi
/scsi_base.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
/usr/src/sys-realclean/kern/kern_synch.c (type: mutex, depth: 0) -- last acquir
ed @ /usr/src/sys-realclean/scsi/scsi_base.c:1338
&sched_lock (type: sched_lock, depth: 3) -- (already displayed)
uvm_fault(0xffffffff81cd9200, 0xd00, 0, 1) -> e
kernel: page fault trap, code=0
Faulted in DDB; continuing...
ddb{0}>
