Fwd: Reboot (dounmount) crash with latest kernel on amd64
I just answered Grégoire, but maybe it might interest the list. -- Forwarded message -- From: Olivier Antoine Date: Tue, Dec 12, 2017 at 8:36 PM Subject: Re: Reboot (dounmount) crash with latest kernel on amd64 To: Grégoire Jadi Hi, I did two VMs. One with fully encrypted disk and the other without. The VM with softraid crypto systematically hangs in ddb on reboot. The one without encryption doesn't crash. syncing disks... done kernel: protection fault trap, code=0 Stopped at dounmount+0x35: movq0x28(%r13),%rax ddb> trace dounmount(8000b410,ff001f19f250,400) at dounmount+0x35 vop_generic_revoke(1f) at vop_generic_revoke+0x5d VOP_REVOKE(ab67f26a1630ea8e,8001) at VOP_REVOKE+0x31 vdevgone(81711550,10,4,1f) at vdevgone+0x8a disk_gone(8001ef10,8001ee00) at disk_gone+0x53 sddetach(1,8001ee00) at sddetach+0x2a config_detach(80081600,8009b000) at config_detach+0x14a scsi_detach_lun(0,80096000,80093000,80096980) at scsi_d etach_lun+0xa8 sr_discipline_shutdown(370,80093000) at sr_discipline_shutdown+0x10e sr_shutdown() at sr_shutdown+0x2a boot(81a8d740) at boot+0x51 reboot(8e3dccf0) at reboot+0x49 sys_reboot(8000b410,ff001f7d2960,815f5545) at sys_reboot+0x 25 syscall() at syscall+0x206 --- syscall (number 55) --- end of kernel end trace frame: 0x7f7e7f40, count: -14 0x573a750151a: ddb> ps PID TID PPIDUID S FLAGS WAIT COMMAND *96234 522304 1 0 7 0x3reboot 12747 383130 0 0 2 0x14200zerothread 39318 475681 0 0 3 0x14200 aiodoned aiodoned 40196 451875 0 0 3 0x14200 syncerupdate 7254 108576 0 0 3 0x14200 cleaner cleaner 15239 90739 0 0 3 0x14200 reaperreaper 5353 269048 0 0 3 0x14200 pgdaemon pagedaemon 96962 12685 0 0 3 0x14200 bored sensors 24843 466294 0 0 3 0x14200 bored srdis 99137 348339 0 0 3 0x14200 bored crynlk 58193 78054 0 0 3 0x14200 bored crypto 61692 524230 0 0 3 0x14200 bored softnet 13415 80733 0 0 3 0x14200 bored systqmp 11167 16722 0 0 3 0x14200 bored systq 70413 458708 0 0 3 0x40014200 bored softclock 16129 120481 0 0 3 0x40014200idle0 1 414329 0 0 30x82 wait init 0 0 -1 0 3 0x10200 scheduler swapper ddb> show reg rdi 0x802e9400 rsi0x808__kernel_end_phys+0x648 rbp 0x8e3dca90 rbx 0x8000b410 rdx 0x8000b410 rcx0 rax0 r80x8195bf80rw_ops+0x10 r9 0x21 r100 r11 0x1526f000 r120x808__kernel_end_phys+0x648 r13 0xdeaf0014deafbead r14 0x802e9400 r15 0x8000b410 rip 0x811be9b5dounmount+0x35 cs 0x8 rflags 0x10286__ALIGN_SIZE+0xf286 rsp 0x8e3dca60 ss 0x10 dounmount+0x35: movq0x28(%r13),%rax OpenBSD 6.2-current (GENERIC) #269: Mon Dec 11 00:12:56 MST 2017 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC real mem = 520085504 (495MB) avail mem = 497549312 (474MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0 acpi at bios0 not configured cpu0 at mainbus0: (uniprocessor) cpu0: Intel(R) Core(TM) i3-3227U CPU @ 1.90GHz, 1897.04 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,XSAVE,AVX,F16C,HV,NXE,RDTSCP,LONG,LAHF,ITSC,FSGSBASE,SMEP,ERMS cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 pvbus0 at mainbus0: OpenBSD pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "OpenBSD VMM Host" rev 0x00 virtio0 at pci0 dev 1 function 0 "Qumranet Virtio RNG" rev 0x00 viornd0 at virtio0 virtio0: irq 3 virtio1 at pci0 dev 2 function 0 "Qumranet Virtio Storage" rev 0x00 vioblk0 at virtio1 scsibus1 at vioblk0: 2 targets sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed sd0: 4608MB, 512 bytes/sector, 9437184 sectors virtio1: irq 5 virtio2 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00 vio0 at virtio2: address fe:e1:bb:d1:6e:e0 virtio2: irq 6 virtio3 at pci0 dev 4 func
Re: Reboot (dounmount) crash with latest kernel on amd64
On Tue, Dec 12, 2017 at 04:33:41PM +0100, Grégoire Jadi wrote:
> >Synopsis:Reboot (dounmount) crash with latest kernel on amd64
> >Category:kernel amd64
> >Environment:
> System : OpenBSD 6.2
> Details : OpenBSD 6.2-current (GENERIC.MP) #1: Tue Dec 12 15:51:42
> CET 2017
>
> daimrod@puffy:/home/daimrod/src/openbsd/src/sys/arch/amd64/compile/GENERIC.MP
>
> Architecture: OpenBSD.amd64
> Machine : amd64
> >Description:
> I compiled the latest version of the kernel from CVS to try stsp@ patch on
> ieee80211_node_checkrssi and rebooted several times.
> Everytime I rebooted my computer dounmount crashed.
>
> Sorry I was a bit tired of copying the addresses so I replaced them with
> "...". Since it happens everytime I can provide them if needed.
>
> Here is the trace/ps/show reg :
>
> syncing disks... done
> kernel: protection fault trap, code=0
> Stopped atdounmount+0x35: movq0x28(%r13), %rax
>
> ddb{0}> trace
> dounmount(800032c77890,ff0118c43190,400) at dounmount+0x35
> vop_generic_revoke(1f) at vop_generic_revoke+0x65
> VOP_REVOKE(2e2303f764c92ff3,1) at VOP_REVOKE+0x31
> vdevgone(...,10,4,1f) at vdevgone+0x8a
> disk_gone(...,...) at disk_gone+0x53
> sddetach(1,...) at sddetach+0x2a
> config_detach(...,...) at config_detach+0x14c
> scsi_detach_lun(0,...,...,...) at scsi_detach_lun+0xa8
> sr_discipline_shutdown(370,...) at sr_discipline_shutdown+0x10e
> sr_shutdown() at sr_shutdown+0x2a
> boot(2) at boot+0x59
> reboot(...) at reboot+0x4f
> sys_reboot(...,...,...) at sys_reboot+0x5c
> syscall() at syscall+0x279
> --- syscall (number 55) ---
> end of kernel
> end trace frame: 0x7f7c5980, count: -15
> 0x10922110151a:
> ddb{0}> ps
> FLAGS WAITCOMMAND
> 0x3 reboot
> [skipped]
>
This is a known issue in recent snapshots/builds. It is being actively worked
on by a few of us, but thanks for the report.
-ml
> ddb{0}> show reg
> rdi 0x8034e000
> rsi 0x808 __kernel_end_phys+0x648
> rbp 0x800032c83810
> rbx 0x800032c77890
> rdx 0x800032c77890
> rcx 0
> rax 0
> r80x818addd0 rw_ops+0x10
> r90x8034e040
> r10 0x21
> r11 0x2
> r12 0x808 __kernel_end_phys+0x648
> r13 0xdead4110
> r14 0x8034e000
> r15 0x800032c77890
> rip 0x8113fe75 dounmount+0x35
> cs0x8
> rflags0x10286 __ALIGN_SIZE+0xf286
> rsp 0x800032c837e0
> ss0x10
> dounmount+0x35: movq0x28(%r13),%rax
> ddb{0}>
>
>
> >How-To-Repeat:
> # reboot
>
> >Fix:
> No idea
>
>
> dmesg:
> OpenBSD 6.2-current (GENERIC.MP) #1: Tue Dec 12 15:51:42 CET 2017
>
> daimrod@puffy:/home/daimrod/src/openbsd/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 4062691328 (3874MB)
> avail mem = 3932639232 (3750MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xe0010 (78 entries)
> bios0: vendor LENOVO version "6QET47WW (1.17 )" date 07/14/2010
> bios0: LENOVO 3680BA5
> acpi0 at bios0: rev 2
> acpi0: sleep states S0 S3 S4 S5
> acpi0: tables DSDT FACP SSDT ECDT APIC MCFG HPET ASF! SLIC BOOT SSDT TCPA
> DMAR SSDT SSDT SSDT
> acpi0: wakeup devices LID_(S3) SLPB(S3) IGBE(S4) EXP1(S4) EXP2(S4) EXP3(S4)
> EXP4(S4) EXP5(S4) EHC1(S3) EHC2(S3) HDEF(S4)
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpiec0 at acpi0
> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 2660.49 MHz
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
> cpu0: 256KB 64b/line 8-way L2 cache
> acpitimer0: recalibrated TSC frequency 2393996550 Hz
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
> cpu0: apic clock running at 132MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
> cpu1 at mainbus0: apid 1 (application processor)
> cpu1: Inte
Reboot (dounmount) crash with latest kernel on amd64
>Synopsis: Reboot (dounmount) crash with latest kernel on amd64
>Category: kernel amd64
>Environment:
System : OpenBSD 6.2
Details : OpenBSD 6.2-current (GENERIC.MP) #1: Tue Dec 12 15:51:42
CET 2017
daimrod@puffy:/home/daimrod/src/openbsd/src/sys/arch/amd64/compile/GENERIC.MP
Architecture: OpenBSD.amd64
Machine : amd64
>Description:
I compiled the latest version of the kernel from CVS to try stsp@ patch on
ieee80211_node_checkrssi and rebooted several times.
Everytime I rebooted my computer dounmount crashed.
Sorry I was a bit tired of copying the addresses so I replaced them with
"...". Since it happens everytime I can provide them if needed.
Here is the trace/ps/show reg :
syncing disks... done
kernel: protection fault trap, code=0
Stopped at dounmount+0x35: movq0x28(%r13), %rax
ddb{0}> trace
dounmount(800032c77890,ff0118c43190,400) at dounmount+0x35
vop_generic_revoke(1f) at vop_generic_revoke+0x65
VOP_REVOKE(2e2303f764c92ff3,1) at VOP_REVOKE+0x31
vdevgone(...,10,4,1f) at vdevgone+0x8a
disk_gone(...,...) at disk_gone+0x53
sddetach(1,...) at sddetach+0x2a
config_detach(...,...) at config_detach+0x14c
scsi_detach_lun(0,...,...,...) at scsi_detach_lun+0xa8
sr_discipline_shutdown(370,...) at sr_discipline_shutdown+0x10e
sr_shutdown() at sr_shutdown+0x2a
boot(2) at boot+0x59
reboot(...) at reboot+0x4f
sys_reboot(...,...,...) at sys_reboot+0x5c
syscall() at syscall+0x279
--- syscall (number 55) ---
end of kernel
end trace frame: 0x7f7c5980, count: -15
0x10922110151a:
ddb{0}> ps
FLAGS WAITCOMMAND
0x3 reboot
[skipped]
ddb{0}> show reg
rdi 0x8034e000
rsi 0x808 __kernel_end_phys+0x648
rbp 0x800032c83810
rbx 0x800032c77890
rdx 0x800032c77890
rcx 0
rax 0
r8 0x818addd0 rw_ops+0x10
r9 0x8034e040
r10 0x21
r11 0x2
r12 0x808 __kernel_end_phys+0x648
r13 0xdead4110
r14 0x8034e000
r15 0x800032c77890
rip 0x8113fe75 dounmount+0x35
cs 0x8
rflags 0x10286 __ALIGN_SIZE+0xf286
rsp 0x800032c837e0
ss 0x10
dounmount+0x35: movq0x28(%r13),%rax
ddb{0}>
>How-To-Repeat:
# reboot
>Fix:
No idea
dmesg:
OpenBSD 6.2-current (GENERIC.MP) #1: Tue Dec 12 15:51:42 CET 2017
daimrod@puffy:/home/daimrod/src/openbsd/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4062691328 (3874MB)
avail mem = 3932639232 (3750MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xe0010 (78 entries)
bios0: vendor LENOVO version "6QET47WW (1.17 )" date 07/14/2010
bios0: LENOVO 3680BA5
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT APIC MCFG HPET ASF! SLIC BOOT SSDT TCPA DMAR
SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) IGBE(S4) EXP1(S4) EXP2(S4) EXP3(S4)
EXP4(S4) EXP5(S4) EHC1(S3) EHC2(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 2660.49 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
acpitimer0: recalibrated TSC frequency 2393996550 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 132MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 2660.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 2660.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,
