On Mon, 2019-10-07 at 13:15 -0400, Kurt Miller wrote: > On Sun, 2019-10-06 at 19:23 -0600, Theo de Raadt wrote: > > > > Kurt Miller <[email protected]> wrote: > > > > > > > > > > > On Sun, 2019-10-06 at 20:01 -0400, Kurt Miller wrote: > > > > > > > > > > > > On Sun, 2019-10-06 at 17:47 -0600, Theo de Raadt wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I applied your diff and built RAMDISK to test it. However, the > > > > > > boot failed with this (full boot message later on): > > > > > > > > > > > > root on rd0a swap on rd0b dump on rd0b > > > > > > panic: cannot open disk, 0x1100/0x2f02, error 2 > > > > > I believe your bsd.rd has no ramdisk filesystem inside it. > > > > > > > > > You are correct, thank you. bsd.mp booted fine. I'll make > > > > a proper bsd.rd and report back results of testing install > > > > with it. > > > I completed 5 bsd.rd upgrades off /mnt/home/_sysupgrade > > > without issue and performed a series of fio tests also > > > without issue. The bsd.rd upgrades would not have survived > > > 2 in a row prior to your change. > > > > > > Disabling command queueing is looking like it stabilized > > > the card for me. I haven't noticed a drop in performance > > > either in an unscientific comparison to numbers I jotted > > > down on Sep 23. > > > > > > Thank you for the quick fix. > > I think that cynical workaround is fine for release, it probably > > affects other people with this ahci block (which has a bad reputation > > in other forums) > > > I hit the issue again using the latest snapshot which > includes the work-around. > > ahci0: log page read failed, slot 31 was still active. > ahci0: stopping the port, softreset slot 31 was still active. > ahci0: failed to reset port during timeout handling, disabling it > > No panic this time. > > The work-around helped with stability in the RAMDISK env where > it was very unstable. Perhaps it is still a good idea. >
This time I got a panic so perhaps there's something helpful in the info below. I had 2x rm -rf on some larger directories going at the time of this panic: ahci0: log page read failed, slot 31 was still active. panic: uvm_fault failed: ffffff80002b4628 Stopped at panic+0x150: TID PID UID PRFLAGS PFLAGS C PU COMMAND db_enter() at panic+0x14c panic() at $x.0+0x6c $x.0() at ahci_port_softreset+0x104 ahci_port_softreset() at ahci_ata_cmd_timeout+0xc4 ahci_ata_cmd_timeout() at softclock+0x130 softclock() at softintr_biglock_wrap+0x1c softintr_biglock_wrap() at softintr_dispatch+0x9c https://www.openbsd.org/ddb.html describes the minimum info required in bug reports. Insufficient info makes it difficult to find and fix bugs. ddb{0}> bt db_enter() at panic+0x14c panic() at $x.0+0x6c $x.0() at ahci_port_softreset+0x104 ahci_port_softreset() at ahci_ata_cmd_timeout+0xc4 ahci_ata_cmd_timeout() at softclock+0x130 softclock() at softintr_biglock_wrap+0x1c softintr_biglock_wrap() at softintr_dispatch+0x9c softintr_dispatch() at arm_do_pending_intr+0xe0 arm_do_pending_intr() at agintc_irq_handler+0x278 agintc_irq_handler() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{0}> trace db_enter() at panic+0x14c panic() at $x.0+0x6c $x.0() at ahci_port_softreset+0x104 ahci_port_softreset() at ahci_ata_cmd_timeout+0xc4 ahci_ata_cmd_timeout() at softclock+0x130 softclock() at softintr_biglock_wrap+0x1c softintr_biglock_wrap() at softintr_dispatch+0x9c softintr_dispatch() at arm_do_pending_intr+0xe0 arm_do_pending_intr() at agintc_irq_handler+0x278 agintc_irq_handler() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{0}> machine ddbcpu 1 Stopped at agintc_ipi_ddb+0x1c: db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{1}> trace db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{1}> machine ddbcpu 2 Stopped at agintc_ipi_ddb+0x1c: db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{2}> trace db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{2}> machine ddbcp 3 Stopped at agintc_ipi_ddb+0x1c: db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{3}> trace db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{3}> machine ddbcpu 4 Stopped at agintc_ipi_ddb+0x1c: db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{4}> bt db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{4}> machine ddbcpu 5 Stopped at agintc_ipi_ddb+0x1c: db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{5}> trace db_enter() at agintc_ipi_ddb+0x18 agintc_ipi_ddb() at arm_cpu_intr+0x30 arm_cpu_intr() at handle_el1h_irq+0x6c handle_el1h_irq() at sched_idle+0x220 sched_idle() at proc_trampoline+0x10 ddb{5}> ps /o TID PID UID PRFLAGS PFLAGS CPU COMMAND ddb{5}> ps PID TID PPID UID S FLAGS WAIT COMMAND 36475 461032 41749 1000 3 0x100003 inode ls 52713 175583 41749 0 3 0x100003 biowait rm 10482 317217 41749 0 3 0x100003 biowait rm 41749 113421 24283 1000 3 0x10008b pause ksh 24283 13517 89760 1000 3 0x90 select sshd 89760 75825 17053 0 3 0x92 poll sshd 84258 33546 1 0 3 0x100083 ttyin ksh 76357 432527 1 0 3 0x100098 poll cron 79562 239967 1 99 3 0x100090 poll sndiod 66858 49310 1 110 3 0x100090 poll sndiod 75045 271937 18580 95 3 0x100092 kqread smtpd 43519 395288 18580 103 3 0x100092 kqread smtpd 56090 418933 18580 95 3 0x100092 kqread smtpd 99024 370275 18580 95 3 0x100092 kqread smtpd 88978 455206 18580 95 3 0x100092 kqread smtpd 73147 64226 18580 95 3 0x100092 kqread smtpd 18580 27418 1 0 3 0x100080 kqread smtpd 17053 452596 1 0 3 0x80 select sshd 1697 131255 1 0 3 0x100080 poll ntpd 95614 56344 22050 83 3 0x100092 poll ntpd 22050 224947 1 83 3 0x100092 poll ntpd 5584 175979 42164 74 3 0x100092 bpf pflogd 42164 168512 1 0 3 0x80 netio pflogd 75784 23156 24896 73 3 0x100010 ffs_fsync syslogd 24896 236518 1 0 3 0x100082 netio syslogd 17545 494366 1 77 3 0x100090 poll dhclient 90523 330978 1 0 3 0x80 poll dhclient 51002 28152 34215 115 3 0x100092 kqread slaacd 45308 447710 34215 115 3 0x100092 kqread slaacd 34215 288252 1 0 3 0x100080 kqread slaacd 29951 158156 0 0 3 0x14200 pgzero zerothread 6723 366506 0 0 3 0x14200 aiodoned aiodoned 48695 474648 0 0 3 0x14200 bqwait update 74435 163002 0 0 3 0x14200 cleaner cleaner 2088 272467 0 0 3 0x14200 reaper reaper 74842 392793 0 0 3 0x14200 pgdaemon pagedaemon 45636 161139 0 0 3 0x14200 bored crynlk 37512 15756 0 0 3 0x14200 bored crypto 11342 37970 0 0 3 0x14200 bored tztq *20523 252715 0 0 7 0x40014200 idle5 82218 153747 0 0 7 0x40014200 idle4 90923 75218 0 0 7 0x40014200 idle3 92620 362709 0 0 7 0x40014200 idle2 44453 517988 0 0 7 0x40014200 idle1 52190 332617 0 0 3 0x14200 bored sensors 16085 331753 0 0 3 0x14200 usbtsk usbtask 96363 218165 0 0 3 0x14200 usbatsk usbatsk 79032 522320 0 0 3 0x14200 mmctsk sdmmc1 92309 486467 0 0 3 0x14200 mmctsk sdmmc0 66781 357474 0 0 2 0x14200 softnet 32037 254087 0 0 3 0x14200 bored systqmp 85348 4219 0 0 3 0x14200 bored systq 25652 481708 0 0 3 0x40014200 bored softclock 76554 183769 0 0 7 0x40014200 idle0 15035 498790 0 0 3 0x14200 bored smr 82235 383808 0 0 3 0x14200 kmalloc kmthread 1 176461 0 0 3 0x82 wait init 0 0 -1 0 3 0x10200 scheduler swapper ddb{5}> show uvm Current UVM status: pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12 961557 VM pages: 6093 active, 51173 inactive, 0 wired, 781467 free (97687 zer o) min 10% (25) anon, 10% (25) vnode, 5% (12) vtext freemin=32051, free-target=42734, inactive-target=0, wired-max=320519 faults=1769238, traps=0, intrs=0, ctxswitch=1526473 fpuswitch=0 softint=56745, syscalls=990596, kmapent=14 fault counts: noram=0, noanon=0, noamap=0, pgwait=0, pgrele=0 ok relocks(total)=76263(76440), anget(retries)=1534607(0), amapcopy=65572 neighbor anon/obj pg=4435/63903, gets(lock/unlock)=133427/76440 cases: anon=1524599, anoncow=10008, obj=129456, prcopy=3794, przero=101380 daemon and swap counts: woke=0, revs=0, scans=0, obscans=0, anscans=0 busy=0, freed=0, reactivate=0, deactivate=0 pageouts=0, pending=0, nswget=0 nswapdev=1 swpages=1050088, swpginuse=0, swpgonly=0 paging=0 kernel pointers: objs(kern)=0xffffff8000cac3c0 ddb{5}> show bcstats Current Buffer Cache status: numbufs 23731 busymapped 128, delwri 143 kvaslots 12019 avail kva slots 11891 bufpages 100112, dmapages 100112, dirtypages 1140 pendingreads 2, pendingwrites 126 highflips 0, highflops 0, dmaflips 0 ddb{5}> dmesg OpenBSD 6.6 (GENERIC.MP) #261: Sun Oct 6 23:03:15 MDT 2019 [email protected]:/usr/src/sys/arch/arm64/compile/GENERIC.MP real mem = 4094251008 (3904MB) avail mem = 3895128064 (3714MB) mainbus0 at root: Pine64 RockPro64 cpu0 at mainbus0 mpidr 0: ARM Cortex-A53 r0p4 cpu0: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache cpu0: 512KB 64b/line 16-way L2 cache efi0 at mainbus0: UEFI 2.8 efi0: Das U-Boot rev 0x20191000 apm0 at mainbus0 psci0 at mainbus0: PSCI 1.1, SMCCC 1.1 agintc0 at mainbus0 sec shift 3:3 nirq 288 nredist 6 ipi: 0, 1: "interrupt-cont roller" agintcmsi0 at agintc0 syscon0 at mainbus0: "qos" syscon1 at mainbus0: "qos" syscon2 at mainbus0: "qos" syscon3 at mainbus0: "qos" syscon4 at mainbus0: "qos" syscon5 at mainbus0: "qos" syscon6 at mainbus0: "qos" syscon7 at mainbus0: "qos" syscon8 at mainbus0: "qos" syscon9 at mainbus0: "qos" syscon10 at mainbus0: "qos" syscon11 at mainbus0: "qos" syscon12 at mainbus0: "qos" syscon13 at mainbus0: "qos" syscon14 at mainbus0: "qos" syscon15 at mainbus0: "qos" syscon16 at mainbus0: "qos" syscon17 at mainbus0: "qos" syscon18 at mainbus0: "qos" syscon19 at mainbus0: "qos" syscon20 at mainbus0: "qos" syscon21 at mainbus0: "qos" syscon22 at mainbus0: "qos" syscon23 at mainbus0: "qos" syscon24 at mainbus0: "qos" syscon25 at mainbus0: "power-management" "power-controller" at syscon25 not configured syscon26 at mainbus0: "syscon" "io-domains" at syscon26 not configured syscon27 at mainbus0: "syscon" syscon28 at mainbus0: "syscon" rkclock0 at mainbus0 rkclock1 at mainbus0 syscon29 at mainbus0: "syscon" "io-domains" at syscon29 not configured "usb2-phy" at syscon29 not configured "usb2-phy" at syscon29 not configured "phy" at syscon29 not configured "pcie-phy" at syscon29 not configured rkpinctrl0 at mainbus0: "pinctrl" rkgpio0 at rkpinctrl0 rkgpio1 at rkpinctrl0 rkgpio2 at rkpinctrl0 rkgpio3 at rkpinctrl0 rkgpio4 at rkpinctrl0 pwmreg0 at mainbus0 "fit-images" at mainbus0 not configured "pmu_a53" at mainbus0 not configured "pmu_a72" at mainbus0 not configured agtimer0 at mainbus0: tick rate 24000 KHz "xin24m" at mainbus0 not configured simplebus0 at mainbus0: "amba" "dma-controller" at simplebus0 not configured "dma-controller" at simplebus0 not configured rkpcie0 at mainbus0 pci0 at rkpcie0 ppb0 at pci0 dev 0 function 0 "Rockchip RK3399 Root Complex" rev 0x00: msi pci1 at ppb0 bus 1 ahci0 at pci1 dev 0 function 0 "ASMedia ASM1061 AHCI" rev 0x01: msi, AHCI 1.2 ahci0: port 1: 6.0Gb/s scsibus0 at ahci0: 32 targets sd0 at scsibus0 targ 1 lun 0: <ATA, Samsung SSD 840, EXT0> naa.50025388a003c28f sd0: 476940MB, 512 bytes/sector, 976773168 sectors, thin dwge0 at mainbus0: address 12:e7:22:42:f7:96 rgephy0 at dwge0 phy 0: RTL8169S/8110S/8211 PHY, rev. 6 dwmmc0 at mainbus0: 50 MHz base clock sdmmc0 at dwmmc0: 4-bit, sd high-speed, mmc high-speed, dma sdhc0 at mainbus0 sdhc0: SDHC 3.0, 200 MHz base clock sdmmc1 at sdhc0: 8-bit, sd high-speed, mmc high-speed, dma ehci0 at mainbus0 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1 ohci0 at mainbus0: version 1.0 ehci1 at mainbus0 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1 ohci1 at mainbus0: version 1.0 rkdwusb0 at mainbus0: "usb" xhci0 at rkdwusb0, xHCI 1.10 usb2 at xhci0: USB revision 3.0 uhub2 at usb2 configuration 1 interface 0 "Generic xHCI root hub" rev 3.00/1.00 addr 1 rkdwusb1 at mainbus0: "usb" xhci1 at rkdwusb1, xHCI 1.10 usb3 at xhci1: USB revision 3.0 uhub3 at usb3 configuration 1 interface 0 "Generic xHCI root hub" rev 3.00/1.00 addr 1 "saradc" at mainbus0 not configured rkiic0 at mainbus0 iic0 at rkiic0 rkiic1 at mainbus0 iic1 at rkiic1 com0 at mainbus0: ns16550, no working fifo com1 at mainbus0: ns16550, no working fifo com1: console "thermal-zones" at mainbus0 not configured rktemp0 at mainbus0 rkiic2 at mainbus0 iic2 at rkiic2 rkpmic0 at iic2 addr 0x1b: RK808 fanpwr0 at iic2 addr 0x40: SYR827, 1.00 VDC fanpwr1 at iic2 addr 0x41: SYR828, 1.00 VDC rkiic3 at mainbus0 iic3 at rkiic3 fusbtc0 at iic3 addr 0x22 "pwm" at mainbus0 not configured "pwm" at mainbus0 not configured "dmc" at mainbus0 not configured "efuse" at mainbus0 not configured "phy" at mainbus0 not configured "phy" at mainbus0 not configured "watchdog" at mainbus0 not configured "rktimer" at mainbus0 not configured "i2s" at mainbus0 not configured "i2s" at mainbus0 not configured "i2s" at mainbus0 not configured "vop" at mainbus0 not configured "iommu" at mainbus0 not configured "vop" at mainbus0 not configured "iommu" at mainbus0 not configured "hdmi-sound" at mainbus0 not configured "hdmi" at mainbus0 not configured "gpu" at mainbus0 not configured "opp-table0" at mainbus0 not configured "opp-table1" at mainbus0 not configured "opp-table2" at mainbus0 not configured "external-gmac-clock" at mainbus0 not configured "gpio-keys" at mainbus0 not configured "leds" at mainbus0 not configured "sdio-pwrseq" at mainbus0 not configured "vcc12v-dcin" at mainbus0 not configured "vcc1v8-s3" at mainbus0 not configured "vcc3v3-pcie-regulator" at mainbus0 not configured "vcc3v3-sys" at mainbus0 not configured "vcc5v0-host-regulator" at mainbus0 not configured "vcc5v0-typec-regulator" at mainbus0 not configured "vcc5v0-sys" at mainbus0 not configured "vcc5v0-usb" at mainbus0 not configured cpu1 at mainbus0 mpidr 1: ARM Cortex-A53 r0p4 cpu1: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache cpu1: 512KB 64b/line 16-way L2 cache cpu2 at mainbus0 mpidr 2: ARM Cortex-A53 r0p4 cpu2: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache cpu2: 512KB 64b/line 16-way L2 cache cpu3 at mainbus0 mpidr 3: ARM Cortex-A53 r0p4 cpu3: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache cpu3: 512KB 64b/line 16-way L2 cache cpu4 at mainbus0 mpidr 100: ARM Cortex-A72 r0p2 cpu4: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache cpu4: 1024KB 64b/line 16-way L2 cache cpu5 at mainbus0 mpidr 101: ARM Cortex-A72 r0p2 cpu5: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache cpu5: 1024KB 64b/line 16-way L2 cache usb4 at ohci0: USB revision 1.0 uhub4 at usb4 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00 addr 1 usb5 at ohci1: USB revision 1.0 uhub5 at usb5 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00 addr 1 scsibus1 at sdmmc0: 2 targets, initiator 0 sd1 at scsibus1 targ 1 lun 0: <SD/MMC, SD16G, 0020> removable sd1: 29862MB, 512 bytes/sector, 61157376 sectors sdmmc1: can't enable card vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root scsibus3 at softraid0: 256 targets bootfile: sd0a:/bsd boot device: sd0 root on sd1a (4cbb6909064a3cdc.a) swap on sd1b dump on sd1b
