Ted Unangst wrote: >Philippe Meunier wrote: >> - is the panic intended (well, known to the developers and considered >> normal; I hesitate to call it a feature) or is it an oversight? > >no, nothing bioctl does should kill init like that.
Well, it does, and it's reproducible. >> - I would have thought that, once the softraid volume has been created, its >> metadata wouldn't need to change (unless the passphrase is changed, or the >> volume is roaming, as seen above). Any idea why part of it gets trashed? > >that's true, but maybe a stray write killed it? It happens even in single-user mode when only / is mounted read-only and only init and a shell are running. Here are all the gory details, if you want them, including the ddb trace when the kernel panics. # wget http://ftp.jaist.ac.jp/pub/OpenBSD/6.1/amd64/SHA256.sig # wget http://ftp.jaist.ac.jp/pub/OpenBSD/6.1/amd64/install61.fs # signify -Cp /etc/signify/openbsd-61-base.pub -x SHA256.sig install61.fs Signature Verified install61.fs: OK # dd if=install61.fs of=/dev/rsd1c bs=1m Then boot from install USB on the t61 and go to shell: # dd if=/dev/random of=/dev/rwd0c bs=1m # fdisk -iy wd0 # disklabel -E wd0 a a 64 * RAID w q # bioctl -c C -l wd0a softraid0 New passphrase: Re-type passphrase: sd1 at scsibus2 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006> SCSI2 0/direct fixed sd1: 114470MB, 512 bytes/sector, 234435008 sectors softraid0: CRYPTO volume attached as sd1 # cd /dev && sh MAKEDEV sd1 # dd if=/dev/zero of=/dev/rsd1c bs=1m count=1 # exit I default System hostname = t61 Which network interface do you wish to configure = done DNS domain name = my.domain DNS nameservers = none Start sshd(8) by default = no Do you want the X Window System to be started by xenodm(1) = no Setup a user = no Which disk is the root disk = sd1 Use (W)hole disk MBR, whole disk (G)PT or (E)dit = W Use (A)uto layout, (E)dit auto layout, or create (C)ustom layout = A Which disk do you wish to initialize = done Location of sets = disk Is the disk partition already mounted = no Which disk contains the install media = sd0 Which sd0 partition has the install sets = a Pathname to the sets = 6.1/amd64 Set name(s) = done Directory does not contain SHA256.sig. Continue without verification = yes Location of sets = done What timezone are you in = PRC # reboot Then everything works fine. Here's the dmesg: OpenBSD 6.1 (GENERIC.MP) #20: Sat Apr 1 13:45:56 MDT 2017 [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 1030422528 (982MB) avail mem = 994574336 (948MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0010 (73 entries) bios0: vendor LENOVO version "7LETD0WW (2.30 )" date 02/27/2012 bios0: LENOVO 7659AE6 acpi0 at bios0: rev 2 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SSDT ECDT TCPA APIC MCFG HPET SLIC BOOT ASF! SSDT SSDT SSDT SSDT SSDT acpi0: wakeup devices LID_(S3) SLPB(S3) IGBE(S4) EXP0(S4) EXP1(S4) EXP2(S4) EXP3(S4) EXP4(S4) PCI1(S4) USB0(S3) USB1(S3) USB2(S3) USB3(S3) USB4(S3) EHC0(S3) EHC1(S3) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpiec0 at acpi0 acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz, 2194.89 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF,SENSOR cpu0: 4MB 64b/line 16-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 199MHz cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2, IBE cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz, 1995.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG,LAHF,PERF,SENSOR cpu1: 4MB 64b/line 16-way L2 cache cpu1: smt 0, core 1, package 0 ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins acpimcfg0 at acpi0 addr 0xf0000000, bus 0-63 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus -1 (AGP_) acpiprt2 at acpi0: bus 2 (EXP0) acpiprt3 at acpi0: bus 3 (EXP1) acpiprt4 at acpi0: bus 4 (EXP2) acpiprt5 at acpi0: bus 5 (EXP3) acpiprt6 at acpi0: bus 13 (EXP4) acpiprt7 at acpi0: bus 21 (PCI1) acpicpu0 at acpi0: !C3(250@17 mwait.3@0x20), !C2(500@1 mwait.1@0x10), C1(1000@1 mwait.1), PSS acpicpu1 at acpi0: !C3(250@17 mwait.3@0x20), !C2(500@1 mwait.1@0x10), C1(1000@1 mwait.1), PSS acpipwrres0 at acpi0: PUBS, resource for USB0, USB2, USB4, EHC0, EHC1 acpitz0 at acpi0: critical temperature is 127 degC acpitz1 at acpi0: critical temperature is 100 degC acpibtn0 at acpi0: LID_ acpibtn1 at acpi0: SLPB "PNP0303" at acpi0 not configured "IBM0057" at acpi0 not configured tpm0 at acpi0: TPM_ addr 0xfed40000/0x5000: Atmel 97SC3203 rev 0x5 acpibat0 at acpi0: BAT0 model "92P1131" serial 8030 type LION oem "SANYO" acpiac0 at acpi0: AC unit online acpithinkpad0 at acpi0 "PNP0C14" at acpi0 not configured acpidock0 at acpi0: GDCK not docked (0) acpivideo0 at acpi0: VID_ acpivout0 at acpivideo0: LCD0 acpivideo1 at acpi0: VID_ cpu0: Enhanced SpeedStep 2194 MHz: speeds: 2001, 2000, 1600, 1200, 800 MHz pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel GM965 Host" rev 0x0c inteldrm0 at pci0 dev 2 function 0 "Intel GM965 Video" rev 0x0c drm0 at inteldrm0 intagp0 at inteldrm0 agp0 at intagp0: aperture at 0xe0000000, size 0x10000000 inteldrm0: msi inteldrm0: 1440x900, 32bpp wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation) wsdisplay0: screen 1-5 added (std, vt100 emulation) "Intel GM965 Video" rev 0x0c at pci0 dev 2 function 1 not configured em0 at pci0 dev 25 function 0 "Intel ICH8 IGP M AMT" rev 0x03: msi, address 00:1a:6b:d2:10:56 uhci0 at pci0 dev 26 function 0 "Intel 82801H USB" rev 0x03: apic 1 int 20 uhci1 at pci0 dev 26 function 1 "Intel 82801H USB" rev 0x03: apic 1 int 21 ehci0 at pci0 dev 26 function 7 "Intel 82801H USB" rev 0x03: apic 1 int 22 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 azalia0 at pci0 dev 27 function 0 "Intel 82801H HD Audio" rev 0x03: msi azalia0: codecs: Analog Devices AD1984, Conexant/0x2bfa, using Analog Devices AD1984 audio0 at azalia0 ppb0 at pci0 dev 28 function 0 "Intel 82801H PCIE" rev 0x03: msi pci1 at ppb0 bus 2 ppb1 at pci0 dev 28 function 1 "Intel 82801H PCIE" rev 0x03: msi pci2 at ppb1 bus 3 iwn0 at pci2 dev 0 function 0 "Intel Wireless WiFi Link 4965" rev 0x61: msi, MIMO 2T3R, MoW1, address 00:13:e8:a3:b8:29 ppb2 at pci0 dev 28 function 2 "Intel 82801H PCIE" rev 0x03: msi pci3 at ppb2 bus 4 ppb3 at pci0 dev 28 function 3 "Intel 82801H PCIE" rev 0x03: msi pci4 at ppb3 bus 5 ppb4 at pci0 dev 28 function 4 "Intel 82801H PCIE" rev 0x03: msi pci5 at ppb4 bus 13 uhci2 at pci0 dev 29 function 0 "Intel 82801H USB" rev 0x03: apic 1 int 16 uhci3 at pci0 dev 29 function 1 "Intel 82801H USB" rev 0x03: apic 1 int 17 uhci4 at pci0 dev 29 function 2 "Intel 82801H USB" rev 0x03: apic 1 int 18 ehci1 at pci0 dev 29 function 7 "Intel 82801H USB" rev 0x03: apic 1 int 19 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb5 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xf3 pci6 at ppb5 bus 21 cbb0 at pci6 dev 0 function 0 "Ricoh 5C476 CardBus" rev 0xba: apic 1 int 16 "Ricoh 5C832 Firewire" rev 0x04 at pci6 dev 0 function 1 not configured sdhc0 at pci6 dev 0 function 2 "Ricoh 5C822 SD/MMC" rev 0x21: apic 1 int 18 sdhc0: SDHC 1.0, 33 MHz base clock sdmmc0 at sdhc0: 4-bit, sd high-speed, mmc high-speed "Ricoh 5C843 MMC" rev 0x11 at pci6 dev 0 function 3 not configured "Ricoh 5C592 Memory Stick" rev 0x11 at pci6 dev 0 function 4 not configured "Ricoh 5C852 xD" rev 0x11 at pci6 dev 0 function 5 not configured cardslot0 at cbb0 slot 0 flags 0 cardbus0 at cardslot0: bus 22 device 0 cacheline 0x8, lattimer 0xb0 pcmcia0 at cardslot0 pcib0 at pci0 dev 31 function 0 "Intel 82801HEM LPC" rev 0x03 pciide0 at pci0 dev 31 function 2 "Intel 82801HBM SATA" rev 0x03: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility wd0 at pciide0 channel 0 drive 0: <HITACHI HTS541612J9SA00 3H> wd0: 16-sector PIO, LBA48, 114473MB, 234441648 sectors wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5 atapiscsi0 at pciide0 channel 1 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, DVDRAM GSA-U10N, 1.05> ATAPI 5/cdrom removable cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2 ichiic0 at pci0 dev 31 function 3 "Intel 82801H SMBus" rev 0x03: apic 1 int 23 iic0 at ichiic0 usb2 at uhci0: USB revision 1.0 uhub2 at usb2 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci1: USB revision 1.0 uhub3 at usb3 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb4 at uhci2: USB revision 1.0 uhub4 at usb4 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb5 at uhci3: USB revision 1.0 uhub5 at usb5 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb6 at uhci4: USB revision 1.0 uhub6 at usb6 configuration 1 interface 0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) wsmouse0 at pms0 mux 0 wsmouse1 at pms0 mux 0 pms0: Synaptics touchpad, firmware 6.2 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 aps0 at isa0 port 0x1600/31 vmm0 at mainbus0: VMX uvideo0 at uhub1 port 5 configuration 1 interface 0 "Chicony Electronics Co., Ltd. Integrated Camera" rev 2.00/3.20 addr 2 video0 at uvideo0 vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root scsibus3 at softraid0: 256 targets sd0 at scsibus3 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006> SCSI2 0/direct fixed sd0: 114470MB, 512 bytes/sector, 234435008 sectors softraid0: volume sd0 is roaming, it used to be sd1, updating metadata root on sd0a (1a45e2363357077d.a) swap on sd0b dump on sd0b Automatic boot in progress: starting file system checks. /dev/sd0a (1a45e2363357077d.a): file system is clean; not checking /dev/sd0k (1a45e2363357077d.k): file system is clean; not checking /dev/sd0d (1a45e2363357077d.d): file system is clean; not checking /dev/sd0f (1a45e2363357077d.f): file system is clean; not checking /dev/sd0g (1a45e2363357077d.g): file system is clean; not checking /dev/sd0h (1a45e2363357077d.h): file system is clean; not checking /dev/sd0j (1a45e2363357077d.j): file system is clean; not checking /dev/sd0i (1a45e2363357077d.i): file system is clean; not checking /dev/sd0e (1a45e2363357077d.e): file system is clean; not checking setting tty flags pf enabled starting network reordering libraries: done. openssl: generating isakmpd/iked RSA keys... done. ssh-keygen: generating new host keys: RSA DSA ECDSA ED25519 starting early daemons: syslogd pflogd ntpd. starting RPC daemons:. savecore: no core dump checking quotas: done. clearing /tmp kern.securelevel: 0 -> 1 creating runtime link editor directory cache. preserving editor files. starting network daemons: smtpd sndiod. Path to firmware: http://firmware.openbsd.org/firmware/6.1/ Installing: uvideo-firmware iwn-firmware vmm-firmware http://firmware.openbsd.org/firmware/6.1/: ftp: firmware.openbsd.org: no address associated with name http://firmware.openbsd.org/firmware/6.1/: empty Can't find uvideo-firmware Can't find iwn-firmware Can't find vmm-firmware starting local daemons: cron. Sun Aug 20 01:43:27 CST 2017 Reboot into single user mode (I re-mounted / with the sync option before remembering that / was read-only anyway; it doesn't seem to change anything to the result: either way you get a panic): Using drive 0, partition 3. Loading...... probing: pc0 mem[630K 997M a20=on] disk: hd0+ sr0* >> OpenBSD/amd64 BOOT 3.33 Passphrase: boot> boot -s [...] Enter pathname of shell or RETURN for sh: # mount -u -o sync / # mount /dev/sd0a on / type ffs (local, read-only, synchronous) # sync; bioctl -d sd0 sd0 detached panic: init died (signal 11, exit 0) Stopped at Debugger+0x9: leave TID PID UID PRFLAGS PFLAGS CPU COMMAND 328004 52033 0 0x14000 0x200 0 reaper *503417 1 0 0x802 0x2000 1 init Debugger() at Debugger+0x9 panic() at panic+0xfe exit1() at exit1+0x592 trapsignal() at trapsignal+0x110 trap() at trap+0x6c0 --- trap (number 6) --- end of kernel end trace frame: 0x10c1d, count: 10 0x4031aa01590: https://www.openbsd.org/ddb.html describes the minimum info required in bug reports. Insufficient info makes it difficult to find and fix bugs. ddb[1]> mach ddbcpu 0 Stopped at Debugger+0x9: leave Debugger() at Debugger+0x9 x86_ipi_handler() at at x86_ipi_handler+0x76 Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c --- interrupt --- __mp_lock() at __mp_lock+0x48 softintr_dispatch() at softintr_dispatch+0x43 Xsoftclock() at Xsoftclock+0x1f --- interrupt --- end trace frame: 0x0, count: 9 0x8: ddb{0}> mach ddbcpu 1 Stopped at Debugger+0x9: leave Debugger() at Debugger+0x9 panic() at panic+0xfe exit1() at exit1+0x592 trapsignal() at trapsignal+0x110 trap() at trap+0x6c0 --- trap (number 6) --- end of kernel end trace frame: 0x10c1d, count: 10 0x4031aa01590: ddb{1}> ps PID TID PPID UID S FLAGS WAIT COMMAND 2618 161906 0 0 2 0x14200 zerothread 16099 377664 0 0 3 0x14200 aiodoned aiodoned 99842 88767 0 0 3 0x14200 syncer update 74443 90149 0 0 3 0x14200 cleaner cleaner 52033 328004 0 0 7 0x14200 reaper 98927 437205 0 0 3 0x14200 pgdameon pagedaemon 89000 87772 0 0 3 0x14200 bored crynlk 8771 42175 0 0 3 0x14200 bored crypto 7178 54837 0 0 3 0x14200 pftm pfpurge 70306 164952 0 0 3 0x14200 mmctsk sdmmc0 42523 38750 0 0 3 0x14200 usbtsk usbtask 73518 244458 0 0 3 0x14200 usbatsk usbatsk 31971 224750 0 0 3 0x14200 bored i915 23589 454162 0 0 3 0x40014200 acpi0 acpi0 16513 75343 0 0 3 0x40014200 idle1 49806 157139 0 0 3 0x14200 bored sensors 91350 19031 0 0 3 0x14200 bored softnet 63816 309740 0 0 3 0x14200 bored systqmp 62045 153648 0 0 3 0x40014200 bored softclock 96214 61001 0 0 3 0x40014200 idle0 2726 16281 0 0 3 0x14200 bored sbar * 1 503417 0 0 7 0x2802 init 0 0 -1 0 3 0x10200 scheduler swapper Interestingly, there's no shell or bioctl showing in that list. ddb{1}> show uvm Current UVM status: pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12 244888 VM pages: 67 active, 0 inactive, 1300 wired, 236711 free (29585 zero) min 10% (25) anon, 10% (25) vnode, 5% (12) vtext pages 0 anon, 0 vnode, 0 vtext freemin=8162, free-target=10882, inactive-target=0, wired-max=81629 faults=985, traps=985, intrs=463, ctxswitch=6487 fpuswitch=0 softint=17662, syscalls=733, kmapent=17 fault counts: noram=0, noanon=0, noamap=0, pgwait=0, pgrele=0 ok relocks(total)=200(200), anget(retries)=472(0), amapcopy=485 neighbor anon/obj pg=58/377, gets(lock/unlock)=322/204 cases: anon=347, anoncow=125, obj=299, prcopy=19, przero=191 daemon and swap counts: woke=0, revs=0, scans=0, obscans=0, anscans=0 busy=0, freed=0, reactivate=0, deactivate=0 pageouts=0, pending=0, nswget=0 nswapdev=1, nanon=0, nanonneeded=0, swpgonly=0, paging=0 kernel pointers: objs(kern)=0xffffffff81941c80 ddb{1}> show bcstats Current Buffer Cache status: numbufs 0 busymapped 0, delwri 0 kvaslots 3061 avail kva slots 3061 bufpages 0, dirtypages 0 pendingreads 0, pendingwrites 0 ddb{1}> boot reboot At that point the machine hangs for ever so a hard reboot is necessary. Then: Using drive 0, partition 3. Loading...... probing: pc0 mem[630K 997M a20=on] disk: hd0+ sr0 >> OpenBSD amd64 BOOT 3.33 open(hd0a:/etc/boot.conf): Invalid argument boot> cannot open hd0a:/etc/random.seed: Invalid argument booting hd0a:/bsd: open hd0a:/bsd: Invalid argument failed(22). will try /bsd boot> cannot open hd0a:/etc/random.seed: Invalid argument booting hd0a:/bsd: open hd0a:/bsd: Invalid argument failed(22). will try /bsd Turning timeout off. boot> Booting from the install USB again and going to shell: # bioctl -c C -l /dev/wd0a softraid0 Passphrase: sd1 at scsibus2 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006> SCSI2 0/direct fixed sd1: 114470MB, 512 bytes/sector, 234435008 sectors softraid0: volume sd1 is roaming, it used to be sd0, updating metadata softraid0: volume sd1 is roaming, it used to be sd0, updating metadata softraid0: CRYPTO volume attached as sd1 # cd /dev && sh MAKEDEV sd1 # fsck /dev/rsd1a ** /dev/rsd1a ** File system is clean; not checking # mount -t ffs /dev/sd1a /mnt # ls /mnt .cshrc altroot bsd bsd.sp etc mnt sbin tmp var .profile bin bsd.rd dev home root sys use # umount /mnt >> - is there a way to get the computer to boot again, short of wiping the >> disk with dd and starting from scratch again? > >you can run installboot. use the softraid disk name. I tried, from the install USB: # installboot wd0 installboot: /usr/mdec/biosboot: No such file or directory # mount -t ffs /dev/sd0a /mnt # cd / # gzip -d -c /mnt/6.1/amd64/base61.tgz | tar xfv - ./usr/mdec/biosboot ./usr/mdec/biosboot # installboot wd0 installboot: open /usr/mdec/boot: No such file or directory # gzip -d -c /mnt/6.1/amd64/base61.tgz | tar xfv - ./usr/mdec/boot ./usr/mdec/boot # installboot wd0 installboot: cross-device install # I don't know whether the "cross-device install" message is supposed to be just informative or indicates an error... Anyway, upon rebooting the machine, I still get the same "open(hd0a:/etc/boot.conf): Invalid argument" error message. Philippe
