On Sun, Sep 04, 2022 at 07:08:51PM +0000, Mikolaj Kucharski wrote:
> Hi,
>
> I have strange setup on some of my machines, when I want to encrypt disk
> where OpenBSD is installed, but still be able to boot them up without
> any user interaction, like passphrase entry for CRYPTO softraid(4). I
> have this so I can with little time spent lock out access to the disk,
> by wiping beginning of the disk, instead of entire disk. I do recognise
> magnitute of limitations of this. I still try to wipe entire disk, when
> it's time for a machine decommission, but first I break CRYPTO softraid
> by wiping beginning and then switch to proper full disk wipe.
I don't see that as a supported use-case at all; this reads like
hand-waving to me and I fail to see any benefit in such a setup.
Keydisks ought to be separate devices (not just slices) by design.
Actively supporting such a setup feels like a step backwards.
>
> All in all that brings me to the below diff. I was only able to test on
> amd64, as this is the only type of machine which I have.
>
>
> Index: i386_softraid.c
> ===================================================================
> RCS file: /cvs/src/usr.sbin/installboot/i386_softraid.c,v
> retrieving revision 1.19
> diff -u -p -u -r1.19 i386_softraid.c
> --- i386_softraid.c 29 Aug 2022 18:54:43 -0000 1.19
> +++ i386_softraid.c 3 Sep 2022 11:28:55 -0000
> @@ -65,6 +65,13 @@ sr_install_bootblk(int devfd, int vol, i
> return;
> }
>
> + /* Key disk has size of zero */
> + if (bd.bd_size == 0) {
> + fprintf(stderr, "softraid chunk %u looks like key disk - "
> + "skipping...\n", disk);
> + return;
> + }
> +
> if (strlen(bd.bd_vendor) < 1)
> errx(1, "invalid disk name");
> part = bd.bd_vendor[strlen(bd.bd_vendor) - 1];
>
>
> Below follows my test and comments what happens without the diff
> and with the diff.
>
> First without the diff machine doesn't boot when I use keydisk on the
> same disk which has the OpenBSD operaring system, wd0a and wd0d:
You discovered it works half-way, I wouldn't be surprised if other
platforms/boot loaders behaved differently.
Pushing this keydisk-on-same-device design would only encourage users,
which is bad idea, imho.
It seems like stating the obvious, but maybe softraid(4) should contain
clear words like 'keeping the key next to the crypto volume is insecure'
or 'put the keydisk on separate device'.
>
> Booting from Hard Disk...
> Using drive 0, partition 3.
> Loading......
> ERR M
>
>
> To keep it short, it is because of installboot(8) installs boot blocks
> on both wd0a and wd0d:
>
> ramdisk# bioctl sd0
> Volume Status Size Device
> softraid0 0 Online 268426960384 sd0 CRYPTO
> 0 Online 268426960384 0:0.0 noencl <wd0d>
> 1 Online key disk 0:1.0 noencl <wd0a>
So CRYPTO on wd0d and keydisk on wd0a...
>
>
> ramdisk# installboot -r /mnt -nv sd0
> Using /mnt as root
> would install bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage
> /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: would install boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> wd0a: would install boot blocks on /dev/rwd0c, part offset 144
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
>
> We see above that sd0 is softraid volume with 2 disks and then boot
> blocks is installed on wd0d and wd0a. With my change boot blocks are
> only installed on wd0d and wd0a (keydisk) is skipped.
... and installboot writes to the keydisk, making it bootable.
Is this failing to boot for you because wd0a is tried first but fails
because it is a keydisk? Put differently, would your questionable
setup boot without a diff if CRYPTO was on wd0a and the keydisk on wd0d?
Seing installboot write to keydisks seems unexpected, I'd say it should
always ignore them.
Are keydisks expected to be present after unlocking the volume?
I'm picturing users having a USB stick they put in temporarily to unlock
and then boot/run the system without the stick plugged in.
But installboot seems to expect keydisks to be present, which is really
odd. It means you can't install bootstraps on your currently unlocked
crypto volume without having the keydisk available...
That does not sound right, but I'm not a keydisk user myself, so I have
to test/dig around a little.
>
>
> ramdisk# cd /mnt
> ramdisk# ftp https://example.com/installboot.bin
> Trying 10.123.123.123...
> Requesting https://example.com/installboot.bin
> 149040 bytes received in 0.21 seconds (671.19 KB/s)
> ramdisk# chmod 0755 /mnt/installboot.bin
> ramdisk# /mnt/installboot.bin -r /mnt -nv sd0
> Using /mnt as root
> would install bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: would install boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> softraid chunk 1 looks like key disk - skipping...
>
>
> Here is full installation log on OpenBSD -current when installaton is
> not bootable with above ERR M error.
>
>
> root on rd0a swap on rd0b dump on rd0b
> erase ^?, werase ^W, kill ^U, intr ^C, status ^T
>
> Welcome to the OpenBSD/amd64 7.2 installation program.
> (I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s
> # cd /dev
> # sysctl -n hw.disknames
> wd0:ccb72943e9945c9c,rd0:73efb2b48b898ab3
> # sh MAKEDEV wd0 wd1 wd2 wd3
> # sh MAKEDEV sd0 sd1 sd2 sd3
> # dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0c
> 10240+0 records in
> 10240+0 records out
> 10485760 bytes transferred in 5.606 secs (1870132 bytes/sec)
> # fdisk -iy wd0
> Writing MBR at offset 0.
> # disklabel -E wd0
> Label editor (enter '?' for help at any prompt)
> wd0> p g
> OpenBSD area: 64-524288000; size: 250.0G; free: 250.0G
> # size offset fstype [fsize bsize cpg]
> c: 250.0G 0 unused
> wd0> a a
> offset: [64]
> size: [524287936] 4m
> FS type: [4.2BSD] RAID
> wd0*> a d
> offset: [16065]
> size: [524271935] *
> FS type: [4.2BSD] RAID
> wd0*> w
> wd0> q
> No label changes.
> # dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0d
> 10240+0 records in
> 10240+0 records out
> 10485760 bytes transferred in 4.258 secs (2462425 bytes/sec)
> # sync
> # bioctl -c C -l wd0d -k wd0a softraid0
> sd0 at scsibus0 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006>
> sd0: 255991MB, 512 bytes/sector, 524271407 sectors
> softraid0: CRYPTO volume attached as sd0
> # dd if=/dev/zero bs=1024 count=10240 of=/dev/rsd0c
> 10240+0 records in
> 10240+0 records out
> 10485760 bytes transferred in 5.696 secs (1840830 bytes/sec)
> # fdisk -iy sd0
> Writing MBR at offset 0.
> # bioctl sd0
> Volume Status Size Device
> softraid0 0 Online 268426960384 sd0 CRYPTO
> 0 Online 268426960384 0:0.0 noencl <wd0d>
> 1 Online key disk 0:1.0 noencl <wd0a>
> # exit
> erase ^?, werase ^W, kill ^U, intr ^C, status ^T
>
> Welcome to the OpenBSD/amd64 7.2 installation program.
> (I)nstall, (U)pgrade, (A)utoinstall or (S)hell? i
> At any prompt except password prompts you can escape to a shell by
> typing '!'. Default answers are shown in []'s and are selected by
> pressing RETURN. You can exit this program at any time by pressing
> Control-C, but this can leave your system in an inconsistent state.
>
> Terminal type? [vt220]
> System hostname? (short form, e.g. 'foo') keydisk
>
> Available network interfaces are: re0 vlan0.
> Which network interface do you wish to configure? (or 'done') [re0]
> IPv4 address for re0? (or 'autoconf' or 'none') [autoconf]
> IPv6 address for re0? (or 'autoconf' or 'none') [none]
> Available network interfaces are: re0 vlan0.
> Which network interface do you wish to configure? (or 'done') [done]
> Using DNS domainname my.domain
> Using DNS nameservers at 192.168.122.1
>
> Password for root account? (will not echo)
> Password for root account? (again)
> Start sshd(8) by default? [yes]
> Do you expect to run the X Window System? [yes] no
> Change the default console to com0? [yes]
> Available speeds are: 9600 19200 38400 57600 115200.
> Which speed should com0 use? (or 'done') [115200]
> Setup a user? (enter a lower-case loginname, or 'no') [no]
> Since no user was setup, root logins via sshd(8) might be useful.
> WARNING: root is targeted by password guessing attacks, pubkeys are safer.
> Allow root ssh login? (yes, no, prohibit-password) [no] prohibit-password
> What timezone are you in? ('?' for list) [UTC]
>
> Available disks are: wd0 sd0.
> Which disk is the root disk? ('?' for details) [wd0] sd0
> Disk: sd0 geometry: 32634/255/63 [524271407 Sectors]
> Offset: 0 Signature: 0xAA55
> Starting Ending LBA Info:
> #: id C H S - C H S [ start: size ]
> -------------------------------------------------------------------------------
> 0: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
> 1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
> 2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
> *3: A6 0 1 2 - 32634 98 23 [ 64: 524271343 ] OpenBSD
> Use (W)hole disk MBR, whole disk (G)PT, (O)penBSD area or (E)dit? [OpenBSD] w
> Setting OpenBSD MBR partition to whole sd0...done.
> The auto-allocated layout for sd0 is:
> # size offset fstype [fsize bsize cpg]
> a: 1.0G 64 4.2BSD 2048 16384 1 # /
> b: 2.2G 2097216 swap
> c: 250.0G 0 unused
> d: 4.0G 6783040 4.2BSD 2048 16384 1 # /tmp
> e: 8.0G 15171616 4.2BSD 2048 16384 1 # /var
> f: 25.4G 31883264 4.2BSD 2048 16384 1 # /usr
> g: 1.0G 85133280 4.2BSD 2048 16384 1 # /usr/X11R6
> h: 20.0G 87230432 4.2BSD 2048 16384 1 # /usr/local
> i: 3.0G 129173472 4.2BSD 2048 16384 1 # /usr/src
> j: 6.0G 135464928 4.2BSD 2048 16384 1 # /usr/obj
> k: 179.4G 148047872 4.2BSD 4096 32768 1 # /home
> Use (A)uto layout, (E)dit auto layout, or create (C)ustom layout? [a] a
> /dev/rsd0a: 1024.0MB in 2097152 sectors of 512 bytes
> 6 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0k: 183702.9MB in 376223488 sectors of 512 bytes
> 226 cylinder groups of 814.44MB, 26062 blocks, 52224 inodes each
> /dev/rsd0d: 4096.0MB in 8388576 sectors of 512 bytes
> 21 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0f: 26001.0MB in 53250016 sectors of 512 bytes
> 129 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0g: 1024.0MB in 2097152 sectors of 512 bytes
> 6 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0h: 20480.0MB in 41943040 sectors of 512 bytes
> 102 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0j: 6144.0MB in 12582912 sectors of 512 bytes
> 31 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0i: 3072.0MB in 6291456 sectors of 512 bytes
> 16 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> /dev/rsd0e: 8160.0MB in 16711648 sectors of 512 bytes
> 41 cylinder groups of 202.50MB, 12960 blocks, 25920 inodes each
> Available disks are: wd0.
> Which disk do you wish to initialize? (or 'done') [done]
> /dev/sd0a (09b2208037fe8a1b.a) on /mnt type ffs (rw, asynchronous, local)
> /dev/sd0k (09b2208037fe8a1b.k) on /mnt/home type ffs (rw, asynchronous,
> local, nodev, nosuid)
> /dev/sd0d (09b2208037fe8a1b.d) on /mnt/tmp type ffs (rw, asynchronous, local,
> nodev, nosuid)
> /dev/sd0f (09b2208037fe8a1b.f) on /mnt/usr type ffs (rw, asynchronous, local,
> nodev)
> /dev/sd0g (09b2208037fe8a1b.g) on /mnt/usr/X11R6 type ffs (rw, asynchronous,
> local, nodev)
> /dev/sd0h (09b2208037fe8a1b.h) on /mnt/usr/local type ffs (rw, asynchronous,
> local, nodev)
> /dev/sd0j (09b2208037fe8a1b.j) on /mnt/usr/obj type ffs (rw, asynchronous,
> local, nodev, nosuid)
> /dev/sd0i (09b2208037fe8a1b.i) on /mnt/usr/src type ffs (rw, asynchronous,
> local, nodev, nosuid)
> /dev/sd0e (09b2208037fe8a1b.e) on /mnt/var type ffs (rw, asynchronous, local,
> nodev, nosuid)
>
> Let's install the sets!
> Location of sets? (disk http nfs or 'done') [http]
> HTTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none]
> HTTP Server? (hostname, list#, 'done' or '?') [cdn.openbsd.org]
> Server directory? [pub/OpenBSD/snapshots/amd64]
>
> Select sets by entering a set name, a file name pattern or 'all'. De-select
> sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'.
> [X] bsd [X] comp72.tgz [X] xbase72.tgz [X] xserv72.tgz
> [X] bsd.rd [X] man72.tgz [X] xshare72.tgz
> [X] base72.tgz [X] game72.tgz [X] xfont72.tgz
> Set name(s)? (or 'abort' or 'done') [done] all
> [X] bsd [X] comp72.tgz [X] xbase72.tgz [X] xserv72.tgz
> [X] bsd.rd [X] man72.tgz [X] xshare72.tgz
> [X] base72.tgz [X] game72.tgz [X] xfont72.tgz
> Set name(s)? (or 'abort' or 'done') [done]
> Get/Verify SHA256.sig 100% |**************************| 2144 00:00
> Signature Verified
> Get/Verify bsd 100% |**************************| 22447 KB 00:10
> Get/Verify bsd.rd 100% |**************************| 4532 KB 00:03
> Get/Verify base72.tgz 100% |**************************| 331 MB 02:07
> Get/Verify comp72.tgz 100% |**************************| 74595 KB 00:32
> Get/Verify man72.tgz 100% |**************************| 7607 KB 00:04
> Get/Verify game72.tgz 100% |**************************| 2745 KB 00:02
> Get/Verify xbase72.tgz 100% |**************************| 55180 KB 00:20
> Get/Verify xshare72.tgz 100% |**************************| 4559 KB 00:03
> Get/Verify xfont72.tgz 100% |**************************| 22967 KB 00:10
> Get/Verify xserv72.tgz 100% |**************************| 20119 KB 00:10
> Installing bsd 100% |**************************| 22447 KB 00:02
> Installing bsd.rd 100% |**************************| 4532 KB 00:00
> Installing base72.tgz 100% |**************************| 331 MB 01:21
> Extracting etc.tgz 100% |**************************| 257 KB 00:00
> Installing comp72.tgz 100% |**************************| 74595 KB 00:32
> Installing man72.tgz 100% |**************************| 7607 KB 00:03
> Installing game72.tgz 100% |**************************| 2745 KB 00:00
> Installing xbase72.tgz 100% |**************************| 55180 KB 00:20
> Extracting xetc.tgz 100% |**************************| 7238 00:00
> Installing xshare72.tgz 100% |**************************| 4559 KB 00:05
> Installing xfont72.tgz 100% |**************************| 22967 KB 00:04
> Installing xserv72.tgz 100% |**************************| 20119 KB 00:05
> Location of sets? (disk http nfs or 'done') [done]
> Saving configuration files... done.
> Making all device nodes... done.
> fw_update: added none; updated none; kept none
> Relinking to create unique kernel... done.
>
> CONGRATULATIONS! Your OpenBSD install has been successfully completed!
>
> When you login to your new system the first time, please read your mail
> using the 'mail' command.
>
> Exit to (S)hell, (H)alt or (R)eboot? [reboot] s
> To boot the new system, enter 'reboot' at the command prompt.
>
> keydisk# installboot -r /mnt -nv sd0
> Using /mnt as root
> would install bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: would install boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> wd0a: would install boot blocks on /dev/rwd0c, part offset 144
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> keydisk# reboot
> syncing disks... done
> rebooting...
>
>
> Press ESC for boot menu.
>
> Booting from Hard Disk...
> Using drive 0, partition 3.
> Loading......
> ERR M
>
>
>
> Above installation when I use modified installboot(8) and wd0a (keydisk
> partition) is skipped by installboot(8) and OS is booting properly:
>
>
> keydisk# installboot -r /mnt -nv sd0
> Using /mnt as root
> would install bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: would install boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> wd0a: would install boot blocks on /dev/rwd0c, part offset 144
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
>
>
> keydisk# cd /mnt
> keydisk# ftp https://example.com/installboot.bin
> Trying 10.123.123.123...
> Requesting https://example.com/installboot.bin
> 149040 bytes received in 0.21 seconds (671.19 KB/s)
> keydisk# chmod 0755 /mnt/installboot.bin
> keydisk# /mnt/installboot.bin -r /mnt -nv sd0
> Using /mnt as root
> would install bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: would install boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> softraid chunk 1 looks like key disk - skipping...
> keydisk# /mnt/installboot.bin -r /mnt -v sd0
> Using /mnt as root
> installing bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: installing boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: installing boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> softraid chunk 1 looks like key disk - skipping...
> keydisk# cd /
> keydisk# reboot
> syncing disks... done
> rebooting...
>
>
>
> Press ESC for boot menu.
>
> Booting from Hard Disk...
> Using drive 0, partition 3.
> Loading......
> probing: pc0 com0 mem[638K 2046M a20=on]
> disk: hd0+ sr0*
> >> OpenBSD/amd64 BOOT 3.55
> switching console to com0
> >> OpenBSD/amd64 BOOT 3.55
> boot>
> booting sr0a:/bsd: 15607064|
>
>
> As you can see above, OpenBSD starting to boot up.
>
>
> --
> Regards,
> Mikolaj
>
