Re: raidframe and gpt
hello. If you put a gpt label on a raid set and add partitions to it, those partitions will show up as dkn devices automatically, when the raid set is configured. This is true whether the raid set is configured automatically by the kernel at boot or manually via raidctl -C or -c. This has been true since NetBSD-5. The NAME= syntax in the fstab file works as well. -thanks -Brian
Re: raidframe and gpt
On 2024-03-17 12.04, B Harder wrote: On Mar 16, 2024, at 23:16, Greg Oster wrote: RAIDframe doesn't understand NAME=gpt-label. Use /dev/dk , and then set the RAID set to autoconfigure. It'll magically figure out what devices need to be glued together. Hey Greg, can you explain that magic in any level of detail? Sure... How about a somewhat lower level with a bit of hand-waving :) As the kernel is booting up, rf_autoconfig() (from rf_netbsdkintf.c) is set to run after all devices are initialized. rf_autoconfig() calls rf_find_raid_components(), which goes through all the devices on the system (first looking at wedges, and then at 'disks'), and identifying any partitions that are of type "RAID" (handwaving occurs here.) The component labels of all those RAID partitions are stuffed into a single list for later processing. One rf_find_raid_components() is complete, rf_create_auto_sets() is then called. This goes through the list of components created earlier, and sorts them into groups of components based on the component labels. The next step happens in rf_buildroothack(), which takes the set of sets of components. For each set of components, rf_have_enough_components() is called, and that determines whether or not there are sufficient components to bring up that RAID set (possibly in degraded mode). If there are sufficient components, then rf_auto_config_set() is called with that set of components, and if all is still well, then that particular RAID set is configured (to the RAID device on which it was last configured, if that device is available). So in a nutshell, the component labels (preferably with a different serial number for ease of differentiation) are used to figure out which components belong together. RAIDframe doesn't care if they've moved to a different SCSI or SATA or dk device -- it only cares that the labels are consistent and that it can find enough components to get the RAID set operational. Hope that's a useful level of detail. :) Later... Greg Oster One more quuestion: the raidctl man page talks about partitioning the raid device using mbr partitions. Is it possible to use GPT here? Will the resulting wedges show up automatically? You can use GPT instead of mbr. Wedges should show up automatically (I'm pretty sure they do, but since I do NFS exporting of mine, I stuck with MBR so the filesystem ID doesn't change every time the n in dk for the RAID deivce changes :( ) It seems so much simpler to use ccd(4) but there's a nasty memory allocation bug which makes it unuseable for now. Thanks in advance! If you're using -current, note that you can now use: raidctl raid0 create mirror /dev/dk0 /dev/dk1 to do a simple RAID config without needing to use the config file... Later... Greg Oster
Re: raidframe and gpt
> On Mar 16, 2024, at 23:16, Greg Oster wrote: > > > >> On 2024-03-16 21.58, Paul Goyette wrote: >>> On Sat, 16 Mar 2024, Paul Goyette wrote: >>> Does anyone have an example of how to configure raid0 on a GPT disk? >>> >>> I can easily set the partition type with gpt, but how do I reserve >>> space for the raid component label? Do I need to reserve that space? > > You don't need to reserve the space. RAIDframe takes care of that. So you > just set the partition type appropriately, and tell RAIDframe about the > partition... > >>> Also, does raidframe understand the NAME=gpt-label syntax in the >>> config file? Or does it require me to specify the particular dk ? >>> (And what happens if something moves and changes?) > > RAIDframe doesn't understand NAME=gpt-label. Use /dev/dk , and then set > the RAID set to autoconfigure. It'll magically figure out what devices need > to be glued together. Hey Greg, can you explain that magic in any level of detail? -bch > >> One more quuestion: the raidctl man page talks about partitioning the >> raid device using mbr partitions. Is it possible to use GPT here? >> Will the resulting wedges show up automatically? > > You can use GPT instead of mbr. Wedges should show up automatically (I'm > pretty sure they do, but since I do NFS exporting of mine, I stuck with MBR > so the filesystem ID doesn't change every time the n in dk for the RAID > deivce changes :( ) > >>> It seems so much simpler to use ccd(4) but there's a nasty memory >>> allocation bug which makes it unuseable for now. >>> >>> Thanks in advance! > > If you're using -current, note that you can now use: > > raidctl raid0 create mirror /dev/dk0 /dev/dk1 > > to do a simple RAID config without needing to use the config file... > > Later... > > Greg Oster
Re: raidframe and gpt
On Sat, Mar 16, 2024 at 10:58:18PM -0600, Greg Oster wrote: > On 2024-03-16 21.58, Paul Goyette wrote: > > On Sat, 16 Mar 2024, Paul Goyette wrote: > > You can use GPT instead of mbr. Wedges should show up automatically (I'm > pretty sure they do, but since I do NFS exporting of mine, I stuck with MBR > so the filesystem ID doesn't change every time the n in dk for the RAID > deivce changes :( ) They do show up automatically, with their gpt name label, and can be mounted with that name in /etc/fstab - eg pts/3 dogcow@kuroi: /usr/obj 3124 % gpt show -l raid1 |grep part 64 19532798976 1 GPT part - 10taux pts/3 dogcow@kuroi: /usr/obj 3130 % dmesg |grep 10taux [ 3.990347] dk10 at raid1: "10taux", 19532798976 blocks at 64, type: ffs pts/3 dogcow@kuroi: /usr/obj 3125 % grep 10taux /etc/fstab NAME=10taux /archive ffsrw,log,noatime 1 2 pts/3 dogcow@kuroi: /usr/obj 3129 % mount | grep /archive /dev/dk10 on /archive type ffs (log, noatime, local) I haven't tried booting off of a raidframe partition, though.
Re: raidframe and gpt
On 2024-03-16 21.58, Paul Goyette wrote: On Sat, 16 Mar 2024, Paul Goyette wrote: Does anyone have an example of how to configure raid0 on a GPT disk? I can easily set the partition type with gpt, but how do I reserve space for the raid component label? Do I need to reserve that space? You don't need to reserve the space. RAIDframe takes care of that. So you just set the partition type appropriately, and tell RAIDframe about the partition... Also, does raidframe understand the NAME=gpt-label syntax in the config file? Or does it require me to specify the particular dk ? (And what happens if something moves and changes?) RAIDframe doesn't understand NAME=gpt-label. Use /dev/dk , and then set the RAID set to autoconfigure. It'll magically figure out what devices need to be glued together. One more quuestion: the raidctl man page talks about partitioning the raid device using mbr partitions. Is it possible to use GPT here? Will the resulting wedges show up automatically? You can use GPT instead of mbr. Wedges should show up automatically (I'm pretty sure they do, but since I do NFS exporting of mine, I stuck with MBR so the filesystem ID doesn't change every time the n in dk for the RAID deivce changes :( ) It seems so much simpler to use ccd(4) but there's a nasty memory allocation bug which makes it unuseable for now. Thanks in advance! If you're using -current, note that you can now use: raidctl raid0 create mirror /dev/dk0 /dev/dk1 to do a simple RAID config without needing to use the config file... Later... Greg Oster
Re: raidframe and gpt
p...@whooppee.com (Paul Goyette) writes: >> Does anyone have an example of how to configure raid0 on a GPT disk? For a manual setup, you can just reference the wedges like: # raidctl config file for /dev/rraid0 START array # numRow numCol numSpare 1 2 0 START disks NAME=raid0.0 NAME=raid0.1 START layout # sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_1 128 1 1 1 START queue fifo 100 Auto detection of a RAID works by collecting disks with valid RAID labels into RAID sets. This also works with wedges, the actual device or unit number is irrelevant, the RAID set is identified by the unique serial number in the RAID label. The serial number is specified with the -I option of raidctl. Even booting from a RAID on GPT is possible with a recent bootloader and an autoconfigured RAID set. Here is more about RAID on GPT: https://wiki.netbsd.org/users/spz/moderndisk/ >> I can easily set the partition type with gpt, but how do I reserve >> space for the raid component label? Do I need to reserve that space? You don't, the components are the GPT partitions. >> Also, does raidframe understand the NAME=gpt-label syntax in the >> config file? Or does it require me to specify the particular dk ? >> (And what happens if something moves and changes?) I've added support for the NAME= Syntax some time ago. >One more quuestion: the raidctl man page talks about partitioning the >raid device using mbr partitions. Is it possible to use GPT here? >Will the resulting wedges show up automatically? Whenever you create a raid device (raidN attaches), it will be scanned for wedges like a regular disk device. # dkctl dk4 getwedgeinfo dk4 at raid2: tank dk4: 4294967296 blocks at 128, type: ffs # grep tank /etc/fstab NAME=tank /tank ffs rw,log 1 2 This system configures a RAID1 of two RAID0 sets. That's not supported by the RAID autoconfiguration, so it is created manually at boot time and obviously the system can't boot from it.
re: raidframe and gpt
Paul Goyette writes: > Does anyone have an example of how to configure raid0 on a GPT disk? these are my notes i refer to every so often: https://www.netbsd.org/~mrg/gpt-raid-setup.txt it's gpt on each with type raid, which gives you dkN @ diskN, you then create a raid with those dkNs, and then you create another gpt on the raid device itself, with a ffs partition. (see below; but skip the raidN.conf method, and just use the newer raidctl create.) > I can easily set the partition type with gpt, but how do I reserve > space for the raid component label? Do I need to reserve that space? note how i pick "-b 128" above to get my partitions aligned on at least 64K bounaries. nvme/sata probably wants higher (check your disk specs, it can vary a lot, and you could go as high as 6MB alignment to catch all known alignment...) > Also, does raidframe understand the NAME=gpt-label syntax in the > config file? Or does it require me to specify the particular dk ? > (And what happens if something moves and changes?) NAME= works. use autoconfig raid.. actually just use the new in -current "raidctl create", since it does all the intro set and good default choices. > It seems so much simpler to use ccd(4) but there's a nasty memory > allocation bug which makes it unuseable for now. you can't root-on-ccd like you can root-on-raidframe :-) you could, using the same initrd method root-on-cgd uses. .mrg.
Re: raidframe and gpt
On Sat, 16 Mar 2024, Paul Goyette wrote: Does anyone have an example of how to configure raid0 on a GPT disk? I can easily set the partition type with gpt, but how do I reserve space for the raid component label? Do I need to reserve that space? Also, does raidframe understand the NAME=gpt-label syntax in the config file? Or does it require me to specify the particular dk ? (And what happens if something moves and changes?) One more quuestion: the raidctl man page talks about partitioning the raid device using mbr partitions. Is it possible to use GPT here? Will the resulting wedges show up automatically? It seems so much simpler to use ccd(4) but there's a nasty memory allocation bug which makes it unuseable for now. Thanks in advance! +-+--+--+ | Paul Goyette (.sig) | PGP Key fingerprint: | E-mail addresses:| | (Retired) | 1B11 1849 721C 56C8 F63A | p...@whooppee.com| | Software Developer | 6E2E 05FD 15CE 9F2D 5102 | pgoye...@netbsd.org | | & Network Engineer | | pgoyett...@gmail.com | +-+--+--+ !DSPAM:65f65ac4122352312412431! +-+--+--+ | Paul Goyette (.sig) | PGP Key fingerprint: | E-mail addresses:| | (Retired) | 1B11 1849 721C 56C8 F63A | p...@whooppee.com| | Software Developer | 6E2E 05FD 15CE 9F2D 5102 | pgoye...@netbsd.org | | & Network Engineer | | pgoyett...@gmail.com | +-+--+--+
raidframe and gpt
Does anyone have an example of how to configure raid0 on a GPT disk? I can easily set the partition type with gpt, but how do I reserve space for the raid component label? Do I need to reserve that space? Also, does raidframe understand the NAME=gpt-label syntax in the config file? Or does it require me to specify the particular dk ? (And what happens if something moves and changes?) It seems so much simpler to use ccd(4) but there's a nasty memory allocation bug which makes it unuseable for now. Thanks in advance! +-+--+--+ | Paul Goyette (.sig) | PGP Key fingerprint: | E-mail addresses:| | (Retired) | 1B11 1849 721C 56C8 F63A | p...@whooppee.com| | Software Developer | 6E2E 05FD 15CE 9F2D 5102 | pgoye...@netbsd.org | | & Network Engineer | | pgoyett...@gmail.com | +-+--+--+
Re: NetBSD 9_RC1 not booting from disklabel partition on RAIDframe on GPT
Hello Chavdar, thanks for looking at this issue. In the meantime Emmanuel Dreyfus contacted me and did prepare a patch which solves the issue for me. It is attached to the PR misc/54748 [1]. The patch is committed to HEAD, so newer 9.99.x daily builds should already contain it. It would be good if it receives some more testing so hopefully it finds its way to 9.0. Kind regards Matthias [1] http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=54748 Am 06.12.2019 um 13:13 schrieb Chavdar Ivanov: I haven't done raidframe installations for a long time, but followed your sequence using two days old amd64-current -9.99.19- and it failed the same way for me. It was under VirtualBox, though. Chavdar smime.p7s Description: S/MIME Cryptographic Signature
Re: NetBSD 9_RC1 not booting from disklabel partition on RAIDframe on GPT
I haven't done raidframe installations for a long time, but followed your sequence using two days old amd64-current -9.99.19- and it failed the same way for me. It was under VirtualBox, though. Chavdar On Fri, 6 Dec 2019 at 01:39, Matthias Petermann wrote: > > Hello, > > there seems to be a deviation of the boot behavior in NetBSD 9_RC1 > compared to NetBSD 8.1 under the following circumstances: > > 1) system has 2 disks (wd0, wd1) > 2) disks are GPT-partitioned > 3) GPT contains one GPT-partition (RAIDframe component) each > 4) RAIDframe raid0 contains a NetBSD disklabel > > After installing NetBSD 9_RC1 it will not boot, instead the primary > bootstrap is telling me: > > Unexpected raidframe label version > NAME=sys0 not found > > The closest I could find is this commit: > > https://www.mail-archive.com/source-changes-d@netbsd.org/msg21099.html > > where GPT and RAIDframe support for the boot loaders was enhanced (which > is really good!). > > Here is where I set up the system: > > * Boot from USB installation media > * Select keyboard type > * Utility Menu - Run /bin/sh > > # ksh > > # gpt create wd0 > # gpt create wd1 > > # gpt add -l sys0 -a 2m -s 96G -t raid wd0 > # gpt add -l sys1 -a 2m -s 96G -t raid wd1 > > # gpt show -l wd0 > # gpt show -l wd1 > > # dkctl wd0 listwedges > # dkctl wd1 listwedges > > # gpt biosboot -A -i 1 wd0 > # gpt biosboot -A -i 1 wd1 > > # vi /tmp/raid0.conf > > START array > 1 2 0 > > START disks > /dev/dk0 > absent > > START layout > 128 1 1 1 > > START queue > fifo 100 > > # raidctl -C /tmp/raid0.conf raid0 > # raidctl -I 201912500 raid0 > # raidctl -i raid0 > # raidctl -A softroot raid0 > > * Exit shell and continue with installer > > * Install NetBSD to hard disk > * Select raid0 > * Select disklabel as partitioning scheme for raid0 > * Set sizes of NetBSD Partitions > -> Change input units (MB) > * root 16384 > * swap 16384 > * usr 32768+ > * var 8192 > * Bootblock: BIOS console > * Installation w/out X11 > > * Utility Menu - Run /bin/sh > > # installboot -o timeout=30 -v -t raid /dev/rdk0 /usr/mdec/bootxx_ffsv2 > > * Reboot > > > Kind regards > Matthias --
