Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 7:36 PM Grant Taylor wrote: > > That assumes that there is a boot loader. There wasn't one with the old > Slackware boot & root disks. > Linux no longer supports direct booting from the MBR. arch/x86/boot/header.S bugger_off_msg: .ascii "Use a boot loader.\r\n" .ascii "\n" .ascii "Remove disk and press any key to reboot...\r\n" .byte 0 -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
Peter Humphrey: ... > In my case I > haven't needed an initramfs so far, and now I see I still don't need one - > why > add complication? Having set the kernel option to assemble raid devices at > boot time, now that /dev/md0 has been created I find it ready to go as soon > as > I boot up and log in. No jiggery-pokery needed. > > A reminder: this is not the boot device. Works on a boot device as long as you use old 0.90 format superblock on your md devices. You can boot with autodetect or specify how to assemble root raid fs on command line like: # cat /proc/cmdline BOOT_IMAGE=18/3 ro root=902 md=2,/dev/sda2,/dev/sdb2 raid=noautodetect # ls -l /dev/md2 brw-rw 1 root disk 9, 2 Apr 12 2015 /dev/md2 # Regards, /Karl Hammar --- Aspö Data Lilla Aspö 148 S-742 94 Östhammar Sweden
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 02:17 PM, Neil Bothwick wrote: AFAIR the initramfs code is built into the kernel, not as an option. The reason given for using a cpio archive is that it is simple and available in the kernel. The kernel itself has an initramfs built into it which is executed automatically, it's just that this initramfs is usually empty. So loading an initramfs is trivial for any kernel, and loading anything after that is handled by the initramfs. That may be the case now. But when I started messing with Linux nearly 20 years ago that was not the case. The kernel and the initramfs / initrd were two distinct things. I remember having to calculate where the kernel stopped on a floppy disk so that you could start writing the initramfs / initrd image after the kernel. Or for fun, modify the flag (bit?) to tell the kernel to prompt to to swap disks for the initramfs / initrd. Both of which needed to tell the kernel where the initramfs / initrd started on the medium. That was a LONG time ago. More than a few things have changed since then. That only leaves loading the initramfs file from disk, which is handled by the bootloader along with the kernel file. That assumes that there is a boot loader. There wasn't one with the old Slackware boot & root disks. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tuesday, 29 January 2019 20:37:31 GMT Wol's lists wrote: > On 28/01/2019 16:56, Peter Humphrey wrote: > > I must be missing something, in spite of following the wiki instructions. > > Can someone help an old duffer out? > > Gentoo wiki, or kernel raid wiki? Gentoo wiki. It's fascinating to see what a hornets' nest I've stirred up. In my case I haven't needed an initramfs so far, and now I see I still don't need one - why add complication? Having set the kernel option to assemble raid devices at boot time, now that /dev/md0 has been created I find it ready to go as soon as I boot up and log in. No jiggery-pokery needed. A reminder: this is not the boot device. -- Regards, Peter.
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, 29 Jan 2019 13:37:43 -0700, Grant Taylor wrote: > > An initramfs typically loads kernel modules, assuming there are any > > that need to be loaded. > > And where is it going to load them from if said kernel doesn't support > initrds or loop back devices or the archive or file system type that > the initramfs is using? AFAIR the initramfs code is built into the kernel, not as an option. The reason given for using a cpio archive is that it is simple and available in the kernel. The kernel itself has an initramfs built into it which is executed automatically, it's just that this initramfs is usually empty. So loading an initramfs is trivial for any kernel, and loading anything after that is handled by the initramfs. That only leaves loading the initramfs file from disk, which is handled by the bootloader along with the kernel file. -- Neil Bothwick TEXAS VIRUS: Makes sure that it's bigger than any other file. pgpjYJRwaLpQH.pgp Description: OpenPGP digital signature
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 20:58:37 +, Wol's lists wrote: > On 29/01/2019 19:41, Grant Taylor wrote: > > The kernel /must/ have (at least) the minimum drivers (and dependencies) > > to be able to boot strap. It doesn't matter if it's boot strapping an > > initramfs or otherwise. > > All of these issues about lack of a driver are avoided by having the > > driver statically compiled into the kernel. > I'm not sure to what extent it's true of 64-bit hardware, but one of the > big problems with non-module kernels is actually being able to load them > into the available ram ... something to do with BG's "640K should be > enough for anyone". Uh? I've never had problems with my hand-configured kernels fitting into RAM, regardless of whether it's a 32-bit or 64-bit processor. A modular kernel is a workaround for binary kernels needing to support any and all hardware devices. If you load drivers for all these devices at the same time, you may well run out of RAM (or have done so in the relatively recent past). If you're configuring the kernel for a specific machine, I can't see how you could run out of RAM, unless there's too little of it to run GNU/Linux anyway. > Cheers, > Wol -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] RAID-1 on secondary disks how?
On 29/01/2019 19:41, Grant Taylor wrote: The kernel /must/ have (at least) the minimum drivers (and dependencies) to be able to boot strap. It doesn't matter if it's boot strapping an initramfs or otherwise. All of these issues about lack of a driver are avoided by having the driver statically compiled into the kernel. I'm not sure to what extent it's true of 64-bit hardware, but one of the big problems with non-module kernels is actually being able to load them into the available ram ... something to do with BG's "640K should be enough for anyone". Cheers, Wol
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 3:37 PM Grant Taylor wrote: > > On 01/29/2019 01:26 PM, Rich Freeman wrote: > > Uh, an initramfs typically does not exec a second kernel. I guess it > > could, in which case that kernel would need its own initramfs to get > > around to mounting its root filesystem. Presumably at some point you'd > > want to have your system stop kexecing kernels and start actually doing > > something useful... > > Which ever type of initramfs you use, the kernel that you are running > MUST have support for the minimum number of devices and file systems it > needs to be able to load other things. Certainly, which is what I've been saying. And what you've been saying as well. > Hence the difference between > built-in and modular drivers that I'm talking about. The kernel doesn't care where the driver came from. If you want to put root on ext4 then the kernel needs ext4 support. It doesn't matter if it is built-into the kernel or in a module stored in the initramfs. > And where is it going to load them from if said kernel doesn't support > initrds or loop back devices or the archive or file system type that the > initramfs is using? Why would you use an initramfs with a kernel incapable of using an initramfs? > > > Sure, and those are in the kernel that runs the initramfs. > > Not if they aren't compiled in. > Sure, in that case they would be in modules. I don't really get your point here. > I feel like this (sub)thread has become circular and unproductive. Clearly... -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 01:26 PM, Rich Freeman wrote: Uh, an initramfs typically does not exec a second kernel. I guess it could, in which case that kernel would need its own initramfs to get around to mounting its root filesystem. Presumably at some point you'd want to have your system stop kexecing kernels and start actually doing something useful... Which ever type of initramfs you use, the kernel that you are running MUST have support for the minimum number of devices and file systems it needs to be able to load other things. Hence the difference between built-in and modular drivers that I'm talking about. An initramfs typically loads kernel modules, assuming there are any that need to be loaded. And where is it going to load them from if said kernel doesn't support initrds or loop back devices or the archive or file system type that the initramfs is using? Sure, and those are in the kernel that runs the initramfs. Not if they aren't compiled in. I feel like this (sub)thread has become circular and unproductive. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On 28/01/2019 16:56, Peter Humphrey wrote: I must be missing something, in spite of following the wiki instructions. Can someone help an old duffer out? Gentoo wiki, or kernel raid wiki? Cheers, Wol
Re: [gentoo-user] RAID-1 on secondary disks how?
On 29/01/2019 19:01, Rich Freeman wrote: It would surely be a bug if the kernel were capable of manipulating RAIDs, but not of initialising and mounting them. Linus would disagree with you there, and has said as much publicly. He does not consider initialization to be the responsibility of kernel space long-term, and prefers that this happen in user-space. Some of the lvm and mdadm support remains for legacy reasons, but you probably won't see initialization of newer volume/etc managers supported directly in the kernel. Actually, the kernel isn't capable of manipulating raid. The reason you need raid 0.9 (or 1.0, actually) is that all the raid metadata is at the *end* of the partition, so the system boots off the file-system completely ignorant that it's actually a raid. The raid then gets assembled and when root is remounted rw, it's the raid version that's mounted not a single-disk filesystem. In other words, if you have sda1 and sdb1 raided together as your root, the system will boot off a read-only sda1 before switching to a read-write md1. Cheers, Wol
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 3:15 PM Grant Taylor wrote: > > On 01/29/2019 01:08 PM, Rich Freeman wrote: > > You seem to be focusing on the second kernel that the initramfs execs. > Uh, an initramfs typically does not exec a second kernel. I guess it could, in which case that kernel would need its own initramfs to get around to mounting its root filesystem. Presumably at some point you'd want to have your system stop kexecing kernels and start actually doing something useful... If an initramfs did kexec a second kernel then that initramfs would basically be wiped out along with anything the first kernel did. Unless you're talking about something like Xen a linux kernel generally takes complete control over the system. An initramfs typically loads kernel modules, assuming there are any that need to be loaded. They're loaded by the kernel that was run by grub, and they stay around after the new root/init is pivoted. > The initramfs won't be able to do crap if it doesn't have the device and > file system drives necessary for the initramfs kernel & init scripts to > boot. Sure, and those are in the kernel that runs the initramfs. Remember, it is the kernel that runs the initramfs, not the other way around, though the initramfs might modprobe some modules just as you might do 5 minutes after booting. If those drivers are already built-in to the kernel then there is no need to modprobe them. -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 01:08 PM, Rich Freeman wrote: Obviously. Hence the reason I said that it shouldn't matter if the module is built in-kernel. I'm saying it does matter. I'm not sure why it seems like we're talking past each other here... You seem to be focusing on the second kernel that the initramfs execs. I'm talking about the first kernel that the initramfs uses. The initramfs won't be able to do crap if it doesn't have the device and file system drives necessary for the initramfs kernel & init scripts to boot. Now, it might not support a kernel that doesn't support module loading at all, though I'm not sure why not. If it doesn't I could see why the developers wouldn't be bothered to address the use case. Yet another possible reason to dislike dracut. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 2:59 PM Grant Taylor wrote: > > On 01/29/2019 12:47 PM, Rich Freeman wrote: > > It couldn't. Hence the reason I said, "obviously it needs whatever > > drivers it needs, but I don't see why it would care if they are built > > -in-kernel vs in-module." > > You are missing what I'm saying. > > Even the kernel the initramfs uses MUST have support for the file > systems and devices that the initramfs uses. You can't load the module > if you can't get to where the module is or have a place to write it to > load it. Obviously. Hence the reason I said that it shouldn't matter if the module is built in-kernel. If your root is on ext4 then the kernel needs an ext4 driver. That could be built-in, or in a module present in an initramfs, and an intramfs could support either. Dracut definitely supports either config. Dracut can mount btrfs just fine if btrfs is built in-kernel and not as a module. I'm not sure why it seems like we're talking past each other here... Now, it might not support a kernel that doesn't support module loading at all, though I'm not sure why not. If it doesn't I could see why the developers wouldn't be bothered to address the use case. -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 2:52 PM Grant Taylor wrote: > > On 01/29/2019 12:33 PM, Rich Freeman wrote: > > > However, as soon as you throw so much as a second hard drive in a system > > that becomes unreliable. > > Mounting the root based on UUID (or labels) is *WONDERFUL*. It makes > the system MUCH MORE resilient. Even if a device somehow gets inserted > before your root device in the /dev/sd* order. Interesting. I didn't realize that linux supported PARTUUID natively. I'll agree that addresses many more use cases. I was under the impression that it required an initramfs - maybe that is a recent change... > > > I'm not saying you can't use linux without an initramfs. I'm just > > questioning why most normal people would want to. I bet that 98% of > > people who use Linux run an initramfs, and there is a reason for that... > > I don't doubt your numbers. > > But I do question the viability of them. How many people that are > running Linux even know that they have an option? Most of them probably don't even realize they're running Linux. :) People design systems with an initramfs because it is more robust and covers more use cases, including the use cases that don't require an initramfs. It also allows a fully modular kernel, which means less RAM use/etc on distros that use one-size-fits-all kernels (which is basically all of them but Gentoo). -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 12:47 PM, Rich Freeman wrote: It couldn't. Hence the reason I said, "obviously it needs whatever drivers it needs, but I don't see why it would care if they are built -in-kernel vs in-module." You are missing what I'm saying. Even the kernel the initramfs uses MUST have support for the file systems and devices that the initramfs uses. You can't load the module if you can't get to where the module is or have a place to write it to load it. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On 29/01/2019 16:48, Alan Mackenzie wrote: Hello, All. On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote: On 01/29/2019 09:08 AM, Peter Humphrey wrote: I'd rather not have to create an initramfs if I can avoid it. Would it be sensible to start the raid volume by putting an mdadm --assemble command into, say, /etc/local.d/raid.start? The machine doesn't boot from /dev/md0. Drive by comment. I thought there was a kernel option / command line parameter that enabled the kernel to automatically assemble arrays as it's initializing. Would something like that work for you? I have no idea where that is in the context of what you're working on. I use mdadm with a RAID-1 pair of SSDs, without an initramfs (YUCK!). My root partition is on the RAID. For this, the kernel needs to be able to assemble the drives into the raid at booting up time, and for that you need version 0.90 metadata. (Or, at least, you did back in 2017.) You still do. 0.9 is deprecated and bit-rotting. If it breaks, nobody is going to fix it!!! My command for building my array was: # mdadm --create /dev/md2 --level=1 --raid-devices=2 \ --metadata=0.90 /dev/nvme0n1p2 /dev/nvme1n1p2. However, there's another quirk which bit me: something in the Gentoo installation disk took it upon itself to renumber my /dev/md2 to /dev/md127. I raised bug #539162 for this, but it was decided not to fix it. (This was back in February 2015.) This is nothing to do with gentoo - it's mdadm. And like with sdX for drives, it's explicitly not guaranteed that the number remains consistent. You're supposed to use names now, eg /dev/md/root, or /dev/md/home for example. Cheers, Wol
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 12:33 PM, Rich Freeman wrote: If all my boxes could function reliably without an initramfs I probably would do it that way. ;-) However, as soon as you throw so much as a second hard drive in a system that becomes unreliable. I disagree. I've been reliably booting and running systems with multiple drives for almost two decades. Including various combinations of PATA, SATA, SCSI, USB, SAN, drives. Mounting the root based on UUID (or labels) is *WONDERFUL*. It makes the system MUCH MORE resilient. Even if a device somehow gets inserted before your root device in the /dev/sd* order. I'm not saying you can't use linux without an initramfs. I'm just questioning why most normal people would want to. I bet that 98% of people who use Linux run an initramfs, and there is a reason for that... I don't doubt your numbers. But I do question the viability of them. How many people that are running Linux even know that they have an option? I suspect the answer to that question is less extreme than you are wanting. I also suspect that it's more in your favor than I want. But 70 / 30 (I pulled those from you know where) is significantly different than 98 / 2. A lot of that is situational. If you have a kernel without btrfs support, and you build btrfs as a module and switch your root filesystem to btrfs, then obviously you'll need to rebuild your initramfs since the one you have can't do btrfs. But, most people would just rebuild their initramfs anytime they rebuild a kernel just to be safe. If you added btrfs support to the kernel (built-in) then it is more of a toss-up, though in the case of btrfs specifically you might still need to regenerate the initramfs to add the btrfs userspace tools to it if you didn't already have them in /usr when you generated it the first time. But, if you're running btrfs you're probably forced to use an initramfs in any case. That's one of many reasons that I'm not using btrfs. In any case, it isn't some kind of automatic thing. Just as some things require rebuilding a kernel, some things require rebuilding an initramfs. I just find it simplest to build an initramfs anytime I build a kernel, and use the make install naming convention so that grub-mkconfig just does its thing automatically. Simplicity is completely independent of necessity. You have made a choice. That's your choice to make. IMO Dracut is one of the most robust solutions for these sorts of situations. It is highly modular, easy to extend, and it really tries hard to respect your existing config in /etc. In fact, not only does it put a copy of fstab in the initramfs to help it find your root, but after it mounts the root it checks that version of fstab to see if it is different and then remounts things accordingly. I find it simpler and more robust to remove the initramfs complexity if I have no /need/ for it. If you haven't guessed I'm a bit of a Dracut fan. :) And I'm a fan of simplicity. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 2:41 PM Grant Taylor wrote: > > On 01/29/2019 12:01 PM, Rich Freeman wrote: > > > > That is news to me. Obviously it needs whatever drivers it needs, but > > I don't see why it would care if they are built in-kernel vs in-module. > > How is a kernel going to be able to mount the root file system if it > doesn't have the driver (and can't load) for said root file system type? It couldn't. Hence the reason I said, "obviously it needs whatever drivers it needs, but I don't see why it would care if they are built -in-kernel vs in-module." -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 12:01 PM, Rich Freeman wrote: Not sure why you would think this. It is just a cpio archive of a root filesystem that the kernel runs as a generic bootstrap. IMHO the simple fact that such is used when it is not needed is ugly part. This means that your bootstrap for initializing your root and everything else can use any userspace tool that exists for linux. Why would I want to do that if I don't /need/ to do that? I can use IPs on VXLAN VTEPs to communicate between two hosts in the same L2 broadcast domain too. But why would I want to do that when the simple IP address on the Ethernet interface will suffice? A similar concept lies at the heart of coreboot - using a generic kernel/userpace as a firmware bootloader making it far more flexible. Coreboot is not part of the operating system. If you want to talk about the kernel in coreboot taking over the kernel's job and removing the boot loader + (2nd) kernel, I'm interested in discussing. An initramfs is basically just a fairly compact linux distro. It works the same as any distro. IP over the VXLAN VTEP works the same as IP over Ethernet too. The simple fact that there are two distros (kernel & init scripts) that run in succession when there is no /need/ for them is the ugly bit. Why stop at two distros? Why not three or four or more? The kernel runs init, and init does its thing. By convention that init will mount the real root and then exec the init inside, but it doesn't have to work that way. Heck, you can run a system with nothing but an initramfs and no other root filesystem. You can also run a system in the halt run level with everything mounted read-only. I used to run firewalls this way. Makes them really hard to modify without rebooting and altering how they boot. }:-) Linus would disagree with you there, and has said as much publicly. He does not consider initialization to be the responsibility of kernel space long-term, and prefers that this happen in user-space. ~chuckle~ That wouldn't be the first or last time that Linus disagreed with someone. Some of the lvm and mdadm support remains for legacy reasons, but you probably won't see initialization of newer volume/etc managers supported directly in the kernel. That is news to me. Obviously it needs whatever drivers it needs, but I don't see why it would care if they are built in-kernel vs in-module. O.o‽ How is a kernel going to be able to mount the root file system if it doesn't have the driver (and can't load) for said root file system type? Or how is it going to extract the initramfs if it doesn't have the driver for a ram disk? The kernel /must/ have (at least) the minimum drivers (and dependencies) to be able to boot strap. It doesn't matter if it's boot strapping an initramfs or otherwise. All of these issues about lack of a driver are avoided by having the driver statically compiled into the kernel. IMHO a kernel and a machine is quite a bit more secure if it doesn't support modules. Almost all of the root kits that I've seen require modules. So the root kits are SIGNIFICANTLY impeded if not completely broken if the kernel doesn't support modules. Sure, if you want to know exactly how it works that is true, but the same is true of openrc or any other piece of software on your system. Yep. I think it's a LOT easier to administer a system that I understand how and why it works. Dracut is highly modular and phase/hook-driven so it isn't too hard to grok. Most of it is just bash/dash scripts. That may be the case. But why even spend time with it if it's not actually /needed/. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 2:22 PM Grant Taylor wrote: > > On 01/29/2019 12:04 PM, Rich Freeman wrote: > > I don't see the value in using a different configuration on a box simply > > because it happens to work on that particular box. Dracut is a more > > generic solution that allows me to keep hosts the same. > > And if all the boxes in the fleet can function without an initramfs? > Then why have it? Why not apply Occam's Razor & Parsimony and use the > simpler solution. Especially if more complex solutions introduce > additional things that need to be updated. If all my boxes could function reliably without an initramfs I probably would do it that way. However, as soon as you throw so much as a second hard drive in a system that becomes unreliable. I'm not saying you can't use linux without an initramfs. I'm just questioning why most normal people would want to. I bet that 98% of people who use Linux run an initramfs, and there is a reason for that... > > Sure, and I wouldn't expect them to require rebuilding your initramfs > > either. I was speaking generally. > > Modifying things like crypttab and / or adding / removing file systems > from the kernel that are required for boot have caused me to need to > rebuild an initramfs in the past. But that was not necessarily Gentoo, > so it may not be a fair comparison. A lot of that is situational. If you have a kernel without btrfs support, and you build btrfs as a module and switch your root filesystem to btrfs, then obviously you'll need to rebuild your initramfs since the one you have can't do btrfs. But, most people would just rebuild their initramfs anytime they rebuild a kernel just to be safe. If you added btrfs support to the kernel (built-in) then it is more of a toss-up, though in the case of btrfs specifically you might still need to regenerate the initramfs to add the btrfs userspace tools to it if you didn't already have them in /usr when you generated it the first time. But, if you're running btrfs you're probably forced to use an initramfs in any case. In any case, it isn't some kind of automatic thing. Just as some things require rebuilding a kernel, some things require rebuilding an initramfs. I just find it simplest to build an initramfs anytime I build a kernel, and use the make install naming convention so that grub-mkconfig just does its thing automatically. IMO Dracut is one of the most robust solutions for these sorts of situations. It is highly modular, easy to extend, and it really tries hard to respect your existing config in /etc. In fact, not only does it put a copy of fstab in the initramfs to help it find your root, but after it mounts the root it checks that version of fstab to see if it is different and then remounts things accordingly. If you haven't guessed I'm a bit of a Dracut fan. :) -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 12:04 PM, Rich Freeman wrote: I don't see the value in using a different configuration on a box simply because it happens to work on that particular box. Dracut is a more generic solution that allows me to keep hosts the same. And if all the boxes in the fleet can function without an initramfs? Then why have it? Why not apply Occam's Razor & Parsimony and use the simpler solution. Especially if more complex solutions introduce additional things that need to be updated. Kinda sorta. The kernel boots one distro which then chroots and execs another. The initramfs follows the exact same rules as any other userspace rootfs. There's no "kinda" to it. Booting one distro (kernel & set of init scripts), then chrooting and execing a new kernel and the booting another distro is at least twice as complex as booting a single distro. This is even more complicated if the first initramfs distro is not identical to the main installed distro. Which is quite likely to be the case, or at least a subset of the main installed distro. IMHO an initramfs is usually, but not always, an unnecessary complication. Sure, and I wouldn't expect them to require rebuilding your initramfs either. I was speaking generally. Modifying things like crypttab and / or adding / removing file systems from the kernel that are required for boot have caused me to need to rebuild an initramfs in the past. But that was not necessarily Gentoo, so it may not be a fair comparison. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 1:54 PM Grant Taylor wrote: > > On 01/29/2019 10:58 AM, Rich Freeman wrote: > > Can't say I've tried it recently, but I'd be shocked if it changed much. > > The linux kernel guys generally consider this somewhat deprecated > > behavior, and prefer that users use an initramfs for this sort of thing. > > It is exactly the sort of problem an initramfs was created to fix. > > I see no reason to use an initramfs (swingroot) if the kernel can do > what is needed by itself. Personally I use dracut on boxes with a single ext4 partition... To each his own. I don't see the value in using a different configuration on a box simply because it happens to work on that particular box. Dracut is a more generic solution that allows me to keep hosts the same. > > Honestly, I'd just bite the bullet and use dracut if you want your OS > > on RAID/etc. > > You obviously have a different opinion than Alan and I do. Thank you! :) > > It is basically a one-liner at this point to install and a relatively > > small tweak to your GRUB config (automatic if using mkconfig). > > The dracut command may be a one-liner. But the alteration to the system > and it's boot & mount process is CONSIDERABLY more significant. Kinda sorta. The kernel boots one distro which then chroots and execs another. The initramfs follows the exact same rules as any other userspace rootfs. > > Dracut will respect your mdadm.conf, and just about all your other > > config info in /etc. The only gotcha is rebuilding your initramfs if > > it drastically changes (but, drastically changing your root filesystem > > is something that requires care anyway). > > I can think of some drastic changes to the root file system that would > not require changing the kernel, boot loader, or command line options. Sure, and I wouldn't expect them to require rebuilding your initramfs either. I was speaking generally. -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 1:39 PM Alan Mackenzie wrote: > > On Tue, Jan 29, 2019 at 12:58:38 -0500, Rich Freeman wrote: > > Can't say I've tried it recently, but I'd be shocked if it changed > > much. The linux kernel guys generally consider this somewhat > > deprecated behavior, and prefer that users use an initramfs for this > > sort of thing. It is exactly the sort of problem an initramfs was > > created to fix. > > An initramfs is conceptually so ugly that I view it as a workaround, not > a fix, to whatever problem it's applied to. Not sure why you would think this. It is just a cpio archive of a root filesystem that the kernel runs as a generic bootstrap. This means that your bootstrap for initializing your root and everything else can use any userspace tool that exists for linux. A similar concept lies at the heart of coreboot - using a generic kernel/userpace as a firmware bootloader making it far more flexible. An initramfs is basically just a fairly compact linux distro. It works the same as any distro. The kernel runs init, and init does its thing. By convention that init will mount the real root and then exec the init inside, but it doesn't have to work that way. Heck, you can run a system with nothing but an initramfs and no other root filesystem. > It would surely be a bug if the kernel were capable of manipulating RAIDs, > but not of initialising > and mounting them. Linus would disagree with you there, and has said as much publicly. He does not consider initialization to be the responsibility of kernel space long-term, and prefers that this happen in user-space. Some of the lvm and mdadm support remains for legacy reasons, but you probably won't see initialization of newer volume/etc managers supported directly in the kernel. > > Honestly, I'd just bite the bullet and use dracut if you want your OS > > on RAID/etc. It is basically a one-liner at this point to install and > > a relatively small tweak to your GRUB config (automatic if using > > mkconfig). Dracut will respect your mdadm.conf, and just about all > > your other config info in /etc. The only gotcha is rebuilding your > > initramfs if it drastically changes (but, drastically changing your > > root filesystem is something that requires care anyway). > > Well, at the moment my system's not broken, hence doesn't need fixing. > Last time I looked at Dracut, it would only work in a kernel built with > modules enabled, ruling out my setup. That is news to me. Obviously it needs whatever drivers it needs, but I don't see why it would care if they are built in-kernel vs in-module. > Also, without putting in a LOT of time and study, dracut is a massive, > opaque mystery. I've got a pretty good mental picture of how my system > works, and introducing an initramfs would degrade that picture > enormously. That means if any problems happened with the initramfs, I'd > be faced with many days study to get to grips with it. Sure, if you want to know exactly how it works that is true, but the same is true of openrc or any other piece of software on your system. Dracut is highly modular and phase/hook-driven so it isn't too hard to grok. Most of it is just bash/dash scripts. -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 10:58 AM, Rich Freeman wrote: Can't say I've tried it recently, but I'd be shocked if it changed much. The linux kernel guys generally consider this somewhat deprecated behavior, and prefer that users use an initramfs for this sort of thing. It is exactly the sort of problem an initramfs was created to fix. I see no reason to use an initramfs (swingroot) if the kernel can do what is needed by itself. Honestly, I'd just bite the bullet and use dracut if you want your OS on RAID/etc. You obviously have a different opinion than Alan and I do. I dislike using an initramfs (swingroot) without a specific reason to actually /need/ one. As in the kernel is unable to do what is necessary by itself and /requires/ assistance of an initramfs (swingroot). (An encrypted root device or iSCSI connected root device comes to mind as a legitimate /need/ for an initramfs (swingroot).) It is basically a one-liner at this point to install and a relatively small tweak to your GRUB config (automatic if using mkconfig). The dracut command may be a one-liner. But the alteration to the system and it's boot & mount process is CONSIDERABLY more significant. Dracut will respect your mdadm.conf, and just about all your other config info in /etc. The only gotcha is rebuilding your initramfs if it drastically changes (but, drastically changing your root filesystem is something that requires care anyway). I can think of some drastic changes to the root file system that would not require changing the kernel, boot loader, or command line options. But, if you're not using an initramfs you can get the kernel to handle this. Just don't be surprised when it changes your device name or whatever. There are ways to get around the device naming issue. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
Hello, Rich. On Tue, Jan 29, 2019 at 12:58:38 -0500, Rich Freeman wrote: > On Tue, Jan 29, 2019 at 11:48 AM Alan Mackenzie wrote: > > On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote: > > > On 01/29/2019 09:08 AM, Peter Humphrey wrote: > > > > I'd rather not have to create an initramfs if I can avoid it. Would it > > > > be sensible to start the raid volume by putting an mdadm --assemble > > > > command into, say, /etc/local.d/raid.start? The machine doesn't boot > > > > from /dev/md0. > > For this, the kernel needs to be able to assemble the drives into the > > raid at booting up time, and for that you need version 0.90 metadata. > > (Or, at least, you did back in 2017.) > Can't say I've tried it recently, but I'd be shocked if it changed > much. The linux kernel guys generally consider this somewhat > deprecated behavior, and prefer that users use an initramfs for this > sort of thing. It is exactly the sort of problem an initramfs was > created to fix. An initramfs is conceptually so ugly that I view it as a workaround, not a fix, to whatever problem it's applied to. It would surely be a bug if the kernel were capable of manipulating RAIDs, but not of initialising and mounting them. > Honestly, I'd just bite the bullet and use dracut if you want your OS > on RAID/etc. It is basically a one-liner at this point to install and > a relatively small tweak to your GRUB config (automatic if using > mkconfig). Dracut will respect your mdadm.conf, and just about all > your other config info in /etc. The only gotcha is rebuilding your > initramfs if it drastically changes (but, drastically changing your > root filesystem is something that requires care anyway). Well, at the moment my system's not broken, hence doesn't need fixing. Last time I looked at Dracut, it would only work in a kernel built with modules enabled, ruling out my setup. Also, without putting in a LOT of time and study, dracut is a massive, opaque mystery. I've got a pretty good mental picture of how my system works, and introducing an initramfs would degrade that picture enormously. That means if any problems happened with the initramfs, I'd be faced with many days study to get to grips with it. > But, if you're not using an initramfs you can get the kernel to handle > this. Just don't be surprised when it changes your device name or > whatever. The kernel seems to leave it alone. Any Gentoo installation CD I've used has corrupted the setup, changing all the names to /dev/md127, /dev/md126, , leaving the victim PC unbootable. Hence my root partition is /dev/md127, despite me originally creating it as something like /dev/md4. > -- > Rich -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tue, Jan 29, 2019 at 11:48 AM Alan Mackenzie wrote: > > On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote: > > On 01/29/2019 09:08 AM, Peter Humphrey wrote: > > > I'd rather not have to create an initramfs if I can avoid it. Would it > > > be sensible to start the raid volume by putting an mdadm --assemble > > > command into, say, /etc/local.d/raid.start? The machine doesn't boot > > > from /dev/md0. > > > For this, the kernel needs to be able to assemble the drives into the > raid at booting up time, and for that you need version 0.90 metadata. > (Or, at least, you did back in 2017.) > Can't say I've tried it recently, but I'd be shocked if it changed much. The linux kernel guys generally consider this somewhat deprecated behavior, and prefer that users use an initramfs for this sort of thing. It is exactly the sort of problem an initramfs was created to fix. Honestly, I'd just bite the bullet and use dracut if you want your OS on RAID/etc. It is basically a one-liner at this point to install and a relatively small tweak to your GRUB config (automatic if using mkconfig). Dracut will respect your mdadm.conf, and just about all your other config info in /etc. The only gotcha is rebuilding your initramfs if it drastically changes (but, drastically changing your root filesystem is something that requires care anyway). But, if you're not using an initramfs you can get the kernel to handle this. Just don't be surprised when it changes your device name or whatever. -- Rich
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 09:48 AM, Alan Mackenzie wrote: However, there's another quirk which bit me: something in the Gentoo installation disk took it upon itself to renumber my /dev/md2 to /dev/md127. I raised bug #539162 for this, but it was decided not to fix it. (This was back in February 2015.) I tend to treat the /dev/md to be somewhat fluid, much like I do /dev/sd. I prefer to use UUID for raw file systems or LV names when using LVM for this reason. $WORK uses sym-links from persistent names to the actual disk that is currently the desired disk. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
Hello, All. On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote: > On 01/29/2019 09:08 AM, Peter Humphrey wrote: > > I'd rather not have to create an initramfs if I can avoid it. Would it > > be sensible to start the raid volume by putting an mdadm --assemble > > command into, say, /etc/local.d/raid.start? The machine doesn't boot > > from /dev/md0. > Drive by comment. > I thought there was a kernel option / command line parameter that > enabled the kernel to automatically assemble arrays as it's > initializing. Would something like that work for you? > I have no idea where that is in the context of what you're working on. I use mdadm with a RAID-1 pair of SSDs, without an initramfs (YUCK!). My root partition is on the RAID. For this, the kernel needs to be able to assemble the drives into the raid at booting up time, and for that you need version 0.90 metadata. (Or, at least, you did back in 2017.) My command for building my array was: # mdadm --create /dev/md2 --level=1 --raid-devices=2 \ --metadata=0.90 /dev/nvme0n1p2 /dev/nvme1n1p2. However, there's another quirk which bit me: something in the Gentoo installation disk took it upon itself to renumber my /dev/md2 to /dev/md127. I raised bug #539162 for this, but it was decided not to fix it. (This was back in February 2015.) > -- > Grant. . . . > unix || die -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] RAID-1 on secondary disks how?
On 01/29/2019 09:08 AM, Peter Humphrey wrote: I'd rather not have to create an initramfs if I can avoid it. Would it be sensible to start the raid volume by putting an mdadm --assemble command into, say, /etc/local.d/raid.start? The machine doesn't boot from /dev/md0. Drive by comment. I thought there was a kernel option / command line parameter that enabled the kernel to automatically assemble arrays as it's initializing. Would something like that work for you? I have no idea where that is in the context of what you're working on. -- Grant. . . . unix || die
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tuesday, 29 January 2019 16:08:27 GMT Peter Humphrey wrote: > On Tuesday, 29 January 2019 09:20:46 GMT Mick wrote: > > Hello Mick, > > --->8 > > > Do you have CONFIG_MD_RAID1 (or whatever it should be these days) built in > > your kernel? > > Yes, I have, but something else was missing: CONFIG_DM_RAID=y. This is in > the SCSI section, which I'd overlooked (I hadn't needed it before because > the main storage is an NVMe drive). After setting that and rebooting, mdadm > --create is working as expected. Good! I had assumed this was already selected. ;-) > > You need to update your initramfs after you configure your array, so your > > kernel knows what to assemble at boot time when it doesn't yet have access > > to your mdadm.conf. > > I'd rather not have to create an initramfs if I can avoid it. Would it be > sensible to start the raid volume by putting an mdadm --assemble command > into, say, /etc/local.d/raid.start? The machine doesn't boot from /dev/md0. I think yes, as long as OS filesystem(s) are not on the array, or not mounted until the array has been assembled with the correct mdadm.config. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] RAID-1 on secondary disks how?
On Tuesday, 29 January 2019 09:20:46 GMT Mick wrote: Hello Mick, --->8 > Do you have CONFIG_MD_RAID1 (or whatever it should be these days) built in > your kernel? Yes, I have, but something else was missing: CONFIG_DM_RAID=y. This is in the SCSI section, which I'd overlooked (I hadn't needed it before because the main storage is an NVMe drive). After setting that and rebooting, mdadm --create is working as expected. The wiki needs a small addition. I submitted a bug against it but was told not to use bugs for this purpose: I should use the wiki page's discussion page. I see more head-scratchery coming on... --->8 > You need to update your initramfs after you configure your array, so your > kernel knows what to assemble at boot time when it doesn't yet have access > to your mdadm.conf. I'd rather not have to create an initramfs if I can avoid it. Would it be sensible to start the raid volume by putting an mdadm --assemble command into, say, /etc/local.d/raid.start? The machine doesn't boot from /dev/md0. -- Regards, Peter.
Re: [gentoo-user] RAID-1 on secondary disks how?
Hello Peter, On Monday, 28 January 2019 16:56:57 GMT Peter Humphrey wrote: > Hello list, > When I run "mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda2 > /dev/ sdb2", this is what I get: > > # mdadm --stop /dev/md0 > mdadm: stopped /dev/md0 > # mdadm: /dev/sda2 appears to contain an ext2fs file system >size=524288000K mtime=Thu Jan 1 01:00:00 1970 > mdadm: /dev/sdb2 appears to contain an ext2fs file system >size=524288000K mtime=Thu Jan 1 01:00:00 1970 > Continue creating array? y > mdadm: RUN_ARRAY failed: Invalid argument Do you have CONFIG_MD_RAID1 (or whatever it should be these days) built in your kernel? > If I boot from SysRescCD I also get "device /dev/sda2 exists but is not an > md device." I also had to "mdadm --stop /dev/md127" since that OS calls the > first array that. The SysRescueCD kernel you boot with does not (yet) have access to an mdadm.conf and is automatically compiling the RAID array with default settings. > I've tried with a GPT disk header and with an MSDOS one, with similar > results. Also with /etc/init.d/mdraid not running, with it started on my > command and with it in the boot runlevel. Each time I changed anything I > rebooted before trying anything else. > > I must be missing something, in spite of following the wiki instructions. > Can someone help an old duffer out? You need to update your initramfs after you configure your array, so your kernel knows what to assemble at boot time when it doesn't yet have access to your mdadm.conf. It could help if you examined/posted the contents of your: cat /etc/mdadm/mdadm.conf cat /proc/mdstat dmesg |grep md mdadm --detail --scan ls -l /dev/md* I haven't worked with RAID for some years now, but going from memory the above should reveal any discrepancies. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
[gentoo-user] RAID-1 on secondary disks how?
Hello list, (I've been off-line for ten days and I haven't yet caught up with the list. I had to send my machine to its maker to have a cooling-system hardware fault fixed.) I've added two SSDs to my workstation, intending to create a RAID-1 array on them to store backups (which may be another question when I've got the setup right), but I cannot create the array to create LVM. Each disk has an 80GiB NTFS partition in case I want to install Windows, then a 500GiB ext2 partition. When I run "mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda2 /dev/ sdb2", this is what I get: # mdadm --stop /dev/md0 mdadm: stopped /dev/md0 # mdadm: /dev/sda2 appears to contain an ext2fs file system size=524288000K mtime=Thu Jan 1 01:00:00 1970 mdadm: /dev/sdb2 appears to contain an ext2fs file system size=524288000K mtime=Thu Jan 1 01:00:00 1970 Continue creating array? y mdadm: RUN_ARRAY failed: Invalid argument If I boot from SysRescCD I also get "device /dev/sda2 exists but is not an md device." I also had to "mdadm --stop /dev/md127" since that OS calls the first array that. I've tried with a GPT disk header and with an MSDOS one, with similar results. Also with /etc/init.d/mdraid not running, with it started on my command and with it in the boot runlevel. Each time I changed anything I rebooted before trying anything else. I must be missing something, in spite of following the wiki instructions. Can someone help an old duffer out? -- Regards, Peter.