Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
...on Wed, Jul 08, 2020 at 05:12:54PM +0200, Alexander Bochmann wrote: > ...on Wed, Jul 08, 2020 at 06:14:51PM +1000, Andrew McGlashan via Dng wrote: > > Using the /dev/mapper device name would likely have been just as good, > but I'm not sure as I didn't try that > I'll try if using UUIDs in the fstab makes a difference in the > boot process later tonight (and maybe why, if it indeed does). So yes, using a UUID for a split /usr mount (on lvm) in the fstab totally doesn't work. I'm landing in the "Begin: Running /scripts/local-block ... done." loop too, ending with "ALERT! UUID= does not exist. Dropping to a shell!" I apparently changed my fstab from UUIDs to the /dev/mapper symlinks some time in 2019, way before my upgrade to beowulf (possibly when I migrated that machine into a VM) - so at least for some configurations, this problem has also been present in ascii. Searching for the error messages, the hint to use device names shows up for older Ubuntu versions too, but I haven't found a good explanation why this happens. Initramfs scripting for lvm was never fixed to work with UUIDs? Alex. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
...on Wed, Jul 08, 2020 at 06:14:51PM +1000, Andrew McGlashan via Dng wrote: > Using the /dev/mapper device name would likely have been just as good, but > I'm not sure as I didn't try that I'll try if using UUIDs in the fstab makes a difference in the boot process later tonight (and maybe why, if it indeed does). The system has been migrated from hardware into a VM some time ago, so I can recover easily. Alex. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On 8/7/20 10:07 pm, Hendrik Boom wrote: > On Wed, Jul 08, 2020 at 06:14:51PM +1000, Andrew McGlashan via Dng wrote: >> >> >> On 8/7/20 7:31 am, Alexander Bochmann wrote: >>> Hi, >>> >>> ...on Tue, Jul 07, 2020 at 02:00:38AM +1000, Andrew McGlashan via Dng wrote: >>> >>> > After the dist-upgrade, it failed to boot and remained at the >>> ministrants shell environment after having complained about not being able >>> to find the /usr file system via it's UUID. >>> >>> I have a system mostly like this (minus mdraid) with split root and /usr >>> on lvm each, and didn't run into your problem. >>> >>> My fstab uses /dev/mapper device names instead of UUIDs, but I don't see >>> why that should make a difference, seeing as it isn't used in the initramfs. >> >> Apparently with initramfs-tools it will try to mount /usr if it is in >> /etc/fstab ... not being able to mount /usr stopped normally boot from >> progressing further. >> >> Using the /dev/mapper device name would likely have been just as good, but >> I'm not sure as I didn't try that; I adjusted the >> /usr/share/initramfs-tools/scripts/local-top/lvm2 file >> to specifically activate the lv so it could be found to be mounted as it >> should have been. >> >>> (On the other hand, I usually use UUIDs too, so there might be a reason it >>> looks that way, and I just don't remember about it right now...) >> >> Yes, that makes sense. >> >> I would think that you fixed the problem by using the /dev/mapper >> entry and I fixed it in the lvm2 script. > > > I quite agree. There's a bug that needs fixing for Devuan, but not > Debian. > I may delay upgrading until it's fixed. Not sure it will get fixed... :( - it seems that the problem is a bit of an edge case and won't effect anybody whom doesn't split /usr from root. - if they have split them and they don't "merge" them, - then the problem /may/ only arise if UUIDs are used for mount reference in /etc/fstab. I don't really like my fix, but I'll probably merge /usr into root myself next time I'm onsite where that machine lives to avoid future issues. > My /boot is on an old-style RAID by itself, so either copy can be used > directly. > > My /usr, by the way, is on lvm2 on RAID. > Do I need both adjustments? I would think that the /dev/mapper/VG-LV in /etc/fstab would probably be fine. Otherwise, expand the root file system LV (hopefully you have space), boot from a LIVE USB and move /usr back to root as well as remove the /usr entry in your /etc/fstab file. Once /usr is back inside the root filesystem, then there is no need to keep the /usr lv. Cheers A. signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On Wed, Jul 08, 2020 at 06:14:51PM +1000, Andrew McGlashan via Dng wrote: > > > On 8/7/20 7:31 am, Alexander Bochmann wrote: > > Hi, > > > > ...on Tue, Jul 07, 2020 at 02:00:38AM +1000, Andrew McGlashan via Dng wrote: > > > > > After the dist-upgrade, it failed to boot and remained at the > > ministrants shell environment after having complained about not being able > > to find the /usr file system via it's UUID. > > > > I have a system mostly like this (minus mdraid) with split root and /usr > > on lvm each, and didn't run into your problem. > > > > My fstab uses /dev/mapper device names instead of UUIDs, but I don't see > > why that should make a difference, seeing as it isn't used in the initramfs. > > Apparently with initramfs-tools it will try to mount /usr if it is in > /etc/fstab ... not being able to mount /usr stopped normally boot from > progressing further. > > Using the /dev/mapper device name would likely have been just as good, but > I'm not sure as I didn't try that; I adjusted the > /usr/share/initramfs-tools/scripts/local-top/lvm2 file > to specifically activate the lv so it could be found to be mounted as it > should have been. > > > (On the other hand, I usually use UUIDs too, so there might be a reason it > > looks that way, and I just don't remember about it right now...) > > Yes, that makes sense. > > I would think that you fixed the problem by using the /dev/mapper > entry and I fixed it in the lvm2 script. I quite agree. There's a bug that needs fixing for Devuan, but not Debian. I may delay upgrading until it's fixed. My /boot is on an old-style RAID by itself, so either copy can be used directly. My /usr, by the way, is on lvm2 on RAID. Do I need both adjustments? -- hendrik > Either way, I think there is a bug that needs to be fixed with > initramfs-tools so that neither adjustment should be necessary. Quite agree. This is a bug in Devuan that originates in Debian but is not considered a bug there. So, as I understand it, if /usr is mentioned in /etc/fstab, initramfstools will generate an initramfs that tries to mount /usr. And that will succeed it /etc/fstab specifies /usr by the /dev/mapper name, but not by the uuid? So updating /etc/fstab to use the /dev/mapper name instead of a uuid will make things work? Even for LVM2 partitions? As it happens, my /etc/fstab alrady uses /dev/mapper names, though it uses a uuid for /boot. At the very least, this should be mentioned in the upgrade instructions. -- hendrik ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On 8/7/20 7:31 am, Alexander Bochmann wrote: > Hi, > > ...on Tue, Jul 07, 2020 at 02:00:38AM +1000, Andrew McGlashan via Dng wrote: > > > After the dist-upgrade, it failed to boot and remained at the ministrants > shell environment after having complained about not being able to find the > /usr file system via it's UUID. > > I have a system mostly like this (minus mdraid) with split root and /usr > on lvm each, and didn't run into your problem. > > My fstab uses /dev/mapper device names instead of UUIDs, but I don't see > why that should make a difference, seeing as it isn't used in the initramfs. Apparently with initramfs-tools it will try to mount /usr if it is in /etc/fstab ... not being able to mount /usr stopped normally boot from progressing further. Using the /dev/mapper device name would likely have been just as good, but I'm not sure as I didn't try that; I adjusted the /usr/share/initramfs-tools/scripts/local-top/lvm2 file to specifically activate the lv so it could be found to be mounted as it should have been. > (On the other hand, I usually use UUIDs too, so there might be a reason it > looks that way, and I just don't remember about it right now...) Yes, that makes sense. I would think that you fixed the problem by using the /dev/mapper entry and I fixed it in the lvm2 script. Either way, I think there is a bug that needs to be fixed with initramfs-tools so that neither adjustment should be necessary. Cheers A. signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
Hi, ...on Tue, Jul 07, 2020 at 02:00:38AM +1000, Andrew McGlashan via Dng wrote: > After the dist-upgrade, it failed to boot and remained at the ministrants > shell environment after having complained about not being able to find the > /usr file system via it's UUID. I have a system mostly like this (minus mdraid) with split root and /usr on lvm each, and didn't run into your problem. My fstab uses /dev/mapper device names instead of UUIDs, but I don't see why that should make a difference, seeing as it isn't used in the initramfs. (On the other hand, I usually use UUIDs too, so there might be a reason it looks that way, and I just don't remember about it right now...) Alex. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
Forget it - you have Thank you. On Tue, 7 Jul 2020, Jim Jackson wrote: > > > > On Tue, 7 Jul 2020, Steve Litt wrote: > > > On Mon, 6 Jul 2020 18:58:48 -0400 > > Hendrik Boom wrote: > > > > > > > Doesn't systemd require a merged /usr partition? It sounds as if a > > > systemd-ism has crept into our boot process. > > > > > > Fortunately I haven't upgraded my server to beowulf yet. > > > > > > -- hendrik > > > > Void Linux also symlinks all the various *bin directories together, > > even though it's a runit distro. My objection is this merge forces you > > to have an initramfs. > > Ok, maybe I've been asleep at the back - but can you explain why, and in > what circumstances? > > Jim-being-a-bit-slow > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng > ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On Tue, 7 Jul 2020, Steve Litt wrote: > On Mon, 6 Jul 2020 18:58:48 -0400 > Hendrik Boom wrote: > > > > Doesn't systemd require a merged /usr partition? It sounds as if a > > systemd-ism has crept into our boot process. > > > > Fortunately I haven't upgraded my server to beowulf yet. > > > > -- hendrik > > Void Linux also symlinks all the various *bin directories together, > even though it's a runit distro. My objection is this merge forces you > to have an initramfs. Ok, maybe I've been asleep at the back - but can you explain why, and in what circumstances? Jim-being-a-bit-slow ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On Mon, 6 Jul 2020 18:58:48 -0400 Hendrik Boom wrote: > Doesn't systemd require a merged /usr partition? It sounds as if a > systemd-ism has crept into our boot process. > > Fortunately I haven't upgraded my server to beowulf yet. > > -- hendrik Void Linux also symlinks all the various *bin directories together, even though it's a runit distro. My objection is this merge forces you to have an initramfs. SteveT Steve Litt May 2020 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On 7/7/20 8:58 am, Hendrik Boom wrote: > Doesn't systemd require a merged /usr partition? It sounds as if a > systemd-ism has crept into our boot process. > > Fortunately I haven't upgraded my server to beowulf yet. Probably I know that Debian wants merged /usr, wasn't sure it was specifically due to systemd, but I think you are right. I've upgraded 6 machines now from Ascii to Beowulf and it turns out the only one that I've done with this particular problem is the only one that had /usr as it's own file system and not part of the root file system. A. signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
On Tue, Jul 07, 2020 at 02:00:38AM +1000, Andrew McGlashan via Dng wrote: > Hi, > > I had another "simple" server upgrade from Devuan Ascii to Devuan Beowulf, > these are the details and my work around for the problem. > > > There was nothing particularly special about this server, it doesn't use > encrypted file systems; it started out life as a Debian Wheezy installation, > migrated to Devuan Jessie and > later to Devuan Ascii and now Beowulf. > > > The server has /boot on it's own RAID1 partition with another RAID1 volume > for the rest of the disk being an LVM2 volume group having a number of > logical volumes for root, swap, > /usr/, /var/, /home/ and more. Sounds just like my configuration. > > > After the dist-upgrade, it failed to boot and remained at the ministrants > shell environment after having complained about not being able to find the > /usr file system via it's UUID. > > It had another error as well which was fixed by allocating 25% to RUNSIZE > variable (up from 10%) in /etc/initramfs-tools/initramfs.conf > > - it was unable to find "rm" when running the boot up scripts before > dumping itself to the initramfs shell. > > > Once at the initramfs prompt after fixing the first problem, I was able to do > the following: > > (initramfs) lvm > > lvm> vgchange -ay > > lvm> exit > > (initramfs) exit > > > And then the server would continue to boot properly. > > > _The second fix, which I consider to be "clunky", was to adjust the > /usr/share/initramfs-tools/scripts/local-top/lvm2 file, adding in a line near > the bottom as highlighted_ > > activate "$ROOT" > *activate "/dev/mapper/vg0-usr"* > activate "$resume" > > > Then I rebuilt the initramfs in the usual way. > > update-initramfs -u -k all > > > The original lvm2 script specifically only activated the root file system > (/dev/mapper/vg0-root), even though /usr (/dev/mapper/vg0-usr) was in the > exact same volume group as a > separate file system, thus stopping boot from succeeding as expected. > > Other volumes come online in due course okay. > > > All was good with subsequent reboots. > > > Now, cludge or clunky, this was required because the /usr file system was > [and continues to be] separate to the root file system and the initramfs only > cared to enable the root > file system, leaving all other logical volumes as being "NOT AVAILABLE", > including /usr which was definitely required! > > > Have I fixed this appropriately, or should I some how fix it another way? > Doesn't systemd require a merged /usr partition? It sounds as if a systemd-ism has crept into our boot process. Fortunately I haven't upgraded my server to beowulf yet. -- hendrik ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] Upgrade problem [ ascii -> beowulf ] failed to boot, left at initramfs shell -- with fix and query
Hi, I had another "simple" server upgrade from Devuan Ascii to Devuan Beowulf, these are the details and my work around for the problem. There was nothing particularly special about this server, it doesn't use encrypted file systems; it started out life as a Debian Wheezy installation, migrated to Devuan Jessie and later to Devuan Ascii and now Beowulf. The server has /boot on it's own RAID1 partition with another RAID1 volume for the rest of the disk being an LVM2 volume group having a number of logical volumes for root, swap, /usr/, /var/, /home/ and more. After the dist-upgrade, it failed to boot and remained at the ministrants shell environment after having complained about not being able to find the /usr file system via it's UUID. It had another error as well which was fixed by allocating 25% to RUNSIZE variable (up from 10%) in /etc/initramfs-tools/initramfs.conf - it was unable to find "rm" when running the boot up scripts before dumping itself to the initramfs shell. Once at the initramfs prompt after fixing the first problem, I was able to do the following: (initramfs) lvm lvm> vgchange -ay lvm> exit (initramfs) exit And then the server would continue to boot properly. _The second fix, which I consider to be "clunky", was to adjust the /usr/share/initramfs-tools/scripts/local-top/lvm2 file, adding in a line near the bottom as highlighted_ activate "$ROOT" *activate "/dev/mapper/vg0-usr"* activate "$resume" Then I rebuilt the initramfs in the usual way. update-initramfs -u -k all The original lvm2 script specifically only activated the root file system (/dev/mapper/vg0-root), even though /usr (/dev/mapper/vg0-usr) was in the exact same volume group as a separate file system, thus stopping boot from succeeding as expected. Other volumes come online in due course okay. All was good with subsequent reboots. Now, cludge or clunky, this was required because the /usr file system was [and continues to be] separate to the root file system and the initramfs only cared to enable the root file system, leaving all other logical volumes as being "NOT AVAILABLE", including /usr which was definitely required! Have I fixed this appropriately, or should I some how fix it another way? Kind Regards AndrewM signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
