Re: Root is in RO after boot
Hi George, Run fsck on your root filesystem. On 8/1/20 5:15 am, George Shuklin wrote: On 1/7/20 4:33 PM, David Wright wrote: On Tue 07 Jan 2020 at 14:58:08 (+0200), George Shuklin wrote: After recent minor package upgrade (I'm on sid) I found that the root filesystem is in 'RO' mode. I changed 'ro' into 'rw' in grub.conf and after reboot everything works fine. Any update which triggers update-grub causes 'ro' back. Assuming you mean grub.cfg, you shouldn't do that. Your initialisation system will take care of switching from ro to rw at the correct time. I know, I've shouldn't, but at least I got a system with working UI. I want to fix this problem, but I can't find a place where 'rw' remount happens. I tried to find what's broke, but failed. I can't find who remounts root filesystem into rw mode. Is this some systemd generators magic? Or is this happens inside initrd? You need to look at your logs. I've only had root switch to ro upon getting i/o errors, which is what the fstab entry for root specifies: LABEL=swan07 / ext4 errors=remount-ro 0 1 It's a safety feature to stop those errors from gradually corrupting the filesystem further. RO on errors is a separate thing. As far as I understand (the fact I started the thread is a proof I don't), initrd mounts root as readonly, and something after pivot_root doing mount -o remount,rw for /. Unfortunately, I absolutely coudn't find it. There is a unit '-.mount' which claims to be root mount generated by systemd-fstab-generator, but I can't see any problems with it. Moreover, funny or not, I've just tried to reproduce the problem (run update-grub, reboot), and problem had gone. Now root is mounted as 'ro', remounted with no issues. So, I suppose, it gonna be a mystery. This problem (I reproduced it few times) was, and now it gone by itself. -- "Bother" said Pooh, "It's your husband and he has a gun."
Re: Root is in RO after boot
On 1/7/20 4:33 PM, David Wright wrote: On Tue 07 Jan 2020 at 14:58:08 (+0200), George Shuklin wrote: After recent minor package upgrade (I'm on sid) I found that the root filesystem is in 'RO' mode. I changed 'ro' into 'rw' in grub.conf and after reboot everything works fine. Any update which triggers update-grub causes 'ro' back. Assuming you mean grub.cfg, you shouldn't do that. Your initialisation system will take care of switching from ro to rw at the correct time. I know, I've shouldn't, but at least I got a system with working UI. I want to fix this problem, but I can't find a place where 'rw' remount happens. I tried to find what's broke, but failed. I can't find who remounts root filesystem into rw mode. Is this some systemd generators magic? Or is this happens inside initrd? You need to look at your logs. I've only had root switch to ro upon getting i/o errors, which is what the fstab entry for root specifies: LABEL=swan07 / ext4 errors=remount-ro 0 1 It's a safety feature to stop those errors from gradually corrupting the filesystem further. RO on errors is a separate thing. As far as I understand (the fact I started the thread is a proof I don't), initrd mounts root as readonly, and something after pivot_root doing mount -o remount,rw for /. Unfortunately, I absolutely coudn't find it. There is a unit '-.mount' which claims to be root mount generated by systemd-fstab-generator, but I can't see any problems with it. Moreover, funny or not, I've just tried to reproduce the problem (run update-grub, reboot), and problem had gone. Now root is mounted as 'ro', remounted with no issues. So, I suppose, it gonna be a mystery. This problem (I reproduced it few times) was, and now it gone by itself.
Re: Root is in RO after boot
Kenneth Parker wrote: > I've been wrong before. So are you saying that initrd is called without / > being mounted at all! > there is init script inside the initrd (which is an archive that gets extracted into memory) the init script there is a complex set of scripts that do various tasks, before the boot process can be handed over to the system installed on disk. this includes loading of drivers and mounting the root file system. initially (when this init script loads the drivers, probably asks for luks passphrase and enables logical volumes) it is mounted ro and when it hands over it remounts to rw. from there on, the init (binary or systemd) is executed and handles the second stage of the boot process. https://wiki.debian.org/initramfs you can check also /etc/initramfs-tools/scripts/ or /usr/share/initramfs-tools/scripts/ you can extract the initrd and study it, it is very interesting regards
Re: Root is in RO after boot
Le 07/01/2020 à 16:45, Kenneth Parker a écrit : So are you saying that initrd is called without / being mounted at all! Of course. The main purpose of the initramfs is to mount the final root filesystem before starting the final init.
Re: Root is in RO after boot
On Tue, Jan 7, 2020, 10:24 AM Pascal Hambourg wrote: > Le 07/01/2020 à 15:28, Kenneth Parker a écrit : > > > > As far as I know, it's mounted ro, so that initrd can check if it is okay > > Nonsense. The initramfs can check the root filesystem before mounting > it, so it does not need to mount it read-only. > I've been wrong before. So are you saying that initrd is called without / being mounted at all! > > > (via fsck). And initrd is then supposed to remount it rw. > > I don't think so. If you change the default final init with something > like init=/bin/bash for example, you can see that the root filesystem is > still mounted read-only. So something in the init system remounts it > read-write. With sysvinit it is done in checkroot.sh according to mount > options in fstab. I don't know about systemd. > Something more, for me to learn. Kenneth Parker >
Re: Root is in RO after boot
Le 07/01/2020 à 15:28, Kenneth Parker a écrit : As far as I know, it's mounted ro, so that initrd can check if it is okay Nonsense. The initramfs can check the root filesystem before mounting it, so it does not need to mount it read-only. (via fsck). And initrd is then supposed to remount it rw. I don't think so. If you change the default final init with something like init=/bin/bash for example, you can see that the root filesystem is still mounted read-only. So something in the init system remounts it read-write. With sysvinit it is done in checkroot.sh according to mount options in fstab. I don't know about systemd.
Re: Root is in RO after boot
On Tue 07 Jan 2020 at 14:58:08 (+0200), George Shuklin wrote: > > After recent minor package upgrade (I'm on sid) I found that the root > filesystem is in 'RO' mode. I changed 'ro' into 'rw' in grub.conf and > after reboot everything works fine. Any update which triggers > update-grub causes 'ro' back. Assuming you mean grub.cfg, you shouldn't do that. Your initialisation system will take care of switching from ro to rw at the correct time. > I tried to find what's broke, but failed. > > I can't find who remounts root filesystem into rw mode. Is this some > systemd generators magic? Or is this happens inside initrd? You need to look at your logs. I've only had root switch to ro upon getting i/o errors, which is what the fstab entry for root specifies: LABEL=swan07 / ext4 errors=remount-ro 0 1 It's a safety feature to stop those errors from gradually corrupting the filesystem further. Cheers, David.
Re: Root is in RO after boot
On Tue, Jan 7, 2020, 7:58 AM George Shuklin wrote: > Hello. > > After recent minor package upgrade (I'm on sid) I found that the root > filesystem is in 'RO' mode. I changed 'ro' into 'rw' in grub.conf and > after reboot everything works fine. Any update which triggers > update-grub causes 'ro' back. > > I tried to find what's broke, but failed. > > I can't find who remounts root filesystem into rw mode. Is this some > systemd generators magic? Or is this happens inside initrd? > As far as I know, it's mounted ro, so that initrd can check if it is okay (via fsck). And initrd is then supposed to remount it rw. If it can't be automatically verified, you get dumped, unceremoniously into a busybox prompt *inside* initrd. (I know, because I troubleshooted a friend's Buster laptop, after he pressed the power button, during an apt upgrade. Ugh!) Good luck! Kenneth Parker >
Root is in RO after boot
Hello. After recent minor package upgrade (I'm on sid) I found that the root filesystem is in 'RO' mode. I changed 'ro' into 'rw' in grub.conf and after reboot everything works fine. Any update which triggers update-grub causes 'ro' back. I tried to find what's broke, but failed. I can't find who remounts root filesystem into rw mode. Is this some systemd generators magic? Or is this happens inside initrd?
