Re: Read-only rootfs on systemd
On Thu, Mar 06, 2014 at 11:05:20PM +, Amit wrote: 0) After reboot and running 'lsof +L1': COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME cupsd935 root8r REG8,1 1392 0 132095 /etc/passwd (deleted) So it's reproducible. 1) Shutting down cups: sudo service cups stop No /etc/passwd in 'lsof +L1' output Since cupsd is the only one who writes in there, and you've just stopped cupsd - that's expected. 2) Starting cups: $ sudo /etc/init.d/cups start [ ok ] Starting cups (via systemctl): cups.service. Oops. That's something I've forgot. I expected sysvinit compatibility layer to take care of cupsd starting. 3) No /etc/passwd in 'lsof +L1' output 4) fuser output shows cupsd process using /etc/passwd but no output in lsof showing '(deleted)'. Weird. Just checked again, and on my Wheezy install nobody is using /etc/passwd. Are you running Jessie? At this point remounting as 'ro' works. As it should be, as nobody is writing to the / filesystem now. As a workaround to all this, you probably can just restart cups on system's boot, and remount '/' read-only after that. As a real solution to the problem, I suggest you to fill a bug report to http://bugs.debian.org Reco -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140307092141.GA26358@x101h
Re: Read-only rootfs on systemd
On Thu 06 Mar 2014 at 23:05:20 +, Amit wrote: 0) After reboot and running 'lsof +L1': COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME cupsd935 root8r REG8,1 1392 0 132095 /etc/passwd (deleted) I upgraded my wheezy install to jessie and cannot reproduce this. 1) Shutting down cups: sudo service cups stop No /etc/passwd in 'lsof +L1' output 2) Starting cups: $ sudo /etc/init.d/cups start [ ok ] Starting cups (via systemctl): cups.service. 3) No /etc/passwd in 'lsof +L1' output It would appear to be relevant to investigate why starting cups at boottime produces a different outcome. It might also be worthwhile using another init system to determine whether 'lsof +L1' gives the same as above. 4) fuser output shows cupsd process using /etc/passwd but no output in lsof showing '(deleted)'. I get that. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/07032014181859.8901992de...@desktop.copernicus.demon.co.uk
Re: Read-only rootfs on systemd [SOLVED]
Thanks for your help and the replies. So this issue is now resolved. Summary of Issue: Mounting root as read-only as documented in (https://wiki.debian.org/ReadonlyRoot) because rootfs is busy. Summary of Solution: 1. 'lsof +L1' showed cupsd getting stuck on /etc/passwd (deleted). Looking at systemd journalctl, a custom service on this server was creating users blindly on every startup due to a bug in the code logic of this custom user management application. This caused /etc/passwd to keep changing. By fixing the bug, the /etc/passwd (deleted) was no longer an issue. 2. After reading the debian wiki on ReadOnlyRoot, there is a section on samba creating a file /etc/samba/dhcp.conf everytime dhclient was called. This server has a dual ethernet port with one port not plugged. As a result, dhclient kept trying to get a lease for this eth1 port and thus samba kept creating that file in /etc and rootfs was always busy. Not sure how this issue was solved but by adding the following line to /etc/fstab, I don't see the repeated dhclient messages anymore: tmpfs /tmptmpfs nodev,nosuid 0 0 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140307t202848-...@post.gmane.org
Re: Read-only rootfs on systemd
On Thu 06 Mar 2014 at 01:21:03 +, Amit wrote: I need cups, so is there a way around this? This doesn't answer your question but I have a spare Wheezy with separate /, /home, and /var. I installed systemd, made the rootfs ro in fstab and booted with init=/lib/systemd/systemd. The rootfs was mounted ro. cupsd is also running. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/06032014144345.2d9a7ff52...@desktop.copernicus.demon.co.uk
Re: Read-only rootfs on systemd
On 6 March 2014 01:21, Amit amit.ut...@gmail.com wrote: Amit amit.uttam at gmail.com writes: [snip] However, setting up a fresh install of systemd, the readonly does not have any effect. The rootfs is still mounted as rw. All I did was changed /etc/fstab. Based on the systemd man pages, this should be enough. How do I go about debugging/fixing this issue? The problem is cupsd. Doing an 'lsof +L1': COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME cupsd 825 root8r REG8,1 1392 0 132009 /etc/passwd (deleted) Stopping cups service and re-mounting as 'ro' now works. I need cups, so is there a way around this? Thanks, Amit Just a suggestion have you tried a re-install of cups since fresh install of systemd -- rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAOZWb-qJMcCFK5VF9S-m4UQ7T_Domh=psv2rttsxw29vnpp...@mail.gmail.com
Re: Read-only rootfs on systemd
Hi. On Thu, 6 Mar 2014 14:49:30 + Brian a...@cityscape.co.uk wrote: On Thu 06 Mar 2014 at 01:21:03 +, Amit wrote: I need cups, so is there a way around this? This doesn't answer your question but I have a spare Wheezy with separate /, /home, and /var. I installed systemd, made the rootfs ro in fstab and booted with init=/lib/systemd/systemd. The rootfs was mounted ro. cupsd is also running. https://wiki.debian.org/ReadonlyRoot#cups says: CUPS stores any kind of state files under /etc (classes.conf, cupsd.conf, printers.conf subscriptions.conf) and upstream is against any modification. Personally I worked around similar problem by moving /etc/cups to /var/opt, and symlinking /var/opt/cups to /etc. Reco -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140306191540.8ed53c576ecd29e1043cb...@gmail.com
Re: Read-only rootfs on systemd
Reco recoverym4n at gmail.com writes: https://wiki.debian.org/ReadonlyRoot#cups says: CUPS stores any kind of state files under /etc (classes.conf, cupsd.conf, printers.conf subscriptions.conf) and upstream is against any modification. Personally I worked around similar problem by moving /etc/cups to /var/opt, and symlinking /var/opt/cups to /etc. Thanks for the reply. The problem doesn't seem to be the statefiles, it seems to be '/etc/passwd'. I don't think I can safely move this to /var right? Thanks, Amit -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140306t193345-...@post.gmane.org
Re: Read-only rootfs on systemd
Robin rc.rattusrattus at gmail.com writes: Just a suggestion have you tried a re-install of cups since fresh install of systemd Thanks for the reply. Yes, the first thing I did was install systemd and then all the other packages but anyways I tried reinstalling again but no luck. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140306t193534-...@post.gmane.org
Re: Read-only rootfs on systemd
Brian ad44 at cityscape.co.uk writes: On Thu 06 Mar 2014 at 01:21:03 +, Amit wrote: I need cups, so is there a way around this? This doesn't answer your question but I have a spare Wheezy with separate /, /home, and /var. I installed systemd, made the rootfs ro in fstab and booted with init=/lib/systemd/systemd. The rootfs was mounted ro. cupsd is also running. Thanks for testing. I am running jessie at the moment. I kept a backup of sysvinit as recommended by wiki.debian.org/systemd and used that. I see the same issue of cups using /etc/passwd (deleted). So doesn't seem to be a systemd issue, more of a cups issue I guess. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140306t193742-...@post.gmane.org
Re: Read-only rootfs on systemd
On Thu, 6 Mar 2014 18:35:06 + (UTC) Amit amit.ut...@gmail.com wrote: Reco recoverym4n at gmail.com writes: https://wiki.debian.org/ReadonlyRoot#cups says: CUPS stores any kind of state files under /etc (classes.conf, cupsd.conf, printers.conf subscriptions.conf) and upstream is against any modification. Personally I worked around similar problem by moving /etc/cups to /var/opt, and symlinking /var/opt/cups to /etc. Thanks for the reply. The problem doesn't seem to be the statefiles, it seems to be '/etc/passwd'. I don't think I can safely move this to /var right? IMO it's not safe indeed to move /etc/passwd somewhere else. But, I've checked cups on my print-server and it does not keep open /etc/passwd at all. And, given what cups should do, I find it unlikely that it should behave that way. Can you do the following, please: 1) Shutdown cups by systemd's way (systemctl blahblah …). 2) Start it by /etc/init.d/cups start. 3) Confirm with lsof whenever /etc/passwd is kept open. 4) While you're at it, invoke 'fuser /etc/passwd' to ensure that nobody else is using it. Reco -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140306234036.27ebda625335038c18fb3...@gmail.com
Re: Read-only rootfs on systemd
Reco recoverym4n at gmail.com writes: Can you do the following, please: 1) Shutdown cups by systemd's way (systemctl blahblah …). 2) Start it by /etc/init.d/cups start. 3) Confirm with lsof whenever /etc/passwd is kept open. 4) While you're at it, invoke 'fuser /etc/passwd' to ensure that nobody else is using it. Reco 0) After reboot and running 'lsof +L1': COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME cupsd935 root8r REG8,1 1392 0 132095 /etc/passwd (deleted) 1) Shutting down cups: sudo service cups stop No /etc/passwd in 'lsof +L1' output 2) Starting cups: $ sudo /etc/init.d/cups start [ ok ] Starting cups (via systemctl): cups.service. 3) No /etc/passwd in 'lsof +L1' output 4) fuser output shows cupsd process using /etc/passwd but no output in lsof showing '(deleted)'. At this point remounting as 'ro' works. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140307t000512-...@post.gmane.org
Re: Read-only rootfs on systemd
Amit amit.uttam at gmail.com writes: [snip] However, setting up a fresh install of systemd, the readonly does not have any effect. The rootfs is still mounted as rw. All I did was changed /etc/fstab. Based on the systemd man pages, this should be enough. How do I go about debugging/fixing this issue? The problem is cupsd. Doing an 'lsof +L1': COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME cupsd 825 root8r REG8,1 1392 0 132009 /etc/passwd (deleted) Stopping cups service and re-mounting as 'ro' now works. I need cups, so is there a way around this? Thanks, Amit -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140306t021826-...@post.gmane.org
Read-only rootfs on systemd
Hello, I always run my debian systems with a separate /, /home, and /var. I added read-only 'ro' mount to fstab for the root / partition. So far it has been working great. However, setting up a fresh install of systemd, the readonly does not have any effect. The rootfs is still mounted as rw. All I did was changed /etc/fstab. Based on the systemd man pages, this should be enough. How do I go about debugging/fixing this issue? Thanks, Amit -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/loom.20140304t231522-...@post.gmane.org
