Re: [systemd-devel] [Help] Can't log in to homed user account: "No space left on device"
On Mon, Aug 24, 2020 at 2:44 AM Andrii Zymohliad wrote: > > > I suspect that the workaround until > > this is figured out why the fallocate fails (I suspect shared extents, > > there's evidence this home file has been snapshot and I don't know > > what effect that has on fallocating the file) is to use > > --luks-discard=true ? That should avoid the need to fallocate when > > opening. > > Just to confirm, "homectl update --luks-discard=on azymohliad" fixed the > issue for me. I can log in again. Thanks a lot to Chris and Andrei! > > > > And the user will have to be vigilant about the space usage > > of user home because it's now possible to overcommit. > > I guess it's better to reduce home size (to something like 300-350G in my > case) to decrease the probability of overcommit? > Yes. But mainly you'll just want to keep an eye on the underlying file system free space. If it goes empty, the home file system won't know this and can try to write anyway, and then we get a pretty icky kind of failure with the home file system, possibly very bad. Of course this is a temporary situation, we need to find a long term solution because it's definitely not intended that the user babysit the two file systems like this. There should be built-in logic to make sure It Just Works. But also in your case you should try to find out why there are shared extents. That's more of a btrfs question so i'll resume that conversation on the btrfs list. -- Chris Murphy ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [Help] Can't log in to homed user account: "No space left on device"
23.08.2020 09:34, Andrii Zymohliad пишет: > Hello! I've lost the ability to log in to my systemd-homed user account. I > would be very grateful for any help! > > If I log in as root and try to authenticate: > > # homectl authenticate azymohliad > > Then after typing my password I get the following output: > > Operation on home azymohliad failed: Not enough disk space for home azymohliad > ... > LUKS Discard: online=no offline=yes ... > > My root partition is 475G, and as you can see, home file size is ~400G (I > guess it was stupid to leave only 75G for root in the first place). But for > some reason `btrfs fi usage /` shows that only 352G are allocated on the > device before I try to authenticate (every time after boot), and full 475G > after authentication attempt. In discussion on btrfs list it was pointed out that btrfs fallocate by design always tries to reserve space for full file size. So in this case there are 123GB available which are not enough to reserve 400GB ... https://lore.kernel.org/linux-btrfs/798a9077-bcbd-076c-a458-3403010ce...@libero.it/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with setting up systemd for Apache on Debian 10
On Sun, Aug 23, 2020 at 12:19 PM Tom Browder wrote: > There is no official Apache systemd setup for Apache from source, and ... My original post was too harsh. My requests for help haven't been very clear. Thanks to hints and questions here, I took the time to do a little scarier experimentation with my live Apache installation and now have a service file that seems to work. In it I have added documentation to refresh tired and failing memories: $ cat apache2.service # Debian 10 installation instructions: # # See systemd documentation in: # # man systemd.unit # man systemd.service # # Place a copy of this file into: # /etc/systemd/system/ # # After creating or modifying any unit files, we must tell systemd # that we want it to look for new things: # # # systemctl daemon-reload # # Our new service should be recognized at this point, but it won't run # yet. We need to do two more things. First, tell systemd to enable # it, so that it will start every time we boot: # # # systemctl enable apache2.service # # Second, start it now: # # # systemctl start apache2.service # # Note that you don't get feedback from this command, because all it # does it send a message to systemd telling it to start your # service. The command you typed doesn't hang around to see what # happens next. You may use systemctl status myservice.service (or # systemctl status myservice) to check on your service, to make sure # it seems OK. # # # systemctl status apache2.service # # systemctl status apache2 # # Also note that systemctl status myservice gives more information if # you run it as root, compared to running it as a normal user. [Unit] Description=The Apache HTTP Server After=network.target remote-fs.target nss-lookup.target [Service] Type=forking Environment=APACHE_STARTED_BY_SYSTEMD=true ExecStart=/usr/local/apache2/bin/apachectl -k start ExecStop=/usr/local/apache2/bin/apachectl -k stop ExecReload=/usr/local/apache2/bin/apachectl -k graceful PrivateTmp=true Restart=on-abort [Install] WantedBy=multi-user.target # EOF Thank you. -Tom ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Antw: [EXT] Re: [Help] Can't log in to homed user account: "No space left on device"
> If you don't want to use quotas, I'd use separate partitions or logival > volumes... I'd love to have encrypted home on a separate partition or volume. In that case if something happens to my root fs, I'd still be able to access home. But as I understand from "Storage Mechanism: luks Home Directories" part here https://systemd.io/HOME_DIRECTORY/, it needs the full device with partition table. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Antw: [EXT] Re: [Help] Can't log in to homed user account: "No space left on device"
>>> Andrii Zymohliad schrieb am 24.08.2020 um 10:49 in Nachricht <2j3201oowK1vlEa9-nZ6zsUwIuIwqtmetdg1EYSf1Wq1h4SThYqY5IVNEyam4YzS422dZ1TxJJPTRTH fPWGpO8A9zyLz8HeLkF_xPfWhI8=@protonmail.com>: >> I suspect that the workaround until >> this is figured out why the fallocate fails (I suspect shared extents, >> there's evidence this home file has been snapshot and I don't know >> what effect that has on fallocating the file) is to use >> ‑‑luks‑discard=true ? That should avoid the need to fallocate when >> opening. > > Just to confirm, "homectl update ‑‑luks‑discard=on azymohliad" fixed the issue > for me. I can log in again. Thanks a lot to Chris and Andrei! > > >> And the user will have to be vigilant about the space usage >> of user home because it's now possible to overcommit. > > I guess it's better to reduce home size (to something like 300‑350G in my > case) to decrease the probability of overcommit? Actually I think separating system and user data is important: If your OS upgrade went wrong and you want to revert to the previous OS version via snapshot, you typically do not want to revert the user data as well. Or the other way 'round. Despite of that you probably don't want a user to fill up the whole root filesystem, possibly preventing logins. If you don't want to use quotas, I'd use separate partitions or logival volumes... > > ___ > systemd‑devel mailing list > systemd‑de...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/systemd‑devel ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd-fsck doesn't check data partiton
On Mo, 24.08.20 12:39, Benjamin Beckmeyer (b...@eks-engel.de) wrote: > root@em-switch:~ systemctl status systemd-fsck* > ● > systemd-fsck@dev-disk-by\x2duuid-1073706a\x2da8c1\x2d4d59\x2db701\x2d73b07f603 > Loaded: loaded (/usr/lib/systemd/system/systemd-fsck@.service; static; > vendor > Active: active (exited) since Sat 2018-12-01 01:00:07 CET; 2min 3s ago > Docs: man:systemd-fsck@.service(8) > Process: 210 ExecStart=/usr/lib/systemd/systemd-fsck > /dev/disk/by-uuid/1073706 > Main PID: 210 (code=exited, status=0/SUCCESS) > > Dec 01 01:00:07 em-switch systemd[1]: Starting File System Check on > /dev/disk/by > Dec 01 01:00:07 em-switch systemd-fsck[210]: read_bad_blocks_file: No such > file LMGTFY: https://www.mail-archive.com/openembedded-core@lists.openembedded.org/msg114560.html Please see README, we require the util-linux mount command. If you don't use that, you are on your own. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [Help] Can't log in to homed user account: "No space left on device"
> I suspect that the workaround until > this is figured out why the fallocate fails (I suspect shared extents, > there's evidence this home file has been snapshot and I don't know > what effect that has on fallocating the file) is to use > --luks-discard=true ? That should avoid the need to fallocate when > opening. Just to confirm, "homectl update --luks-discard=on azymohliad" fixed the issue for me. I can log in again. Thanks a lot to Chris and Andrei! > And the user will have to be vigilant about the space usage > of user home because it's now possible to overcommit. I guess it's better to reduce home size (to something like 300-350G in my case) to decrease the probability of overcommit? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemd-userdbd and other stuff running all the time
is taht growing amount of services running on all systems really necessary and why are things like "repart.service" and "homed.service" are started "static" which makes the concept of enable/disable things more and more obsolete 7.4 MB is a lot and i don't see any issue by just mask "systemd-userdbd.service" and "systemd-userdbd.socket" on sever systems at the moment, but who knows how deep the dependencies will become in the future lrwxrwxrwx 1 root root9 2020-05-28 09:06 systemd-homed.service -> /dev/null lrwxrwxrwx 1 root root9 2020-07-06 17:45 systemd-repart.service -> /dev/null lrwxrwxrwx 1 root root9 2020-08-06 18:48 systemd-timesyncd.service -> /dev/null lrwxrwxrwx 1 root root9 2020-08-24 12:47 systemd-userdbd.service -> /dev/null lrwxrwxrwx 1 root root9 2020-08-24 12:47 systemd-userdbd.socket -> /dev/null [root@srv-rhsoft:~]$ systemctl status systemd-userdbd ● systemd-userdbd.service - User Database Manager Loaded: loaded (/usr/lib/systemd/system/systemd-userdbd.service; static; vendor preset: disabled) Active: active (running) since Mon 2020-08-24 12:45:34 CEST; 48min ago TriggeredBy: ● systemd-userdbd.socket Docs: man:systemd-userdbd.service(8) Main PID: 719025 (systemd-userdbd) Status: "Processing requests..." Tasks: 4 (limit: 512) Memory: 7.4M CPU: 253ms CGroup: /system.slice/systemd-userdbd.service ├─719025 /usr/lib/systemd/systemd-userdbd ├─725718 systemd-userwork ├─725842 systemd-userwork └─725843 systemd-userwork ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemd-fsck doesn't check data partiton
Hi all, we have a problem on an embedded device wit emmc with systemd-fsck. It just doesn't check the partition. The exit status 8 means the partition is mounted, because when I do a check on the live system I get the same error. root@em-switch:~ systemctl status systemd-fsck* ● systemd-fsck@dev-disk-by\x2duuid-1073706a\x2da8c1\x2d4d59\x2db701\x2d73b07f603 Loaded: loaded (/usr/lib/systemd/system/systemd-fsck@.service; static; vendor Active: active (exited) since Sat 2018-12-01 01:00:07 CET; 2min 3s ago Docs: man:systemd-fsck@.service(8) Process: 210 ExecStart=/usr/lib/systemd/systemd-fsck /dev/disk/by-uuid/1073706 Main PID: 210 (code=exited, status=0/SUCCESS) Dec 01 01:00:07 em-switch systemd[1]: Starting File System Check on /dev/disk/by Dec 01 01:00:07 em-switch systemd-fsck[210]: read_bad_blocks_file: No such file Dec 01 01:00:07 em-switch systemd-fsck[210]: fsck failed with exit status 8. Dec 01 01:00:07 em-switch systemd-fsck[210]: Ignoring error. Dec 01 01:00:07 em-switch systemd[1]: Started File System Check on /dev/disk/by- root@em-switch:~ systemctl --version systemd 239 -PAM -AUDIT -SELINUX -IMA -APPARMOR -SMACK -SYSVINIT -UTMP -LIBCRYPTSETUP -GCRYPT -GNUTLS -ACL -XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=hybrid The embedded device has 3 partitions. 2 rootfs(ro) and one data partition(rw), between the rootfs we could do updates so we have everytime one partition to fallback if something got wrong. systemd-journald is writing its log to /data/journal and is configured to store=persistent. /etc/fstab: */ snip UUID=1073706a-a8c1-4d59-b701-73b07f603c4c /data ext4 defaults,sync 0 2 /data/journal /var/log none bind 0 0 I'm asking because we get some crazy write errors on the ext4 data partition. and the system won't boot anymore. After I boot the system from SD card and do a e2fsck on the data partition, the system will run again witho errors. Hopefully you can help me with this error, I will write to the ext4 mailing list for the error messages too but let me know if you need some more information. Best regards, Benjamin Beckmeyer ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [Help] Can't log in to homed user account: "No space left on device"
On Sun, Aug 23, 2020 at 6:47 AM Andrei Borzenkov wrote:
>
> 23.08.2020 15:34, Andrii Zymohliad пишет:
> >> Here is the log after authentication attempt:
> >> https://gitlab.com/-/snippets/2007113
> >> And just in case here is the full log since boot:
> >> https://gitlab.com/-/snippets/2007112
> >
> > Sorry, links are broken, re-uploaded:
> >
> > Authentication part: https://gitlab.com/-/snippets/2007123
> > Full log: https://gitlab.com/-/snippets/2007124
> >
>
> Yes, as suspected:
>
> > сер 23 14:12:48 az-wolf-pc systemd-homed[917]: Not enough disk space
> to fully allocate home.
>
> This comes from
>
> if (fallocate(backing_fd, FALLOC_FL_KEEP_SIZE, 0, st->st_size) <
> 0) {
>
> ...
> if (ERRNO_IS_DISK_SPACE(errno)) {
> log_debug_errno(errno, "Not enough disk space to
> fully allocate home.");
> return -ENOSPC; /* make recognizable */
> }
>
> return log_error_errno(errno, "Failed to allocate
> backing file blocks: %m");
> }
>
> So fallocate syscall failed. Try manually
>
> fallocate -l 403G -n /home/azymohliad.home
>
> if it fails too, the question is better asked on btrfs list.
User reports this from 'homectl inspect'
LUKS Discard: online=no offline=yes
Does this mean 'fstrim' is issued before luksClose? And 'fallocate' is
issued before luksOpen?
If so, it seems it'd be trivial to run into a fatal attempt to
activate, just by deactivating a user home of default size. And then
consuming free space on the underlying volume, such that there isn't
enough free space to fallocate the home file before opening it again.
What am I missing?
What it seems homed needs to do if fallocate fails for whatever reason
is to have some kind of fallback. Otherwise the user is stuck being
unable to open their user home. I suspect that the workaround until
this is figured out why the fallocate fails (I suspect shared extents,
there's evidence this home file has been snapshot and I don't know
what effect that has on fallocating the file) is to use
--luks-discard=true ? That should avoid the need to fallocate when
opening. And the user will have to be vigilant about the space usage
of user home because it's now possible to overcommit.
--
Chris Murphy
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
