Re: [systemd-devel] Please help: timeout waiting for /dev/tty* console device
On Fri, Jan 06, 2023 at 07:15:14PM -0500, Gabriel L. Somlo wrote: > Hi, > > I'm failing to get a login prompt on the serial console of my system, > because a few steps earlier serial-getty@.service fails due to a > dependency on the actual tty device, which times out: > > [ TIME ] Timed out waiting for device …ttyLXU0.device - /dev/ttyLXU0. > [DEPEND] Dependency failed for seri…ice - Serial Getty on ttyLXU0. > > This eventually results in "Failed to start systemd-logind.service", > and no login prompt on the serial console. > > I'm trying get a riscv64 port of fedora (systemd version 251) running > on a system that can be considered "exotic" and rather on the slow > side -- it's an FPGA soft-cpu system using the RocketChip running at > 50MHz. > > I got it working successfully on an older version of the fedora-riscv > port, f33, using systemd 246. > > I can also get it working with the current systemd on a 4-core Rocket > chip deployed on a large enough FPGA. > > It (udev?) times out on the single-core version of the design, and I'm > wondering if there are any available options to get systemd and/or > udev to be more "patient". > > I tried booting with `udev.children_max=1` and `udev.event_timeout=800` > on the kernel command line. > > I also tried modifying systemd-udev-settle.service like so: > TimeoutSec=720 > ExecStart=udevadm settle --timeout=680 > > None of the above seem to help on the single-core 50MHz rocket-chip > system (and are not needed on the 4-core version). > > Any other tricks I can use to force it to wait (or to otherwise > encourage it to find, sooner) /dev/ttyLXU0? > > I can't log into this machine to run any tests. It does manage connect > to the network, start NetworkManager.service and sshd.service, but any > attempt to ssh in over the network fails (after successful auth) with > "conection closed by port 22". Not sure if due to the > user login service having failed, or some other unrelated timeout. Turns out it was just really slow loading bash :) After a while, I did manage to log in and obtain a root shell over ssh. From there I did: udevadm trigger --type=devices --action=add udevadm trigger --type=subsystems --action=add which had no visible effect. After that, I did: systemctl restart serial-getty@ttyLXU0.service which resulted in a login prompt showing up on the serial console! I *can* run any tests y'all might suggest to further debug the state of the system. But at this point I really do believe there is (or should be :) a way to extend the timeout during initial boot to force the system to wait for /dev/ttyLXU0 to become available (via udev?). Thanks much, --Gabriel
[systemd-devel] Please help: timeout waiting for /dev/tty* console device
Hi, I'm failing to get a login prompt on the serial console of my system, because a few steps earlier serial-getty@.service fails due to a dependency on the actual tty device, which times out: [ TIME ] Timed out waiting for device …ttyLXU0.device - /dev/ttyLXU0. [DEPEND] Dependency failed for seri…ice - Serial Getty on ttyLXU0. This eventually results in "Failed to start systemd-logind.service", and no login prompt on the serial console. I'm trying get a riscv64 port of fedora (systemd version 251) running on a system that can be considered "exotic" and rather on the slow side -- it's an FPGA soft-cpu system using the RocketChip running at 50MHz. I got it working successfully on an older version of the fedora-riscv port, f33, using systemd 246. I can also get it working with the current systemd on a 4-core Rocket chip deployed on a large enough FPGA. It (udev?) times out on the single-core version of the design, and I'm wondering if there are any available options to get systemd and/or udev to be more "patient". I tried booting with `udev.children_max=1` and `udev.event_timeout=800` on the kernel command line. I also tried modifying systemd-udev-settle.service like so: TimeoutSec=720 ExecStart=udevadm settle --timeout=680 None of the above seem to help on the single-core 50MHz rocket-chip system (and are not needed on the 4-core version). Any other tricks I can use to force it to wait (or to otherwise encourage it to find, sooner) /dev/ttyLXU0? I can't log into this machine to run any tests. It does manage connect to the network, start NetworkManager.service and sshd.service, but any attempt to ssh in over the network fails (after successful auth) with "conection closed by port 22". Not sure if due to the user login service having failed, or some other unrelated timeout. I can (also) ssh into the 4-core version of the system, and can run tests and report results from that one if it would help troubleshoot the issue (the systems are otherwise 100% identical in terms of "hardware" -- same HDL sources -- and only differ by the core count of the CPU module). Any help and/or ideas much appreciated, thanks in advance! Best, --Gabriel
Re: [systemd-devel] Ordering issues
On Do, 05.01.23 21:39, Scott Ellentuch (tuct...@gmail.com) wrote: > > > Restarted the machine and all good. So the next thing I tried was for > > > keepalived . It started up and failed with not being able to see its > > config > > > file (Moved to gluster). > > > > It checks for /etc/keepalived/keepalived.conf file. If you moved this > > file somewhere else, you presumably need to adjust path to this file in > > unit definition as well. > > > File is in the same location, just /etc/keepalived is a link to > /gfs/etc/keepalived . This sounds like a bad idea. use gluster for storing data, not for storing configuration. You are using a cluster storage where you should just use configuration management. Don't confuse the two concepts. One is totally not like the other, and you cannot replace one with the other. This will create tons of cyclic deps. This all sounds like a terrible idea, you are actively working on making things hard for you. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Unmountable mounts and systemd-fsck@.service conflicting with shutdown.target
It is a call to systemd-mount done from initramfs. It ends up in /run/systemd/transient and survives the root switch. The generated unit contains Requires=systemd-fsck@service. Is the conflict on shutdown.target to make shutdown kill fsck if it is running? Generated systemd-cryptsetup@.service units have "DefaultDependencies=no" and no conflict on shutdown. Maybe this is missing then. "cryptsetup attach" might be running. On Fri, Jan 6, 2023 at 1:34 PM Lennart Poettering wrote: > On Do, 05.01.23 14:18, Valentin David (valentin.da...@canonical.com) > wrote: > > > Hello, > > > > In Ubuntu Core, we have some mounts that cannot be unmounted until we > have > > switched root. > > > > To simplify, this looks like that: > > > > / mounts a ro loop devices backed by /some/disk/some/path/image.img > > /some/disk mounts a block device (let's say /dev/some-block0p1) > > > > In this case, /some/disk cannot be unmounted. > > > > We do not want to lazily unmount, we cannot get errors if something > fails. > > (Unless we had a lazy unmount that would only work when read-only) > > > > We do remount /some/disk read-only on shutdown. And in the shutdown > > intramfs, we unmount /oldroot/some/disk. > > > > However, we get an error message with systemd trying to unmount it. While > > functionally, it does not matter, it is still very problematic to have > > error messages. > > > > Using `DefaultDependencies=no` is not enough. I have tried to be clever > and > > add some-disk.mount to shutdown.target.wants so it would not try to > unmount > > it. But systemd got confused with conflicts and randomly kills stop jobs > > until there is no conflict. > > > > Debugging it, I have found that this is because some-disk.mount depends > on > > systemd-fsck@some\x2dblock0p1.service. And systemd-fsck@.service > conflicts > > with shutdown.target. > > > > I wonder if having conflict on shutdown.target really needed. Could we > > remove it? (And also add DefaultDepenencies=no to > > system-systemd\x2dfsck.slice) With this, mounts with > DefaultDependencie=no > > do not get unmounted as part of shutdown.target. (They do during > > systemd-shutdown) > > hmm, so we generally want system services to go away before > shutdown. This is a very special case though. I wonder if we can just > override systemd-fsck@….service for that specific case? > > How are those mounts established? i.e. by which unit is the > systemd-fsck@.service instance pulled in? and how was that configured? > fstab? ubuntu-own code? > > Lennart > > -- > Lennart Poettering, Berlin >
Re: [systemd-devel] Unmountable mounts and systemd-fsck@.service conflicting with shutdown.target
On Do, 05.01.23 14:18, Valentin David (valentin.da...@canonical.com) wrote: > Hello, > > In Ubuntu Core, we have some mounts that cannot be unmounted until we have > switched root. > > To simplify, this looks like that: > > / mounts a ro loop devices backed by /some/disk/some/path/image.img > /some/disk mounts a block device (let's say /dev/some-block0p1) > > In this case, /some/disk cannot be unmounted. > > We do not want to lazily unmount, we cannot get errors if something fails. > (Unless we had a lazy unmount that would only work when read-only) > > We do remount /some/disk read-only on shutdown. And in the shutdown > intramfs, we unmount /oldroot/some/disk. > > However, we get an error message with systemd trying to unmount it. While > functionally, it does not matter, it is still very problematic to have > error messages. > > Using `DefaultDependencies=no` is not enough. I have tried to be clever and > add some-disk.mount to shutdown.target.wants so it would not try to unmount > it. But systemd got confused with conflicts and randomly kills stop jobs > until there is no conflict. > > Debugging it, I have found that this is because some-disk.mount depends on > systemd-fsck@some\x2dblock0p1.service. And systemd-fsck@.service conflicts > with shutdown.target. > > I wonder if having conflict on shutdown.target really needed. Could we > remove it? (And also add DefaultDepenencies=no to > system-systemd\x2dfsck.slice) With this, mounts with DefaultDependencie=no > do not get unmounted as part of shutdown.target. (They do during > systemd-shutdown) hmm, so we generally want system services to go away before shutdown. This is a very special case though. I wonder if we can just override systemd-fsck@….service for that specific case? How are those mounts established? i.e. by which unit is the systemd-fsck@.service instance pulled in? and how was that configured? fstab? ubuntu-own code? Lennart -- Lennart Poettering, Berlin
