Re: [systemd-devel] [SPAM]Re: Mount units with After=autofs.service cause ordering cycles
On 10/31/19 2:59 PM, Lennart Poettering wrote: On Do, 31.10.19 14:09, John Florian (jflor...@doubledog.org) wrote: # /etc/systemd/system/var-www-pub.mount [Unit] Description=mount /pub served via httpd Requires=autofs.service After=autofs.service [Mount] What=/mnt/pub Where=/var/www/pub Options=bind,context=system_u:object_r:httpd_sys_content_t [Install] WantedBy=multi-user.target ~~~ The above worked for a long time, but once again a `dnf upgrade` seems to have broken things because now I have a ordering cycle that systemd must break. Since I haven't changed my mount units, my ability to mesh with those shipped by the OS proves fragile. I'm deliberately avoiding too much detail here because it would seem that there should be a relatively simple solution to this general sort of task -- I just can't seem to discover it. Any recommendations that don't involve an entirely different approach? What precisely is the ordering cycle you are seeing? It's usually dumped along with the log message. systemd[1]: local-fs.target: Found ordering cycle on var-www-pub.mount/start systemd[1]: local-fs.target: Found dependency on autofs.service/start systemd[1]: local-fs.target: Found dependency on rpc-statd.service/start systemd[1]: local-fs.target: Found dependency on network-online.target/start systemd[1]: local-fs.target: Found dependency on network.target/start systemd[1]: local-fs.target: Found dependency on NetworkManager.service/start systemd[1]: local-fs.target: Found dependency on sysinit.target/start systemd[1]: local-fs.target: Found dependency on systemd-update-done.service/start systemd[1]: local-fs.target: Found dependency on local-fs.target/start systemd[1]: local-fs.target: Job var-www-pub.mount/start deleted to break ordering cycle starting with local-fs.target/start The ordering dep between local-fs.target and var-ww-pub.mount is what you have to get rid of to remove the cycle. Set: … [Unit] DefaultDependencies=no Conflicts=umount.target Before=umount.target … [Install] WantedBy=remote-fs.target … i.e. make this a dep of remote-fs.target, not the implicit local-fs.target, so that we don#t pull it in early boot, but only during late boot, before remote-fs.target. Thanks Lennart! That did the trick. I and others I know have knocked heads on this one several times and somehow never came to this conclusion. It makes sense now that I see it, however. Maybe local-fs.target should have stood out to me, but I think it was mostly accepted since if you follow all deps far enough, you'll eventually cover (most?) everything. I think this just means I need to use `systemctl show` more even though `systemctl cat` is so much easier to digest for what I think I need to know. Abstracting the default deps is both good in expression but also difficult in comprehension. I wish there was something "in between", but I don't even know how to define what that means. Maybe just grouping all the settings from `show` somehow, e.g.: ordering, deps, etc. or maybe by unit type: unit, exec, mount, etc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [SPAM]Re: Mount units with After=autofs.service cause ordering cycles
On 10/31/19 1:08 PM, Lennart Poettering wrote: On Mo, 14.10.19 16:23, John Florian (j...@doubledog.org) wrote: So, I much prefer the expressiveness of systemd's mount units to the naive era of /etc/fstab, but I've found one situation where I seem to always get stuck and am never able to find a reliable solution that survives OS (Fedora & CentOS) updates. I have a NFS filesystem mounted by autofs at /pub that needs to be bind mounted in various places such as /var/www/pub and /var/ftp/pub. So I create a unit that looks like: ~~~ # /etc/systemd/system/var-www-pub.mount [Unit] Description=mount /pub served via httpd Requires=autofs.service After=autofs.service [Mount] What=/mnt/pub Where=/var/www/pub Options=bind,context=system_u:object_r:httpd_sys_content_t [Install] WantedBy=multi-user.target ~~~ The above worked for a long time, but once again a `dnf upgrade` seems to have broken things because now I have a ordering cycle that systemd must break. Since I haven't changed my mount units, my ability to mesh with those shipped by the OS proves fragile. I'm deliberately avoiding too much detail here because it would seem that there should be a relatively simple solution to this general sort of task -- I just can't seem to discover it. Any recommendations that don't involve an entirely different approach? What precisely is the ordering cycle you are seeing? It's usually dumped along with the log message. systemd[1]: local-fs.target: Found ordering cycle on var-www-pub.mount/start systemd[1]: local-fs.target: Found dependency on autofs.service/start systemd[1]: local-fs.target: Found dependency on rpc-statd.service/start systemd[1]: local-fs.target: Found dependency on network-online.target/start systemd[1]: local-fs.target: Found dependency on network.target/start systemd[1]: local-fs.target: Found dependency on NetworkManager.service/start systemd[1]: local-fs.target: Found dependency on sysinit.target/start systemd[1]: local-fs.target: Found dependency on systemd-update-done.service/start systemd[1]: local-fs.target: Found dependency on local-fs.target/start systemd[1]: local-fs.target: Job var-www-pub.mount/start deleted to break ordering cycle starting with local-fs.target/start ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Mount units with After=autofs.service cause ordering cycles
So, I much prefer the expressiveness of systemd's mount units to the naive era of /etc/fstab, but I've found one situation where I seem to always get stuck and am never able to find a reliable solution that survives OS (Fedora & CentOS) updates. I have a NFS filesystem mounted by autofs at /pub that needs to be bind mounted in various places such as /var/www/pub and /var/ftp/pub. So I create a unit that looks like: ~~~ # /etc/systemd/system/var-www-pub.mount [Unit] Description=mount /pub served via httpd Requires=autofs.service After=autofs.service [Mount] What=/mnt/pub Where=/var/www/pub Options=bind,context=system_u:object_r:httpd_sys_content_t [Install] WantedBy=multi-user.target ~~~ The above worked for a long time, but once again a `dnf upgrade` seems to have broken things because now I have a ordering cycle that systemd must break. Since I haven't changed my mount units, my ability to mesh with those shipped by the OS proves fragile. I'm deliberately avoiding too much detail here because it would seem that there should be a relatively simple solution to this general sort of task -- I just can't seem to discover it. Any recommendations that don't involve an entirely different approach? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] `Found ordering cycle on _SERVICE_` loops due to `local-fs.target` with `Requires` instead of `Wants` for mounts generated from `/etc/fstab`
On 2018-04-27 03:56, Jérémy Rosen wrote: Ok, there are a couple of tools that have not be mentionned.. "systemd-analyze verify " will check all sorts of stuff on a service, including checking recursively the deps and finding ordering cycles systemctl list-dependencies can show ordering/req dependencies, up or down, recursively or not to be completely honest, I never use dot for cycles I sometime read/grep the .dot file itself, but I never generate the image... systemd-analyze is my favorite tool for that sort of problems On 27/04/2018 06:01, Andrei Borzenkov wrote: Well, you could use "systemctl show -p Id -p After -p Before" but the problem is to find the correct unit set. You really need something that simulates transaction and evaluates dependencies like systemd does it. Note that systemd should print offending cycle when it hits it: апр 27 06:54:14 bor-Latitude-E5450 systemd[1582]: foo.service: Found ordering cycle on foo.service/start апр 27 06:54:14 bor-Latitude-E5450 systemd[1582]: foo.service: Found dependency on bar.service/start апр 27 06:54:14 bor-Latitude-E5450 systemd[1582]: foo.service: Found dependency on baz.service/start апр 27 06:54:14 bor-Latitude-E5450 systemd[1582]: foo.service: Found dependency on foo.service/start апр 27 06:54:14 bor-Latitude-E5450 systemd[1582]: foo.service: Breaking ordering cycle by deleting job baz.service/start Well these are good things to know. I can't recall if I ever tried to use them when faced with a loop. I'd guess that I tried `systemd-analyze verify` as it sort of jumps off the man page whereas `systemctl list-dependencies` is kind of buried in the details. It would be helpful it if additionally printed kind of dependency (like foo.service is After bar.service) because this is by far not obvious for average user. Maybe one really helpful immediate change would be to have systemd log some hints when it breaks a loop to suggest a few commands that might be run to help understand the situation more clearly. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] `Found ordering cycle on _SERVICE_` loops due to `local-fs.target` with `Requires` instead of `Wants` for mounts generated from `/etc/fstab`
On 2018-04-26 16:04, John Florian wrote: On 2018-04-25 04:53, Lennart Poettering wrote: There have been requests in improving the cycle breaking algorithm, but not much has been done in this area, since it's not clear what can be done. Ultimately it's just polishing a broken situation, and the better way out is to fix things properly, i.e. just correct the cycle in the dependencies in the first place. Having been the author of numerous custom services that are intended to wedge into those provided by systemd/Fedora I've faced resolving these several times and I've never felt all that competent at the task. Lennart, you were immensely helpful on one occasion by pointing me to `systemctl show FOO` but even then hunting the problem down was far from simple (for me). I've done the dot/graphviz thing and found it just as useless to me as when I've tried applying it to Puppet's ordering/dependency looping. I'm not blaming those tools (or systemd) because I'm well aware much of the problem is my inability to use them effectively. The graphs always seem to be overly simple and revealing no problem or overly detailed and obscuring the problem. Compound that with "an arrow pointing this way means what exactly?" Is there anything else that could be done to make hunting these loops down easier? Is there an example of any product that has a similar situation where they excel at helping the developer? Or are we already there and I just need more practice? I'm sure part of my struggle is just not encountering these very regularly and integrating into much that I'm only partly familiar with, but the result is the same, trepidation and loathing. Since I hate to grumble w/o so much as offering any possible suggestions... It seems like it might be helpful to have something like `systemctl show` but only dumping the properties used for ordering and dependencies. Ideally, only ordering OR dependencies as the case warrants and for all units involved, but only those units. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] `Found ordering cycle on _SERVICE_` loops due to `local-fs.target` with `Requires` instead of `Wants` for mounts generated from `/etc/fstab`
On 2018-04-25 04:53, Lennart Poettering wrote: There have been requests in improving the cycle breaking algorithm, but not much has been done in this area, since it's not clear what can be done. Ultimately it's just polishing a broken situation, and the better way out is to fix things properly, i.e. just correct the cycle in the dependencies in the first place. Having been the author of numerous custom services that are intended to wedge into those provided by systemd/Fedora I've faced resolving these several times and I've never felt all that competent at the task. Lennart, you were immensely helpful on one occasion by pointing me to `systemctl show FOO` but even then hunting the problem down was far from simple (for me). I've done the dot/graphviz thing and found it just as useless to me as when I've tried applying it to Puppet's ordering/dependency looping. I'm not blaming those tools (or systemd) because I'm well aware much of the problem is my inability to use them effectively. The graphs always seem to be overly simple and revealing no problem or overly detailed and obscuring the problem. Compound that with "an arrow pointing this way means what exactly?" Is there anything else that could be done to make hunting these loops down easier? Is there an example of any product that has a similar situation where they excel at helping the developer? Or are we already there and I just need more practice? I'm sure part of my struggle is just not encountering these very regularly and integrating into much that I'm only partly familiar with, but the result is the same, trepidation and loathing. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] About supporting relative values (e.g. 50%) in configuration
On 2018-03-09 03:13, Zbigniew Jędrzejewski-Szmek wrote: On Thu, Mar 08, 2018 at 03:48:54PM +0800, ChenQi wrote: Hi All, I'd like to know if systemd community thinks that adding support for relative values in configuration is a good idea. I found a related patch and discussion in https://lists.freedesktop.org/archives/systemd-devel/2015-May/032191.html. Quoting from the comments: """ I'd always keep our basic structures as generic as possible, and as close to the way CPUs work. Hence: store things as fixed-point 32bit/32bit internally, but make it configurable in percent as user-pointing UI. """ According to the comments, it seems that adding such support is acceptable? I' sending out this email because I want to check with the community before I start to investigate more. Hi, yes, a patch to allow relative values would be very much welcome. Indeed, very much! Please see my bug[0] for some history on this. [0] https://bugzilla.redhat.com/show_bug.cgi?id=874631 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Retaining boot messages on (near-)stateless systems
I maintain a derivative of Fedora Live (built using lorax) that gets deployed on hundreds of systems, far more than my team has the man- power to keep a watchful eye. Occasionally we are notified of a problematic node and often it would be helpful to see the full journal for say, the first 15 minutes of run-time in which the node transforms itself from a generic appliance to a specific role. Unfortunately, in many cases we are notified too late and journald has tail-trimmed the logs we desire most. Reviewing journald.conf(5) I don't see options to achieve what I want directly so my thought is to create a service and timer with OnBootSec=15m to duplicate the journal over to persistent storage, which would then require non-standard techniques for viewing but at least the availability would be guaranteed. Is there a better way to achieve this? If not, what's the best way to duplicate the journal data? Simply cp -a /var/log/journal/ ... or use journalctl to dump or export it somehow? I'll also add that I also wish to retain higher priority messages for a longer period, though for the first ~15m I want everything, DEBUG included. That leads me to think about a 2nd logger (journald, ideally) which had messages forwarded to it, but with different retention characteristics.___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How do I easily resolve conflicting jobs in custom units?
On Thu, 2017-03-30 at 16:04 +0200, Lennart Poettering wrote: > On Thu, 30.03.17 09:53, John Florian (j...@doubledog.org) wrote: > > > On Thu, 2017-03-30 at 15:32 +0200, Lennart Poettering wrote: > > > On Wed, 22.03.17 16:47, John Florian (j...@doubledog.org) wrote: > > > > > > > Mar 22 16:11:42 localhost systemd[1]: sshd.service: Looking at > > > > job > > > > sshd.service/start conflicted_by=no > > > > Mar 22 16:11:42 localhost systemd[1]: sshd.service: Looking at > > > > job > > > > sshd.service/stop conflicted_by=yes > > > > Mar 22 16:11:42 localhost systemd[1]: sshd.service: Fixing > > > > conflicting > > > > jobs sshd.service/start,sshd.service/stop by deleting job > > > > sshd.service/start > > > > > > So this suggests that you have both a job that pulls in > > > sshd.service > > > and one that conflicts with it in your initial transaction. Try > > > figuring out what that is... > > > > > > (hint: use "systemctl show sshd.service" for that, and look for > > > Conflicts= and ConlictedBy=) > > > > > > > That seems like a reasonable theory, but given the following I > > can't > > make any sense of it. > > > > # journalctl -b | grep -i conflict > > Mar 30 09:35:57 localhost systemd[1]: sshd.service: Looking at job > > sshd.service/start conflicted_by=no > > Mar 30 09:35:57 localhost systemd[1]: sshd.service: Looking at job > > sshd.service/stop conflicted_by=yes > > Mar 30 09:35:57 localhost systemd[1]: sshd.service: Fixing > > conflicting > > jobs sshd.service/start,sshd.service/stop by deleting job > > sshd.service/start > > Mar 30 09:35:57 localhost systemd[1]: sshd-keygen.target: Looking > > at > > job sshd-keygen.target/start conflicted_by=no > > Mar 30 09:35:57 localhost systemd[1]: sshd-keygen.target: Looking > > at > > job sshd-keygen.target/stop conflicted_by=no > > Mar 30 09:35:57 localhost systemd[1]: sshd-keygen.target: Fixing > > conflicting jobs sshd-keygen.target/start,sshd-keygen.target/stop > > by > > deleting job sshd-keygen.target/stop > > > > # systemctl show sshd.service | grep -i conflict > > Conflicts=shutdown.target > > ConflictedBy=sshd.socket > > Note that you can use sshd on Fedora in socket-activated mode or > non-socket-activated mode. In socket-activated mode, sshd.socket is > started at boot, and for each connection one sshd@.service instance > is > created. sshd.service otoh is only used in the non-socket-activated > mode, and in that case is the singleton service processing all > connections... Yup, I was aware of that of neat feature... > My educated guess is that you somehow ended up pull in both > sshd.service and sshd.socket, but you cannot really do that, you have > to make a choice between them, as there's only one port 22, and it's > either systemd that listens on it or sshd. Bingo! That was just what I needed to put my train of thought on a different set of tracks. systemctl status sshd.socket showed that it was enabled and the vendor preset was also enabled. Wait... what My package that contains the sshd-persist-keys.service and sshd-restore-keys.service units also provides a file with unit presets and in there I had "enabled sshd.*". That was fine back in F21, but no so with F25 so I replaced that with a "enable sshd.service" and "disabled sshd.socket" and all is well now. It's funny, the problem seems so obvious now yet only an hour ago I was totally blind to the problem. Thank you so much for your time and wisdom.___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How do I easily resolve conflicting jobs in custom units?
On Thu, 2017-03-30 at 15:32 +0200, Lennart Poettering wrote: > On Wed, 22.03.17 16:47, John Florian (j...@doubledog.org) wrote: > > > Mar 22 16:11:42 localhost systemd[1]: sshd.service: Looking at job > > sshd.service/start conflicted_by=no > > Mar 22 16:11:42 localhost systemd[1]: sshd.service: Looking at job > > sshd.service/stop conflicted_by=yes > > Mar 22 16:11:42 localhost systemd[1]: sshd.service: Fixing > > conflicting > > jobs sshd.service/start,sshd.service/stop by deleting job > > sshd.service/start > > So this suggests that you have both a job that pulls in sshd.service > and one that conflicts with it in your initial transaction. Try > figuring out what that is... > > (hint: use "systemctl show sshd.service" for that, and look for > Conflicts= and ConlictedBy=) > That seems like a reasonable theory, but given the following I can't make any sense of it. # journalctl -b | grep -i conflict Mar 30 09:35:57 localhost systemd[1]: sshd.service: Looking at job sshd.service/start conflicted_by=no Mar 30 09:35:57 localhost systemd[1]: sshd.service: Looking at job sshd.service/stop conflicted_by=yes Mar 30 09:35:57 localhost systemd[1]: sshd.service: Fixing conflicting jobs sshd.service/start,sshd.service/stop by deleting job sshd.service/start Mar 30 09:35:57 localhost systemd[1]: sshd-keygen.target: Looking at job sshd-keygen.target/start conflicted_by=no Mar 30 09:35:57 localhost systemd[1]: sshd-keygen.target: Looking at job sshd-keygen.target/stop conflicted_by=no Mar 30 09:35:57 localhost systemd[1]: sshd-keygen.target: Fixing conflicting jobs sshd-keygen.target/start,sshd-keygen.target/stop by deleting job sshd-keygen.target/stop # systemctl show sshd.service | grep -i conflict Conflicts=shutdown.target ConflictedBy=sshd.socket ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How do I easily resolve conflicting jobs in custom units?
On Thu, 2017-03-23 at 06:38 +0300, Andrei Borzenkov wrote: > 22.03.2017 23:47, John Florian пишет: > > I build an mostly-stateless appliance OS derived from Fedora (25 > > ATM) > > and have several custom units to make it all possible. My units > > had > > worked great with F21, but are now giving me problems with > > F25. One > > pair of the custom units do some trickery to relocate sshd host > > keys > > from /etc/ssh to an alternate location that provides persistence: > > > > > > Please make full log available as well as actual unit definitions > that > are not started. > Okay, attached are a complete boot.log (journalctl > boot.log) as well as the most relevant units. Please note that while I need a solution, it's just as important I understand HOW to resolve this kind of issue. I've struggled through similar problems in the past and I never feel adept at the process, like I'm not using a fundamental tool for the job and instead relying more on trial and error. While there's no doubt a lot of change between F21 and F25, the one thing that forced a change on my part was systemd transitioning from a single sshd-keygen.service to a templated sshd-keygen@.service. For me that meant the following change: diff --git lib/systemd/sshd-persist-keys.service lib/systemd/sshd- persist-keys.service index d3508a8..7500b68 100644 --- lib/systemd/sshd-persist-keys.service +++ lib/systemd/sshd-persist-keys.service @@ -1,8 +1,8 @@ [Unit] Description=OpenSSH server - persist volatile keys for the AOS -After=sshd-keygen.service +After=sshd-keygen.target Before=sshd.service -Wants=sshd-keygen.service +Wants=sshd-keygen.target [Service] ExecStart=/usr/sbin/sshd-persist-keys diff --git lib/systemd/sshd-restore-keys.service lib/systemd/sshd- restore-keys.service index 475cede..85ac3bc 100644 --- lib/systemd/sshd-restore-keys.service +++ lib/systemd/sshd-restore-keys.service @@ -1,7 +1,7 @@ [Unit] Description=OpenSSH server - restore persisted keys for the AOS After=aos-storage-init.service -Before=sshd-keygen.service +Before=sshd-keygen@rsa.service sshd-keygen@ecdsa.service sshd-keygen@e d25519.service [Service] ExecStart=/usr/sbin/sshd-restore-keys I'd love to know of a better way for my sshd-restore-kes.service to be coded such that it's unnecessary to list individual key types, but my experience has led me to believe that targets only define a point in time when a collection of units have completed and that there's no equivalent to a point in time prior to any of them having begun. Thus I found I couldn't use Before=sshd-keygen.target because keys were being generated and restored concurrently rather than serially (restored if possible and then the missing generated as needed). sshd-startup-failure.tar.gz Description: application/compressed-tar ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] How do I easily resolve conflicting jobs in custom units?
I build an mostly-stateless appliance OS derived from Fedora (25 ATM) and have several custom units to make it all possible. My units had worked great with F21, but are now giving me problems with F25. One pair of the custom units do some trickery to relocate sshd host keys from /etc/ssh to an alternate location that provides persistence: $ systemctl cat sshd-persist-keys.service # /usr/lib/systemd/system/sshd-persist-keys.service [Unit] Description=OpenSSH server - persist volatile keys for the AOS After=sshd-keygen.target Before=sshd.service Wants=sshd-keygen.target [Service] ExecStart=/usr/sbin/sshd-persist-keys Type=oneshot RemainAfterExit=yes [Install] WantedBy=multi-user.target $ systemctl cat sshd-restore-keys.service # /usr/lib/systemd/system/sshd-restore-keys.service [Unit] Description=OpenSSH server - restore persisted keys for the AOS After=aos-storage-init.service Before=sshd-keygen@rsa.service sshd-keygen@ecdsa.service sshd-keygen@ed 25519.service [Service] ExecStart=/usr/sbin/sshd-restore-keys Type=oneshot RemainAfterExit=yes [Install] WantedBy=multi-user.target I found that on some boots, sshd wasn't getting started. With the help of booting with systemd.log_level=debug I learned: $ sudo journalctl | grep conflict Mar 22 16:11:42 localhost systemd[1]: sshd.service: Looking at job sshd.service/start conflicted_by=no Mar 22 16:11:42 localhost systemd[1]: sshd.service: Looking at job sshd.service/stop conflicted_by=yes Mar 22 16:11:42 localhost systemd[1]: sshd.service: Fixing conflicting jobs sshd.service/start,sshd.service/stop by deleting job sshd.service/start Mar 22 16:11:42 localhost systemd[1]: sshd.socket: Looking at job sshd.socket/stop conflicted_by=no Mar 22 16:11:42 localhost systemd[1]: sshd.socket: Looking at job sshd.socket/start conflicted_by=no Mar 22 16:11:42 localhost systemd[1]: sshd.socket: Fixing conflicting jobs sshd.socket/stop,sshd.socket/start by deleting job sshd.socket/stop Mar 22 16:11:42 localhost systemd[1]: sshd-keygen.target: Looking at job sshd-keygen.target/start conflicted_by=no Mar 22 16:11:42 localhost systemd[1]: sshd-keygen.target: Looking at job sshd-keygen.target/stop conflicted_by=no Mar 22 16:11:42 localhost systemd[1]: sshd-keygen.target: Fixing conflicting jobs sshd-keygen.target/start,sshd-keygen.target/stop by deleting job sshd-keygen.target/stop I'm sure systemd-analyze is my friend here, but I've not found an effective way to grok the problem. s-a dot either gives me too much to comprehend or not enough to make evident the problem. I've tried feeding it different PATTERNS like: none at all, sshd.service, ssh- keygen.target, sshd-{persist,restore}-keys.service and more. I either see a whole forest or just a few trees that seem to be getting along happily together. What advice can you suggest?___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel