Re: [systemd-devel] Spurious failures starting ConnMan with systemd

2016-11-21 Thread Simon McVittie 
On Tue, 15 Nov 2016 at 16:02:07 +0100, Daniel Wagner wrote:
> commit 09aa0243aac40ec4e5bd0fbe41e702be4952a382
> Author: Patrik Flykt 
> Date:   Thu Sep 17 10:42:46 2015 +0300
> 
> connman.service: Fix dependencies for early boot
> 
> Unset default dependencies in order to properly run at early boot and
> require the save directory to be mounted before starting.
> 
> See the systemd.unit man page, Debian's wiki page
> https://wiki.debian.org/Teams/pkg-systemd/rcSMigration and the upstream
> systemd-networkd.service file for details.

dbus-daemon is not (currently) designed to be usable as an early-boot
service. You cannot be an early-boot service and depend on D-Bus - you
don't get to "have your cake and eat it".

See also
https://bugs.freedesktop.org/show_bug.cgi?id=98254 and
https://bugs.freedesktop.org/show_bug.cgi?id=89847

Especially these comments:
https://bugs.freedesktop.org/show_bug.cgi?id=98254#c6
https://bugs.freedesktop.org/show_bug.cgi?id=98254#c8

(#c6 explains the significant feature work that would have to happen in
dbus-daemon to make it a viable early-boot service.)

Regards,
S
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Spurious failures starting ConnMan with systemd

2016-11-15 Thread Colin Guthrie 
Hi,

Daniel Wagner wrote on 15/11/16 15:02:
> Hi Colin,
> 
> On 11/15/2016 03:46 PM, Colin Guthrie wrote:
>> Hope you're keeping well?
> 
> Doing fine! How is live in the north? 

Grand thank you! :)

>> So, by default, dbus should be socket activated. That means that when

>> If socket activation is used, then there shouldn't be any need to
>> mention dbus.socket/service in connman.service at all.
> 
> """
> [Unit]
> Description=Connection service
> DefaultDependencies=false
> Conflicts=shutdown.target
> RequiresMountsFor=/var/lib/connman
> After=dbus.service network-pre.target systemd-sysusers.service
> Before=network.target multi-user.target shutdown.target
> Wants=network.target
...
> Hmm, now I am confused...

There may be valid reasons for not wanting some of the default
dependants but there are probably some you still want.

e.g. maybe you should do After=sockets.target (and perhaps
Requires=sockets.target although practically speaking I'm not sure if
this is needed), this would be sufficient. sockets.target is pretty low
level and can start quite early on. I suspect starting after that should
still allow connman to function as you require but without this sporadic
dbus connection issue.

HTHs

Col

-- 

Colin Guthrie
colin(at)mageia.org
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Spurious failures starting ConnMan with systemd

2016-11-15 Thread Colin Guthrie 
Hey Daniel,

Hope you're keeping well?

So, by default, dbus should be socket activated. That means that when
dbus.service eventually starts shouldn't really matter, provided it is
eventually started. This is because it's actually dbus.socket that's the
important unit. It should be started by sockets.target which is pulled
in as part of the default dependencies that all units get automatically
(provided they've not disabled this)

So, check for dbus.socket, and check that connman.service doesn't
disable default deps.

If socket activation is used, then there shouldn't be any need to
mention dbus.socket/service in connman.service at all.

HTHs

Col

Daniel Wagner wrote on 15/11/16 10:59:
> [Cc: systemd mailing list"
> 
> what happened until now
> """
> I am working on a system based on Yocto Project which uses ConnMan
> v1.33 for network management. The system uses systemd and generally
> works well so far, but once in a blue moon ConnMan fails to start up.
> Here is the relevant log from systemd:
> 
> [   15.886270] systemd[1]: connman.service: Main process exited,
> code=exited, status=1/FAILURE
> [   15.912838] systemd[1]: Failed to start Connection service.
> 
> There were no logs from connmand in this case, so in the
> connman.service file I've added the -d option to ExecStart= and set
> StandardOutput to syslog. I have restarted the system many times until
> the failure occurred again. There is only a single line from connmand
> in the log:
> 
> [   12.174528] connmand[278]: Failed to connect to socket
> /var/run/dbus/system_bus_socket: No such file or directory
> [   12.209448] systemd[1]: connman.service: Main process exited,
> code=exited, status=1/FAILURE
> [   12.242872] systemd[1]: Failed to start Connection service.
> 
> So the system D-Bus is not up and running the time ConnMan is started.
> In case of failure, I've noticed there is also no mentioning about
> D-Bus starting up in the log, but in all other logs there is.
> 
> Now, I think the After=dbus.service line in src/connman.service.in is
> not enough because After= specifies order, but not a requirement
> dependency. This what Requires= is there for. I've added the line
> 
> Requires=dbus.service
> 
> to the service file, reverted my other changes, and tried again. I've
> been unable to reproduce the problem ever since, so for me the problem
> seems to be fixed.
> 
> What do you think? Should src/connman.service.in be changed to include
> the Requires= line by default or is this something to be fixed
> somewhere else?
> 
> Should network-pre.target and systemd-sysusers.service be added to
> Requires= as well?
> """
> ]
> 
> On 11/15/2016 10:45 AM, Robert Tiemann wrote:
>> On 11/14/2016 01:30 PM, Patrik Flykt wrote:
>>> On Fri, 2016-11-11 at 12:04 +0100, Daniel Wagner wrote:

 I am no systemd expert. I just did a quick read in the documentation
 and
 it seems it is recommended to have After= and Requires= together:
>>
>> Yes, this seems to be required here.
>>
 Care to send a patch which adds Requires=? Please include your
 excellent error report into the commit message.
>>
>> I can create a patch that adds Requires=dbus.service to the service
>> file if you like, but please read my comments below.
>>
>>> Hmm, then again man systemd.service says that "...Service units with
>>> this option configured implicitly gain dependencies on the dbus.socket
>>> unit. This type is the default if BusName= is specified..."
>>>
>>> ConnMan defines both Type=dbus and Busname=net.connman, so it should
>>> already have these dependencies. Unless the DefaultDependencies=false
>>> overrides this behavior. Do we have any systemd experts around?
>>
>> I am by no means a systemd expert, so instead I've tried a few things.
>> The output of
>>
>> $ systemctl show connman.service
>>
>> with the service file from v1.33 installed does not show any
>> dependencies on dbus.socket, but only an After= dependency on
>> dbus.service. The only Requires= dependency is on -.mount (and then
>> there is also the RequiredMountsFor= dependency). I always thought
>> this should be enough because dbus.service is actually required by
>> other services in the system, therefore ConnMan should be started
>> *after* it, right?
>>
>> Well. As I've come to realize, the troubles started after an upgrade
>> of ConnMan v1.30 to v1.33 a few weeks ago. So, I've checked the
>> changes of connman.service.in.
>>
>> With the service file shipped with v1.30, the output of the above
>> command shows Requires=basic.target, which depends on sockets.target,
>> which depends on dbus.socket. I think that's why I didn't have any
>> trouble with v1.30.
>>
>> In 09aa024 (first change to the service file after v1.30 release),
>> DefaultDependencies was set to false. According to
>> https://www.freedesktop.org/software/systemd/man/systemd.special.html,
>> an After= dependency on basic.target is added automatically if
>> DefaultDependencies is set to yes, so this is missing

Re: [systemd-devel] Spurious failures starting ConnMan with systemd

2016-11-15 Thread Daniel Wagner 
Hi Colin,

On 11/15/2016 03:46 PM, Colin Guthrie wrote:
> Hope you're keeping well?

Doing fine! How is live in the north? 
 
> So, by default, dbus should be socket activated. That means that when
> dbus.service eventually starts shouldn't really matter, provided it is
> eventually started. This is because it's actually dbus.socket that's the
> important unit. It should be started by sockets.target which is pulled
> in as part of the default dependencies that all units get automatically
> (provided they've not disabled this)

Okay, that makes sense.

> So, check for dbus.socket, and check that connman.service doesn't
> disable default deps.

I think we do, see below.

> If socket activation is used, then there shouldn't be any need to
> mention dbus.socket/service in connman.service at all.

"""
[Unit]
Description=Connection service
DefaultDependencies=false
Conflicts=shutdown.target
RequiresMountsFor=/var/lib/connman
After=dbus.service network-pre.target systemd-sysusers.service
Before=network.target multi-user.target shutdown.target
Wants=network.target

[Service]
Type=dbus
BusName=net.connman
Restart=on-failure
ExecStart=/usr/sbin/connmand -n
StandardOutput=null
CapabilityBoundingSet=CAP_KILL CAP_NET_ADMIN CAP_NET_BIND_SERVICE CAP_NET_RAW 
CAP_SYS_TIME CAP_SYS_MODULE
ProtectHome=true
ProtectSystem=full

[Install]
WantedBy=multi-user.target
"""

Okay, so the DefaultDependencies=false is causing this problem then.
In our commit history I found following explanation for it:

"""
commit 09aa0243aac40ec4e5bd0fbe41e702be4952a382
Author: Patrik Flykt 
Date:   Thu Sep 17 10:42:46 2015 +0300

connman.service: Fix dependencies for early boot

Unset default dependencies in order to properly run at early boot and
require the save directory to be mounted before starting.

See the systemd.unit man page, Debian's wiki page
https://wiki.debian.org/Teams/pkg-systemd/rcSMigration and the upstream
systemd-networkd.service file for details.

diff --git a/src/connman.service.in b/src/connman.service.in
index 8f7f3429f7dc..0a8f15c9f90b 100644
--- a/src/connman.service.in
+++ b/src/connman.service.in
@@ -1,7 +1,10 @@
 [Unit]
 Description=Connection service
+DefaultDependencies=false
+Conflicts=shutdown.target
+RequiresMountsFor=@localstatedir@/lib/connman
 After=dbus.service network-pre.target
-Before=network.target remote-fs-pre.target
+Before=network.target shutdown.target remote-fs-pre.target
 Wants=network.target remote-fs-pre.target
 
 [Service]
"""

Hmm, now I am confused...

Thanks,
Daniel
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Spurious failures starting ConnMan with systemd

2016-11-15 Thread Daniel Wagner 

[Cc: systemd mailing list"

what happened until now
"""
I am working on a system based on Yocto Project which uses ConnMan
v1.33 for network management. The system uses systemd and generally
works well so far, but once in a blue moon ConnMan fails to start up.
Here is the relevant log from systemd:

[   15.886270] systemd[1]: connman.service: Main process exited, 
code=exited, status=1/FAILURE

[   15.912838] systemd[1]: Failed to start Connection service.

There were no logs from connmand in this case, so in the
connman.service file I've added the -d option to ExecStart= and set
StandardOutput to syslog. I have restarted the system many times until
the failure occurred again. There is only a single line from connmand
in the log:

[   12.174528] connmand[278]: Failed to connect to socket 
/var/run/dbus/system_bus_socket: No such file or directory
[   12.209448] systemd[1]: connman.service: Main process exited, 
code=exited, status=1/FAILURE

[   12.242872] systemd[1]: Failed to start Connection service.

So the system D-Bus is not up and running the time ConnMan is started.
In case of failure, I've noticed there is also no mentioning about
D-Bus starting up in the log, but in all other logs there is.

Now, I think the After=dbus.service line in src/connman.service.in is
not enough because After= specifies order, but not a requirement
dependency. This what Requires= is there for. I've added the line

Requires=dbus.service

to the service file, reverted my other changes, and tried again. I've
been unable to reproduce the problem ever since, so for me the problem
seems to be fixed.

What do you think? Should src/connman.service.in be changed to include
the Requires= line by default or is this something to be fixed
somewhere else?

Should network-pre.target and systemd-sysusers.service be added to
Requires= as well?
"""
]

On 11/15/2016 10:45 AM, Robert Tiemann wrote:

On 11/14/2016 01:30 PM, Patrik Flykt wrote:

On Fri, 2016-11-11 at 12:04 +0100, Daniel Wagner wrote:


I am no systemd expert. I just did a quick read in the documentation
and
it seems it is recommended to have After= and Requires= together:


Yes, this seems to be required here.


Care to send a patch which adds Requires=? Please include your
excellent error report into the commit message.


I can create a patch that adds Requires=dbus.service to the service
file if you like, but please read my comments below.


Hmm, then again man systemd.service says that "...Service units with
this option configured implicitly gain dependencies on the dbus.socket
unit. This type is the default if BusName= is specified..."

ConnMan defines both Type=dbus and Busname=net.connman, so it should
already have these dependencies. Unless the DefaultDependencies=false
overrides this behavior. Do we have any systemd experts around?


I am by no means a systemd expert, so instead I've tried a few things.
The output of

$ systemctl show connman.service

with the service file from v1.33 installed does not show any
dependencies on dbus.socket, but only an After= dependency on
dbus.service. The only Requires= dependency is on -.mount (and then
there is also the RequiredMountsFor= dependency). I always thought
this should be enough because dbus.service is actually required by
other services in the system, therefore ConnMan should be started
*after* it, right?

Well. As I've come to realize, the troubles started after an upgrade
of ConnMan v1.30 to v1.33 a few weeks ago. So, I've checked the
changes of connman.service.in.

With the service file shipped with v1.30, the output of the above
command shows Requires=basic.target, which depends on sockets.target,
which depends on dbus.socket. I think that's why I didn't have any
trouble with v1.30.

In 09aa024 (first change to the service file after v1.30 release),
DefaultDependencies was set to false. According to
https://www.freedesktop.org/software/systemd/man/systemd.special.html,
an After= dependency on basic.target is added automatically if
DefaultDependencies is set to yes, so this is missing now. The After=
dependency on dbus.service is still there, but it is not Required. Out
of curiosity, I've removed the dbus.service dependency completely to
check if Type=dbus is enough as is documented. Turns out it is not.
Neither the output of "systemctl show connman.service" nor that of
"systemctl list-dependencies connman.service" show _any_ dependencies
on dbus.service/dbus.socket in this case.

Why the automatically added After= dependency on basic.target has
turned into a Requires= dependency with the v1.30 file, but the
explicit After=dbus.service dependency in the file from 09aa024
remains an After= dependency is beyond me. My guess is that
DefaultDependencies does a little bit more than is documented, or
maybe it's just a bug in my version of systemd (v225). Maybe it's some
strange interaction between unit files that I am unable to see.

A comment from someone with a little more experience with systemd
would be most