Re: [systemd-devel] [PATCH weston] doc/systemd: system service example

[Matt Hoosier]

On Mon, Jan 22, 2018 at 7:49 AM, Pekka Paalanen <ppaala...@gmail.com> wrote:
> On Fri, 29 Dec 2017 15:09:28 -0600
> Matt Hoosier <matt.hoos...@gmail.com> wrote:
>
>> Hi Lennart,
>>
>> On Mon, Dec 4, 2017 at 9:11 AM, Pekka Paalanen <ppaala...@gmail.com> wrote:
>> > On Fri, 1 Dec 2017 18:25:35 +0100
>> > Lennart Poettering <lenn...@poettering.net> wrote:
>> >
>
>> >> So, as long as weston runs from a PAM enabled service (and its PAM
>> >> snippet pulls in pam_systemd) all should be good and race-free: as the
>> >> PAM session is set up XDG_RUNTIME_DIR will be made available and the
>> >> systemd --user instance is invoked in the background.
>> >
>> > This is exactly what we attempted with the User and PAMName directive,
>> > and it turned out to be racy somehow.
>
>> >> > > > > > +# Set up a full user session for the user, required by Weston.
>> >> > > > > > +PAMName=login
>> >> > > > >
>> >> > > > > Piggy-backing on "login" is a bad idea. "login" is a text tool, 
>> >> > > > > and
>> >> > > > > thus the PAM rules for it usually pull in some TTY specific PAM
>> >> > > > > modules. YOu shoudl really use your own PAM fragment here, and
>> >> > > > > configure only the bits you need.
>> >> > > >
>> >> > >
>> >> > > Oh, so could it just be that we needed something other than
>> >> > > "PAMName=login"?
>> >> >
>> >> > What are they key bits in the PAM configuration we must have, and are
>> >> > there any often used bits we must not have to avoid the race Martyn
>> >> > describes?
>> >>
>> >> well, pam_systemd needs to be pulled in from it, that's the most
>> >> important thing. A PAM snippet that pulls in pam_systemd means you get
>> >> a session allcoated in logind, which in turn sets up XDG_RUNTIME_DIR
>> >> for you.
>
>>
>> The approach that you and Pekka most recently put on record here:
>>
>> * User=foo
>> * PAMName=weston
>>
>> with a /etc/pam.d/weston that just does minimal stuff (enforce the
>> account exists and then execute pam_systemd.so for the session phase)
>> works well for me.
>
> Hi Matt,
>
> that is cool. Could you share your PAM file for 'weston'? When I get to
> re-spinning the example patch, I could use that instead of trying to
> craft my own which I probably could not test in any reasonable time.

Sure, although I'm not certain about its applicability to desktop
distributions. Mine is intended for an OpenEmbedded-style device, and
I basically just adapted the PAM file that some XDM recipe had:

  auth required pam_unix.so null
  account required pam_unix.so
  session required pam_unix.so
  -session optional pam_systemd.so

The entry about pam_systemd.so could probably be promoted to be
strictly required.

>
>> One thing I can't figure out though: using PAMName= causes the service
>> process's journal entries emitted by regular stdout and stderr not to
>> be visible with 'journalctl -u weston.service' anymore. Only the
>> messages coming internally from systemd ("Started Weston." and
>> similar) show in that journal.
>>
>> I've tacked in StandardOutput=journal and StandardError=journal to
>> compensate for the StandardInput=tty-fail. The messages do make it
>> across to journald; you can view them with 'journalctl
>> /usr/bin/weston'. But somehow they're not associated with the system
>> unit weston.service anymore. Does using the PAMName= directive cause
>> the stdout/stderr messages to be reassigned to a user-session unit or
>> something?
>
> That was an interesting point and it's nice to see Mantas' reply
> explaining what happens there. I should make a note of that as well.
>
>
> Thanks,
> pq
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH weston] doc/systemd: system service example

[Matt Hoosier]

On Dec 29, 2017 15:32, "Mantas Mikulėnas" <graw...@gmail.com> wrote:

On Fri, Dec 29, 2017 at 11:09 PM, Matt Hoosier <matt.hoos...@gmail.com>
wrote:

>
> The approach that you and Pekka most recently put on record here:
>
> * User=foo
> * PAMName=weston
>
> with a /etc/pam.d/weston that just does minimal stuff (enforce the
> account exists and then execute pam_systemd.so for the session phase)
> works well for me.
>
> One thing I can't figure out though: using PAMName= causes the service
> process's journal entries emitted by regular stdout and stderr not to
> be visible with 'journalctl -u weston.service' anymore. Only the
> messages coming internally from systemd ("Started Weston." and
> similar) show in that journal.
>
> I've tacked in StandardOutput=journal and StandardError=journal to
> compensate for the StandardInput=tty-fail. The messages do make it
> across to journald; you can view them with 'journalctl
> /usr/bin/weston'. But somehow they're not associated with the system
> unit weston.service anymore. Does using the PAMName= directive cause
> the stdout/stderr messages to be reassigned to a user-session unit or
> something?
>

No, it's done by pam_systemd specifically.

The main purpose of pam_systemd is to create a user session and move the
process to the session's own cgroup (from the weston.service cgroup). As a
result systemd no longer considers it as belonging to the weston.service
unit, but to session-c123.scope or such.)

(The same happens with all other interactive login types -- e.g. when you
log in at the console, you get moved out of getty@.service and into your
own cgroup.)

-- 
Mantas Mikulėnas


Okay, thanks. So that's just hardwired behavior that we should expect? I
tried looking around the source to pam_systemd a bit, but quickly lost
track of the way that the 'type', 'class', and other parameters to logind's
CreateSession() eventually interact with systemd recordkeeping.

-Matt
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH weston] doc/systemd: system service example

[Matt Hoosier]

Hi Lennart,

On Mon, Dec 4, 2017 at 9:11 AM, Pekka Paalanen  wrote:
> On Fri, 1 Dec 2017 18:25:35 +0100
> Lennart Poettering  wrote:
>
>> On Fr, 01.12.17 13:42, Pekka Paalanen (ppaala...@gmail.com) wrote:
>>
>> > > > > This is racy, as the session ID is not really reliably predictable,
>> > > > > and is synthesized in different contexts in different ways, for
>> > > > > example depnding on whether audit is enabled in the kernel it might 
>> > > > > be
>> > > > > session-1.scope rather than session-c1.scope.
>> >
>> > > > If we could determine the bug doesn't exist anymore, that would be
>> > > > awesome and I could just drop this.
>> >
>> > Hi Lennart,
>> >
>> > taking a step back, the session-c1.scope directive is definitely not
>> > wanted and we should drop it. We should also use a custom PAM name
>> > setup. If we do that, is the service file otherwise good for
>> > guaranteeing:
>> >
>> > - a full user session setup (I presume we want it), specifically
>> >   XDG_RUNTIME_DIR being set up
>> >
>> > - running exclusively on a VT that is made current
>>
>> This really depends on how weston sets up a VT. I really don't know
>> Weston and what it does.
>
> Weston doesn't set up the VT, we rely on the systemd unit directives to
> set it up.
>
> Weston calls sd_pid_get_session(getpid()), sd_session_get_seat(), and wants
> sd_session_get_vt() to succeed and give a VT number. Then it connects
> to logind, wants TakeControl to succeed, and calls Activate. It uses
> TakeDevice to open the DRM KMS device and input devices. I think that's
> the start-up sequence, it also listens on signals from logind etc.
>
>> > - access to DRM and input devices via logind
>>
>> So, I can't comment on Weston really.
>
> No worries, that was more of a general question about whether the
> example unit file was making any unwarranted assumptions.
>
>> Here are brief (and possibly slightly out-of-date, but probably not)
>> notes on how to write display managers with logind:
>>
>> https://www.freedesktop.org/wiki/Software/systemd/writing-display-managers/
>
> Thanks, I had a quick read through. We expect the systemd unit to also
> set up PAM, Weston itself does not touch PAM.
>
>> > so that all these are already in place by the time the Weston process
>> > is started?
>> >
>> > As you can see from Martyn below, the first issue that prevented Weston
>> > from running was that XDG_RUNTIME_DIR was not set. Furthermore, this
>> > failure did not occur always, sometimes things just worked as we
>> > expected.
>>
>> So, as long as weston runs from a PAM enabled service (and its PAM
>> snippet pulls in pam_systemd) all should be good and race-free: as the
>> PAM session is set up XDG_RUNTIME_DIR will be made available and the
>> systemd --user instance is invoked in the background.
>
> This is exactly what we attempted with the User and PAMName directive,
> and it turned out to be racy somehow.
>
>> What currently is not supported is to run things independently of any
>> session, i.e. run weston as systemd --user service with nothing that
>> creates a session in the first place. In that case XDG_RUNTIME_DIR
>> will not be set up, and no devices are made available to weston...
>
> Weston never was a --user service.
>
> As far as I know, there was also nothing that would manually attempt to
> start user@.service, the only trigger for that were the User and PAMName
> directives in the system weston.service.
>
>> > > > > > +# Set up a full user session for the user, required by Weston.
>> > > > > > +PAMName=login
>> > > > >
>> > > > > Piggy-backing on "login" is a bad idea. "login" is a text tool, and
>> > > > > thus the PAM rules for it usually pull in some TTY specific PAM
>> > > > > modules. YOu shoudl really use your own PAM fragment here, and
>> > > > > configure only the bits you need.
>> > > >
>> > >
>> > > Oh, so could it just be that we needed something other than
>> > > "PAMName=login"?
>> >
>> > What are they key bits in the PAM configuration we must have, and are
>> > there any often used bits we must not have to avoid the race Martyn
>> > describes?
>>
>> well, pam_systemd needs to be pulled in from it, that's the most
>> important thing. A PAM snippet that pulls in pam_systemd means you get
>> a session allcoated in logind, which in turn sets up XDG_RUNTIME_DIR
>> for you.
>
> Yes, it was, but apparently somewhere in the PAM stack or something it
> calls there was a race, which sometimes let Weston to start before
> XDG_RUNTIME_DIR environment variable was set, causing weston to fail.
>
> We all here were quite baffled on what could even be racing, unless it
> is possible that the weston process gets started in parallel with the
> PAM setup done by the User/PAMName in weston.service. We assumed that
> all the setup described in the systemd unit file would be guaranteed to
> complete before the actual process gets started.
>
> It seems our and your expectations are aligned. Maybe we 

[systemd-devel] systemd-bus-proxyd policy differences from dbus-daemon?

[Matt Hoosier]

I'm trying to figure out the root cause of some failures of regular old
libdbus1-using programs when /var/run/dbus/system_bus_socket happens to be
provided by systemd-bus-proxyd.

The programs in question attempt to invoke org.freedesktop.DBus.AddMatch()
with the rule-parameter having the value:


"type='signal',member='NameOwnerChanged',interface='org.freedesktop.DBus',path='/org/freedesktop/DBus',eavesdrop='false'".

On a regular (non-kdbus) system, this doesn't cause dbus-daemon to raise
its eyebrows, and the call succeeds. On a kdbus system, the bus proxy
daemon reports back SD_BUS_ERROR_INVALID_ARGS (i.e.,
"org.freedesktop.DBus.Error.InvalidArgs" in the dbus wire protocol
parlance).  The source code to the bus proxy daemon seems to only reach
this error condition when it judges the method request to have the wrong
signature -- something other than a single in-parameter string. But that's
not the case here as nearly as I can tell.

Any chance that systemd-bus-proxyd has some nonobvious hard-wired allergy
to the presence of the 'eavesdrop' attribute in that rule-string?

-Matt
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Adopt processes spawned before /lib/systemd/systemd takes over as PID 1?

[Matt Hoosier]

One final question on this topic:

I'm not sure from the available discussion of the 'Alias=' directive in
systemd.unit(5) whether it might be possible for me to have two units:

  /lib/systemd/system/dummy-adopted-service.service
...
[Unit]
DefaultDependencies=no

[Service]
Type=forking
PIDFile=/var/run/already-forked-process.pid
ExecStart=/bin/true

[Install]
Alias=the-service.service
Wanted-By=sysinit.target

  /lib/systemd/system/real-service.service
...
[Install]
Alias=the-service.service

such that clients always phrase their dependencies in terms of the aliased
name (the-service.service). At startup time, the virtual unit name would
always happen to be satisfied by dummy-adopted-service.service, but any
subsequent restarts would be accomplished by systemd noticing that
real-service.service exists and using it in preference over
dummy-adopted-service.service.

My gut feeling is that this approach doesn't play well with the way that
'systemctl enable' wants to place alias symlinks into /etc/systemd/system/.
I.e., I think that systemd will only ever recognize one provider of the
virtual/aliased service name.

Am I wrong in that assessment in the preceding paragraph?

On Mon, Apr 20, 2015 at 12:49 PM, Lennart Poettering lenn...@poettering.net
 wrote:

 On Mon, 20.04.15 12:12, Matt Hoosier (matt.hoos...@gmail.com) wrote:

inexperienced poking around the internal default suite of units
 packaged
with systemd.
  
   This is not available, though often requested. But I doubt this can
   ever work, since running before 'everything' or running after
   'everything' is not particularly usefully defined as this all breaks
   down as soon as you have two services that want to be run like this
 
 
  Okay, I appreciate that there's no built-in meta-unit that would permit
 me
  to say start me to the exclusion of absolutely everything else. I agree
  that would fail the what if two processes each tried to play that game?
  test.
 
  I was just hoping that some unit exists that is synonymous with the
 point
  upon which all traditional systemd work is dependent. I suppose nothing
  still exists matching that kind of weaker description? E.g., .slice or
  something like that.

 You could order yourself before local-fs-pre.target plus the various
 early-boot services we ship. That list is pretty limited and
 relatively stable, but there's nothing nicer currently, no.

 Lennart

 --
 Lennart Poettering, Red Hat

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Adopt processes spawned before /lib/systemd/systemd takes over as PID 1?

[Matt Hoosier]

On Mon, Apr 20, 2015 at 9:11 AM, Lennart Poettering lenn...@poettering.net
wrote:

 On Fri, 17.04.15 14:06, Matt Hoosier (matt.hoos...@gmail.com) wrote:

  The bootcharting that I do seems to show that about 1.2 - 1.5 sec are
 spent
  internal to systemd before any external processes get run for the
  particular embedded CPU I'm using. That gap is a killer at the moment.
 
  I'm sure this is quite the naive question, but: is there a simple option
 I
  can insert into my super-important first service that would cause all
  other units to be forestalled until my service reports that it's
  initialized? I.e., something like:
 
  [Unit]
  DefaultDependencies=no
  Before=very-first-normal-systemd-unit.service
 
  I didn't have luck identifying such a
  very-first-normal-systemd-unit.service on my own, but I'm admittedly
 rather
  inexperienced poking around the internal default suite of units packaged
  with systemd.

 This is not available, though often requested. But I doubt this can
 ever work, since running before 'everything' or running after
 'everything' is not particularly usefully defined as this all breaks
 down as soon as you have two services that want to be run like this


Okay, I appreciate that there's no built-in meta-unit that would permit me
to say start me to the exclusion of absolutely everything else. I agree
that would fail the what if two processes each tried to play that game?
test.

I was just hoping that some unit exists that is synonymous with the point
upon which all traditional systemd work is dependent. I suppose nothing
still exists matching that kind of weaker description? E.g., .slice or
something like that.

.

 The right way is usually to simply add the right deps of the stuff you
 really want to be run before.

 That all said, I would be open to adding a priority concept to
 units: if we are about to fork off a large number of processes at the
 same time we do so sequentially but provide no control currently about
 the order then. I'd be open making this configurable with a priority
 values that can ensure we fork off some things before others when both
 are runnable. This would not really fix your issue, sicne it wouldn't
 really delay any other services, it would only configure the order of
 the fork()s, but they'd still take place in a tight loop.

 Lennart

 --
 Lennart Poettering, Red Hat

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Adopt processes spawned before /lib/systemd/systemd takes over as PID 1?

[Matt Hoosier]

Hi,

I'm writing to see whether there's a best way to allow systemd to inherit
ownership of a process forked from a hand-crafted /sbin/init process before
that hand-crafted process turns over the keys to systemd by doing
exec(/lib/systemd/systemd) over the top of itself and allowing it to take
over as PID 1.

I know that sounds like an odd thing to ask about. The use-case has to do
with being able to start some work extremely early during boot of embedded
systems to achieve performance goals. I don't wish to subvert systemd, and
in fact would love for systemd to be able to monitor the process, stop it,
restart according to the normal [Service] configuration in a unit file
describing the process.

I can achieve a poor man's version of this right now by telling my
hand-spawned process to write out a pid file, and then supplying a unit
file that looks something like:

  #foo.service
  [Unit]
  Description=Proxy unit file for foo run prior to systemd
  ...

  [Service]
  Type=forking
  ExecStart=/bin/true
  PIDFile=/var/run/my-hand-forked-process-pidfile.txt

This is at least enough to get my process into the overall dependency graph
and to allow other units to depend on the process nominally described by
foo.service. But any kind of use of the 'Restart' option is out of the
question using this approach. Similarly, 'systemctl start foo.service'
wouldn't be able to really start the service explicitly.

Is there a (sneaky?) way to inform systemd that a daemon described property
in a service file, has already been launched and that systemd should simply
adopt the running instance?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Adopt processes spawned before /lib/systemd/systemd takes over as PID 1?

[Matt Hoosier]

On Fri, Apr 17, 2015 at 12:22 PM, Lennart Poettering lenn...@poettering.net
 wrote:

 On Fri, 17.04.15 09:00, Matt Hoosier (matt.hoos...@gmail.com) wrote:

  Hi,
 
  I'm writing to see whether there's a best way to allow systemd to
 inherit
  ownership of a process forked from a hand-crafted /sbin/init process
 before
  that hand-crafted process turns over the keys to systemd by doing
  exec(/lib/systemd/systemd) over the top of itself and allowing it to
 take
  over as PID 1.

 We support this only really for kernel-like processes that are
 started from the initrd, and basically run as long as the system is up
 without every being restarted in between, thus effectively appearing
 much like a kernel process and nothing systemd should
 manage. Processes like this should be marked with argv[0][0] = '@',
 see for details:

 https://wiki.freedesktop.org/www/Software/systemd/RootStorageDaemons/

  I know that sounds like an odd thing to ask about. The use-case has to do
  with being able to start some work extremely early during boot of
 embedded
  systems to achieve performance goals. I don't wish to subvert systemd,
 and
  in fact would love for systemd to be able to monitor the process, stop
 it,
  restart according to the normal [Service] configuration in a unit file
  describing the process.

 Hmm, are you sure that invoking the binary from systemd as first
 service is really that much slower than starting systemd only afterwards?


The bootcharting that I do seems to show that about 1.2 - 1.5 sec are spent
internal to systemd before any external processes get run for the
particular embedded CPU I'm using. That gap is a killer at the moment.

I'm sure this is quite the naive question, but: is there a simple option I
can insert into my super-important first service that would cause all
other units to be forestalled until my service reports that it's
initialized? I.e., something like:

[Unit]
DefaultDependencies=no
Before=very-first-normal-systemd-unit.service

I didn't have luck identifying such a
very-first-normal-systemd-unit.service on my own, but I'm admittedly rather
inexperienced poking around the internal default suite of units packaged
with systemd.



  I can achieve a poor man's version of this right now by telling my
  hand-spawned process to write out a pid file, and then supplying a unit
  file that looks something like:
 
#foo.service
[Unit]
Description=Proxy unit file for foo run prior to systemd
...
 
[Service]
Type=forking
ExecStart=/bin/true
PIDFile=/var/run/my-hand-forked-process-pidfile.txt

 urks, but yeah, this should work, however, this will not move the
 process into the service's cgroup. For that you can probably script
 something easily that echoes the PID manually into the cgroups dir of
 the service (which you can reference via %c).

  This is at least enough to get my process into the overall dependency
 graph
  and to allow other units to depend on the process nominally described by
  foo.service. But any kind of use of the 'Restart' option is out of the
  question using this approach. Similarly, 'systemctl start foo.service'
  wouldn't be able to really start the service explicitly.
 
  Is there a (sneaky?) way to inform systemd that a daemon described
 property
  in a service file, has already been launched and that systemd should
 simply
  adopt the running instance?

 Not really. Maybe you can make the start routine of your daemon smart
 enough to check if the daemon is already up. If so, you move that
 process to the cgroup the routine is started in. If not you fork off
 the real daemon. That way, you can start the daemon before systemd,
 then systemd will start it again, but instead of forking off a new
 instance it would just adopt the existing instance by systemd. When
 the daemon is the restarted through systemd then it will terminate the
 daemon cleanly and the new forked of process will be a regular again,
 if you follow what I mean.


Fair enough. That approach had occurred to me also.



 But this is all ugly, this is really nothing we want to support
 officially I am sure...


Understood.



 Lennart

 --
 Lennart Poettering, Red Hat


-Matt
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Adopt processes spawned before /lib/systemd/systemd takes over as PID 1?

[Matt Hoosier]

On Fri, Apr 17, 2015 at 3:52 PM, Cristian Rodríguez 
crrodrig...@opensuse.org wrote:

 On Fri, Apr 17, 2015 at 4:06 PM, Matt Hoosier matt.hoos...@gmail.com
 wrote:
  On Fri, Apr 17, 2015 at 12:22 PM, Lennart Poettering
  lenn...@poettering.net wrote:
 
  On Fri, 17.04.15 09:00, Matt Hoosier (matt.hoos...@gmail.com) wrote:
 
   Hi,
  
   I'm writing to see whether there's a best way to allow systemd to
   inherit
   ownership of a process forked from a hand-crafted /sbin/init process
   before
   that hand-crafted process turns over the keys to systemd by doing
   exec(/lib/systemd/systemd) over the top of itself and allowing it to
   take
   over as PID 1.
 
  We support this only really for kernel-like processes that are
  started from the initrd, and basically run as long as the system is up
  without every being restarted in between, thus effectively appearing
  much like a kernel process and nothing systemd should
  manage. Processes like this should be marked with argv[0][0] = '@',
  see for details:
 
  https://wiki.freedesktop.org/www/Software/systemd/RootStorageDaemons/
 
   I know that sounds like an odd thing to ask about. The use-case has to
   do
   with being able to start some work extremely early during boot of
   embedded
   systems to achieve performance goals. I don't wish to subvert systemd,
   and
   in fact would love for systemd to be able to monitor the process, stop
   it,
   restart according to the normal [Service] configuration in a unit file
   describing the process.
 
  Hmm, are you sure that invoking the binary from systemd as first
  service is really that much slower than starting systemd only
 afterwards?
 
 
  The bootcharting that I do seems to show that about 1.2 - 1.5 sec are
 spent
  internal to systemd before any external processes get run for the
 particular
  embedded CPU I'm using. That gap is a killer at the moment.

 Did you watch this presentation ?

 https://www.youtube.com/watch?v=RFVlbaDqll8

 what part of systemd is taking 1.5 seconds to start, on what CPU and
 how much of RAM does the board has ?


Thanks, I hadn't found that presentation before. My board is essentially a
Panda ES, with gigabytes of RAM.

A small point of clarification: when I say that systemd takes 1.5 seconds,
I'm referring to the time that elapses between the moment that
/lib/systemd/systemd is exec'ed and the time that the first unit is shown
in the 'systemd-analyze plot'. I haven't done an internal profile on the
systemd binary to see what might be happening during that window yet.

Could you say a word more about the sys_accept4() and /sys/fs/cgroup issue?
Was its only symptom that it caused systemd to run slower?

-Matt
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel