Re: [systemd-devel] [PATCH] Add handling for bind/unbind actions

[systemd github import bot]

Patchset imported to github.
To create a pull request, one of the main developers has to initiate one via:


--
Generated by https://github.com/haraldh/mail2git
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] [PATCH] Add handling for bind/unbind actions

[Dmitry Torokhov]

Newer kernels will emit uevents with "bind" and "unbind" actions. These
uevents will be issued when driver is bound to or unbound from a device.
"Bind" events are helpful when device requires a firmware to operate
properly, and driver is unable to create a child device before firmware
is properly loaded.

For some reason systemd validates actions and drops the ones it does not
know, instead of passing them on through as old udev did, so we need to
explicitly teach it about them.
---

BIND/UNBIND will be in 4.14.

 src/libsystemd/sd-device/device-internal.h | 2 ++
 src/libsystemd/sd-device/device-private.c  | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/src/libsystemd/sd-device/device-internal.h 
b/src/libsystemd/sd-device/device-internal.h
index f4783deef..0505a2730 100644
--- a/src/libsystemd/sd-device/device-internal.h
+++ b/src/libsystemd/sd-device/device-internal.h
@@ -104,6 +104,8 @@ typedef enum DeviceAction {
 DEVICE_ACTION_MOVE,
 DEVICE_ACTION_ONLINE,
 DEVICE_ACTION_OFFLINE,
+DEVICE_ACTION_BIND,
+DEVICE_ACTION_UNBIND,
 _DEVICE_ACTION_MAX,
 _DEVICE_ACTION_INVALID = -1,
 } DeviceAction;
diff --git a/src/libsystemd/sd-device/device-private.c 
b/src/libsystemd/sd-device/device-private.c
index b4cd676c1..8839c3266 100644
--- a/src/libsystemd/sd-device/device-private.c
+++ b/src/libsystemd/sd-device/device-private.c
@@ -466,6 +466,8 @@ static const char* const 
device_action_table[_DEVICE_ACTION_MAX] = {
 [DEVICE_ACTION_MOVE] = "move",
 [DEVICE_ACTION_ONLINE] = "online",
 [DEVICE_ACTION_OFFLINE] = "offline",
+[DEVICE_ACTION_BIND] = "bind",
+[DEVICE_ACTION_UNBIND] = "unbind",
 };
 
 DEFINE_STRING_TABLE_LOOKUP(device_action, DeviceAction);
-- 
2.14.1.581.gf28d330327-goog


-- 
Dmitry
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] RFE and DISCUSSION: Custom commands in systemd services

[Introoter]

Hello,
As discussed in https://github.com/systemd/systemd/issues/6690, there's a need 
to add an option to the specific systemd services to be able to execute 
arbitrary actions in the context of that particular service (think service 
postgres initdb). Although, I do know that the current way to get this working 
isn't too bad either, my making `foobar-action.service` like oneshot units to 
do custom actions and split it into unit files, as these are the basic 
constituent logic units that participate in the dependency graph. Though, there 
are times when a service particularly needs many actions which may After 
reflecting enough on the idea of units, I have something else to add. (I cannot 
post to the mailing list because none of my email providers have a way to send 
plain text mails, though this will be my last comment here, and probably the 
closing one before this is taken to the mailing list for discussion).

THE CURRENT SITUATION:
systemctl mainly conforms to the LSB standard for service actions, and is not 
supposed to support custom actions. Though, again, nothing stops it from 
separating the standard service actions namespace from the generic actions 
namespace.

The way systemd sets up dependency graphs, they're mainly done as dependencies 
between units, participating in a boot transaction graph. Thus, adding custom 
actions would mean there should be a way to depend on these custom defined 
additions, which really does not fit into the working unit model which is a 
really superior way of handling things. The boot is divided into these logical 
units. The better way to do this is to have a foobar-action.service to the 
desired action as a oneshot, but there still is a way to create these custom 
actions as something that have dependencies in the unit itself as I showed 
earlier, which aptly fits the use case where the action is too weak to lie in a 
separate unit (again, very specific to the requirement). This model is not 
perfect and I'll be happy to receive feedback on the ML. I'm also aware that 
this will a lot of changes if we are to add them in the units, as there would 
be change in the parsing engine, and the transaction and dependency graph 
generator. After reflecting enough, it is also possible that the [Action] may 
not quite fit the requirements of the custom action, for example, if it depends 
on external units (which would lie outside of the dependency scope of the 
block), in which case, a separate unit makes more sense.

As an immediate measure, you can use separate units to do your 
graceful-restarts and logrotate and maybe add them as dependencies if required, 
or configure your controlling client (or write a small script) to call these 
units on the desired action, which is very neat tbh.

The current proposal needs some more polishing and *time*need to rely on a 
certain action already strongly typed in the service like ExecStart= or 
ExecStop= (or there Pre and Post) or ExecReload, or even before the stop+start 
logic of restart (in simple words). Here, I am putting up some ideas I had 
about the basic prototype of something that fits this particular need in 
services.

WHAT I HAVE IN MIND:
Something along the lines of systemctl action=logrotate foobar.service 
(actually, the action could lie after the service name so the bash/zsh 
completion script could parse the unit and provide suggestions), and something 
like Name= and then Exec= (and maybe still have the ExecPre=... and 
ExecPost=... options for the Exec= action) in the service files, with these 
custom commands in their own context block, like [Action] to be able to depend 
on the generic ExecStart=, ExecStop=, ExecReload= commands, because then the 
reopening of log files can be subjected with a restart or reload or w/e. (When 
forming dependencies with a certain action, that action itself can have a 
parameter action to call the custom command.) The order of execution will be 
defined by the Before= or After= dependency of the command on the generic 
action.

Example for doing a job:

[Unit]
...

[Service]
...
ExecStart=...
ExecStop=...
ExecReload=...
...

[Action]
Name=logrotate
# queue it before ExecStartPre or ExecStartPost, or After, as required.
Before=ExecStart
# Pre post for the Exec= directive
ExecPre=...
Exec=...
ExecPost=..

and then,

systemctl action=logrotate foobar.service
or
systemctl start action=logrotate foobar.service # would mean the same thing

Support could also be added to queue them as dependencies when being invoked 
from the command line when a dependency isn't specified in the unit or cannot 
be.

systemctl restart before-action=logrotate foobar.service
or
systemctl restart after-action=logrotate foobar.service

This though does have some use cases, there are times when the custom action 
doesn't have to be queued before a generic action, and therefore it could be 
something like this

[Unit]
...

[Service]
...

[Action]
Name=jobfoo
Exec=...
...
and then,
systemctl 

Re: [systemd-devel] Permission/updating problems; different behaviour of two identical nspawn containers

[Lennart Poettering]

On Mi, 30.08.17 17:24, Olaf the Lost Viking (olaf.the.lost.vik...@gmail.com) 
wrote:

> Hi ML,
> 
> 
> currently I am seeing differences between two, what I consider identical, 
> nspawn-containers which prevents me to update one of them. (Lots of) details 
> are at the end of the mail.
> 
> I set up two (hopefully) identical debian containers in nspawn for a single 
> service (DNS) on a debian host. Today's "apt upgrade" now throws permissions 
> problem on _one_ of the containers (ns4 fails, all others still work - ns3 
> should be identical but some service data):

Most likely something went wrong with the userns UID mapping... Not
sure what though...

> As you could see the few lines above, the groups in ns4 aren't correct for 
> certain files/directories. But correcting them in the guest as well as the 
> host fails:
> 
>   root@ns4:/var/cache/apt/archives# ls -l
>   total 0
>   -rw-r- 1 root root   0 Apr 28 22:04 lock
>   drwx-- 1 _apt nogroup 5000 Aug 30 17:01 partial
>   root@ns4:/var/cache/apt/archives# chgrp root partial/
>   chgrp: changing group of 'partial/': Operation not permitted
>   root@ns4:/var/cache/apt/archives#
> 
>   root@HOST:/var/lib/machines/ns4/var/cache/apt/archives# ls -l
>   total 0
>   -rw-r- 1 vu-ns4-0   vg-ns4-00 Apr 28 22:04 lock
>   drwx-- 1 vu-ns4-104 root 5000 Aug 30 17:01 partial
>   root@HOST:/var/lib/machines/ns4/var/cache/apt/archives# chgrp vg-ns4-0 
> _ partial/
>   root@HOST:/var/lib/machines/ns4/var/cache/apt/archives# ls -l
>   total 0
>   -rw-r- 1 vu-ns4-0   vg-ns4-00 Apr 28 22:04 lock
>   drwx-- 1 vu-ns4-104 root 5000 Aug 30 17:01 partial
>   root@HOST:/var/lib/machines/ns4/var/cache/apt/archives#

Are you suggesting that doing this on the host has no effect at all?
That's seriously strange...

When you ran this, was the container running?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Requirements for successful mounting of RootImage?

[Lennart Poettering]

On So, 20.08.17 13:20, Topi Miettinen (toiwo...@gmail.com) wrote:

> >> The file /fs has a MBR partition table:
> >> Disk /dev/loop0: 1.1 MiB, 1192960 bytes, 2330 sectors
> >> Units: sectors of 1 * 512 = 512 bytes
> >> Sector size (logical/physical): 512 bytes / 512 bytes
> >> I/O size (minimum/optimal): 512 bytes / 512 bytes
> >> Disklabel type: dos
> >> Disk identifier: 0x3990f3e6
> >>
> >> Device   Boot Start   End Sectors  Size Id Type
> >> /dev/loop0p1 *   34  23292296  1.1M 83 Linux
> > 
> > That should work. See if "systemd-nspawn -i" can get a shell in it. If
> > so, RootImage= should work too, it uses the same code.
> > 
> > Also, consider invoking /usr/lib/systemd/systemd-dissect on the image
> > file, it will tell you whether it can make sense of the image, and how
> > it would mount it.
> 
> # /lib/systemd/systemd-dissect /root.sqsh
> Found writable 'root' partition of type squashfs without verity
> (/dev/block/7:0)

Hmm, I figure we shouldn't claim "writable" here, given that it is
squashfs, which is r/o

> 
> >> Perhaps I miss some RootImage requirements? What exactly they are?
> > 
> > They are documented briefly in "systemd-nspawn's" --image= setting.
> 
> I tried systemd-nspawn with the image, but that also refuses. There's
> this error:
> # systemd-nspawn --image=/root.sqsh
> Spawning container root.sqsh on /root.sqsh.
> Press ^] three times within 1s to kill container.
> Timezone Europe/Helsinki does not exist in container, not updating
> container timezone.
> Failed to create /var/log: Read-only file system
> 
> It looks like the image is mounted read-only:
> 2427  mkdir("/tmp/nspawn-root-jlYu4k/var/log", 0755) = -1 EROFS
> (Read-only file system)

Yeah, it's squashfs, squashfs is read-only by definition...

If you are using a read-only image you need to populate /var properly,
or you --volatile= in some form... (which is similar to --tmpfs=/var...

> If I add "--tmpfs=/var" and move the mount_custom() call in nspawn.c
> between setup_seccomp() and setup_timezone(), there's no error and
> systemd-nspawn can mount the image and run the command. But it would be
> nice to understand why the image is mounted read-only in the first
> place.

squashfs...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Reliable way to finish system units before termination through systemd shutdown / reboot routines

[Rick Beldin]

On 08/31/2017 10:05 AM, Lennart Poettering wrote:
> I have running some virtualbox instances under systemd control and use 
> vboxautostart-service
> for starting and stopping (saving). This is running on Centos7
> As long as I use systemctl stop vboxautostart-service for termination, 
> everything is fine, 
> but it takes some time for completion.

I'm going to guess that this causes vbox to attempt an acpi shutdown for each
guest.   You may want to monitor the guest console as you do this to verify 
this.

Just a thought...

Rick
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] type=notify not working as intended

[Lennart Poettering]

On Di, 15.08.17 09:29, Kalpa Gunarathna (ka...@sigscale.com) wrote:

> Hello all,
> 
> I have scripted a service unit as follow.
> 
> [Unit]
> Description="A test service"
> After=epmd.service epmd.socket
> 
> [Service]
> ExecStart=/home/otpuser/bin/start
> Type=notify
> NotifyAccess=all
> WorkingDirectory=~
> User=otpuser
> Group=otp
> Restart=always
> RestartSec=3
> 
> [Install]
> WantedBy=multi-user.target
> 
> 
> ExecStart directs to a shell script which starts a Erlang run_erl daemon.
> This daemon is forking a process which is the BEAM virtual machine. In the
> forked process Im sending "READY=1\nSTATUS=Initialized\nMAINPID="
> <> "\n to systemd via sd_notify. The problem is service
> is getting restarted constantly.

When you use Type=notify, then the main process of the daemon defines
the runtime of your service. That means you probably want to use the
shell's "exec" command to replace the shell script with your erlang
daemon. if you instead just fork it off, and exit in the shell script,
then systemd will notice that it exited and will consider the service
dead.

> journalctl give this
> 
> abc.service: Service hold-off time over, scheduling restart.
> Stopped "A test service.".
> 
> 
> Is it required to send READY=1 from the daemon it self ? or can systemd
> allow forked process of the daemon to send REDAY=1 notification?

It needs to come from any process that is witelisted using
NotifyAccess=. Note though that there's a bit of a race: if you send
READY=1 and immediately exit from the process doing that, then systemd
can't properly relate the message to the service (that's a race we
can#t fix without kernel support), except if the process is actually
the one systemd forked off. Hence, sd_notify(READY=1) works reliably only if 
either:

1. it is invoked by the immediate process systemd forked off

or

2. the process is long-running and stays around for longer after
   sending READY=1

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Outputting STDOUT and STDERR to current $SSH_TTY

[Lennart Poettering]

On Di, 15.08.17 09:14, Sergei Franco (sergei.fra...@gmail.com) wrote:

> Please accept my apologies for HTML email (using gmail client).
> 
> For our organisation mid-2016 was the year of systemd, and exposure
> has been minimal (only new builds), so I guess you had a head start. I
> fully grasp the config file layout and overrides (I use them to deal
> with things like LimitNOFILE and ExecStartPost). My comments were from
> perspective of someone recently coming from init scripts. I still
> thinks /lib/ is bad place for system config (should have been another
> directory in /etc/systemd).
> 
> In contrast I had no issues with upstart what so ever.
> 
> In any way, since it is going down towards personal attacks, lets go
> back to original problem:
> 
> How does one convey the service output (stdout/stderr) to the console
> from which user initiated systemctl? I guess consensus is that you
> cannot.
> 
> If so, is it possible to make systemctl to display status after a state 
> change?
> Eg: running systemctl restart apache2 it fires off systemctl status
> apache2 after?

Try "journalctl -u apache2 -f" to get a live view on the stuff apache2
does, and everything systemd does with it.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Is it possible to config rate limit for journal based on severity

[Lennart Poettering]

On Mi, 16.08.17 10:35, P.R.Dinesh (pr.din...@gmail.com) wrote:

> With respect to Journal rate limit (RateLimitIntervalSec=, RateLimitBurst=)
> I would like to rate limit more for low severity logs and less for high
> severity logs.  Is this feasible? also can we have different rate limit
> configured for different services?

This is currently not available. But it sounds like something
worthwile to add. Can you file an RFE issue about this on github?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Outputting STDOUT and STDERR to current $SSH_TTY

[Lennart Poettering]

On Mo, 14.08.17 22:31, Sergei Franco (sergei.fra...@gmail.com) wrote:

> Admins expect output when starting and stopping services.
> This is even more important when there are runbooks and other processes
> involved.
> The journalctl output is very messy (because it is a log). When you are on
> call and you have to deal with an issue at 3am in the morning, the messy
> output adds to the already high cognitive load.
> As system engineers we used to be able write our own init scripts that
> would leave colourized and concise messages about particular sub-services
> of a service.
> This would make it easy for admin on call to determine if everything OK or
> not.

Note that there's "systemctl status" as a quick way to determine the
service status in one go, along with a brief excerpt of the most
recent logs.

> What is even more annoying when start up fails, it just tells you that it
> failed and for more info to see the status or journalctl. Would it not be
> more logical dump the standard error into current tty and status instead of
> pointless message?

There has been a TODO list item for a while to optionally show the log
output of a service during "systemctl start" as it
happens. i.e. "systemctl start -v" or so as a something of a
combination of "sstemctl start" + "journalctl -uf", but so far nobody
implemented that. It's not trivial though, due to the races involved,
i.e. it's hard ot know when precisely to stop waiting for logs, after
we got the start completion notification. This is fixable, but it#s
not trivial.

> There needs to be a setting to pass through stdout and error to whatever
> the tty is used to invoke the systemctl.
> 
> Warning: rant follows!
> 
> I am yet to see a systems engineer that praised systemd. All I hear
> from my

Maybe I live in a bubble, but I have seen quite a few actually...

> colleagues is swearing when they deal with systemd. What makes them angry?
> Inability to actually see what is going on. Has mysql actually failed to
> start or is it repairing tables after a crash? Who knows, systemd ate the
> output. Is the firewall script actually doing what is supposed to do or
> not, who knows, there is no way to pass that information directly without
> systemd hiding it. The only way to catch errors with bash scripts is using
> "set -e" (there is no eye balling it), which means even partial
> functionality now is a failure.

Well, this is deeply unfair.. "systemctl status" shows you ton of
information SysV never was able to show you. Please play around with
it before coming to early conclusions. Yes, it's different than Sysv,
but in many good ways, too.

> Even simple things like where is the bloody unit defined, is it in
> /etc/systemd/system/ or /lib/systemd/system/ or is it in
> /usr/lib/systemd/system/?.

Try "systemctl cat foobar.service" to see where something is defined,
and how exactly.

> What about tail -f 1000 /var/log/messages | grep
> bla | awk .. Can't really do that any more, as journalctl does not support
> number of lines to follow from (and again, I could be wrong here, I have
> been wrong many times).

Yes, you are wrong... "journalctl -f -n 1000" works just fine. Did you
actually play around with systemd at all?

> Why /lib/ for essentially config files? Isn't what /etc/ is for?

They aren't configuration files, they are vendor supplied defaults. If
you want to change them, copy them to /etc and modify them there, they
will then override the vendor defaults. "systemctl edit" does that for
you automatically btw.

> Then we have the whole "predictable" interface naming scheme, with names
> like enp35s0f1. Very "predictable" (they change!) and very "easy" to type
> in middle of night.

Well, then turn them off, if they don#t work on your hardware...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Reading an "ao" signature message with the sdbus c library?

[Lennart Poettering]

On Do, 17.08.17 13:43, James Johnson (jjjwallyh...@gmail.com) wrote:

> Hello,
> 
> Can someone please point me to the correct method for reading an "ao"
> signature dbus message using the sd_bus C api?
> 
> I am trying to read all the network devices with the following call:
> 
>   result = sd_bus_get_property(m_bus, "org.freedesktop.NetworkManager",
>  "/org/freedesktop/NetworkManager",
>  "org.freedesktop.NetworkManager",
>  "AllDevices", ,
>  , "ao");
> I have tried:
> 
> char * ptr;
> size_t size;
> 
> result = sd_bus_message_read_array(msgCall, 'o', , );

_read_array() only works for trivial types, not for strings and
other variable length types. If you have such complex types you need
to enter the array with _enter_container() and then _read() each item
individually...

But I see you already figure that out...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] systemd-logind fails to start

[Lennart Poettering]

On Mo, 14.08.17 10:09, Robin Becker (ro...@reportlab.com) wrote:

> After a recent update I started having problems logging in.
> 
> It seems systemd-logind is timing out for some reason.
> 
> I see this  in the journalctl -xe output
> 
> 
> 
> > -- Unit systemd-logind.service has begun starting up.
> > Aug 14 09:54:27 delilah systemd-logind[484]: Failed to enable subscription: 
> > Connection timed out
> > Aug 14 09:54:27 delilah systemd-logind[484]: Failed to fully start
> > up daemon: Connection timed out

That looks like dbus-daemon is not running properly. You should
probably check that. Maybe just invoke "busctl" which queries the bus
for running names. If it succeeds the bus is properly up...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Reliable way to finish system units before termination through systemd shutdown / reboot routines

[Lennart Poettering]

On Di, 15.08.17 14:25, Reiner Wenke (reiner.we...@nxp.com) wrote:

> Hello,
> 
> I have running some virtualbox instances under systemd control and use 
> vboxautostart-service
> for starting and stopping (saving). This is running on Centos7
> As long as I use systemctl stop vboxautostart-service for termination, 
> everything is fine, 
> but it takes some time for completion.
> 
> If I shutdown or reboot the host system, then I get a premature kill of all 
> running instances,
> most likely through the systemd reboot / shutdown routines .
> I tried some configuration changes, but nothing worked for me.
> 
> Is there a reliable way to avoid this behavior?

At shutdown systemd will terminate all services in a similar way as
"systemctl stop" does it. I don#t really know vbox though, maybe they
are not setting everything up correctly there. Consider contacting
your downstream distro or the vbox community for help on this.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel