Re: [systemd-devel] Starting one service when another one starts

2022-04-08 Thread Andrei Borzenkov
On 08.04.2022 23:35, Nick Howitt wrote:
> Sorry, for the delay. Big internet outage.
> 
> On 08/04/2022 15:15, Andrei Borzenkov wrote:
>>
>> On 08.04.2022 14:54, Nick Howitt wrote:
>>> Hi,
>>> I apologise if this is not the right place for user help. If it is not,
>>> please point me to the best place.
>>>
>>> I am trying to start a service (clearshare-scheduler) when another
>>> service (siad) starts. Clearshare-scheduler is an odd service. When you
>>> start it it may run for ages (days+) or it may terminate immediately so
>>> I have set it up as a oneshot:
>>>
> clearshare-scheduler.service
>>> [Unit]
>>> Description=Clearshare Scheduler
>>> PartOf=siad.service
>>> After=siad.service
>>>
>>> [Service]
>>> Type=oneshot
>>> Environment="TERM=dumb"
>>> ExecStartPre=-/usr/bin/killall -15 -q /usr/sbin/clearshare-scheduler.sh
>>> ExecStartPre=-/usr/bin/echo "$(/usr/bin/date) Starting scheduler from
>>> systemd" >> /var/log/scheduler.log
>>> ExecStart=/usr/sbin/clearshare-scheduler.sh > /dev/null
>>> ExecStop=-/usr/bin/killall -15 -q /usr/sbin/clearshare-scheduler.sh
>>>
>>> [Install]
>>> WantedBy=multi-user.target
>>>
>>> The siad service looks like:
>>>
> siad.service
>>> [Unit]
>>> Description=Siad
>>> After=syslog.target network.target clearsync.service
>>>
>>> [Service]
>>> Type=simple
>>> OOMScoreAdjust=500
>>> PIDFile=/var/run/siad.pid
>>> EnvironmentFile=/etc/sysconfig/siad
>>> Environment="SIA_DATA_DIR=/var/lib/siad-data"
>>> ExecStartPre=-/usr/bin/killall -15 -q clearshare-scheduler.sh
>>> ExecStartPre=-/usr/bin/rm -f /var/run/siad.pid
>>> ExecStart=/usr/bin/siad $EXTRA_ARGS
>>> ExecStop=/usr/bin/siac stop
>>> WorkingDirectory=/var/lib/sia/
>>> ExecStartPost=/usr/bin/sh -c 'umask 022; /usr/bin/pgrep siad >
>>> /var/run/siad.pid'
>>>
>>> [Install]
>>> WantedBy=multi-user.target
>>>
>>
>> You do not show actual unit names which makes it rather difficult to follow.
> Done. See above
>>
>>> A "systemctl show clearshare-scheduler" lists the PartOf=siad.service as
>>> one of its properties but, in reverse, "systemctl show siad" does not
>>> list the corresponding ConsistsOf property.
>>>
>>> When I start siad, nothing happens to the clearshare-scheduler service.
>>
>> Why do you expect to happen? There is no Wants or Requires in the unit
>> that is /probably/ siad.service so request to start siad.service will
>> not pull in any additional units.
> Perhaps I have misunderstood, but from the documentation I understood 
> you could PartOf in force (in this case) clearshare-scheduler.service to 
> respond when siad.service was stopped or started. Have I misunderstood 
> the docs? I am hoping to not do any changes to the siad.service.
> 

Documentation for PartOf says "limited to stopping and restarting of
units". Nothing about "starting". PartOf complements normal startup
dependencies, not replaces them. And yes, this is confusing, as are the
names of almost any systemd dependency which mean something entirely
different from what these names imply in English.

You can add WantedBy=siad.service to [Install] section of
clearshare-scheduler.service. In general you can always extend Wants by
manually creating necessary links. This does not require you to edit
unit definition itself. You can also create drop-in (although I am not
sure whether they are already supported in your systemd version).

> As an alternative, which does affect the siad.service, is there any way 
> I can run the clearshare-scheduler.sh script from the siad.service? I 
> have tried starting it as a ExecStartPost, but it does not appear to 
> work if the script does not exit immediately. If it runs for a while, 
> then systemd says siad has failed to start.

You can increase TimeoutStartSec.

> I've tried launching it with
> ExecStartPost=-/usr/sbin/clearshare-scheduler.sh

"-" affects command that completed with failure status, in your case
command does not complete so this does not have any effect.

> and
> ExecStartPost=-/usr/sbin/clearshare-scheduler.sh &
> and
> ExecStartPost=-/usr/bin/nohup /usr/sbin/clearshare-scheduler.sh &
> 

sytsemd is not shell, what made you think this would work? If you want
to use shell syntax, you need to invoke shell

/bin/sh -c "/usr/sbin/clearshare-scheduler.sh &"

>>
>>> It does not try to start but it runs when I run it on its own. Have I
>>> misunderstood something?
>>>
>>> My version of systems is systemd-219-78.el7_9.5.
>>>
>>> Thanks,
>>>
>>> Nick
>>



Re: [systemd-devel] Starting one service when another one starts

2022-04-08 Thread Nick Howitt

Sorry, for the delay. Big internet outage.

On 08/04/2022 15:15, Andrei Borzenkov wrote:


On 08.04.2022 14:54, Nick Howitt wrote:

Hi,
I apologise if this is not the right place for user help. If it is not,
please point me to the best place.

I am trying to start a service (clearshare-scheduler) when another
service (siad) starts. Clearshare-scheduler is an odd service. When you
start it it may run for ages (days+) or it may terminate immediately so
I have set it up as a oneshot:


clearshare-scheduler.service

[Unit]
Description=Clearshare Scheduler
PartOf=siad.service
After=siad.service

[Service]
Type=oneshot
Environment="TERM=dumb"
ExecStartPre=-/usr/bin/killall -15 -q /usr/sbin/clearshare-scheduler.sh
ExecStartPre=-/usr/bin/echo "$(/usr/bin/date) Starting scheduler from
systemd" >> /var/log/scheduler.log
ExecStart=/usr/sbin/clearshare-scheduler.sh > /dev/null
ExecStop=-/usr/bin/killall -15 -q /usr/sbin/clearshare-scheduler.sh

[Install]
WantedBy=multi-user.target

The siad service looks like:


siad.service

[Unit]
Description=Siad
After=syslog.target network.target clearsync.service

[Service]
Type=simple
OOMScoreAdjust=500
PIDFile=/var/run/siad.pid
EnvironmentFile=/etc/sysconfig/siad
Environment="SIA_DATA_DIR=/var/lib/siad-data"
ExecStartPre=-/usr/bin/killall -15 -q clearshare-scheduler.sh
ExecStartPre=-/usr/bin/rm -f /var/run/siad.pid
ExecStart=/usr/bin/siad $EXTRA_ARGS
ExecStop=/usr/bin/siac stop
WorkingDirectory=/var/lib/sia/
ExecStartPost=/usr/bin/sh -c 'umask 022; /usr/bin/pgrep siad >
/var/run/siad.pid'

[Install]
WantedBy=multi-user.target



You do not show actual unit names which makes it rather difficult to follow.

Done. See above



A "systemctl show clearshare-scheduler" lists the PartOf=siad.service as
one of its properties but, in reverse, "systemctl show siad" does not
list the corresponding ConsistsOf property.

When I start siad, nothing happens to the clearshare-scheduler service.


Why do you expect to happen? There is no Wants or Requires in the unit
that is /probably/ siad.service so request to start siad.service will
not pull in any additional units.
Perhaps I have misunderstood, but from the documentation I understood 
you could PartOf in force (in this case) clearshare-scheduler.service to 
respond when siad.service was stopped or started. Have I misunderstood 
the docs? I am hoping to not do any changes to the siad.service.


As an alternative, which does affect the siad.service, is there any way 
I can run the clearshare-scheduler.sh script from the siad.service? I 
have tried starting it as a ExecStartPost, but it does not appear to 
work if the script does not exit immediately. If it runs for a while, 
then systemd says siad has failed to start. I've tried launching it with

ExecStartPost=-/usr/sbin/clearshare-scheduler.sh
and
ExecStartPost=-/usr/sbin/clearshare-scheduler.sh &
and
ExecStartPost=-/usr/bin/nohup /usr/sbin/clearshare-scheduler.sh &




It does not try to start but it runs when I run it on its own. Have I
misunderstood something?

My version of systems is systemd-219-78.el7_9.5.

Thanks,

Nick




Re: [systemd-devel] Intercepting/Delaying the boot process

2022-04-08 Thread Andreas Hartmann
On Fri, 2022-04-08 at 15:14 +0200, Lennart Poettering wrote:
> On Fr, 08.04.22 09:01, Andreas Hartmann (har...@7x.de) wrote:
> > To this end I was wondering whether it would be feasible to "hook" into the
> > boot
> > process, somewhere before the disks are decrypted, to only have it charge 
> > (or
> > continue to boot when I press some magic button maybe).
> 
> The way I'd implement this would be via a generator. See
> https://www.freedesktop.org/software/systemd/man/systemd.generator.html
> for details on that.
> 
> Generators are in a way "plugins" to systemd's unit graph logic. They
> can insert units and dependencies into the tree from external
> sources. They can also override the regular tree. So here's what I
> would specifically do: write a tiny generator that figures out if it
> is in this charging mode. And if so, it overrides the "default.target"
> symlink so that it points to a new target "charge-mode.target" or
> so. Note that "default.target" is the default target the system will
> boot into. Thus with this generator you made sure that if this charge
> mode is selected only a tiny subset of units can be activated. Now,
> add in a service into "charge-mode.target" that brings some nice
> graphics to screen, and then just waits. If eventually something
> happens so that the system shall power up fully, you can do the
> equivalent of "systemctl isolate graphical.target" so that you boot up
> into the regular system again.

Great, that indeed sounds much better than delaying the boot process!

> 
> > - After which stage in the boot process is the sysfs available?
> 
> sysfs is mounted by very early PID 1 code. It's basically always there.
> 
> > - Can I delay the boot for an indefinite amount of time, or will this cause
> > some
> > services later on in the process to timeout and fail?
> 
> You can, but as mentioned I'd rather redirect boot to a separate
> target, see above.
> 
> > - Is it possible for a service during early boot to command a system 
> > shutdown
> > instead of continuing to boot?
> 
> Sure.
> 
> > - May I simply take control of the TTY and clear/rewrite it as I like, or
> > does
> > systemd use some magic for this?
> 
> Use StandardInput=tty and your program will be started once there is
> noone else who is the controller of the TTY.
> 
> This reminds of an RFE we have had for a while, and which I think
> would make sense to add directly to systemd: a generator that detects
> whether the battery is nearly empty at boot, and if so redirects boot
> to some special target showing a nice message for a bit and then
> powering off.
> 
> How do you intend to discern "power-up due to charger plugged in" from
> "power-up due to power button pressed" btw? Is there generic infra for
> discerning that? Or just something pinephone specific thing? Ideally
> we had some generic infra for that in the kernel.
> 
> (On PCs there's a field for that in DMI called "Wake-up Type". I
> wonder how well that works these days.)

Right now: I have no idea how I'll do that. The pinephones PMIC at least is able
to discern these two power-on reasons and they can be read out from internal
registers. I had a quick glance at the kernel driver for the IC and the 
registers
are at least mentioned there, but that's all I know so far. I don't know whether
this is exposed to sysfs somewhere related to the PMIC, or whether the kernel 
has
its own way of collecting such information and (not) telling me... I get a
feeling this project will keep me busy for quite some time.

Don't take this as a real commitment, but maybe I'll start with the "battery 
low,
turning off" thingy to get a feeling for the generators *before* I start adding
such hardware specifics as power-on reasons. I'm curious where this will lead...

Thank you for all the hints!


hartan

> 
> Lennart
> 
> --
> Lennart Poettering, Berlin


Re: [systemd-devel] Starting one service when another one starts

2022-04-08 Thread Andrei Borzenkov
On 08.04.2022 14:54, Nick Howitt wrote:
> Hi,
> I apologise if this is not the right place for user help. If it is not, 
> please point me to the best place.
> 
> I am trying to start a service (clearshare-scheduler) when another 
> service (siad) starts. Clearshare-scheduler is an odd service. When you 
> start it it may run for ages (days+) or it may terminate immediately so 
> I have set it up as a oneshot:
> 
> [Unit]
> Description=Clearshare Scheduler
> PartOf=siad.service
> After=siad.service
> 
> [Service]
> Type=oneshot
> Environment="TERM=dumb"
> ExecStartPre=-/usr/bin/killall -15 -q /usr/sbin/clearshare-scheduler.sh
> ExecStartPre=-/usr/bin/echo "$(/usr/bin/date) Starting scheduler from 
> systemd" >> /var/log/scheduler.log
> ExecStart=/usr/sbin/clearshare-scheduler.sh > /dev/null
> ExecStop=-/usr/bin/killall -15 -q /usr/sbin/clearshare-scheduler.sh
> 
> [Install]
> WantedBy=multi-user.target
> 
> The siad service looks like:
> 
> [Unit]
> Description=Siad
> After=syslog.target network.target clearsync.service
> 
> [Service]
> Type=simple
> OOMScoreAdjust=500
> PIDFile=/var/run/siad.pid
> EnvironmentFile=/etc/sysconfig/siad
> Environment="SIA_DATA_DIR=/var/lib/siad-data"
> ExecStartPre=-/usr/bin/killall -15 -q clearshare-scheduler.sh
> ExecStartPre=-/usr/bin/rm -f /var/run/siad.pid
> ExecStart=/usr/bin/siad $EXTRA_ARGS
> ExecStop=/usr/bin/siac stop
> WorkingDirectory=/var/lib/sia/
> ExecStartPost=/usr/bin/sh -c 'umask 022; /usr/bin/pgrep siad > 
> /var/run/siad.pid'
> 
> [Install]
> WantedBy=multi-user.target
> 

You do not show actual unit names which makes it rather difficult to follow.

> A "systemctl show clearshare-scheduler" lists the PartOf=siad.service as 
> one of its properties but, in reverse, "systemctl show siad" does not 
> list the corresponding ConsistsOf property.
> 
> When I start siad, nothing happens to the clearshare-scheduler service. 

Why do you expect to happen? There is no Wants or Requires in the unit
that is /probably/ siad.service so request to start siad.service will
not pull in any additional units.

> It does not try to start but it runs when I run it on its own. Have I 
> misunderstood something?
> 
> My version of systems is systemd-219-78.el7_9.5.
> 
> Thanks,
> 
> Nick



Re: [systemd-devel] Intercepting/Delaying the boot process

2022-04-08 Thread Lennart Poettering
On Fr, 08.04.22 09:01, Andreas Hartmann (har...@7x.de) wrote:

> Hi,
>
>
> I've been owning a Pinephone for a while now and one thing that I find rather
> annoying is that whenever I plug the charger in, the thing will perform a full
> boot. I often find myself just wanting to plug it in without performing a full
> boot, i.e. only to have it charge and maybe see how far it has charged 
> already.
>
> To this end I was wondering whether it would be feasible to "hook" into the 
> boot
> process, somewhere before the disks are decrypted, to only have it charge (or
> continue to boot when I press some magic button maybe).

The way I'd implement this would be via a generator. See
https://www.freedesktop.org/software/systemd/man/systemd.generator.html
for details on that.

Generators are in a way "plugins" to systemd's unit graph logic. They
can insert units and dependencies into the tree from external
sources. They can also override the regular tree. So here's what I
would specifically do: write a tiny generator that figures out if it
is in this charging mode. And if so, it overrides the "default.target"
symlink so that it points to a new target "charge-mode.target" or
so. Note that "default.target" is the default target the system will
boot into. Thus with this generator you made sure that if this charge
mode is selected only a tiny subset of units can be activated. Now,
add in a service into "charge-mode.target" that brings some nice
graphics to screen, and then just waits. If eventually something
happens so that the system shall power up fully, you can do the
equivalent of "systemctl isolate graphical.target" so that you boot up
into the regular system again.

> - After which stage in the boot process is the sysfs available?

sysfs is mounted by very early PID 1 code. It's basically always there.

> - Can I delay the boot for an indefinite amount of time, or will this cause 
> some
> services later on in the process to timeout and fail?

You can, but as mentioned I'd rather redirect boot to a separate
target, see above.

> - Is it possible for a service during early boot to command a system shutdown
> instead of continuing to boot?

Sure.

> - May I simply take control of the TTY and clear/rewrite it as I like, or does
> systemd use some magic for this?

Use StandardInput=tty and your program will be started once there is
noone else who is the controller of the TTY.

This reminds of an RFE we have had for a while, and which I think
would make sense to add directly to systemd: a generator that detects
whether the battery is nearly empty at boot, and if so redirects boot
to some special target showing a nice message for a bit and then
powering off.

How do you intend to discern "power-up due to charger plugged in" from
"power-up due to power button pressed" btw? Is there generic infra for
discerning that? Or just something pinephone specific thing? Ideally
we had some generic infra for that in the kernel.

(On PCs there's a field for that in DMI called "Wake-up Type". I
wonder how well that works these days.)

Lennart

--
Lennart Poettering, Berlin


Re: [systemd-devel] Samba Config Reload

2022-04-08 Thread Lennart Poettering
On Do, 07.04.22 15:38, Kenneth Porter (sh...@sewingwitch.com) wrote:

> --On Thursday, April 07, 2022 12:30 PM +0200 Lennart Poettering
>  wrote:
>
> > The other two options are likely similar, i.e. synchronous and talk to
> > smbd directly. But I don't know samba that well, so it's just an
> > assumption. In fact, if ExecStop= in smbd.service just calls the
> > smbcontrol they behave very very similar.
>
> FWIW, here's the unit file from samba-4.10.16-13 in CentOS 7.9:
>
…
> ExecReload=/bin/kill -HUP $MAINPID

This could be done better. Plugging in just a "kill" here, means the
reload is async. i.e. "systemctl reload" will basically return
immediately without the reload being complete, thus subsequent
commands can't rely the new config is already in place.

It's typically nicer to invoke some synchronous command from
ExecReload=.

Lennart

--
Lennart Poettering, Berlin


Re: [systemd-devel] Intercepting/Delaying the boot process

2022-04-08 Thread Mantas Mikulėnas
On Fri, Apr 8, 2022, 12:44 Andreas Hartmann  wrote:

> On Fri, 2022-04-08 at 12:07 +0300, Mantas Mikulėnas wrote:
> > On Fri, Apr 8, 2022 at 10:08 AM Andreas Hartmann  wrote:
> >
> > >
> > > To this end I was wondering whether it would be feasible to "hook" into
> > > the boot
> > > process, somewhere before the disks are decrypted, to only have it
> charge
> > > (or
> > > continue to boot when I press some magic button maybe). Looking at the
> > > order of
> > > service startup I was thinking about maybe intercepting the boot
> between
> > > "local-
> > > fs-pre.target" and "machines.target" because nothing happens there on
> my
> > > setup.
> > >
> >
> > That "between" doesn't make sense, because the boot process waits for
> > machines.target *in parallel* with waiting for all other services that
> are
> > started via multi-user. The systemd boot process is non-linear and
> doesn't
> > have a fixed order, it's a dependency graph.
> >
> > According to bootup(7), there are two main synchronization points,
> > sysinit.target and basic.target, which all non-"early" services will wait
> > for. So if you have a service with Before=basic.target, it'll delay
> startup
> > of all normal services. (It would then need to explicitly specify After=
> > for things it needs, like specific devices or sockets.)
>
> Good hint, thank you!
>
> >
> > If the device was using an initramfs (which has a fully separate boot
> > process, its own udevd, etc.) then you could make the initramfs decide
> > between starting a normal boot vs minimal boot depending on charger
> status.
> > If it doesn't have an initramfs, then systemd *generators* could also be
> > used to dynamically swap default.target (or alter units in general)
> > depending on some condition, as they also run after /sys is mounted but
> > before any units at all are started... though this would only work if
> your
> > wanted sysfs entries appear without needing udev.
>
> Interesting, I wasn't aware such things could be performed. About udev:
> The way I
> understand it, udev is mostly involved in dynamic device probing or
> managing/reacting to device events. Is that correct?
>

It doesn't probe devices (the kernel does that), but it does react to the
kernel's "device detected" events – loads additional kernel modules (e.g.
if a USB device with specific vid:pid was probed, udev loads the
corresponding .ko module), forwards events to userspace (e.g. to systemd),
etc.


> In my particular scenario, I have a complete devicetree that contains the
> nodes
> for all the hardware I need. So from my understanding that's enough for the
> kernel to load the drivers et al, or am I mistaken?
>

DT tells the kernel what devices exist without it needing to e.g. scan a
bus (which the kernel would do on its own anyway – that's not udev's job),
but it *doesn't* tell the kernel what .ko modules to load for them (that's
udev's job).

So if all necessary drivers are built-in, great. But if some of them are
modular, udev is usually still needed, whether the device was discovered
through DT or ACPI or something else.


> > > - After which stage in the boot process is the sysfs available?
> > >
> >
> > It is mounted by init before any units are started. (sysfs *is* how you
> > interact with the kernel though...)
>
> That will make my life a lot easier. I really meant I don't want to
> program the
> syscalls to the kernel myself etc.
>

There aren't any other syscalls that would do what /sys does, anyway. The
file-based interface (i.e. open/read/write syscalls on /sys) is often the
*only* interface.


> >
> > But that doesn't mean all *devices* are available in it – many things
> could
> > take some time to appear, and devices requiring drivers to be loaded as
> > modules would only become available some unspecified time after
> > systemd-udevd.service is started.
>
> But isn't really any device driver a module?


No, they can be compiled into the main kernel image as well (and on
embedded systems, most likely they all are, but...still). "Module"
specifically means a separate .ko file in /lib/modules, as opposed to the
driver being literally part of /vmlinuz. Usually the kernel has some mix of
built-in and modular drivers.

If udev is responsible for loading
> the kernel modules (drivers) then that would mean I can't access any
> device at
> this stage, right?
> I thought that statically defining the devices and their drivers in the
> devicetree makes them available without ''extra effort''.
>

The device tree doesn't specify Linux drivers directly, there's still a
layer of translating the abstract DT 'compatible' strings (or
vendor/product IDs, or ACPI IDs, or whatever) to Linux-specific paths. For
built-in drivers, that mapping is indeed also built in to the kernel, but
for modular ones it's in /lib/.../modules.alias and has to be read by
udev/libkmod, and then the corresponding .ko file has to be given to the
kernel.


> >
> > If you need a specific /sys or /dev device that's not 

Re: [systemd-devel] Intercepting/Delaying the boot process

2022-04-08 Thread Andreas Hartmann
On Fri, 2022-04-08 at 12:07 +0300, Mantas Mikulėnas wrote:
> On Fri, Apr 8, 2022 at 10:08 AM Andreas Hartmann  wrote:
> 
> > 
> > To this end I was wondering whether it would be feasible to "hook" into
> > the boot
> > process, somewhere before the disks are decrypted, to only have it charge
> > (or
> > continue to boot when I press some magic button maybe). Looking at the
> > order of
> > service startup I was thinking about maybe intercepting the boot between
> > "local-
> > fs-pre.target" and "machines.target" because nothing happens there on my
> > setup.
> > 
> 
> That "between" doesn't make sense, because the boot process waits for
> machines.target *in parallel* with waiting for all other services that are
> started via multi-user. The systemd boot process is non-linear and doesn't
> have a fixed order, it's a dependency graph.
> 
> According to bootup(7), there are two main synchronization points,
> sysinit.target and basic.target, which all non-"early" services will wait
> for. So if you have a service with Before=basic.target, it'll delay startup
> of all normal services. (It would then need to explicitly specify After=
> for things it needs, like specific devices or sockets.)

Good hint, thank you!

> 
> If the device was using an initramfs (which has a fully separate boot
> process, its own udevd, etc.) then you could make the initramfs decide
> between starting a normal boot vs minimal boot depending on charger status.
> If it doesn't have an initramfs, then systemd *generators* could also be
> used to dynamically swap default.target (or alter units in general)
> depending on some condition, as they also run after /sys is mounted but
> before any units at all are started... though this would only work if your
> wanted sysfs entries appear without needing udev.

Interesting, I wasn't aware such things could be performed. About udev: The way 
I
understand it, udev is mostly involved in dynamic device probing or
managing/reacting to device events. Is that correct?

In my particular scenario, I have a complete devicetree that contains the nodes
for all the hardware I need. So from my understanding that's enough for the
kernel to load the drivers et al, or am I mistaken?

> > - After which stage in the boot process is the sysfs available?
> > 
> 
> It is mounted by init before any units are started. (sysfs *is* how you
> interact with the kernel though...)

That will make my life a lot easier. I really meant I don't want to program the
syscalls to the kernel myself etc.

> 
> But that doesn't mean all *devices* are available in it – many things could
> take some time to appear, and devices requiring drivers to be loaded as
> modules would only become available some unspecified time after
> systemd-udevd.service is started.

But isn't really any device driver a module? If udev is responsible for loading
the kernel modules (drivers) then that would mean I can't access any device at
this stage, right?
I thought that statically defining the devices and their drivers in the
devicetree makes them available without ''extra effort''.

> 
> If you need a specific /sys or /dev device that's not guaranteed to be
> available statically, the correct way would be to use an After= dependency
> on that specific device (like After=dev-sda.device). Systemd relies on udev
> events for this.
> 
> 
> > - Can I delay the boot for an indefinite amount of time, or will this
> > cause some
> > services later on in the process to timeout and fail?
> > 
> 
> In theory you could, similar to how systemd-fsck works, but I'm not sure if
> that's the right way to do what you want.

Likely not, but I think it's the most simple to wrap my head around for 
starters.
I'll do some more research about generators and when my devices become available
in the sysfs.

Thank you very much!


hartan




Re: [systemd-devel] Antw: [EXT] Dropping split-usr/unmerged-usr support

2022-04-08 Thread Luca Boccassi
On Thu, 2022-04-07 at 10:39 +0100, Luca Boccassi wrote:
> On Wed, 2022-04-06 at 08:39 -0400, Neal Gompa wrote:
> > On Wed, Apr 6, 2022 at 8:07 AM Luca Boccassi  
> > wrote:
> > > 
> > > On Wed, 2022-04-06 at 06:51 -0400, Neal Gompa wrote:
> > > > On Wed, Apr 6, 2022 at 6:45 AM Luca Boccassi  
> > > > wrote:
> > > > > 
> > > > > On Wed, 2022-04-06 at 08:05 +0200, Ulrich Windl wrote:
> > > > > > > > > Luca Boccassi  schrieb am 05.04.2022
> > > > > > > > > um 22:07 in
> > > > > > Nachricht 
> > > > > > <05cf10d04274dcbff07fed88e98dca2eebb24b7d.ca...@gmail.com>:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > As part of our spring cleaning effort, we are considering when to
> > > > > > > drop
> > > > > > > support for split/unmerged-usr filesystem layouts.
> > > > > > > 
> > > > > > > A build-time warning was added last year:
> > > > > > > 
> > > > > > > https://github.com/systemd/systemd/commit/9afd5e7b975e8051c011ff9c07c95e80bd
> > > > > > > 954469
> > > > > > 
> > > > > > Honestly to me the requirement that /usr be part of the root
> > > > > > filesystem never had a reasonable argument.
> > > > > > Instead I think systemd quit the concept of a simple scaled-down
> > > > > > subset to bring up the system.
> > > > > > Also with initrd/dracut the concept is even more odd, because the
> > > > > > /usr found there is just some arbitrary subset of the real /usr
> > > > > > (similar for other filesystems).
> > > > > > So why couldn't that work with a really scaled-down /sbin?
> > > > > > 
> > > > > > > 
> > > > > > > We are now adding a runtime taint as well.
> > > > > > > 
> > > > > > > Which distributions are left running with systemd on a
> > > > > > > split/unmerged-
> > > > > > > usr system?
> > > > > > > 
> > > > > > > (reminder: we refer to a system that boots without a populated 
> > > > > > > /usr
> > > > > > > as
> > > > > > > split-usr, and a system where bin, sbin and lib* are not symlinks
> > > > > > > to
> > > > > > > their counterparts under /usr as unmerged-usr)
> > > > > > 
> > > > > > Symlinking /sbin or /usr/sbin binaries to /usr is also a bad concept
> > > > > > IMHO.
> > > > > > 
> > > > > > It seems systemd is the new Microsoft ("We know what is good for 
> > > > > > you;
> > > > > > just accept it!") ;-)
> > > > > > 
> > > > > > Regards,
> > > > > > Ulrich
> > > > > 
> > > > > Sorry, but you are about ~10 years late to this debate :-) The 
> > > > > question
> > > > > today is not whether it's good or bad, but who's left to do the 
> > > > > switch.
> > > > > 
> > > > > We know Fedora/RHEL/CentOS/SUSE/Arch/Ubuntu have done the switch, and
> > > > > presumably any of their derivatives.
> > > > > 
> > > > > We know Debian is, er, working on it, as per the most recent article 
> > > > > on
> > > > > LWN.
> > > > > 
> > > > 
> > > > Debian is expected to complete this with Debian 12, I believe.
> > > 
> > > Yeah it's, uhm, complicated :-) Working on it...
> > > 
> > > > > What about other distros that are not derivatives of the aboves and
> > > > > that use systemd? Does anybody have any insight?
> > > > > 
> > > > 
> > > > OpenMandriva and Yocto both haven't done the switch yet, as far as I'm
> > > > aware. Might be worth reaching out to them and finding out when
> > > > they're going to do it.
> > > 
> > > Thanks, I'm not familiar with OpenMandriva at all, is anyone here? Any
> > > pointers on where to reach out to?
> > > 
> > 
> > You could try filing an issue here:
> > https://github.com/OpenMandrivaAssociation/distribution
> > 
> > Alternatively, I believe Bernhard Rosenkraenzer (berolinux on GitHub)
> > is someone to reach out to. He does a lot of OpenMandriva
> > architectural work.
> 
> Thank you, opened an issue:
> 
> https://github.com/OpenMandrivaAssociation/distribution/issues/2792

Good news, the migration to usr-merged is scheduled for OpenMandriva
5.0:

https://github.com/OpenMandrivaAssociation/distribution/issues/2792#issuecomment-1092605536

-- 
Kind regards,
Luca Boccassi


signature.asc
Description: This is a digitally signed message part


Re: [systemd-devel] Intercepting/Delaying the boot process

2022-04-08 Thread Mantas Mikulėnas
On Fri, Apr 8, 2022 at 10:08 AM Andreas Hartmann  wrote:

> Hi,
>
>
> I've been owning a Pinephone for a while now and one thing that I find
> rather
> annoying is that whenever I plug the charger in, the thing will perform a
> full
> boot. I often find myself just wanting to plug it in without performing a
> full
> boot, i.e. only to have it charge and maybe see how far it has charged
> already.
>
> To this end I was wondering whether it would be feasible to "hook" into
> the boot
> process, somewhere before the disks are decrypted, to only have it charge
> (or
> continue to boot when I press some magic button maybe). Looking at the
> order of
> service startup I was thinking about maybe intercepting the boot between
> "local-
> fs-pre.target" and "machines.target" because nothing happens there on my
> setup.
>

That "between" doesn't make sense, because the boot process waits for
machines.target *in parallel* with waiting for all other services that are
started via multi-user. The systemd boot process is non-linear and doesn't
have a fixed order, it's a dependency graph.

According to bootup(7), there are two main synchronization points,
sysinit.target and basic.target, which all non-"early" services will wait
for. So if you have a service with Before=basic.target, it'll delay startup
of all normal services. (It would then need to explicitly specify After=
for things it needs, like specific devices or sockets.)

If the device was using an initramfs (which has a fully separate boot
process, its own udevd, etc.) then you could make the initramfs decide
between starting a normal boot vs minimal boot depending on charger status.
If it doesn't have an initramfs, then systemd *generators* could also be
used to dynamically swap default.target (or alter units in general)
depending on some condition, as they also run after /sys is mounted but
before any units at all are started... though this would only work if your
wanted sysfs entries appear without needing udev.


> For this to work ideally I would like to have the sysfs available so I
> don't have
> to interact with the kernel directly. So here are my questions:


> - After which stage in the boot process is the sysfs available?
>

It is mounted by init before any units are started. (sysfs *is* how you
interact with the kernel though...)

But that doesn't mean all *devices* are available in it – many things could
take some time to appear, and devices requiring drivers to be loaded as
modules would only become available some unspecified time after
systemd-udevd.service is started.

If you need a specific /sys or /dev device that's not guaranteed to be
available statically, the correct way would be to use an After= dependency
on that specific device (like After=dev-sda.device). Systemd relies on udev
events for this.


> - Can I delay the boot for an indefinite amount of time, or will this
> cause some
> services later on in the process to timeout and fail?
>

In theory you could, similar to how systemd-fsck works, but I'm not sure if
that's the right way to do what you want.


> - Is it possible for a service during early boot to command a system
> shutdown
> instead of continuing to boot?
> - May I simply take control of the TTY and clear/rewrite it as I like, or
> does
> systemd use some magic for this?
>
> Thank you in advance!
>
>
> hartan
>
>
>

-- 
Mantas Mikulėnas


Re: [systemd-devel] Antw: [EXT] Re: Dropping split-usr/unmerged-usr support

2022-04-08 Thread Lennart Poettering
On Fr, 08.04.22 10:26, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote:

> I always thought it was a distro-hostile systemd ;-)

Ulrich, that's not constructive, and you keep posting stuff like that
the last days here. Not the right forum. Stop that. Or I'll put you on
moderation.

Lennart

--
Lennart Poettering, Berlin


Re: [systemd-devel] Antw: [EXT] Re: [systemd‑devel] Dropping split‑usr/unmerged‑usr support

2022-04-08 Thread Lennart Poettering
On Fr, 08.04.22 10:11, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote:

> > So, my guess would be that the people who dislike merged‑/usr are also
> > the ones who dislike systemd, no? i.e. do they really matter if we are
> > talking about what to support in systemd? They'd not use our stuff
> > anyway, so why bother?
>
> Maybe explain why systemd required /usr to be part of the root filesystem
> instead.

We do not require that. What we do require is that /usr/ is mounted
during the initrd → host transition.

> Maybe also explain the chicken-egg race with temporary directories that
> systemd added.

Hm?

> Maybe just add a manual page systemd-filesystem-concepts ;-)

https://www.freedesktop.org/software/systemd/man/file-hierarchy.html

Lennart

--
Lennart Poettering, Berlin


[systemd-devel] Antw: [EXT] Re: Dropping split-usr/unmerged-usr support

2022-04-08 Thread Ulrich Windl
>>> Wol  schrieb am 08.04.2022 um 00:02 in Nachricht
:
> On 07/04/2022 17:47, Mike Gilbert wrote:
>>> So, my guess would be that the people who dislike merged-/usr are also
>>> the ones who dislike systemd, no? i.e. do they really matter if we are
>>> talking about what to support in systemd? They'd not use our stuff
>>> anyway, so why bother?
> 
> There's probably also a big minority of users (like me) who may be 
> pro-systemd, but run a systemd-hostile distro for reasons that are 
> nothing to do with systemd ...

I always thought it was a distro-hostile systemd ;-)
(You know what was first)

...

Ulrich






[systemd-devel] Antw: [EXT] all numeric usernames not allowed in systemd (version 245 onward)

2022-04-08 Thread Ulrich Windl
>>> Nikhil Kshirsagar  schrieb am 07.04.2022 um 13:07 in
Nachricht
:
> Hello,
> 
> I gather from the discussion on
> https://github.com/systemd/systemd/issues/15141 that all numeric usernames
> would no longer be supported on servers running systemd version 245 onward.
> This was also reiterated by Frantisek and Lennart (thank you for your email
> responses and for redirecting me to this mailing list.)

AFAIK there never were numeric user names in UNIX: If a user specification ws 
numeric, it was the UID of a user, not the name.
(VAX had different concepts however)
I mean: If you wanted to shoot yourself in the foot you would use numeric user 
names like this:

(pwasswd)
123:333:...
333:123:...

chown 333 file...

> 
> While I completely understand the motivation to do this, my concern is that
> this change will break logins for users on a lot of servers that upgrade to
> the new systemd.

Can you give examples of numeric user names being used?

> 
> Is there any chance systemd could support a configuration option in the
> future to get the earlier "all numeric" user logins to work? With an
> understanding that it would be at the users own risk?
> 
> Are there any pam_systemd configuration options under consideration that
> might help anyone looking for such functionality? Or is the only option
> "not to upgrade" for someone interested in preserving their all numeric
> usernames?
> 
> Regards,
> Nikhil.






[systemd-devel] Antw: [EXT] Re: [systemd‑devel] Dropping split‑usr/unmerged‑usr support

2022-04-08 Thread Ulrich Windl
>>> Lennart Poettering  schrieb am 07.04.2022 um 10:59
in
Nachricht :
> On Mi, 06.04.22 11:24, Mike Gilbert (flop...@gentoo.org) wrote:
> 
>> We are not likely to require merging of / and /usr for the foreseeable
>> future. We are a "rolling release" distro and basically never require
>> our users to re‑install, which makes large file system migrations
>> difficult. Also, many of our users would resist any attempt to force
>> merged‑/usr on them.
> 
> So, my guess would be that the people who dislike merged‑/usr are also
> the ones who dislike systemd, no? i.e. do they really matter if we are
> talking about what to support in systemd? They'd not use our stuff
> anyway, so why bother?

Maybe explain why systemd required /usr to be part of the root filesystem
instead.
Maybe also explain the chicken-egg race with temporary directories that
systemd added.

> 
>> I think it would be ok if systemd drops support for installing itself
>> in /lib/systemd; we would just move everything under /usr/lib/systemd,
>> and possibly set up some symlinks in /lib/systemd for the
>> transition.
> 
> You guys are making your life hell, because you are afraid if making
> it difficult...

Maybe just add a manual page systemd-filesystem-concepts ;-)

Regards,
Ulrich Windl


> 
> Lennart
> 
> ‑‑
> Lennart Poettering, Berlin





[systemd-devel] Antw: Re: Antw: Re: Antw: [EXT] Dropping split-usr/unmerged-usr support

2022-04-08 Thread Ulrich Windl
>>> Neal Gompa  schrieb am 07.04.2022 um 10:53 in
Nachricht
:

...
> On the contrary, Windows has been much more organized than UNIX has
> been. In the C:\ hierarchy, the "Windows" directory contains all the

Ok, A fan of Windows!

> resources to run Windows itself. System-wide applications are all in
> "Program Files", and user data is in "Users". Those three directories

Several EXEs are in c:\Windows (like notepad.exe), more are in
C:\Windows\System32 like write.exe.
Those are really _not_ system programs.

There are more in SysWOW64, WinSxS, Microsoft.NET, etc.

> form the core of the Windows experience. There are obviously more
> directories, but those three are essential for Windows itself. And if
> you don't need any applications (just the base Windows OS), then you
> can get away with just C:\Windows.

Windows is a hug pot of mess: Why are the ICC color profiles in a "spool"
directory (C:\Windows\System32\spool\drivers\color)?
Why are drivers in a spool directory?

> 
> UNIX, meanwhile, didn't have an opportunity to be thoughtful
> about how the hierarchy worked. Things got stuffed in where they could

UNIX had a hierarchy when MS-DOS still had everything in the root directory 10
years later.
Early versions of the FAT filesytem did not even have subdirectories.

> based on the size of diskettes and what could be held in memory. The
> fact that /usr doesn't actually represent where user data is proves
> it. "Unix System Resources" is a backronym to attempt to deal with the

In fact early UNIX systems _had_ user data in /usr.
Subdirectory in MS-DOS was so terribly slow that subdirectories were avoided.
In fact I think they started using subdirectories when the root filesystem was
full with 144 entries (or so).

---citation (UNIX Version 7 Volume 2B)---
New Users
Install new users by editing the password file /etc/passwd (passwd(5)). This
procedure should be
done once multi-user mode is entered (see init(8)). You’ll have to make a
current directory for each
new user and change its owner to the newly installed name. Login as each user
to make sure the password
file is correctly edited. For example:
ed /etc/passwd
$a
joe::10:1::/usr/joe:
w
q
mkdir /usr/joe
chown joe /usr/joe
login joe
ls – la
login root
This will make a new login entry for joe, who should be encouraged to use
passwd(1) to give himself a
password. His default current directory is /usr/joe which has been created.
The delivered password file
has the user bin in it to be used as a prototype.

(ed once was considered as genuine as systemd is now ;-)

> mistake of not renaming the directory when it evolved away from
> holding user data. The Unix hierarchy is *full* of mistakes and
> post-rationalizations ideally would be fixed someday but probably
> won't be.

Oh well: Windows 10 still has ini-files in \Windows, probably going back to
the times of Windows <=3 when there was no users and permissions, so they had
no concept of protection at all.
Now I think Windows is really not an example how to do things right.

Regards,
Ulrich Windl



[systemd-devel] Intercepting/Delaying the boot process

2022-04-08 Thread Andreas Hartmann
Hi,


I've been owning a Pinephone for a while now and one thing that I find rather
annoying is that whenever I plug the charger in, the thing will perform a full
boot. I often find myself just wanting to plug it in without performing a full
boot, i.e. only to have it charge and maybe see how far it has charged already.

To this end I was wondering whether it would be feasible to "hook" into the boot
process, somewhere before the disks are decrypted, to only have it charge (or
continue to boot when I press some magic button maybe). Looking at the order of
service startup I was thinking about maybe intercepting the boot between "local-
fs-pre.target" and "machines.target" because nothing happens there on my setup.

For this to work ideally I would like to have the sysfs available so I don't 
have
to interact with the kernel directly. So here are my questions:

- After which stage in the boot process is the sysfs available?
- Can I delay the boot for an indefinite amount of time, or will this cause some
services later on in the process to timeout and fail?
- Is it possible for a service during early boot to command a system shutdown
instead of continuing to boot?
- May I simply take control of the TTY and clear/rewrite it as I like, or does
systemd use some magic for this?

Thank you in advance!


hartan