Re: [systemd-devel] Hints for upgrading systemd on a running system

[Lennart Poettering]

On Mi, 21.02.18 08:10, Paul Menzel (pmenzel+systemd-de...@molgen.mpg.de) wrote:

> > Note that PID 1 itself is probably pretty Ok with such a massive
> > update in one step, but the unit files have been rearranged quite a
> > bit since then. Downstream distributions generally expect you to
> > reboot even between single-step distro updates, but this becomes much
> > more of a necessity if you jump even further.
> 
> But if reboot wouldn’t be an option, is there a way to get rid of not-found
> services?

stop them.

How did you upgrade your kernel without rebooting?

Note that there's also stuff such as dbus-daemon which cannot be
updated without rebooting.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Paul Menzel]

Dear Lennart and others,


Thank you for your prompt replies.


Am 20.02.2018 um 23:12 schrieb Lennart Poettering:

On Di, 20.02.18 20:00, Paul Menzel (pmenzel+systemd-de...@molgen.mpg.de) wrote:



We finally are going to upgrade from a very old systemd version 27 from 2011
to the current systemd v237. (Historical reasons.)

Anyway, I already was told about `systemctl daemon-reexec`, and we got it
working.


While we try to ensure that live upgrades of PID 1 like that work
quite well, this is generally tested only for small steps. Jumping 6
years ahead in one go is not something people typically test.


Indeed, but it seems to work. During the upgrade you have to make sure, 
that both versions are installed in parallel when doing `systemctl 
daemon-reexec`, so the old systemd still finds it dependencies/libraries 
and can terminate properly. Then the old version can be removed.



After that, looking at the output of `systemctl`, there are many units from
the old version, which were removed in the meantime.

```
$ systemctl --state=not-found
   UNIT LOAD  ACTIVE   SUB DESCRIPTION
● dev-hugepages.automount  not-found active   waiting
dev-hugepages.automount
● dev-mqueue.automount not-found active   waiting
dev-mqueue.automount
● sys-kernel-debug.automount   not-found active   waiting
sys-kernel-debug.automount
● sys-kernel-security.automountnot-found active   waiting
sys-kernel-security.automount
● auditd.service   not-found inactive dead
auditd.service
● console-kit-log-system-start.service not-found active   exited
console-kit-log-system-start.service
● display-manager.service  not-found inactive dead
display-manager.service
● hwclock-load.service not-found active   exited
hwclock-load.service
● plymouth-quit-wait.service   not-found inactive dead
plymouth-quit-wait.service
● plymouth-start.service   not-found inactive dead
plymouth-start.service
● remount-rootfs.service   not-found active   exited
remount-rootfs.service
● syslog.service   not-found inactive dead
syslog.service
● systemd-kmsg-syslogd.service not-found active   running
systemd-kmsg-syslogd.service
● systemd-remount-api-vfs.service  not-found active   exited
systemd-remount-api-vfs.service
● systemd-sysusers.service not-found inactive dead
systemd-sysusers.service
● udev-retry.service   not-found active   exited
udev-retry.service
● udev-settle.service  not-found active   exited
udev-settle.service
● systemd-logger.socketnot-found active   listening
systemd-logger.socket
● systemd-shutdownd.socket not-found active   listening
systemd-shutdownd.socket
● cryptsetup.targetnot-found active   active
cryptsetup.target
● syslog.targetnot-found active   active
syslog.target


My recommendation: simply reboot. That should clean up everything
properly.

Note that PID 1 itself is probably pretty Ok with such a massive
update in one step, but the unit files have been rearranged quite a
bit since then. Downstream distributions generally expect you to
reboot even between single-step distro updates, but this becomes much
more of a necessity if you jump even further.


But if reboot wouldn’t be an option, is there a way to get rid of 
not-found services?



Note that systemd upstream currently requires kernel 3.13 at least
which was released in 2014. Hence, if you update from a 2011 system
you have to reboot anyway, already to update the kernel...


We already run later Linux Kernels, so that is not a problem. But thank 
you for mentioning it.



Do I need to stop those manually beforehand, or is there another way to
clean up?

Is the recommended update procedure documented somewhere?


Usually distributions document that invididually as systemd is just
one component of many that make up the distribution.


Understood.


Kind regards,

Paul
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Lennart Poettering]

On Di, 20.02.18 20:06, Reindl Harald (h.rei...@thelounge.net) wrote:

> 
> 
> Am 20.02.2018 um 20:00 schrieb Paul Menzel:
> > Dear systemd folks,
> > 
> > We finally are going to upgrade from a very old systemd version 27 from
> > 2011 to the current systemd v237. (Historical reasons.)
> 
> hopefully you have a working backup
> 
> > Anyway, I already was told about `systemctl daemon-reexec`, and we got
> > it working.
> 
> the "reexec" is misleading because it's not possible to terminate PID1 and
> start it again on a running system with a new binary

Hmm? "systemctl daemon-reexec" is supposed to support upgrades between
systemd versions. Doing massive jumps v27 → v237 in one go isn't
precisely well tested, but smaller ones should work quite well.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Lennart Poettering]

On Di, 20.02.18 20:00, Paul Menzel (pmenzel+systemd-de...@molgen.mpg.de) wrote:

> Dear systemd folks,
> 
> 
> We finally are going to upgrade from a very old systemd version 27 from 2011
> to the current systemd v237. (Historical reasons.)
> 
> Anyway, I already was told about `systemctl daemon-reexec`, and we got it
> working.

While we try to ensure that live upgrades of PID 1 like that work
quite well, this is generally tested only for small steps. Jumping 6
years ahead in one go is not something people typically test.

> After that, looking at the output of `systemctl`, there are many units from
> the old version, which were removed in the meantime.
> 
> ```
> $ systemctl --state=not-found
>   UNIT LOAD  ACTIVE   SUB DESCRIPTION
> ● dev-hugepages.automount  not-found active   waiting
> dev-hugepages.automount
> ● dev-mqueue.automount not-found active   waiting
> dev-mqueue.automount
> ● sys-kernel-debug.automount   not-found active   waiting
> sys-kernel-debug.automount
> ● sys-kernel-security.automountnot-found active   waiting
> sys-kernel-security.automount
> ● auditd.service   not-found inactive dead
> auditd.service
> ● console-kit-log-system-start.service not-found active   exited
> console-kit-log-system-start.service
> ● display-manager.service  not-found inactive dead
> display-manager.service
> ● hwclock-load.service not-found active   exited
> hwclock-load.service
> ● plymouth-quit-wait.service   not-found inactive dead
> plymouth-quit-wait.service
> ● plymouth-start.service   not-found inactive dead
> plymouth-start.service
> ● remount-rootfs.service   not-found active   exited
> remount-rootfs.service
> ● syslog.service   not-found inactive dead
> syslog.service
> ● systemd-kmsg-syslogd.service not-found active   running
> systemd-kmsg-syslogd.service
> ● systemd-remount-api-vfs.service  not-found active   exited
> systemd-remount-api-vfs.service
> ● systemd-sysusers.service not-found inactive dead
> systemd-sysusers.service
> ● udev-retry.service   not-found active   exited
> udev-retry.service
> ● udev-settle.service  not-found active   exited
> udev-settle.service
> ● systemd-logger.socketnot-found active   listening
> systemd-logger.socket
> ● systemd-shutdownd.socket not-found active   listening
> systemd-shutdownd.socket
> ● cryptsetup.targetnot-found active   active
> cryptsetup.target
> ● syslog.targetnot-found active   active
> syslog.target

My recommendation: simply reboot. That should clean up everything
properly.

Note that PID 1 itself is probably pretty Ok with such a massive
update in one step, but the unit files have been rearranged quite a
bit since then. Downstream distributions generally expect you to
reboot even between single-step distro updates, but this becomes much
more of a necessity if you jump even further.

Note that systemd upstream currently requires kernel 3.13 at least
which was released in 2014. Hence, if you update from a 2011 system
you have to reboot anyway, already to update the kernel...

> Do I need to stop those manually beforehand, or is there another way to
> clean up?
> 
> Is the recommended update procedure documented somewhere?

Usually distributions document that invididually as systemd is just
one component of many that make up the distribution.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Reindl Harald]

Am 20.02.2018 um 22:04 schrieb Mantas Mikulėnas:
On Tue, Feb 20, 2018, 21:06 Reindl Harald > wrote:



Am 20.02.2018 um 20:00 schrieb Paul Menzel:
 > Dear systemd folks,
 >
 > We finally are going to upgrade from a very old systemd version
27 from
 > 2011 to the current systemd v237. (Historical reasons.)

hopefully you have a working backup

 > Anyway, I already was told about `systemctl daemon-reexec`, and
we got
 > it working.

the "reexec" is misleading because it's not possible to terminate PID1
and start it again on a running system with a new binary


But you don't *have to* terminate pid1 to change the binary – you can 
just exec the new one. Hence the "reexec".


It's possible. That's how initramfs works


a lot of other threads here made it clear that there is not much 
difference between "daemon-reload" and "daemon-reexec" at all in the 
past and i strongly doubt that it's technically possible to seamless 
switch from version 27 (the first Fedora GA AFAIK hat version 44) to 237 
and keep the system happily running as like nothing has changed without 
reboot

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Mantas Mikulėnas]

On Tue, Feb 20, 2018, 21:06 Reindl Harald  wrote:

>
>
> Am 20.02.2018 um 20:00 schrieb Paul Menzel:
> > Dear systemd folks,
> >
> > We finally are going to upgrade from a very old systemd version 27 from
> > 2011 to the current systemd v237. (Historical reasons.)
>
> hopefully you have a working backup
>
> > Anyway, I already was told about `systemctl daemon-reexec`, and we got
> > it working.
>
> the "reexec" is misleading because it's not possible to terminate PID1
> and start it again on a running system with a new binary
>

But you don't *have to* terminate pid1 to change the binary – you can just
exec the new one. Hence the "reexec".

It's possible. That's how initramfs works.

> --

Mantas Mikulėnas 
Sent from my phone
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Hints for upgrading systemd on a running system

[Paul Menzel]

Dear systemd folks,


We finally are going to upgrade from a very old systemd version 27 from 
2011 to the current systemd v237. (Historical reasons.)


Anyway, I already was told about `systemctl daemon-reexec`, and we got 
it working.


After that, looking at the output of `systemctl`, there are many units 
from the old version, which were removed in the meantime.


```
$ systemctl --state=not-found
  UNIT LOAD  ACTIVE   SUB 
DESCRIPTION
● dev-hugepages.automount  not-found active   waiting 
dev-hugepages.automount
● dev-mqueue.automount not-found active   waiting 
dev-mqueue.automount
● sys-kernel-debug.automount   not-found active   waiting 
sys-kernel-debug.automount
● sys-kernel-security.automountnot-found active   waiting 
sys-kernel-security.automount
● auditd.service   not-found inactive dead 
auditd.service
● console-kit-log-system-start.service not-found active   exited 
console-kit-log-system-start.service
● display-manager.service  not-found inactive dead 
display-manager.service
● hwclock-load.service not-found active   exited 
hwclock-load.service
● plymouth-quit-wait.service   not-found inactive dead 
plymouth-quit-wait.service
● plymouth-start.service   not-found inactive dead 
plymouth-start.service
● remount-rootfs.service   not-found active   exited 
remount-rootfs.service
● syslog.service   not-found inactive dead 
syslog.service
● systemd-kmsg-syslogd.service not-found active   running 
systemd-kmsg-syslogd.service
● systemd-remount-api-vfs.service  not-found active   exited 
systemd-remount-api-vfs.service
● systemd-sysusers.service not-found inactive dead 
systemd-sysusers.service
● udev-retry.service   not-found active   exited 
udev-retry.service
● udev-settle.service  not-found active   exited 
udev-settle.service
● systemd-logger.socketnot-found active   listening 
systemd-logger.socket
● systemd-shutdownd.socket not-found active   listening 
systemd-shutdownd.socket
● cryptsetup.targetnot-found active   active 
cryptsetup.target
● syslog.targetnot-found active   active 
syslog.target


LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB= The low-level unit activation state, values depend on unit type.

21 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
```

Do I need to stop those manually beforehand, or is there another way to 
clean up?


Is the recommended update procedure documented somewhere?


Kind regards,

Paul
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Michael Biebl]

2018-02-20 20:00 GMT+01:00 Paul Menzel :
>
> Do I need to stop those manually beforehand, or is there another way to
> clean up?

reboot.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hints for upgrading systemd on a running system

[Reindl Harald]

Am 20.02.2018 um 20:00 schrieb Paul Menzel:

Dear systemd folks,

We finally are going to upgrade from a very old systemd version 27 from 
2011 to the current systemd v237. (Historical reasons.)


hopefully you have a working backup

Anyway, I already was told about `systemctl daemon-reexec`, and we got 
it working.


the "reexec" is misleading because it's not possible to terminate PID1 
and start it again on a running system with a new binary


After that, looking at the output of `systemctl`, there are many units 
from the old version, which were removed in the meantime.


i doubt that anybody has done such a version jump at all and even if the 
environment won't match anyways - you need to reboot and hope it works 
as well as be prepared for the case it won't boot - it's that easy


make at least sure that initrd has been updated with the new systemd!


$ systemctl --state=not-found
   UNIT LOAD  ACTIVE   SUB DESCRIPTION
● dev-hugepages.automount  not-found active   waiting 
dev-hugepages.automount
● dev-mqueue.automount not-found active   waiting 
dev-mqueue.automount
● sys-kernel-debug.automount   not-found active   waiting 
sys-kernel-debug.automount
● sys-kernel-security.automount    not-found active   waiting 
sys-kernel-security.automount
● auditd.service   not-found inactive dead 
auditd.service
● console-kit-log-system-start.service not-found active   exited 
console-kit-log-system-start.service
● display-manager.service  not-found inactive dead 
display-manager.service
● hwclock-load.service not-found active   exited 
hwclock-load.service
● plymouth-quit-wait.service   not-found inactive dead 
plymouth-quit-wait.service
● plymouth-start.service   not-found inactive dead 
plymouth-start.service
● remount-rootfs.service   not-found active   exited 
remount-rootfs.service
● syslog.service   not-found inactive dead 
syslog.service
● systemd-kmsg-syslogd.service not-found active   running 
systemd-kmsg-syslogd.service
● systemd-remount-api-vfs.service  not-found active   exited 
systemd-remount-api-vfs.service
● systemd-sysusers.service not-found inactive dead 
systemd-sysusers.service
● udev-retry.service   not-found active   exited 
udev-retry.service
● udev-settle.service  not-found active   exited 
udev-settle.service
● systemd-logger.socket    not-found active   listening 
systemd-logger.socket
● systemd-shutdownd.socket not-found active   listening 
systemd-shutdownd.socket
● cryptsetup.target    not-found active   active 
cryptsetup.target
● syslog.target    not-found active   active 
syslog.target


LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

21 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
```

Do I need to stop those manually beforehand, or is there another way to 
clean up?


Is the recommended update procedure documented somewhere?

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel