dpkg sysusers and file metadata (was Re: Integration with systemd)

[Guillem Jover]

Hi!

On Fri, 2019-11-01 at 11:36:19 +, Simon McVittie wrote:
> On Thu, 31 Oct 2019 at 17:51:28 -0700, Russ Allbery wrote:
> > I think we should adopt sysusers.d fragments as the preferred mechanism
> > for creating system users
> 
> I have been tempted to write a small reimplementation of systemd-sysusers
> suitable for init-less containers and sysvinit systems, so that we can
> rely on its declarative syntax even on non-systemd systems - even though
> I use systemd myself and am happy with it as my init system, so it's
> entirely possible that I would never *use* the reimplementation.

I guess I'll need to prioritize implementing
.

> I've vaguely considered the same thing for tmpfiles.d, although a full
> reimplementation of tmpfiles.d is somewhat more difficult because it's
> more featureful.

And part of this would be covered too by
, which is
currently blocked by packages accessing the dpkg database, where I
need to sit down and file bugs. :/

> > And presumably you would instead propose banning use of systemd-sysusers
> > and sysusers.d and requiring continuing to use adduser from maintainer
> > scripts as we currently do.  I would object because to me that's obviously
> > inferior to a declarative syntax.
> 
> Whether declarative or imperative, it's also Debian-specific - which
> I think is not *necessarily* a problem, but runs a risk of becoming an
> instance of this frequent anti-pattern:

In this case (or parts of it), these are in the realm of dpkg and its
handling of the filesystem.

Thanks,
Guillem

Re: Integration with systemd

[Simon McVittie]

On Fri, 01 Nov 2019 at 14:16:37 +0100, Ansgar wrote:
> Possibly also tmpfiles, but without an init system nothing would start
> the service and it would have to be invoked manually.  Maintainer
> scripts might use it though to setup directories in /var/lib or similar
> locations.

Yes, that's roughly what I had in mind.

Some container managers (I'm thinking particularly of Docker and
bubblewrap here) already include a minimal process-reaper to act as pid
1 in the container, which is one side of what init traditionally does;
it wouldn't seem like a huge leap for container managers to have some
relatively minimal startup management (either running "hook" scripts
during container setup, or implementing things like the tmpfiles.d spec
themselves) to provide the other side of what init does.

Arguably, things like Docker already have the most minimal possible
service management: it runs the container's entry-point command, whether
that's a web server, an interactive shell or even a full init system,
then cleans up the rest of the container after that entry point exits.

smcv

Re: Integration with systemd

[Stephan Seitz]

On Di, Nov 05, 2019 at 05:45:34 +0100, Ansgar wrote:

Simon Richter writes:
Yes, that's one of the questions I have asked: is systemd a core 
system component that we want to provide a stable release for, or is 
it one of the peripheral packages that users can upgrade to 
a backported version if they need a new feature, or has Debian relaxed 
its standards to accomodate systemd?

I guess it can work the same as for src:linux?  That is a core system
component, but users can upgrade to a backported version.


Well, you can have different linux kernels installed at the same time and 
choose from grub. So installing a newer kernel from backports won’t 
delete the kernel from the stable release.


This won’t work for systemd.

Shade and sweet water!

Stephan

--
| Stephan Seitz  E-Mail: s...@fsing.rootsland.net |
|If your life was a horse, you'd have to shoot it.|

Re: Integration with systemd

[Ansgar]

Simon Richter writes:
> On Sat, Nov 02, 2019 at 07:40:52PM +0100, Thomas Goirand wrote:
>> There's no need to do that. If a backported package is using such
>> features, then it just should depend on the correct version of systemd.
>> You may have seen that systemd 242 is already in buster-backports...
>
> Yes, that's one of the questions I have asked: is systemd a core system
> component that we want to provide a stable release for, or is it one of the
> peripheral packages that users can upgrade to a backported version if they
> need a new feature, or has Debian relaxed its standards to accomodate
> systemd?

I guess it can work the same as for src:linux?  That is a core system
component, but users can upgrade to a backported version.

I don't think Debian has relaxed its standards to accomodate linux.

systemd isn't any different from other packages.

> Regardless of whether Debian wants to have diverse init systems or
> standardize on one, we need to have a clear position on what kind of
> support users can expect for the stable release. If a significant fraction
> of users runs a backported systemd, can we provide security updates for
> them, or will we shunt these users onto a "rolling release" track, and if
> yes, who manages that track?

Currently the systemd maintainers also maintain a backport for systemd.

Ansgar

Re: Integration with systemd

[Simon Richter]

Hi,

On Sat, Nov 02, 2019 at 07:40:52PM +0100, Thomas Goirand wrote:

> There's no need to do that. If a backported package is using such
> features, then it just should depend on the correct version of systemd.
> You may have seen that systemd 242 is already in buster-backports...

Yes, that's one of the questions I have asked: is systemd a core system
component that we want to provide a stable release for, or is it one of the
peripheral packages that users can upgrade to a backported version if they
need a new feature, or has Debian relaxed its standards to accomodate
systemd?

Regardless of whether Debian wants to have diverse init systems or
standardize on one, we need to have a clear position on what kind of
support users can expect for the stable release. If a significant fraction
of users runs a backported systemd, can we provide security updates for
them, or will we shunt these users onto a "rolling release" track, and if
yes, who manages that track?

These are all questions we never had to answer with sysvinit, because the
interface was stable for decades. I can understand why systemd does not
want that kind of "stability", but that means that we need a policy for
dealing with the consequences of that decision.

   Simon

Re: Integration with systemd

[Wouter Verhelst]

On Thu, Oct 31, 2019 at 03:45:47PM +0100, Marco d'Itri wrote:
> http://www.islinuxaboutchoice.com/

https://grep.be/blog/en/computer/cluebat/Systemd__Devuan__and_Debian/

-- 
To the thief who stole my anti-depressants: I hope you're happy

  -- seen somewhere on the Internet on a photo of a billboard

Re: Integration with systemd

[Sam Hartman]

> "Svante" == Svante Signell  writes:
Svante> Marco, I think your information about elogind is not
Svante> up-to-date: https://tracker.debian.org/pkg/elogind

Svante> testing migrations: excuses: Migration status for elogind
Svante> (239.3+20190131-1+debian1 to 241.3-1+debian1): Will attempt
Svante> migration (Any information below is purely informational)
Svante> Additional info: Updating elogind fixes old bugs: #939101
Svante> Piuparts tested OK -
Svante> https://piuparts.debian.org/sid/source/e/elogind.html 115
Svante> days old (needed 5 days)

Svante> Yes, 115 days old...

So, that's a bit misleading.
The last RC bug was closed the last week in October.
Yes, the process took longer than several of us wish it did.
However, the 105 days is definitely misleadingly long.

Mark has looked into why elogind still hasn't migrated.

Re: Integration with systemd

[Thomas Goirand]

On 11/1/19 3:20 PM, Simon Richter wrote:
>> If we do need to have a GR, we need to be very careful how the choices
>> are worded.  We should be clear whether we are giving carte blanche
>> for Debian developers to use every possible systemd feature under the
>> sun, whether or not there are non-systemd emulation possibilities yet.
> 
> Taking off the "I'm running sysvinit" hat, and putting on the "if we're
> going to do this, we should at least do it properly" hat: The obvious
> middle ground would be to allow features that exist in the previous
> release, so we can have working backports.

There's no need to do that. If a backported package is using such
features, then it just should depend on the correct version of systemd.
You may have seen that systemd 242 is already in buster-backports...

> It absolutely makes sense to teach start-stop-daemon to parse a .service
> file instead of its command line, that is a net win for everyone. If unit
> files can have shebang lines, we could even use them as init scripts
> directly, and that would probably cover 90% of use cases for bullseye
> without adding any more emulation features.

If you want to do a good thing for sysv-like stuff, then IMO, you'd
better work on having OpenRC to replace sysv-rc. That'd be a real net
improvement, and from there, you could improve OpenRC to have it in
feature parity with systemd. Then we could simply drop sysv-rc.

Later on, I don't think it'd be so hard to write a .service -> runscript
converter. The declarative way of OpenRC is very close to the one of
systemd, even though the internals are very different. Then init script
could start slowly dying too, for good! :)

If only I had spare time, I'd have work on that and have that ready for
Buster... Maybe this can be achieve for Bullseye? (I do have enough on
my plate with OpenStack, and I'm convince my time is better spent there)

Cheers,

Thomas Goirand (zigo)

Re: Integration with systemd

[Russ Allbery]

Vincent Bernat  writes:

> An alternative for many system users is to use the DynamicUser feature
> of systemd.

Yeah, I completely agree, and we haven't even started talking about that
yet.  This is what I mean by ten or more facilities like this that we
probably want to approach from the same philosophical basis.

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Simon Richter]

Hi,

On Thu, Oct 31, 2019 at 04:32:28PM -0400, Theodore Y. Ts'o wrote:

> That's true SO FAR.  The fact remains that systemd has *tons* and
> *tons* of new features which to date, aren't yet getting used in huge
> numbers of open source software packages or in Debian packaging --- YET.

I think that the open source world splitting between systemd and
non-systemd is pretty much unavoidable by now.

Systemd will not go away, and the BSDs plus a few stragglers on Linux are
keeping non-systemd alive as well.

The question for us as a project is whether we want to ship both branches,
or drop the non-systemd branch and shunt off the users to Devuan.

If we do, we should probably decide that before the next release, while it
is still easy for users to migrate (switch to sysvinit in Debian first,
then change the sources.list).

> If we do need to have a GR, we need to be very careful how the choices
> are worded.  We should be clear whether we are giving carte blanche
> for Debian developers to use every possible systemd feature under the
> sun, whether or not there are non-systemd emulation possibilities yet.

Taking off the "I'm running sysvinit" hat, and putting on the "if we're
going to do this, we should at least do it properly" hat: The obvious
middle ground would be to allow features that exist in the previous
release, so we can have working backports.

Whether emulation for these features is implemented for non-systemd is
secondary. If it doesn't work, we lock out the package combination, and
that's it.

IMO, emulation is a red herring for most features.

It absolutely makes sense to teach start-stop-daemon to parse a .service
file instead of its command line, that is a net win for everyone. If unit
files can have shebang lines, we could even use them as init scripts
directly, and that would probably cover 90% of use cases for bullseye
without adding any more emulation features.

I can even see socket activation being emulated to some extent, even though
I'm personally not interested in that beyond saving a few kilobytes on my
laptop by demand-starting lpd), but the ratio of usefulness vs complexity
diminishes quickly after that.

> Again, look at Josh's list of all of the random esoteric systemd
> features that people *could* be using.  I think you're painting a far
> too optismistic picture that there will always be enough programmer
> interest to keep up with systemd's many and varied new features.

The disagreement about systemd is not about whether these features should
be better implemented in C or in shell scripts, but about whether building
such a complex stack is worthwhile, because every layer introduces new and
interesting failure modes, and makes it more difficult to learn how the
entire system works.

So for me it is obvious that there will never be full emulation, because
that would give us the worst of both worlds, complexity without a system
designed from the ground up to organize it.

   Simon

Re: Integration with systemd

[Ansgar]

Simon McVittie writes:
> On Thu, 31 Oct 2019 at 17:51:28 -0700, Russ Allbery wrote:
>> I think we should adopt sysusers.d fragments as the preferred mechanism
>> for creating system users
>
> I have been tempted to write a small reimplementation of systemd-sysusers
> suitable for init-less containers and sysvinit systems, so that we can
> rely on its declarative syntax even on non-systemd systems - even though
> I use systemd myself and am happy with it as my init system, so it's
> entirely possible that I would never *use* the reimplementation.

I think splitting out services that are useful in environments without
an init system installed, such as sysusers, might be useful.

Possibly also tmpfiles, but without an init system nothing would start
the service and it would have to be invoked manually.  Maintainer
scripts might use it though to setup directories in /var/lib or similar
locations.

I think we should agree that using them is okay before considering to
split them into a separate package from systemd.

> I've vaguely considered the same thing for tmpfiles.d, although a full
> reimplementation of tmpfiles.d is somewhat more difficult because it's
> more featureful.

There is already an alternative implementation for tmpfiles.d:

https://github.com/OpenRC/opentmpfiles

I don't know more than that it exists though.

Ansgar

Re: Integration with systemd

[Alf Gaida]

On 01.11.19 02:33, Thomas Goirand wrote:


...the bigger question is: why systemd-sysusers is part of systemd, and
not a standalone thing, which we could make an essential package. If we
want it to be part of a package standard toolkit, it means systemd
becomes an essential package, which isn't really what we want (we don't
need an init system in a chroot, as you know). For that reason alone,
it's probably a bad idea to recommend systemd-sysusers everywhere.


Just file a bug against systemd and request a split out. Done. And i 
still don't get it. Why should upstream care and put the sources into 
another project - i don't get it in the discussion about systemd in the 
past years. If the handling of the sources is so evil, nobody should use 
*BSD anymore.



Cheers

Re: Integration with systemd

[Simon McVittie]

On Thu, 31 Oct 2019 at 17:51:28 -0700, Russ Allbery wrote:
> I think we should adopt sysusers.d fragments as the preferred mechanism
> for creating system users

I have been tempted to write a small reimplementation of systemd-sysusers
suitable for init-less containers and sysvinit systems, so that we can
rely on its declarative syntax even on non-systemd systems - even though
I use systemd myself and am happy with it as my init system, so it's
entirely possible that I would never *use* the reimplementation.

I've vaguely considered the same thing for tmpfiles.d, although a full
reimplementation of tmpfiles.d is somewhat more difficult because it's
more featureful.

The major problem with either of these is that, at the moment, packages
that ship a sysusers.d or tmpfiles.d fragment can assume that the consumer
of those fragments has the complete feature-set of systemd (at least the
version in stable), whereas if there was a reimplementation, we'd need to
have an answer for packages that use features that are supported by
systemd's reference implementation but not yet supported by the
reimplementation.

> And presumably you would instead propose banning use of systemd-sysusers
> and sysusers.d and requiring continuing to use adduser from maintainer
> scripts as we currently do.  I would object because to me that's obviously
> inferior to a declarative syntax.

Whether declarative or imperative, it's also Debian-specific - which
I think is not *necessarily* a problem, but runs a risk of becoming an
instance of this frequent anti-pattern:

* we identify a problem that needs some sort of system-integration glue
* we solve it in a Debian-specific way (either intentionally or
  unintentionally)
* other distributions later realise they have the same problem
* they solve it in upstream projects or in a cross-distro way
* often, we end up dropping our solution and using theirs - even if ours
  had significant advantages over theirs! - because theirs has wider
  upstream support, or because it keeps up with new requirements where
  ours does not

The Debian menu being replaced by .desktop files is a classic example.

It seems to be somewhat common that the Debian-specific solution
is more versatile - imperative maintainer scripts vs. declarative
configuration, or /etc/X11/Xsession.d vs. environment.d(5) - which makes
it flexible, but hard to reason about and hard to change or replace,
because packages that integrate with the Debian-specific solution could
be doing *anything*. Xsession.d has this particularly badly, because
it's a mixture of setting environment variables (which should ideally be
inherited by all session services) and starting session services (which
obviously can't inherit environment variables that haven't been set yet),
making it difficult to sequence the script fragments correctly.

smcv

Re: Integration with systemd

[Thomas Goirand]

On 11/1/19 3:16 AM, Russ Allbery wrote:
> Thomas Goirand  writes:
> 
>> ...the bigger question is: why systemd-sysusers is part of systemd, and
>> not a standalone thing, which we could make an essential package. If we
>> want it to be part of a package standard toolkit, it means systemd
>> becomes an essential package, which isn't really what we want (we don't
>> need an init system in a chroot, as you know). For that reason alone,
>> it's probably a bad idea to recommend systemd-sysusers everywhere.
> 
> I don't claim to know the full answer to this question, but if I had to
> guess, it's not particularly exciting: (a) the people who wrote it decided
> to include it in the systemd project for watever reason, probably because
> it was convenient and they were systemd developers, and (b) in the absence
> of any particular reason to break parts of systemd off into separate
> packages, the systemd maintainers have quite rationally minimized their
> work and packaging complexity.
> 
> Note that upstream is probably not interested in promising
> systemd-sysusers will always run under non-systemd init systems.  I don't
> know if there's any current or future reason why it wouldn't (maybe the %b
> or %m specifiers use some systemd library, although maybe they don't; I
> have done precisely zero investigation), but I doubt they'd make such a
> guarantee.  Folks may with some reason think they're wrong for not caring
> about that, but they're of course entitled to care and not care about
> whatever they want.  It therefore might be as simple as just making a new
> binary package, or it might not, and even if it is now, it might not
> always be.
> 
> On my side, what I find interesting is the declarative syntax and
> therefore the configuration files.  I don't really care if we use the
> systemd-sysusers program or something else (although obviously it's easier
> to use the thing that's already written and that someone else is
> maintaining for us).  I do (mildly) care that we use the same syntax and
> features as systemd because I think there's value in convergence between
> Debian, Red Hat, and other distributions.  The divergence between Debian
> and Red Hat and the correspondingly large differences in how you
> administered systems used to be really irritating; reducing gratuitous
> difference where neither approach is better, just different ways of doing
> the same thing, is a feature to me.
> 
> What I want out of a GR is to decide the deeper meta question of just how
> much effort the project wants to put into problems like this.  The
> *easiest* approach for Debian (assuming you agree with me about moving to
> a declarative syntax) would be to just say sysusers.d is now supported and
> preferred (except possibly in edge cases where it won't work; places where
> adduser is a pre-dependency will require special handling), and use the
> existing implementation and move on.  This would obviously break sysvinit
> until someone wrote an equivalent program.
> 
> A more moderate approach would be to say that once that alternative
> implementation was written, or alternately we've established that
> systemd-sysuers will work on sysvinit systems and we're at least somewhat
> committed to keeping it that way or writing something new if it doesn't,
> *then* we can start using sysusers.d, but until then it's not allowed (and
> there's a bug in the tomcat9 package).
> 
> Yet another option would be to say that we like adduser and maintainers
> are still required to use adduser, and systemd-sysusers is not supported
> and not allowed.
> 
> Of course, we shouldn't use a GR to decide this for systemd-sysusers
> specifically.  That's way too detailed for a GR.  But the point of many of
> us in the thread is that pretty much exactly this same set of alternatives
> are present for something like ten different facilities (if not more).  I
> don't think we actually want to make separate conflicting decisions about
> each one; I'm pretty sure that there is a general *philosophy* that we
> want to apply here, which is roughly somewhere on a spectrum between
> "let's start using systemd native services whenever they're available,
> stable, and solve some Debian problem, regardless of whether that breaks
> sysvinit" to "all this systemd stuff is unappealing and inferior to what
> we can do ourselves; let's decide to not use it and stick with our
> facilities."
> 
> (Note that there is another option, which is "let's go all in on systemd
> and use the systemd-native way of doing *everything*, right away."  I'm
> fairly sure that at most a tiny minority of folks in Debian want to do
> that; I'm pretty sure not even the systemd maintainers want that.  Rather,
> the most aggressive option I expect anyone to support in significant
> numbers still implies some sort of Policy vetting of a new facility to
> make sure it solves a real problem and that we've given some thought to
> how to integrate with it before we just start using it.  For 

Re: Integration with systemd

[Ansgar]

Martin Steigerwald writes:
> Of course that raises the question on what relationship with a 
> downstream like Devuan to aim for.

Debian has so many downstream distributions, one more fringe
distribution doesn't make any difference in relationships with
downstream distributions.

Besides that, from earlier communication from the 2000 active developers
of the Devuan distribution, they were planning to stop importing updates
from Debian anyway.

Ansgar

Re: Integration with systemd

[Svante Signell]

On Fri, 2019-11-01 at 06:47 +, Adam D. Barratt wrote:
> On Fri, 2019-11-01 at 00:54 +0100, Svante Signell wrote:
> > On Thu, 2019-10-31 at 22:40 +0100, Marco d'Itri wrote:
> > > On Oct 31, Svante Signell  wrote:
> > > 
> > 
> > Marco, I think your information about elogind is not up-to-date:
> > https://tracker.debian.org/pkg/elogind
> > 
> No, Marco's statement was entirely correct, and proved so by your own
> quote above.
> 
> That line says that elogind 239.3+20190131-1+debian1 is currently in
> testing, and that an updated version is attempting to migrate. A
> version of elogind has been in testing for 10 months now.

I should have added that the version in testing is _not_ usable for
desktops ... Sorry for being unclear.

Re: Integration with systemd

[Martin Steigerwald]

Martin Steigerwald - 01.11.19, 09:25:07 CET:
> Adam D. Barratt - 01.11.19, 07:47:48 CET:
> > On Fri, 2019-11-01 at 00:54 +0100, Svante Signell wrote:
> > > On Thu, 2019-10-31 at 22:40 +0100, Marco d'Itri wrote:
> > > > On Oct 31, Svante Signell  wrote:
> > > > > When elogind enters testing there would be many more people
> > > > > running
> > > > > Debian with sysvinit/elogind. elogind is needed for desktop
> > > > > usage
> > > > > when not using systemd as PID 1.
> > > > 
> > > > elogind is already in testing: I will be delighted to see how
> > > > the
> > > > number  of testing/unstable users running sysvinit will change
> > > > in
> > > > November.
> > > 
> > > Marco, I think your information about elogind is not up-to-date:
> > > https://tracker.debian.org/pkg/elogind
> > > 
> > > testing migrations:
> > > excuses:
> > > Migration status for elogind (239.3+20190131-1+debian1 to
> > > 
> > > 241.3-1+debian1): Will attempt migration (Any information below is
> > > purely informational)
> > 
> > No, Marco's statement was entirely correct, and proved so by your
> > own
> > quote above.
> > 
> > That line says that elogind 239.3+20190131-1+debian1 is currently in
> > testing, and that an updated version is attempting to migrate. A
> > version of elogind has been in testing for 10 months now.
> 
> But this is the version where it is still difficult to switch over. It
> usually involves uninstalling the desktop and installing it again.
> Also as to what I know it needs a package "that is not supposed to
> exist" from Debian experimental
> 
> So asking for this version how many will adopt it, in my viewpoint, is
> kind of unfair. I migrated two desktop laptops and it was not exactly
> fun with at least the second one. With the first it worked a bit
> smoother.
> 
> As long as it involves that hassle, likely only a few will migrate.

Aside from that the second laptop disappeared from Debian statistics 
already. Cause it is a Devuan meanwhile.

Of course that raises the question on what relationship with a 
downstream like Devuan to aim for. Make it easy for them to keep the 
differences at a minimum or not? And also specifically people using Debian 
asked about using elogind *with* Debian.

-- 
Martin

Re: Integration with systemd

[Vincent Bernat]

 ❦ 31 octobre 2019 17:51 -07, Russ Allbery :

> I think we should adopt sysusers.d fragments as the preferred mechanism
> for creating system users (with some rules, such as a standard for how to
> name the users and a requirement that the UID be specified as - unless one
> goes through the normal base-passwd registration process).  It supports a
> declarative syntax, doesn't require putting runes of code into a shell
> script, moves us farther down the path towards reducing us of maintainer
> scripts for most packages, and avoids the whole dependency and
> pre-dependency mess with adduser that took forever to sort out.  The
> syntax for sysusers.d is straighforward to parse, and support for
> non-systemd init systems via a trigger or boot-time script (or both) via
> adduser could be easily written, hiding the distinction between init
> systems.
[...]

An alternative for many system users is to use the DynamicUser feature
of systemd.
-- 
Use statement labels that mean something.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: Integration with systemd

[Martin Steigerwald]

Adam D. Barratt - 01.11.19, 07:47:48 CET:
> On Fri, 2019-11-01 at 00:54 +0100, Svante Signell wrote:
> > On Thu, 2019-10-31 at 22:40 +0100, Marco d'Itri wrote:
> > > On Oct 31, Svante Signell  wrote:
> > > > When elogind enters testing there would be many more people
> > > > running
> > > > Debian with sysvinit/elogind. elogind is needed for desktop
> > > > usage
> > > > when not using systemd as PID 1.
> > > 
> > > elogind is already in testing: I will be delighted to see how the
> > > number  of testing/unstable users running sysvinit will change in
> > > November.
> > 
> > Marco, I think your information about elogind is not up-to-date:
> > https://tracker.debian.org/pkg/elogind
> > 
> > testing migrations:
> > excuses:
> > Migration status for elogind (239.3+20190131-1+debian1 to
> > 
> > 241.3-1+debian1): Will attempt migration (Any information below is
> > purely informational)
> 
> No, Marco's statement was entirely correct, and proved so by your own
> quote above.
> 
> That line says that elogind 239.3+20190131-1+debian1 is currently in
> testing, and that an updated version is attempting to migrate. A
> version of elogind has been in testing for 10 months now.

But this is the version where it is still difficult to switch over. It 
usually involves uninstalling the desktop and installing it again. Also 
as to what I know it needs a package "that is not supposed to exist" 
from Debian experimental

So asking for this version how many will adopt it, in my viewpoint, is 
kind of unfair. I migrated two desktop laptops and it was not exactly 
fun with at least the second one. With the first it worked a bit 
smoother.

As long as it involves that hassle, likely only a few will migrate.

-- 
Martin

Re: Integration with systemd

[Adam D. Barratt]

On Fri, 2019-11-01 at 00:54 +0100, Svante Signell wrote:
> On Thu, 2019-10-31 at 22:40 +0100, Marco d'Itri wrote:
> > On Oct 31, Svante Signell  wrote:
> > 
> > > When elogind enters testing there would be many more people
> > > running
> > > Debian with sysvinit/elogind. elogind is needed for desktop usage
> > > when not using systemd as PID 1.
> > elogind is already in testing: I will be delighted to see how the
> > number  of testing/unstable users running sysvinit will change in
> > November.
> 
> Marco, I think your information about elogind is not up-to-date:
> https://tracker.debian.org/pkg/elogind
> 
> testing migrations:
> excuses:
> Migration status for elogind (239.3+20190131-1+debian1 to
> 241.3-1+debian1): Will attempt migration (Any information below is
> purely informational)

No, Marco's statement was entirely correct, and proved so by your own
quote above.

That line says that elogind 239.3+20190131-1+debian1 is currently in
testing, and that an updated version is attempting to migrate. A
version of elogind has been in testing for 10 months now.

Regards,

Adam

Re: Integration with systemd

[Russ Allbery]

Thomas Goirand  writes:

> ...the bigger question is: why systemd-sysusers is part of systemd, and
> not a standalone thing, which we could make an essential package. If we
> want it to be part of a package standard toolkit, it means systemd
> becomes an essential package, which isn't really what we want (we don't
> need an init system in a chroot, as you know). For that reason alone,
> it's probably a bad idea to recommend systemd-sysusers everywhere.

I don't claim to know the full answer to this question, but if I had to
guess, it's not particularly exciting: (a) the people who wrote it decided
to include it in the systemd project for watever reason, probably because
it was convenient and they were systemd developers, and (b) in the absence
of any particular reason to break parts of systemd off into separate
packages, the systemd maintainers have quite rationally minimized their
work and packaging complexity.

Note that upstream is probably not interested in promising
systemd-sysusers will always run under non-systemd init systems.  I don't
know if there's any current or future reason why it wouldn't (maybe the %b
or %m specifiers use some systemd library, although maybe they don't; I
have done precisely zero investigation), but I doubt they'd make such a
guarantee.  Folks may with some reason think they're wrong for not caring
about that, but they're of course entitled to care and not care about
whatever they want.  It therefore might be as simple as just making a new
binary package, or it might not, and even if it is now, it might not
always be.

On my side, what I find interesting is the declarative syntax and
therefore the configuration files.  I don't really care if we use the
systemd-sysusers program or something else (although obviously it's easier
to use the thing that's already written and that someone else is
maintaining for us).  I do (mildly) care that we use the same syntax and
features as systemd because I think there's value in convergence between
Debian, Red Hat, and other distributions.  The divergence between Debian
and Red Hat and the correspondingly large differences in how you
administered systems used to be really irritating; reducing gratuitous
difference where neither approach is better, just different ways of doing
the same thing, is a feature to me.

What I want out of a GR is to decide the deeper meta question of just how
much effort the project wants to put into problems like this.  The
*easiest* approach for Debian (assuming you agree with me about moving to
a declarative syntax) would be to just say sysusers.d is now supported and
preferred (except possibly in edge cases where it won't work; places where
adduser is a pre-dependency will require special handling), and use the
existing implementation and move on.  This would obviously break sysvinit
until someone wrote an equivalent program.

A more moderate approach would be to say that once that alternative
implementation was written, or alternately we've established that
systemd-sysuers will work on sysvinit systems and we're at least somewhat
committed to keeping it that way or writing something new if it doesn't,
*then* we can start using sysusers.d, but until then it's not allowed (and
there's a bug in the tomcat9 package).

Yet another option would be to say that we like adduser and maintainers
are still required to use adduser, and systemd-sysusers is not supported
and not allowed.

Of course, we shouldn't use a GR to decide this for systemd-sysusers
specifically.  That's way too detailed for a GR.  But the point of many of
us in the thread is that pretty much exactly this same set of alternatives
are present for something like ten different facilities (if not more).  I
don't think we actually want to make separate conflicting decisions about
each one; I'm pretty sure that there is a general *philosophy* that we
want to apply here, which is roughly somewhere on a spectrum between
"let's start using systemd native services whenever they're available,
stable, and solve some Debian problem, regardless of whether that breaks
sysvinit" to "all this systemd stuff is unappealing and inferior to what
we can do ourselves; let's decide to not use it and stick with our
facilities."

(Note that there is another option, which is "let's go all in on systemd
and use the systemd-native way of doing *everything*, right away."  I'm
fairly sure that at most a tiny minority of folks in Debian want to do
that; I'm pretty sure not even the systemd maintainers want that.  Rather,
the most aggressive option I expect anyone to support in significant
numbers still implies some sort of Policy vetting of a new facility to
make sure it solves a real problem and that we've given some thought to
how to integrate with it before we just start using it.  For instance, we
would want to make sure that systemd-sysusers honors our system UID ranges
and naming rules.)

I truly don't know where on that spectrum the *project* wants to be.  I
know where a 

Re: Integration with systemd

[Thomas Goirand]

On 11/1/19 1:51 AM, Russ Allbery wrote:
> Thomas Goirand  writes:
> 
>> IMO, this type of decision should go in the policy, case by case, and
>> I'm not sure a GR is the solution: it's going to be a generic "use all
>> of systemd" vs a "be careful to use only things implemented elsewhere".
>> I don't think this works, as often, there is maybe a middle ground
>> "well, it depends on the situation". For the systemd-sysusers in
>> tomcat9, probably best would have been to keep thinks as they were
>> rather than using an implementation that only has the side effect as to
>> get locked-in, especially when it's easy to avoid the problem. For other
>> cases, maybe it's nice to be able to use systemd-only features, and here
>> I'm thinking namely about cgroup stuff, for example.
> 
> So, let's explore this "Policy on a case-by-case basis" approach.
> 
> I think we should adopt sysusers.d fragments as the preferred mechanism
> for creating system users (with some rules, such as a standard for how to
> name the users and a requirement that the UID be specified as - unless one
> goes through the normal base-passwd registration process).  It supports a
> declarative syntax, doesn't require putting runes of code into a shell
> script, moves us farther down the path towards reducing us of maintainer
> scripts for most packages, and avoids the whole dependency and
> pre-dependency mess with adduser that took forever to sort out.  The
> syntax for sysusers.d is straighforward to parse, and support for
> non-systemd init systems via a trigger or boot-time script (or both) via
> adduser could be easily written, hiding the distinction between init
> systems.
> 
> So I should propose putting that into Policy, right?  Presumably you would
> object.
> 
> And presumably you would instead propose banning use of systemd-sysusers
> and sysusers.d and requiring continuing to use adduser from maintainer
> scripts as we currently do.  I would object because to me that's obviously
> inferior to a declarative syntax.  I've been beating the drum for
> declarative syntax to replace maintainer scripts in Debian since before
> systemd existed, and I personally don't care whether systemd happens to be
> the project that came up with a good facility or not.  If I see a good
> opportunity for moving to declarative syntax, I'll support it.
> 
> So now neither of our proposals has consensus, and Policy continues to be
> somewhat ambiguous about systemd-sysusers.  (Policy currently says, in
> kind of a weird place, that using adduser is a "should," which makes it a
> non-RC bug to not do so, but shoulds are often interpreted by the project
> to imply a certain amount of maintainer discretion.)
> 
> Now what?

I agree with some of the things you wrote above, but...

...the bigger question is: why systemd-sysusers is part of systemd, and
not a standalone thing, which we could make an essential package. If we
want it to be part of a package standard toolkit, it means systemd
becomes an essential package, which isn't really what we want (we don't
need an init system in a chroot, as you know). For that reason alone,
it's probably a bad idea to recommend systemd-sysusers everywhere.

Cheers,

Thomas Goirand (zigo)

Re: Integration with systemd

[Russ Allbery]

Thomas Goirand  writes:

> IMO, this type of decision should go in the policy, case by case, and
> I'm not sure a GR is the solution: it's going to be a generic "use all
> of systemd" vs a "be careful to use only things implemented elsewhere".
> I don't think this works, as often, there is maybe a middle ground
> "well, it depends on the situation". For the systemd-sysusers in
> tomcat9, probably best would have been to keep thinks as they were
> rather than using an implementation that only has the side effect as to
> get locked-in, especially when it's easy to avoid the problem. For other
> cases, maybe it's nice to be able to use systemd-only features, and here
> I'm thinking namely about cgroup stuff, for example.

So, let's explore this "Policy on a case-by-case basis" approach.

I think we should adopt sysusers.d fragments as the preferred mechanism
for creating system users (with some rules, such as a standard for how to
name the users and a requirement that the UID be specified as - unless one
goes through the normal base-passwd registration process).  It supports a
declarative syntax, doesn't require putting runes of code into a shell
script, moves us farther down the path towards reducing us of maintainer
scripts for most packages, and avoids the whole dependency and
pre-dependency mess with adduser that took forever to sort out.  The
syntax for sysusers.d is straighforward to parse, and support for
non-systemd init systems via a trigger or boot-time script (or both) via
adduser could be easily written, hiding the distinction between init
systems.

So I should propose putting that into Policy, right?  Presumably you would
object.

And presumably you would instead propose banning use of systemd-sysusers
and sysusers.d and requiring continuing to use adduser from maintainer
scripts as we currently do.  I would object because to me that's obviously
inferior to a declarative syntax.  I've been beating the drum for
declarative syntax to replace maintainer scripts in Debian since before
systemd existed, and I personally don't care whether systemd happens to be
the project that came up with a good facility or not.  If I see a good
opportunity for moving to declarative syntax, I'll support it.

So now neither of our proposals has consensus, and Policy continues to be
somewhat ambiguous about systemd-sysusers.  (Policy currently says, in
kind of a weird place, that using adduser is a "should," which makes it a
non-RC bug to not do so, but shoulds are often interpreted by the project
to imply a certain amount of maintainer discretion.)

Now what?

This is why I don't think your strategy of avoiding a GR and just having
Policy sort this out somehow doesn't work.

(Also, just to head this off in advance in case anyone wants to argue it,
I personally am dead-set against a position that says "if we can't get
consensus, we require everyone to do what we've always done."  I think
that's a recipe for endless sniping, hard feelings, and eventual
obsolescence.  Debian is too large now to be able to do literally
*everything* by consensus; there are some things where we'll always find
someone who objects to every course of action.)

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Thomas Goirand]

On 10/31/19 9:32 PM, Theodore Y. Ts'o wrote:
> Let's take e2fsprogs for example.  I had applied a patch which had a
> cron script alternative on top of the timer unit file.  It turns out
> the cron script was buggy, and it took multiple tries before we got it
> right --- because I don't maintain a test system with sysvinit to test
> it.  So I applied patches, but I was *not* doing my own testing before
> releasing updates with the cron support.  I'd call that "best efforts"
> support.
> 
> The GR should make clear whether or not what I did was sufficient ---
> I took patches and attempted bug fixes to support sysvinit --- or
> whether I should have been doing more explicit testing for sysvinit.

As much as I understand, at this point in time, the project considers
that what you did was enough. I don't think we need a GR for this.

> The GR should also make clear whether it would be a good thing if I,
> as the Debian Maintainer, were to deliberately use some esoteric
> systemd feature for which there is not non-systemd alternative in a
> package's packaging scripts. (I wouldn't do such a thing, in general,
> since I personally a good programmer should do things portably, and
> using an esoteric systemd feature is *not* good programming practice.
> But it's clear that others, like perhaps Josh Triplett, feel
> differently.  And I don't feel that I should necessarily be imposing
> my personal beliefs on everyone.)

IMO, this type of decision should go in the policy, case by case, and
I'm not sure a GR is the solution: it's going to be a generic "use all
of systemd" vs a "be careful to use only things implemented elsewhere".
 I don't think this works, as often, there is maybe a middle ground
"well, it depends on the situation". For the systemd-sysusers in
tomcat9, probably best would have been to keep thinks as they were
rather than using an implementation that only has the side effect as to
get locked-in, especially when it's easy to avoid the problem. For other
cases, maybe it's nice to be able to use systemd-only features, and here
I'm thinking namely about cgroup stuff, for example.

Cheers,

Thomas Goirand (zigo)

Re: Integration with systemd

[Svante Signell]

On Thu, 2019-10-31 at 22:40 +0100, Marco d'Itri wrote:
> On Oct 31, Svante Signell  wrote:
> 
> > When elogind enters testing there would be many more people running
> > Debian with sysvinit/elogind. elogind is needed for desktop usage
> > when not using systemd as PID 1.

> elogind is already in testing: I will be delighted to see how the
> number  of testing/unstable users running sysvinit will change in
> November.

Marco, I think your information about elogind is not up-to-date:
https://tracker.debian.org/pkg/elogind

testing migrations:
excuses:
Migration status for elogind (239.3+20190131-1+debian1 to
241.3-1+debian1): Will attempt migration (Any information below is
purely informational)
Additional info:
Updating elogind fixes old bugs: #939101
Piuparts tested OK - 
https://piuparts.debian.org/sid/source/e/elogind.html
115 days old (needed 5 days)

Yes, 115 days old...

Re: Integration with systemd

[Thomas Goirand]

On 10/31/19 11:30 PM, Russ Allbery wrote:
> Craig Small  writes:
>> On Fri, 1 Nov 2019 at 08:27, Thomas Goirand  wrote:
> 
>>> However, this doesn't mean that anything non-systemd must implement all
>>> things that systemd does, or just die. It really doesn't make sense to
>>> tell that, for example, OpenRC should be forced into implementing a
>>> parser of .timer files, just because some maintainers wont care about
>>> cron jobs. These cron jobs could be maintained by those who care, just
>>> like with sysv-rc scripts, as a best-effort basis.
> 
>> I think this, or something like this, is the policy update the project
>> needs.  I think this is a fine option, but does the project as a whole
>> think so, or perhaps they do with some changes?
> 
>> At the moment, my guess is most developers have no idea what the "right"
>> (e.g. what the project as a whole believe is right) solution for this.
>> Should I just use systemd support only? With sysvinit? If I get bug
>> reports about lack of support for one or the other how important is it?
> 
> One variation on this that's worth at least thinking about, although
> perhaps it would be unworkable, is if those cron jobs, init scripts, and
> other similar components that need to be maintained could be shipped in
> their own Debian package.
> 
> A lot of the process friction is in getting Debian maintainers to add this
> support to their packages.  Even if all concerns about testing, support,
> and so forth go away, we know that this is still not painless (see, for
> instance, translations, where we do *reasonably* well but where
> translations can still sit in the BTS for extended periods of time).  If
> it's possible to allow the people who are maintaining those files to
> upload them directly, it would remove that friction and process and might
> be easier to maintain.

While I very much like the idea of letting others do their work without
the maintainer to even notice to avoid frictions, I don' think adding
cron jobs in a separate package is the sensible solution.

Thomas Goirand (zigo)

Re: Integration with systemd

[Russ Allbery]

Craig Small  writes:
> On Fri, 1 Nov 2019 at 08:27, Thomas Goirand  wrote:

>> However, this doesn't mean that anything non-systemd must implement all
>> things that systemd does, or just die. It really doesn't make sense to
>> tell that, for example, OpenRC should be forced into implementing a
>> parser of .timer files, just because some maintainers wont care about
>> cron jobs. These cron jobs could be maintained by those who care, just
>> like with sysv-rc scripts, as a best-effort basis.

> I think this, or something like this, is the policy update the project
> needs.  I think this is a fine option, but does the project as a whole
> think so, or perhaps they do with some changes?

> At the moment, my guess is most developers have no idea what the "right"
> (e.g. what the project as a whole believe is right) solution for this.
> Should I just use systemd support only? With sysvinit? If I get bug
> reports about lack of support for one or the other how important is it?

One variation on this that's worth at least thinking about, although
perhaps it would be unworkable, is if those cron jobs, init scripts, and
other similar components that need to be maintained could be shipped in
their own Debian package.

A lot of the process friction is in getting Debian maintainers to add this
support to their packages.  Even if all concerns about testing, support,
and so forth go away, we know that this is still not painless (see, for
instance, translations, where we do *reasonably* well but where
translations can still sit in the BTS for extended periods of time).  If
it's possible to allow the people who are maintaining those files to
upload them directly, it would remove that friction and process and might
be easier to maintain.

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Craig Small]

On Fri, 1 Nov 2019 at 08:27, Thomas Goirand  wrote:

> However, this doesn't mean that anything non-systemd must implement all
> things that systemd does, or just die. It really doesn't make sense to
> tell that, for example, OpenRC should be forced into implementing a
> parser of .timer files, just because some maintainers wont care about
> cron jobs. These cron jobs could be maintained by those who care, just
> like with sysv-rc scripts, as a best-effort basis.
>
I think this, or something like this, is the policy update the project
needs.  I think this is a fine option, but does the project as a whole
think so, or perhaps they do with some changes?

At the moment, my guess is most developers have no idea what the "right"
(e.g. what the project as a whole believe is right) solution for this.
Should I just use systemd support only? With sysvinit? If I get bug reports
about lack of support for one or the other how important is it?

 - Craig


> Thomas Goirand (zigo)
>
>

Re: Integration with systemd

[Marco d'Itri]

On Oct 31, Svante Signell  wrote:

> When elogind enters testing there would be many more people running
> Debian with sysvinit/elogind. elogind is needed for desktop usage when
> not using systemd as PID 1. And as said numerous times Debian
elogind is already in testing: I will be delighted to see how the number 
of testing/unstable users running sysvinit will change in November.

> maintainers don't have to create sysvinit scripts, they have only to
> _accept_ patches to add or fix sysvinit scripts.
The problem is not just shipping a sysvinit script to start a daemon.
The problem is integrating in the boot system packages to support things 
like LVM, FCoE and multipath.
The problem is all the brittle machinery needed to support switching 
among differeny init systems.
The problem is having a daemon whose listening ports and addresses 
should be configured with systemd units or their configuration file 
depending on the init system being used.
The problem is starting X and then starting all the programs in the user 
session with systemd user units or with something totally different 
which maybe 1% of the users base has tested.
The problem is other packages having to support two totally different 
kinds of suspend/resume hooks.
And so on.

It was never just about sysvinit scripts: if all needed to make our 
few sysvinit users happy were just accepting patches to add missing init 
scripts then who would object?

> Any ideas of the non-linux ports: Scrap them, and leave them in the
> cold?
The non-Linux ports already died due to lack of users and developers 
because they are pointless, this has nothing to do with systemd.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: Integration with systemd

[Thomas Goirand]

On 10/30/19 10:54 PM, Josh Triplett wrote:
> It seems evident based on the history of such efforts that there is
> *not* sufficient people/interest/motivation to reimplement the majority
> of systemd features, let alone doing so in a way that meets the
> additional constraints imposed on such solutions. That support isn't
> going to materialize out of hope.

You are very right with the above, some features are probably never
going to be implemented elsewhere than in systemd.

However, this doesn't mean that anything non-systemd must implement all
things that systemd does, or just die. It really doesn't make sense to
tell that, for example, OpenRC should be forced into implementing a
parser of .timer files, just because some maintainers wont care about
cron jobs. These cron jobs could be maintained by those who care, just
like with sysv-rc scripts, as a best-effort basis.

Thomas Goirand (zigo)

Re: Integration with systemd

[Theodore Y. Ts'o]

On Thu, Oct 31, 2019 at 01:44:58PM +, Ian Jackson wrote:
> Martin Steigerwald writes ("Re: Integration with systemd"):
> > As to this, I did not yet see that the migration of elogind to testing 
> > has been accepted.
> 
> Yes.
> 
> I find these conversations draining, exhausting, awful.  I am sure
> that most people who are sceptical of systemd agree.  The constant
> negging and doom-saying is very unpleasant.
> 
> And, are we going to continue to wear people down with awful threads
> like this one, where a parade of doomsayers tell us we can't have what
> we want *even though it already exists and is maintained* ?

That's true SO FAR.  The fact remains that systemd has *tons* and
*tons* of new features which to date, aren't yet getting used in huge
numbers of open source software packages or in Debian packaging --- YET.

If we do need to have a GR, we need to be very careful how the choices
are worded.  We should be clear whether we are giving carte blanche
for Debian developers to use every possible systemd feature under the
sun, whether or not there are non-systemd emulation possibilities yet.

Or whether we are simply saying that we are not mandating that Debain
Maintainers don't have to do extra work to support non-systemd
systems, but they shouldn't be actively working to make things.

Let's take e2fsprogs for example.  I had applied a patch which had a
cron script alternative on top of the timer unit file.  It turns out
the cron script was buggy, and it took multiple tries before we got it
right --- because I don't maintain a test system with sysvinit to test
it.  So I applied patches, but I was *not* doing my own testing before
releasing updates with the cron support.  I'd call that "best efforts"
support.

The GR should make clear whether or not what I did was sufficient ---
I took patches and attempted bug fixes to support sysvinit --- or
whether I should have been doing more explicit testing for sysvinit.

The GR should also make clear whether it would be a good thing if I,
as the Debian Maintainer, were to deliberately use some esoteric
systemd feature for which there is not non-systemd alternative in a
package's packaging scripts.  (I wouldn't do such a thing, in general,
since I personally a good programmer should do things portably, and
using an esoteric systemd feature is *not* good programming practice.
But it's clear that others, like perhaps Josh Triplett, feel
differently.  And I don't feel that I should necessarily be imposing
my personal beliefs on everyone.)

Again, look at Josh's list of all of the random esoteric systemd
features that people *could* be using.  I think you're painting a far
too optismistic picture that there will always be enough programmer
interest to keep up with systemd's many and varied new features.

- Ted

Re: Integration with systemd

[John Goerzen]

On Thu, Oct 31 2019, Theodore Y. Ts'o wrote:
> It may be that sysvinit is doomed.  But we shouldn't be accelerating
> the process.

You are quite right.  I have also found myself wondering, though, what
are the BSDs doing?  Clearly systemd isn't going to be workable for
them.  Is their approach something that would be at all useful here?

Re: Integration with systemd

[Ole Streicher]

Svante Signell  writes:
> And as said numerous times Debian maintainers don't have to create
> sysvinit scripts, they have only to _accept_ patches to add or fix
> sysvinit scripts.

Even as someone who does not really care about the init system (being a
desktop user, I use whatever is the base there): I fully support you.

I really don't get the point of the discussion here: We have several
people trying to reach their goal with Debian. There are people fighting
for reproducibility, we have people supporting cross-building,
inofficial ports etc.

And we have people that want to support several init systems.

As a maintainer, my own interest in all these topics varies, by topic
and by package; therefore my own active support varies as well. But for
every topic I see some rationale, and so the minimum support is always
to accept patches (maybe after some review and discussion if it
influences my own packaging). If this creates a bug, I would however
just forward it to the patch creator, and just revert the patch if it
doesn't get fixed.

I think the openness of our distribution for well-defined goals is one
of our big strengths, and we should keep this up. I don't see a reason
why a maintainer should not accept sysvinit patches, or crossbuilding
patches, as long as they don't interfere with the rest of the package.

Being liberal would IMO really help here, and hurt nobody.

Just my 2 cents

Ole

Re: Integration with systemd

[Ansgar]

Simon Richter writes:
>> > No, and that's not our job. There are a lot of people out there building
>> > non-systemd systems.
[...]
> My point with that sentence is a different one though: a lot of Free
> Software exists outside the Linux sphere that does neither anticipate nor
> require tight integration with systemd, and that software is still useful,
> so it makes sense to ship it. It is niche, but it is the niche we have been
> traditionally good at, so I don't see a reason for leaving that and
> focusing on an oversaturated field without even marginally preparing.

I'm not sure what that has to do with Debian supporting multiple init
systems?  Nobody is suggesting to not package software that doesn't need
tight integration with systemd; most software probably won't.

Ansgar

Re: Integration with systemd

[Russ Allbery]

Ian Jackson  writes:

> The question is: are we going to permit those technical contributions
> into Debian ?  Are we going to keep making it awkward or are we actually
> going to _welcome_ them ?

> Are we going to say to those of our contributors who want to see a nice
> tidy hegemony, "sure, throw all the sysvinit stuff away, it is OK to
> reject the patches with init scripts, it is OK to work to make it
> impossible to have this stuff in Debian, even if the software works" ?

> And, are we going to continue to wear people down with awful threads
> like this one, where a parade of doomsayers tell us we can't have what
> we want *even though it already exists and is maintained* ?

Exactly.  And this to me is why a GR is a good idea, because right now we
don't have a clear project decision to point to, and therefore everyone
feels free to argue their side of the issue every time it comes up.  And
there isn't a clear project statement that people shouldn't remove init
scripts from their package, or that people shouldn't reject patches with
init scripts, that's documented somewhere and that we support as a
community.

We have a rough compromise that those of us who were there at the time
remember and try to support, but it's not written down, and there are
far-flung segments of Debian that don't read debian-devel and only
interact with systemd systems and aren't going to naturally support that
compromise.  Plus, that compromise was understood at the time to be
contingent on future developments, so we need to revisit it.

This is the deferred decision that I'm talking about: what is the project
going to say is Debian's position on what we're supporting and what we're
not supporting?  One option is to ratify the existing compromise, which
would have the advantage of putting some teeth into it and which I will
volunteer to try to write Policy text for.  But I do think it's not 100%
obvious that a majority in Debian supports that compromise, or understands
it in the same way.  We gain a lot of clarity by figuring that out.

I should also point out that this is exactly why Ian wanted a clear TC
ruling or GR on this after the original systemd debate, and what we're
seeing are the negative consequences of not doing that, just as he had
predicted.  As mentioned elsewhere, I still support the decision the
project made at the time for other reasons, but this is the downside that
we have not yet come to grips with, and I think it's worth noting that Ian
was correct about the negative consequences.

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Simon Richter]

Hi,

On Thu, Oct 31, 2019 at 03:45:47PM +0100, Marco d'Itri wrote:

> > That is work we have to do regardless of whether we want to support
> > alternatives or not, but in the simple case we just list what is supported
> > by the systemd version we have decided to ship in the last stable release,
> > so we can have backport packages with reasonable effort.

> It is not obvious to me why we would need a different policy for systemd 
> than for other packages which backports may depend on.

We need *a* policy in the first place.

When a package requires a systemd feature that isn't present in the current
stable release,

 - how do we detect this?

There is not even infrastructure in place that can map the features used in
unit files to required minimum systemd versions. Detecting dynamically used
features may be possible by looking at library symbols used, but this would
still have to be developed.

Right now, if I take a package from testing, compile it in a stable chroot
and get an installable binary package that has no dependencies not
satisfiable in stable, how do I verify that the package will work?

 - how should we react?

If a backported package does not work, and we find out somehow, what is the
correct course of action? Should the package be removed from the backports
archive, and users told to upgrade to testing? Should the backporter create
workarounds for missing features?

If you look at the history of the Debian Policy, the question of backwards
compatibility has always been a major point of contention. It took several
years to implement the "build-arch" target in debian/rules, because there
was no reliable way to detect whether the target was supported, so it was
impossible to distinguish between a build failure and an old package.

For some reason, all that caution seems to have gone out of the window.

We have had systemd as default pid 1 for longer than it took to get
"build-arch" documented, added to policy as optional, transitioning all
packages in the archive and making it mandatory so autobuilders could use
it. In all this time the only mention of systemd in policy are two
instances of adding dh_installsystemd after a mention of dh_installinit,
and section 9.3.5., titled "Example", pointing to a manpage instead of
giving an actual example.

Debian is entirely unprepared to do backports of anything that uses newer
systemd features. For GNOME that isn't a problem, large installations will
simply wait for the next stable release and laptop users will switch to
testing, but for libvirt, that is going to be a big deal.

> > No, and that's not our job. There are a lot of people out there building
> > non-systemd systems.

> Data says: not really a lot.

Even if we limit our view to "Debian installations", there is no sensible
way to measure that, or to distill it down to a useful number. Is an
administrator for a large installation a single user? Should we look at
downloads or at popcon statistics? Do autobuilders pulling systemd-sysv as
a dependency of libgtk-3-dev count?

My point with that sentence is a different one though: a lot of Free
Software exists outside the Linux sphere that does neither anticipate nor
require tight integration with systemd, and that software is still useful,
so it makes sense to ship it. It is niche, but it is the niche we have been
traditionally good at, so I don't see a reason for leaving that and
focusing on an oversaturated field without even marginally preparing.

Mind you, we still need to cover that field, but the reason people have
forgiven us to usually be the last distribution to ship new versions is
that when we ship, we get it right -- and that's what I'm not seeing here.

We're not only short on volunteers to backport systemd features to
sysvinit, we're also short on volunteers to lay the groundwork for systemd
based services. Is that is an indication of "lack of interest", as it seems
to be when sysvinit support is concerned?

The other big elephant in the room is what Ansgar mentioned in his reply to
me: containers where the systemd instance lives outside the container. I
would expect that several users want to run "stable" systemd, and import
containers for specific services. What infrastructure, what policies are in
place to ensure that services inside the container are compatible with
systemd outside?

   Simon

Re: Integration with systemd

[Svante Signell]

On Thu, 2019-10-31 at 15:45 +0100, Marco d'Itri wrote:
> > On Oct 31, Simon Richter  wrote:
> > 
> > No, and that's not our job. There are a lot of people out there
> > building non-systemd systems.
> Data says: not really a lot.

When elogind enters testing there would be many more people running
Debian with sysvinit/elogind. elogind is needed for desktop usage when
not using systemd as PID 1. And as said numerous times Debian
maintainers don't have to create sysvinit scripts, they have only to
_accept_ patches to add or fix sysvinit scripts.

Additionally, do you know that sysvinit (and elogind) has an active
upstream and an active maintainer. The number of bugs on sysvinit-
related packages has been reduced considerably recently! I can try to
find the actual numbers for the last months if somebody is interested.

Any ideas of the non-linux ports: Scrap them, and leave them in the
cold?

Re: Integration with systemd

[Josh Triplett]

Russ Allbery wrote:
> Josh Triplett  writes:
> > Part of the problem is that the people interested in sysvinit don't tend
> > to care about those features and often argue that others shouldn't care
> > either, and the people interested in those features don't tend to care
> > about sysvinit. It's difficult to motivate people to work on
> > alternatives for a system they already have and prefer.
>
> *This* is the thing that I feel we need to tackle head-on.  Because there
> are a lot of reasons to care about those features, but possibly more
> convincingly, as Ted points out, *upstreams* care about those features and
> are not going to stop caring about those features just because Debian
> packagers do not.

(And some Debian packagers care about those features, too.)

The problem isn't that nobody cares; the problem is that the people who
care most about those features have a solution that works and don't feel
the need for a new one. The more people are (or become) convinced they
want and care about these features, the more likely they are to *also*
want the existing system that already implements them. Resources
available to implement an alternative that compatibly supports those
features but doesn't use or work anything like the existing solution
would have to come from the small intersection of "wants those features"
and "doesn't want the existing system implementing those features".

I'm not suggesting that group doesn't exist, but conditions don't look
good for it to grow substantially, or become sustainable.

(And unfortunately, by comparison, the more acrimonious approach of
fighting against those features and their usage is relatively easier, at
least within Debian.)

> Sure, some systemd features are only marginally better than what came
> before (I admit to not being much of a fan of timer units, for instance).
> But this will vary by person.  For example, as a security person, I care a
> *lot* about namespacing, capability reduction, and seccomp filters, and I
> want to use those features in my packages to make the default
> configuration more secure.  Other people are going to care a lot about
> D-Bus integration, and some upstreams are going to fully embrace socket
> activation (which, again, I'll point out is one of the easiest features of
> systemd to implement outside of systemd; implementations of the same idea
> predated systemd by many years) and just not support any way of spawning
> their service that doesn't satisfy the systemd socket activation API.
> 
> One can individually say that one doesn't care about those features, but
> we just cannot say Debian as a whole should not care about those features.
> It doesn't work.  We have to take an affirmative stance on what Debian is
> going to do with software that does care about and use those features;
> whether we're committing to porting it,

"committing to porting it" isn't a stance that "Debian as a whole" can
take. (Nor is committing to reimplement those features in another init
system.) That's certainly a stance that developers could take,
individually or as a dedicated group, but otherwise it's just "someone
should bell the cat".

> But saying "no one should care about those features" isn't a choice and
> isn't a path forward and we can't stop there as a project.

Agreed.

> > That's leaving aside that a reimplementation of systemd's features will
> > tend towards becoming systemd, and we have one of those already. What is
> > the actual goal?
> 
> The actual goal is to allow for different ecosystems that provide the same
> features while embracing different implementation philosophies.  I know
> that you don't think this is a valuable goal

That wasn't the point I was making, and I'm sorry if it came across
otherwise.

My point was that that makes any potential reimplementation even
*harder*. A simple reimplementation won't suffice. Reuse of existing
code won't suffice. It wouldn't help for folks who like systemd to
reimplement those features, because the reimplementation is not going to
satisfy the additional constraints of people who don't like systemd.
And the dual constraints of "must be compatible with systemd" and "must
have a fundamentally different approach than systemd" *inherently*
creates impedance mismatches and implementation challenges.

There's a reason why this reimplementation work hasn't happened, and
doesn't seem likely to happen. And *because* of that, there's instead
effort directed to try to stop people from using and depending on such
features.

> > It seems evident based on the history of such efforts that there is
> > *not* sufficient people/interest/motivation to reimplement the majority
> > of systemd features, let alone doing so in a way that meets the
> > additional constraints imposed on such solutions.
> 
> I don't think the question has yet been forced.  Up through buster, one
> has been able to mostly run Debian on sysvinit without doing this work,
> with some pain and some gaps and some issues.  I 

Re: Integration with systemd

[Martin Steigerwald]

Marco d'Itri - 31.10.19, 15:45:47 CET:
> On Oct 31, Simon Richter  wrote:
[…]
> > The freedom to configure a system without things I do not want is
> > one of the main reasons that made me switch over from Windows to
> > Debian, a bit more than twenty years ago.
> 
> http://www.islinuxaboutchoice.com/

I kindly ask you to stop repeating old arguments.

We have been there before and if this is about throwing links around… I 
can for sure throw in a dozen links how and why using something else 
than Systemd would be better, including with more links to more pages 
and so on.

Please, we have been there before and there is no need to go there 
again.

Can you just agree to disagree so we can see where to go from there?

Please stop making this another thread pro and contra Systemd.

-- 
Martin

Re: Integration with systemd

[Martin Steigerwald]

Theodore Y. Ts'o - 31.10.19, 16:03:29 CET:
> On Thu, Oct 31, 2019 at 01:19:56PM +0100, Martin Steigerwald wrote:
> > alienate me away from Debian. This laptop, for the sake of packaging
> > flexible I/O tester, is the last of my machines still running on
> > Debian. All the others are running Devuan. I am not looking back. I
> > have no intention what so ever to switch back to Systemd again. For
> > that reason I for example use unbound instead of knot-resolver.
> > Cause I still have a choice which upstream projects to choose. If I
> > would not be able to run Debian with sysvinit or runit in the end I
> > might also drop my packaging and at least some other contributions
> > to Debian at one point in time.
> I think there was mis-undrestanding about what I was trying to say.
> My basic premise is that we can't *force* people to work on
> technology.  Debian is a volunteer organization.  So we can't force
> people to work on improving elogind.

Sure. I totally get that. I replied to your mail, since you mentioned 
that for you integrating Debian core infrastructure like dpkg or apt 
with Systemd would be something new and different. So while I chose your 
mail to reply to as you summarized this so accurately, I really 
responded to the intention to use Systemd features within core Debian 
packages.

> However, that's *different* from adopting new dependencies on systemd
> as part of Debian packaging or core Debian packages.  If KDE or GNOME
> wants to add new systemd dependencies, that's GNOME's or KDE's
> decision.  But if Debian decides to do something which makes it harder

Sure. For KDE I am pretty confident they won't. And if they would… I and 
others would probably speak out loudly that this may not be such a good 
idea.

> for sysvinit, we need to think ***really** hard about whether it's
> worth it, and I would claim the default answer is we shouldn't make
> the change unless there are darned good reasons to do so.

I fully second this. 

> It may be that sysvinit is doomed.  But we shouldn't be accelerating
> the process.

Or it may not be.

It is similar with "the" cloud. All say it will disrupt everything 
around… and it does. Yet, there are attempts to disrupt the cloud 
already by making machines powerful enough that they do not need a 
centralized cloud provider even if storing a huge lot of data.

Or with the climate.

I can respond to things like this in a way "Oh, we are doomed it does 
not make sense to do anything about it" or say "Let's challenge this and 
see what comes out of it". For the climate it is of course way more 
important to challenge how we are doing things, but it has been and will 
continue to be challenged for Systemd as well.

And yes, in a way Systemd challenged the status quo. As well as "the" 
cloud. But eventually both will be challenged again.

As for Debian for me the question would also be: Are we just following 
the lead of other distributions? Or are we actually inventing things not 
seen elsewhere. Just implementing more of Systemd integration would not 
be anything new. Others have been there before. What makes Debian unique 
these days if not the community?

-- 
Martin

Re: Integration with systemd

[Theodore Y. Ts'o]

On Thu, Oct 31, 2019 at 01:19:56PM +0100, Martin Steigerwald wrote:
> alienate me away from Debian. This laptop, for the sake of packaging 
> flexible I/O tester, is the last of my machines still running on Debian. 
> All the others are running Devuan. I am not looking back. I have no 
> intention what so ever to switch back to Systemd again. For that reason 
> I for example use unbound instead of knot-resolver. Cause I still have a 
> choice which upstream projects to choose. If I would not be able to run 
> Debian with sysvinit or runit in the end I might also drop my packaging 
> and at least some other contributions to Debian at one point in time.

I think there was mis-undrestanding about what I was trying to say.
My basic premise is that we can't *force* people to work on
technology.  Debian is a volunteer organization.  So we can't force
people to work on improving elogind.

However, that's *different* from adopting new dependencies on systemd
as part of Debian packaging or core Debian packages.  If KDE or GNOME
wants to add new systemd dependencies, that's GNOME's or KDE's
decision.  But if Debian decides to do something which makes it harder
for sysvinit, we need to think ***really** hard about whether it's
worth it, and I would claim the default answer is we shouldn't make
the change unless there are darned good reasons to do so.

It may be that sysvinit is doomed.  But we shouldn't be accelerating
the process.

- Ted

Re: Integration with systemd

[Marco d'Itri]

On Oct 31, Simon Richter  wrote:

> However, a lot of our software comes from the BSD world and will never
> fully switch to systemd, and that software is often used in server contexts
> by people who know exactly what they are doing. I don't see why we
> shouldn't support these people anymore, especially as they are the ones who
> cannot be served equally well by other distributions.
Because by using systemd features in the Debian packaging we make better 
packages: simpler to maintain and more robust.

(And then everything else that Ansgar patiently explained.)

> The freedom to configure a system without things I do not want is one of
> the main reasons that made me switch over from Windows to Debian, a bit
> more than twenty years ago.
http://www.islinuxaboutchoice.com/

> That is work we have to do regardless of whether we want to support
> alternatives or not, but in the simple case we just list what is supported
> by the systemd version we have decided to ship in the last stable release,
> so we can have backport packages with reasonable effort.
It is not obvious to me why we would need a different policy for systemd 
than for other packages which backports may depend on.

> No, and that's not our job. There are a lot of people out there building
> non-systemd systems.
Data says: not really a lot.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: Integration with systemd

[Ian Jackson]

Martin Steigerwald writes ("Re: Integration with systemd"):
> As to this, I did not yet see that the migration of elogind to testing 
> has been accepted.

Yes.

I find these conversations draining, exhausting, awful.  I am sure
that most people who are sceptical of systemd agree.  The constant
negging and doom-saying is very unpleasant.

Meanwhile, elsewhere, where the real technical work is being done,
systems are being built that run modern software (including GNOME,
whatever the latest daemons, etc.) without systemd.

The question is: are we going to permit those technical contributions
into Debian ?  Are we going to keep making it awkward or are we
actually going to _welcome_ them ?

Are we going to say to those of our contributors who want to see a
nice tidy hegemony, "sure, throw all the sysvinit stuff away, it is OK
to reject the patches with init scripts, it is OK to work to make it
impossible to have this stuff in Debian, even if the software works" ?

And, are we going to continue to wear people down with awful threads
like this one, where a parade of doomsayers tell us we can't have what
we want *even though it already exists and is maintained* ?

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Integration with systemd

[Martin Steigerwald]

Martin Steigerwald - 31.10.19, 13:19:56 CET:
> While I do not expect maintainers of Debian packages to implement
> support for alternate init systems themselves, I still believe if
> someone works constructively and consistently on making such support
> available in Debian, it would be good to be inclusive enough to allow
> him or her to do this work and find a good way to integrate.
> 
> Like with elogind.
> 
> Refusing to work together constructively due to emotional exhaustion
> even tough Mark did all he can to remain as constructive and
> supportive as he could… is understandable, but ultimately does not
> help and may ultimately alienate those who use Debian with an
> alternative init system.

As to this, I did not yet see that the migration of elogind to testing 
has been accepted.

Thank you very much, Sam, Ian, Mark!

However, as to the broader topic of integrating with systemd, what I 
wrote still applies.

Best,
-- 
Martin

Re: Integration with systemd

[Martin Steigerwald]

Hi!

I thought about just silently unsubscribing from debian-devel… but as I 
got the impression that almost no one argues for the freedom to choose 
the init system here in this thread, I decided to speak up:

Theodore Y. Ts'o - 31.10.19, 00:57:48 CET:
> And if we do this in core Debian infrastructure, such as say, in dpkg,
> then that's *really* different.  That's completely ruling out
> sysvinit.  And that to me is different from something like "GNOME no
> longer works on sysvinit" (until someone enhances elogind).  In that
> case, it's the GNOME Project which screwed over sysvinit, not Debian
> --- and we're just saying that it's not the GNOME Debian packaging
> team which is obligated to fix things up.
> 
> But the dpkg maintainer making a change which screws over sysvinit
> *feels* different, both in that it affects all Debian installations,
> versus just the ones using a particular GUI, and whether it was Debian
> or GNOME that made that particular decision.
> 
> Let me be clear, my personal opinion is that Lennart, and the
> acceptance of systemd by the vast majority of the major distributions,
> means that eventually, most upstreams will be using more and more
> systemd features, and people who like sysvinit should just get over
> it.  But whether we should accelerate that transition, or let it
> happen at a more natural pace, is something which IMHO, needs to be
> handled on a case by case basis.  Exactly how much of a win do we get
> if we use a particular systemd feature in core Debian packaging?  How
> hard is it to emulate that for non-systemd systems?  I don't think
> that decision can be made in the abstract, unless we as an entire
> project want to vote to deliberately, and with malice aforethought,
> kill off sysvinit support in Debian.

While I do not expect maintainers of Debian packages to implement 
support for alternate init systems themselves, I still believe if 
someone works constructively and consistently on making such support 
available in Debian, it would be good to be inclusive enough to allow 
him or her to do this work and find a good way to integrate.

Like with elogind.

Refusing to work together constructively due to emotional exhaustion 
even tough Mark did all he can to remain as constructive and supportive 
as he could… is understandable, but ultimately does not help and may 
ultimately alienate those who use Debian with an alternative init 
system.

For me one of the core issues is that with how systemd is designed it 
can be challenging to find a win-win solution, at least if you like to 
use all its features. Systemd appears to be an all-or-nothing-approach. 
And thus the analogy you made, Ted, that Josh judged and silenced as 
"gratuitous flame" appeared to be perfectly valid to me as for it 
described what I see happening here. In this concrete example Mark came 
up with several such win-win solutions to the best of his ability 
anyway.

Making core Debian infrastructure dependent on Systemd will very likely 
alienate me away from Debian. This laptop, for the sake of packaging 
flexible I/O tester, is the last of my machines still running on Debian. 
All the others are running Devuan. I am not looking back. I have no 
intention what so ever to switch back to Systemd again. For that reason 
I for example use unbound instead of knot-resolver. Cause I still have a 
choice which upstream projects to choose. If I would not be able to run 
Debian with sysvinit or runit in the end I might also drop my packaging 
and at least some other contributions to Debian at one point in time. If 
Debian is not inclusive enough for me to run it the way I like it to, 
what is the point of contributing to it any longer unless I can still 
use the Debian packaging as a base for the package in Devuan?

Making core Debian infrastructure dependent on Systemd would also deepen 
the split between Debian and Devuan. So far it has been possible to keep 
the differences at a minimum, making Devuan more of a slight variation to 
Debian instead of a fork that would develop in a different direction. 
Keeping the differences minimal is also an intention of Devuan project 
members as far as I got.

As for upstream I'd wait for what actually would really happen instead 
of trying to predict the future. So far, I am able to run all I need on 
sysvinit and/or openrc+sysvinit based systems. Others are to¹. That even 
includes third party software like rspamd, now available in Debian too, 
and I also expect the Elastic Stack to work. Cause in the end, most of 
them are services which do not really care for how they have been 
started.

Also the KDE project shows no signs limiting itself to Systemd based 
systems. The Plasma desktop needs some logind, but that is basically it. 
I expect that to stay this way at least for as long as FreeBSD is 
supported and quite some KDE applications even run and in part are also 
supported on Windows. For the KDE project portability is a feature not a 
hindrance.

Re: Integration with systemd

[Ansgar]

Russ Allbery writes:
> Josh Triplett  writes:
>
>> Part of the problem is that the people interested in sysvinit don't tend
>> to care about those features and often argue that others shouldn't care
>> either, and the people interested in those features don't tend to care
>> about sysvinit. It's difficult to motivate people to work on
>> alternatives for a system they already have and prefer.
>
> *This* is the thing that I feel we need to tackle head-on.  Because there
> are a lot of reasons to care about those features, but possibly more
> convincingly, as Ted points out, *upstreams* care about those features and
> are not going to stop caring about those features just because Debian
> packagers do not.

But many people *don't* care.  See the history of consolekit and
systemd-shim which nobody wanted to maintain and then were unhappy when
they were eventually gone...  The argument there has always been "I
don't care about these features", but upstream projects used them and
eventually systemd was the only implementation.

>> It seems evident based on the history of such efforts that there is
>> *not* sufficient people/interest/motivation to reimplement the majority
>> of systemd features, let alone doing so in a way that meets the
>> additional constraints imposed on such solutions.
>
> I don't think the question has yet been forced.  Up through buster, one
> has been able to mostly run Debian on sysvinit without doing this work,
> with some pain and some gaps and some issues.  I think we're coming up to
> a point where this issue will be forced because both packaging practices
> and upstreams are diverging away from sysvinit support.
>
> I think there are two questions here:
>
> 1. Do we as a project want to do the work to leave open the possibility
>that such resources will materialize?  This means doing things like
>defining a subset of systemd unit features that packages can rely on,
>and requiring (some? all?) packages not use features outside of that
>set.  "No" is a possible answer here, but this is a political question
>with significant consequences, and I think it should be decided
>democratically.

Is there someone willing to do the work you envision?  If not, what
would be the use of deciding about it democratically?

I think we shouldn't spend large amounts of resources to provide support
for such a hypothetical possibility.

I can imaging doing the same for other system interfaces, such as
mandating only a subset of the Linux kernel API to be used to support
alternative implementations of that API.  These already exist,
e.g. Microsoft's Linux subsystem (the old one) and I believe BSDs also
can pretend to be a Linux kernel.

We could also standardize the libc ABI to allow alternative
implementations, or at least not use non-standard extensions that make
replacing glibc harder.

I don't see why systemd should be treated different from other parts of
Debian.

> 2. Are we as a project *capable* of producing a non-systemd alternative
>that's viable?  If the answer to question 1 is "no," then this question
>doesn't arise.  But it's entirely possible that the answer to question
>1 is "yes" but the answer to this question is still "no."

I don't really believe writing init systems is Debian's goal, just like
implementing our own X server or other stuff.

That other implementations that use systemd's unit files might be
useful, but so far no other implementation has come into existence for
years.

Ansgar

Re: Integration with systemd

[Ansgar]

Simon Richter writes:
> On Wed, Oct 30, 2019 at 05:17:57PM -0700, Russ Allbery wrote:
>
>> One can individually say that one doesn't care about those features, but
>> we just cannot say Debian as a whole should not care about those features.
>> It doesn't work.  We have to take an affirmative stance on what Debian is
>> going to do with software that does care about and use those features;
>> whether we're committing to porting it, whether we're going to kick it out
>> of the archive (that seems highly unlikely to be viable), whether we're
>> going to say that software with a hard dependency on systemd features is
>> allowed to only work on systems running systemd, or some other
>> alternative.
>
> From an integration point of view we need to make sure that software that
> is installed also works, and I have little trouble when packages declare an
> explicit dependency on systemd running as pid 1.

Such a dependency doesn't work well with services installed in
containers which can still access stuff outside the container.
Or services started by a user (who doesn't have to use the provided
system integration).

> They need to declare a
> dependency nonetheless, because init systems have never been Essential, and
> also because the dependency likely needs a minimum version.

No: init systems have been essential until systemd (or init) replaced
sysvinit.

Note that there are other cases where we don't have explicit
dependencies such as depending on the Linux kernel itself, even though
some programs require minimum versions (including glibc).

> However, a lot of our software comes from the BSD world and will never
> fully switch to systemd, and that software is often used in server contexts
> by people who know exactly what they are doing. I don't see why we
> shouldn't support these people anymore, especially as they are the ones who
> cannot be served equally well by other distributions.

Sure, you can just not install any init system.  This doesn't require
Debian to support other init systems and provide integration with them.

> Infrastructure I'd like to see is
>
>  - dh_systemd generating a versioned dependency for the unit files
>  given

That makes supporting other init systems even less possible as
everything shipping any systemd service will start to pull in systemd,
even though one might not use it to start the service.

I think this is not a good idea.

>  - apt being able to blacklist packages and hide packages that depend on
>those

Implicitly hiding some packages seems very confusing.

Ansgar

Re: Integration with systemd

[Simon Richter]

Hi,

On Wed, Oct 30, 2019 at 05:17:57PM -0700, Russ Allbery wrote:

> One can individually say that one doesn't care about those features, but
> we just cannot say Debian as a whole should not care about those features.
> It doesn't work.  We have to take an affirmative stance on what Debian is
> going to do with software that does care about and use those features;
> whether we're committing to porting it, whether we're going to kick it out
> of the archive (that seems highly unlikely to be viable), whether we're
> going to say that software with a hard dependency on systemd features is
> allowed to only work on systems running systemd, or some other
> alternative.

>From an integration point of view we need to make sure that software that
is installed also works, and I have little trouble when packages declare an
explicit dependency on systemd running as pid 1. They need to declare a
dependency nonetheless, because init systems have never been Essential, and
also because the dependency likely needs a minimum version.

There is no point for Debian in taking a stance against using systemd
features, because upstreams will do that anyway, and we neither have the
manpower as a volunteer project to provide alternatives, nor as a
distribution the duty to.

Our goal is to provide packages that allow you to put together a working
system. If upstream decides that their software will only work with
systemd, we make note of that and move on. We already have enough work to
deal with the fallout in our own infrastructure -- I've mentioned
earlier[1] that libgtk-3-0 pulls in systemd-sysv, so autobuilders will
install it quite often, and we need to make sure this doesn't break
anything.

However, a lot of our software comes from the BSD world and will never
fully switch to systemd, and that software is often used in server contexts
by people who know exactly what they are doing. I don't see why we
shouldn't support these people anymore, especially as they are the ones who
cannot be served equally well by other distributions.

> > That's leaving aside that a reimplementation of systemd's features will
> > tend towards becoming systemd, and we have one of those already. What is
> > the actual goal?

> The actual goal is to allow for different ecosystems that provide the same
> features while embracing different implementation philosophies.

Not quite, I think. The goal is to allow people to omit complexity from
features they will not use. My laptop will never automount USB sticks, so
the code path for that doesn't even need to exist. My web server will die
in a horrible way if someone finds a buffer overflow in the SSL code, and
it will stay down until I can manually investigate, there is no code path
that will give the attacker another chance to get the offset right.

The freedom to configure a system without things I do not want is one of
the main reasons that made me switch over from Windows to Debian, a bit
more than twenty years ago.

> 1. Do we as a project want to do the work to leave open the possibility
>that such resources will materialize?  This means doing things like
>defining a subset of systemd unit features that packages can rely on,
>and requiring (some? all?) packages not use features outside of that
>set.  "No" is a possible answer here, but this is a political question
>with significant consequences, and I think it should be decided
>democratically.

That is work we have to do regardless of whether we want to support
alternatives or not, but in the simple case we just list what is supported
by the systemd version we have decided to ship in the last stable release,
so we can have backport packages with reasonable effort.

Infrastructure I'd like to see is

 - dh_systemd generating a versioned dependency for the unit files given

 - apt being able to blacklist packages and hide packages that depend on
   those

This cuts both ways: for packages with optional systemd integration, I'd
expect "sysvinit" variants to pop up, and we don't want users running
systemd to accidentally install these if a tighter integrated variant is
available.

> 2. Are we as a project *capable* of producing a non-systemd alternative
>that's viable?  If the answer to question 1 is "no," then this question
>doesn't arise.  But it's entirely possible that the answer to question
>1 is "yes" but the answer to this question is still "no."

No, and that's not our job. There are a lot of people out there building
non-systemd systems. We do what we've always done, packaging them to the
best of our ability.

   Simon

[1] https://lists.debian.org/debian-devel/2019/08/msg00278.html

Re: Integration with systemd

[Marco d'Itri]

On Oct 31, "Theodore Y. Ts'o"  wrote:

> handled on a case by case basis.  Exactly how much of a win do we get
> if we use a particular systemd feature in core Debian packaging?  How
> hard is it to emulate that for non-systemd systems?  I don't think
> that decision can be made in the abstract, unless we as an entire
> project want to vote to deliberately, and with malice aforethought,
> kill off sysvinit support in Debian.
We have already voted with apt: less than 1% of new installs use 
sysvinit. If you do not believe that 1% is low enough, then how low do 
you want to put the bar?

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: Integration with systemd

[Russ Allbery]

Josh Triplett  writes:

> Part of the problem is that the people interested in sysvinit don't tend
> to care about those features and often argue that others shouldn't care
> either, and the people interested in those features don't tend to care
> about sysvinit. It's difficult to motivate people to work on
> alternatives for a system they already have and prefer.

*This* is the thing that I feel we need to tackle head-on.  Because there
are a lot of reasons to care about those features, but possibly more
convincingly, as Ted points out, *upstreams* care about those features and
are not going to stop caring about those features just because Debian
packagers do not.

Sure, some systemd features are only marginally better than what came
before (I admit to not being much of a fan of timer units, for instance).
But this will vary by person.  For example, as a security person, I care a
*lot* about namespacing, capability reduction, and seccomp filters, and I
want to use those features in my packages to make the default
configuration more secure.  Other people are going to care a lot about
D-Bus integration, and some upstreams are going to fully embrace socket
activation (which, again, I'll point out is one of the easiest features of
systemd to implement outside of systemd; implementations of the same idea
predated systemd by many years) and just not support any way of spawning
their service that doesn't satisfy the systemd socket activation API.

One can individually say that one doesn't care about those features, but
we just cannot say Debian as a whole should not care about those features.
It doesn't work.  We have to take an affirmative stance on what Debian is
going to do with software that does care about and use those features;
whether we're committing to porting it, whether we're going to kick it out
of the archive (that seems highly unlikely to be viable), whether we're
going to say that software with a hard dependency on systemd features is
allowed to only work on systems running systemd, or some other
alternative.

But saying "no one should care about those features" isn't a choice and
isn't a path forward and we can't stop there as a project.

> That's leaving aside that a reimplementation of systemd's features will
> tend towards becoming systemd, and we have one of those already. What is
> the actual goal?

The actual goal is to allow for different ecosystems that provide the same
features while embracing different implementation philosophies.  I know
that you don't think this is a valuable goal; please let's not have that
argument yet again.  I'm sure you realize by now that some people simply
do not agree with you and do not find your arguments convincing.

> It seems evident based on the history of such efforts that there is
> *not* sufficient people/interest/motivation to reimplement the majority
> of systemd features, let alone doing so in a way that meets the
> additional constraints imposed on such solutions.

I don't think the question has yet been forced.  Up through buster, one
has been able to mostly run Debian on sysvinit without doing this work,
with some pain and some gaps and some issues.  I think we're coming up to
a point where this issue will be forced because both packaging practices
and upstreams are diverging away from sysvinit support.

I think there are two questions here:

1. Do we as a project want to do the work to leave open the possibility
   that such resources will materialize?  This means doing things like
   defining a subset of systemd unit features that packages can rely on,
   and requiring (some? all?) packages not use features outside of that
   set.  "No" is a possible answer here, but this is a political question
   with significant consequences, and I think it should be decided
   democratically.

2. Are we as a project *capable* of producing a non-systemd alternative
   that's viable?  If the answer to question 1 is "no," then this question
   doesn't arise.  But it's entirely possible that the answer to question
   1 is "yes" but the answer to this question is still "no."

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Theodore Y. Ts'o]

On Wed, Oct 30, 2019 at 01:51:07PM -0700, Josh Triplett wrote:
> > So mostly, this isn't going to be up to us.  It's going to be up to
> > the upstream.  Eventually, for each package where upstream has chosen
> > to use these technologies, our choice will be (a) to drop the package
> > from Debian, (b) add backwards compatibility support for systems which
> > haven't drunk the systemd kool-aid, or (c) mark that the package only
> > works for systemd.
> >
> > I think we've mostly accepted that we can't force package maintainers
> > to do (b), and for many packages, such as for example GNOME, (a) will
> > be a non-starter, which means we're left with (c).
> 
> There's a big difference between "we've mostly accepted that we can't do
> otherwise" and "people can safely rely on not getting flamed". The point
> of my mail was the latter.

So I believe that if we believe the former, then we *have* to say that
it's not fair that Debian Maintainers get flamed, and that it is
**not** an obligation of the Debian Maintainer to "fix" what might be
perceived as a regression.

If we need to have a GR, or a TC ruling, to make a statement in Debian
Policy which to me is self-evident, then so be it.  But when we don't
have a choice, we don't have a choice, and it's not fair to make life
unfun for Debian Maintainres who can't dictate the choices of upstream.

> It's not just about upstream. Many of the capabilities of systemd would
> be useful in Debian-native components and packaging. And given that, as
> you observed, upstream can already use such capabilities, then it
> doesn't make sense to stop Debian developers from using those
> capabilities, where doing so provides a benefit.

Perhaps; although if there aren't yet significant number of critical
upstream packages using some particular new systemd functionality,
deliberlately using that feature and completely ruling out some
alternative init system seems to me to be different.  I guess the
difference is one of an act of comission versus act of omission.

And if we do this in core Debian infrastructure, such as say, in dpkg,
then that's *really* different.  That's completely ruling out
sysvinit.  And that to me is different from something like "GNOME no
longer works on sysvinit" (until someone enhances elogind).  In that
case, it's the GNOME Project which screwed over sysvinit, not Debian
--- and we're just saying that it's not the GNOME Debian packaging
team which is obligated to fix things up.

But the dpkg maintainer making a change which screws over sysvinit
*feels* different, both in that it affects all Debian installations,
versus just the ones using a particular GUI, and whether it was Debian
or GNOME that made that particular decision.

Let me be clear, my personal opinion is that Lennart, and the
acceptance of systemd by the vast majority of the major distributions,
means that eventually, most upstreams will be using more and more
systemd features, and people who like sysvinit should just get over
it.  But whether we should accelerate that transition, or let it
happen at a more natural pace, is something which IMHO, needs to be
handled on a case by case basis.  Exactly how much of a win do we get
if we use a particular systemd feature in core Debian packaging?  How
hard is it to emulate that for non-systemd systems?  I don't think
that decision can be made in the abstract, unless we as an entire
project want to vote to deliberately, and with malice aforethought,
kill off sysvinit support in Debian.

Is it really worth it to make that choice now, as opposed to letting
other upstream projects make that choice for the entire Linux
ecosystem naturally?

And some good might come from making this transition more slowly;
maybe it will encourage people who have that sysvinit itch to scratch
to gradually separate out some of systemd's functionality, and
implement it in modules that can be used without pulling in all of
systemd.  I personally don't have that itch; but perhaps others do.

Cheers,

- Ted

Re: Integration with systemd

[Sean Whitton]

Hello,

On Wed 30 Oct 2019 at 12:16PM -07, Russ Allbery wrote:

> It's not clear to me whether we need a faster policy *process* or if we
> just need more hands, but I completely agree that the current policy
> process is too slow.  I haven't had much time to work on it for about five
> years; if it weren't for Sean, it would have stalled entirely.  We still
> don't have good documentation of triggers in Policy, and I think there are
> major gaps in the documentation of multi-arch.  :(

My working assumption is that the problem of not having enough people
interested in writing patches strongly outweighs any slowness inherent
in the process we use, however...

> If anyone reading this has thought about working on Policy and has been
> driven away or turned off by something, please reach out either publicly
> on the list or privately to me (and I suspect Sean would be happy to hear
> from you too).  I'd love to know what drove you away, and think about
> whether there's anything we can do about it.  Deciding what changes to
> make involves some necessary social wrangling, and that may be the hardest
> obstacle, but even hearing specifics about that would be helpful.

... I, likewise, would be grateful for people's experiences, and am
certainly open to making process simplifications/changes.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Integration with systemd

[Josh Triplett]

Russ Allbery wrote:
> One of the options that I find interesting is to enumerate a list of
> features in unit files that Debian supports, and require that any
> Debian init system be able to handle unit files with those features.
> This standardzes an *API* for both package maintainers and upstream

I sincerely hope that we never have to support multiple implementations
of the unit file format with varying levels of support for its
directives. The point of my mail was to propose acknowledging the end of
the lowest-common-denominator approach.

(Also, the surface area of systemd is far broader than just unit file
directives, and many of the directives exist to integrate with other
features.)

> It's similar to what we did for /bin/sh (although this is of course
> much more complicated).

With /bin/sh, we just required that *if* you write #!/bin/sh you only
use /bin/sh features. (And a non-bash /bin/sh already existed, as well
as an industry standard defining the required behavior of /bin/sh, which
pre-dated bash.)

Critically, people *always* had the option to write #!/bin/bash
if they actually used the features of bash.

> Of course, this approach is not viable if it turns out that too few people
> are interested in init system diversity sufficiently to do the reasonably
> substantial implementation work required to maintain a competing
> implementation of the systemd unit features we care about.

Part of the problem is that the people interested in sysvinit don't tend
to care about those features and often argue that others shouldn't care
either, and the people interested in those features don't tend to care
about sysvinit. It's difficult to motivate people to work on
alternatives for a system they already have and prefer.

That's leaving aside that a reimplementation of systemd's features will
tend towards becoming systemd, and we have one of those already. What is
the actual goal? As a limiting case to prove the point, suppose someone
patched systemd to support running as a PID other than 1, underneath
sysvinit, using mechanisms like prctl PR_SET_CHILD_SUBREAPER and cgroups
to let it act like init without being PID 1. (By comparison, that would
*not* be an especially hard problem.) Would that solve the problem of
"must run under sysvinit"?  Something tells me that that solution would
not be considered acceptable by folks who want to keep running sysvinit.
There are additional unspoken constraints here.

It seems evident based on the history of such efforts that there is
*not* sufficient people/interest/motivation to reimplement the majority
of systemd features, let alone doing so in a way that meets the
additional constraints imposed on such solutions. That support isn't
going to materialize out of hope.  What would it take for us to document
the situation and move forward, rather than assuming that it might
change?

Re: Integration with systemd

[Josh Triplett]

On Wed, Oct 30, 2019 at 03:30:17PM -0400, Theodore Y. Ts'o wrote:
> On Wed, Oct 30, 2019 at 05:14:02AM -0700, Josh Triplett wrote:
> > Today, people can't use systemd persistent timers in place of cron (and
> > in place of anacron's "wake up periodically" approach). You have to have
> > a cron job as well, and there's no good mechanism to automatically
> > prevent a cron job from running when running systemd and a corresponding
> > timer exists, so systemd systems would still have to have a pile of cron
> > jobs that wake up just to say "oh, I'm on systemd, exit".
>
> Yep, that's what we did for e2fsprogs's e2scrub.timer unit.  We
> shipped a cron job as well, which exited if systemd was running.  It
> was kludgy as heck, and to be honest, I'm not sure it actually added
> that much value to do it as a systemd timer unit.

(I did explicitly say I'd prefer to avoid delving into any one feature,
rather than the overall issue.)

A few values of using a .timer unit:

- Support for spreading the timer activations out so a network full of
  systems doesn't all poke a server at the same time.

- Persistent timers, which make anacron unnecessary, and which don't
  require waking up and poking anacron periodically.

- Support for timers that only run if another unit is active (without
  having to unconditionally wake up and check), and that only count time
  another unit is active.

- Support for coalescing timer wakeups to improve power management.

- Support for waking the system from sleep, if desired.

But in any case, the point isn't to advocate a specific feature, the
point is to unblock the possibility of using features *when they provide
an advantage*.

> > Systemd user sessions, socket activation, sysusers, dynamic users,
> > systemd-homed, temporary directory setup, transient units, anything
> > talking to the slice or control group APIs, containerization, firstboot,
> > systemd's whole "preset" system-wide configuration and policy mechanism
> > for admins to say "what services do I want launched when installed and
> > what services do I want to leave stopped until I configure them",
> > "stateless system" capabilities, and I'm probably forgetting another
> > dozen.
> 
> Yep, and this is the "embrace, extend, and extinguish" phenomenom of
> systemd which caused so much fear and loathing.  Lennart is the
> Ballmer of the Linux infrastructure world.  :-)

Can we not? Can we just...not? I would love to have a thread about
systemd that manages to avoid gratuitous flames and personal attacks.

> if upstream developers start using more and more of these Systemd
> features, if we're not willing to backport solution for systems that
> don't have socket activation, systemd containerization, etc., we're
> going to be left in a very painful place.

Only "painful" if we continue to attempt to make the world conform to
the limitations and capabilities of sysvinit.

> > Note: I'm not trying to say "we should wholeheartedly adopt every one of
> > these technologies"; please don't make this thread about that, or any
> > one specific technology. The issue is that we don't even have the option
> > of *considering* most of these technologies in the current environment.
> > Even if Policy changed tomorrow to have a full "unless you're using
> > capabilities that alternate init systems don't have" clause, people
> > would still be afraid of using those capabilities or merging patches
> > that do so, lest their work become the subject of a giant flamewar. We
> > should get to a state where people building something interesting using
> > these capabilities and technologies can expect useful feedback, and
> > potentially excitement and collaboration, rather than flames.
> 
> So mostly, this isn't going to be up to us.  It's going to be up to
> the upstream.  Eventually, for each package where upstream has chosen
> to use these technologies, our choice will be (a) to drop the package
> from Debian, (b) add backwards compatibility support for systems which
> haven't drunk the systemd kool-aid, or (c) mark that the package only
> works for systemd.
>
> I think we've mostly accepted that we can't force package maintainers
> to do (b), and for many packages, such as for example GNOME, (a) will
> be a non-starter, which means we're left with (c).

There's a big difference between "we've mostly accepted that we can't do
otherwise" and "people can safely rely on not getting flamed". The point
of my mail was the latter.

> > If we're going to have a GR, part of the goal should be to either
> > confirm the current state that we're never moving very far past the
> > capabilities of sysvinit even when most people don't run it, or that
> > we're allowed to use the full capabilities of our default init system
> > even when there's no equivalent elsewhere.
> 
> I agree, and I think the only thing we can do is to say that upstream
> is allowed to use the full capability of systemd --- not that we could
> dictate to upstream anyway.

It's not 

Re: Integration with systemd

[Sam Hartman]

> "Russ" == Russ Allbery  writes:

Russ> I also completely agree with Josh's message and with your
Russ> other message that we need to make a lot of decisions about
Russ> what systemd features packages can assume, or what workarounds
Russ> they have to have in place if they can't assume those
Russ> features.  There are a variety of different options here,
Russ> ranging from requiring software be fully ported to a sysvinit
Russ> system before it can be packaged for Debian to assuming that
Russ> all Debian systems run systemd, with multiple interesting
Russ> points in between.

Russ> I don't think a GR should attempt to resolve all of the
Russ> specific details, but I do think it should resolve where the
Russ> project wants to be on that spectrum so that we can then roll
Russ> up our sleeves and start turning that into specific policy
Russ> guidance.  There is quite a lot to say in this area (and that
Russ> needs to be said), once we have that direction from the
Russ> project as a whole.

That's my hope.
I'm trying to phrase the GR options as  concrete options about  what we
face today, but I also am hoping that we end up with a general
direction.

It's possible that our general direction might change based on something
we find downstream.  For example perhaps having an alternative for
simple socket activations is doable.  But later perhaps we discover that
we don't have the energy to come up with an alternative for some systemd
feature that *everyone* is using.
Something like that might cause us to need to change direction.

--Sam

Re: Integration with systemd

[Russ Allbery]

"Theodore Y. Ts'o"  writes:

> So mostly, this isn't going to be up to us.  It's going to be up to
> the upstream.  Eventually, for each package where upstream has chosen
> to use these technologies, our choice will be (a) to drop the package
> from Debian, (b) add backwards compatibility support for systems which
> haven't drunk the systemd kool-aid, or (c) mark that the package only
> works for systemd.

> I think we've mostly accepted that we can't force package maintainers
> to do (b), and for many packages, such as for example GNOME, (a) will
> be a non-starter, which means we're left with (c).

I don't think the analysis is *quite* that obvious, since much of (b) can
be done in the form of implementing equivalent features to systemd.  The
unit file format is basically fine -- it might not be what everyone would
have picked, but it's fairly straightforward to parse.  Some systemd
features are already implemented by start-stop-daemon; many others would
be fairly straightforward to implement in at least a basic form (such as
socket activation via a tcpserver-style wrapper, and even a lot of the
namespace and capability support).  A cron daemon could add a parser for
systemd timer units and invoke the corresponding service unit via the same
mechanism.  And so forth.

One of the options that I find interesting is to enumerate a list of
features in unit files that Debian supports, and require that any Debian
init system be able to handle unit files with those features.  This
standardzes an *API* for both package maintainers and upstream, without
going all the way to saying that systemd will always be the init system.
It's similar to what we did for /bin/sh (although this is of course much
more complicated).  The space between those options leaves the door open
for folks who care about init system diversity to implement those features
in some other way, such as via separate stand-alone binaries chained
together with pipes and other normal UNIX tools.

Of course, this approach is not viable if it turns out that too few people
are interested in init system diversity sufficiently to do the reasonably
substantial implementation work required to maintain a competing
implementation of the systemd unit features we care about.

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Theodore Y. Ts'o]

On Wed, Oct 30, 2019 at 05:14:02AM -0700, Josh Triplett wrote:
> 
> Today, people can't use systemd persistent timers in place of cron (and
> in place of anacron's "wake up periodically" approach). You have to have
> a cron job as well, and there's no good mechanism to automatically
> prevent a cron job from running when running systemd and a corresponding
> timer exists, so systemd systems would still have to have a pile of cron
> jobs that wake up just to say "oh, I'm on systemd, exit".

Yep, that's what we did for e2fsprogs's e2scrub.timer unit.  We
shipped a cron job as well, which exited if systemd was running.  It
was kludgy as heck, and to be honest, I'm not sure it actually added
that much value to do it as a systemd timer unit.  The person who
contributed the code for upstream did it that way, I suspect because
he was curious about learning about systemd, and fortunately he also
submitted a non-systemd alternative, probably because the enterprise
distro that his $COMPANY supported wasn't using systemd yet.

So not a lot of upside; and interestingly enough, the Debian sysvinit
support for e2scrub was slightly buggy, it took a surprisingly long
time before someone reported it, and we finally got it fixed.

> Systemd user sessions, socket activation, sysusers, dynamic users,
> systemd-homed, temporary directory setup, transient units, anything
> talking to the slice or control group APIs, containerization, firstboot,
> systemd's whole "preset" system-wide configuration and policy mechanism
> for admins to say "what services do I want launched when installed and
> what services do I want to leave stopped until I configure them",
> "stateless system" capabilities, and I'm probably forgetting another
> dozen.

Yep, and this is the "embrace, extend, and extinguish" phenomenom of
systemd which caused so much fear and loathing.  Lennart is the
Ballmer of the Linux infrastructure world.  :-)

We were kind of lucky because in Debian Stretch, we were stuck on a
pretty old version of systemd that didn't have a lot of these "embrace
and extend" features.  I'm kind of surprised this didn't bite us more
in Buster, but it's inevitably going to be an issue moving forward,
and it might not be up to us at Debian; if upstream developers start
using more and more of these Systemd features, if we're not willing to
backport solution for systems that don't have socket activation,
systemd containerization, etc., we're going to be left in a very
painful place.

> Note: I'm not trying to say "we should wholeheartedly adopt every one of
> these technologies"; please don't make this thread about that, or any
> one specific technology. The issue is that we don't even have the option
> of *considering* most of these technologies in the current environment.
> Even if Policy changed tomorrow to have a full "unless you're using
> capabilities that alternate init systems don't have" clause, people
> would still be afraid of using those capabilities or merging patches
> that do so, lest their work become the subject of a giant flamewar. We
> should get to a state where people building something interesting using
> these capabilities and technologies can expect useful feedback, and
> potentially excitement and collaboration, rather than flames.

So mostly, this isn't going to be up to us.  It's going to be up to
the upstream.  Eventually, for each package where upstream has chosen
to use these technologies, our choice will be (a) to drop the package
from Debian, (b) add backwards compatibility support for systems which
haven't drunk the systemd kool-aid, or (c) mark that the package only
works for systemd.

I think we've mostly accepted that we can't force package maintainers
to do (b), and for many packages, such as for example GNOME, (a) will
be a non-starter, which means we're left with (c).

Of course, someone other than the package maintainer can provide the
backwards compatibility support, and they can either try to pursuade
upstream to accept the patch, or if the change is not too invasive and
not oo onerous to support, they can try to pursuade the Debian
Packager to include it in debian/patches.

> If we're going to have a GR, part of the goal should be to either
> confirm the current state that we're never moving very far past the
> capabilities of sysvinit even when most people don't run it, or that
> we're allowed to use the full capabilities of our default init system
> even when there's no equivalent elsewhere.

I agree, and I think the only thing we can do is to say that upstream
is allowed to use the full capability of systemd --- not that we could
dictate to upstream anyway.  And unless we are willing to force
package maintainers to drop packages or to do the work to add the
backwards compatibility, which I think very much goes against the
Debian ethos, especially given the extent of systemd's "embrace and
extend" strategy, I don't think we have a choice but acknowledge
reality and accept that some packages may simply 

Re: Integration with systemd

[Russ Allbery]

Simon Richter  writes:

> I believe this GR is less about technical than about organizational
> aspects. If we want to fully adopt systemd, we need a faster policy
> process, which will disenfranchise users with less-common use cases,
> because there is no time to integrate their concerns (I'm also sceptical
> that we have the necessary influence upstream to alter the trajectory of
> development).

It's not clear to me whether we need a faster policy *process* or if we
just need more hands, but I completely agree that the current policy
process is too slow.  I haven't had much time to work on it for about five
years; if it weren't for Sean, it would have stalled entirely.  We still
don't have good documentation of triggers in Policy, and I think there are
major gaps in the documentation of multi-arch.  :(

If anyone reading this has thought about working on Policy and has been
driven away or turned off by something, please reach out either publicly
on the list or privately to me (and I suspect Sean would be happy to hear
from you too).  I'd love to know what drove you away, and think about
whether there's anything we can do about it.  Deciding what changes to
make involves some necessary social wrangling, and that may be the hardest
obstacle, but even hearing specifics about that would be helpful.

I also completely agree with Josh's message and with your other message
that we need to make a lot of decisions about what systemd features
packages can assume, or what workarounds they have to have in place if
they can't assume those features.  There are a variety of different
options here, ranging from requiring software be fully ported to a
sysvinit system before it can be packaged for Debian to assuming that all
Debian systems run systemd, with multiple interesting points in between.

I don't think a GR should attempt to resolve all of the specific details,
but I do think it should resolve where the project wants to be on that
spectrum so that we can then roll up our sleeves and start turning that
into specific policy guidance.  There is quite a lot to say in this area
(and that needs to be said), once we have that direction from the project
as a whole.

-- 
Russ Allbery (r...@debian.org)  

Re: Integration with systemd

[Josh Triplett]

[Please CC me on replies.]

Simon Richter wrote:
> On Wed, Oct 30, 2019 at 05:14:02AM -0700, Josh Triplett wrote:
> > If we're going to have a GR, part of the goal should be to either
> > confirm the current state that we're never moving very far past the
> > capabilities of sysvinit even when most people don't run it, or that
> > we're allowed to use the full capabilities of our default init system
> > even when there's no equivalent elsewhere.
>
> I doubt we have that choice. Those packages embracing systemd will
> require all the features, and it will be hard enough to put our foot
> down and insist that they restrict themselves to the set in the stable
> release.

(Leaving the gratuitously adversarial assumption of inevitable problems
aside...)

By "the stable release", do you mean "the stable release of systemd" or
"the stable release of Debian"? For the former, I certainly don't think
it's reasonable to package software that depends on an unreleased or
unpackaged version of systemd, and I doubt much software does so. For
the latter, 1) that doesn't seem like a very reasonable limitation, 2)
systemd supports upgrading itself without a reboot, and 3) even if we
needed that for some reason, the ability to use the features present in
stable is still far better than not being able to use any of the
features at all.

> If we want to fully adopt systemd, we need a faster policy process

As Sam's original mail suggested, part of the problem is that policy
folks don't feel empowered to create any policy around usage of systemd
features, because the *official* stance is still "your package must work
with sysvinit". Let's fix that, and *then* we can worry about whether we
have a problem with policy keeping up. I have confidence in the Debian
Policy Editors and their ability to handle various use cases and
requirements.

Also, not every aspect of the way two packages interact is or needs to
be covered by Policy. Some aspects do (we should absolutely have policy
regarding any use of presets, or how we generally want to handle users
distro-wide, or when a periodic timer is appropriate or inappropriate),
while other aspects don't (usage of transient units to run jobs, usage
of systemd-tmpfiles, usage of timer units assuming a periodic timer is
appropriate).

> Priority: extra

https://lintian.debian.org/tags/priority-extra-is-replaced-by-priority-optional.html

Re: Integration with systemd

[Simon Richter]

Hi,

On Wed, Oct 30, 2019 at 05:14:02AM -0700, Josh Triplett wrote:

> If we're going to have a GR, part of the goal should be to either
> confirm the current state that we're never moving very far past the
> capabilities of sysvinit even when most people don't run it, or that
> we're allowed to use the full capabilities of our default init system
> even when there's no equivalent elsewhere.

I doubt we have that choice. Those packages embracing systemd will require
all the features, and it will be hard enough to put our foot down and
insist that they restrict themselves to the set in the stable release.

Debian is a community distribution, and we're competing with distributions
maintained by full-time employees that have less trouble following the
releases of a project driven by other full-time employees. Of course we're
having a hard time catching up if we keep our processes.

On the other hand, these very same processes have been what distinguished
us from other distributions in the past, and what has led to our current
reputation as a very stable distribution with very few quality issues.

I believe this GR is less about technical than about organizational
aspects. If we want to fully adopt systemd, we need a faster policy
process, which will disenfranchise users with less-common use cases,
because there is no time to integrate their concerns (I'm also sceptical
that we have the necessary influence upstream to alter the trajectory of
development).

This is a massive change for Debian, because the technology decisions drive
policy decisions here, and we need to formulate a vision for where the
project will be in a few years. Can we provide something unique?

One option I'd like to see on the ballot is splitting packages into
systemd-integrated and standalone versions where these deviate too much
from each other, and making it normal that such packages are
team-maintained. If we can't find people for one variant, that is fine, but
it should be the exception, and it should be normal and acceptable to
depend on or conflict with systemd-sysv as needed (those that conflict
would go to Priority: extra).

There is still a massive Free Software scene outside the Linux+systemd
world, so I expect that we won't run out of software that works without
systemd anytime soon.

   Simon