Re: Policy consensus on transition when removing initscripts.

[Matthew Vernon]

Andreas Henriksson  writes:

> If you want me to take suggestions like coordination seriously then
> please consider adressing https://bugs.debian.org/934463 soon or admit
> that sysvinit maintenance lacks the resources to do coordinated
> transitions. Dropping things and letting people pick them up if they
> think they are still useful seems to be the only practical way forward.

I thought the suggestion I'd made as the last comment on that bug
(roughly, consider a migration to orphan-sysvinit-scripts after the
bookworm release) was not entirely unreasonable? It's not entirely
straightforward (and I'm not sure udev rules are best handled thus), but
still probably better co-ordinated via that bug report rather than this
-devel thread...

Regards,

Matthew

-- 
"At least you know where you are with Microsoft."
"True. I just wish I'd brought a paddle."
http://www.debian.org

Re: Policy consensus on transition when removing initscripts.

[Ivan Shmakov]

> "Sam" == Sam Hartman  wrote:
> "Simon" == Simon Richter  writes:
> "Sam" == On 6/29/23 01:56, Sam Hartman wrote:

 Sam> It also seems a bit strange to require more from the maintainer
 Sam> when they are dropping an init script than we would if a
 Sam> maintainer started depending on a feature like socket activation
 Sam> such that their package simply would not work without systemd.

 Simon> This would be a case where the init script and the systemd
 Simon> unit deviate in functionality.  I don’t see a problem with
 Simon> that, and my expectation is generally that the people running
 Simon> sysvinit and the people running systemd have different
 Simon> expectations here anyway.

That’s my impression as well.  Personally, I’m not going to
expect for daemons started via different init mechanisms to
behave identically, either.

 Sam> It would be entirely permitted under GR 2019-002 for me to build
 Sam> a version of krb5-kdc that was compiled to depend on socket
 Sam> activation and would not work without systemd.

 Sam> I would not be required to provide any transition when doing that.
 Sam> (To be clear, I have no such plans, and in the case of krb5kdc
 Sam> don’t currently think it would be a good idea).

I’m not sure it’s solely within the scope of the GR.

When doing development and testing, I’ve found it useful to be
able to start server processes from my own code.  So, for example,
I’d have a wrapper that’d do initdb on a temporary directory,
start a PostgreSQL server process there, load the schema and
test data, run a test suite, kill the server and remove the
directory.  No system-wide PostgreSQL instance is ever started
on that system, so the choice of the init system is hardly
relevant.  (Never had a reason to start a KDC in such a way,
but I’m not going to swear no one will ever need it, either.)

I’d rather appreciate if this particular rug isn’t pulled from
under my feet at some future date.

Not that I’d expect it to be, mind.  So far my experience with
Debian packages has been that they tend to come with as much
upstream features enabled as possible (to the point where I
sometimes wish it weren’t the case.)

If there’re concerns that a package maintainer might decide to
deviate from such a practice for ill reasons, I suppose it should
be codified within the policy.  Otherwise, so long as as a
given daemon can be started via fork and exec, it can be started
from an init.d script just as well; while removing support for
such use might affect users beyond those who rely on init.d.

PS.  The orphan-sysvinit-scripts package now has my attention.

-- 
FSF associate member #7257  np. A Gentle Place by Lisa Lynne

Re: Replaces without Breaks or Conflicts harmful? [was: Re: Policy consensus on transition when removing initscripts.]

[Simon Richter]

Hi,

On 6/29/23 04:49, Helmut Grohne wrote:


* Package A 1.0-1 is installed providing file F.
* File F is moved to package B as of package A 1.0-3.
* User installs package B, which replaces the file in package A.
* User uninstalls package B.



F is now gone, even though it's supposed to be still shipped by A 1.0-1.



I am convinced by this. I think this is a sufficiently bad footgun to
simply forbid Replaces that are not covered by a suitable Breaks or
Conflicts relation.


That is already in Policy 7.6.1, with a footnote that gives exactly this 
explanation.


   Simon

Re: Replaces without Breaks or Conflicts harmful? [was: Re: Policy consensus on transition when removing initscripts.]

[Luca Boccassi]

On Thu, 29 Jun 2023 at 13:34, Helmut Grohne  wrote:
>
> Hi Bas,
>
> On Thu, Jun 29, 2023 at 08:19:51AM +0200, Sebastiaan Couwenberg wrote:
> > On 6/28/23 21:49, Helmut Grohne wrote:
> > > Debian GIS Project 
> > > postgis
> > > qgis
> >
> > Why is postgis on this list?
> >
> >  $ grep -c Replaces debian/control*
> >  debian/control:0
> >  debian/control.in:0
>
> Thanks for asking. You identified another source of false positives that
> slipped my mind when doing the analysis. The underlying data source did
> not use unstable, but every suite from bullseye to experimental
> including -security and -backports. As it happens, bookworm's
> postgresql-15-postgis-3-scripts has versioned Replaces that are not
> matched with Breaks or Conflicts. I don't think we are going to fix that
> in bookworm and you've fixed it in unstable. So yeah, this list has more
> false positives than originally assumed.

In case it's useful, src:dpdk is also a false positive, I suspect
because the versions in the breaks vs replaces are slightly different
- probably clerical mistakes, like a missing '~'.

> I could improve the numbers, but to me the numbers I've given being a
> tight upper bound seems good enough and lintian.debian.org will give us
> precise and current numbers once my patch is merged. Does that seem
> sensible to you as well?

Sadly, as I found out recently for the scripts mbf, lintian.d.o is
borken and has ~2 years old stale data. We should probably consider
taking it down, given it appears fully working and can be queried, but
just returns stale data with no indication that it is stale on the
face of it, without manual checks.

Kind regards,
Luca Boccassi

Re: Replaces without Breaks or Conflicts harmful? [was: Re: Policy consensus on transition when removing initscripts.]

[Sebastiaan Couwenberg]

On 6/29/23 12:32, Helmut Grohne wrote:

On Thu, Jun 29, 2023 at 08:19:51AM +0200, Sebastiaan Couwenberg wrote:

On 6/28/23 21:49, Helmut Grohne wrote:

Debian GIS Project 
 postgis
 qgis


Why is postgis on this list?

  $ grep -c Replaces debian/control*
  debian/control:0
  debian/control.in:0


Thanks for asking. You identified another source of false positives that
slipped my mind when doing the analysis. The underlying data source did
not use unstable, but every suite from bullseye to experimental
including -security and -backports. As it happens, bookworm's
postgresql-15-postgis-3-scripts has versioned Replaces that are not
matched with Breaks or Conflicts. I don't think we are going to fix that
in bookworm and you've fixed it in unstable. So yeah, this list has more
false positives than originally assumed.


Thanks for clarifying your data sources.

FWIW, qgis is fixed in git.


I could improve the numbers, but to me the numbers I've given being a
tight upper bound seems good enough and lintian.debian.org will give us
precise and current numbers once my patch is merged. Does that seem
sensible to you as well?


If you intend to file bugs you should only look at unstable & experimental.

For the sake of argument your list is fine.

Kind Regards,

Bas

--
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

Re: Replaces without Breaks or Conflicts harmful? [was: Re: Policy consensus on transition when removing initscripts.]

[Helmut Grohne]

Hi Bas,

On Thu, Jun 29, 2023 at 08:19:51AM +0200, Sebastiaan Couwenberg wrote:
> On 6/28/23 21:49, Helmut Grohne wrote:
> > Debian GIS Project 
> > postgis
> > qgis
> 
> Why is postgis on this list?
> 
>  $ grep -c Replaces debian/control*
>  debian/control:0
>  debian/control.in:0

Thanks for asking. You identified another source of false positives that
slipped my mind when doing the analysis. The underlying data source did
not use unstable, but every suite from bullseye to experimental
including -security and -backports. As it happens, bookworm's
postgresql-15-postgis-3-scripts has versioned Replaces that are not
matched with Breaks or Conflicts. I don't think we are going to fix that
in bookworm and you've fixed it in unstable. So yeah, this list has more
false positives than originally assumed.

I could improve the numbers, but to me the numbers I've given being a
tight upper bound seems good enough and lintian.debian.org will give us
precise and current numbers once my patch is merged. Does that seem
sensible to you as well?

Helmut

Re: Replaces without Breaks or Conflicts harmful? [was: Re: Policy consensus on transition when removing initscripts.]

[Sebastiaan Couwenberg]

On 6/28/23 21:49, Helmut Grohne wrote:

Debian GIS Project 
postgis
qgis


Why is postgis on this list?

 $ grep -c Replaces debian/control*
 debian/control:0
 debian/control.in:0

Kind Regards,

Bas

--
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Michael Biebl  writes:

> While I find this discussion really interesting, is this really relevant
> for orphan-sysvinit-scripts? After all, it doesn't ship any conffiles in
> /etc, i.e. it doesn't take over any (conf)files from packages that
> dropped their initscript.

> Have you actually looked at what orphan-sysvinit-scripts does?

Uh... good call.  *embarrassed look*

I'm sorry, I appear to have added a lot of noise to this thread based on
some entirely incorrect assumptions.  I also missed the (in retrospect
rather obvious) observation that the init script is a conffile.

I have now looked at the package, which I should have done at that the
start, and I think the points that remain valid now that I've corrected my
misunderstanding are:

* There is some documentation in the package about how to add a new
  script, but it's aimed at sysvinit maintainers.  I continue to think
  it's reasonable to ask maintainers who are dropping an init script to
  open a bug against orphan-sysvinit-scripts, although I respect the
  opinion that it would be better to automate this (and I do think
  automating it will be more effective in the long run, since we have no
  way of reaching maintainers that will get them all to open bugs or even
  be aware that they should, as much as we might prefer otherwise).  But
  we need documentation aimed at package maintainers somewhere, whether
  that be Policy, the Developers' Reference, the wiki, etc., in order to
  make that request.

* Coordination still seems like something we'd normally do with this type
  of transition, even though unpurged packages might keep their init
  scripts around and Replaces is arguably not required because of how the
  package works (although I don't really know what the interaction between
  Replaces and ucf is and would hesitate to say for certain whether
  Replaces is helpful here or not).  I saw Sam's argument that an
  uncoordinated transition here may be reasonable, and I don't have a
  strong opinion about this, although it also feels (at least to me) like
  the costs of not coordinating outweigh the costs of doing the
  coordination.

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Sam Hartman]

> "Simon" == Simon Richter  writes:

>> It also seems a bit strange to require more from the maintainer
>> when they are dropping an init script than we would if a
>> maintainer started depending on a feature like socket activation
>> such that their packkage simply would not work without systemd.

Simon> This would be a case where the init script and the systemd
Simon> unit deviate in functionality. I don't see a problem with
Simon> that, and my expectation is generally that the people running
Simon> sysvinit and the people running systemd have different
Simon> expectations here anyway.

It would be entirely permitted under GR 2019-002 for me to build a
version of krb5-kdc that was compiled to depend on socket activation and
would not work without systemd.
I would not be required to provide any transition when doing that.
(To be clear, I have no such plans, and in the case of krb5kdc don't
currently think it would be a good idea).

Re: Policy consensus on transition when removing initscripts.

[Simon Richter]

Hi,

On 6/29/23 01:56, Sam Hartman wrote:


 Russ> This feels like exactly the kind of problem that normally
 Russ> Debian goes to some lengths to avoid creating, but it sounds
 Russ> like several commenters here don't think the effort is worth
 Russ> it?



Normally, Debian spends a fair bit of effort to avoid these kind of
breakages.
But I think init systems other than systemd are in kind of a special case.


I agree, but it doesn't make a difference anyway.

The init scripts are conffiles. Dpkg should not remove them on upgrade, 
so no such window exists, unless maintainers explicitly remove these 
files, which would be "deleting user data" and therefore wrong, except 
in very specific cases.


Longer term, we might either want to also develop a strategy to remove 
these files, or decide to not bother because the impact is minimal, 
especially without the compatibility layer and with immutable images.



It also seems a bit strange to require more from the maintainer when
they are dropping an init script than we would if a maintainer started
depending on a feature like socket activation such that their packkage
simply would not work without systemd.


This would be a case where the init script and the systemd unit deviate 
in functionality. I don't see a problem with that, and my expectation is 
generally that the people running sysvinit and the people running 
systemd have different expectations here anyway.


Very few services depend on systemd and are completely unable to manage 
their own listener sockets, and those that do never had an init script 
in the first place and are not in use on any machine not running 
systemd, so there is no regression.


In general, the sysvinit crowd prefers "traditional" Unix daemons, many 
of which also exist on BSD, so these are unlikely to lose the ability to 
work outside the systemd environment.


   Simon

Re: Policy consensus on transition when removing initscripts.

[Sam Hartman]

> "Russ" == Russ Allbery  writes:

Russ> I think the open question is whether we're happy to just break
Russ> init scripts in unstable, since that migration path means
Russ> there will always be a window where a fully-upgraded unstable
Russ> system has no init script for a package that previously had
Russ> one and in the future will have one again.

Russ> This feels like exactly the kind of problem that normally
Russ> Debian goes to some lengths to avoid creating, but it sounds
Russ> like several commenters here don't think the effort is worth
Russ> it?

Normally, Debian spends a fair bit of effort to avoid these kind of
breakages.
But I think init systems other than systemd are in kind of a special case.

When I go back and look at GR 2019-002, I think the following are true:

* At the time of that GR, only systemd was  an officially supported init
  system for normal operation.  Rationale: GR 2019-002 allows
  maintainers to use any systemd facility within some broad constraints
  and does not require them to provide an alternative.

* At the time of the GR, services like elogind that supported
* development of alternate init systems *were important to support the
* development of alternate init systems*.
That is, at the time of the GR, when thinking about how important some
* breakage is, we need to consider two cases:

  * Does it break the systemd use case?  If it breaks things for someone
running systemd then Debian's normal approach to breakage is  in
play.  I.E. we try to avoid breakage and breakage of upgrades tends
to be a (often RC) bug.

  * Does it break things for people doing development of 
alternate init systems.

  * At the time of GR 2019-002,  init systems other than systemd were
only important in so far as they were tools in the process of
developing alternatives to systemd.  The use case of someone just
using an init system other than systemd outside of the context of
developing init systems or systemd alternatives was not supported.

My take is that the normal rules about breakage in unstable might well
be relaxed in such a situation.  After all the use case we care about is
explicitly the case where someone is actively working on the moving
parts involved.
It also seems a bit strange to require more from the maintainer when
they are dropping an init script than we would if a maintainer started
depending on a feature like socket activation such that their packkage
simply would not work without systemd.
In that case, GR 2019-002 leaves it entirely up to the maintainer
whether they will use that systemd facility and whether they will
support alternatives.

While we're thinking back to 2019, a few other notes:

  
* At the time of GR 2019-002, init scripts were explicitly supported as a
  way to start services.


* The GR explicitly allows us to change the position of the GR without
  resorting to a new GR.  Processes such as the policy process or  a
  debian-devel discussion are sufficient.  Rationale: it is a statement
  of the day, and explicitly has text to that effect at the top.

* So, it is fine that we're effectively removing init scripts as a
  first-class option for starting services and saying that the
  officially supported option is service units.

* It is possible that the importance of supporting development of
  alternate init systems has changed since the time of  GR 2019-002.  We
  are not bound by that GR.  For example we could consider what
  developments of init system technology have been facilitated since
  2019.

* Even if our position has changed since GR 2019-002, I suspect that the text 
from proposal F still sets a minimum bound on
  the support for init systems other than systemd:
>  We believe that the
>  benefits of supporting multiple init systems do not outweigh the
>  costs.
>Debian can continue to provide and explore other init systems, but
>systemd is the only officially supported init system. Wishlist bug
>reports with patches can be submitted, which package maintainers should
>review like other bug reports with patches. 

Here's some quotes from GR 2019-002's winning option to justify my
thoughts above.

  

>However, Debian remains an environment where developers and users can
>explore and develop alternate init systems and alternatives to systemd
>features. Those interested in exploring such alternatives need to
>provide the necessary development and packaging resources to do that
>work. Technologies such as elogind that facilitate exploring
>alternatives while running software that depends on some systemd
>interfaces remain important to Debian. It is important that the project
>support the efforts of developers working on such technologies where
>there is overlap between these technologies and the rest of the project,
>for example by reviewing patches and participating in discussions in a
>timely manner. 

>Packages should include service units or init scripts to start 

Re: Policy consensus on transition when removing initscripts.

[Holger Levsen]

On Sun, Jun 25, 2023 at 03:15:24PM +0100, Mark Hindley wrote:
> Debian Policy no longer requires that packages which provide a systemd 
> .service
> file also provide an initscript. This permits maintainers who so wish to 
> remove
> initscripts from their packages. However, initscripts remain used and 
> useful[1],
> and uncoordinated removal can have significant effects on users' systems[2].
> 
> With the encouragement of the Technical Committee[3] and despite some
> unavoidable deficiencies resulting consequent on keeping initscripts without
> their intended package[4], orphan-sysvinit-scripts has collected and 
> maintained
> some dropped initscripts. However, the process surrounding this has not been
> defined in Policy. Indeed, #975075[5] contains a number of suggestions that 
> have
> not yet been followed through.

I'm not sure Debian Policy is the best place to document this, because Debian 
Policy
documents what packages *must* comply with, while legacy initscripts are a thing
of the past which still are permitted (and liked & prefered by some), so *maybe*
src:dev-ref would be a better place for documenting those best practices?


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Homelessness exists not because the housing systemn is not working, but because
this is the way it works. - Peter Marcuse.


signature.asc
Description: PGP signature

Re: Policy consensus on transition when removing initscripts.

[Michael Biebl]

Am 27.06.23 um 19:31 schrieb Russ Allbery:

Simon Richter  writes:


The only thing we actually need is a versioned Replaces that allows
orphan-sysvinit-scripts to take over ownership of the conffile.



Conflicts is unneeded here, and the daemon package does not need to
declare any relationship. They can use



 Depends: systemd-sysv | orphan-sysvinit-scripts



but really that doesn't do much because orphan-sysvinit-scripts is going
to be pulled in anyway, so I'd rather avoid the clutter.


Normally Conflicts is always added with Replaces because otherwise you can
have the following situation:

* Package A version 1.0-1 is installed providing file F.
* File F is moved to package B as of package A 1.0-3.
* User installs package B, which replaces the file in package A.
* User upgrades package A to version 1.0-2 (*not* 1.0-3). Or, rather,
   tries, because this will fail with an error due to the file conflict.

Unless I misunderstand how this works, I believe the Conflicts is
necessary to force dependency resolution to realize version 1.0-3 of
package A is needed when the F-providing version of package B is
installed.

My personal opinion is that if we're going to do this, we should do it
correctly.


Less prone to errors than a manual process might be to watch
automatically where legacy startup scripts disappear anyway; it's not
that complicated to do. People tend to forget things.



No, we shouldn't set the bar this low. We're talking about DDs here,
they should all have passed P and T tests.


Passing P and T doesn't help you not forget things.  Debian packaging
is already hard enough; we should automate as much as we possibly can.  A
comprehensive checklist of everything you're supposed to think about when
packaging a Debian package doesn't exist (Policy is certainly shy of
that), and if it did, would form a hardcover book large enough to use as a
boat anchor.  We should be tryingn to whittle that down over time with
automation and tools, not rely on people's memory and constantly
re-reading Policy.



While I find this discussion really interesting, is this really relevant 
for orphan-sysvinit-scripts? After all, it doesn't ship any conffiles in 
/etc, i.e. it doesn't take over any (conf)files from packages that 
dropped their initscript.

Have you actually looked at what orphan-sysvinit-scripts does?


OpenPGP_signature
Description: OpenPGP digital signature

Re: Policy consensus on transition when removing initscripts.

[Andreas Henriksson]

Hello Thomas Goirand,

On Wed, Jun 28, 2023 at 10:01:13AM +0200, Thomas Goirand wrote:
> On 6/27/23 17:45, Andreas Henriksson wrote:
> > Dropping things and letting people pick them up if they
> > think they are still useful seems to be the only practical way forward.
> 
> It's not. As written by Russ in this thread, filling a bug against
> orphan-sysvinit-scripts so it takes over the abandoned script is. I wouldn't
> mind seeing this mandatory, and written in the policy.

You seem to have missed this part of my message:
https://bugs.debian.org/934463

Please note: Date: Sun, 11 Aug 2019 12:46:52 +0200
This soon turns 4 years old!

(And my original message still contains my advice to put this in the
initscripts package because as I understand it the orphan-* is
supposed to be optional, but you do as you please.)

As an absolute minimum here, I request that any kind of mandatory
coordination comes with a specified official timeout (that's way less
than 4 years obviously).

Regards,
Andreas Henriksson

Re: Policy consensus on transition when removing initscripts.

[Bastian Blank]

On Wed, Jun 28, 2023 at 10:01:13AM +0200, Thomas Goirand wrote:
> It's not. As written by Russ in this thread, filling a bug against
> orphan-sysvinit-scripts so it takes over the abandoned script is. I wouldn't
> mind seeing this mandatory, and written in the policy.

I do.  This also does not have anything to do with how packages look or
work.  This is about non-technical interaction, so at most should be
part of the developers reference.

But this still bears the question why this needs manual work and in
doing this distribute the work away to other people.

Bastian

-- 
We fight only when there is no other choice.  We prefer the ways of
peaceful contact.
-- Kirk, "Spectre of the Gun", stardate 4385.3

Re: Policy consensus on transition when removing initscripts.

[Luca Boccassi]

On Wed, 28 Jun 2023, 06:31 Paul Wise,  wrote:

> On Tue, 2023-06-27 at 09:36 +0100, Luca Boccassi wrote:
>
> > That has been implemented a long time ago, services can set
> > ProtectProc= so that processes run with hidepid:
> >
> >
> https://freedesktop.org/software/systemd/man/systemd.exec.html#ProtectProc=
>
> Thats opt-in and for services only, there are folks who want to run
> their entire system with hidepid=2, including root/user command-line
> and graphical sessions, systemd doesn't support this setup.
>

On the global proc instance yes that cannot possibly work, as the kernel
doesn't make the required information to do process management available.
But with the per service setting you can set it on the graphical session
(eg gdm.service), however it's likely that it won't cover everything, and
also unrelated bits might break.

>

Re: Policy consensus on transition when removing initscripts.

[Thomas Goirand]

On 6/27/23 17:45, Andreas Henriksson wrote:

Dropping things and letting people pick them up if they
think they are still useful seems to be the only practical way forward.


It's not. As written by Russ in this thread, filling a bug against 
orphan-sysvinit-scripts so it takes over the abandoned script is. I 
wouldn't mind seeing this mandatory, and written in the policy.


Cheers,

Thomas Goirand (zigo)

Re: Policy consensus on transition when removing initscripts.

[Guillem Jover]

On Tue, 2023-06-27 at 21:05:57 -0700, Russ Allbery wrote:
> Simon Richter  writes:
> > On 6/28/23 02:31, Russ Allbery wrote:
> >> Normally Conflicts is always added with Replaces because otherwise you can
> >> have the following situation:
> 
> >> * Package A version 1.0-1 is installed providing file F.
> >> * File F is moved to package B as of package A 1.0-3.
> >> * User installs package B, which replaces the file in package A.
> >> * User upgrades package A to version 1.0-2 (*not* 1.0-3). Or, rather,
> >>tries, because this will fail with an error due to the file conflict.
> 
> > No, that is fine. "Replaces: A (<< 1.0-3)" is sufficient here that the
> > file is not unpacked from A 1.0-2.
> 
> Oh!  Of course.  Okay.

Yes, Replaces works regardless of the order. Maybe you had in mind
that this was not always the case, and dpkg could fail to unpack
depending on the order, but that was fixed long long time ago
(dpkg 1.13.x for the general fix and 1.15.x to handle downgrades).

> In that case, I don't actually know why we usually use Conflicts with
> Replaces.  Maybe we don't really need to?

We use Breaks to avoid getting into a situation where neither of the
packages ship the file, which might make things well, break. :)
(And when we do not want to impose a Depends, but instead just force a
versioned constraint on something else.)

Regards,
Guillem

Re: Policy consensus on transition when removing initscripts.

[Jakub Wilk]

* Russ Allbery , 2023-06-27 23:19:
for some reason I thought we normally always combine Replaces with 
Breaks or Conflicts even for other cases.


There is a good reason.

Consider the following scenario:

* Package A 1.0-1 is installed providing file F.
* File F is moved to package B as of package A 1.0-3.
* User installs package B, which replaces the file in package A.
* User uninstalls package B.

F is now gone, even though it's supposed to be still shipped by A 1.0-1.

--
Jakub Wilk

Re: Policy consensus on transition when removing initscripts.

[Simon Richter]

Hi,

On 6/28/23 15:19, Russ Allbery wrote:


Yeah, I knew that part, but for some reason I thought we normally always
combine Replaces with Breaks or Conflicts even for other cases.  Maybe I
just made that up and confused myself.


No, we just have very few use cases for Replaces alone these days, since 
typically files moving from one package to another will take place 
within the same source package, and they will just either have a strong 
dependency on the exact same version, or a conflict with all other versions.


   Simon

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Simon Richter  writes:
> On 6/28/23 13:05, Russ Allbery wrote:

>> In that case, I don't actually know why we usually use Conflicts with
>> Replaces.  Maybe we don't really need to?

> Replaces+Conflicts together has a special meaning, that is used for
> replacing a package completely in an atomic operations, such as when a
> package is being renamed.

Yeah, I knew that part, but for some reason I thought we normally always
combine Replaces with Breaks or Conflicts even for other cases.  Maybe I
just made that up and confused myself.

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Simon Richter]

Hi,

On 6/28/23 13:05, Russ Allbery wrote:


In that case, I don't actually know why we usually use Conflicts with
Replaces.  Maybe we don't really need to?


Replaces+Conflicts together has a special meaning, that is used for 
replacing a package completely in an atomic operations, such as when a 
package is being renamed.


Replaces alone defines how file conflicts are to be resolved. Conflicts 
alone stops installation if the conflicting package is still installed, 
and the frontend is responsible for uninstalling the other package 
first. With both Replaces and Conflicts, the new package can be 
installed over the old one, causing the old package to be deinstalled at 
the same time, but only if unpacking the new package succeeds.


This is very useful for replacing a package that other packages depend 
on without going through an invalid state that needs --force to get to.


Policy 7.4 specifies this behaviour.

   Simon

Re: Policy consensus on transition when removing initscripts.

[Paul Wise]

On Tue, 2023-06-27 at 09:36 +0100, Luca Boccassi wrote:

> That has been implemented a long time ago, services can set
> ProtectProc= so that processes run with hidepid:
> 
> https://freedesktop.org/software/systemd/man/systemd.exec.html#ProtectProc=

Thats opt-in and for services only, there are folks who want to run
their entire system with hidepid=2, including root/user command-line
and graphical sessions, systemd doesn't support this setup. 

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Simon Richter  writes:
> On 6/28/23 02:31, Russ Allbery wrote:

>> Normally Conflicts is always added with Replaces because otherwise you can
>> have the following situation:

>> * Package A version 1.0-1 is installed providing file F.
>> * File F is moved to package B as of package A 1.0-3.
>> * User installs package B, which replaces the file in package A.
>> * User upgrades package A to version 1.0-2 (*not* 1.0-3). Or, rather,
>>tries, because this will fail with an error due to the file conflict.

> No, that is fine. "Replaces: A (<< 1.0-3)" is sufficient here that the
> file is not unpacked from A 1.0-2.

Oh!  Of course.  Okay.

In that case, I don't actually know why we usually use Conflicts with
Replaces.  Maybe we don't really need to?

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Simon Richter]

Hi,

On 6/28/23 02:31, Russ Allbery wrote:


Normally Conflicts is always added with Replaces because otherwise you can
have the following situation:



* Package A version 1.0-1 is installed providing file F.
* File F is moved to package B as of package A 1.0-3.
* User installs package B, which replaces the file in package A.
* User upgrades package A to version 1.0-2 (*not* 1.0-3). Or, rather,
   tries, because this will fail with an error due to the file conflict.


No, that is fine. "Replaces: A (<< 1.0-3)" is sufficient here that the 
file is not unpacked from A 1.0-2.


(Reading database ... 3 files and directories currently installed.)
Preparing to unpack A-2.deb ...
Unpacking a (2) over (1) ...
Replaced by files in installed package b (1) ...
Setting up a (2) ...


Debian packaging
is already hard enough; we should automate as much as we possibly can.


Nothing against automating it, but this doesn't need to be done 
centrally, so I'd expect the orphan-sysvinit-scripts maintainers to set 
up a cronjob or systemd timer[1] that checks for removed init scripts if 
they felt that was necessary.


I don't see the need for mandating that automation for what is 
essentially a very small transition that is low-impact and 
well-supported by our existing tools, and doing so does not decrease the 
workload on individual maintainers because they still need to look up 
the appropriate action to take on removal of an init script even if that 
action is "do nothing."



A
comprehensive checklist of everything you're supposed to think about when
packaging a Debian package doesn't exist (Policy is certainly shy of
that), and if it did, would form a hardcover book large enough to use as a
boat anchor.


We have an upgrade checklist for moving to a newer Standards-Version, 
and we have the Developers' Handbook.


   Simon

[1] probably not

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Simon Richter  writes:

> The only thing we actually need is a versioned Replaces that allows
> orphan-sysvinit-scripts to take over ownership of the conffile.

> Conflicts is unneeded here, and the daemon package does not need to
> declare any relationship. They can use

> Depends: systemd-sysv | orphan-sysvinit-scripts

> but really that doesn't do much because orphan-sysvinit-scripts is going
> to be pulled in anyway, so I'd rather avoid the clutter.

Normally Conflicts is always added with Replaces because otherwise you can
have the following situation:

* Package A version 1.0-1 is installed providing file F.
* File F is moved to package B as of package A 1.0-3.
* User installs package B, which replaces the file in package A.
* User upgrades package A to version 1.0-2 (*not* 1.0-3). Or, rather,
  tries, because this will fail with an error due to the file conflict.

Unless I misunderstand how this works, I believe the Conflicts is
necessary to force dependency resolution to realize version 1.0-3 of
package A is needed when the F-providing version of package B is
installed.

My personal opinion is that if we're going to do this, we should do it
correctly.

>> Less prone to errors than a manual process might be to watch
>> automatically where legacy startup scripts disappear anyway; it's not
>> that complicated to do. People tend to forget things.

> No, we shouldn't set the bar this low. We're talking about DDs here,
> they should all have passed P and T tests.

Passing P and T doesn't help you not forget things.  Debian packaging
is already hard enough; we should automate as much as we possibly can.  A
comprehensive checklist of everything you're supposed to think about when
packaging a Debian package doesn't exist (Policy is certainly shy of
that), and if it did, would form a hardcover book large enough to use as a
boat anchor.  We should be tryingn to whittle that down over time with
automation and tools, not rely on people's memory and constantly
re-reading Policy.

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Simon Richter]

Hi Ansgar,

On 6/27/23 01:45, Ansgar wrote:

[systemd service wrapper provided by init]


I think sysvinit maintainers looked at such ideas in the past, but
weren't capable to get it to work. That might be a blocker for such
approaches. There was also a GSoC project in 2012 and some other work.


I'm fairly sure they are capable of doing that, but there is no point in 
doing so. There is already an implementation of the systemd service API, 
and it's called systemd.


This interface is not useful as an abstraction layer because it is 
tightly coupled with the existing implementation, to the extent that 
writing a conforming implementation requires largely making the same 
design choices. This work would be redundant, and if we had volunteers 
for that, they could be working on systemd instead, that would make more 
sense.


The init.d interface, too, is directly tied to an implementation. It can 
be implemented by a compatibility layer, which doesn't work the other 
way around, but you are losing most of the benefits of the systemd 
interface, so the desire to drop this layer is understandable, and has 
been predicted (and those predictions rejected as paranoid) back when 
systemd was introduced.


There is a fundamental disagreement between the systemd and sysvinit 
camps, and it is not at an implementation level, but at conceptual and 
community management levels. We cannot implement our way out of this 
disagreement.


Fortunately, we don't have to. Proposed policy:

 - Each package that contains something that is useful as a systemd 
unit should also ship the unit definitions.


There may be exceptions, so I'm writing "should", not "must".

 - Each package that contains something that can also be a daemon can, 
but does not need to ship an init script.


The maintainer of a service package should know best how to invoke the 
program, so there is a technical benefit in having them maintain the 
init script, but that only extends as far as their motivation to do a 
good job.


 - If an init script exists, a service unit with the same name needs to 
exist as well so systemd before the removal of the compatibility layer 
ignores the init script.


We will have a full release cycle with an old systemd and backports.

 - Unit files and init scripts do not need to provide the same 
functionality.


The sysvinit users have different requirements than the systemd users.

There are services that have their own implementation of a graceful 
restart that cannot be used from systemd, and there are services you 
would not find on a machine running sysvinit anyway, so any effort 
writing an init script for those would be wasted.


 - Maintainers dropping init scripts are asked to file a bug on 
orphan-sysvinit-scripts so they can be added there.


Orphaning an init script is just like orphaning a package: you're 
handing it over to a team that is not necessarily that invested in your 
software as you are, but if you feel you're not doing a good job at it, 
stepping down gracefully is expected of a maintainer.


Extending this notion to a part of a package is not a big stretch.


Neither orphan-sysvinit-scripts now! nor the packages it ships legacy
startup scripts for seem to have Replaces or Conflicts though?


The only thing we actually need is a versioned Replaces that allows 
orphan-sysvinit-scripts to take over ownership of the conffile.


Conflicts is unneeded here, and the daemon package does not need to 
declare any relationship. They can use


Depends: systemd-sysv | orphan-sysvinit-scripts

but really that doesn't do much because orphan-sysvinit-scripts is going 
to be pulled in anyway, so I'd rather avoid the clutter.



Less prone to errors than a manual process might be to watch
automatically where legacy startup scripts disappear anyway; it's not
that complicated to do. People tend to forget things.


No, we shouldn't set the bar this low. We're talking about DDs here, 
they should all have passed P and T tests.


   Simon

Re: Policy consensus on transition when removing initscripts.

[Andreas Henriksson]

Hello all,

On Sun, Jun 25, 2023 at 03:15:24PM +0100, Mark Hindley wrote:
> Hello friends and colleagues,
[...]
> To avoid breakage of existing systems and facilitate ongoing support for
> non-systemd inits, I would like to establish a consensus for
> 
>  - stating that initscripts remain useful.

If you add '... for non-default init systems' I might agree, but for
the default init system I consider init scripts harmful.

> 
>  - requiring a coordinated transition of any initscript a maintainer wishes to
>drop to the orphan-sysvinit-scripts package and providing the relevant
>copyright information.

Consider my vote AGAINST anything that puts the burden on package
maintainers to do any extra work at this point.

"Policy is not a stick to beat people with", etc.

If you want me to take suggestions like coordination seriously then
please consider adressing https://bugs.debian.org/934463 soon or admit
that sysvinit maintenance lacks the resources to do coordinated
transitions. Dropping things and letting people pick them up if they
think they are still useful seems to be the only practical way forward.

Regards,
Andreas Henriksson

Re: Policy consensus on transition when removing initscripts.

[Thomas Goirand]

On 6/25/23 16:15, Mark Hindley wrote:

Hello friends and colleagues,

Debian Policy no longer requires that packages which provide a systemd .service
file also provide an initscript. This permits maintainers who so wish to remove
initscripts from their packages. However, initscripts remain used and useful[1],
and uncoordinated removal can have significant effects on users' systems[2].

With the encouragement of the Technical Committee[3] and despite some
unavoidable deficiencies resulting consequent on keeping initscripts without
their intended package[4], orphan-sysvinit-scripts has collected and maintained
some dropped initscripts. However, the process surrounding this has not been
defined in Policy. Indeed, #975075[5] contains a number of suggestions that have
not yet been followed through.

The most recent proposal[6] for updating the Policy with a requirement to use
tmpfiles.d(5) states

  "Init systems other than ``systemd`` should allow providing the same
  functionality as appropriate for each system, for example managing the
  directories from the init script shipped by the package."

This creates an inconsistency whereby non-systemd inits are required to provide
functionality in their initscript, but that initscript is not required to be
present.
   
To avoid breakage of existing systems and facilitate ongoing support for

non-systemd inits, I would like to establish a consensus for

  - stating that initscripts remain useful.

  - requiring a coordinated transition of any initscript a maintainer wishes to
drop to the orphan-sysvinit-scripts package and providing the relevant
copyright information.

Best wishes

Mark


There's some ongoing (or already working?) work for OpenRC to support 
systemd .service files. Contributing to this would be a much better use 
of your time, IMO.


Cheers,

Thomas Goirand (zigo)

Re: Policy consensus on transition when removing initscripts.

[Simon McVittie]

On Mon, 26 Jun 2023 at 20:04:11 -0400, nick black wrote:
> Simon McVittie left as an exercise for the reader:
> > started as root and dropped privileges to some other uid, that permanently
> > restricts its ability to read information out of its own /proc, which is
> > not always desirable. If the daemon starts up unprivileged, then it can
> 
> i assume by "its own /proc" you mean /proc/getpid()?

Yes, the same directory that the /proc/self magic symlink points to.

> furthermore, this is only true when procfs is mounted with a
> nonzero hidepid, right?

No. Mounting /proc with hidepid does make a subset of the /proc/[0-9]*
directories appear to be nonexistent (which breaks surprisingly many
things), but I'm talking about the permissions and ownership of the
pseudo-files inside those directories, not about the presence/absence
of the directories themselves.

If you look at the /proc/$pid of `dbus-daemon --system` (which is a
convenient example of a daemon present on most Debian systems that
currently starts as uid 0 and then drops privileges), you'll see that
for example /proc/$pid/fd/ has 0500 root:root ownership and permissions,
/proc/$pid/auxv is 0400 root:root, and so on. Some (mostly older)
pseudo-files such as /proc/*/stat are world-readable by default, and some
such as /proc/*/net are owned by its post-privilege-dropping identity
messagebus:messagebus, but many remain restricted to root access even
after the process has dropped privileges.

Compare that with `dbus-daemon --session`, which starts as its final uid
(your ordinary user account) and does not drop privileges; everything in
its /proc/$pid is owned by that uid.

> how this is different from any other resource one might need
> consider acquiring prior to dropping privileges. if i want to
> open a privileged port, i'd better do that before i change my
> user (or otherwise yield CAP_NET_BIND_SERVICE).

Not all of the information in /proc/$pid is something that you can
usefully load ahead of time: a lot of it is a dynamic representation
of the state of the process *now*. In particular, some programs rely on
the ability to access /proc/$pid/fd/* to reopen or otherwise manipulate
the program's current set of file descriptors, which needs to act on
the current file descriptor set, and not the file descriptors as they
existed at program start time.

As a concrete example, one of the things on my lengthy to-do list is
reviewing a proposed feature for dbus-daemon to allow use of process file
descriptors (pidfds) in future kernels, which will eventually let D-Bus
clients like polkitd use pidfds to refer to processes by using a pidfd as
a "process handle" without the time-of-check/time-of-use race conditions
inherent in using numeric process IDs.

It's looking as though that will require access to /proc/$pid/fd/*
in order to "pin" process file descriptors, and that in turn will
require dbus.service to start `dbus-daemon --system` with its final uid
(messagebus) instead of starting as root and then dropping privileges
to messagebus.

smcv

Re: Policy consensus on transition when removing initscripts.

[Luca Boccassi]

On Tue, 27 Jun 2023 at 01:04, nick black  wrote:
>
> Simon McVittie left as an exercise for the reader:
> > started as root and dropped privileges to some other uid, that permanently
> > restricts its ability to read information out of its own /proc, which is
> > not always desirable. If the daemon starts up unprivileged, then it can
>
> i assume by "its own /proc" you mean /proc/getpid()? i don't see
> how this is different from any other resource one might need
> consider acquiring prior to dropping privileges. if i want to
> open a privileged port, i'd better do that before i change my
> user (or otherwise yield CAP_NET_BIND_SERVICE).
>
> furthermore, this is only true when procfs is mounted with a
> nonzero hidepid, right? all my /proc/PID directories are 0755,
> with contents likewise generally world-readable. hidepid=off is
> the default according to
> https://www.kernel.org/doc/html/latest/filesystems/proc.html.

Resources are dynamic, and come and go - for example, file
descriptors. The 'run as root and drop privileges' pattern might have
been fine in the past century, but in 2023 we have much, much better
patterns to secure running services. And using hidepid (ie,
ProtectProc) and other sandboxing on services is one such pattern.

Kind regards,
Luca Boccassi

Re: Policy consensus on transition when removing initscripts.

[Luca Boccassi]

On Tue, 27 Jun 2023 at 04:10, Paul Wise  wrote:
>
> On Mon, 2023-06-26 at 20:04 -0400, nick black wrote:
>
> > furthermore, this is only true when procfs is mounted with a
> > nonzero hidepid, right?
>
> I note that systemd does not support non-zero hidepid, so
> procfs hidepid will always be off on systemd based systems:
>
> https://github.com/systemd/systemd/issues/12955
>
> At least until Linux offers a way for systemd to have access
> to /proc but other programs to not have access to it.

That has been implemented a long time ago, services can set
ProtectProc= so that processes run with hidepid:

https://freedesktop.org/software/systemd/man/systemd.exec.html#ProtectProc=

Kind regards,
Luca Boccassi

Re: Policy consensus on transition when removing initscripts.

[Paul Wise]

On Mon, 2023-06-26 at 20:04 -0400, nick black wrote:

> furthermore, this is only true when procfs is mounted with a
> nonzero hidepid, right?

I note that systemd does not support non-zero hidepid, so
procfs hidepid will always be off on systemd based systems:

https://github.com/systemd/systemd/issues/12955

At least until Linux offers a way for systemd to have access
to /proc but other programs to not have access to it.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: Policy consensus on transition when removing initscripts.

[nick black]

Simon McVittie left as an exercise for the reader:
> started as root and dropped privileges to some other uid, that permanently
> restricts its ability to read information out of its own /proc, which is
> not always desirable. If the daemon starts up unprivileged, then it can

i assume by "its own /proc" you mean /proc/getpid()? i don't see
how this is different from any other resource one might need
consider acquiring prior to dropping privileges. if i want to
open a privileged port, i'd better do that before i change my
user (or otherwise yield CAP_NET_BIND_SERVICE).

furthermore, this is only true when procfs is mounted with a
nonzero hidepid, right? all my /proc/PID directories are 0755,
with contents likewise generally world-readable. hidepid=off is
the default according to
https://www.kernel.org/doc/html/latest/filesystems/proc.html.

-- 
nick black -=- https://www.nick-black.com
to make an apple pie from scratch,
you need first invent a universe.


signature.asc
Description: PGP signature

Re: Policy consensus on transition when removing initscripts.

[Scott Kitterman]

On Monday, June 26, 2023 2:02:24 PM EDT Bastian Blank wrote:
> On Mon, Jun 26, 2023 at 01:22:38PM -0400, Scott Kitterman wrote:
> > > Less prone to errors than a manual process might be to watch
> > > automatically where legacy startup scripts disappear anyway; it's not
> > > that complicated to do. People tend to forget things.
> > 
> > That sounds reasonable.  It might be sufficient for the policy discussion
> > to say that maintainers who drop sysv init scripts should file a bug
> > against orphan- sysvinit-scripts with the final version of the provided
> > init attached and which lists the lowest version that won't include it
> > (so that orphan-sysvinit- scripts can Replaces << that version).
> 
> Automatically is clearly not the same as "file a bug".  It is not a
> useful way to ask hundreds of people to do things if you can do it on
> your own.

Are hundreds of people dropping init scripts?  I'm curious how you measure 
that?

Scott K

signature.asc
Description: This is a digitally signed message part.

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Bastian Blank  writes:
> On Mon, Jun 26, 2023 at 01:22:38PM -0400, Scott Kitterman wrote:

>>> Less prone to errors than a manual process might be to watch
>>> automatically where legacy startup scripts disappear anyway; it's not
>>> that complicated to do. People tend to forget things.

>> That sounds reasonable.  It might be sufficient for the policy
>> discussion to say that maintainers who drop sysv init scripts should
>> file a bug against orphan- sysvinit-scripts with the final version of
>> the provided init attached and which lists the lowest version that
>> won't include it (so that orphan-sysvinit- scripts can Replaces << that
>> version).

> Automatically is clearly not the same as "file a bug".  It is not a
> useful way to ask hundreds of people to do things if you can do it on
> your own.

I think the open question is whether we're happy to just break init
scripts in unstable, since that migration path means there will always be
a window where a fully-upgraded unstable system has no init script for a
package that previously had one and in the future will have one again.

This feels like exactly the kind of problem that normally Debian goes to
some lengths to avoid creating, but it sounds like several commenters here
don't think the effort is worth it?

If we don't want to break init scripts in unstable, then the package
maintainer should wait for the upload of orphan-sysvinit-scripts that
contains the script before uploading the version that drops it.

I thought for a bit that some dependency other than Replaces/Conflicts
would be appropriate, but the more I think about it, the more I'm not sure
there is.  (Well, Enhances in orphan-sysvinit-scripts, but I'm not sure
that changes apt behavior in any way.)  Presumably the sysvinit init
system package should depend on orphan-sysvinit-scripts, so the
Replaces/Conflicts should then force the right upgrade to happen.

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Bastian Blank]

On Mon, Jun 26, 2023 at 01:22:38PM -0400, Scott Kitterman wrote:
> > Less prone to errors than a manual process might be to watch
> > automatically where legacy startup scripts disappear anyway; it's not
> > that complicated to do. People tend to forget things.
> 
> That sounds reasonable.  It might be sufficient for the policy discussion to 
> say 
> that maintainers who drop sysv init scripts should file a bug against orphan-
> sysvinit-scripts with the final version of the provided init attached and 
> which 
> lists the lowest version that won't include it (so that orphan-sysvinit-
> scripts can Replaces << that version).

Automatically is clearly not the same as "file a bug".  It is not a
useful way to ask hundreds of people to do things if you can do it on
your own.

Bastian

-- 
Military secrets are the most fleeting of all.
-- Spock, "The Enterprise Incident", stardate 5027.4

Re: Policy consensus on transition when removing initscripts.

[Scott Kitterman]

On Monday, June 26, 2023 12:45:11 PM EDT Ansgar wrote:
> On Sun, 2023-06-25 at 11:15 -0700, Russ Allbery wrote:
> > Bastian Blank  writes:
> > > Sorry no.  Please add a conversion layer that adopts service and
> > > maybe other systemd units to initrc if you care about it.  This is
> > > what systemd does to adopt existing init scripts, so you can do the
> > > opposite.
> > 
> > I don't think it's useful to tell people who are working on sysvinit
> > support how best to do that work.  We decided to not require support
> > in packages and put the maintenance burden on the sysvinit
> > maintainers.  It feels rude to me to do that and then try to second-
> > guess how they choose to do that.
> 
> I think sysvinit maintainers looked at such ideas in the past, but
> weren't capable to get it to work. That might be a blocker for such
> approaches. There was also a GSoC project in 2012 and some other work.
> 
> > [...] we should respect it.
> 
> But not necessarily much more than the community the thread starter
> works with shows towards others.
> 
> > > And it can detect easily if no init script exists for a given unit,
> > > so no coordination necessary.
> > 
> > Replaces and Conflicts are required at the very least so that
> > upgrades work properly, and I don't see any reason why we shouldn't
> > provide instructions to package maintainers to do that smoothly,
> > rather than having the init script disappear and then reappear and
> > break users who are using unstable.  It's not that difficult to slow
> > down a little bit and follow a process.
> 
> Neither orphan-sysvinit-scripts now! nor the packages it ships legacy
> startup scripts for seem to have Replaces or Conflicts though?
> 
> I also don't think we should put anything here on the critical path:
> that lead to problems with, for example, systemd-shim which was
> promised to get maintained and timely updated...
> 
> Less prone to errors than a manual process might be to watch
> automatically where legacy startup scripts disappear anyway; it's not
> that complicated to do. People tend to forget things.

That sounds reasonable.  It might be sufficient for the policy discussion to 
say 
that maintainers who drop sysv init scripts should file a bug against orphan-
sysvinit-scripts with the final version of the provided init attached and which 
lists the lowest version that won't include it (so that orphan-sysvinit-
scripts can Replaces << that version).

Scott K

signature.asc
Description: This is a digitally signed message part.

Re: Policy consensus on transition when removing initscripts.

[Ansgar]

On Sun, 2023-06-25 at 11:15 -0700, Russ Allbery wrote:
> Bastian Blank  writes:
> > Sorry no.  Please add a conversion layer that adopts service and
> > maybe other systemd units to initrc if you care about it.  This is
> > what systemd does to adopt existing init scripts, so you can do the
> > opposite.
> 
> I don't think it's useful to tell people who are working on sysvinit
> support how best to do that work.  We decided to not require support
> in packages and put the maintenance burden on the sysvinit
> maintainers.  It feels rude to me to do that and then try to second-
> guess how they choose to do that.

I think sysvinit maintainers looked at such ideas in the past, but
weren't capable to get it to work. That might be a blocker for such
approaches. There was also a GSoC project in 2012 and some other work.

> [...] we should respect it.

But not necessarily much more than the community the thread starter
works with shows towards others.

> > And it can detect easily if no init script exists for a given unit,
> > so no coordination necessary.
> 
> Replaces and Conflicts are required at the very least so that
> upgrades work properly, and I don't see any reason why we shouldn't
> provide instructions to package maintainers to do that smoothly,
> rather than having the init script disappear and then reappear and
> break users who are using unstable.  It's not that difficult to slow
> down a little bit and follow a process.

Neither orphan-sysvinit-scripts now! nor the packages it ships legacy
startup scripts for seem to have Replaces or Conflicts though?

I also don't think we should put anything here on the critical path:
that lead to problems with, for example, systemd-shim which was
promised to get maintained and timely updated...

Less prone to errors than a manual process might be to watch
automatically where legacy startup scripts disappear anyway; it's not
that complicated to do. People tend to forget things.

Ansgar

Re: Policy consensus on transition when removing initscripts.

[Simon McVittie]

On Mon, 26 Jun 2023 at 15:03:51 +0900, Simon Richter wrote:
> On 6/25/23 23:15, Mark Hindley wrote:
> > The most recent proposal[6] for updating the Policy with a requirement to 
> > use
> > tmpfiles.d(5) states
> 
> >   "Init systems other than ``systemd`` should allow providing the same
> >   functionality as appropriate for each system, for example managing the
> >   directories from the init script shipped by the package."
> 
> The way I see it, we are getting a split between "daemons" and "services",
> and it simply does not make sense to start a "service" directly from an init
> script, because it requires the service orchestrator as its runtime
> environment.

That's certainly not how I understand the difference between a service
and a daemon. The LSB specification refers to the functionality provided
by a LSB init script as a service: this is not a new term introduced
by systemd, and daemons that need some one-time setup before the actual
daemon can be started are not a new concept either.

It's true that a lot of system services are a single daemon (long-running
background process) or a wrapper around a daemon, maybe with some extra
setup/teardown: for example /etc/init.d/dbus has always needed to ensure
that the machine ID exists before it can start dbus-daemon, and various
other init scripts need to create a directory in /run or whatever, owned
by the user ID with which they are going to start the actual daemon.
In systemd jargon, the init script is a "forking" service (it forks in
order to run the daemon process in the background, or instructs the
daemon process to use the double-fork pattern).

Other system services have no daemons at all: /etc/init.d/sudo and
/etc/init.d/apparmor do some one-time actions during boot, and then exit,
leaving no processes running (assuming they're working correctly). In
systemd jargon they're "oneshot" services.

A somewhat rarer category of system services involves a top-level init
script or systemd unit that results in multiple daemons being started,
for example /etc/init.d/postfix.

> "If it is present, it also needs to work." sounds like a reasonable
> statement though.

I'm fairly sure that's the intention here.

The typical case where a daemon has a functional dependency on
tmpfiles.d(5) is that the daemon needs some state directories, either on
disk (typically /var/lib) or in RAM (typically /run), to be set up as
root and chown'd to the appropriate user ID before it can do its work.
For example /usr/sbin/foo might run as user ID _foo and require /run/foo
and/or /var/lib/foo to be 0750 _foo:_foo.

Historically a lot of daemons have required being started as root and
then dropped privileges internally, allowing the daemon to set up those
directories while it is still root. However, not all daemon maintainers
will want their daemon implementation to be responsible for doing that:
when touching a non-root-owned directory as root, it's necessary to be
careful to avoid time-of-check/time-of-use attacks (perhaps involving
symlinks or hard links) that could allow a local attacker to trick the
daemon into changing the ownership or permissions of the wrong file.

Not wanting to drop privileges internally at all is also a design choice
for the daemon, and a valid one IMO: as well as the security impact of
any bugs that might exist in the privilege-dropping code, if a process
started as root and dropped privileges to some other uid, that permanently
restricts its ability to read information out of its own /proc, which is
not always desirable. If the daemon starts up unprivileged, then it can
read its own /proc in the usual way, but obviously it can't create a new
subdirectory of /run or /var/lib itself, so someone else needs to do that.

If a daemon needs its state directories set up in advance like that,
and if someone provides an init script, I hope it's uncontroversial
to say that the init script needs to take responsibility for creating
those state directories, either directly or by delegating it to a
dependency. That's equally true regardless of whether the init script
is provided by the same package as the daemon, or by some other package
like orphan-sysvinit-scripts, or by a local sysadmin.

One way to implement pre-creation of state directories is to use
the declarative tmpfiles.d(5) files. On systemd systems, the package
providing the service can rely on systemd to process those files. On
non-systemd-booted systems, one option is to run on the systemd-tmpfiles
executable (from the systemd-standalone-tmpfiles or systemd package)
during boot and during postinst to achieve the same thing. Another
option is to use a reimplementation of the tmpfiles.d(5) interface; a
third option is to open-code the directory creation in the init script
with some appropriate `mkdir -p`, `chmod` and `chown`, or `install -d`.

Looking at reimplementations of tmpfiles.d(5), OpenRC's opentmpfiles
exists as an upstream project, but was removed from Debian, and my
understanding is 

Re: Policy consensus on transition when removing initscripts.

[Simon Richter]

Hi,

On 6/25/23 23:15, Mark Hindley wrote:


The most recent proposal[6] for updating the Policy with a requirement to use
tmpfiles.d(5) states



  "Init systems other than ``systemd`` should allow providing the same
  functionality as appropriate for each system, for example managing the
  directories from the init script shipped by the package."


The way I see it, we are getting a split between "daemons" and 
"services", and it simply does not make sense to start a "service" 
directly from an init script, because it requires the service 
orchestrator as its runtime environment.


Init scripts are useful for starting daemons. If you need a service 
started, you need an appropriate service wrapper, and that is outside 
the scope of an init system. The lack of these interfaces is not a 
deficiency, but a deliberate design decision.


My expectation is that some daemons will gain some systemd integration, 
but keep the non-systemd code because it is still useful during 
debugging and in container environments, and init scripts would use the 
non-systemd invocation interface.



This creates an inconsistency whereby non-systemd inits are required to provide
functionality in their initscript, but that initscript is not required to be
present.


"If it is present, it also needs to work." sounds like a reasonable 
statement though.


   Simon

Re: Policy consensus on transition when removing initscripts.

[RL]

Mark Hindley  writes:

> Debian Policy no longer requires that packages which provide a systemd
> .service file also provide an initscript. This permits maintainers who
> so wish to remove initscripts from their packages. However,
> initscripts remain used and useful[1], and uncoordinated removal can
> have significant effects on users' systems[2].
>
> With the encouragement of the Technical Committee[3] and despite some
> unavoidable deficiencies resulting consequent on keeping initscripts without
> their intended package[4], orphan-sysvinit-scripts has collected and 
> maintained
> some dropped initscripts. However, the process surrounding this has not been
> defined in Policy. Indeed, #975075[5] contains a number of suggestions that 
> have
> not yet been followed through.


This seems like something that the trixie release notes should mention -
although quite what it should say is perhaps not yet clear. Can you file
a bug against release notes suggestiong what users should do. Presumably
people using non-systemd should install orphan-susvinit-scripts while
people using the default systemd dont need to do anything?

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Bastian Blank  writes:
> On Sun, Jun 25, 2023 at 03:15:24PM +0100, Mark Hindley wrote:

>> To avoid breakage of existing systems and facilitate ongoing support
>> for non-systemd inits, I would like to establish a consensus for

>>  - stating that initscripts remain useful.

>>  - requiring a coordinated transition of any initscript a maintainer
>>wishes to drop to the orphan-sysvinit-scripts package and providing
>>the relevant copyright information.

> Sorry no.  Please add a conversion layer that adopts service and maybe
> other systemd units to initrc if you care about it.  This is what
> systemd does to adopt existing init scripts, so you can do the opposite.

I don't think it's useful to tell people who are working on sysvinit
support how best to do that work.  We decided to not require support in
packages and put the maintenance burden on the sysvinit maintainers.  It
feels rude to me to do that and then try to second-guess how they choose
to do that.

The orphan-sysvinit-scripts approach fully abides by the result of GR
2019-002.  They're doing what the project asked for even though it's a lot
of work for them, and they're volunteering to let maintainers transfer the
support they don't want to maintain to them.  That's exactly what we asked
for in the GR, and we should respect it.

> And it can detect easily if no init script exists for a given unit, so
> no coordination necessary.

Replaces and Conflicts are required at the very least so that upgrades
work properly, and I don't see any reason why we shouldn't provide
instructions to package maintainers to do that smoothly, rather than
having the init script disappear and then reappear and break users who are
using unstable.  It's not that difficult to slow down a little bit and
follow a process.

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Russ Allbery]

Mark Hindley  writes:

> The most recent proposal[6] for updating the Policy with a requirement
> to use tmpfiles.d(5) states

>  "Init systems other than ``systemd`` should allow providing the same
>  functionality as appropriate for each system, for example managing the
>  directories from the init script shipped by the package."

For those not following the Policy bug, while the wording you mention is
in a patch that in part is addressing use of tmpfiles.d, that paragraph is
not about the tmpfiles.d mechanism.  It's about systemd-specific unit file
configuration such as RuntimeDirectory.

tmpfiles.d itself is easier to handle since systemd-tmpfiles has already
been packaged as a standalone utility for systems using other init
systems.  Other init systems should arrange to invoke it (or some other
implementation that consumes the same configuration files) on boot so that
all Debian packages can use the same configuration files for temporary
file creation.

> This creates an inconsistency whereby non-systemd inits are required to
> provide functionality in their initscript, but that initscript is not
> required to be present.

Note that this wording hasn't been accepted yet, and I'd want to address
the inconsistency you've identified before merging it, probably by
clarifying that this support should be provided *if* other init systems
are supported.

> To avoid breakage of existing systems and facilitate ongoing support for
> non-systemd inits, I would like to establish a consensus for

>  - stating that initscripts remain useful.

>  - requiring a coordinated transition of any initscript a maintainer
>wishes to drop to the orphan-sysvinit-scripts package and providing the
>relevant copyright information.

The first point may be outside the scope of Policy, although Policy can
point out as part of the second bullet that there is ongoing work to
support init scripts in Debian.

For the second point, is there an existing policy for how you want init
scripts to be contributed to orphan-sysvinit-scripts that is already
written down somewhere?  (If not, that would be the place to start.)

I think the part of the transition that cannot easily be automated is
ensuring that an appropriate dependency and Replaces relationship is in
place.  This does sound like something that can be usefully documented in
Policy, although I think (at least at first glance; I've not thought about
it a lot) we'd prefer to point to documentation maintained by the
orphan-sysvinit-scripts maintainers rather than putting the entire
procedure directly in Policy.  A bunch of it is going to be about
coordination and communication rather than technical specifics, and I
think it's better for you to be able to change those details as you see
fit without having to update Policy.

-- 
Russ Allbery (r...@debian.org)  

Re: Policy consensus on transition when removing initscripts.

[Luca Boccassi]

On Sun, 25 Jun 2023 at 15:21, Mark Hindley  wrote:
>  "Init systems other than ``systemd`` should allow providing the same
>  functionality as appropriate for each system, for example managing the
>  directories from the init script shipped by the package."
>
> This creates an inconsistency whereby non-systemd inits are required to 
> provide
> functionality in their initscript, but that initscript is not required to be
> present.

This wasn't intended as a hard rule, I used "should" and "for example"
to try and make that clear. That text has now been made more generic
after a suggestion:

-directories from the init script shipped by the package.
+directories from the appropriate service startup configuration.

which should hopefully clarify that nothing is required to be
implemented from scripts or packages themselves, but it is merely a
hint to developers and maintainers of alternatives that such
functionality should be supported in whichever way is most appropriate
for their systems.

Does this change help to clarify the intentions?

Kind regard,
Luca Boccassi

Re: Policy consensus on transition when removing initscripts.

[Bastian Blank]

On Sun, Jun 25, 2023 at 03:15:24PM +0100, Mark Hindley wrote:
> To avoid breakage of existing systems and facilitate ongoing support for
> non-systemd inits, I would like to establish a consensus for
>  - stating that initscripts remain useful.
>  - requiring a coordinated transition of any initscript a maintainer wishes to
>drop to the orphan-sysvinit-scripts package and providing the relevant
>copyright information.

Sorry no.  Please add a conversion layer that adopts service and maybe
other systemd units to initrc if you care about it.  This is what
systemd does to adopt existing init scripts, so you can do the opposite.

And it can detect easily if no init script exists for a given unit, so
no coordination necessary.

Bastian

-- 
Intuition, however illogical, is recognized as a command prerogative.
-- Kirk, "Obsession", stardate 3620.7

Policy consensus on transition when removing initscripts.

[Mark Hindley]

Hello friends and colleagues,

Debian Policy no longer requires that packages which provide a systemd .service
file also provide an initscript. This permits maintainers who so wish to remove
initscripts from their packages. However, initscripts remain used and useful[1],
and uncoordinated removal can have significant effects on users' systems[2].

With the encouragement of the Technical Committee[3] and despite some
unavoidable deficiencies resulting consequent on keeping initscripts without
their intended package[4], orphan-sysvinit-scripts has collected and maintained
some dropped initscripts. However, the process surrounding this has not been
defined in Policy. Indeed, #975075[5] contains a number of suggestions that have
not yet been followed through.

The most recent proposal[6] for updating the Policy with a requirement to use
tmpfiles.d(5) states

 "Init systems other than ``systemd`` should allow providing the same
 functionality as appropriate for each system, for example managing the
 directories from the init script shipped by the package."

This creates an inconsistency whereby non-systemd inits are required to provide
functionality in their initscript, but that initscript is not required to be
present.
  
To avoid breakage of existing systems and facilitate ongoing support for
non-systemd inits, I would like to establish a consensus for

 - stating that initscripts remain useful.

 - requiring a coordinated transition of any initscript a maintainer wishes to
   drop to the orphan-sysvinit-scripts package and providing the relevant
   copyright information.

Best wishes

Mark 

[1] they continue to be used by sysvinit, openrc and runit which are all viable
 non-systemd inits at the time of writing.

[2] https://bugs.debian.org/1038903 is the most recent example of which I am 
aware.

[3]  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=975075#311

[4]  See Bugs/Limitations in 
https://salsa.debian.org/matthew/orphan-sysvinit-scripts/-/blob/master/README.org

[5]  https://bugs.debian.org/975075 

[6]  
https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=945269;filename=0001-Define-service-directories-and-tmpfiles.d-interfaces.patch;msg=160

-- 
Mark Hindley
GPG: 506C 15A4 2B0A F5A0 A854  23EE D28A 45BF 3287 D649


signature.asc
Description: PGP signature