Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Sam Hartman]

> "Sean" == Sean Whitton  writes:
Sean> In general, I agree with Santiago.  I find Policy's current
Sean> scope and working process effective, and not especially
Sean> ambiguous.  I think everyone should read it during the NM
Sean> process, if not sooner.

Sean> Russ has concretely considered these issues much more than me,
Sean> and Niels has worked extensively on an implementation, and I
Sean> have not.  These things count, so my sense that things are
Sean> broadly okay as they are only goes so far.

I agree with the above.
When Russ brought up concerns I didn't want to add stop energy to making
things better.
But I also am generally happy with policy today.  I find it useful and
refer to it often.  I find that with almost all of my NM applicants I
end up asking them to look at policy at various points in the process
and they do not run into trouble using the document.
The only reason they did not before is they were not in the practice of
doing so.

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Sean Whitton]

Hello,

On Mon 11 Sep 2023 at 01:04pm +02, Santiago Vila wrote:

> I wrote:
>> I believe that by choosing the wording appropriately, we can still keep this
>> desired property of Policy while still not mandating any given 
>> implementation.
>
> Sorry, that was terribly worded. I meant that we can avoid the hassle of
> documenting everything dh_installsystemd does and at the same time not
> *formally* mandating the use of dh_installsystemd.

In general, I agree with Santiago.  I find Policy's current scope and
working process effective, and not especially ambiguous.
I think everyone should read it during the NM process, if not sooner.

Russ has concretely considered these issues much more than me, and Niels
has worked extensively on an implementation, and I have not.
These things count, so my sense that things are broadly okay as they are
only goes so far.

I do think we should put weight on our actual experiences as Debian
contributors, and what's useful, and, as elsewhere in software, focus on
incremental improvements over rewrites.  For example, I think that the
knowledge of good documentation practices that's already implicit in how
we all actually write Policy text counts for more than abstract
discussions of best practice.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Bill Allombert]

Hello Russ,

In my view the main purpose of policy is to allow interoperability by defining
interfaces between packages.
 
We used to have a separate Packaging Manual, but it has been merged with
Policy a long time ago. The intent was to reduce duplication which lead to
outdated information.
However, this directly lead to the structural problem we have now.

While it would be OK for a Packaging Manual to say "just use debhelper"
that does not help the debhelper developers that need to know what it
is expected of debhelper.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Niels Thykier]

Russ Allbery:

Niels Thykier  writes:

Russ Allbery:



Ooo, this is a great framing of the problem.  I have a lot of thoughts
about this.  Unfortunately, I'm not sure if they're actionable thoughts
since my grand vision requires someone to sit down and do some serious
Policy restructuring and produce a radically different document.  But
maybe if I write them all down and enough people feel similarly, it
would be worth doing.  I would love to work on this, I am just afraid
that it is the sort of project that I would start and never finish and
thus never accomplish anything of use.



Indeed, it is definitely the thing I would personally prefer to
pre-align on before adventuring on something of this scale myself (were
I in your shoes), so I totally feel your concern about actionability.


I do to some extent want people to encourage me to work on this if they
think it would be awesome, since people being happy with it would be what
would make all the work worthwhile (although I will probably also need
help).  :)



Personally, I feel such a change would be for the better. +1 from here.


Interesting choice with the mixing. I was of the understanding that the
idea was one should try to avoid mixing documentation styles according
to Divio. Admittedly, I find it hard to fully stick to exactly one type
of style, so I would be happy if I had just overlooked the "advice on
mixing".


So, I have to admit that I have not read Divio in detail although I have
been pointed at it many times and have had the high-level concepts
explained to me.  I've read bits and pieces, but I'm not sure what it says
about mixing.



I think it was my reading of their Introduction section. It says:

"""
Each of them requires a distinct mode of writing. People working with 
software need these four different kinds of documentation at different 
times, in different circumstances - so software usually needs them all, 
and they should all be integrated into your documentation.


And documentation needs to be explicitly structured around them, and 
they all must be kept separate and distinct from each other.

"""

Which I interpreted as "don't mix" (particular the "must be kept 
separate and distinct").  But re-reading here it might just be that the 
policy should use "How to" as a baseline and clearly mark explainer 
paragraphs as "Here is the background" as something you can choose to 
dive into or easily skip if you are not interested (or have distinct 
subsections for how to vs. explainers, which I think you are proposing 
in the part that I snipped)


Anyway, as long as you are going into it with open eyes and it seems 
like you are!



[...]


As for the content: The "How-to" style would make sense for the target
audience.  I am less clear if all of these headlines makes up a "Policy"
as they sounds like something you could find in a "Debian Packaging 101"
guide.


I think that about a quarter of Policy currently is already how-to text.
For example, look at Policy 3.4:

https://www.debian.org/doc/debian-policy/ch-binary.html#s-descriptions

This is a disguised how-to document on how to write a good package
description.  There's a ton of stuff in Policy like this.  What
distinguishes it from a Debian Packaging 101 guide is that the how-to goes
into *way* more depth than a 101 guide: edge cases, exceptions, advanced
bits of packaging, etc.



Sounds good to me.  For me it was mostly a question to ensure that 
Policy would not end up like "yet another new maintainers guide". You 
seem to have a clear vision here for how the Policy would set itself 
apart and I think you make a compelling argument.



The main problem from the perspective of helping the typical packager, is
that this is mixed in with oodles of advice that is irrelevant to anyone
except debhelper maintainers.  To take another short example, look at the
section on symbolic links:

https://www.debian.org/doc/debian-policy/ch-files.html#symbolic-links

Half of that section is a specification for the packaging helper, which
will fix symbolic links to follow those rules.  The rest is sort of a
how-to (mixed in with some basic shell command advice).



Indeed, I am not a fan of the those sections in our Policy!


Side question: Does Policy add anything on the specification for
`symbols` and `shlibs` files as a reference document that is not covered
by dpkg's documentation for these formats? I assume the "symbols guide"
(on /when/ to use symbols and when not too) would go in the previous
section.


Well, one can read the two side-by-side and see, and I'm biased as the
person who wrote it, but I think it does.

Policy duplicates dpkg documentation quite a lot, so this is a question
worth asking, but I do think Policy has a good answer.  The value that
Policy adds over the dpkg documentation generally falls into three
categories:

[...]


Again, compelling argumentation for those. I think the rule of thumb is 
that for Policy to have a section on something, it would actually have 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Russ Allbery]

Niels Thykier  writes:
> Russ Allbery:

>> Ooo, this is a great framing of the problem.  I have a lot of thoughts
>> about this.  Unfortunately, I'm not sure if they're actionable thoughts
>> since my grand vision requires someone to sit down and do some serious
>> Policy restructuring and produce a radically different document.  But
>> maybe if I write them all down and enough people feel similarly, it
>> would be worth doing.  I would love to work on this, I am just afraid
>> that it is the sort of project that I would start and never finish and
>> thus never accomplish anything of use.

> Indeed, it is definitely the thing I would personally prefer to
> pre-align on before adventuring on something of this scale myself (were
> I in your shoes), so I totally feel your concern about actionability.

I do to some extent want people to encourage me to work on this if they
think it would be awesome, since people being happy with it would be what
would make all the work worthwhile (although I will probably also need
help).  :)

> Interesting choice with the mixing. I was of the understanding that the
> idea was one should try to avoid mixing documentation styles according
> to Divio. Admittedly, I find it hard to fully stick to exactly one type
> of style, so I would be happy if I had just overlooked the "advice on
> mixing".

So, I have to admit that I have not read Divio in detail although I have
been pointed at it many times and have had the high-level concepts
explained to me.  I've read bits and pieces, but I'm not sure what it says
about mixing.

But in terms of what makes an effective Policy document, I think a general
structure of an explanation section introducing a part of packaging
followed by how-to sections for the underlying tasks would work.

> As for the content: The "How-to" style would make sense for the target
> audience.  I am less clear if all of these headlines makes up a "Policy"
> as they sounds like something you could find in a "Debian Packaging 101"
> guide.

I think that about a quarter of Policy currently is already how-to text.
For example, look at Policy 3.4:

https://www.debian.org/doc/debian-policy/ch-binary.html#s-descriptions

This is a disguised how-to document on how to write a good package
description.  There's a ton of stuff in Policy like this.  What
distinguishes it from a Debian Packaging 101 guide is that the how-to goes
into *way* more depth than a 101 guide: edge cases, exceptions, advanced
bits of packaging, etc.

The main problem from the perspective of helping the typical packager, is
that this is mixed in with oodles of advice that is irrelevant to anyone
except debhelper maintainers.  To take another short example, look at the
section on symbolic links:

https://www.debian.org/doc/debian-policy/ch-files.html#symbolic-links

Half of that section is a specification for the packaging helper, which
will fix symbolic links to follow those rules.  The rest is sort of a
how-to (mixed in with some basic shell command advice).

> Side question: Does Policy add anything on the specification for
> `symbols` and `shlibs` files as a reference document that is not covered
> by dpkg's documentation for these formats? I assume the "symbols guide"
> (on /when/ to use symbols and when not too) would go in the previous
> section.

Well, one can read the two side-by-side and see, and I'm biased as the
person who wrote it, but I think it does.

Policy duplicates dpkg documentation quite a lot, so this is a question
worth asking, but I do think Policy has a good answer.  The value that
Policy adds over the dpkg documentation generally falls into three
categories:

1. Policy is much more opinionated about what features supported by dpkg
   should be used in packages and how they should be used.  There are
   sometimes things dpkg *can* do that we don't do.  A bunch of this is
   how-to, but I think some of this is reference.

2. Policy is sometimes a lot more verbose and offers worked examples, and
   sometimes benefits from the additional formatting tools available in
   Sphinx.  This could be imported into the dpkg documentation to some
   extent, of course, but as it stands I think there's value there.

3. dpkg documentation has to cover the complete operation of dpkg, whereas
   Policy, even in reference sections and packaging helper sections,
   should only need to cover the surface area that's visible to packaging
   at the lowest level.

I agree that this is a source of some duplication of effort, although in
my experience it's not the part that takes the most time.  (But I haven't
written triggers or multiarch documentation for Policy yet, so maybe it's
part of the problem.)

> What is it you would see go into this section [packaging helper
> specification] that is not the previous section?

Literally everything in Policy that you shouldn't think about because the
packaging helpers will cover it.  For example, the symlink
canonicalization stuff mentioned above.  How to run 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Niels Thykier]

Russ Allbery:

Niels Thykier  writes:


I had a look at the introduction section of Policy to check who the
target audience is.  I cannot find an explicit mention of the target
audience. I suspect there is a conflict here on the content because we
have different audiences in mind for the Policy and the Policy does not
seem to be explicit here.



[...]


Ooo, this is a great framing of the problem.  I have a lot of thoughts
about this.  Unfortunately, I'm not sure if they're actionable thoughts
since my grand vision requires someone to sit down and do some serious
Policy restructuring and produce a radically different document.  But
maybe if I write them all down and enough people feel similarly, it would
be worth doing.  I would love to work on this, I am just afraid that it is
the sort of project that I would start and never finish and thus never
accomplish anything of use.



Indeed, it is definitely the thing I would personally prefer to 
pre-align on before adventuring on something of this scale myself (were 
I in your shoes), so I totally feel your concern about actionability.



I think that ideally Policy targets all three of your audiences, but not
at the same time, and not with the same priority.

I have a lot of problems with the current structure of Policy, to the
extent that it sometimes interferes with my motivation for working on it,
and one of the big ones is that it doesn't follow any structured design
pattern for documentation, such as Divio [1].


I had Divio in an earlier draft of my email - should have kept it! :D


[...]

I think my ideal structure of Policy would have three major parts.

First, there would be Policy for Debian packagers.  This would focus on
the things someone packaging for Debian needs to know, and would be
organized roughly around task.  Example sections here would be:

* Choosing an archive area
* Files on the file system (FHS, ownership, permissions, etc.)
* Writing a debian/control file
* Writing a changelog
* Writing a copyright file
* Packaging a shared library
* Packaging a system service
* Using maintainer scripts

In other words, this section would consist primarily of Divio how-to
guides, with some intermixed Divio explanations.  (Tutorials I think are
outside the scope of Policy and are better handled elsewhere, such as in
debhelper documentation.)

[...]


Interesting choice with the mixing. I was of the understanding that the 
idea was one should try to avoid mixing documentation styles according 
to Divio. Admittedly, I find it hard to fully stick to exactly one type 
of style, so I would be happy if I had just overlooked the "advice on 
mixing".


As for the content: The "How-to" style would make sense for the target 
audience.  I am less clear if all of these headlines makes up a "Policy" 
as they sounds like something you could find in a "Debian Packaging 101" 
guide.  That is a "risk" (or maybe exactly what you are going for), 
which would be fine with me and I would support the idea.


Nevertheless, I feel it has quite some potential to make Policy more 
accessible to new contributors and that alone is worth supporting in my 
book. Though I am hardly the target audience nor "paying the upkeep 
cost", so supporting it is basically gratis for me.





[...]

Second section would be Policy the reference manual for interfaces in the
distribution.  Sections here would be:

* Complete list of control fields and their meanings.
* Specifications for the .dsc and .changes files (which packagers mostly
   never need to care about, but tool maintainers do).
* The detailed reference documentation on all the ways maintainer scripts
   can be called.
* Specification for the symbols and shlibs files.

This is all Divio reference stuff.  Whenever we have a comprehensive
explanation of the details of something, it goes here, and then we
liberally link to the reference section from the packaging-focused how-to
section for more details.



Makes sense to me as well. I would indeed also need documentation somewhere.

Side question: Does Policy add anything on the specification for 
`symbols` and `shlibs` files as a reference document that is not covered 
by dpkg's documentation for these formats? I assume the "symbols guide" 
(on /when/ to use symbols and when not too) would go in the previous 
section.



Finally, there is the reference manual for toolchain maintainers.  My
position on this is that it's best-effort documentation and should
probably be a non-normative appendix where toolchain maintainers are
relatively free to just make changes without going through a very formal
process as long as those changes reflect reality.  This is, by nature,
going to be incomplete and possibly out of date, but I do think the
project should have *somewhere* outside of any specific package where
people can write down the details of, oh, the other options to
Rules-Requires-Root that we aren't currently using.  But we would stick a
lot of disclaimers on it and make it clear 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Russ Allbery]

Niels Thykier  writes:

> I had a look at the introduction section of Policy to check who the
> target audience is.  I cannot find an explicit mention of the target
> audience. I suspect there is a conflict here on the content because we
> have different audiences in mind for the Policy and the Policy does not
> seem to be explicit here.

> If the target audience is package maintainers, then 100% of all debian
> contributors should read it. Then we need "simple and easy to
> understand" language and examples, because we want *everybody* to
> understand this.  I agree with Russ that long winded sections of "here
> is how you do this manually but really just use debhelper" is directly
> counter productive to this audience.

> If it is cross-package integration, are we targeting the ones
> integrating or the ones providing the integration points?  If it is
> both, then for more complex topics, it would make sense to split the
> topic into two. One for consumer and one for the provider of the
> integration, because for the consumer it will probably boil down to
> "install path at $X in your deb and run tool $Y" and then the consumer
> can skip the provider section as "not relevant".

> If the target audience is tool-chain maintainers, then only 5-10 people
> need to read the policy and the entire document + related process seems
> completely over-engineered.  But then, for the same reason, I suspect
> this is an unlikely target audience for the Policy.

Ooo, this is a great framing of the problem.  I have a lot of thoughts
about this.  Unfortunately, I'm not sure if they're actionable thoughts
since my grand vision requires someone to sit down and do some serious
Policy restructuring and produce a radically different document.  But
maybe if I write them all down and enough people feel similarly, it would
be worth doing.  I would love to work on this, I am just afraid that it is
the sort of project that I would start and never finish and thus never
accomplish anything of use.

I think that ideally Policy targets all three of your audiences, but not
at the same time, and not with the same priority.

I have a lot of problems with the current structure of Policy, to the
extent that it sometimes interferes with my motivation for working on it,
and one of the big ones is that it doesn't follow any structured design
pattern for documentation, such as Divio [1].  It may have at one point,
but iteratively over the years, and due to some decisions like trying to
retain section numbering, we've diverged from that.  Even if it were
trying to be only one type of documentation, it doesn't consistently stay
in that lane.

[1] https://documentation.divio.com/

I think my ideal structure of Policy would have three major parts.

First, there would be Policy for Debian packagers.  This would focus on
the things someone packaging for Debian needs to know, and would be
organized roughly around task.  Example sections here would be:

* Choosing an archive area
* Files on the file system (FHS, ownership, permissions, etc.)
* Writing a debian/control file
* Writing a changelog
* Writing a copyright file
* Packaging a shared library
* Packaging a system service
* Using maintainer scripts

In other words, this section would consist primarily of Divio how-to
guides, with some intermixed Divio explanations.  (Tutorials I think are
outside the scope of Policy and are better handled elsewhere, such as in
debhelper documentation.)

(This is very rough, this is not the right order, etc.  This is just to
provide a feel of the nature of this section.)

This section would assume that you're using a packaging helper and not
tell you how to do things that the packaging helper is going to do for
you.  There's an awkward line here between describing what to do and being
debhelper documentation, but the basic idea is that this would tell you
what you are aiming for, and then your packaging helper documentation will
tell you where to put configuration files to achieve that.

Second section would be Policy the reference manual for interfaces in the
distribution.  Sections here would be:

* Complete list of control fields and their meanings.
* Specifications for the .dsc and .changes files (which packagers mostly
  never need to care about, but tool maintainers do).
* The detailed reference documentation on all the ways maintainer scripts
  can be called.
* Specification for the symbols and shlibs files.

This is all Divio reference stuff.  Whenever we have a comprehensive
explanation of the details of something, it goes here, and then we
liberally link to the reference section from the packaging-focused how-to
section for more details.

Finally, there is the reference manual for toolchain maintainers.  My
position on this is that it's best-effort documentation and should
probably be a non-normative appendix where toolchain maintainers are
relatively free to just make changes without going through a very formal
process as long as those changes reflect 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Niels Thykier]

Russ Allbery:

Bill Allombert  writes:

[...]  Quite a lot of Policy is of the general format "here's a bunch of
complex things you need to do, wait, never mind, just use debhelper, go
see its documentation for the configuration files you should use instead"
and some of the rest of Policy is "here's a bunch of complex things you
need to do but if you follow these instructions instead of using debhelper
your package will be buggy."  This is not ideal!

I think there's a lot of appeal of having a thorough specification for
what debhelper is supposed to be doing.  It would enable future
competition around better packaging helpers (and I do think debhelper will
not be the last word).  But I also want to be realistic about whether
we're really capable of maintaining that specification.

--
Russ Allbery (r...@debian.org)  



Indeed. I have had the same challenge with the Policy.

I had a look at the introduction section of Policy to check who the 
target audience is.  I cannot find an explicit mention of the target 
audience. I suspect there is a conflict here on the content because we 
have different audiences in mind for the Policy and the Policy does not 
seem to be explicit here.


If the target audience is package maintainers, then 100% of all debian 
contributors should read it. Then we need "simple and easy to 
understand" language and examples, because we want *everybody* to 
understand this.  I agree with Russ that long winded sections of "here 
is how you do this manually but really just use debhelper" is directly 
counter productive to this audience.


If it is cross-package integration, are we targeting the ones 
integrating or the ones providing the integration points?  If it is 
both, then for more complex topics, it would make sense to split the 
topic into two. One for consumer and one for the provider of the 
integration, because for the consumer it will probably boil down to 
"install path at $X in your deb and run tool $Y" and then the consumer 
can skip the provider section as "not relevant".


  There is a separate here in whether the Policy editors have the 
capacity to maintain the documentation for the provider side of the 
integration points for all the integrations.  I think this is where Russ 
is arguing that we do not that capacity.  If so, it would also make 
sense to explicitly cut *out* that side of from policy in the Scope 
section. Maybe along the lines of "The Policy does not document the 
provider side of integration points directly. Instead, we provide links 
to the external documentation where available.".



If the target audience is tool-chain maintainers, then only 5-10 people 
need to read the policy and the entire document + related process seems 
completely over-engineered.  But then, for the same reason, I suspect 
this is an unlikely target audience for the Policy.



Either way, I think the root problem is that we are not all agreeing on 
scope and audience for the Policy. Until resolved, we can argue forever 
about whether X belongs in Policy.


Best regards,
Niels


PS: I also have other things to say about the concrete topic, but I will 
leave that for a later mail. I think the point above is so important 
that it should not be diluted with other topics.

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Russ Allbery]

Bill Allombert  writes:

> But we do: we support debhelper 13.11.4 and debhelper 13.11.6.  Even if
> we support a single implementation, we still need to know what is
> expected of it.

Policy already requires a single implementation of quite a lot of tools,
does not specify a version, and does not specify how they do what they do.
Bugs are handled as bugs against the packages that supply those tools, and
what they do, except at a very high level, is opaque to Policy.  Examples
include update-rc.d, invoke-rc.d, adduser, and ldconfig, as well as the
obvious example of dpkg (which doesn't really count).

Now, I realize that debhelper feels conceptually different than those, and
I think to some extent it *is* conceptually different.  Those are all
tools that do a single thing, whereas debhelper is an infrastructure that
has a lot of implications for how the package is defined and assembled.
It is a big step to go from requiring that all packages use a specific
tool to requiring that packages of a specific type use a particular
packaging framework.  That's why I wanted to have a discussion about that
first.

The alternative is to adopt deb-systemd-invoke and deb-systemd-helper as
tools similar to update-rc.d and invoke-rc.d and specify how packages
should call them.  This is indeed what I had started doing.  But when I
started writing the language for this, it quickly became obvious that
those tools were written for debhelper and debhelper embeds a lot of
information about how to call them.  (deb-systemd-invoke, for an obvious
example, has a very minimal man page.)

The invocation added by dh_installsystemd is also not entirely trivial.
Here is an example from wpasupplicant, reformatted a bit to make it more
readable in a mail message (and also note that my system has several
versions of this, so it's clearly under active development):

# Automatically added by dh_installsystemd/13.11.6
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] \
   || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
  # The following line should be removed in trixie or trixie+1
  deb-systemd-helper unmask 'wpa_supplicant.service' >/dev/null || true

  # was-enabled defaults to true, so new installations run enable.
  if deb-systemd-helper --quiet was-enabled 'wpa_supplicant.service'; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable 'wpa_supplicant.service' >/dev/null || true
  else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state 'wpa_supplicant.service' >/dev/null || true
  fi
fi
# End automatically added section
# Automatically added by dh_installsystemd/13.11.6
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] \
   || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
  if [ -d /run/systemd/system ]; then
systemctl --system daemon-reload >/dev/null || true
if [ -n "$2" ]; then
  _dh_action=restart
else
  _dh_action=start
fi
deb-systemd-invoke $_dh_action 'wpa_supplicant.service' >/dev/null || true
  fi
fi
# End automatically added section

This is postinst, which is the most complex, but prerm and postrm snippets
are also required.

This felt like kind of a lot to put into Policy.  There is a line that
apparently won't be needed in the future, there's an update-state concept
that's internal to the tool, and the logic differs based on whether this
is a package upgrade or a new installation.  We can put this all into
Policy, but I would just be blindly copying and pasting code from
debhelper, and I'm very dubious that it is going to stay accurate and up
to date (as witnessed by the fact that the existing snippets in Policy for
using update-rc.d and invoke-rc.d do not match what debhelper does and are
likely broken for packages that also include systemd units or packages
that need to care about DPKG_ROOT).

Also, honestly, nearly all packagers think of debhelper as an opaque tool
that everyone uses, and I have a larger concern that this makes Policy
much less useful for a large part of its intended audience than it could
be.  Quite a lot of Policy is of the general format "here's a bunch of
complex things you need to do, wait, never mind, just use debhelper, go
see its documentation for the configuration files you should use instead"
and some of the rest of Policy is "here's a bunch of complex things you
need to do but if you follow these instructions instead of using debhelper
your package will be buggy."  This is not ideal!

I think there's a lot of appeal of having a thorough specification for
what debhelper is supposed to be doing.  It would enable future
competition around better packaging helpers (and I do think debhelper will
not be the last word).  But I also want to be realistic about whether
we're really capable of maintaining that 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Sam Hartman]

> "Bill" == Bill Allombert  writes:
Bill> But we do: we support debhelper 13.11.4 and debhelper 13.11.6.
Bill> Even if we support a single implementation, we still need to
Bill> know what is expected of it.

At that level, I think the answer is roughly that if you call
dh_installsystemd, then any units in the package installation directory,
or any units matching certain file patterns in the debian directory will
be installed and if appropriate enabled.


I understand there's some work to do:

* do we support units in /lib/systemd/system, /usr/lib/systemd/system,
  or both.

* What are those filename patterns for finding service units under
  debian?

* You might implicitly call dh_installsystemd via dh

I think Russ might be arguing that we should actually push all that out
to the Debhelper documentation.  I'd support that, although I understand
that makes the debhelper 13.4 vs 13.6 issue more relevant.  I'd also
support describing a bit of the interface between debian/rules and
debhelper wrt systemd units if it allowed us to move forward.

My argument is that even if we support multiple versions of debhelper,
we do not need to define the interface between debhelper and systemd in
policy: we do not need to specify deb-systemd-helper and
deb-systemd-invoke.

--Sam

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Bill Allombert]

On Mon, Sep 11, 2023 at 09:21:56AM -0600, Sam Hartman wrote:
> > "Santiago" == Santiago Vila  writes:
> 
> Santiago> El 10/9/23 a las 4:09, Russ Allbery escribió:
> >> I therefore would like to propose a first: I think Policy should
> >> simply say that any package that provides a system service should
> >> use debhelper and rely on dh_installsystemd to put the
> >> appropriate commands in its maintainer scripts.  (We can then
> >> discuss whether we should do the same for init scripts and
> >> dh_installinit, although its stanzas are simpler.)
> 
> Santiago> Hello. I don't like this approach, and I believe we are
> Santiago> mixing two different things here. One of them is our
> Santiago> ability (or lack thereof) of policy to catch up with real
> Santiago> development done elsewhere. Another one is the fact that
> Santiago> policy does not mandate any given implementation.
> 
> The question in my mind is whether it is valuable to support multiple
> implementations, and I think the answer is no.

But we do: we support debhelper 13.11.4 and debhelper 13.11.6.
Even if we support a single implementation, we still need to know what is
expected of it.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Sam Hartman]

> "Santiago" == Santiago Vila  writes:

Santiago> El 10/9/23 a las 4:09, Russ Allbery escribió:
>> I therefore would like to propose a first: I think Policy should
>> simply say that any package that provides a system service should
>> use debhelper and rely on dh_installsystemd to put the
>> appropriate commands in its maintainer scripts.  (We can then
>> discuss whether we should do the same for init scripts and
>> dh_installinit, although its stanzas are simpler.)

Santiago> Hello. I don't like this approach, and I believe we are
Santiago> mixing two different things here. One of them is our
Santiago> ability (or lack thereof) of policy to catch up with real
Santiago> development done elsewhere. Another one is the fact that
Santiago> policy does not mandate any given implementation.

The question in my mind is whether it is valuable to support multiple
implementations, and I think the answer is no.

In the past, there was not a preference for using debhelper.  So, back
then, we did need to support multiple implementations of debian/rules,
and we needed to specify the things debhelper did.

Having a specified interface like policy is expensive.  I know; I've
spent a good chunk of my life working on various protocols and
standards.
Having specified interfaces brings value when you need multiple
implementations and in a few other cases, like where you need to plan
for certain forms of extensibility or isolation.

There's a part of this where we do need an interface.  We will have
multiple packages wanting their debian/rules to install systemd units.
So, we do need to at least say or imply that if you stick systemd units
in the right place and call dh_installsystemd, then your systemd units
will be properly handled.

But today, we have a policy preference for using debhelper.  We do not
need multiple implementations of debhelper.  There does sort of need to
be an interface between debhelper and systemd if for no other reason
than to allow for systemd to change and to control which details are
hard-coded in maintainer scripts.  But if we agree that we do not need
multiple implementations of debhelper, the policy team does not need to
be part of defining that interface.  We can be more efficient by not
being involved in that process.  Also, we can do a better job of
focusing on the interface that the project does care about (how to tell
debhelper to install your systemd units) and not confuse people with the
details between debhelper and systemd.

Also, by explicitly acknowledging that the deb-systemd-helper and
deb-systemd-invoke interfaces are effectively between debhelper and
systemd, those interfaces can be simpler because they do not need to
allow for multiple implementations of debhelper or systemd.

In effect by making this decision, we are strengthening our preference
for saying packages should use debhelper.  For a significant class of
packages (those with service units) there really will be no easy
alternative.  And if we go down that route, over time, we will probably
prefer debhelper more and more, and it will be less possible to generate
a policy-compliant package that does not use debhelper.

I think that's desirable.
I think we can still find ways to experiment with new more declarative
ways of handling package building.  But I think that having more
uniformity in the cases where we have a single solution that has broad
support will make things easier for everyone.

Put another way, having multiple implementations is often very expensive
in terms of interface complexity, testing complexity, and especially
complexity that developers need to deal with.
In this instance, I do not think that cost is justified.

--Sam


signature.asc
Description: PGP signature

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Bill Allombert]

On Mon, Sep 11, 2023 at 12:47:15PM +0200, Santiago Vila wrote:
> El 10/9/23 a las 4:09, Russ Allbery escribió:
> > I therefore would like to propose a first: I think Policy should simply
> > say that any package that provides a system service should use debhelper
> > and rely on dh_installsystemd to put the appropriate commands in its
> > maintainer scripts.  (We can then discuss whether we should do the same
> > for init scripts and dh_installinit, although its stanzas are simpler.)
> 
> Hello. I don't like this approach, and I believe we are mixing two different 
> things
> here. One of them is our ability (or lack thereof) of policy to catch up with
> real development done elsewhere. Another one is the fact that policy does
> not mandate any given implementation.

I agree. The issue is that we need to document what dh_installsystemd should do,
otherwise we will not be able to distinguish between bug in dh_installsystemd 
and
intended behaviour, and we will end up freezing dh_installsystemd to avoid 
introducing
problem by breaking incidental interfaces.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Santiago Vila]

I wrote:

I believe that by choosing the wording appropriately, we can still keep this
desired property of Policy while still not mandating any given implementation.


Sorry, that was terribly worded. I meant that we can avoid the hassle of
documenting everything dh_installsystemd does and at the same time not
*formally* mandating the use of dh_installsystemd.

Thanks.

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Santiago Vila]

El 10/9/23 a las 4:09, Russ Allbery escribió:

I therefore would like to propose a first: I think Policy should simply
say that any package that provides a system service should use debhelper
and rely on dh_installsystemd to put the appropriate commands in its
maintainer scripts.  (We can then discuss whether we should do the same
for init scripts and dh_installinit, although its stanzas are simpler.)


Hello. I don't like this approach, and I believe we are mixing two different 
things
here. One of them is our ability (or lack thereof) of policy to catch up with
real development done elsewhere. Another one is the fact that policy does
not mandate any given implementation.

I believe that by choosing the wording appropriately, we can still keep this
desired property of Policy while still not mandating any given implementation.

(i.e. we could essentially say "you should do the same thing
as dh_installsystemd does", but in an elegant way).

Thanks.

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Sam Hartman]

> "Russ" == Russ Allbery  writes:

Russ> I therefore would like to propose a first: I think Policy
Russ> should simply say that any package that provides a system
Russ> service should use debhelper and rely on dh_installsystemd to
Russ> put the appropriate commands in its maintainer scripts.  (We
Russ> can then discuss whether we should do the same for init
Russ> scripts and dh_installinit, although its stanzas are simpler.)

For a variety of reasons, I support this.


--Sam

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Edward Little]

Please remove the following email address:  e.little...@gmail.com

On Sat, Sep 9, 2023 at 10:12 PM Russ Allbery  wrote:

> Package: debian-policy
> Version: 4.6.2.0
> Severity: important
> X-Debbugs-Cc: r...@debian.org
>
> As part of reviewing #1039102, I took a detailed look at Policy 9.3
> on system services and realized that it is largely obsolete and is
> not followed by most Debian packages that provide system services.
> Specifically:
>
> * There is no real documentation of systemd units except in the
>   introduction, and most of the section describes init scripts as
>   if they were the only way to start a service.
>
> * All packages are told they must use update-rc.d, but systemd units
>   no longer use update-rc.d.  They instead use deb-systemd-helper in
>   ways that are not documented in Policy and I believe are maintained
>   primarily in debhelper.
>
> * All packages are told they should use invoke-rc.d, but systemd units
>   no longer use invoke-rc.d.  They instead use deb-systemd-invoke,
>   which also supports the policy-rc.d interface.
>
> * The policy-rc.d interface is undocumented.
>
> This part of Policy is also a bit of a mess of deleted sections due to
> a desire to avoid section renumbering.
>
> I started a rewrite of this section, but quickly ran into the question
> of how to document the correct invocations of deb-systemd-helper and
> deb-systemd-invoke.  After thinking about this for a while, the
> conclusion I reached was that documenting this in Policy is both extra
> make-work that we do not have the resources to keep up with, and serves
> no real purpose for nearly every reader of Debian.  debhelper is already
> maintaining the correct code to set up systemd services in close
> cooperation with the systemd and init-system-helpers maintainers, that
> code contains temporary workarounds and other fixes that Policy is not
> going to keep up with, and it's hard to justify duplicating that work in
> a Policy statement.
>
> I therefore would like to propose a first: I think Policy should simply
> say that any package that provides a system service should use debhelper
> and rely on dh_installsystemd to put the appropriate commands in its
> maintainer scripts.  (We can then discuss whether we should do the same
> for init scripts and dh_installinit, although its stanzas are simpler.)
>
> Previously we have tried to avoid doing this, and have maintained the
> principle that debhelper is simply an *implementation* of Policy, and it
> still falls to Policy to describe what debhelper should do.  However, I
> think it is now abundantly obvious that debhelper has considerably more
> development resources available to it than Policy has writers, debhelper
> developers are quite rightfully not waiting for things to be added to
> Policy before they can be implemented, and essentially every Debian
> package that does anything non-trivial uses debhelper now.  I also don't
> see any truly useful purpose served by dumping a wad of shell code into
> Policy that essentially no one will use because it's what debhelper would
> add for them.
>
> I have some other questions about the rewrite of these sections (such as
> how hard we should try to retain section numbering), but I think we should
> resolve this question first, since it has dramatic effects on what text
> we should write.
>
>
> -- System Information:
> Debian Release: trixie/sid
>   APT prefers unstable
>   APT policy: (990, 'unstable'), (500, 'unstable-debug'), (1,
> 'experimental')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
>
> Kernel: Linux 6.4.0-4-amd64 (SMP w/8 CPU threads; PREEMPT)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE
> not set
> Shell: /bin/sh linked to /usr/bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
>
> debian-policy depends on no packages.
>
> Versions of packages debian-policy recommends:
> ii  libjs-sphinxdoc  5.3.0-7
>
> Versions of packages debian-policy suggests:
> pn  doc-base  
>
> -- no debconf information
>
>

Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete

[Russ Allbery]

Package: debian-policy
Version: 4.6.2.0
Severity: important
X-Debbugs-Cc: r...@debian.org

As part of reviewing #1039102, I took a detailed look at Policy 9.3
on system services and realized that it is largely obsolete and is
not followed by most Debian packages that provide system services.
Specifically:

* There is no real documentation of systemd units except in the
  introduction, and most of the section describes init scripts as
  if they were the only way to start a service.

* All packages are told they must use update-rc.d, but systemd units
  no longer use update-rc.d.  They instead use deb-systemd-helper in
  ways that are not documented in Policy and I believe are maintained
  primarily in debhelper.

* All packages are told they should use invoke-rc.d, but systemd units
  no longer use invoke-rc.d.  They instead use deb-systemd-invoke,
  which also supports the policy-rc.d interface.

* The policy-rc.d interface is undocumented.

This part of Policy is also a bit of a mess of deleted sections due to
a desire to avoid section renumbering.

I started a rewrite of this section, but quickly ran into the question
of how to document the correct invocations of deb-systemd-helper and
deb-systemd-invoke.  After thinking about this for a while, the
conclusion I reached was that documenting this in Policy is both extra
make-work that we do not have the resources to keep up with, and serves
no real purpose for nearly every reader of Debian.  debhelper is already
maintaining the correct code to set up systemd services in close
cooperation with the systemd and init-system-helpers maintainers, that
code contains temporary workarounds and other fixes that Policy is not
going to keep up with, and it's hard to justify duplicating that work in
a Policy statement.

I therefore would like to propose a first: I think Policy should simply
say that any package that provides a system service should use debhelper
and rely on dh_installsystemd to put the appropriate commands in its
maintainer scripts.  (We can then discuss whether we should do the same
for init scripts and dh_installinit, although its stanzas are simpler.)

Previously we have tried to avoid doing this, and have maintained the
principle that debhelper is simply an *implementation* of Policy, and it
still falls to Policy to describe what debhelper should do.  However, I
think it is now abundantly obvious that debhelper has considerably more
development resources available to it than Policy has writers, debhelper
developers are quite rightfully not waiting for things to be added to
Policy before they can be implemented, and essentially every Debian
package that does anything non-trivial uses debhelper now.  I also don't
see any truly useful purpose served by dumping a wad of shell code into
Policy that essentially no one will use because it's what debhelper would
add for them.

I have some other questions about the rewrite of these sections (such as
how hard we should try to retain section numbering), but I think we should
resolve this question first, since it has dramatic effects on what text
we should write.


-- System Information:
Debian Release: trixie/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (500, 'unstable-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 6.4.0-4-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

debian-policy depends on no packages.

Versions of packages debian-policy recommends:
ii  libjs-sphinxdoc  5.3.0-7

Versions of packages debian-policy suggests:
pn  doc-base  

-- no debconf information