Re: BITS from the DPL For September/October 2019

[Paul Wise]

On Sat, Nov 2, 2019 at 7:06 AM Bernd Zeimetz wrote:

> That leads to the question how long it takes until these bugs are being
> noticed.
>
> I am definitely not going to test init scripts properly when the systemd
> units are exactly doing what they are supposed to. The number of people
> not using systemd are probably low enough to ensure bugs are not found
> until its to late.

It seems like autopkgtests might be the right answer here, but I don't
know if debci also runs the tests on sysvinit/etc based systems.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: BITS from the DPL For September/October 2019

[Bernd Zeimetz]

On 10/31/19 8:07 PM, Alf Gaida wrote:
> I read it the same way - and also a logical consequece: if these
> patches lead to bugs, the maintainer should not be forced to fix the
> mess. I for myself would just remove buggy things that nobody care in a
> certain amount of time.

That leads to the question how long it takes until these bugs are being
noticed.

I am definitely not going to test init scripts properly when the systemd
units are exactly doing what they are supposed to. The number of people
not using systemd are probably low enough to ensure bugs are not found
until its to late.


-- 
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F

Re: BITS from the DPL For September/October 2019

[Vincent Bernat]

 ❦ 31 octobre 2019 21:49 +01, Thomas Goirand :

> The idea has always been that it would be on best-effort from people who
> volunteer, without forcing anyone to do any sysv-rc support if they
> don't feel like it. What you describe goes along this line.

I have raised my concern about this a few months ago and it seems this
is not the consensus:

 
 

My impression is that there are some people pushing sysvinit related
work to all maintainers through this policy item but keeping quiet
during discussions like this. A GR would give us a better view of what
the project thinks.
-- 
I do desire we may be better strangers.
-- William Shakespeare, "As You Like It"


signature.asc
Description: PGP signature

Re: BITS from the DPL For September/October 2019

[Moritz Mühlenhoff]

Thomas Goirand  schrieb:
> "I’d very happily maintain the init script."
>
> I haven't read all the bug entry, but if someone is claiming that
> accepting such contribution is mandatory, then that's very much right,
> at least that the consensus we're in right now, indeed.

This isn't limited to just shipping an init script, have a look at the
tomcat9 9.0.13-1 changelog entry which dropped the sysvinit script.
Continuing to support an init script also means to retain on all the
packaging boilerplate which got obsoleted by systemd config declaratives,
forcing the least common denominator of init system features.

Cheers,
Moritz

Re: BITS from the DPL For September/October 2019

[Thomas Goirand]

On 10/31/19 8:07 PM, Alf Gaida wrote:
> I read it the same way - and also a logical consequece: if these
> patches lead to bugs, the maintainer should not be forced to fix the
> mess. I for myself would just remove buggy things that nobody care in a
> certain amount of time.

I'd be very much fine with this type of move.

The idea has always been that it would be on best-effort from people who
volunteer, without forcing anyone to do any sysv-rc support if they
don't feel like it. What you describe goes along this line.

Thomas Goirand (zigo)

Re: BITS from the DPL For September/October 2019

[Thomas Goirand]

On 10/31/19 7:36 PM, Moritz Mühlenhoff wrote:
> Thomas Goirand  schrieb:
>> My understanding is that the current guidance is that doing init script
>> isn't mandatory.
> 
> With policy not updated, people claim it's mandatory, see e.g.
> #925473 on Tomcat.

Well, the policy is wrong, and should be updated then. I do believe we
all agree on this already.

Now, looking at the said bug, we have Thorsten Glaser proposing:
"I’d very happily maintain the init script."

I haven't read all the bug entry, but if someone is claiming that
accepting such contribution is mandatory, then that's very much right,
at least that the consensus we're in right now, indeed. Which is *very*
different from mandating the package maintainers to do the work all by
themselves.

> The discussed GRs would provide clarification on what the project
> at large actually wants (and what policy should spell out).

So, if I understand well, you guys want to remove the mandatory
acceptation of contributed sysv-rc init scripts. At this point in time,
we could do that, but well, if it's this way, I do believe we must as well:
- stop any non-linux port effort, and remove them from ports.d.o.
- remove sysv-rc, filerc and openrc from Debian.
- edit the policy and mandate each package having .service files.
- file (probably non-rc) bugs against things providing a sysv-rc script,
asking for its removal.
- make a release goal for Bullseye so that we get rid of any non-systemd
startup scripts.

Because if we can't provide support for these, we may as well remove
them all together, and if we do, we should do it the right way.

In the text of the said GR, please make all of this very clear,
otherwise, we'd be misleading the voters!

Cheers,

Thomas Goirand (zigo)

Re: BITS from the DPL For September/October 2019

[Gunnar Wolf]

Simon Richter dijo [Wed, Oct 30, 2019 at 12:46:21PM +0100]:
> Hi,
> 
> On Tue, Oct 29, 2019 at 10:38:32PM +0100, Thomas Goirand wrote:
> 
> > If we have such vote again, I'll continue on this direction: I'd prefer
> > if we didn't have to vote.
> 
> >From a Policy perspective, packages are supposed to integrate into the
> system by providing init scripts, and Policy has a lengthy definition of
> how init scripts need to behave.
> 
> We need to at least mention systemd unit files at some point, so action is
> needed, and that action needs to be backed up by a vote to avoid being
> more divisive than necessary.

I completely agree with this -- It is time for Policy to specify at
least the equivalence of init scripts to systemd units.

Policy has always documented standard practice, so it was right for it
to lag before doing this. It is, I guess, time to revisit...

> So far, systemd adoption has been mostly smooth sailing because the
> ecosystem effectively consists of two blocks: the systemd core, which is
> centrally coordinated, and some traditional Unix daemons hanging off it,
> but essentially not integrating. This is about to change as projects
> actually start using systemd.

Yes. And that will be a thorn for people investing time in maintaining
Debian as an init-system-agnostic distribution. Or worse, following
the examples you provide :-\

> (...)
> All that requires way more complex unit files than anything we normally
> deal with in Debian, and we need to decide how much control we want to keep
> over package integration here, because that is what Debian does: take
> upstream software and provide a coherent integrated whole.
> 
> By leaving it unspecified in Policy what systemd features are expected in a
> particular Debian release, we essentially take a back seat in what should
> be our core competence.

Although by taking advantage of them, we will not be able to support
on an equal tier other init systems.

> If the project feels that these use cases are not worth supporting, then
> that is a policy decision that needs to be voted on, not just silently
> implemented, last but not least so we can update our marketing materials
> with footnotes on the "universal operating system" tagline, and users (who,
> after all, are one of our priorities) can adjust their expectations.
> 
> So yes, having a vote is necessary at this point. We've neglected setting
> policy way too long, the scope of the project is unclear at this point, and
> people are pulling in all kinds of directions.

Very much agree with this. It is time to revisit and think clearly
about what we win/lose, and where are we heading as a
project. Probably the only way forward is via the full GR process.


signature.asc
Description: PGP signature

Re: BITS from the DPL For September/October 2019

[Alf Gaida]

On Thu, 31 Oct 2019 18:40:58 +0100
Thomas Goirand  wrote:
> On 10/29/19 11:19 PM, Russ Allbery wrote:

> > of clear project guidance, no one is clearly empowered to prevent
> > it from bitrotting.  
What is wrong with bitrotting if nobody cares about - in case nobody
cares about it's the logical consequence. And please don't even try to
enforce this care. 

> My understanding is that the current guidance is that doing init
> script isn't mandatory. What is mandatory, is to ACK init scripts /
> patches when contributed (through the BTS).
I read it the same way - and also a logical consequece: if these
patches lead to bugs, the maintainer should not be forced to fix the
mess. I for myself would just remove buggy things that nobody care in a
certain amount of time.
 
> I think that's fair enough for everyone, and I don't see why we need
> to change anything to this rule.
Dito

> Make what deferred decision? That we want to actively dismiss patches
> adding init script support? This IMO would just destroy any work
> attempted in the non-linux ports, or anyone willing to add support
> for a new init system. I don't think that's fair for these. Voting
> them out would not feel right.
See it different: if the provided scripts/patches are good, why not
take and keep them - systemd users are not hurt. If the scripts/patches
don't work - ignore the upcoming bugs and downgrade everything about to
whishlist - or if interested in: Fix the bugs. Default will work and
good. And i don't want to provoke, i see it exactly this way, if people
spend time one something, why blocking or discourage if it do no harm.
I even merge patches for kfreebsd and the hurd upstream - but i would
never active work on it.

> As much as I understand, we never wrote or decided we would. We just
> need to accept patches to add or fix sysv-rc scripts. What makes you
> think sysv-rc scripts are mandatory?
Good question - if these things are not mandatory nothing needs to
change.

> 
> Thomas Goirand (zigo)
> 



-- 
Alf Gaida
BDBF C688 EFAD BA89 5A9F  464B CD28 0A0B 4D72 827C

Re: BITS from the DPL For September/October 2019

[Moritz Mühlenhoff]

Thomas Goirand  schrieb:
> My understanding is that the current guidance is that doing init script
> isn't mandatory.

With policy not updated, people claim it's mandatory, see e.g.
#925473 on Tomcat.

The discussed GRs would provide clarification on what the project
at large actually wants (and what policy should spell out).

Cheers,
Moritz

Re: BITS from the DPL For September/October 2019

[Thomas Goirand]

Russ,

On 10/29/19 11:19 PM, Russ Allbery wrote:
> [...] we do not have clear
> project consensus that sysvinit support is mandatory in new packages, so
> the support is starting to bitrot, and given the lack of clear project
> guidance, no one is clearly empowered to prevent it from bitrotting.

I think we mostly agree on the topic, but now what the consensus is.

My understanding is that the current guidance is that doing init script
isn't mandatory. What is mandatory, is to ACK init scripts / patches
when contributed (through the BTS).

I think that's fair enough for everyone, and I don't see why we need to
change anything to this rule.

> The current Policy text is a mess, and everything it says on the topic is
> essentially accidental, being left over from text that was added to
> clarify how to support upstart, before the TC decision on the default init
> system.  It's unclear what requirements Policy should actually set on
> packages that want to ship daemons, and any attempt to hammer that out is
> likely to be contentious and have great difficulty reaching consensus.

Sure, the Debian policy text needs some love and contributions, though
what I wrote above is more or less what we all agreed on. Besides this,
the state of the policy is (unfortunately, and because the policy needs
updates) orthogonal to this discussion.

> I think it was the right thing to do to refuse to make a clear long-term
> decision at the time, when the project had just gone through a bruising
> and awful argument.  Now that we have some distance and have seen how the
> ecosystem has subsequently evolved, I think it's time to circle back and,
> hopefully with more accumulated wisdom, a bit of emotional distance, and a
> bit more calm, make the deferred decision.

Make what deferred decision? That we want to actively dismiss patches
adding init script support? This IMO would just destroy any work
attempted in the non-linux ports, or anyone willing to add support for a
new init system. I don't think that's fair for these. Voting them out
would not feel right.

> If we're really going to continue to fully support sysvinit

As much as I understand, we never wrote or decided we would. We just
need to accept patches to add or fix sysv-rc scripts. What makes you
think sysv-rc scripts are mandatory?

Thomas Goirand (zigo)

Re: BITS from the DPL For September/October 2019

[Holger Levsen]

Sam,

thanks for the bits mail. Very interesting as usual! Just a minor
nitpick...

On Tue, Oct 29, 2019 at 01:16:21PM -0400, Sam Hartman wrote:
> * I spent some money.

No, you didn't, Debian did, as you correctly described below. You
approved that spending.


-- 
cheers,
Holger

(I'm not sure why it is important to me to point this out, but as I
thought to write this mail yesterday and still want to write it
today... I hope this is just a joke/witty way of putting things which
got lost in translation...)

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C



signature.asc
Description: PGP signature

Re: BITS from the DPL For September/October 2019

[Simon Richter]

Hi,

On Tue, Oct 29, 2019 at 10:38:32PM +0100, Thomas Goirand wrote:

> If we have such vote again, I'll continue on this direction: I'd prefer
> if we didn't have to vote.

>From a Policy perspective, packages are supposed to integrate into the
system by providing init scripts, and Policy has a lengthy definition of
how init scripts need to behave.

We need to at least mention systemd unit files at some point, so action is
needed, and that action needs to be backed up by a vote to avoid being
more divisive than necessary.

So far, systemd adoption has been mostly smooth sailing because the
ecosystem effectively consists of two blocks: the systemd core, which is
centrally coordinated, and some traditional Unix daemons hanging off it,
but essentially not integrating. This is about to change as projects
actually start using systemd.

For example, libvirt needs to tightly integrate with systemd because a lot
of the automatisms for the desktop are actively harmful in a VM hosting
setup.

My VM host starts with two visible network cards, loading the driver
enables fourteen more virtual functions, so I start with sixteen Ethernet
devices, fifteen of which need to be completely ignored by
systemd-networkd, and one has static configuration. When VMs start up, LVM
logical volumes are activated, but should not have fsck run on them,
because they are then having their permissions changed so KVM can open them
exclusively and make them available to the guest. The virtual network cards
then have their drivers unbound from them, making the interface disappear,
and the PCIe device is passed through to the guest. In addition, libvirt
creates a tap device, and connects it to a bridge device whose IP
configuration lives inside libvirt's configuration database.

All that requires way more complex unit files than anything we normally
deal with in Debian, and we need to decide how much control we want to keep
over package integration here, because that is what Debian does: take
upstream software and provide a coherent integrated whole.

By leaving it unspecified in Policy what systemd features are expected in a
particular Debian release, we essentially take a back seat in what should
be our core competence.

At the very least, I'd like Policy to spell out what types of unit files
are supported, what keywords in those unit files exist, and what target
names are used to denote certain points in the boot sequence (similar to
how we define priority ranges for init scripts). We probably don't need to
replicate the full documentation for these things, but we are deviating
from upstream systemd by doing stable releases, and we need to at least
document the deviation.

We should probably also make some inroads into when unit files should
specify dependencies and which. Systemd explicitly differentiates between
functional and order dependencies, and lots of daemons just specify both.
Are these required, or just cargo-culted from somewhere? I'd expect Debian
maintainers of services that ship unit files to understand exactly what
these dependencies do, and whether they are needed, and that means adding
that information to Tasks in the New Maintainer process.

All of these are exactly the kind of bureaucracy that Debian is (in)famous
for, but which has enabled us to build a high-quality distribution, and
which will be necessary going forward if we actually want to use more of
systemd's features and still keep our reputation.

In my opinion, keeping the other init systems around is also still useful,
because there will always be installations where the default behaviour of
systemd is counterproductive, and keeping the system running after updates
is playing whack-a-mole with newly introduced features that also need to be
turned off.

If the project feels that these use cases are not worth supporting, then
that is a policy decision that needs to be voted on, not just silently
implemented, last but not least so we can update our marketing materials
with footnotes on the "universal operating system" tagline, and users (who,
after all, are one of our priorities) can adjust their expectations.

So yes, having a vote is necessary at this point. We've neglected setting
policy way too long, the scope of the project is unclear at this point, and
people are pulling in all kinds of directions.

   Simon

Re: BITS from the DPL For September/October 2019

[Sam Hartman]

> "Wouter" == Wouter Verhelst  writes:


Wouter> Having said that,

Wouter> Sam: I notice that you've not sent a draft of your GR
Wouter> proposal to the -vote mailing list yet. It has been my
Wouter> experience over the years that that is not generally a good
Wouter> idea. The DPL does have the power to bypass much of the GR
Wouter> procedure, and in urgent situations this is probably a good
Wouter> thing; but I believe that the drafting of GR ballots in the
Wouter> open on a public mailinglist is actually an essential part
Wouter> of the whole GR procedure, which allows people to form an
Wouter> opinion together, resulting in fewer (but better) options on
Wouter> the ballot.

I think we agree in part.  I think that an open discussion of the ballot
prior to any formal procedure happening is important.

When I'm ready for that I'll send a draft to debian-vote.

I'll also note that it's rare that the DPL has used their power to call
for a general resolution, and so we actually don't have a lot of
experience with how best to handle things when the DPL is trying to
facilitate a project poll.

Both in DPL facilitated GRs and in other GRs I actually think there's
some important leg work that can be done ahead of the public process.  I
want to come with a ballot that I think is reasonable and an
understanding of some of the issues involved at the beginning of the
public process.  It will make ith easier for me to answer questions and
to facilitate a discussion.

I absolutely agree that a public discussion prior to anyone pushing the
button and getting the secretary involved is important.

I expected to be ready for that a week ago, but I got distracted by my
day job.

--Sam

Re: BITS from the DPL For September/October 2019

[Wouter Verhelst]

On Tue, Oct 29, 2019 at 03:19:03PM -0700, Russ Allbery wrote:
> We've now had several years of essentially declining to make a decision
> and trying to see if the project can muddle through, and while I feel
> somewhat vindicated by the fact that this didn't immediately fall apart
> and has sort of worked, I think it's becoming increasingly untenable.  We
> now have contributors who are far-removed from the original debate and who
> may have only used a systemd-based Debian system and we do not have clear
> project consensus that sysvinit support is mandatory in new packages, so
> the support is starting to bitrot, and given the lack of clear project
> guidance, no one is clearly empowered to prevent it from bitrotting.

Hear hear.

While I was reading Sam's message, I was a bit apprehensive about having
a vote about this; but his arguments, and yours, make sense in that it
is a good idea to either tell people we're no longer interested in
multiple init systems as a project, or to empower those who want to work
on it that we, as a project, think it is a sensible idea to do so and
that not supporting alternative init systems should be considered an RC
bug.

(FWIW, even though one of my packages doesn't currently have an init
script where it should, I happen to think the latter is true; but it has
become very clear to me over the past few years that this opinion is far
from common in the project, and that is precisely the reason why this
vote would be a good idea).

Having said that,

Sam: I notice that you've not sent a draft of your GR proposal to the
-vote mailing list yet. It has been my experience over the years that
that is not generally a good idea. The DPL does have the power to bypass
much of the GR procedure, and in urgent situations this is probably a
good thing; but I believe that the drafting of GR ballots in the open on
a public mailinglist is actually an essential part of the whole GR
procedure, which allows people to form an opinion together, resulting in
fewer (but better) options on the ballot.

It is times when we are divided, such as in the case of GR 2004_004,
that this process breaks down and the number of options on the ballot
skyrockets. This is also the reason, I think, why ballots that are
drafted in secret almost always immediately receive amendments as soon
as they are proposed, thereby negating any perceived benefit that they
might have provided.

For that reason, I would like to urge you to draft the ballot in the
open, unless you think there is little time for this and you need to
have something to move forward urgently -- which would be something I
would disagree with.

Thanks,

-- 
To the thief who stole my anti-depressants: I hope you're happy

  -- seen somewhere on the Internet on a photo of a billboard

Re: BITS from the DPL For September/October 2019

[Russ Allbery]

Thomas Goirand  writes:

> The last time we've had a GR on init systems, the general response was
> that we don't want to vote, and we preferred the TC to decide.

I don't think the core questions here are technical questions.  I think
they're more fundamental questions of project direction and questions
about what burden we want to put on ourselves as package maintainers in
order to support a variety of init systems.  This seems like a good set of
issues to resolve democratically, at least for lack of a better
alternative.  Resolution by TC on this issue seems likely to lead to lead
to challenges to the TC's impartiality or legitimacy.  That's really not
very much fun to go through when you're on the TC.

We've now had several years of essentially declining to make a decision
and trying to see if the project can muddle through, and while I feel
somewhat vindicated by the fact that this didn't immediately fall apart
and has sort of worked, I think it's becoming increasingly untenable.  We
now have contributors who are far-removed from the original debate and who
may have only used a systemd-based Debian system and we do not have clear
project consensus that sysvinit support is mandatory in new packages, so
the support is starting to bitrot, and given the lack of clear project
guidance, no one is clearly empowered to prevent it from bitrotting.

The current Policy text is a mess, and everything it says on the topic is
essentially accidental, being left over from text that was added to
clarify how to support upstart, before the TC decision on the default init
system.  It's unclear what requirements Policy should actually set on
packages that want to ship daemons, and any attempt to hammer that out is
likely to be contentious and have great difficulty reaching consensus.

I think it was the right thing to do to refuse to make a clear long-term
decision at the time, when the project had just gone through a bruising
and awful argument.  Now that we have some distance and have seen how the
ecosystem has subsequently evolved, I think it's time to circle back and,
hopefully with more accumulated wisdom, a bit of emotional distance, and a
bit more calm, make the deferred decision.

If we're really going to continue to fully support sysvinit, we should
commit to doing so clearly and empower people to test that support and
report bugs of appropriate severity (and also to create a stronger
incentive for making that support easier to achieve).  If we're not, we
should unambiguously free people from doing additional work that they
don't want to do and can't test easily, and also more clearly communicate
the project's intentions so that our partners like Devuan can make
informed decisions about how to proceed.  The simmering uncertainty and
low-level tension of not having a decision eventually becomes its own
problem.

A GR may not be the best mechanism to make that decision, but I'm not sure
what method would be better.

-- 
Russ Allbery (r...@debian.org)  

Re: BITS from the DPL For September/October 2019

[Thomas Goirand]

On 10/29/19 6:16 PM, Sam Hartman wrote:
> Init System Policy
> ==
> 
> last Bits mail, I talked about how I was considering whether we needed a
> GR on Init System Policy.

The last time we've had a GR on init systems, the general response was
that we don't want to vote, and we preferred the TC to decide.

If we have such vote again, I'll continue on this direction: I'd prefer
if we didn't have to vote.

Thomas