date:20131230

]] Russ Allbery 

First, thanks to both you and Ian for the quite comprehensive
write-ups.

> If the package later changes the flags in some orthogonal way, it's
> easy for the system to miss that change.  This is something that,
> under systemd, will probably require development of new tools to warn
> the adminsitrator of what's happened.  upstart avoids this problem by
> having the whole configuration be managed as a conffile.

You might want to have a look at systemd-delta.  On my quite boring home
machine:

: tfheen@xoog ~ > systemd-delta
[MASKED] /etc/udev/rules.d/75-persistent-net-generator.rules → 
/lib/udev/rules.d/75-persistent-net-generator.rules

1 overridden configuration files found.
: tfheen@xoog ~ > 

It can also output diffs.

> I think the upstart approach is better, but I think the drawbacks of the
> systemd approach could be mostly overcome with some fairly simple Debian
> tools.

We were initially considering using ucf and checking if the file in /etc
had changed (before we had per-line overrides and such), but with the
more complex overrides now available, I think systemd-delta is a better
solution.  I guess we could integrate that into the packaging somehow
and present a useful UI if you've overridden a line that changes.

[...]

> But I do think it blunts some of the porting argument.  The non-Linux
> ports are going to have to port, fork, or replace systemd components
> anyway, regardless of the choice of init system, or drop out of
> feature parity with the Linux ports.

It's not like we have feature parity today.  Hurd has the whole
translators setup.  kFreeBSD has jails and ZFS.  Nobody is arguing that
we must ship those with Linux too, so why should the feature parity have
to go in the other direction?  Personally, I think the more interesting
use case for kFreeBSD is on the server side, and not as a GNOME
workstation.  I also realise a file system is not on the same magnitude
for a distribution as an entire desktop environment, but we're looking
at degrees here anyway, not a black and white picture.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/m2k3emk9z6@rahvafeir.err.no

Bug#727708: upstart and upgrading from sysvinit scripts

On Sun, Dec 29, 2013 at 10:05:17PM +0100, Tollef Fog Heen wrote:
> > It would, however, be nice if this were more clearly stated, since the
> > guidance to the author of the unit file about what dependencies one should
> > or should not explicitly add is a bit sparse.  In particular, I wonder if
> > there is an implicit After= dependency in a service unit on a socket unit
> > of the same name (which would make sense given how Sockets= works), or if
> > that's something that one should explicitly add.

> http://www.freedesktop.org/software/systemd/man/bootup.html has some
> graphs.  In addition, as long as your service is not doing anything in
> the early boot, DefaultDependencies will be true and you'll have an
> After=basic.target in your service.

On Sun, Dec 29, 2013 at 01:18:00PM -0800, Russ Allbery wrote:
> Tollef Fog Heen  writes:

> > ]] Russ Allbery 

> >> It would, however, be nice if this were more clearly stated, since the
> >> guidance to the author of the unit file about what dependencies one should
> >> or should not explicitly add is a bit sparse.  In particular, I wonder if
> >> there is an implicit After= dependency in a service unit on a socket unit
> >> of the same name (which would make sense given how Sockets= works), or if
> >> that's something that one should explicitly add.

> > http://www.freedesktop.org/software/systemd/man/bootup.html has some
> > graphs.  In addition, as long as your service is not doing anything in
> > the early boot, DefaultDependencies will be true and you'll have an
> > After=basic.target in your service.

> This by itself doesn't fully replace the implicit dependency.  It ensures
> ordering is correct for boot, but not that ordering is correct when
> starting deactivated services, where the service startup is not happening
> in the context of target processing.  (Unless target processing happens in
> more places than I think it does.)

> It sounds, from Uoti's investigations, that the code also directly adds an
> implicit dependency, which would ensure correct behavior in those cases as
> well.

Right.  So between these two aspects of systemd's implied dependencies
(which I believe postdate my previous investigations, given the behavior I
observed at the time), it sounds like there are no races here for the
general case, and I agree that this provides a solid mechanism for
activation of services whose authors wish to avoid handling sockets
directly.

I'm also interested to know how systemd purports to handle the exceptional
cases, where a dependency on basic.target is not possible.  For instance,
NFS mounts at boot: modern NFS on Linux requires a complex set of
interdependent RPC daemons to be started on the client before an NFS share
can be mounted.  In my experience, these are some of the most complex
boot-time interdependencies around, which would most benefit from something
like socket-based activation to implicitly resolve ordering constraints.

The nfs-common and rpcbind packages currently have no integration with
systemd, so they get the default behavior for SysV services in rcS.d:
WantedBy=sysinit.target, Before=sysinit.target.  But there's nothing which
documents that sysinit.target is a precondition for remote-fs.target, so in
its current state, mounting of remote filesystems at boot is almost
certainly racy with systemd in Debian.  What does a correct implementation
of NFS support on systemd look like?  I've tried installing nfs-utils on
Fedora 20, and while it provides a full set of native systemd units,
surprisingly, the only one that references remote-fs-pre.target is nfs-lock. 
So it looks to me like this is another case where no one has actually done
the work yet to properly integrate with systemd, and a migration to systemd
would cause regressions for users of this configuration (and probably
others).

What is the right way to make nfs-common behave correctly on startup with
systemd?  Is there a reason that sockets.target is only a dependency of
basic.target, not of remote-fs-pre.target, which would enable use of
socket-based activation for fs helper daemons like those in nfs-common? 
(Note, of course, that nfs-utils currently has no support for the systemd
socket activation protocol.  However, if one of the arguments for systemd is
socket activation, then I think we should explore the limits of how we think
it should be used.)

Thanks,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: upstart proposed policy in Debian [and 1 more messages]

[Started drafting this before Ian forked the bug; sending to both bug
reports now]

On Fri, Dec 20, 2013 at 03:41:55PM -0800, Russ Allbery wrote:
> Ian Jackson  writes:
> > Russ Allbery writes:

> >> I'd like to see both of them support systemd's method, since it's
> >> extensible and provides more general functionality.  I think the
> >> benefit of support for SIGSTOP is marginal; adding sd_notify calls is
> >> not that much harder and doesn't have the conceptual weirdness of
> >> stopping the daemon to notify the init system that it's running.

> > I strongly disagreee.

> > As the upstream author of a daemon I'm
> >  - not willing to add a library dependency for this
> >  - not willing to write a 50-100 lines of tiresome socket code for this
> >  - not willing to copy the library file into my source tree
> > so the result is that userv upstream won't support systemd's readiness
> > protocol.

> Yes, we completely disagree.

> Among other things, that's about the smallest and least-impactful library
> dependency I can imagine.  My intent in integrating support for sd_notify
> is to just stub out the call when not built with systemd support, which is
> pretty trivial to do with Autoconf and means that those who want systemd
> integration get it and those who don't care can easily ignore it.

The lowest-impact library dependency is still pretty large, from the
perspective of the typical daemon upstream.  Plenty of projects don't use
autoconf.  Some aren't written in C at all and would need bindings (or to
reimplement the base logic natively).

There is an elegance to sd_notify() that appeals to you, I can understand
that.  But as Ian's response demonstrates, it doesn't appeal to everybody.

I think the next-generation init system should bring solutions that improve
things for all consumers, not just to those maintainers for whom introducing
an init-system-specific external library dependency is compatible with their
sense of aesthetics.  You say that both upstart and systemd are
backwards-compatible, for those who don't want to adopt their respective
readiness interfaces.  That's true, but that just means
backwards-compatibility with a world of bugs that we should strive to
eliminate.  There are all kinds of race conditions in the init scripts we
have today in Debian, caused by improper daemonization (forking/detaching
before listening, etc.).  I think declaring that upstreams that don't want
to use sd_notify() (or change to use socket-based activation) can just use
the backwards-compatibility means we fall short of the kind of
across-the-board uplift that we should seek to achieve.

This discussion makes it clear to me that the SIGSTOP approach /also/
doesn't achieve that, due to equal but opposite aesthetic preferences on the
part of different groups of maintainers.  :) So I'm in favor of upstart
implementing sd_notify, alongside its existing SIGSTOP handling.

But I think it doesn't make sense to treat it as a mark against upstart, for
Debian's purposes, that upstart started from the more conservative end of
the spectrum on this question while systemd was more ambitious.  If
anything, the long time it's taken Debian to even seriously consider the
question of moving to upstart shows that by at least one relevant measure,
even upstart was being too aggressive. :/

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: systemd and support for other distros

On Mon, Dec 02, 2013 at 02:48:52PM -0800, Russ Allbery wrote:
> > Sure.  Both systemd and upstart manage to avoid the problem of
> > inconsistent behavior due to tainted admin environments, because daemons
> > are always started as children of init and not of the admin's login
> > shell.  That being the case, hard-coding the path to an executable in
> > your initscript equivalent doesn't buy you much added protection,
> > compared with just using the system $PATH, and does cause gratuitous
> > incompatibilities in exactly those cases that Debian Policy's
> > prohibition on hard-coded paths is meant to address.

> I have never seen a gratuitous incompatibility caused by this.  Do you
> have any examples?

I would argue that every single result returned by
'ls -l /usr/sbin/ /usr/bin|grep /bin', preventing us from merging /usr/
into / by default, is an example of such historical incompatibilities.  But
any other cases where binaries have moved from one directory or another
without providing such compat symlinks would also qualify.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Bug#727708: upstart and upgrading from sysvinit scripts

]] Steve Langasek 

> I'm also interested to know how systemd purports to handle the exceptional
> cases, where a dependency on basic.target is not possible.  

In general «you need to write the dependencies manually, then».  As
you're pointing out in your mail, that can get tricky to get right.

> The nfs-common and rpcbind packages currently have no integration with
> systemd, so they get the default behavior for SysV services in rcS.d:
> WantedBy=sysinit.target, Before=sysinit.target.  But there's nothing which
> documents that sysinit.target is a precondition for remote-fs.target, so in
> its current state, mounting of remote filesystems at boot is almost
> certainly racy with systemd in Debian.

That looks buggy or racy indeed.

> What is the right way to make nfs-common behave correctly on startup with
> systemd?  Is there a reason that sockets.target is only a dependency of
> basic.target, not of remote-fs-pre.target, which would enable use of
> socket-based activation for fs helper daemons like those in nfs-common? 

rpcbind.socket would have a Before=rpcbind.service automatically
(assuming we want socket activation).  In addition, I imagine you'd want
the various daemons to have After + Wants on rpcbind.service and
rpcbind.service to be before=remote-fs-pre.target and
WantedBy=remote-fs-pre.target.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/m2bnzyk7m2@rahvafeir.err.no

Bug#727708: upstart user jobs

Hi Ian,

On Sat, Dec 14, 2013 at 12:31:57PM +, Ian Jackson wrote:
> I have some questions about these.  Forgive me if I could just have
> looked up the answers:

> Are they enabled by default in jessie/sid ?
> (If the answer is "no" then feel free not to answer the rest...)

"No" :)

Using upstart user jobs in Debian would imply a whole added level of
transition above and beyond adoption of upstart as init, and would require
coordination with maintainers of e.g. desktop environment packages and
display manager packages.  I think it would be a logical next step for
Debian to consider, if it adopted upstart as a default; but so long as
upstart is not the default in Debian I don't think it would be a good idea
to try to support this in Debian.

> In the manpage I read:
>   | Note that a user job configuration file cannot have the same name
>   | as a system job configuration file.
> I don't understand this restriction.  It's sounds like it's referring
> to the pathnames in which case it's trivially true, so I assume it's
> referring to job names.

Hmm, this sounds like a documentation bug, a throwback to an earlier
iteration of the user job support.  Which manpage did you find this in?

The current implementation certainly has no such restriction.  For instance,
on an Ubuntu system there are both /etc/init/dbus.conf and
/usr/share/upstart/sessions/dbus.conf, for the system bus and the session
bus respectively, and these do not collide.  System-level events are visible
to the user session, but they are qualified with a disambiguating ":sys:"
prefix.

> Does anything that user jobs do depend on upstart being pid 1, or
> being root ?  Does the thing which reads (and watches) the user's
> configuration files run as root, or as the user ?  I.e., what is the
> privilege separation ?

The upstart "session init" runs as the user, not as root.  I'm not sure if
upstart as a user session has any dependencies on upstart being PID 1. 
Cc:ing James, who would know better - James, do you know if upstart session
init works on non-upstart systems?

> The docs say:
>  | Files in this directory will be read and an inotify(7) watch
>  | created the first time a user runs initctl(8).
> Does this really mean that if I'm fiddling around with writing some
> jobs, but not quite ready yet, and say "initctl --help", my jobs will
> start to run ?  Also, it would appear to imply that user jobs aren't
> started automatically at boot.

This seems to be a quote from the 1.6.1 version of the manpage, in wheezy. 
The user session support in the current releases of upstart (the only
implementation that's been used in production in Ubuntu) doesn't work this
way; and the manpage has been updated to match.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: systemd jessie -> jessie+1 upgrade problems

2013-12-30 Thread Florian Weimer

* Josselin Mouette:

> Le mardi 17 décembre 2013 à 12:26 -0800, Russ Allbery a écrit :
>> > Is there actually any implementation other than glib2.0 and libdbus that
>> > would be affected by a switch to kdbus?
>> 
>> This is an interesting question.  Josselin, is GNOME (for example) likely
>> to acquire a hard dependency on kdbus through some mechanism?  I don't
>> understand the plumbing of this stuff well enough to know where the
>> dependencies could surface.
>
> That doesn’t seem very likely to me. In terms of library API, kdbus
> doesn’t change anything, so there is no need for applications to depend
> on it.

Isn't it fairly likely that some libraries or applications will use
kdbus directly because their developers dislike the existing
libraries?  It happens all the time with other kernel functionality
(inotify and its cousins, netlink, etc.).


--
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ha9qe8p3@mid.deneb.enyo.de

Bug#727708: upstart proposed policy in Debian [and 1 more messages]

Steve Langasek  writes:

> The lowest-impact library dependency is still pretty large, from the
> perspective of the typical daemon upstream.  Plenty of projects don't
> use autoconf.  Some aren't written in C at all and would need bindings
> (or to reimplement the base logic natively).

I still don't entirely agree with this, although the point about language
bindings is certainly valid.  (I'm tempted to just fix that problem for
Perl, since no one seems to have to date, but that supports your point.)
But I don't think there's a need to debate it, since as noted below I'm
not sure it really matters.

> But I think it doesn't make sense to treat it as a mark against upstart,
> for Debian's purposes, that upstart started from the more conservative
> end of the spectrum on this question while systemd was more ambitious.
> If anything, the long time it's taken Debian to even seriously consider
> the question of moving to upstart shows that by at least one relevant
> measure, even upstart was being too aggressive. :/

Note that I didn't include anything about the notification policy in my
writeup.  That's because after further consideration, I agreed: I don't
consider this a notable advantage to either system.

(I probably owe everyone an additional followup message listing some of
the things that I don't think are differentiators, in part to call out
what I consider exemplary support from multiple different projects over
the course of this evaluation.  I'm very grateful to both the upstart and
the systemd teams for the work they've already done on integration and
their responses during my discovery process.)

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/874n5qwap4@windlord.stanford.edu

Bug#727708: systemd and support for other distros

Steve Langasek  writes:
> On Mon, Dec 02, 2013 at 02:48:52PM -0800, Russ Allbery wrote:

>> I have never seen a gratuitous incompatibility caused by this.  Do you
>> have any examples?

> I would argue that every single result returned by 'ls -l /usr/sbin/
> /usr/bin|grep /bin', preventing us from merging /usr/ into / by default,
> is an example of such historical incompatibilities.  But any other cases
> where binaries have moved from one directory or another without
> providing such compat symlinks would also qualify.

Ah, yes, thank you.  Having to maintain those compatibility symlinks is
indeed a burden that's incurred by using fully-qualified paths.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87zjniuw2m@windlord.stanford.edu

Bug#727708: systemd and upstart, a view from a daemon Debian maintainer

Steve Langasek writes ("Bug#727708: systemd and upstart, a view from a daemon 
Debian maintainer"):
> I also think that the extensive maintainer script changes required for any
> upstart-using package are quite deplorable (whether or not they're wrapped
> in a helper script + debhelper snippet).  [...]

Did you mean 
  the extensive maintainer script changes required for any
  systemd-using package are quite deplorable
  ^^^
?

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.42155.711320.219...@chiark.greenend.org.uk

Bug#727708: init system other points, and conclusion

Tollef Fog Heen  writes:
> ]] Russ Allbery 

>> I think the upstart approach is better, but I think the drawbacks of
>> the systemd approach could be mostly overcome with some fairly simple
>> Debian tools.

> We were initially considering using ucf and checking if the file in /etc
> had changed (before we had per-line overrides and such), but with the
> more complex overrides now available, I think systemd-delta is a better
> solution.  I guess we could integrate that into the packaging somehow
> and present a useful UI if you've overridden a line that changes.

Right, several people have pointed me at systemd-delta, but your last
sentence was the tool that I was driving at.  systemd-delta tells you
whether you're currently overriding something, which is a useful building
block, but which doesn't restore conffile upgrade handling.  What's needed
to keep existing Debian behavior is a tool that tells you whether the
maintainer and the local system administrator have changed the
configuration in conflicting ways so that something is now being
overridden that was not being overridden before.

In other words, systemd-delta does a two-way diff, but what's needed here
is a three-way diff plus integration into the packaging system so that, as
with conffile changes now, the upgrade (by default) stops and prompts the
local systems administrator with the orthogonal differences and lets them
choose how to handle the situation.

I don't think this would be particularly difficult to implement on top of
systemd-delta, although the details may be tricky to get right and may
require a few iterations.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87vby6uvsu@windlord.stanford.edu

Bug#727708: init system other points, and conclusion [and 1 more messages]

Tollef Fog Heen writes ("Bug#727708: init system other points, and conclusion"):
> Ian Jackson:
> > This is exacerbated by the fact that systemd's Debian maintainers are
> > (IMO) much too deferential to upstream.
> 
> That's because the bits of systemd you've asked to change isn't just a
> piece of software.  It's a standardised API, and you're effectively
> asking us to change that API.  I think it's pretty clear why that is a
> bad idea.

Which of my three rejected proposals are you discussing ?

That systemd support SIGSTOP readiness indication; that it support a
different simpler readiness protocol; or that it allow relative
pathnames in unit files ?

Of these your objection clearly doesn't apply to the first two.
systemd already supports around four readiness signally protocols
(depending how count them); adding another would certainly not break
anything.

The last one - relative pathnames - is a strict enhancement.  The
only effect would be that Debian's unit files wouldn't necessarily
work on older systemds which didn't have the patch.

Tollef Fog Heen writes ("Bug#733452: init system daemon readiness protocol"):
> It seems Lennart read the proposal and responded in
> https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/37AWJLE3XcJ

In the light of this message from Lennart, and the example code, it
seems that the documentation is not adequate.

Lennart writes:
 | sending a message to $NOTIFY_SOCKET would require setting
 | SCM_CREDENTIALS (wrong!),

But the only documentaton I can find for this protocol is in
sd_notify(3), which says[1]:

 | The datagram is accompanied by the process
 | credentials of the sending daemon, using SCM_CREDENTIALS.

If this is not required by systemd, why is it done by sd_notify ?
Is this actually the right documentation to be reading ?

Ian.

[1] 
http://manpages.debian.net/cgi-bin/man.cgi?query=sd_notify&sektion=3&apropos=0&manpath=Debian+testing+jessie&locale=

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.42733.823185.940...@chiark.greenend.org.uk

Bug#733452: Minimal code for systemd protocol

Nikolaus Rath writes ("Bug#733452: Minimal code for systemd protocol"):
> It's already been mentioned elsewhere, but I think it should be included in 
> this bug for reference. The minimum code to support systemd style readyness 
> notification is (from 
> https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/37AWJLE3XcJ):

This code snippet does not do what sd_notify(3) says is required, but
perhaps the documentation is wrong.

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.42794.348553.475...@chiark.greenend.org.uk

Bug#733452: init system daemon readiness protocol

Tollef Fog Heen writes ("Bug#733452: init system daemon readiness protocol"):
> Ian Jackson:
> > I conclude therefore that we should design another simple protocol -
> > preferably, a variation on one of the existing ones - and have (at
> > least) both Debian's systemd and Debian's upstart implement it.
> 
> I think you're into ever-multiplying power socket standards territory
> here.  I am not going to carry patches in systemd in Debian for a
> Debian-only notification protocol because you don't want to use the
> upstream protocol.

I would like an easier protocol to be supported by upstream too.
Perhaps that wasn't clear from my message.  I would certainly prefer
to avoid some Debian-specific protocol.

Later:
> It seems Lennart read the proposal and responded in
> https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/37AWJLE3XcJ

Nevertheless, even if SCM_CREDENTIALS is not required by systemd, my
proposed protocol is still simpler than the systemd one.

Lennart doesn't seem to have addressed this point.

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/21185.42952.83261.226...@chiark.greenend.org.uk

Bug#727708: init system other points, and conclusion

Russ Allbery  writes:

> * Red Hat adopted upstart but never did a wholescale conversion, and then
>   abandoned upstart in favor of systemd.  Obviously, one should not put
>   too much weight on this; Red Hat is a commercial company that has a
>   wealth of reasons for its actions that do not apply to Debian.  But I
>   think it's still worth noting that the only non-Ubuntu major adopter of
>   upstart backed away from it.

[...]

>   By comparison, upstart is effectively used only by Ubuntu, [...]

Both of these statements are incorrect.

I'm sure that somewhere in the many vast threads that we've had over the
init system, someone pointed out to me that Google's Chromium OS and
Chrome OS use upstart.  Now that I've been reminded of this, I think Steve
also mentioned it in the discussion of how upstart's upstream and
packaging development were handled.  However, it had completely dropped
out of my mind and I was unaware of it while writing the above statements.

I apologize for the misrepresentation.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87iou6uv4a@windlord.stanford.edu

Bug#733452: init system daemon readiness protocol

(Sorry, 2nd copy here because I missed up the change of To field in
the previous one.)

cameron writes ("Re: Bug#733452: init system daemon readiness protocol"):
> I was curious: why should SOCK_STREAM be used instead of SOCK_DGRAM in 
> your proposed protocol?

SOCK_DGRAM sockets do not offer reliable delivery (at least, not on
all unices).

> Have you seen Lennart Poettering's pastebin of a short daemon side 
> implementation of that protocol: http://fpaste.org/64821/32737713/? It 
> meets all your desired criteria, it is used in one init system already, 
> and it is very extensible. Now that you know that systemd does not 
> actually use SOCK_SEQPACKET, but SOCK_DGRAM, do you have any changes in 
> opinion of the systemd approach?

I still think it would be simpler to pass the ready-connected socket
(or whatever) to the daemon by inheritance, rather than having the
daemon call socket() etc.

Do you know why in systemd it was done the way it was ?

Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.44921.223653.273...@chiark.greenend.org.uk

Bug#727708: upstart user jobs

Steve Langasek writes ("Re: Bug#727708: upstart user jobs"):
> On Sat, Dec 14, 2013 at 12:31:57PM +, Ian Jackson wrote:
> > I have some questions about these.  Forgive me if I could just have
> > looked up the answers:
> 
> > Are they enabled by default in jessie/sid ?
> > (If the answer is "no" then feel free not to answer the rest...)
> 
> "No" :)
> 
> Using upstart user jobs in Debian would imply a whole added level of
> transition above and beyond adoption of upstart as init, and would require
> coordination with maintainers of e.g. desktop environment packages and
> display manager packages.  I think it would be a logical next step for
> Debian to consider, if it adopted upstart as a default; but so long as
> upstart is not the default in Debian I don't think it would be a good idea
> to try to support this in Debian.

I'm not sure I see the connection.  AIUI user jobs are a way to do
roughly what cron's @reboot facility does, only better.

> > In the manpage I read:
> >   | Note that a user job configuration file cannot have the same name
> >   | as a system job configuration file.
> > I don't understand this restriction.  It's sounds like it's referring
> > to the pathnames in which case it's trivially true, so I assume it's
> > referring to job names.
> 
> Hmm, this sounds like a documentation bug, a throwback to an earlier
> iteration of the user job support.  Which manpage did you find this in?

http://manpages.debian.net/cgi-bin/man.cgi?query=init&sektion=5&apropos=0&manpath=Debian+testing+jessie&locale=

> > The docs say:
> >  | Files in this directory will be read and an inotify(7) watch
> >  | created the first time a user runs initctl(8).
> > Does this really mean that if I'm fiddling around with writing some
> > jobs, but not quite ready yet, and say "initctl --help", my jobs will
> > start to run ?  Also, it would appear to imply that user jobs aren't
> > started automatically at boot.
> 
> This seems to be a quote from the 1.6.1 version of the manpage, in wheezy. 
> The user session support in the current releases of upstart (the only
> implementation that's been used in production in Ubuntu) doesn't work this
> way; and the manpage has been updated to match.

OK, good, thanks.

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.44789.411269.837...@chiark.greenend.org.uk

Bug#727708: init system other points, and conclusion [and 1 more messages]

]] Ian Jackson 

> Tollef Fog Heen writes ("Bug#727708: init system other points, and 
> conclusion"):
> > Ian Jackson:
> > > This is exacerbated by the fact that systemd's Debian maintainers are
> > > (IMO) much too deferential to upstream.
> > 
> > That's because the bits of systemd you've asked to change isn't just a
> > piece of software.  It's a standardised API, and you're effectively
> > asking us to change that API.  I think it's pretty clear why that is a
> > bad idea.
> 
> Which of my three rejected proposals are you discussing ?

All of them.  It means that you can't test and verify that a daemon
works correctly with systemd in Debian and expect it to work the same
with systemd in other distributions.

I'm not saying those kinds of changes are unwanted.  I'm saying they
should go upstream and that I am unwilling to carry those patches in the
Debian package.

> Tollef Fog Heen writes ("Bug#733452: init system daemon readiness protocol"):
> > It seems Lennart read the proposal and responded in
> > https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/37AWJLE3XcJ
> 
> In the light of this message from Lennart, and the example code, it
> seems that the documentation is not adequate.

I think you're just being confused by it giving you some information you
don't need.  The documentation in question reads: 

  Internally, these functions send a single datagram with the state
  string as payload to the AF_UNIX socket referenced in the
  $NOTIFY_SOCKET environment variable. If the first character of
  $NOTIFY_SOCKET is «@», the string is understood as Linux abstract
  namespace socket. The datagram is accompanied by the process
  credentials of the sending daemon, using SCM_CREDENTIALS.

  For details about the algorithms check the liberally licensed
  reference implementation sources:
  
http://cgit.freedesktop.org/systemd/systemd/plain/src/libsystemd-daemon/sd-daemon.c
  and
  http://cgit.freedesktop.org/systemd/systemd/plain/src/systemd/sd-daemon.h

SCM_CREDENTIALS is generally not something the sender sends, it's
something the receiver sets on the socket using SO_PASSCRED.

> If this is not required by systemd, why is it done by sd_notify ?

It's not.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/m27gamjkus@rahvafeir.err.no

Bug#727708: init system other points, and conclusion

2013-12-30 Thread gregor herrmann

On Mon, 30 Dec 2013 09:05:57 -0800, Russ Allbery wrote:

> >   By comparison, upstart is effectively used only by Ubuntu, [...]
> Both of these statements are incorrect.
> I'm sure that somewhere in the many vast threads that we've had over the
> init system, someone pointed out to me that Google's Chromium OS and
> Chrome OS use upstart.  

Maemo5 also uses upstart (a quite old version but after all, almost
everything there is quite old).

Cheers,
gregor

-- 
 .''`.  Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
 : :' : Debian GNU/Linux user, admin, and developer  -  http://www.debian.org/
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   NP: Supertramp: Ain't Nobody But Me


signature.asc
Description: Digital signature

Bug#727708: systemd-shim uploaded to NEW

Steve Langasek writes ("Bug#727708: systemd-shim uploaded to NEW"):
> So I repeat here my request that the systemd maintainers make a suitable
> split of the systemd binary package, so that systemd-shim will be
> coinstallable with the systemd-provided implementations of the other dbus
> services.

Is there a summary of the systemd maintainers' response to this
request ?  I glanced #726763 and #729576 but they were very long and
if the answer is in there I probably wouldn't have found it.

>  The only alternative I see is for systemd-shim to declare a
> Replaces: against systemd without a Conflicts,

This would be quite wrong; Replaces is not supposed to be used like
that (but you apparently know that).

How do the systemd maintainers think this problem should be solved ?

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.47990.599239.983...@chiark.greenend.org.uk

Bug#727708: systemd-shim uploaded to NEW

2013-12-30 Thread Raphael Hertzog

On Mon, 30 Dec 2013, Ian Jackson wrote:
> >  The only alternative I see is for systemd-shim to declare a
> > Replaces: against systemd without a Conflicts,
> 
> This would be quite wrong; Replaces is not supposed to be used like
> that (but you apparently know that).
> 
> How do the systemd maintainers think this problem should be solved ?

I don't know the specifics of systemd-shim, but dpkg-divert is the usual
answer when you need/want to replace something without the consent of the
other side.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131230184258.gc24...@x230-buxy.home.ouaza.com

Bug#727708: loose ends for init system decision

This message is about a transition plan for an init system replacement and
about how to handle portability to our non-Linux ports.  I'm keeping the
same subject as Ian's message on the same basic topics and attaching it to
the same thread, but this is more of a separate writeup than a reply.
I'll reply to Ian's message separately.

I've expressed my opinion separately about which init system we should
choose.  This message is written from the assumption that we will choose
either upstart or systemd as the init system for the non-Linux ports.  I
believe either system would pose basically the same concerns.


1. Role of Non-Linux Ports in Debian

There has been a lot of discussion in various places, including the Debian
mailing lists, about the utility of the non-Linux ports and about how much
they should play a role in project decisions.  I think this deserves to be
addressed directly here.

One of the things that I love about Debian, and one of the reasons why I
am involved in this project as opposed to a different distribution, is
that it is a labor of love.  Debian is not driven by market forces; we
have users and we want the distribution to work for them, but we're not
competing for market share.  Debian does not make decisions based on
simple popularity, or on customer counts.  Rather, Debian is a project of
mutual cooperation among Debian's contributors.  We work on what we care
about, regardless of whether that makes economic sense.

It is not, in general, necessary to justify what you want to do in Debian.
It doesn't matter if it's going to be used by thousands of people or two
people.  If you can do your work to the standards that we expect to be
maintained across the archive, and without negative impact on Debian's
other contributors, you can work on whatever you love.  And, furthermore,
we all support each other in our passions.  Debian is built on a culture
of deference to other people's work, and reasonable accomodation to the
projects that other people want to work on.

Now, there is a fine balance here.  Deference to other people's work does
not mean a requirement to join their work.  Reasonable accomodation does
not mean that every Debian developer is required to test their software on
non-Linux ports.  The goal is that all of us should be empowered to work
on the things we are passionate about, which implicitly includes being
empowered to not work on the things that are not of interest to us.
Therefore, for some efforts, and specifically for the non-Linux port
efforts, the work is mostly born by the porters.  They're expected to
test, diagnose problems, and submit patches.  The deference and reasonable
accomodation that we expect of Debian contributors is to merge those
patches when they are reasonable, to not undermine the porting effort when
there is a reasonable approach that preserves it, and to be aware of the
implications of their packaging for those ports in the broad sense (such
as qualifying build-dependencies with [linux-any] where appropriate).

We do not expect Debian contributors to reject significant new upstream
versions or significant new integrations because they will affect the
non-Linux ports, or, for that matter, other projects in Debian.  We do
expect those changes to follow the standards that we expect of Debian as a
whole, and that porting efforts will be treated with deference and
reasonable accomodation.

I think this status applies to both our Hurd port and our kFreeBSD port.
Those ports have different challenges, and arguably different levels of
utility to general users, but as mentioned, I don't consider popularity to
be a useful metric here.  It doesn't make any difference to me if the port
is used by thousands of people or if it's only used by the people actively
working on it: the same principles of deference and reasonable
accomodation apply.

I believe strongly that this is something that defines Debian as a project
and is worth preserving.

It's also worth saying here that I don't believe any of the above is
particularly controversial within the project.  We have wide-ranging
arguments about it in the abstract, and quite a few disparaging comments
have been thrown around about the non-Linux ports in the init system
discussion, which makes me sad.  But when it comes to the day-to-day work
in the project, nearly all maintainers are quite good (and have been for
years) about merging required patches for the Hurd and pushing them
upstream, responding to bug reports on systems they don't use, and making
a reasonable effort to accomodate other people's projects.

Similarly, our non-Linux porters have been exemplary.  Neither the Hurd
nor the kFreeBSD ports have expected every Debian contributor to directly
support their platforms.  They track down problems, develop patches, and
submit tested patches to the BTS.  They have developed their own solutions
for packages that are not portable and worked out how to integrate them
with the archive.

Despite the often off-

Bug#727708: init system other points, and conclusion

Russ Allbery writes ("Bug#727708: init system other points, and conclusion"):
> We seem to be at the point of the process where at least those of us who
> did early investigation are stating conclusions.  I think I have enough
> information to state mine, so will attempt to do so here.

Thanks.

> First, other choices besides systemd and upstart.

I agree with your comments here; it appears you've investigated OpenRC
in more detail than I have but I'm happy to take your word for it.


> 2. Core Service Management Functionality
...
> To my surprise, that's not what happened.  Rather, I concluded that
> systemd has a substantial technical advantage over upstart, primarily in
> terms of available useful features, but also in terms of fundamental
> design.

I think we have fundamentally different ideas about what the
replacement init ought to be like.  I don't think either of us will
convince the other.


> 3. Ecosystem and Portability
...
> 3.1. Ecosystem Reality Check
...
> Rather, we're talking about whether or not to swap out a core component of
> an existing integrated ecosystem with a component that we like better.

Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.

Also, I get the impression me that the "integration" of much of this
functionality into the systemd source package has been done for
political rather than technical reasons.  Indeed to the extent that
there is a problematically tight technical coupling between these
components, I find it difficult to avoid doubting the good faith of
the people making those decisions.  At the very least, I worry that
the systemd project as a whole is far too willing to impose decisions
of all kinds on its downstreams.

Under those kind of circumstances I am willing for the Debian project
as a whole to go to quite some effort (and, indeed, impose some effort
even on the maintainers of systemd in Debian) to retain the
flexibility that I think is important.  That flexibility certainly
includes the ability for a user to drop use of parts of systemd that
they don't find desirable.

> Now, I am generally on the side that says loose coupling of ecosystems is
> an inherent good.  However, I don't agree that it's such an inherent good
> that we should disassemble things just for the sake of having disassembled
> things.  At feature parity, and absent any compelling reason to swap
> components, I think we should take the path of least resistance and use
> the integrations that other people have already developed.  Debian has
> more than enough hard integration problems to solve without creating new
> ones for ourselves unnecessarily.  But that's the key word: unnecessarily.
> If we do have a reason for doing this, we should seriously consider it.

If these problems have been created with reckless disregard for the
flexibility needs of downstreams and users, then I take the opposite
view.


> 3.2. Portability
...
> So, in short, I consider portability to be a possible benefit of upstart,
> but I'm inclined to discount that advantage for several reasons.  One,
> it's not yet actually materialized and still may not,

My understanding was that Dimitri had got upstart running on BSD.
I can't imagine that this problem won't be solved (in Debian) for
jessie.  We're not talking about some nonexistent hypothetical
patchset.


> 3.3. Project Momentum

I don't see the outlook here the same way as you do.

Furthermore, I am much less worried about Debian going it alone
(although, of course, it's not alone) than you seem to be.  We have
historically been entirely unafraid to do our own better things, even
if it is more work and takes us longer.

I felt that way when Debian was very much a minority interest.  That's
far from the case nowadays; I've heard it asserted that Debian-based
distros now account for a majority of traditional distro installs.  I
guess numbers on that are all speculative but it is certainly true
that we have a thriving ecosystem of our own.

We have got where we are by doing things the way we think is best, not
by simply following the herd.

Of course you say that you prefer systemd, which still leads you to
the opposite conclusion.  But I wanted to explicitly deal with this
argument.

Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.49757.755100.804...@chiark.greenend.org.uk

Bug#727708: loose ends for init system decision

Russ Allbery writes ("Bug#727708: loose ends for init system decision"):
> 1. Role of Non-Linux Ports in Debian

I agree with most of this.

> 2. Impact of Multiple Init Systems

I don't want to do a blow-by-blow quote/rebuttal of this.

> 3. systemd and upstart As Multiple Systems
..
> I therefore think that, regardless of which init system we pick, we should
> keep in mind the above principle of Debian's deference and reasonable
> accomodation to other people's projects and apply that to systemd and
> upstart as well as to the non-Linux ports.  Obviously, this also has the
> same issues mentioned above: Debian contributors can only be expected to
> test on the primary init system, other configurations will tend to bitrot
> without active porter attention, and so forth.  But if people want to take
> on the work, that deserves our respect.

Right.


> 4. Conclusions
...
> 1. We should select a new init system for jessie, either systemd or
>upstart.  Support for that init system should be release-critical, but
>only in the sense that the daemon works properly under that init
>system.  In other words, use of the sysvinit compatibility of either
>init system is acceptable support for jessie.

I agree.

> 2. All packages providing init scripts must continue to support sysvinit
>scripts through the jessie release.  Such support will continue to be
>release-critical.  This is going to be painful for packages that want
>to do an early conversion, since it means testing two different init
>systems for this release cycle, but I think this is the right thing to
>do regardless for a clean upgrade path and Debian's normal robust
>committment to upgrades.  Here it has the additional advantage of
>giving the non-Linux ports some breathing space to strategize.

Yes.

> 3. Related, up through the jessie release, packages must (where possible;
>it's possible there will be cases where this is simply impossible)
>support switching back and forth between the new default init system
>and sysvinit.  This means that configurations should not be moved out
>of /etc/default and that startup files for the new init system should
>read the same configuration that the existing sysvinit scripts use (or
>both should be modified compatibly).

Yes.

> 4. Post-jessie, support for sysvinit will no longer be release-critical,
>and package maintainers will no longer be expected to ensure that it
>continues working.  [...]
> 
> 5. Support for the other init system that was not chosen should be handled
>in the same fashion, should a team choose to pursue it.  [...]

I think it is probably premature for us to drop support for the other
system in jessie+1.

But we don't need to make this decision now.

> 6. Debian's non-Linux ports should either use the same init system as
>Debian's Linux ports or agree on an init system that they're both going
>to use.  The porting work is going to be hard enough without the ports
>going in different directions on which secondary init system they want
>to use.  I prefer to leave it up to the porters to decide which init
>system to choose, but I do think OpenRC would be a strong contender.

Is there some difficulty expected with upstart on hurd ?

> We should revisit this decision again after the jessie release in case the
> situation has substantially changed.

I would be very reluctant to write now that the support for sysvinit,
or the non-default new system, may be dropped in jessie+1.  That will
result in premature rot and removal.

It's also possible that some kind of compatibility mechanism will
become available.

I would like to leave this decision to the policy maintainers, with
the expectation that the TC will probably need to decide.

Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.50247.915263.539...@chiark.greenend.org.uk

Bug#727708: loose ends for init system decision

Ian Jackson  writes:
> Russ Allbery writes ("Bug#727708: loose ends for init system decision"):

>> 6. Debian's non-Linux ports should either use the same init system as
>>Debian's Linux ports or agree on an init system that they're both going
>>to use.  The porting work is going to be hard enough without the ports
>>going in different directions on which secondary init system they want
>>to use.  I prefer to leave it up to the porters to decide which init
>>system to choose, but I do think OpenRC would be a strong contender.

> Is there some difficulty expected with upstart on hurd ?

I don't know if anyone has looked at it, and in the absence of any
practical experience, I think it's fair to expect some challenges.  The
blog post on kFreeBSD porting indicated that the porting effort to date
required eglibc and FreeBSD kernel changes.  It seems like a reasonable
assumption that at least a similar level of effort will be required on
Hurd, and I don't know if the Hurd has as mature (if incompatible)
capabilities such as kqueue on FreeBSD to use to implement such things as
inotify or epoll.

I know very little about the Hurd, so basically I just don't know.

> I would be very reluctant to write now that the support for sysvinit, or
> the non-default new system, may be dropped in jessie+1.  That will
> result in premature rot and removal.

> It's also possible that some kind of compatibility mechanism will become
> available.

That's a reasonable point.  I have no objections to deferring that
decision until after the jessie release.

> I would like to leave this decision to the policy maintainers, with the
> expectation that the TC will probably need to decide.

Yeah, with my Policy hat on, I don't want any piece of that one.  I would
be inclined to immediately escalate to the TC.  I don't believe that
Policy's conservative consensus approach is going to work here, and I
expect trying to be painful.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87sitatajs@windlord.stanford.edu

Bug#727708: init system other points, and conclusion [and 1 more messages]

On Mon, Dec 30, 2013 at 9:43 AM, Tollef Fog Heen  wrote:

 If this is not required by systemd, why is it done by sd_notify ?

It's not.

You obviously did not read the code. It is. Here is a G+ convo with 
Lennart I had:

> As a sender you only have to set SCM_CREDENTIALS manually if you 
want to 
> fake it (for which you need privs however).

sd_notify() basically impersonates the process. You only need to set 
SCM manually if you are writing an external library. If someone is just 
doing it in the daemon, the kernel will set SCM_CREDENTIALS 
automatically.

Bug#727708: init system other points, and conclusion

Ian Jackson  writes:
> Russ Allbery writes ("Bug#727708: init system other points, and conclusion"):

>> First, other choices besides systemd and upstart.

> I agree with your comments here; it appears you've investigated OpenRC
> in more detail than I have but I'm happy to take your word for it.

I should be clear that I've not actually run the system.  I've tried to
track down documentation and tried to understand what the project goals
were.  It's quite possible that I've gotten some details wrong, and I
would welcome any corrections from the OpenRC folks.

>> Rather, we're talking about whether or not to swap out a core component
>> of an existing integrated ecosystem with a component that we like
>> better.

> Unless you are proposing to make systemd mandatory for all Debian
> installations, this is work that needs to be done anyway.

What I'm hearing from both GNOME and KDE maintainers is that assuming
systemd would save them a great deal of work.  I think having systemd be
the only supported and tested option for at least some large package sets
that heavily use the systemd infrastructure upstream is a not-unlikely
long-term outcome.

In other words, I think the long-term portability future of GNOME (and to
a much lesser extent KDE, where the issue will be bitrot of unused
configurations more than an intentional maintenance choice, and where I
expect the system to at worst degrade functionality rather than stop
working) to non-systemd platforms is already in some jeopardy, because
it's not clear that resources will be available upstream to maintain those
projects outside of the APIs that systemd supports.  That was, after all,
the forcing moment that brought this whole debate to a head.

One can, of course, have a wide variety of reactions to that.  As someone
who considers portability to be an inherent good, I find it sad, although
I don't have a full appreciation for what problems GNOME is trying to
solve by increasing its reliance on systemd.  But I'm highly dubious that
Debian will just single-handedly fix that, or that we would drop GNOME
because we don't like upstream's integration decisions.

> Also, I get the impression me that the "integration" of much of this
> functionality into the systemd source package has been done for
> political rather than technical reasons.

I hear that you have this impression, but I completely disagree.  I find
the amount of bad will and assumption of malice aimed at the systemd
maintainers to be intensely depressing and unwarranted by any of the
interactions that I've seen, and I've been doing a fair bit of reading.

Lennart passionately argues his side.  So do you.  So do a lot of us.  We
all tend to have strong technical opinions and a great deal of confidence
in our own judgement.

Personally, I'm putting contentions that systemd is doing a power grab or
is merging things for political reasons into the same bucket as
contentions that Technical Committee opinions are driven by current or
past employer relationships.  I would prefer to assume good faith among
the people involved and understand their projects on their own terms.

> At the very least, I worry that the systemd project as a whole is far
> too willing to impose decisions of all kinds on its downstreams.

All upstreams do this.  I have yet to meet an upstream for any piece of
software that doesn't care about that software working uniformly on its
supported platforms.  systemd is entirely in line with normal upstream
practice of trying to pick the best solution to a given problem and
supporting it thoroughly, rather than incurring the maintenance burden of
supporting multiple ways to do the same thing.

This always involves imposing some decisions on downstreams unless
downstreams are willing to fork around that decision.  It's already meant
that some of Debian's decisions are being imposed on Red Hat because they
were judged to be better solutions.

In places where Debian needs to take a different approach for reasons of
backward compatibility or existing integrations, obviously we should, and
we should be sure that we have the flexibility to do so.  For example, I
think we should disable the current systemd binfmt support in favor of our
existing working solution.  I have not seen any indication that would be a
problem.

You made multiple proposals that do not reflect what Debian is doing
*today*, and which are not *necessary* for Debian.  I don't think those
are good test cases for upstream's willingness to accomodate Debian's
needs.

> My understanding was that Dimitri had got upstart running on BSD.

The latest that I have seen on this porting effort is here:

http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html

I asked previously on this bug if someone had later news.  Do you have
more information than that?

The above is definitely not "upstart running on BSD."  That's upstart's
underlying support library mostly working on BSD after disabling two
features (that are requir

Bug#733452: Minimal code for systemd protocol

The documentation says what sd_notify() does, not what the minimum 
requirements are. The documentation should be clarified IMO, but 
Lennart does not seem to want to do so (even though he already typed up 
a paragraph about it on G+).


On Mon, Dec 30, 2013 at 9:02 AM, Ian Jackson 
 wrote:
Nikolaus Rath writes ("Bug#733452: Minimal code for systemd 
protocol"):
 It's already been mentioned elsewhere, but I think it should be 
included in this bug for reference. The minimum code to support 
systemd style readyness notification is (from 
https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/37AWJLE3XcJ):



This code snippet does not do what sd_notify(3) says is required, but
perhaps the documentation is wrong.

Ian.


--
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact 
listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21185.42794.348553.475...@chiark.greenend.org.uk

Bug#727708: Go with Upstart

2013-12-30 Thread Pouar

While systemd is my favorite init system. Debian will probably be better 
off with upstart for the reasons Ian said, especially portability. The 
reason Fedora and RHEL didn't have a problem is they only use the Linux 
Kernel, while Debian also uses kFreeBSD and the GNU Hurd.

--
Pouar

Bug#727708: systemd and upstart, a view from a daemon Debian maintainer

On Mon, Dec 30, 2013 at 04:51:55PM +, Ian Jackson wrote:
> Steve Langasek writes ("Bug#727708: systemd and upstart, a view from a daemon 
> Debian maintainer"):
> > I also think that the extensive maintainer script changes required for any
> > upstart-using package are quite deplorable (whether or not they're wrapped
> > in a helper script + debhelper snippet).  [...]

> Did you mean 
>   the extensive maintainer script changes required for any
>   systemd-using package are quite deplorable
>   ^^^
> ?

Ahahaha.  Yes, yes I did. ;)

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Bug#733452: init system daemon readiness protocol

On Mon, Dec 30, 2013 at 9:38 AM, Ian Jackson
wrote:

(Sorry, 2nd copy here because I missed up the change of To field in
the previous one.)

cameron writes ("Re: Bug#733452: init system daemon readiness
protocol"):
I was curious: why should SOCK_STREAM be used instead of SOCK_DGRAM
in
your proposed protocol?

SOCK_DGRAM sockets do not offer reliable delivery (at least, not on
all unices).

I am pretty sure they are reliable for //local// sockets, at least on
Linux.
see this reddit comment:
http://www.reddit.com/r/linux/comments/1tya0c/lennarts_take_on_the_proposed_debian_daemon/cecstgq

Have you seen Lennart Poettering's pastebin of a short daemon side
implementation of that protocol: http://fpaste.org/64821/32737713/?
It
meets all your desired criteria, it is used in one init system
already,
and it is very extensible. Now that you know that systemd does not
actually use SOCK_SEQPACKET, but SOCK_DGRAM, do you have any
changes in
opinion of the systemd approach?

I still think it would be simpler to pass the ready-connected socket
(or whatever) to the daemon by inheritance, rather than having the
daemon call socket() etc.

Do you know why in systemd it was done the way it was ?

Yes, here are Lennart's words:

>We use SOCK_DGRAM because we are interested in the message boundary
and to get SCM_CREDENTIALS attached to each datagram by the kernel.
Note that systemd only has a single notification socket set up for all
the services it starts. All service hence queue their messages into the
same socket, and we need to be able to identify exactly from which
process each message originated, and need to make sure that the
boundaries are intact and not messages from one service are half
written and then mixed with messages from other services which write
inbetween. By using SOCK_DGRAM we can be sure that each datagram is
either fully written or never fully written, but never half-written
interleaved with another half message from somebody else. And the
kernel implicitly attaches SCM_CREDENTIALS to each of these datagrams,
but this does not translate to SOCK_STREAM.

Thanks,
Ian.

Bravo,
Cameron Norman

Re: Bug#733452: Minimal code for systemd protocol

Ian Jackson wrote:
> Nikolaus Rath writes ("Bug#733452: Minimal code for systemd protocol"):
> > It's already been mentioned elsewhere, but I think it should be
> > included in this bug for reference. The minimum code to support
> > systemd style readyness notification is (from
> > https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/37AWJLE3XcJ):
> 
> This code snippet does not do what sd_notify(3) says is required, but
> perhaps the documentation is wrong.

What part of sd_notify(3) are you referring to?  The full implementation
of sd_notify in
http://cgit.freedesktop.org/systemd/systemd/plain/src/libsystemd-daemon/sd-daemon.c
does not contain any semantic additions to the algorithm.

If you're talking about SCM_CREDENTIALS, then as has already been
mentioned elsewhere, that's not the concern of the daemon process.
Perhaps sd_notify(3) could, for clarity, explicitly say "The process
receiving the datagram (e.g. systemd) may wish to use SCM_CREDENTIALS to
obtain the process credentials of the sending daemon."

- Josh Triplett

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131230201340.GA27739@leaf

Bug#727708: upstart user jobs

On Mon, Dec 30, 2013 at 05:35:49PM +, Ian Jackson wrote:
> Steve Langasek writes ("Re: Bug#727708: upstart user jobs"):
> > On Sat, Dec 14, 2013 at 12:31:57PM +, Ian Jackson wrote:
> > > I have some questions about these.  Forgive me if I could just have
> > > looked up the answers:

> > > Are they enabled by default in jessie/sid ?
> > > (If the answer is "no" then feel free not to answer the rest...)

> > "No" :)

> > Using upstart user jobs in Debian would imply a whole added level of
> > transition above and beyond adoption of upstart as init, and would require
> > coordination with maintainers of e.g. desktop environment packages and
> > display manager packages.  I think it would be a logical next step for
> > Debian to consider, if it adopted upstart as a default; but so long as
> > upstart is not the default in Debian I don't think it would be a good idea
> > to try to support this in Debian.

> I'm not sure I see the connection.  AIUI user jobs are a way to do
> roughly what cron's @reboot facility does, only better.

The topic of upstart user jobs has evolved since upstart 1.6.  As they're
used in Ubuntu 13.04 and later, the entire desktop session is run as a set
of upstart jobs, giving service supervision and better cleanup handling on
logout.  That is not something that we should try to enable in the upstart
package in Debian without careful coordination with other maintainers.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Bug#727708: systemd-shim uploaded to NEW

]] Ian Jackson 

> Steve Langasek writes ("Bug#727708: systemd-shim uploaded to NEW"):
> > So I repeat here my request that the systemd maintainers make a suitable
> > split of the systemd binary package, so that systemd-shim will be
> > coinstallable with the systemd-provided implementations of the other dbus
> > services.
> 
> Is there a summary of the systemd maintainers' response to this
> request ?  I glanced #726763 and #729576 but they were very long and
> if the answer is in there I probably wouldn't have found it.

I see no point in doing that, in particular not in the middle of when
the ctte is choosing a new default init system.  If anything, I think
it's pretty rude of Steve to upload systemd-shim and asking the systemd
maintainers to solve the problem for him.

> >  The only alternative I see is for systemd-shim to declare a
> > Replaces: against systemd without a Conflicts,
> 
> This would be quite wrong; Replaces is not supposed to be used like
> that (but you apparently know that).
> 
> How do the systemd maintainers think this problem should be solved ?

Initially, by waiting for the ctte to come to a conclusion.  If that is
that systemd should be the default init system I think we should solve
the problem by not solving it.  If the decision is that another init
system should be solved, I'm probably going to solve it by removing the
init functionality from the systemd package at which point the bug
solves itself, AIUI.

If the systemd-shim maintainers want to solve it in the meantime, going
with Raphael's suggestion seems reasonable enough.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/m238lajc2y@rahvafeir.err.no

Bug#727708: loose ends for init system decision

Ian Jackson  writes:

> I think we should give package maintainers some guidance on what kinds
> of upstart or systemd patches should be accepted.  Without this, it's
> likely we'll find ourselves adjudicating disputes that ought to have
> been settled in principle much earlier.

> I think that patches for either should:
>  - use a non-forking startup method
>  - avoid significant amounts of open-coded AF_UNIX socketry
>  - avoid embedded copies of daemon library code

These all seem reasonable to me, with the caveat that if *upstream*
includes open-coded AF_UNIX socketry or embedded copies of daemon library
code, I don't see any need for Debian package maintainers to remove that
code.

>  - avoid extra build-dependencies (eg on libsystemd)
>  - avoid extra runtime dependencies (eg on deb-systemd-helper)

These requirements, on the other hand, I find completely baffling.

This approach seems completely contrary to Debian best practices.  Our
standard practice with all packages is to fully enable all available
features that make sense on Debian and don't pose some concrete risk.
This means that, for example, I have CUPS libraries on this system despite
the fact that I never print, Avahi libraries depsite the fact that I will
never use that software, and SELinux libraries deeply embedded in core
code despite the fact that few Debian systems currently run SELinux.

These requirements seem like would fit with Gentoo, where much emphasis is
placed on letting people build custom-crafted binaries to their local
requirements, not Debian's approach of providing a full-featured and
uniform distribution.

Needless to say, I don't think we should avoid dependencies on
libsystemd-daemon or init-system-helpers, and I think it would be wholly
inappropriate for the Technical Committee to say anything of the sort.

> (If the current state of the art in readiness protocols persists that
> means that upstart patches using raise(SIGSTOP) ought to be accepted,
> but systemd ones rejected unless they can use socket activation.)

So, as mentioned above, I don't agree with this, although of course I
think using socket activation is, in general, better than using the
readiness protocol.

> We should recommend:
>  - daemon authors and Debian maintainers should prefer command-line
>options to environment variables

I disagree with including this in a statement.  I think it's outside the
scope of what we were asked to resolve, and I don't think it's the place
of the TC to offer this sort of general advise to upstreams.

>  - whatever environment variables and fds are used, measures should be
>taken to avoid them being inherited by daemons' arbitrary children

I agree with this as good practice for daemons, but again, I don't think
it's the role of the TC to issue this sort of general advice that is not
directly related to a question put before us.

If we're going to offer specific advice, we should limit it to the
specific integrations that we're asked to consider.  But I think we should
be mindful of the restriction on the TC not doing design work and let
Policy for packaging support of upstart and/or systemd be hashed out
through the regular Policy process.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ha9qt5vu@windlord.stanford.edu

Bug#727708: systemd-shim uploaded to NEW

On Mon, Dec 30, 2013 at 06:29:10PM +, Ian Jackson wrote:
> Steve Langasek writes ("Bug#727708: systemd-shim uploaded to NEW"):
> > So I repeat here my request that the systemd maintainers make a suitable
> > split of the systemd binary package, so that systemd-shim will be
> > coinstallable with the systemd-provided implementations of the other dbus
> > services.

> Is there a summary of the systemd maintainers' response to this
> request ?  I glanced #726763 and #729576 but they were very long and
> if the answer is in there I probably wouldn't have found it.

The relevant discussion on debian-devel is here:

http://lists.debian.org/msgid-search/20131024012751.ga7...@virgil.dodds.net
http://lists.debian.org/msgid-search/87bo2fyruk.fsf...@qurzaw.varnish-software.com

> >  The only alternative I see is for systemd-shim to declare a
> > Replaces: against systemd without a Conflicts,

> This would be quite wrong; Replaces is not supposed to be used like
> that (but you apparently know that).

Yes.  Raphaël rightly points out that dpkg-divert can be used for this; if
necessary, that's what I'll do.

But I still think the right thing here is for the systemd package to be
split.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Bug#727708: init system other points, and conclusion




On Mon, Dec 30, 2013 at 11:56 AM, Russ Allbery  wrote:


The latest that I have seen on this porting effort is here:

http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html

I asked previously on this bug if someone had later news.  Do you have
more information than that?

The above is definitely not "upstart running on BSD."  That's 
upstart's

underlying support library mostly working on BSD after disabling two
features (that are required for upstart).

I know inotify is working on kFreeBSD with the libinotify-kqueue 
software (recently packaged in Debian). 
https://github.com/dmatveev/libinotify-kqueue


That leaves only abstract domain name sockets left, which should not be 
too complicated.


Upstart still does not run on kFreeBSD, though.

Re: Bug#727708: systemd-shim uploaded to NEW

Tollef Fog Heen wrote:
> Initially, by waiting for the ctte to come to a conclusion.  If that is
> that systemd should be the default init system I think we should solve
> the problem by not solving it.  If the decision is that another init
> system should be solved, I'm probably going to solve it by removing the
> init functionality from the systemd package at which point the bug
> solves itself, AIUI.

Speaking as a happy user of the Debian systemd packages: please don't.
Having seen Russ Albery's mail regarding transition plans
(https://lists.debian.org/debian-ctte/2013/12/msg00258.html), which I'd
agree with wholeheartedly, I hope that even if Debian makes the mistake
of choosing upstart as a default, that the systemd packages remain
viable.  (After all, Debian needs at least one sane init system. ;) )

Regardless of the outcome of the tech-ctte's decision, I would happily
serve as an early-warning system for any breakage that occurs in other
packages when used without upstart, and I'll happily volunteer to help
maintain the systemd packages on an ongoing basis if that would help.

Thanks again for all your work on systemd.

- Josh Triplett

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131230211144.GA32283@leaf

Bug#727708: init system other points, and conclusion [and 1 more messages]

]] cameron 

> On Mon, Dec 30, 2013 at 9:43 AM, Tollef Fog Heen  wrote:
> 
> >>  If this is not required by systemd, why is it done by sd_notify ?
> >>
> > It's not.
> 
> You obviously did not read the code. It is. Here is a G+ convo with
> Lennart I had:
>
> > As a sender you only have to set SCM_CREDENTIALS manually if you
> > want to fake it (for which you need privs however).
> 
> sd_notify() basically impersonates the process. You only need to set
> SCM manually if you are writing an external library. If someone is
> just doing it in the daemon, the kernel will set SCM_CREDENTIALS
> automatically.

You seem to be confusing systemd-notify(1) with sd_notify(3).
sd_notify(3) is the library call that's called by the daemon itself.
systemd-notify(1) is a command line tool to «Notify service manager
about start-up completion and other daemon status changes».

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/m2sitahvjg@rahvafeir.err.no

Bug#727708: init system other points, and conclusion

This message contains some supplemental information to go with my primary
writeup, and some profound thanks for the people involved in this
investigation.

I apologize for the huge volume of mail, and I know it's going to take a
while to digest.  I appreciate people's willingness to read all these
messages in detail.  This is a very complex decision-making process.  I'm
pretty mentally exhausted by the effort of trying to synthesize all these
pieces, so my apologies in advance for the inevitable errors and omissions
(some of which affected my original writeup).


1. Thanks

Throughout this evaluation process, my interactions with upstart and
systemd upstream developers and Debian packagers have been uniformly
excellent.  Bug reports filed against both systemd and upstart have
received excellent and timely response, and all involved have been quite
willing to explain things I've misunderstood, correct my false starts, and
discuss technical and practical aspects of their designs.

I was particularly impressed by the clear effort that the systemd and
upstart maintainers in Debian have put into fully integrating their init
systems in such a way that makes them easy to test and use with existing
Debian packages.  This includes but is not limited to update-rc.d support,
invoke-rc.d support, status synchronization with sysvinit, past Policy
discussion, and attention to upgrade paths and init-switching use cases.

I also want to particularly thank the OpenRC upstream development team for
their involvement in this process and their contributions to the
discussion.  I personally don't think that package is a god match for
Debian's needs on Linux, but that's through no fault of the people
involved, and I think they would be an excellent upstream if that package
looked like a good fit for the needs of any of Debian's non-Linux ports.

I also want to thank Petter Reinholdtsen, Roger Leigh, and everyone else
who has worked on the sysvinit package over the years, the insserv
conversion to dependency-based boot, and the inclusion of LSB support.  If
it weren't for their hard work over the years, we would be in a far worse
position than we are today.  It's often hard to see people discussing the
inadequacies of something into which you put years of hard work.  I want
to call attention to their long-term contributions to the distribution and
the number of Debian systems that have booted through their efforts
through the years.

I am carefully keeping this discussion to the Technical Committee bug
report so that all of my comments stay in context with their rebuttals,
but this one section I think is unrelated to the rest of the discussion
and should be more widely posted, so I will also be posting the above to
my blog and Planet Debian.


2. upstart vs. systemd: The Equivalencies

In doing this evaluation, I looked at quite a few different things.  In a
great number of cases, I found both upstart and systemd to be at an
equally high level of quality.  There are too many of those to list here,
but I wanted to mention a few points that had been the topic of wider
discussion.

* The documentation of both systems was uniformly excellent.  Both systems
  had complete reference manuals plus guidance to packagers for how to
  integrate with them.  systemd also had excellent guidance for upstream
  developers on how to add systemd support relevant to upstream packages.

* Both packages preserve traditional logging behavior, continue to
  properly utilize syslog, and retain logs in the places where I expected
  them.  This is obviously not surprising for upstart, but I wanted to
  mention it explicitly since those unfamiliar with the systemd journal
  may be afraid that it would migrate logs into a separate, unexpected
  store.  This is not the case; systemd logs are available in the normal
  places and can continue to be managed through syslog configuration.  The
  journal is supplementary and enables some additional features discussed
  elsewhere.

* Both packages move to configuration, from code, many of the annoyances
  and fiddly pieces involved in starting daemons.  Both support
  non-forking daemon models and proper PID tracking in their own ways.
  Both directly support such routine needs as setting user and group, OOM
  score adjustment, resource limits, and so forth.

* Both packages provide MAC integration, which is something that I think
  will become more important for Debian in the future.  (systemd provides
  some additional features around Smack, but at least given what I know
  right now, I don't think this is a significant differentiation.)

There was another point that I am calling equivalent since I didn't
evaluate it either way.  It's possible that this would convert into an
advantage for one side or the other after further investigation:

* Both systemd and upstart provide a way to run the system as a user
  process to manage user jobs in a similar fashion to how they manage
  system jobs.  I believe this is alr

Bug#727708: init system other points, and conclusion [and 1 more messages]

On Mon, Dec 30, 2013 at 1:35 PM, Tollef Fog Heen  wrote:
]] cameron 

 On Mon, Dec 30, 2013 at 9:43 AM, Tollef Fog Heen  
wrote:

 >>  If this is not required by systemd, why is it done by sd_notify 
?

 >>
 > It's not.

 You obviously did not read the code. It is. Here is a G+ convo with

 Lennart I had:

 > As a sender you only have to set SCM_CREDENTIALS manually if you
 > want to fake it (for which you need privs however).

 sd_notify() basically impersonates the process. You only need to set

 SCM manually if you are writing an external library. If someone is
 just doing it in the daemon, the kernel will set SCM_CREDENTIALS
 automatically.

You seem to be confusing systemd-notify(1) with sd_notify(3).
sd_notify(3) is the library call that's called by the daemon itself.
systemd-notify(1) is a command line tool to «Notify service manager
about start-up completion and other daemon status changes».

I am not confused, but I am wrong. sd_notify() does not use 
SCM_CREDENTIALS explicitly (only implicitly via the kernel's 
autosetting of them).

Sorry for the misrepresentation,
Cameron Norman

Bug#727708: loose ends for init system decision

]] Russ Allbery 

> Given that, I don't believe a Technical Committee choice of a default init
> system is going to make either the systemd or the upstart maintainers want
> to stop maintaining their packages.

Given what you're basically deciding between is «upstart + castrated
systemd» or «systemd» and I think I've pretty clearly expressed my
thoughts on splitting up systemd, I don't know where that conclusion
comes from.  Were you to choose upstart, I'd be seriously thinking about
stopping maintaining systemd.  (This is meant as a data point, not as a
threat or anything like that.)  My ideas for the package align pretty
well with upstream (which I'm a sometimes-active part of) where the
different services are free to assume that they run with systemd as pid
1. Continously porting new upstream versions to an environment
unsupported by upstream is not what I consider reasonable or fun.  I'm
speaking for myself here, not the other systemd maintainers.

In addition, as per my other message, were you to choose upstart as a
default, I'd be inclined to remove the init functionality from systemd.
I think trying to support multiple init systems is crazy and a recipe
for disaster and I don't want to do that.  (Doing it as part of a
transition is a necessary pain, that much I can agree with, but ongoing?
No thanks.)

> 6. Debian's non-Linux ports should either use the same init system as
>Debian's Linux ports or agree on an init system that they're both going
>to use.  The porting work is going to be hard enough without the ports
>going in different directions on which secondary init system they want
>to use.  I prefer to leave it up to the porters to decide which init
>system to choose, but I do think OpenRC would be a strong contender.

I think allowing them to use a compatible init system should be ok too,
if somebody wants to do that.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/m2r48uhub0@rahvafeir.err.no

Bug#727708: loose ends for init system decision

On Mon, Dec 30, 2013 at 1:47 PM, Russ Allbery wrote:
>
> 4. Conclusions
>
> I previously argued that much of the benefit of a new init system comes
> from when we can stop maintaining init scripts.  I still believe that, but
> after thinking more about the cultural and project issues at stake here,
> as well as the immediate needs for a clean upgrade path, I ended up at
> more of a compromise position than I expected.
>
> I believe Debian should take this path forward:
>
> 1. We should select a new init system for jessie, either systemd or
>upstart.  Support for that init system should be release-critical, but
>only in the sense that the daemon works properly under that init
>system.  In other words, use of the sysvinit compatibility of either
>init system is acceptable support for jessie.
>
> 2. All packages providing init scripts must continue to support sysvinit
>scripts through the jessie release.  Such support will continue to be
>release-critical.  This is going to be painful for packages that want
>to do an early conversion, since it means testing two different init
>systems for this release cycle, but I think this is the right thing to
>do regardless for a clean upgrade path and Debian's normal robust
>committment to upgrades.  Here it has the additional advantage of
>giving the non-Linux ports some breathing space to strategize.
>
> 3. Related, up through the jessie release, packages must (where possible;
>it's possible there will be cases where this is simply impossible)
>support switching back and forth between the new default init system
>and sysvinit.  This means that configurations should not be moved out
>of /etc/default and that startup files for the new init system should
>read the same configuration that the existing sysvinit scripts use (or
>both should be modified compatibly).
>
> 4. Post-jessie, support for sysvinit will no longer be release-critical,
>and package maintainers will no longer be expected to ensure that it
>continues working.  However, for as long as Debian has accepted
>non-Linux ports using a different init system, package maintainers
>should continue to ship init scripts if they work and should apply
>patches and other reasonable fixes from porters for those init scripts.
>In other words, this should be treated the same as merging patches for
>the Hurd to remove hard-coded constant assumptions: if the change is
>reasonable and doesn't break Linux ports (and this should be fairly
>easy to handle for nearly all cases with init scripts), the package
>maintainer should merge it.
>
> 5. Support for the other init system that was not chosen should be handled
>in the same fashion, should a team choose to pursue it.  If we select
>systemd, package maintainers should still be willing to merge
>contributed upstart configuration, with the understanding that they
>can't test it and any support is on a best-effort basis only.
>Similarly, if we select upstart, package maintainers should be willing
>to merge systemd unit files and to enable upstream systemd support
>where requested and where it doesn't interfere with the operation of
>the daemon under upstart, with the understanding that the package
>maintainer is not committing to testing or directly supporting this
>configuration.
>
> 6. Debian's non-Linux ports should either use the same init system as
>Debian's Linux ports or agree on an init system that they're both going
>to use.  The porting work is going to be hard enough without the ports
>going in different directions on which secondary init system they want
>to use.  I prefer to leave it up to the porters to decide which init
>system to choose, but I do think OpenRC would be a strong contender.
>
> 7. After jessie, functionality between systems running the primary Linux
>init system and other init systems (including non-Linux ports) should
>be allowed to drift.  In other words, there will be cases where
>features will only be available with the primary init system.  Possible
>examples include security hardening, socket activation, automatic
>daemon restarts, and so forth.  Packagers are under no obligation to
>port those features to other init systems, but should welcome and merge
>patches that do so.  After jessie, packagers will no longer be required
>to preserve daemon configuration when the init system is switched, so
>use of such facilities as modification of upstart configuration files
>or systemd overrides may be used.
>
> We should revisit this decision again after the jessie release in case the
> situation has substantially changed.

Doesn't a TC mandate on the default init system in some sense violate
Debian's spirit of meritocracy?  May I suggest something like the
following (this isn't meant to be definitive) as a more meritocratic
approach:

1. The TC enco

Bug#727708: loose ends for init system decision

Tollef Fog Heen  writes:
> ]] Russ Allbery 

>> Given that, I don't believe a Technical Committee choice of a default
>> init system is going to make either the systemd or the upstart
>> maintainers want to stop maintaining their packages.

> Given what you're basically deciding between is «upstart + castrated
> systemd» or «systemd» and I think I've pretty clearly expressed my
> thoughts on splitting up systemd, I don't know where that conclusion
> comes from.

I have to admit that I didn't give it a whole lot of thought.  It was an
assumption based on the presence of both in the archive for some time now
as non-default init systems under sysvinit.  In that sense, things don't
change that horribly much, at least up to the jessie release, if the
default changes from one non-systemd thing to another non-systemd thing.

That being said, obviously you should speak for yourself, and I stand
corrected.  Apologies for misprepresenting your feelings here.

>> 6. Debian's non-Linux ports should either use the same init system as
>>Debian's Linux ports or agree on an init system that they're both going
>>to use.  The porting work is going to be hard enough without the ports
>>going in different directions on which secondary init system they want
>>to use.  I prefer to leave it up to the porters to decide which init
>>system to choose, but I do think OpenRC would be a strong contender.

> I think allowing them to use a compatible init system should be ok too,
> if somebody wants to do that.

Oh, yes, good point.  I was thinking more in terms of two different init
systems with different preferred configuration files.

-- 
Russ Allbery (r...@debian.org)   

--
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ha9qrmns@windlord.stanford.edu

Bug#727708: loose ends for init system decision

Michael Gilbert  writes:

> Doesn't a TC mandate on the default init system in some sense violate
> Debian's spirit of meritocracy?

I believe that we have enough information to make an informed choice
already, and that the sides are fairly well-defined and hardened in their
opinions.  That means that this dispute falls under section 6.1.2 of the
constitution:

Decide any technical matter where Developers' jurisdictions overlap.

In cases where Developers need to implement compatible technical
policies or stances (for example, if they disagree about the
priorities of conflicting packages, or about ownership of a command
name, or about which package is responsible for a bug that both
maintainers agree is a bug, or about who should be the maintainer for
a package) the technical committee may decide the matter.

Regardless of how we structure the installer, we need to have a default
init system (unless we plan on making every user choose, which I would
dismiss out of hand as a horrible UI experience for the average user, who
really doesn't care).  We have to mandate support for at least that
default init system.  Realistically, the ability of the Debian developer
community to support more than one init system is limited, since any given
system is generally only going to run one.  That means the level of
quality of integration is going to drop off significantly relative to the
default init system, particularly over time.

Init systems are not like desktop environments: they require work by a
huge swath of the developer community, and the average user does not
generally switch from one to the other.  The reality is that either
systemd or upstart is fully capable of everything the typical user is
going to need (for that matter, sysvinit is capable of most of what the
typical user needs); there isn't the same driving force of user preference
that leads users to switch between desktop environments, and quite a bit
more integration support is required for the init system.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/878uv2rlz0@windlord.stanford.edu

Bug#727708: init system other points, and conclusion

2013-12-30 Thread intrigeri

Hi,

(Sorry if I am duplicating a point that was already made.
These threads are huge, and don't fit entirely into my memory.)

Ian Jackson wrote (30 Dec 2013 18:58:37 GMT) :
> Russ Allbery writes ("Bug#727708: init system other points, and conclusion"):
>> Rather, we're talking about whether or not to swap out a core component of
>> an existing integrated ecosystem with a component that we like better.

> Unless you are proposing to make systemd mandatory for all Debian
> installations, this is work that needs to be done anyway.

"Needs to be done anyway", possibly, but I find it important to make
it clear that, depending on what decision is made, it affects the
project as a whole, and many of its members, in very different ways:

* In one case (upstart is chosen as the default init system), we, as
  a project, are committed to do this work, at the very least because
  Policy mandates it, and we want to release without too many
  RC-buggy packages.

  Some maintainers happily do it because they are glad that upstart
  was picked, some do it reluctantly as they would have preferred
  systemd and they feel this all is unnecessary work put on our
  collective shoulders, some would have preferred systemd but console
  themselves because it feels so good to move away from sysvinit
  eventually, etc.

  Regardless of what our personal preference is, we have to do this
  work, because else it's a RC bug and we can't release.

* In the other case (systemd is chosen as the default init system),
  any individual or self-organized team may tackle this work, if their
  desires or needs make them feel committed to provide choice and
  flexibility, for the init component and potentially the kernel too,
  to Debian users.

  I believe this effort is similar in many ways to porting, and as
  such I trust it will be treated with "deference and reasonable
  accommodation" (thanks, Russ) in our community.

The difference lies in who are the people who "need" to do this work
"anyway", and who else may instead dedicate their time to other tasks,
lead by their own desires and needs.

[Now moving away from my clarification point, and largely off-topic.
Feel free to ignore.]

This specific part of the broader init system discussion boils down to
"what are our collective goals and priorities?", or "what do we find
important enough to put much energy into?". We are currently not that
good at finding exciting collective answers to this, and even at
finding good ways to make such decisions.

I mean: if it were just me, my proposal would be "let's make Debian
better empower its users to protect their privacy in the post-Snowden
world", and to me it feels way more exciting and relevant a thing we
could do for our users and Free Software than "let's allow Debian
users to replace systemd's init component with something else".
How far off-topic my answer is shows, I believe, how hard it is to
focus on the mere technical decision that presently needs to be made
(and rightfully was sent to the TC), given how broad its non-technical
implications could be not only on the Debian project, Debian users,
Debian derivatives and Free Software, but on the world at large.

End of the digression explaining how hard it is not to digress.

Cheers,
-- 
  intrigeri
  | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
  | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/85fvpa3px2@boum.org

Bug#727708: loose ends for init system decision

2013-12-30 Thread Cory


On 12/30/2013 04:31 PM, Michael Gilbert wrote:

On Mon, Dec 30, 2013 at 1:47 PM, Russ Allbery wrote:

4. Conclusions

I previously argued that much of the benefit of a new init system comes
from when we can stop maintaining init scripts.  I still believe that, but
after thinking more about the cultural and project issues at stake here,
as well as the immediate needs for a clean upgrade path, I ended up at
more of a compromise position than I expected.

I believe Debian should take this path forward:

1. We should select a new init system for jessie, either systemd or
upstart.  Support for that init system should be release-critical, but
only in the sense that the daemon works properly under that init
system.  In other words, use of the sysvinit compatibility of either
init system is acceptable support for jessie.

2. All packages providing init scripts must continue to support sysvinit
scripts through the jessie release.  Such support will continue to be
release-critical.  This is going to be painful for packages that want
to do an early conversion, since it means testing two different init
systems for this release cycle, but I think this is the right thing to
do regardless for a clean upgrade path and Debian's normal robust
committment to upgrades.  Here it has the additional advantage of
giving the non-Linux ports some breathing space to strategize.

3. Related, up through the jessie release, packages must (where possible;
it's possible there will be cases where this is simply impossible)
support switching back and forth between the new default init system
and sysvinit.  This means that configurations should not be moved out
of /etc/default and that startup files for the new init system should
read the same configuration that the existing sysvinit scripts use (or
both should be modified compatibly).

4. Post-jessie, support for sysvinit will no longer be release-critical,
and package maintainers will no longer be expected to ensure that it
continues working.  However, for as long as Debian has accepted
non-Linux ports using a different init system, package maintainers
should continue to ship init scripts if they work and should apply
patches and other reasonable fixes from porters for those init scripts.
In other words, this should be treated the same as merging patches for
the Hurd to remove hard-coded constant assumptions: if the change is
reasonable and doesn't break Linux ports (and this should be fairly
easy to handle for nearly all cases with init scripts), the package
maintainer should merge it.

5. Support for the other init system that was not chosen should be handled
in the same fashion, should a team choose to pursue it.  If we select
systemd, package maintainers should still be willing to merge
contributed upstart configuration, with the understanding that they
can't test it and any support is on a best-effort basis only.
Similarly, if we select upstart, package maintainers should be willing
to merge systemd unit files and to enable upstream systemd support
where requested and where it doesn't interfere with the operation of
the daemon under upstart, with the understanding that the package
maintainer is not committing to testing or directly supporting this
configuration.

6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use.  The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use.  I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.

7. After jessie, functionality between systems running the primary Linux
init system and other init systems (including non-Linux ports) should
be allowed to drift.  In other words, there will be cases where
features will only be available with the primary init system.  Possible
examples include security hardening, socket activation, automatic
daemon restarts, and so forth.  Packagers are under no obligation to
port those features to other init systems, but should welcome and merge
patches that do so.  After jessie, packagers will no longer be required
to preserve daemon configuration when the init system is switched, so
use of such facilities as modification of upstart configuration files
or systemd overrides may be used.

We should revisit this decision again after the jessie release in case the
situation has substantially changed.

Doesn't a TC mandate on the default init system in some sense violate
Debian's spirit of meritocracy?  May I suggest something like the
following (this isn't meant to be definitive) as a more meritocratic
approach:

1. The TC encourages a team (probably debian-boot) to provid

Bug#727708: init system other points, and conclusion

On Mon, Dec 30, 2013 at 11:56:33AM -0800, Russ Allbery wrote:
> >> Rather, we're talking about whether or not to swap out a core component
> >> of an existing integrated ecosystem with a component that we like
> >> better.

> > Unless you are proposing to make systemd mandatory for all Debian
> > installations, this is work that needs to be done anyway.

> What I'm hearing from both GNOME and KDE maintainers is that assuming
> systemd would save them a great deal of work.  I think having systemd be
> the only supported and tested option for at least some large package sets
> that heavily use the systemd infrastructure upstream is a not-unlikely
> long-term outcome.

It's clear that being able to assume availability of certain dbus services
is labor-saving for GNOME and KDE upstreams, and I see no reason for them
not to standardize on such a requirement assuming the dbus services have
sane APIs (which they do).

What I object to is referring to this as "assuming systemd".  They are
systemd APIs, sure, but with one exception the systemd implementations are
not presently tied to the use of systemd as init, and there is an alternate
implementation of that one exception (systemd-shim).

From comments made by various GNOME upstream developers on this, I think
they are being suitably cautious about avoiding scope creep where the
systemd dependencies are concerned.  So in what sense are the GNOME and KDE
requirements not already being met on top of upstart?  The only outstanding
issue I see is with non-Linux ports, because logind is not portable; that's
definitely a problem for running GNOME on kfreebsd, but it's actually a
rather narrowly-scoped porting problem and one that the ports need to deal
with one way or another if they want GNOME to continue working.

I'm sure there are GNOME upstream contributors who would be happy to see
GNOME become systemd-only.  But I think their motivations in letting the
lines be blurred in such a way are purely political, not technical; and I
give GNOME upstream as a whole the benefit of the doubt that they would not
so weaken their project to suit someone's political agenda.

> One can, of course, have a wide variety of reactions to that.  As someone
> who considers portability to be an inherent good, I find it sad, although
> I don't have a full appreciation for what problems GNOME is trying to
> solve by increasing its reliance on systemd.  But I'm highly dubious that
> Debian will just single-handedly fix that, or that we would drop GNOME
> because we don't like upstream's integration decisions.

I find the notion that Debian doing this "single-handedly" is an obstacle to
be a bizarre one.  Debian has more than one pair of hands, it's a veritable
multi-tentacled beast.  Have we so atrophied as a community that we'll wail
and gnash our teeth about a non-portable dependency, but have no porters
willing to actually provide a native implementation of a (quite small) dbus
API?

You've said that you think the portability question doesn't weight strongly
in favor of either upstart or systemd, because neither port is done yet. 
But the amount of work that would be involved in porting systemd to kfreebsd
is an order of magnitude greater than the work involved in porting upstart. 
logind is just one small component of systemd, which someone could provide
an API-compatible implementation for without having to contend with the
question of cgroups on non-Linux kernels.  If even that is out of reach for
the Debian porters, how could we ever expect a full BSD port of systemd to
materialize?

I think the reality is that adopting systemd as default init for Debian is
nothing short of a death sentence for the non-Linux ports.  The move to a
different init will have a snowball effect in the distribution, as packages
stop being tested with sysvinit and popular support grows for dropping
compatibility with sysvinit altogether.  This is not a problem if the ports
are in a position to come along on this transition with a moderate
investment in porting init.  But the porting required for systemd as a whole
is far from moderate, and I believe that faced with such a requirement the
non-Linux ports would wither and die.  I know there are Debian developers
(including many in the GNOME/systemd "camp") who think this is a desirable
outcome.  I have no personal attachment to the non-Linux ports, but I do
think that as an existing part of our Debian community, the TC should at
least give these ports a fighting chance, because this kind of competition
is healthy.

> > My understanding was that Dimitri had got upstart running on BSD.

> The latest that I have seen on this porting effort is here:

> http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html

> I asked previously on this bug if someone had later news.  Do you have
> more information than that?

> The above is definitely not "upstart running on BSD."  That's upstart's
> underlying support library mostly working on BSD after disabl

Bug#727708: init system other points, and conclusion

On Mon, Dec 30, 2013 at 01:44:10PM -0800, Russ Allbery wrote:
> * systemd provides really nice command-line tools for understanding the
>   state of the system and the relationships between the unit files.  I
>   don't believe upstart has an equivalent of systemctl list-dependencies,
>   for example.

I think the relevant tools on the upstart side are 'initctl show-config -e'
and 'initctl2dot'.  This doesn't render in a tree format the way 'systemctl
list-dependencies' does, because job relationships are a DAG, not a tree;
but I can see there being room for a simplified view of jobs that works from
a terminal.

Cheers,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: systemd-shim uploaded to NEW

On Mon, Dec 30, 2013 at 01:03:48PM -0800, Steve Langasek wrote:
> > This would be quite wrong; Replaces is not supposed to be used like
> > that (but you apparently know that).

> Yes.  Raphaël rightly points out that dpkg-divert can be used for this; if
> necessary, that's what I'll do.

> But I still think the right thing here is for the systemd package to be
> split.

Looking more closely, I find that one of the conflicting files is a conffile
(/etc/dbus-1/system.d/org.freedesktop.systemd1.conf).  diversions and
conffiles still don't mix, AFAIK (and according to current policy).  So that
seems to still leave us without a proper solution that doesn't involve
splitting the systemd binary package.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: init system other points, and conclusion

2013-12-30 Thread Uoti Urpala

On Mon, 2013-12-30 at 18:58 +, Ian Jackson wrote:
> Also, I get the impression me that the "integration" of much of this
> functionality into the systemd source package has been done for
> political rather than technical reasons.  Indeed to the extent that
> there is a problematically tight technical coupling between these
> components, I find it difficult to avoid doubting the good faith of
> the people making those decisions.  At the very least, I worry that
> the systemd project as a whole is far too willing to impose decisions
> of all kinds on its downstreams.

Your own expressed preference for upstart appeared to be very much
driven by political rather than technical considerations. Using the same
terminology you do, would it not be entirely fair to say that your
decision to support upstart was made in bad faith?

> > 3.3. Project Momentum
> 
> I don't see the outlook here the same way as you do.
> 
> Furthermore, I am much less worried about Debian going it alone
> (although, of course, it's not alone) than you seem to be.  We have
> historically been entirely unafraid to do our own better things, even
> if it is more work and takes us longer.
> 
> I felt that way when Debian was very much a minority interest.  That's
> far from the case nowadays; I've heard it asserted that Debian-based
> distros now account for a majority of traditional distro installs.  I
> guess numbers on that are all speculative but it is certainly true
> that we have a thriving ecosystem of our own.
> 
> We have got where we are by doing things the way we think is best, not
> by simply following the herd.

Who would actually do the work? Getting the amount of development there
is for systemd is not easy. Do you really believe that a Debian decision
in favor of upstart would create that many interested developers working
on it, when that has not happened while it's been used in Ubuntu?

Working on things that they believe to be better than existing ones does
motivate people. But how many Debian developers actually share your
extreme views about portability to the extent that they would be happy
to work on another system motivated by that, when systemd already works
better on Linux? I doubt that group is large enough to create
significant momentum.

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1388447089.3938.340.camel@glyph.nonexistent.invalid

Bug#727708: init system other points, and conclusion

Steve Langasek  writes:

> From comments made by various GNOME upstream developers on this, I think
> they are being suitably cautious about avoiding scope creep where the
> systemd dependencies are concerned.  So in what sense are the GNOME and
> KDE requirements not already being met on top of upstart?  The only
> outstanding issue I see is with non-Linux ports, because logind is not
> portable; that's definitely a problem for running GNOME on kfreebsd, but
> it's actually a rather narrowly-scoped porting problem and one that the
> ports need to deal with one way or another if they want GNOME to
> continue working.

I don't see what you're saying here as substantively different than what I
said in my writeup about the ecosystem.  I feel like we're both presenting
the same facts through different lenses.

> On Mon, Dec 30, 2013 at 11:56:33AM -0800, Russ Allbery wrote:

>> One can, of course, have a wide variety of reactions to that.  As
>> someone who considers portability to be an inherent good, I find it
>> sad, although I don't have a full appreciation for what problems GNOME
>> is trying to solve by increasing its reliance on systemd.  But I'm
>> highly dubious that Debian will just single-handedly fix that, or that
>> we would drop GNOME because we don't like upstream's integration
>> decisions.

> I find the notion that Debian doing this "single-handedly" is an
> obstacle to be a bizarre one.  Debian has more than one pair of hands,
> it's a veritable multi-tentacled beast.  Have we so atrophied as a
> community that we'll wail and gnash our teeth about a non-portable
> dependency, but have no porters willing to actually provide a native
> implementation of a (quite small) dbus API?

Please recall the context here: this whole aside started with an objection
to my contention that adopting upstart requires disassembly and redoing of
an integration that we would otherwise not have to disassemble.  Nowhere
in my message did I say that we would or could not do that disassembly if
we adopted upstart; I said that it was work that we otherwise wouldn't
have to do.

That's the intended context of my point above: I don't think we're going
to port GNOME to a non-systemd infrastructure, in the sense of carrying
significant patches to GNOME to adopt it to, say, not using logind.  I
think GNOME will continue to use systemd APIs heavily, which makes GNOME
less portable.  That means that systems that are not capable of running
those systemd components will need to either port them or develop
alternatives.

I don't consider this wailing or gnashing of teeth, but rather a realistic
look at what efforts the project is talking about committing to, as
opposed to supporting people working on if they so choose.

> You've said that you think the portability question doesn't weight
> strongly in favor of either upstart or systemd, because neither port is
> done yet.  But the amount of work that would be involved in porting
> systemd to kfreebsd is an order of magnitude greater than the work
> involved in porting upstart.

I haven't investigated this myself, but that matches the impressions I've
gotten from discussion both here and in Lennart's blog.  Nothing I'm
writing here is intended as disagreement with this point.  Porting systemd
looks harder.

> logind is just one small component of systemd, which someone could
> provide an API-compatible implementation for without having to contend
> with the question of cgroups on non-Linux kernels.

Yes, as I said explicitly in my writeup, it may well be the case that
porting some of the components of systemd will be substantially easier
than porting the init system.

> If even that is out of reach for the Debian porters, how could we ever
> expect a full BSD port of systemd to materialize?

I never said that a port of logind was out of reach for Debian porters.
I'm saying that there's a lot of work here, and we don't yet know whether
or how any of it is going to happen.  We should not close off possible
future directions unnecesarily, but we need to make a decision based on
the current state without assuming that roadmaps will necessarily come to
fruition.

*If* Debian adopts systemd as an init system, I do think that it's
possible (how likely, I don't know) we'll end up with GNOME unavailable on
the kFreeBSD port because people choose not to do the effort of separating
out and porting the components required.  It's up to the porters, and
depends on what role they see for the port.  If, for example, they're
primarily targeting servers, this may not be a significant loss.  The
point is that it would be their call.

If Debian adopts upstart, obviously we're going to be required to do the
work of separating the init system from the rest of systemd, which will
make some of that effort possibly easier for the kFreeBSD porters
(although that work will not magically port logind to kFreeBSD, or adjust
for any future requirements for D-Bus that might materialize, etc.).

Either way, real

Bug#727708: loose ends for init system decision

On Mon, Dec 30, 2013 at 5:51 PM, Russ Allbery wrote:
> Michael Gilbert writes:
>
>> Doesn't a TC mandate on the default init system in some sense violate
>> Debian's spirit of meritocracy?
>
> I believe that we have enough information to make an informed choice
> already, and that the sides are fairly well-defined and hardened in their
> opinions.  That means that this dispute falls under section 6.1.2 of the
> constitution:

I entirely concur that the social problem resides rightly within the
jurisdiction of the TC.  With that said, however, it is worth
considering whether the role of the TC may be more effective if
directed at the root (the social), rather than the branches (the
technical choice), of the problem.  The key, I think, is for the TC to
provide a reasonable path for those currently identifying with any of
the hardened camps to redirect their negative energy away from
argument and toward something more positive: technical work and actual
code.

> Regardless of how we structure the installer, we need to have a default
> init system (unless we plan on making every user choose, which I would
> dismiss out of hand as a horrible UI experience for the average user, who
> really doesn't care).

As stated in my suggestion, initsel would of course always have a
default, and only "expert" users would be empowered to travel the road
less traveled.  The no default idea that gets thrown about a lot is,
of course, infeasible as a matter of practicality.

> Init systems are not like desktop environments: they require work by a
> huge swath of the developer community, and the average user does not
> generally switch from one to the other.

I think, on the contrary, the nmu procedure is quite effective at
enabling a small subset of the developer community (those that have a
strong shared interest) to effect large changes across the entire
archive (of course when maintainers stay out of the way,).

This process has been demonstrated many times (although slow-going)
for lots of other sweeping changes (e.g. buildflags, usr/share/doc,
etc.).

Best wishes,
Mike

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=mmfcxqxnr2tkqmrz+fcyli-qrjxh9clp4wj7xebapd...@mail.gmail.com

Bug#727708: loose ends for init system decision

2013-12-30 Thread Vincent Bernat

 ❦ 30 décembre 2013 23:31 CET, Michael Gilbert  :

> Doing something like this, the best init system can win based truly on
> merit (if/when the work gets done), rather than as a fuzzy upfront
> judgement call.

Unfortunately, being the best init is the not only the matter of its
maintainers. A good integration implies to modify some packages whose
maintainer may be hostile to some init and may consider that their
package work well enough to not enable some feature needed by some init.

I am pretty vague, by I believe that the TC has currently a case for
such an example.

Only once (and if) systemd gets to be the default init, we will be able
to leverage all its abilities by full integration into Debian.
-- 
Don't compare floating point numbers just for equality.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Bug#727708: loose ends for init system decision

Michael Gilbert  writes:
> On Mon, Dec 30, 2013 at 5:51 PM, Russ Allbery wrote:

>> I believe that we have enough information to make an informed choice
>> already, and that the sides are fairly well-defined and hardened in
>> their opinions.  That means that this dispute falls under section 6.1.2
>> of the constitution:

> I entirely concur that the social problem resides rightly within the
> jurisdiction of the TC.  With that said, however, it is worth
> considering whether the role of the TC may be more effective if directed
> at the root (the social), rather than the branches (the technical
> choice), of the problem.  The key, I think, is for the TC to provide a
> reasonable path for those currently identifying with any of the hardened
> camps to redirect their negative energy away from argument and toward
> something more positive: technical work and actual code.

Well, I think it's worth pointing out that my transition plan looks
somewhat similar to your plan, as far as the jessie release.  (Then it
starts diverging.)  Part of my goal in writing up that plan was, as you
say, to try to provide a means for people who are committed to one system
or the other to continue to work on what they're passionate about even if
it's not chosen as the default init system.

Whether they do so or not is up to them, of course, as it should be.

However, I don't want to understate the amount of effort required to
integrate with the init system across the distribution.  I'm less
pessimistic than Steve, but he's not wrong that the choice of a default
init system will have sweeping consequences for what will work and what
won't.  This will particularly be the case if that init system supports
capabilities beyond the sysvinit set.

I do think it would be possible to maintain package compatibility with
both systemd and upstart.  That was something I was curious about and
therefore explicitly tested as part of my evaluation, and was able to do
so to my satisfaction.  That said, I tackled a fairly simple daemon, and
something like NFS support would require people with deep knowledge of
each supported init system to maintain that support.

I don't think it's a good idea to ask everyone to pursue all paths in
parallel.  I think Debian does a bit too much of that right now.  We
should pick a winner that we believe is technically superior and focus the
mandatory, archive-wide effort on it.  We should then *not get in the way*
of people who also want to pursue alternative paths, but not assume that
they will necessarily be successful, and not require that everyone get
involved in that effort beyond the level of integrating patches that we
currently expect for, say, the Hurd port.

But, anyway, I don't think our positions are really that different.  The
main difference is that I think we should pick a default init system for
jessie now, and you feel like we should do effectively an archive-wide
bake-off and then go with whatever one achieves the best integration.  I
have to admit to a deep personal dislike of "contests" like that, since I
find them stressful and demotivating and think they make the process of
free software development less fun.  I'd rather decide on our default and
on the mandatory work now, so that everyone knows where they stand, and
then let people make their own decisions about what they want to spend
time on beyond the required minimum.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87iou5rhv4@windlord.stanford.edu

Bug#727708: loose ends for init system decision

On Mon, Dec 30, 2013 at 7:16 PM, Vincent Bernat wrote:
>  ❦ 30 décembre 2013 23:31 CET, Michael Gilbert :
>
>> Doing something like this, the best init system can win based truly on
>> merit (if/when the work gets done), rather than as a fuzzy upfront
>> judgement call.
>
> Unfortunately, being the best init is the not only the matter of its
> maintainers. A good integration implies to modify some packages whose
> maintainer may be hostile to some init and may consider that their
> package work well enough to not enable some feature needed by some init.

That is a hypothetical, of course, but it is a very real possibility
for any path that the TC decides here.  However, selectable inits may
(possibly counter-intuitively) reduce this likely-hood.  Since that
maintainer in the way also gets his init of choice, he may be less
likely to oppose an alternative that doesn't in any real sense get in
his way.

However, if/when this does happen, it will be an interesting test of
whether the TC has the political will to override an indignant
maintainer with their 3-to-1 majority power.

Best wishes,
Mike

--
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=mo6ime0tljui9ynzgze+wv58kua7xrhqm+eedme3ba...@mail.gmail.com

Bug#727708: init system other points, and conclusion

On Mon, Dec 30, 2013 at 3:30 PM, Steve Langasek  
wrote:

On Mon, Dec 30, 2013 at 11:56:33AM -0800, Russ Allbery wrote:
 >> Rather, we're talking about whether or not to swap out a core 
component

 >> of an existing integrated ecosystem with a component that we like
 >> better.

 > Unless you are proposing to make systemd mandatory for all Debian
 > installations, this is work that needs to be done anyway.

 What I'm hearing from both GNOME and KDE maintainers is that 
assuming
 systemd would save them a great deal of work.  I think having 
systemd be
 the only supported and tested option for at least some large 
package sets
 that heavily use the systemd infrastructure upstream is a 
not-unlikely

 long-term outcome.

It's clear that being able to assume availability of certain dbus 
services
is labor-saving for GNOME and KDE upstreams, and I see no reason for 
them
not to standardize on such a requirement assuming the dbus services 
have

sane APIs (which they do).

What I object to is referring to this as "assuming systemd".  They are
systemd APIs, sure, but with one exception the systemd 
implementations are
not presently tied to the use of systemd as init, and there is an 
alternate

implementation of that one exception (systemd-shim).

From comments made by various GNOME upstream developers on this, I 
think

they are being suitably cautious about avoiding scope creep where the
systemd dependencies are concerned.  So in what sense are the GNOME 
and KDE
requirements not already being met on top of upstart?  The only 
outstanding
issue I see is with non-Linux ports, because logind is not portable; 
that's
definitely a problem for running GNOME on kfreebsd, but it's actually 
a
rather narrowly-scoped porting problem and one that the ports need to 
deal

with one way or another if they want GNOME to continue working.

I'm sure there are GNOME upstream contributors who would be happy to 
see
GNOME become systemd-only.  But I think their motivations in letting 
the
lines be blurred in such a way are purely political, not technical; 
and I
give GNOME upstream as a whole the benefit of the doubt that they 
would not

so weaken their project to suit someone's political agenda.

 One can, of course, have a wide variety of reactions to that.  As 
someone
 who considers portability to be an inherent good, I find it sad, 
although
 I don't have a full appreciation for what problems GNOME is trying 
to
 solve by increasing its reliance on systemd.  But I'm highly 
dubious that
 Debian will just single-handedly fix that, or that we would drop 
GNOME

 because we don't like upstream's integration decisions.

I find the notion that Debian doing this "single-handedly" is an 
obstacle to
be a bizarre one.  Debian has more than one pair of hands, it's a 
veritable
multi-tentacled beast.  Have we so atrophied as a community that 
we'll wail
and gnash our teeth about a non-portable dependency, but have no 
porters
willing to actually provide a native implementation of a (quite 
small) dbus

API?

You've said that you think the portability question doesn't weight 
strongly
in favor of either upstart or systemd, because neither port is done 
yet. 
But the amount of work that would be involved in porting systemd to 
kfreebsd
is an order of magnitude greater than the work involved in porting 
upstart. 
logind is just one small component of systemd, which someone could 
provide
an API-compatible implementation for without having to contend with 
the
question of cgroups on non-Linux kernels.  If even that is out of 
reach for
the Debian porters, how could we ever expect a full BSD port of 
systemd to

materialize?

systemd lists logind as non-reimplementable, and that was pretty much 
proven when Ubuntu tried to reimplement it and ended up reimplementing 
or pulling in a ton of systemd anyway. Seeing that, both KWin and 
Mutter have denounced portability to non-Linux when running on Wayland. 
They will soon be outright non-portable (when ConsoleKit support is 
dropped, which, AFAIK, is soon in GNOME).

I think the reality is that adopting systemd as default init for 
Debian is
nothing short of a death sentence for the non-Linux ports.  The move 
to a
different init will have a snowball effect in the distribution, as 
packages

stop being tested with sysvinit and popular support grows for dropping
compatibility with sysvinit altogether.  This is not a problem if the 
ports

are in a position to come along on this transition with a moderate
investment in porting init.  But the porting required for systemd as 
a whole
is far from moderate, and I believe that faced with such a 
requirement the
non-Linux ports would wither and die.  I know there are Debian 
developers
(including many in the GNOME/systemd "camp") who think this is a 
desirable
outcome.  I have no personal attachment to the non-Linux ports, but I 
do
think that as an existing part of our Debian community, the TC should 
at
least give these ports a fighting

Bug#727708: init system other points, and conclusion

On Tue, Dec 31, 2013 at 12:27:28AM -0008, cameron wrote:

> systemd lists logind as non-reimplementable, and that was pretty
> much proven when Ubuntu tried to reimplement it and ended up
> reimplementing or pulling in a ton of systemd anyway.

All this proves is that Ubuntu developers have the good sense to not
reinvent the wheel unnecessarily.  No one ever tried to reimplement logind
for Ubuntu, at all.  Why should they, when logind is already a perfectly
usable implementation of logind?

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: loose ends for init system decision

On Mon, Dec 30, 2013 at 7:20 PM, Russ Allbery wrote:
> Michael Gilbert  writes:
>> On Mon, Dec 30, 2013 at 5:51 PM, Russ Allbery wrote:
>
>>> I believe that we have enough information to make an informed choice
>>> already, and that the sides are fairly well-defined and hardened in
>>> their opinions.  That means that this dispute falls under section 6.1.2
>>> of the constitution:
>
>> I entirely concur that the social problem resides rightly within the
>> jurisdiction of the TC.  With that said, however, it is worth
>> considering whether the role of the TC may be more effective if directed
>> at the root (the social), rather than the branches (the technical
>> choice), of the problem.  The key, I think, is for the TC to provide a
>> reasonable path for those currently identifying with any of the hardened
>> camps to redirect their negative energy away from argument and toward
>> something more positive: technical work and actual code.
>
> Well, I think it's worth pointing out that my transition plan looks
> somewhat similar to your plan, as far as the jessie release.  (Then it
> starts diverging.)

The key differences are that it is more succinct, roles and
requirements are defined, no init system is outright rejected, and the
default is selected on demonstrable merit.

> Part of my goal in writing up that plan was, as you
> say, to try to provide a means for people who are committed to one system
> or the other to continue to work on what they're passionate about even if
> it's not chosen as the default init system.

Unfortunately at least two camps will be entirely dejected by any TC
mandate here.

> Whether they do so or not is up to them, of course, as it should be.
>
> However, I don't want to understate the amount of effort required to
> integrate with the init system across the distribution.  I'm less
> pessimistic than Steve, but he's not wrong that the choice of a default
> init system will have sweeping consequences for what will work and what
> won't.  This will particularly be the case if that init system supports
> capabilities beyond the sysvinit set.
>
> I do think it would be possible to maintain package compatibility with
> both systemd and upstart.  That was something I was curious about and
> therefore explicitly tested as part of my evaluation, and was able to do
> so to my satisfaction.  That said, I tackled a fairly simple daemon, and
> something like NFS support would require people with deep knowledge of
> each supported init system to maintain that support.
>
> I don't think it's a good idea to ask everyone to pursue all paths in
> parallel.  I think Debian does a bit too much of that right now.  We
> should pick a winner that we believe is technically superior and focus the
> mandatory, archive-wide effort on it.  We should then *not get in the way*
> of people who also want to pursue alternative paths, but not assume that
> they will necessarily be successful, and not require that everyone get
> involved in that effort beyond the level of integrating patches that we
> currently expect for, say, the Hurd port.

I don

> But, anyway, I don't think our positions are really that different.  The
> main difference is that I think we should pick a default init system for
> jessie now, and you feel like we should do effectively an archive-wide
> bake-off and then go with whatever one achieves the best integration.

And Debian ends up with not only apple pie, but pumpkin and blueberry
pies, and of in a corner there will be others thinking about
cinnamon-spiced apple and blueberry-raspberry and other
never-before-seen flavors.   What a wonderful bake-off that would be
:)

Think about how it would go over if the directors of that metaphorical
bake-off forced all the participants to produce the exact same pie.
No one would really want to participate, and winning would be entirely
meaningless.  It wouldn't be a bake-off at all.  It would be more like
a mass-production assembly line.

Best wishes,
Mike


-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=mnl4j+zmojjp5qxkufy3hglvcuq59yzb5uxgj+hene...@mail.gmail.com

Bug#727708: init system thoughts

2013-12-30 Thread Colin Watson

I see that the LWN commentariat already has my decision selected in
advance, so I might as well write up some more detailed thoughts before
any other words are put into my mouth!


Choice of default
-

Firstly, I've tried to keep my employer affiliation out of this as much
as possible, and I have come under no pressure from that quarter.  I'll
reiterate my previous comments on debian-devel:

  http://lists.debian.org/debian-devel/2013/10/msg00747.html
  http://lists.debian.org/debian-devel/2013/10/msg00777.html
  http://lists.debian.org/debian-devel/2013/10/msg00818.html

It's apparently no surprise that my preference for a default lies with
Upstart.  To the extent that I have a pre-existing bias it has more to
do with the fact that I have substantially more real-world deployment
experience with Upstart in an environment structurally very similar to
Debian (Ubuntu, obviously), which has led me to believe that it would be
a fine choice for deployment in Debian which could be put in place with
a minimum of disruption, and which would close as few doors as possible
to experimentation and innovation.

That said, I will add that I have been impressed with the technical
aspects of systemd that I have seen while working on adding support for
it to my packages by way of research for this debate.  Its documentation
is comprehensive, a great deal of work has clearly gone into the range
of options available in unit files, and it has admirable facilities
available for system administrators.  I am quite happy for it to be my
second choice (which is by no means irrelevant given our voting
protocol); I think it would be a more than worthwhile step forward from
sysvinit, just in a somewhat different direction that does not appeal
quite as much to me.


Reservations with systemd
-

My main concerns with systemd relate to its broad scope regarding units
it provides for system initialisation tasks currently performed by other
packages, and the potential for that to interfere with past and future
work elsewhere in Debian.  I can understand why the systemd authors felt
it valuable to expand its scope in this way, to provide a more
consistent experience across the board.  Nevertheless, it seems to
significantly complicate the job of integrating it with some excellent
features that already exist in Debian and which are superior to those
currently in most other distributions.  I realise that the Debian
systemd maintainers have generally expressed willingness to deal with
this kind of integration problem where appropriate, but I find the
impedance mismatch to be much higher than it is with Upstart which
leaves these tasks up to the distribution.

The two examples which I've run across so far, both ones I was inclined
to look at since I'm familiar with the territories, are:

  #716812 (binfmt-support)

(I'm the author and maintainer of binfmt-support.)

The discussion on this (archived earlier in #727708) does now seem
to have been resolved reasonably amicably so far; I've turned
binfmt-support into an independent upstream package hosted on
nongnu.org, given it a systemd unit file while I was there, and will
work on getting that into more distributions.

However, I maintain that this is really something that has no
particular reason to be integrated with the init system, that it has
much more interesting interactions with packaging than it does with
other system events, and that it is busywork to convert things away
from a system that works measurably better in Debian and derivatives
than (TTBOMK) any other GNU/Linux distribution.

  #608457 (console-setup)

(I've contributed to console-setup in some small ways, although I'm
not its primary maintainer by any stretch of the imagination.)

console-setup is a fantastic piece of innovation in Debian; instead
of shipping two entirely independent sets of keymap files for
virtual consoles and for X, the former of which is mostly unloved,
let's generate virtual console mappings dynamically from XKB!  It's
naturally not an easy task since the console is less capable in
various important ways, but it's demonstrably possible to do a
pretty accurate job.  Like binfmt-support, it probably ought to be
turned into an independent upstream package to make it easier for
other distributions to adopt, but in the meantime it's tremendously
useful for Debian.

This seems a bit conceptually tricky to integrate with systemd,
because localectl(1) exposes the console-data style of keymap naming
in its interface, and with console-setup you only use the X style.
The Debian systemd maintainers haven't yet offered a suggestion in
the bug, but perhaps they'll come up with something appropriate
(maybe just disabling set-keymap/list-keymaps, or converting both
ways and accepting the inevitable round-trip errors).  But my point
is that this is a clas

Bug#727708: init system other points, and conclusion

2013-12-30 Thread Colin Watson

On Mon, Dec 30, 2013 at 04:04:05PM -0800, Russ Allbery wrote:
> My belief, and again I welcome concrete reasons why I'm not correct, is
> that adopting upstart poses a similar risk for the Hurd port as adopting
> systemd, and I care just as much about the Hurd port as kFreeBSD.  And
> while kFreeBSD is in better shape due to the already-begun upstart port,
> there are still significant porting challenges in the way of maintaining
> feature parity in the kFreeBSD port at the level that it has today.  Many
> of those challenges are going to remain regardless of which init system we
> pick.

I haven't looked at this in much detail, and I suspect Dimitri hasn't
yet although IIRC he did express some interest in doing so.  But I
haven't seen anyone else try to outline the scope of a port, so let me
try to do so for the sake of general understanding.  As far as I know,
the hardest parts would be inotify, ptrace, and prctl
(PR_SET_CHILD_SUBREAPER).

inotify is used to notice changes to configuration files.  This is
certainly helpful for users, but it isn't critical as "initctl
reload-configuration" works without it.  We could probably do without
this with the aid of a dpkg trigger.

ptrace is used for "expect fork" and "expect daemon"; as I indicated in
another post, I think it would be preferable to avoid these in Debian
and quite possibly to compile them out.  (This would mean we wouldn't be
able to translate Ubuntu jobs quite as directly, and a number of
important jobs would definitely need to be changed, but the conversion
isn't usually particularly difficult.)

prctl (PR_SET_CHILD_SUBREAPER) is used to make SIGCHLD notification work
properly when Upstart is supervising a user session.  This isn't a
required feature and could easily be compiled out until suitable kernel
support is available (this actually seems like the sort of thing that
could be done in the Hurd without too much difficulty, but I haven't
looked into it).  If absent, it might well impede the ability to do an
advanced desktop port, but it wouldn't get in the way of porting the
bulk of services.

There might also be odds and ends around the details of wait status
handling.

So, I'll certainly concede that I haven't actually tried this, but from
what I know of Upstart/libnih's system dependencies, I think that the
Hurd could be accommodated with at worst possibly some loss of
non-critical features.

> I want to mention again something that you'd dismissed as possibly a joke
> earlier, but which I was actually serious about: I think a world in which
> we use systemd on Linux and upstart on non-Linux ports, should upstart
> prove much more portable, actually makes sense.  As I said in my other
> writeup, I believe the systemd feature advantage is sufficient to choose
> systemd in isolation, without the other issues discussed.  I also believe
> that maintaining a systemd unit plus an upstart configuration is, modulo
> testing (which I realize is a huge issue), much easier than maintaining a
> single sysvinit script.

I wouldn't discard this option out of hand, but I think it would need
significant tool support to be practical (e.g. heuristic generation of
Upstart job files from systemd units), unless we expect all service
maintainers to have kFreeBSD/Hurd VMs lying around.  Keeping the
sysvinit scripts and using Upstart as the init daemon is another
possibility, and at least in that case the sysvinit scripts are probably
still lying around.  We don't even necessarily have to choose between
those up-front.

-- 
Colin Watson   [cjwat...@debian.org]

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131231031739.ga24...@riva.ucam.org

Bug#727708: init system thoughts

Thanks for this write-up, Colin.  This was very interesting to me,
particularly in the concrete examples of where you've felt like systemd
has stepped into areas that will pose integration problems for us.  This
is something that I've seen referred to in the abstract frequently, but
without a lot of specifics that made sense to me.  Both of those examples
make sense to me.

Just a couple of specific comments

Colin Watson  writes:

> (Now, I've been working with Upstart's model for years, and it's now a
> pretty fundamental way of how I think of system operation; so if people
> who are new to *both* models rather than partisans of one side or the
> other consistently tell me that the systemd model is easier to grasp,
> then I'll listen.)

I am new to both models.  I'm not very fond of the upstart model.

Lennart had a comment on this point that I thought phrased the problem in
an interesting way: both systemd and upstart deal with the same data, but
with systemd, the full dependency tree is precalculated and exists as
state within the init system.  With upstart, behavior is dynamic and to
some degree emergent: you know who is listening to what signals, but you
can arbitrarily introduce signals, and you're dealing with a message bus
rather than a dependency tree, so you're reacting to things as they happen
on the bus.  In systemd, you can dump everything that can possibly happen
and work through the state transitions by hand if needed, with the
possible exception of unexpected device events (which are horribly
important in some cases -- don't get me wrong -- but which are
uninteresting in a lot of common debugging scenarios).  I think it's
somewhat harder to do that with upstart, where events are generated more
dynamically.

Basically, the way I'd put it is that I think the upstart event model is
the sort of elegant, minimal model that someone who has thought really
hard about the space comes up with, but which is hard to understand for
people who have *not* thought really hard about the space.  It's elegant
and clever; for me personally, I find it *too* clever.  I understand what
it's trying to do, and it has a certain grace, but I find a dependency
graph more robust and predictable, and even the levels of dependencies
make sense to me given packaging background and familiarity with the
Depends vs. Recommends scenarios.  I can build a relatively complete
mental map more easily than I can with the message bus approach.

I should probably note here that I'm a notorious skeptic about message
buses in general.  People I work with have probably gotten rather tired of
me going on about my feeling that message bus integrations are overly
complex to reason about, hard to guarantee correctness of, and are
overused in system design.  :)

> There is of course also the non-Linux porting issue, which Ian, Russ,
> and Steve have discussed at more length elsewhere, and I won't repeat it
> at length.  I will just add in response to Russ that there is a great
> difference between one project whose lead maintainer has explicitly said
> that he will reject patches for porting to non-Linux architectures
> because they fundamentally do not make sense with its architecture, and
> another project one of whose developers has been working on such a port
> with some degree of success so far.

This is a very valid point, and you're right, I probably understated that
in my writeup.  One of the things that I may turn out to be wrong about
(and would be happy to be wrong about, for the record) is the likelihood
of completing a working port of upstart to Hurd and kFreeBSD.  As I
mentioned but probably not explicitly enough, the existence of those ports
would, for me, turn the whole second part of my analysis into a dead heat
between systemd and upstart as opposed to a general advantage for upstart
in terms of ecosystem integration.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87sit9puow@windlord.stanford.edu

Bug#727708: loose ends for init system decision

On Mon, Dec 30, 2013 at 08:12:19PM -0500, Michael Gilbert wrote:
> > Part of my goal in writing up that plan was, as you
> > say, to try to provide a means for people who are committed to one system
> > or the other to continue to work on what they're passionate about even if
> > it's not chosen as the default init system.

> Unfortunately at least two camps will be entirely dejected by any TC
> mandate here.

I don't agree with that conclusion.

When it comes to technology choices, you win some and you lose some.  If
upstart wins, I will be happy.  If systemd wins, I will also be happy,
because it's long overdue that Debian *make a decision*; and for all that
there are aspects of systemd that make me uncomfortable, it will still be
far better than the status quo.

Your proposal smells like the status quo.  Namely, instead of the project
making a decision and being able to all pull together in the same direction
to provide the best possible OS, we will continue to coast, squandering
efforts on preserving users' ability to make choices about things that no
user should ever be asked to care about.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Re: Bug#727708: init system thoughts

Colin Watson wrote:
> (Now, I've been working with Upstart's model for years, and it's now a
> pretty fundamental way of how I think of system operation; so if people
> who are new to *both* models rather than partisans of one side or the
> other consistently tell me that the systemd model is easier to grasp,
> then I'll listen.)

I'd like to respond to this point, since I have a clear memory of
dealing with both upstart's model and systemd's model for the first
time.  I also work professionally with a system (ChromeOS) that uses
upstart, and I've dealt with systemd quite a bit as well.  For the sake
of full disclosure, my position is that I'd prefer systemd over upstart;
I'd be OK with upstart if and only if systemd remained a viable
alternative (in other words, packages maintaining both upstart jobs and
systemd units but not necessarily maintaining init scripts seems pretty
reasonable).  So, at this point I'm squarely in the systemd camp, but
not for any particular reasons of partisanship; I simply find the
technology and model better.

I started out dealing with the sysvinit model, pre-startpar; I dealt
quite frequently with the magic ordering numbers (S20, K80, etc).  My
first exposure to dependencies was through startpar and LSB, and they
made sense, though I was never a fan of the magic 'S' runlevel.  I
fairly quickly saw that mapping those dependencies to script ordering
numbers was a hack, albeit a nicely done hack for compatibility; using
the dependencies natively made a lot more sense, though.

I read about upstart when it was first announced, and I dove into the
event model quite extensively, thinking that it might help improve
things.  I never found it particularly intuitive, and in particular the
whole "start on starting", "start on stopping" etc model never really
made much sense to me; it didn't seem like a natural mapping of how the
system worked.  I also, from the beginning, disliked the model of
starting everything that was possible to start, though I did not at the
time see what the alternative would be; it simply never really seemed
like the right model for booting quickly.  (This was reinforced when I
started paying attention to boot time, inspired by the original "boot in
5 seconds" presentation; upstart's model seemed entirely unsuited for
that exercise.)  I also found the requirement to know how many times
your daemon forked quite obnoxious (even more so the failure mode if you
get it wrong on even one daemon).  These days, dealing with upstart
professionally, it still seems entirely too easy to get upstart confused
and in a difficult to recover state via a single incorrect upstart job,
and I find it one of the most annoying subsystems to deal with; there's
a general feeling of "oh, joy, that's probably an upstart issue" every
time any issue relating to booting comes up.

By contrast, my initial exposure to systemd (via the "systemd for system
administrators" blog series and the systemd documentation) was actually
via socket activation, with the dependency mechanism coming later.  The
socket activation model, as a replacement for dependencies, was one of
those rare ideas that seemed immediately obvious in hindsight.  My
introduction to systemd's dependencies was more along the lines of "and
explicit dependencies exist for when you can't fix your daemon and unit
file to do things the *right* way".  Similarly, the use of autofs and
other techniques to allow starting things in parallel even when
dependencies exist made immediate sense.  And the entire model,
including its handling of asynchronous events (notably the integration
with udev), simply felt like a closer fit to how the kernel works and
how reality works.  Finally, the concept of factoring out common
elements from daemons seemed quite nice, having dealt with quite a bit
of ugly boilerplate code.

In both cases, I had no particular reason to like or dislike either
model; in each case I very much *wanted* them to work out, since my
exposure to sysvinit versus startpar gave me a very early reason to want
something better than sysvinit.  I had no other reason to prefer one
model over the other.  I'm not attached to either init system, nor do I
have any particular bias or reason to favor one over the other for any
reason other than superior technology or a model I find clearer and more
natural.

So, in summary, upon initial introduction, I found systemd's model far
more intuitive, both innately (in its use of socket activation and
similar mechanisms to *avoid* dependencies) and as a more natural
mapping to how the system, kernel, and hardware work.

I hope this writeup of my first impressions of both systems proves
useful in some way.

- Josh Triplett

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131231032827.GA14382@leaf

Bug#727708: loose ends for init system decision

Steve Langasek  writes:

> When it comes to technology choices, you win some and you lose some.  If
> upstart wins, I will be happy.  If systemd wins, I will also be happy,
> because it's long overdue that Debian *make a decision*; and for all
> that there are aspects of systemd that make me uncomfortable, it will
> still be far better than the status quo.

I concur with this.

I freely admit that I fell in love with systemd while investigating both
systems, but if we agree to use upstart, I will happily go add upstart
configurations to all of my packages and make use of those features.  It's
still a massive improvement over what we have now.

Furthermore, and more philosophically, Debian has to learn how to respect
a governance process and make decisions.  We spent way too much time and
effort not making decisions in this project, which results in big
flamewars and discomfort and everyone sniping at each other while little
productive happens.  Frequently, I think this is like tearing off bandages
over the course of months.  The cumulative pain is much higher than if we
just made a decision and everyone knew where they stood and what reality
they needed to adjust to.

Also, I too am happy when we can successfully pursue all courses of action
at the same time, but at the end of the day, we're trying to create a
distribution, and that means integrating components, and that means
deciding on integration protocols.  It's not useful to try to integrate
everything with everything else in every possible way at the same time.
You just create an unmanageable combinatoric explosion of possibilities,
and then someone who just wants to run a Debian system doesn't know which
paths are supported and tested, and which paths have only been touched by
one developer if that.

We need to pick a winner.  That doesn't mean we need to pick losers.  It
does mean that one solution is going to get better support than everything
else, because that's what integration *means*.  It doesn't mean, or
shouldn't mean, that other solutions are impossible if someone wants to
make them work; it just means that we're not going to *require* that they
work in order to get one's software into Debian.

We probably should have had this discussion for wheezy, to be honest.  But
regardless, we should pick a default, supported init system for jessie.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87lhz1puac@windlord.stanford.edu

Bug#727708: init system other points, and conclusion

On Mon, Dec 30, 2013 at 04:04:05PM -0800, Russ Allbery wrote:
> Steve Langasek  writes:

> > From comments made by various GNOME upstream developers on this, I think
> > they are being suitably cautious about avoiding scope creep where the
> > systemd dependencies are concerned.  So in what sense are the GNOME and
> > KDE requirements not already being met on top of upstart?  The only
> > outstanding issue I see is with non-Linux ports, because logind is not
> > portable; that's definitely a problem for running GNOME on kfreebsd, but
> > it's actually a rather narrowly-scoped porting problem and one that the
> > ports need to deal with one way or another if they want GNOME to
> > continue working.

> I don't see what you're saying here as substantively different than what I
> said in my writeup about the ecosystem.  I feel like we're both presenting
> the same facts through different lenses.

I do see a substantive difference between "GNOME depends on systemd" and
"GNOME depends on interfaces a, b, and c, which are available as part of
systemd".

> >> One can, of course, have a wide variety of reactions to that.  As
> >> someone who considers portability to be an inherent good, I find it
> >> sad, although I don't have a full appreciation for what problems GNOME
> >> is trying to solve by increasing its reliance on systemd.  But I'm
> >> highly dubious that Debian will just single-handedly fix that, or that
> >> we would drop GNOME because we don't like upstream's integration
> >> decisions.

> > I find the notion that Debian doing this "single-handedly" is an
> > obstacle to be a bizarre one.  Debian has more than one pair of hands,
> > it's a veritable multi-tentacled beast.  Have we so atrophied as a
> > community that we'll wail and gnash our teeth about a non-portable
> > dependency, but have no porters willing to actually provide a native
> > implementation of a (quite small) dbus API?

> Please recall the context here: this whole aside started with an objection
> to my contention that adopting upstart requires disassembly and redoing of
> an integration that we would otherwise not have to disassemble.  Nowhere
> in my message did I say that we would or could not do that disassembly if
> we adopted upstart; I said that it was work that we otherwise wouldn't
> have to do.

> That's the intended context of my point above: I don't think we're going
> to port GNOME to a non-systemd infrastructure, in the sense of carrying
> significant patches to GNOME to adopt it to, say, not using logind.  I
> think GNOME will continue to use systemd APIs heavily, which makes GNOME
> less portable.  That means that systems that are not capable of running
> those systemd components will need to either port them or develop
> alternatives.

> I don't consider this wailing or gnashing of teeth, but rather a realistic
> look at what efforts the project is talking about committing to, as
> opposed to supporting people working on if they so choose.

Ok.  I think our core point of disagreement here, then, is in our assessment
of how much work we think this actually is (for the Linux case, not for the
non-Linux case).  I think the actual package maintenance to make this happen
is not even a weekend's worth of free time, and therefore represents a
negligible committment of resources on Debian's part, given that this
dissassembly/integration has already been done in Ubuntu.

> *If* Debian adopts systemd as an init system, I do think that it's
> possible (how likely, I don't know) we'll end up with GNOME unavailable on
> the kFreeBSD port because people choose not to do the effort of separating
> out and porting the components required.  It's up to the porters, and
> depends on what role they see for the port.  If, for example, they're
> primarily targeting servers, this may not be a significant loss.  The
> point is that it would be their call.

Right.  I think that if we adopt systemd, GNOME will be the least of the
kfreebsd porters' problems, because of the overarching init integration
questions.

> My belief, and again I welcome concrete reasons why I'm not correct, is
> that adopting upstart poses a similar risk for the Hurd port as adopting
> systemd, and I care just as much about the Hurd port as kFreeBSD.

Well, certainly there is risk that no one will be interested in doing the
work to port libnih+upstart to the Hurd.  Picking either upstart or systemd
will not guarantee that we have porters willing to do the work to keep up.
But I think the success of the in-progress kfreebsd port shows that upstart
poses a much lower portability barrier than systemd; if we want non-Linux
ports to exist on the same terms as the Linux ones (i.e., without needing
backwards-compatibility solution for sysvinit), I think upstart does give
them a much better fighting chance.

> I want to mention again something that you'd dismissed as possibly a joke
> earlier, but which I was actually serious about: I think a world in which
> we use systemd o

Bug#727708: init system thoughts

2013-12-30 Thread Uoti Urpala

On Tue, 2013-12-31 at 02:55 +, Colin Watson wrote:
> My main concerns with systemd relate to its broad scope regarding units
> it provides for system initialisation tasks currently performed by other
> packages, and the potential for that to interfere with past and future
> work elsewhere in Debian.  I can understand why the systemd authors felt

> The two examples which I've run across so far, both ones I was inclined
> to look at since I'm familiar with the territories, are:
> 
>   #716812 (binfmt-support)

>   #608457 (console-setup)

> Basically, systemd would be more compelling to me if it tried to do
> less.  I don't expect to persuade systemd advocates of this, as I think
> it amounts to different basic views of the world, but the basic approach
> here is probably the single biggest factor influencing my position.

I think this is absolutely ridiculous as a "rationale" for choosing
upstart. If you have done any investigation, you should know that it's
trivial to disable any of those components if Debian decides it's better
off without them. Yet you really seriously present their existence as
the "biggest factor influencing your position"? In what kind of scenario
could their existence possibly cause noticeable problems for Debian
systemd use?

I can imagine some kind of extrapolated arguments about "project scope
issues" that would not be completely laughable, but you haven't really
presented any of those. As is, what you're saying just does not form an
argument that could be taken seriously at all.

> The criticisms of Upstart's event model in the systemd position
> statement simply do not make sense to me.  Events model how things
> actually happen in reality; dependencies are artificial constructions on
> top of them, and making them work requires the plethora of different
> directives in systemd (e.g. Wants, which is sort of a non-depending
> dependency) to avoid blocking the boot process on a single failing
> service.

Dependencies are the natural way to express what people know about the
system and what they need. Events are the internal implementation
details of what the machine does to make it happen.

"I want task X started, and it needs task Y" is what people express.
"Start Y, and when Y is ready, start X" are the "compiled"
implementation instructions to achieve the higher-level goal.

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1388464464.3938.363.camel@glyph.nonexistent.invalid

Bug#727708: init system other points, and conclusion




On Mon, Dec 30, 2013 at 7:17 PM, Colin Watson  
wrote:

On Mon, Dec 30, 2013 at 04:04:05PM -0800, Russ Allbery wrote:
 My belief, and again I welcome concrete reasons why I'm not 
correct, is
 that adopting upstart poses a similar risk for the Hurd port as 
adopting
 systemd, and I care just as much about the Hurd port as kFreeBSD.  
And
 while kFreeBSD is in better shape due to the already-begun upstart 
port,
 there are still significant porting challenges in the way of 
maintaining
 feature parity in the kFreeBSD port at the level that it has today. 
 Many
 of those challenges are going to remain regardless of which init 
system we

 pick.


I haven't looked at this in much detail, and I suspect Dimitri hasn't
yet although IIRC he did express some interest in doing so.  But I
haven't seen anyone else try to outline the scope of a port, so let me
try to do so for the sake of general understanding.  As far as I know,
the hardest parts would be inotify, ptrace, and prctl
(PR_SET_CHILD_SUBREAPER).

inotify is used to notice changes to configuration files.  This is
certainly helpful for users, but it isn't critical as "initctl
reload-configuration" works without it.  We could probably do without
this with the aid of a dpkg trigger.



inotify use can easily be ported to kqueue within Upstart, or 
libinotify-kqueue can be used.




ptrace is used for "expect fork" and "expect daemon"; as I indicated 
in

another post, I think it would be preferable to avoid these in Debian
and quite possibly to compile them out.  (This would mean we wouldn't 
be

able to translate Ubuntu jobs quite as directly, and a number of
important jobs would definitely need to be changed, but the conversion
isn't usually particularly difficult.)



If we are able to settle on a readiness protocol and fully (or almost 
fully) implement it, then ptrace will become irrelevant. I am sure that 
if upstream is in agreement on the proto, then Ubuntu and Debian can 
collaborate to update the jobs (and daemons) for their next releases.




prctl (PR_SET_CHILD_SUBREAPER) is used to make SIGCHLD notification 
work

properly when Upstart is supervising a user session.  This isn't a
required feature and could easily be compiled out until suitable 
kernel

support is available (this actually seems like the sort of thing that
could be done in the Hurd without too much difficulty, but I haven't
looked into it).  If absent, it might well impede the ability to do an
advanced desktop port, but it wouldn't get in the way of porting the
bulk of services.



Unity, likely the only desktop environment using Upstart as a session 
init, is not in Debian. The sacrifice of this functionality on 
non-linux systems is perfectly acceptable.




There might also be odds and ends around the details of wait status
handling.



Two things come to mind: use of epoll in upstart-socket-bridge, and no 
abstract namespace sockets.




So, I'll certainly concede that I haven't actually tried this, but 
from

what I know of Upstart/libnih's system dependencies, I think that the
Hurd could be accommodated with at worst possibly some loss of
non-critical features.

 I want to mention again something that you'd dismissed as possibly 
a joke
 earlier, but which I was actually serious about: I think a world in 
which
 we use systemd on Linux and upstart on non-Linux ports, should 
upstart
 prove much more portable, actually makes sense.  As I said in my 
other
 writeup, I believe the systemd feature advantage is sufficient to 
choose
 systemd in isolation, without the other issues discussed.  I also 
believe
 that maintaining a systemd unit plus an upstart configuration is, 
modulo
 testing (which I realize is a huge issue), much easier than 
maintaining a

 single sysvinit script.


I wouldn't discard this option out of hand, but I think it would need
significant tool support to be practical (e.g. heuristic generation of
Upstart job files from systemd units), unless we expect all service
maintainers to have kFreeBSD/Hurd VMs lying around.  Keeping the
sysvinit scripts and using Upstart as the init daemon is another
possibility, and at least in that case the sysvinit scripts are 
probably

still lying around.  We don't even necessarily have to choose between
those up-front.


Cheers,
Cameron Norman

Bug#727708: init system thoughts




On Mon, Dec 30, 2013 at 6:55 PM, Colin Watson  
wrote:


The ptrace arrangements used for "expect fork" and "expect daemon" 
have
been rather flaky in practice, especially when Upstart jobs are 
written

by people not experts in doing so, and they are an obstacle to
portability.  I think we should strongly discourage use of these in
Debian in favour of some other readiness protocol.  My personal
aesthetic preference is for the "expect stop" scheme or something 
close

to it, but I really don't care that deeply and the sd_notify scheme or
similar would work too.  Indeed, I might well be inclined to support
disabling the ptrace-requiring features entirely in Debian, and 
working
to disable them in Ubuntu in time as well (although that would 
require a

transition plan).




For reasons mentioned by Lennart Poettering in his G+ post, I disagree 
that expect stop is a good scheme for a readiness protocol. Sure, it is 
easy for the daemon and init system, but it is not extensible and can 
be harmful when debugging a process (send it a SIGSTOP, init thinks it 
is successful and gives it a SIGCONT). NOTIFY_SOCKET is a good scheme 
because it is very explicit and can not be misinterpreted by the init 
system. In addition, one init system already has implemented it :)




I think Russ has sold me on systemd's journal being a win, at least in
terms of how it enables much better status output for services, and 
it's

a shame that something equivalent isn't present in Upstart.  My
practical experience of late has been that the per-job log files you 
get

by default are good enough for most purposes where I need to debug
service operations, but it certainly isn't all packaged up as neatly.



Now, since two people agree on this, why do we not file a bug in 
launchpad asking for the per-job logs to be able to be shown by initctl 
(possibly via a --show-logs option)? In fact, I just did it: 
https://bugs.launchpad.net/upstart/+bug/1265123.


(P.S. sorry for the sass, it is just so fun :)

Nighty night,
Cameron Norman

Re: Bug#727708: init system other points, and conclusion

Steve Langasek wrote:
> On Mon, Dec 30, 2013 at 04:04:05PM -0800, Russ Allbery wrote:
>> Please recall the context here: this whole aside started with an objection
>> to my contention that adopting upstart requires disassembly and redoing of
>> an integration that we would otherwise not have to disassemble.  Nowhere
>> in my message did I say that we would or could not do that disassembly if
>> we adopted upstart; I said that it was work that we otherwise wouldn't
>> have to do.
>
>> That's the intended context of my point above: I don't think we're going
>> to port GNOME to a non-systemd infrastructure, in the sense of carrying
>> significant patches to GNOME to adopt it to, say, not using logind.  I
>> think GNOME will continue to use systemd APIs heavily, which makes GNOME
>> less portable.  That means that systems that are not capable of running
>> those systemd components will need to either port them or develop
>> alternatives.
>
>> I don't consider this wailing or gnashing of teeth, but rather a realistic
>> look at what efforts the project is talking about committing to, as
>> opposed to supporting people working on if they so choose.
>
> Ok.  I think our core point of disagreement here, then, is in our assessment
> of how much work we think this actually is (for the Linux case, not for the
> non-Linux case).  I think the actual package maintenance to make this happen
> is not even a weekend's worth of free time, and therefore represents a
> negligible committment of resources on Debian's part, given that this
> dissassembly/integration has already been done in Ubuntu.

I'm making the assumption, here, that the work you're talking about is
making logind and other such services run without systemd, rather than
attempting to make GNOME and other desktop environments run without
those services.

I think you're underestimating the amount of *ongoing* effort required
here.  I'd point out that systemd in Debian is still stuck at version
204, despite the very nice features available in 205 and newer,
specifically because logind dared to make use of those features.  I
fully expect systemd to continue producing new and interesting features
and *using* them, requiring alternative implementations to either
reimplement more of systemd or create an increasingly invasive fork of
it.  And while I think it's *possible* to continue doing so on an
ongoing basis, that's work that could be spent on other productive tasks
that don't involve reimplementation.

In any case, I sincerely hope that the cost of doing that work is borne
entirely by people who find it a worthwhile activity rather than a
monumental waste of time.  And I furthermore hope that an unmangled and
unforked version of systemd continues to be available in Debian for
folks who want to run the init system that continues to create
functionality so useful that the proponents of upstart are willing to do
a huge amount of work in order to adopt most of it other than the init
system itself.

- Josh Triplett

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131231051050.GA18715@leaf

Bug#727708: init system thoughts

On Mon, Dec 30, 2013 at 07:26:23PM -0800, Russ Allbery wrote:
> > (Now, I've been working with Upstart's model for years, and it's now a
> > pretty fundamental way of how I think of system operation; so if people
> > who are new to *both* models rather than partisans of one side or the
> > other consistently tell me that the systemd model is easier to grasp,
> > then I'll listen.)

> I am new to both models.  I'm not very fond of the upstart model.

> Lennart had a comment on this point that I thought phrased the problem in
> an interesting way: both systemd and upstart deal with the same data, but
> with systemd, the full dependency tree is precalculated and exists as
> state within the init system.  With upstart, behavior is dynamic and to
> some degree emergent: you know who is listening to what signals, but you
> can arbitrarily introduce signals, and you're dealing with a message bus
> rather than a dependency tree, so you're reacting to things as they happen
> on the bus.  In systemd, you can dump everything that can possibly happen
> and work through the state transitions by hand if needed, with the
> possible exception of unexpected device events (which are horribly
> important in some cases -- don't get me wrong -- but which are
> uninteresting in a lot of common debugging scenarios).  I think it's
> somewhat harder to do that with upstart, where events are generated more
> dynamically.

So for a concrete example of where I think upstart's model lets us get
things right at boot that systemd's dependency model does not (or at least,
this hasn't been done yet in Debian), I'd like to direct your attention to
/etc/network/if-up.d/upstart .  Conceptually, there are certain kinds of
services in a distro context that we don't want to start by default until
"the network" is up - because they aren't set up for socket-based
activation, or might need to bind to particular addresses/interfaces and not
fall back gracefully, etc.  Of course if we were writing all our services
according to best practices, we wouldn't have to worry about this, as the
service would just handle this gracefully (or maybe hand the complexity off
to the init system for socket-based activation - but then what does init do
with a request for a socket address that's not available? cycles within
cycles, etc).

But in the real world, we have a lot of services that we just want to start
in runlevel 2 and be able to trust that the network and disk are sorted.
This is the classic guarantee that sysvinit gave us pre-udev, but it's
fallen apart since then because a fast system can get all the way through
rcS before the kernel has even managed to enumerate all the network devices.

Upstart (as implemented in Ubuntu) restores this guarantee (with provisions
for failsafe booting in the case of a wrong network config), and it takes
advantage of upstart's capability of sending arbitrary signals to do so.  I
can see how one could implement the equivalent of upstart's
static-network-up event on systemd, using generators.  But what would the
equivalent to /etc/init/failsafe.conf look like?  I think this would be
very difficult to express in systemd language, yet it's altogether vital for
providing a boot that is both reliably ordered, and recoverable in the event
of problems.

Cheers,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org

signature.asc
Description: Digital signature

Bug#727708: init system thoughts

Steve Langasek  writes:

> Upstart (as implemented in Ubuntu) restores this guarantee (with
> provisions for failsafe booting in the case of a wrong network config),
> and it takes advantage of upstart's capability of sending arbitrary
> signals to do so.  I can see how one could implement the equivalent of
> upstart's static-network-up event on systemd, using generators.  But
> what would the equivalent to /etc/init/failsafe.conf look like?  I think
> this would be very difficult to express in systemd language, yet it's
> altogether vital for providing a boot that is both reliably ordered, and
> recoverable in the event of problems.

I'm not sure I completely understand what failsafe.conf actually does, but
I think the systemd answer to this is partly discussed here:

http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

A systemd integration in Debian may want to implement something akin to
upstart's if-up.d hook as a service that waits for ifup notification with
a timeout and then signals network.service to duplicate that upstart
functionality.  (I'm not sure I completely understand all of the issues
involved and whether that would always make sense.)

When using NetworkManager, as mentioned in that discussion, you may want
to enable NetworkManager-wait-online.service instead or in addition.  That
has a failsafe to not wait too long for network before continuing the
boot, including network-dependent services, anyway.

So, in other words, assuming that I understand this correctly, the way
that you implement this in systemd is that you add a Before= dependency to
the network.target (hm, which implies that my assumption about things
meddling with dependencies of core services being less likely is not as
correct as I thought it was, although I still think it's less likely to be
done by accident) that waits for the network to be configured, but
implements a timeout to ensure that you don't stall forever.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87zjnho9dn@windlord.stanford.edu

Re: Bug#727708: init system thoughts

Steve Langasek wrote:
> So for a concrete example of where I think upstart's model lets us get
> things right at boot that systemd's dependency model does not (or at least,
> this hasn't been done yet in Debian), I'd like to direct your attention to
> /etc/network/if-up.d/upstart .  Conceptually, there are certain kinds of
> services in a distro context that we don't want to start by default until
> "the network" is up - because they aren't set up for socket-based
> activation, or might need to bind to particular addresses/interfaces and not
> fall back gracefully, etc.  Of course if we were writing all our services
> according to best practices, we wouldn't have to worry about this, as the
> service would just handle this gracefully (or maybe hand the complexity off
> to the init system for socket-based activation - but then what does init do
> with a request for a socket address that's not available? cycles within
> cycles, etc).
> 
> But in the real world, we have a lot of services that we just want to start
> in runlevel 2 and be able to trust that the network and disk are sorted.
> This is the classic guarantee that sysvinit gave us pre-udev, but it's
> fallen apart since then because a fast system can get all the way through
> rcS before the kernel has even managed to enumerate all the network devices.

udev didn't break this assumption; this assumption became unreasonable
once people started wanting to dynamically change networks.  If you have
a service that can't cope with network interfaces showing up later on,
that same service won't cope with connecting to a new wireless network,
or plugging or unplugging an Ethernet cable.

I've run into this *exact* problem with upstart, and I found it entirely
untenable.  Full support for such a daemon would require not only
waiting to start it until a network becomes available, but also
reloading or restarting it every time the network changes in a way it
can't automatically cope with.

In any case, systemd does indeed "support" this case; simply make your
service depend on network-online.target, which will block for a
reasonable amount of time to see if a network interface comes online,
and eventually time out if that doesn't occur.  This will actually work
even if your primary network is a wireless network managed by
NetworkManager, and even if that network doesn't actually work until the
user has logged in, assuming your service is not actually in the
dependency path of the user logging in.

Personally, I would automatically report a bug any time I found a
service depending on network-online.target or the upstart equivalent.

- Josh Triplett

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20131231055803.GA19839@leaf

Bug#727708: init system thoughts

Oh, sorry, I forgot to respond to this part.

Steve Langasek  writes:

> Of course if we were writing all our services according to best
> practices, we wouldn't have to worry about this, as the service would
> just handle this gracefully (or maybe hand the complexity off to the
> init system for socket-based activation - but then what does init do
> with a request for a socket address that's not available?

This is what IP_FREEBIND is for, which is why it needs to be supported by
the socket activation configuration.  It's been considered best practice
for some time for IPv6 services binding to particular configured IP
addresses to use IP_FREEBIND because IPv6 network setup can take an
unpredictable amount of time.

-- 
Russ Allbery (r...@debian.org)   

-- 
To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ob3xo8ti@windlord.stanford.edu

Bug#727708: init system thoughts

On Mon, Dec 30, 2013 at 10:04:09PM -0800, Russ Allbery wrote:
> Oh, sorry, I forgot to respond to this part.

> Steve Langasek  writes:

> > Of course if we were writing all our services according to best
> > practices, we wouldn't have to worry about this, as the service would
> > just handle this gracefully (or maybe hand the complexity off to the
> > init system for socket-based activation - but then what does init do
> > with a request for a socket address that's not available?

> This is what IP_FREEBIND is for, which is why it needs to be supported by
> the socket activation configuration.  It's been considered best practice
> for some time for IPv6 services binding to particular configured IP
> addresses to use IP_FREEBIND because IPv6 network setup can take an
> unpredictable amount of time.

Ah, thanks for the pointer.  I saw your previous mention of IP_FREEBIND but
hadn't looked it up yet - that certainly does sound like an important
feature to take advantage of in socket activation.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Bug#727708: init system thoughts