Bug#727708: loose ends for init system decision
Russ Allbery r...@debian.org writes: This message is about a transition plan for an init system replacement and about how to handle portability to our non-Linux ports. I'm keeping the same subject as Ian's message on the same basic topics and attaching it to the same thread, but this is more of a separate writeup than a reply. [...] 5. Support for the other init system that was not chosen should be handled in the same fashion, should a team choose to pursue it. If we select systemd, package maintainers should still be willing to merge contributed upstart configuration, with the understanding that they can't test it and any support is on a best-effort basis only. Similarly, if we select upstart, package maintainers should be willing to merge systemd unit files and to enable upstream systemd support where requested and where it doesn't interfere with the operation of the daemon under upstart, with the understanding that the package maintainer is not committing to testing or directly supporting this configuration. Thanks all a lot for all your detailed writeups Russ. It is much appreciated. I think there is one additional questions that will probably need to be decided by the tc but hasn't really been discussed yet: Will packages that explicity depend on a (non-default) init system be allowed in Debian? For example, if I want to package a program that depends on a feature available only in upstart, but systemd was chosen as the default init system, can I upload this package with a depends on upstart, or will this be rejected? If such packages will not be allowed in the archive, does the burden of making them work with the default init lie on the maintainers of the default init (to add the missing feature), or the package maintainer (to work around the features absence if he wants the package in Debian)? The specific situation that I have in mind is of course when upstart becomes the default, and gnome becomes dependent on systemd, but I think the question is larger than just this example. Best, -Nikolaus -- Encrypted emails preferred. PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C »Time flies like an arrow, fruit flies like a Banana.« -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Nikolaus Rath writes (Bug#727708: loose ends for init system decision): I think there is one additional questions that will probably need to be decided by the tc but hasn't really been discussed yet: Will packages that explicity depend on a (non-default) init system be allowed in Debian? My answer to this is no. So, firstly, I would say that all packages must, in jessie at least, continue to support sysvinit. Russ (from the other side of the upstart/systemd fence) agrees. Failure to support sysvinit would be an RC bug. And since all the proposed replacement inits have a compatibility mode, that naturally means they'll work. Contributors who support the non-default new init system will be able to supply patches for native support and should have them accepted. If such packages will not be allowed in the archive, does the burden of making them work with the default init lie on the maintainers of the default init (to add the missing feature), or the package maintainer (to work around the features absence if he wants the package in Debian)? The latter. The specific situation that I have in mind is of course when upstart becomes the default, and gnome becomes dependent on systemd, but I think the question is larger than just this example. Such a decision by Gnome (implying ditching all non-Linux architectures, too) would be very disappointing IMO. I think this is a bridge we should cross if and when we come to it. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On 01/02/2014 10:20 AM, Ian Jackson wrote: Nikolaus Rath writes (Bug#727708: loose ends for init system decision): I think there is one additional questions that will probably need to be decided by the tc but hasn't really been discussed yet: Will packages that explicity depend on a (non-default) init system be allowed in Debian? My answer to this is no. So, firstly, I would say that all packages must, in jessie at least, continue to support sysvinit. Russ (from the other side of the upstart/systemd fence) agrees. Failure to support sysvinit would be an RC bug. And since all the proposed replacement inits have a compatibility mode, that naturally means they'll work. Contributors who support the non-default new init system will be able to supply patches for native support and should have them accepted. If such packages will not be allowed in the archive, does the burden of making them work with the default init lie on the maintainers of the default init (to add the missing feature), or the package maintainer (to work around the features absence if he wants the package in Debian)? The latter. Just to make sure that I expressed myself correctly: I was not thinking about a package that depends on a specific init system to start or stop, but about a program that is really specific to the *new* features of upstart or systemd. For example, a hypothetical future program to interactively adjust program cgroups cannot be sysvinit compatible in any meaningful sense, because it does not need to be supervised, started, or stopped. However, this program would depend on the cgroups API offered by systemd. So this program would not be allowed in Debian, unless its maintainer adds support for whatever cgroups managed we would eventually use with upstart? Best, Nikolaus -- Encrypted emails preferred. PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C »Time flies like an arrow, fruit flies like a Banana.« -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Nikolaus Rath writes (Re: Bug#727708: loose ends for init system decision): For example, a hypothetical future program to interactively adjust program cgroups cannot be sysvinit compatible in any meaningful sense, because it does not need to be supervised, started, or stopped. However, this program would depend on the cgroups API offered by systemd. So this program would not be allowed in Debian, unless its maintainer adds support for whatever cgroups managed we would eventually use with upstart? I would hope that we can standardise on a single API to the system's single cgroup writer. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On 01/02/2014 10:30 AM, Ian Jackson wrote: Nikolaus Rath writes (Re: Bug#727708: loose ends for init system decision): For example, a hypothetical future program to interactively adjust program cgroups cannot be sysvinit compatible in any meaningful sense, because it does not need to be supervised, started, or stopped. However, this program would depend on the cgroups API offered by systemd. So this program would not be allowed in Debian, unless its maintainer adds support for whatever cgroups managed we would eventually use with upstart? I would hope that we can standardise on a single API to the system's single cgroup writer. I certainly hope so too, but I think the question of how such a situation would be handled needs to be answered. Even if we end up with a standardized cgroup API, it may show up in a different disguise. Best, Nikolaus -- Encrypted emails preferred. PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C »Time flies like an arrow, fruit flies like a Banana.« -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Le jeudi 02 janvier 2014 à 18:30 +, Ian Jackson a écrit : I would hope that we can standardise on a single API to the system's single cgroup writer. I have already explained why this is not going to happen. The cgroups API in systemd is already part of the core systemd interface and subject to the stability promise. The only other existing implementation (cgmanager) is merely wrapping in D-Bus the existing kernel API, which is going to die when a single writer becomes necessary. -- .''`. Josselin Mouette : :' : `. `' `- -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Josselin Mouette writes (Bug#727708: loose ends for init system decision): Le jeudi 02 janvier 2014 à 18:30 +, Ian Jackson a écrit : I would hope that we can standardise on a single API to the system's single cgroup writer. I have already explained why this is not going to happen. The cgroups API in systemd is already part of the core systemd interface and subject to the stability promise. The only other existing implementation (cgmanager) is merely wrapping in D-Bus the existing kernel API, which is going to die when a single writer becomes necessary. Err, I'm not sure I see what is stopping the provision of the systemd cgroup manager API by a program other than systemd. But I haven't looked at this in detail. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Thu, Jan 2, 2014 at 11:46 AM, Josselin Mouette j...@debian.org wrote: Le jeudi 02 janvier 2014 à 18:30 +, Ian Jackson a écrit : I would hope that we can standardise on a single API to the system's single cgroup writer. I have already explained why this is not going to happen. The cgroups API in systemd is already part of the core systemd interface and subject to the stability promise. The only other existing implementation (cgmanager) is merely wrapping in D-Bus the existing kernel API, which is going to die when a single writer becomes necessary. It seems like you just explained how it has already happened. systemd's cgroup interface is covered under the stability promise, and seems like a good fit for a standard interface to the single cgroup arbitrator. Cameron
Bug#727708: loose ends for init system decision
]] Ian Jackson Nikolaus Rath writes (Bug#727708: loose ends for init system decision): I think there is one additional questions that will probably need to be decided by the tc but hasn't really been discussed yet: Will packages that explicity depend on a (non-default) init system be allowed in Debian? My answer to this is no. So, firstly, I would say that all packages must, in jessie at least, continue to support sysvinit. Russ (from the other side of the upstart/systemd fence) agrees. Failure to support sysvinit would be an RC bug. I think it would be unwise to require upstart and systemd to continue to support sysvinit. I'm not even sure what that would mean, in particular in the case of systemd-sysv whose sole purpose is to replace sysvinit. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Tollef Fog Heen writes (Bug#727708: loose ends for init system decision): Ian Jackson: So, firstly, I would say that all packages must, in jessie at least, continue to support sysvinit. Russ (from the other side of the upstart/systemd fence) agrees. Failure to support sysvinit would be an RC bug. I think it would be unwise to require upstart and systemd to continue to support sysvinit. I'm not even sure what that would mean, in particular in the case of systemd-sysv whose sole purpose is to replace sysvinit. I was speaking looaely. I meant, still loosely speaking but rather less so, all packages which contain daemons which are to be started by the init system. In my draft resolution I gave an even longer and more precise definition which proves still yet to have edge cases people aren't happy with. But I hope you understand roughly what I'm getting at. For the avoidance of doubt, I mean to include things like inetd and apache and the system dbus and udev and the nameserver, but to exclude things like systemd-sysv and upstart-socket-bridge. The wording in my draft resolution is designed to tolerate (although obviously not encourage) a hypothetical daemon whose bare bones packaging doesn't arrange for the daemon to be started at all, regardless of the init system in use. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Tue, Dec 31, 2013 at 10:18:08PM -0800, Russ Allbery wrote: For upstart readiness, obviously one needs some sort of explicit flag or trigger to enable the raise(SIGSTOP) behavior, since that will otherwise cause rather obvious problems in getting the daemon to work outside of upstart. I prefer to use a command-line flag for this (-Z, per your recommendation) since it has to be explicitly enabled and there isn't the same sort of safe fallback if it's missing the way that there is for the systemd protocol. I don't see any real problem with using an environment variable, though, as long as it's unset afterwards so that it's not inherited by children. Bear in mind that this may often be being done by the Debian maintainer in advance of upstream, and adding a one-character command-line option is problematic since that stands a decent chance of clashing. -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On 30 Dec 2013, at 18:47, Russ Allbery r...@debian.org wrote: However, I think it's the best available approach that balances our ideals as a project against the opportunities offered by a new init system. This approach does permit full use of new init system features for jessie except for eliminating /etc/default files (which I doubt we'd successfully do quickly anyway), and opens up the full spectrum of use cases after jessie. The cost is that packagers should merge contributed patches to the init systems that they don't use. I don't think this is too much to ask, nor do I think it will have serious effects on package complexity based on my own experience configuring a package to work under all three init systems I considered. I’m no longer a RM, but I would echo Russ’ comments here - an ordered transition is very important for Debian releases - and the size of this transition would be something that (in my opinion) would be very hard to achieve in one release cycle. Neil -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Russ Allbery writes (Bug#727708: loose ends for init system decision): Ian Jackson ijack...@chiark.greenend.org.uk writes: - avoid extra build-dependencies (eg on libsystemd) - avoid extra runtime dependencies (eg on deb-systemd-helper) These requirements, on the other hand, I find completely baffling. This approach seems completely contrary to Debian best practices. Our standard practice with all packages is to fully enable all available features that make sense on Debian and don't pose some concrete risk. The case in point is a little different. It's one thing to say that mail daemons should come with ldap support enabled, or whatever (and even then we sometimes have two versions e.g. heavy and light). For me it's a different proposition to suppose that _every_ daemon should bring in these kind of dependencies. This is particularly the case for build-dependencies on an avowedly and intentionally non-portable program. Of course this can be made conditional, but this is IMO too fiddly. We should recommend: - daemon authors and Debian maintainers should prefer command-line options to environment variables I disagree with including this in a statement. I think it's outside the scope of what we were asked to resolve, and I don't think it's the place of the TC to offer this sort of general advise to upstreams. It is very likely that Debian contributors will be implementing native support for whatever readiness protocol, in the upstream parts of the codebase. It is IMO appropriate for those contributors to get advice on how to do this. Policy already gives advice on this kind of thing. Given the context, and the fact that this advice is going to be read by a lot of people as part of a big enhancement project, I think it's appropriate for the TC to answer this question. If we're going to offer specific advice, we should limit it to the specific integrations that we're asked to consider. But I think we should be mindful of the restriction on the TC not doing design work and let Policy for packaging support of upstart and/or systemd be hashed out through the regular Policy process. Are you proposing then that the TC should decide the default init system, but asmk people NOT to start converting packages until the policy questions are settled ? Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Ian Jackson ijack...@chiark.greenend.org.uk writes: Russ Allbery writes (Bug#727708: loose ends for init system decision): These requirements, on the other hand, I find completely baffling. This approach seems completely contrary to Debian best practices. Our standard practice with all packages is to fully enable all available features that make sense on Debian and don't pose some concrete risk. The case in point is a little different. It's one thing to say that mail daemons should come with ldap support enabled, or whatever (and even then we sometimes have two versions e.g. heavy and light). I'm aware of only one case of that in the archive. It's certainly not what we normally do. For me it's a different proposition to suppose that _every_ daemon should bring in these kind of dependencies. It's only going to be *every* daemon if we select systemd as the default init system, in which case we should be doing this in many daemons for either socket activation or for readiness notification as part of a proper systemd integration. If we do not select systemd as the default init system, the only maintainers doing this will be those who want their packages to work well with systemd or whose daemons are of particular interest to people who want to run systemd as a non-default init system. In which case, what's the harm? It's going to be very difficult to even notice this dependency is there. I think the position you're taking here is directly contrary to Debian's position of deference and reasonable accomodation to things that people want to work on. This is particularly the case for build-dependencies on an avowedly and intentionally non-portable program. Of course this can be made conditional, but this is IMO too fiddly. Adding [linux-any] to the Build-Depends line is not too fiddly, and if the goal is to enable optional upstream support for systemd, that will be sufficient. Besides, in general, it's up to the packager to decide what is or is not too fiddly in how they want to package software. Obviously, we should not add an unconditional build dependency for a library that isn't available on some of our ports to software that can work on those ports. But I think that's obvious. I'm happy to have us say that explicitly if if helps. We should recommend: - daemon authors and Debian maintainers should prefer command-line options to environment variables I disagree with including this in a statement. I think it's outside the scope of what we were asked to resolve, and I don't think it's the place of the TC to offer this sort of general advise to upstreams. It is very likely that Debian contributors will be implementing native support for whatever readiness protocol, in the upstream parts of the codebase. It is IMO appropriate for those contributors to get advice on how to do this. Policy already gives advice on this kind of thing. Given the context, and the fact that this advice is going to be read by a lot of people as part of a big enhancement project, I think it's appropriate for the TC to answer this question. If you feel like Policy should give advice on this, you can bring it up in the context of Policy. I don't think the exact mechanism of integration is important enough for the TC to explicitly favor a particular mechanism, and different upstreams may have different approaches. I'm not seeing the important gain to Debian here in trying to standardize exactly how one tells a daemon to use socket activation or readiness notification. I also, for what it's worth, directly disagree with this advice with respect to systemd because I think compatibility with how systemd is used elsewhere is more important than any marginal gain from using a configuration method that's not inherited by subprocesses by default, particularly given that the standard systemd APIs include environment sanitizing. But I also don't see any point in arguing about it here, since I don't think this dispute is ripe (in the legal sense). If you do bring it up in the context of Policy and we can't resolve the debate there, it can get appealed to the TC and we can decide it here separately, but I think this is all premature, given that the whole point becomes basically moot if we don't pick systemd anyway. If you're worried about programs configured with environment variables in the archive, you've got quite a lot of work to do, starting with nearly every Perl program in existence. If we're going to offer specific advice, we should limit it to the specific integrations that we're asked to consider. But I think we should be mindful of the restriction on the TC not doing design work and let Policy for packaging support of upstart and/or systemd be hashed out through the regular Policy process. Are you proposing then that the TC should decide the default init system, but asmk people NOT to start converting packages until the policy questions are settled ? Sure. I think
Bug#727708: loose ends for init system decision
Michael Gilbert writes (Bug#727708: loose ends for init system decision): On Mon, Dec 30, 2013 at 7:16 PM, Vincent Bernat wrote: Unfortunately, being the best init is the not only the matter of its maintainers. A good integration implies to modify some packages whose maintainer may be hostile to some init and may consider that their package work well enough to not enable some feature needed by some init. That is a hypothetical, of course, but it is a very real possibility for any path that the TC decides here. However, selectable inits may (possibly counter-intuitively) reduce this likely-hood. Since that maintainer in the way also gets his init of choice, he may be less likely to oppose an alternative that doesn't in any real sense get in his way. However, if/when this does happen, it will be an interesting test of whether the TC has the political will to override an indignant maintainer with their 3-to-1 majority power. This is why I think the TC resolution should explain what kinds of integration package maintainers shuuld accept. Maintainers who flout that advice will indeed get overruled. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Russ Allbery writes (Bug#727708: loose ends for init system decision): Ian Jackson ijack...@chiark.greenend.org.uk writes: For me it's a different proposition to suppose that _every_ daemon should bring in these kind of dependencies. It's only going to be *every* daemon if we select systemd as the default init system, in which case we should be doing this in many daemons for either socket activation or for readiness notification as part of a proper systemd integration. [...] Well, no. I think that even if we select upstart as the default, we should enable the systemd community to provide as complete a set of integration in Debian as they care to put the work in for. That translates directly to an expectation that the maintainer of any daemon package must be willing to accept reasonable systemd integration patches. If the systemd community is very dynamic (and certainly we can't fault their dynamism so far) this may well translate to all or almost all daemons. Certainly it could be expected to include most of the core daemons where the extra dependencies are most troublesome. So I think we need to say what we regard as reasonable patches, in advance. As the Debian maintainer for uservd (for example), am I entitled to decline to incorporate systemd integration into my package on the grounds that the patch involves additional build- and runtime dependencies which I consider undesirable ? I don't consider it desirable to answer this question differently for different daemons. We need to set out a clear rule for maintainers to follow or we'll end up adjudicating a gazillion individual disputes. And if the answer to this question is, in general, the maintainer must include the patch then the logical conclusion is that it is OK for every daemon in Debian to acquire these additional build- and runtime dependencies. We would be telling non-Linux ports that they need to arrange for these dependencies to be somehow provided despite the lack of systemd on their architecture, or to have conditional compilation of the systemd support in every daemon package. If the TC's answer is the maintainer may reject the patch, we would be telling the systemd community that they need to improve and simplify their integration. I think the latter is the right approach, for two reasons. Firstly, it is less work overall. Secondly, it is proper that the maintainers of an init system should be encouraged to provide as simple and nonintrusive an integration interface as possible. This is particularly the case for build-dependencies on an avowedly and intentionally non-portable program. Of course this can be made conditional, but this is IMO too fiddly. Adding [linux-any] to the Build-Depends line is not too fiddly, and if the goal is to enable optional upstream support for systemd, that will be sufficient. Besides, in general, it's up to the packager to decide what is or is not too fiddly in how they want to package software. Adding [linux-any] to the Build-Depends is not sufficient. It is also necessary to make the actual source code conditionally use sd_notify, using #ifdefs or other similar techniques. The conditional compilation will have to be automated (perhaps with autoconf if the package has it - an then we need new --with[out]-systemd configuration options to avoid miscompilation; or perhaps in an ad hoc way if the daemon doesn't use autoconf). The fact that the systemd-specific command line option may or may not be compiled in needs to be mentioned in the documentation, along with text which allows the user to know whether it's included in the version they actually have. This turns what could be a simple change into a multi-part epic involving all of the components of the package: primary source code, upstream build system and portability layer, Debian build system, documentation, and so on. And it might come about later that someone provides a portable sd_notify so half (but just the right half) of the above would have to be undone. Obviously, we should not add an unconditional build dependency for a library that isn't available on some of our ports to software that can work on those ports. But I think that's obvious. I'm happy to have us say that explicitly if if helps. I think maintainers should not be required to introduce the additional build- and runtime dependencies, if they do not wish to. Obviously if they want to then that's fine by me. It is very likely that Debian contributors will be implementing native support for whatever readiness protocol, in the upstream parts of the codebase. It is IMO appropriate for those contributors to get advice on how to do this. Policy already gives advice on this kind of thing. Given the context, and the fact that this advice is going to be read by a lot of people as part of a big enhancement project, I think it's appropriate for the TC to answer this question. If you feel like Policy should give advice
Bug#727708: loose ends for init system decision
Hi Ian, Ian Jackson ijack...@chiark.greenend.org.uk writes: This is particularly the case for build-dependencies on an avowedly and intentionally non-portable program. Of course this can be made conditional, but this is IMO too fiddly. Adding [linux-any] to the Build-Depends line is not too fiddly, and if the goal is to enable optional upstream support for systemd, that will be sufficient. Besides, in general, it's up to the packager to decide what is or is not too fiddly in how they want to package software. Adding [linux-any] to the Build-Depends is not sufficient. It is also necessary to make the actual source code conditionally use sd_notify, using #ifdefs or other similar techniques. The conditional compilation will have to be automated (perhaps with autoconf if the package has it - an then we need new --with[out]-systemd configuration options to avoid miscompilation; or perhaps in an ad hoc way if the daemon doesn't use autoconf). The fact that the systemd-specific command line option may or may not be compiled in needs to be mentioned in the documentation, along with text which allows the user to know whether it's included in the version they actually have. This turns what could be a simple change into a multi-part epic involving all of the components of the package: primary source code, upstream build system and portability layer, Debian build system, documentation, and so on. No, this is all not true :). libsystemd-daemon already contains the conditionals necessary to be a noop on non-linux: http://sources.debian.net/src/systemd/204-5/src/libsystemd-daemon/sd-daemon.c Therefore, you can just use it, and don’t need to use any conditionals on your end, neither in the compilation process, nor in the code itself. That being said, I just checked and realized the package is not available on non-linux. I’ll look into that now, since intuitively there is no reason for this. -- Best regards, Michael -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Michael Stapelberg stapelb...@debian.org writes: That being said, I just checked and realized the package is not available on non-linux. I’ll look into that now, since intuitively there is no reason for this. Thank you, Michael. That would indeed make things much easier. I think Ian is being excessively dramatic about very simple conditional integrations, but still, avoiding the conditional entirely is useful. One approach that would, I hope, minimize Ian's concerns is to provide a libsystemd-daemon-dev on non-Linux ports that just stubs out the calls, and that, on those architectures, provides some sort of dummy empty library that does nothing and adds no dependency. (There may be an elegant way to do this with a linker script.) That way, the exact same build process can be used everywhere, and everything just quietly disappears on non-Linux ports where systemd isn't available, without even adding a dependency on an (on that platform) useless shared library. What I used for lbcd is: #ifdef HAVE_SD_NOTIFY # include systemd/sd-daemon.h #else # define SD_LISTEN_FDS_START 3 # define sd_listen_fds(u) 0 # define sd_notify(u, s) 0 #endif which of course only stubs out the two major calls and doesn't address the rest of the API. That would need to be expanded to at least account for sd_notifyf (which will require a variadic macro, but that shouldn't be a problem in Debian) and sd_booted. I'm not sure what the best thing to do about the sd_is_* calls would be. One possible approach would be to just define them all to -ENOSYS, since they generally wouldn't be called when sd_listen_fds isn't available, although that would be a problem if any package ever used those APIs outside of a systemd context. (But I'm not sure why one would do that.) -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
]] Ian Jackson So I think we need to say what we regard as reasonable patches, in advance. As the Debian maintainer for uservd (for example), am I entitled to decline to incorporate systemd integration into my package on the grounds that the patch involves additional build- and runtime dependencies which I consider undesirable ? If find it incredible that you consider adding 18 lines of unconditional code to your daemon unreasonable, while forcing the systemd maintainers to split the package, add explicitly rejected upstream-incompatible features reasonable. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Mon, Dec 30, 2013 at 10:24 PM, Steve Langasek wrote: On Mon, Dec 30, 2013 at 08:12:19PM -0500, Michael Gilbert wrote: Part of my goal in writing up that plan was, as you say, to try to provide a means for people who are committed to one system or the other to continue to work on what they're passionate about even if it's not chosen as the default init system. Unfortunately at least two camps will be entirely dejected by any TC mandate here. I don't agree with that conclusion. That no unhappiness will arise out of this decision is, I think, a quite unlikely outcome, so this statement is quite surprising. As Russ eloquently states today [1] The degree to which this all should affect our decision-making process is limited. That's particularly true in this case, which is one of those decisions where, whatever we choose, some group of people in Debian who have put a ton of work into something they care about are going to be at least somewhat unhappy. That may be the systemd maintainers, the GNOME team, the Hurd porters, the kFreeBSD porters, the upstart maintainers, those who care deeply about tight Debian and Ubuntu integration, or any number of other groups. It will probably be several of those. The TC needs to find a path that minimizes unintended social consequences. I have been trying to make the case in this thread that the only reasonable solution that does so is one which empowers all camps to do work to further their cause without feeling excluded. Eventually, and this may take sometime but everything worth doing does, a clearly superior solution will emerge that can, when ready, become the default. When it comes to technology choices, you win some and you lose some. The fact that there may be winners and losers is not the issue at hand. It is the specter of being disqualified before the game even starts. Your proposal smells like the status quo. That is only true if none of the systemd, openrc, and upstart teams get to the point where their init is usable, then we're stuck with sysvinit. This is probably an unlikely circumstance, but sysvinit does work. Namely, instead of the project making a decision and being able to all pull together in the same direction to provide the best possible OS, we will continue to coast, squandering efforts on preserving users' ability to make choices about things that no user should ever be asked to care about The project never goes in the same direction. 1,000 people go in their own direction, creative solutions eventually emerge, one often gains the most traction, but others remain as alternatives (think debhelper/cdbs/etc), and choice and freedom remain. It will be a major change in project governance to select any resolution prior to emergence. In my opinion, the only reasonable TC decision is one that requests project members feel empowered to work toward the solutions that they are interested in, while being open-minded and non-obstructive to all of the other competing solutions. Best wishes, Mike -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Okay, let's see how replying to a mail on my phone while in flight mode goes. Apologies in advance for formatting, quoting and vocabulary issues. On Dec 31, 2013 4:48 AM, Russ Allbery r...@debian.org wrote: 2. Impact of Multiple Init Systems Obviously, the ideal situation for project-wide integration and support, and for Debian's ability to make full use of the capabilities of new init systems, is for all of Debian's ports to be running the same init system. So, reading this (and trimming loss of stuff that makes sense) I wonder if it's worth stepping back and considering what happens when a package doesn't support /any/ init system. The answer I think is that the sysadmin sighs, adds their own configuration, and maybe thinks well at least I didn't have to compile it, I guess. That still sucks compared to the alternative of typing apt-get install and having it just start working, of course. Attempting to support multiple init systems has several obvious drawbacks: * Packages will either be limited by the functionality of the least capable init system or will behave differently (and have to be configured differently) on different ports. I think the Debian way of dealing with multiple init systems would be to provide a compatibility layer for the most common packaging and admin tasks, and allowing packages to provide fancier integration where appropriate. I'm thinking of things like newaliases and sendmail, and cgi-bin and apache modules, I guess. Probably that's already covered for init systems via sysvinit compatibility and update-rc.d. Maybe there's a missing tool that could be written to let you set init specific configuration somehow, which would change /etc/default for sysv and other files for upstart/systemd. It seems like there's maybe three separate questions then: what init system gets used by default, what work gets done to make that experience different/better than everything just having upstart or systemd pretending to be sysvinit, and what's the experience of maintaining packages to support secondary init systems and using secondary init systems on a Debian system? (Personally, I think a gr would be better for choosing the default then having the tech ctte decide that issue, but whatever) Based on what I've read, it seems like the ideas floating around amount to: - if systemd is default: there would be a release goal to include systemd configs added to packages and to get daemons converted to support socket activation. Maybe other stuff too? As far as maintaining sysvinit, openrc or upstart systemd goes, you'd just have to get upstart configs written and packaged, and accept that there's an unused systemd library on your system. Multiple inits must be supported to some extent, since systemd isn't available on ports and that isn't likely to change apparently. - upstart is default, other inits are supported: pull in Ubuntu configs for upstart for various packages. If systemd socket stuff is allowed, dummy library will probably be on most systems, if not, systemd in Debian won't be very interesting. - upstart is default and only init supported by Debian. Same support work for Jessie, except any ports that want to stick around need to get upstart enabled. I don't think the latter is really the Debian way - better to have the choice left in the users' hand if feasible, but it's likely a lot easier to get some impressive results that way. I think the ideal page for tradeoffs like that is in derivatives like Ubuntu personally. I believe Debian should take this path forward: 1. We should select a new init system for jessie, either systemd or upstart. Support for that init system should be release-critical, but only in the sense that the daemon works properly under that init system. In other words, use of the sysvinit compatibility of either init system is acceptable support for jessie. So that would mean switching the init system, and decorating anything that breaks as a result RC buggy. Seems sensible. 2. All packages providing init scripts must continue to support sysvinit scripts through the jessie release. Such support will continue to be release-critical. I'm not sure this makes sense, though. If someone packages a new daemon that works fine with systemd and upstart, I don't see why it shouldn't get released just because it doesn't work with two secondary init systems. Filling a wishlist bug with a patch and doing an NMU seem like they'd be sufficient here. As far as upgrades go, if the idea is that systemd is the way to go for Jessie it seems to me that an update would by default switch you from sysvinit to systemd (likewise for upstart) - I don't think it makes much sense to do systemd/upstart for new installs but leave upgrades with sysv until Jessie+1. 7. After jessie, functionality between systems running the primary Linux init system and other init systems (including non-Linux ports) should be allowed to drift. In other
Bug#727708: loose ends for init system decision
Ian Jackson ijack...@chiark.greenend.org.uk writes: Well, no. I think that even if we select upstart as the default, we should enable the systemd community to provide as complete a set of integration in Debian as they care to put the work in for. That translates directly to an expectation that the maintainer of any daemon package must be willing to accept reasonable systemd integration patches. If the systemd community is very dynamic (and certainly we can't fault their dynamism so far) this may well translate to all or almost all daemons. Certainly it could be expected to include most of the core daemons where the extra dependencies are most troublesome. So I think we need to say what we regard as reasonable patches, in advance. As the Debian maintainer for uservd (for example), am I entitled to decline to incorporate systemd integration into my package on the grounds that the patch involves additional build- and runtime dependencies which I consider undesirable ? Ah! Okay, now I understand why you think this is linked and something we should decide here. Thank you. Okay, so let me see if I understand what we agree on and what we would need to debate. Would you agree with both of these statements? * If we adopt systemd, then obviously integration with systemd would be desirable and should be accepted, including build and runtime dependencies as is appropriate for how systemd is integrated into the distribution as a whole. * If a maintainer wishes to integrate with systemd via build and runtime dependencies in their daemon, that would be their call, not something the TC would decide. (I think you do agree with this given the statement below, but leaving this in as summary.) If so, then the problem reduces to what to do in the case where someone requests integration with systemd via a bug report and the maintainer doesn't want to provide that integration, at least with build or runtime dependencies or both. Let's put aside the question of how to handle this on the ports that don't have systemd for the moment, and hence the conditional build integration parts, since that was discussed in the other thread and I think there are possible lightweight schemes that are specific to the case where we just want to stub out the calls. I assume that the reason why you're bringing this up, per your original message, is that you don't want to introduce either build or runtime dependencies. I don't, however, understand why; the logic doesn't make any sense to me. Both of those dependencies are so lightweight that I have a hard time seeing how anyone would notice they exist, beyond having a couple of packages installed on the system that wouldn't otherwise be. I'm inclined to say that asking maintainers to add those dependencies is appropriate. I'm not aware of a precedent for this for something that we're not committing to supporting for the archive, but it certainly parallels other deployments in Debian, such as SELinux and multi-arch. I think the latter is the right approach, for two reasons. Firstly, it is less work overall. Secondly, it is proper that the maintainers of an init system should be encouraged to provide as simple and nonintrusive an integration interface as possible. I believe that systemd has already provided this, and I find your belief that it hasn't to be strange. I certainly don't think that we will succeed, or *should* succeed, in convincing systemd to adopt a new API that offers no clear benefit over their existing API and is incompatible with it. If I were them, I'd reject it out of hand as a waste of time and effort. Incidentally, the tools in the init-system-helpers package are basically playing the same role as tools that are in the sysv-rc package now. I think one could make a pretty good argument that, were we designing this from scratch today without worrying about history or backward compatibility, update-rc.d and invoke-rc.d would go into init-system-helpers, since they're (now) general tools that support a variety of underlying init systems. If the dependency on that package is really such a burden, one thing we could do is just move the two scripts in it into sysv-rc and use sysv-rc as the package that holds those tools. I don't really care, and I don't mind having two packages, but looking at the current contents of both, they're really quite close to the same thing. (sysv-rc also continues to provide a small amount of additional infrastructure for sysvinit, but it's pretty limited.) I think maintainers should not be required to introduce the additional build- and runtime dependencies, if they do not wish to. Obviously if they want to then that's fine by me. Ah, good, okay. This is less of a conflict than at first I thought we had. I definitely understand the desire to not impose requirements on package maintainers that they don't like, and to minimize the impact of such requirements. If you feel like Policy
Bug#727708: loose ends for init system decision
This message is about a transition plan for an init system replacement and about how to handle portability to our non-Linux ports. I'm keeping the same subject as Ian's message on the same basic topics and attaching it to the same thread, but this is more of a separate writeup than a reply. I'll reply to Ian's message separately. I've expressed my opinion separately about which init system we should choose. This message is written from the assumption that we will choose either upstart or systemd as the init system for the non-Linux ports. I believe either system would pose basically the same concerns. 1. Role of Non-Linux Ports in Debian There has been a lot of discussion in various places, including the Debian mailing lists, about the utility of the non-Linux ports and about how much they should play a role in project decisions. I think this deserves to be addressed directly here. One of the things that I love about Debian, and one of the reasons why I am involved in this project as opposed to a different distribution, is that it is a labor of love. Debian is not driven by market forces; we have users and we want the distribution to work for them, but we're not competing for market share. Debian does not make decisions based on simple popularity, or on customer counts. Rather, Debian is a project of mutual cooperation among Debian's contributors. We work on what we care about, regardless of whether that makes economic sense. It is not, in general, necessary to justify what you want to do in Debian. It doesn't matter if it's going to be used by thousands of people or two people. If you can do your work to the standards that we expect to be maintained across the archive, and without negative impact on Debian's other contributors, you can work on whatever you love. And, furthermore, we all support each other in our passions. Debian is built on a culture of deference to other people's work, and reasonable accomodation to the projects that other people want to work on. Now, there is a fine balance here. Deference to other people's work does not mean a requirement to join their work. Reasonable accomodation does not mean that every Debian developer is required to test their software on non-Linux ports. The goal is that all of us should be empowered to work on the things we are passionate about, which implicitly includes being empowered to not work on the things that are not of interest to us. Therefore, for some efforts, and specifically for the non-Linux port efforts, the work is mostly born by the porters. They're expected to test, diagnose problems, and submit patches. The deference and reasonable accomodation that we expect of Debian contributors is to merge those patches when they are reasonable, to not undermine the porting effort when there is a reasonable approach that preserves it, and to be aware of the implications of their packaging for those ports in the broad sense (such as qualifying build-dependencies with [linux-any] where appropriate). We do not expect Debian contributors to reject significant new upstream versions or significant new integrations because they will affect the non-Linux ports, or, for that matter, other projects in Debian. We do expect those changes to follow the standards that we expect of Debian as a whole, and that porting efforts will be treated with deference and reasonable accomodation. I think this status applies to both our Hurd port and our kFreeBSD port. Those ports have different challenges, and arguably different levels of utility to general users, but as mentioned, I don't consider popularity to be a useful metric here. It doesn't make any difference to me if the port is used by thousands of people or if it's only used by the people actively working on it: the same principles of deference and reasonable accomodation apply. I believe strongly that this is something that defines Debian as a project and is worth preserving. It's also worth saying here that I don't believe any of the above is particularly controversial within the project. We have wide-ranging arguments about it in the abstract, and quite a few disparaging comments have been thrown around about the non-Linux ports in the init system discussion, which makes me sad. But when it comes to the day-to-day work in the project, nearly all maintainers are quite good (and have been for years) about merging required patches for the Hurd and pushing them upstream, responding to bug reports on systems they don't use, and making a reasonable effort to accomodate other people's projects. Similarly, our non-Linux porters have been exemplary. Neither the Hurd nor the kFreeBSD ports have expected every Debian contributor to directly support their platforms. They track down problems, develop patches, and submit tested patches to the BTS. They have developed their own solutions for packages that are not portable and worked out how to integrate them with the archive. Despite the often
Bug#727708: loose ends for init system decision
Russ Allbery writes (Bug#727708: loose ends for init system decision): 1. Role of Non-Linux Ports in Debian I agree with most of this. 2. Impact of Multiple Init Systems I don't want to do a blow-by-blow quote/rebuttal of this. 3. systemd and upstart As Multiple Systems .. I therefore think that, regardless of which init system we pick, we should keep in mind the above principle of Debian's deference and reasonable accomodation to other people's projects and apply that to systemd and upstart as well as to the non-Linux ports. Obviously, this also has the same issues mentioned above: Debian contributors can only be expected to test on the primary init system, other configurations will tend to bitrot without active porter attention, and so forth. But if people want to take on the work, that deserves our respect. Right. 4. Conclusions ... 1. We should select a new init system for jessie, either systemd or upstart. Support for that init system should be release-critical, but only in the sense that the daemon works properly under that init system. In other words, use of the sysvinit compatibility of either init system is acceptable support for jessie. I agree. 2. All packages providing init scripts must continue to support sysvinit scripts through the jessie release. Such support will continue to be release-critical. This is going to be painful for packages that want to do an early conversion, since it means testing two different init systems for this release cycle, but I think this is the right thing to do regardless for a clean upgrade path and Debian's normal robust committment to upgrades. Here it has the additional advantage of giving the non-Linux ports some breathing space to strategize. Yes. 3. Related, up through the jessie release, packages must (where possible; it's possible there will be cases where this is simply impossible) support switching back and forth between the new default init system and sysvinit. This means that configurations should not be moved out of /etc/default and that startup files for the new init system should read the same configuration that the existing sysvinit scripts use (or both should be modified compatibly). Yes. 4. Post-jessie, support for sysvinit will no longer be release-critical, and package maintainers will no longer be expected to ensure that it continues working. [...] 5. Support for the other init system that was not chosen should be handled in the same fashion, should a team choose to pursue it. [...] I think it is probably premature for us to drop support for the other system in jessie+1. But we don't need to make this decision now. 6. Debian's non-Linux ports should either use the same init system as Debian's Linux ports or agree on an init system that they're both going to use. The porting work is going to be hard enough without the ports going in different directions on which secondary init system they want to use. I prefer to leave it up to the porters to decide which init system to choose, but I do think OpenRC would be a strong contender. Is there some difficulty expected with upstart on hurd ? We should revisit this decision again after the jessie release in case the situation has substantially changed. I would be very reluctant to write now that the support for sysvinit, or the non-default new system, may be dropped in jessie+1. That will result in premature rot and removal. It's also possible that some kind of compatibility mechanism will become available. I would like to leave this decision to the policy maintainers, with the expectation that the TC will probably need to decide. Thanks, Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Ian Jackson ijack...@chiark.greenend.org.uk writes: Russ Allbery writes (Bug#727708: loose ends for init system decision): 6. Debian's non-Linux ports should either use the same init system as Debian's Linux ports or agree on an init system that they're both going to use. The porting work is going to be hard enough without the ports going in different directions on which secondary init system they want to use. I prefer to leave it up to the porters to decide which init system to choose, but I do think OpenRC would be a strong contender. Is there some difficulty expected with upstart on hurd ? I don't know if anyone has looked at it, and in the absence of any practical experience, I think it's fair to expect some challenges. The blog post on kFreeBSD porting indicated that the porting effort to date required eglibc and FreeBSD kernel changes. It seems like a reasonable assumption that at least a similar level of effort will be required on Hurd, and I don't know if the Hurd has as mature (if incompatible) capabilities such as kqueue on FreeBSD to use to implement such things as inotify or epoll. I know very little about the Hurd, so basically I just don't know. I would be very reluctant to write now that the support for sysvinit, or the non-default new system, may be dropped in jessie+1. That will result in premature rot and removal. It's also possible that some kind of compatibility mechanism will become available. That's a reasonable point. I have no objections to deferring that decision until after the jessie release. I would like to leave this decision to the policy maintainers, with the expectation that the TC will probably need to decide. Yeah, with my Policy hat on, I don't want any piece of that one. I would be inclined to immediately escalate to the TC. I don't believe that Policy's conservative consensus approach is going to work here, and I expect trying to be painful. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Ian Jackson ijack...@chiark.greenend.org.uk writes: I think we should give package maintainers some guidance on what kinds of upstart or systemd patches should be accepted. Without this, it's likely we'll find ourselves adjudicating disputes that ought to have been settled in principle much earlier. I think that patches for either should: - use a non-forking startup method - avoid significant amounts of open-coded AF_UNIX socketry - avoid embedded copies of daemon library code These all seem reasonable to me, with the caveat that if *upstream* includes open-coded AF_UNIX socketry or embedded copies of daemon library code, I don't see any need for Debian package maintainers to remove that code. - avoid extra build-dependencies (eg on libsystemd) - avoid extra runtime dependencies (eg on deb-systemd-helper) These requirements, on the other hand, I find completely baffling. This approach seems completely contrary to Debian best practices. Our standard practice with all packages is to fully enable all available features that make sense on Debian and don't pose some concrete risk. This means that, for example, I have CUPS libraries on this system despite the fact that I never print, Avahi libraries depsite the fact that I will never use that software, and SELinux libraries deeply embedded in core code despite the fact that few Debian systems currently run SELinux. These requirements seem like would fit with Gentoo, where much emphasis is placed on letting people build custom-crafted binaries to their local requirements, not Debian's approach of providing a full-featured and uniform distribution. Needless to say, I don't think we should avoid dependencies on libsystemd-daemon or init-system-helpers, and I think it would be wholly inappropriate for the Technical Committee to say anything of the sort. (If the current state of the art in readiness protocols persists that means that upstart patches using raise(SIGSTOP) ought to be accepted, but systemd ones rejected unless they can use socket activation.) So, as mentioned above, I don't agree with this, although of course I think using socket activation is, in general, better than using the readiness protocol. We should recommend: - daemon authors and Debian maintainers should prefer command-line options to environment variables I disagree with including this in a statement. I think it's outside the scope of what we were asked to resolve, and I don't think it's the place of the TC to offer this sort of general advise to upstreams. - whatever environment variables and fds are used, measures should be taken to avoid them being inherited by daemons' arbitrary children I agree with this as good practice for daemons, but again, I don't think it's the role of the TC to issue this sort of general advice that is not directly related to a question put before us. If we're going to offer specific advice, we should limit it to the specific integrations that we're asked to consider. But I think we should be mindful of the restriction on the TC not doing design work and let Policy for packaging support of upstart and/or systemd be hashed out through the regular Policy process. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
]] Russ Allbery Given that, I don't believe a Technical Committee choice of a default init system is going to make either the systemd or the upstart maintainers want to stop maintaining their packages. Given what you're basically deciding between is «upstart + castrated systemd» or «systemd» and I think I've pretty clearly expressed my thoughts on splitting up systemd, I don't know where that conclusion comes from. Were you to choose upstart, I'd be seriously thinking about stopping maintaining systemd. (This is meant as a data point, not as a threat or anything like that.) My ideas for the package align pretty well with upstream (which I'm a sometimes-active part of) where the different services are free to assume that they run with systemd as pid 1. Continously porting new upstream versions to an environment unsupported by upstream is not what I consider reasonable or fun. I'm speaking for myself here, not the other systemd maintainers. In addition, as per my other message, were you to choose upstart as a default, I'd be inclined to remove the init functionality from systemd. I think trying to support multiple init systems is crazy and a recipe for disaster and I don't want to do that. (Doing it as part of a transition is a necessary pain, that much I can agree with, but ongoing? No thanks.) 6. Debian's non-Linux ports should either use the same init system as Debian's Linux ports or agree on an init system that they're both going to use. The porting work is going to be hard enough without the ports going in different directions on which secondary init system they want to use. I prefer to leave it up to the porters to decide which init system to choose, but I do think OpenRC would be a strong contender. I think allowing them to use a compatible init system should be ok too, if somebody wants to do that. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Mon, Dec 30, 2013 at 1:47 PM, Russ Allbery wrote: 4. Conclusions I previously argued that much of the benefit of a new init system comes from when we can stop maintaining init scripts. I still believe that, but after thinking more about the cultural and project issues at stake here, as well as the immediate needs for a clean upgrade path, I ended up at more of a compromise position than I expected. I believe Debian should take this path forward: 1. We should select a new init system for jessie, either systemd or upstart. Support for that init system should be release-critical, but only in the sense that the daemon works properly under that init system. In other words, use of the sysvinit compatibility of either init system is acceptable support for jessie. 2. All packages providing init scripts must continue to support sysvinit scripts through the jessie release. Such support will continue to be release-critical. This is going to be painful for packages that want to do an early conversion, since it means testing two different init systems for this release cycle, but I think this is the right thing to do regardless for a clean upgrade path and Debian's normal robust committment to upgrades. Here it has the additional advantage of giving the non-Linux ports some breathing space to strategize. 3. Related, up through the jessie release, packages must (where possible; it's possible there will be cases where this is simply impossible) support switching back and forth between the new default init system and sysvinit. This means that configurations should not be moved out of /etc/default and that startup files for the new init system should read the same configuration that the existing sysvinit scripts use (or both should be modified compatibly). 4. Post-jessie, support for sysvinit will no longer be release-critical, and package maintainers will no longer be expected to ensure that it continues working. However, for as long as Debian has accepted non-Linux ports using a different init system, package maintainers should continue to ship init scripts if they work and should apply patches and other reasonable fixes from porters for those init scripts. In other words, this should be treated the same as merging patches for the Hurd to remove hard-coded constant assumptions: if the change is reasonable and doesn't break Linux ports (and this should be fairly easy to handle for nearly all cases with init scripts), the package maintainer should merge it. 5. Support for the other init system that was not chosen should be handled in the same fashion, should a team choose to pursue it. If we select systemd, package maintainers should still be willing to merge contributed upstart configuration, with the understanding that they can't test it and any support is on a best-effort basis only. Similarly, if we select upstart, package maintainers should be willing to merge systemd unit files and to enable upstream systemd support where requested and where it doesn't interfere with the operation of the daemon under upstart, with the understanding that the package maintainer is not committing to testing or directly supporting this configuration. 6. Debian's non-Linux ports should either use the same init system as Debian's Linux ports or agree on an init system that they're both going to use. The porting work is going to be hard enough without the ports going in different directions on which secondary init system they want to use. I prefer to leave it up to the porters to decide which init system to choose, but I do think OpenRC would be a strong contender. 7. After jessie, functionality between systems running the primary Linux init system and other init systems (including non-Linux ports) should be allowed to drift. In other words, there will be cases where features will only be available with the primary init system. Possible examples include security hardening, socket activation, automatic daemon restarts, and so forth. Packagers are under no obligation to port those features to other init systems, but should welcome and merge patches that do so. After jessie, packagers will no longer be required to preserve daemon configuration when the init system is switched, so use of such facilities as modification of upstart configuration files or systemd overrides may be used. We should revisit this decision again after the jessie release in case the situation has substantially changed. Doesn't a TC mandate on the default init system in some sense violate Debian's spirit of meritocracy? May I suggest something like the following (this isn't meant to be definitive) as a more meritocratic approach: 1. The TC encourages a team (probably debian-boot) to provide a package (similar to
Bug#727708: loose ends for init system decision
Tollef Fog Heen tfh...@err.no writes: ]] Russ Allbery Given that, I don't believe a Technical Committee choice of a default init system is going to make either the systemd or the upstart maintainers want to stop maintaining their packages. Given what you're basically deciding between is «upstart + castrated systemd» or «systemd» and I think I've pretty clearly expressed my thoughts on splitting up systemd, I don't know where that conclusion comes from. I have to admit that I didn't give it a whole lot of thought. It was an assumption based on the presence of both in the archive for some time now as non-default init systems under sysvinit. In that sense, things don't change that horribly much, at least up to the jessie release, if the default changes from one non-systemd thing to another non-systemd thing. That being said, obviously you should speak for yourself, and I stand corrected. Apologies for misprepresenting your feelings here. 6. Debian's non-Linux ports should either use the same init system as Debian's Linux ports or agree on an init system that they're both going to use. The porting work is going to be hard enough without the ports going in different directions on which secondary init system they want to use. I prefer to leave it up to the porters to decide which init system to choose, but I do think OpenRC would be a strong contender. I think allowing them to use a compatible init system should be ok too, if somebody wants to do that. Oh, yes, good point. I was thinking more in terms of two different init systems with different preferred configuration files. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
Michael Gilbert mgilb...@debian.org writes: Doesn't a TC mandate on the default init system in some sense violate Debian's spirit of meritocracy? I believe that we have enough information to make an informed choice already, and that the sides are fairly well-defined and hardened in their opinions. That means that this dispute falls under section 6.1.2 of the constitution: Decide any technical matter where Developers' jurisdictions overlap. In cases where Developers need to implement compatible technical policies or stances (for example, if they disagree about the priorities of conflicting packages, or about ownership of a command name, or about which package is responsible for a bug that both maintainers agree is a bug, or about who should be the maintainer for a package) the technical committee may decide the matter. Regardless of how we structure the installer, we need to have a default init system (unless we plan on making every user choose, which I would dismiss out of hand as a horrible UI experience for the average user, who really doesn't care). We have to mandate support for at least that default init system. Realistically, the ability of the Debian developer community to support more than one init system is limited, since any given system is generally only going to run one. That means the level of quality of integration is going to drop off significantly relative to the default init system, particularly over time. Init systems are not like desktop environments: they require work by a huge swath of the developer community, and the average user does not generally switch from one to the other. The reality is that either systemd or upstart is fully capable of everything the typical user is going to need (for that matter, sysvinit is capable of most of what the typical user needs); there isn't the same driving force of user preference that leads users to switch between desktop environments, and quite a bit more integration support is required for the init system. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On 12/30/2013 04:31 PM, Michael Gilbert wrote: On Mon, Dec 30, 2013 at 1:47 PM, Russ Allbery wrote: 4. Conclusions I previously argued that much of the benefit of a new init system comes from when we can stop maintaining init scripts. I still believe that, but after thinking more about the cultural and project issues at stake here, as well as the immediate needs for a clean upgrade path, I ended up at more of a compromise position than I expected. I believe Debian should take this path forward: 1. We should select a new init system for jessie, either systemd or upstart. Support for that init system should be release-critical, but only in the sense that the daemon works properly under that init system. In other words, use of the sysvinit compatibility of either init system is acceptable support for jessie. 2. All packages providing init scripts must continue to support sysvinit scripts through the jessie release. Such support will continue to be release-critical. This is going to be painful for packages that want to do an early conversion, since it means testing two different init systems for this release cycle, but I think this is the right thing to do regardless for a clean upgrade path and Debian's normal robust committment to upgrades. Here it has the additional advantage of giving the non-Linux ports some breathing space to strategize. 3. Related, up through the jessie release, packages must (where possible; it's possible there will be cases where this is simply impossible) support switching back and forth between the new default init system and sysvinit. This means that configurations should not be moved out of /etc/default and that startup files for the new init system should read the same configuration that the existing sysvinit scripts use (or both should be modified compatibly). 4. Post-jessie, support for sysvinit will no longer be release-critical, and package maintainers will no longer be expected to ensure that it continues working. However, for as long as Debian has accepted non-Linux ports using a different init system, package maintainers should continue to ship init scripts if they work and should apply patches and other reasonable fixes from porters for those init scripts. In other words, this should be treated the same as merging patches for the Hurd to remove hard-coded constant assumptions: if the change is reasonable and doesn't break Linux ports (and this should be fairly easy to handle for nearly all cases with init scripts), the package maintainer should merge it. 5. Support for the other init system that was not chosen should be handled in the same fashion, should a team choose to pursue it. If we select systemd, package maintainers should still be willing to merge contributed upstart configuration, with the understanding that they can't test it and any support is on a best-effort basis only. Similarly, if we select upstart, package maintainers should be willing to merge systemd unit files and to enable upstream systemd support where requested and where it doesn't interfere with the operation of the daemon under upstart, with the understanding that the package maintainer is not committing to testing or directly supporting this configuration. 6. Debian's non-Linux ports should either use the same init system as Debian's Linux ports or agree on an init system that they're both going to use. The porting work is going to be hard enough without the ports going in different directions on which secondary init system they want to use. I prefer to leave it up to the porters to decide which init system to choose, but I do think OpenRC would be a strong contender. 7. After jessie, functionality between systems running the primary Linux init system and other init systems (including non-Linux ports) should be allowed to drift. In other words, there will be cases where features will only be available with the primary init system. Possible examples include security hardening, socket activation, automatic daemon restarts, and so forth. Packagers are under no obligation to port those features to other init systems, but should welcome and merge patches that do so. After jessie, packagers will no longer be required to preserve daemon configuration when the init system is switched, so use of such facilities as modification of upstart configuration files or systemd overrides may be used. We should revisit this decision again after the jessie release in case the situation has substantially changed. Doesn't a TC mandate on the default init system in some sense violate Debian's spirit of meritocracy? May I suggest something like the following (this isn't meant to be definitive) as a more meritocratic approach: 1. The TC encourages a team (probably debian-boot) to
Bug#727708: loose ends for init system decision
On Mon, Dec 30, 2013 at 5:51 PM, Russ Allbery wrote: Michael Gilbert writes: Doesn't a TC mandate on the default init system in some sense violate Debian's spirit of meritocracy? I believe that we have enough information to make an informed choice already, and that the sides are fairly well-defined and hardened in their opinions. That means that this dispute falls under section 6.1.2 of the constitution: I entirely concur that the social problem resides rightly within the jurisdiction of the TC. With that said, however, it is worth considering whether the role of the TC may be more effective if directed at the root (the social), rather than the branches (the technical choice), of the problem. The key, I think, is for the TC to provide a reasonable path for those currently identifying with any of the hardened camps to redirect their negative energy away from argument and toward something more positive: technical work and actual code. Regardless of how we structure the installer, we need to have a default init system (unless we plan on making every user choose, which I would dismiss out of hand as a horrible UI experience for the average user, who really doesn't care). As stated in my suggestion, initsel would of course always have a default, and only expert users would be empowered to travel the road less traveled. The no default idea that gets thrown about a lot is, of course, infeasible as a matter of practicality. Init systems are not like desktop environments: they require work by a huge swath of the developer community, and the average user does not generally switch from one to the other. I think, on the contrary, the nmu procedure is quite effective at enabling a small subset of the developer community (those that have a strong shared interest) to effect large changes across the entire archive (of course when maintainers stay out of the way,). This process has been demonstrated many times (although slow-going) for lots of other sweeping changes (e.g. buildflags, usr/share/doc, etc.). Best wishes, Mike -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
❦ 30 décembre 2013 23:31 CET, Michael Gilbert mgilb...@debian.org : Doing something like this, the best init system can win based truly on merit (if/when the work gets done), rather than as a fuzzy upfront judgement call. Unfortunately, being the best init is the not only the matter of its maintainers. A good integration implies to modify some packages whose maintainer may be hostile to some init and may consider that their package work well enough to not enable some feature needed by some init. I am pretty vague, by I believe that the TC has currently a case for such an example. Only once (and if) systemd gets to be the default init, we will be able to leverage all its abilities by full integration into Debian. -- Don't compare floating point numbers just for equality. - The Elements of Programming Style (Kernighan Plauger) signature.asc Description: PGP signature
Bug#727708: loose ends for init system decision
Michael Gilbert mgilb...@debian.org writes: On Mon, Dec 30, 2013 at 5:51 PM, Russ Allbery wrote: I believe that we have enough information to make an informed choice already, and that the sides are fairly well-defined and hardened in their opinions. That means that this dispute falls under section 6.1.2 of the constitution: I entirely concur that the social problem resides rightly within the jurisdiction of the TC. With that said, however, it is worth considering whether the role of the TC may be more effective if directed at the root (the social), rather than the branches (the technical choice), of the problem. The key, I think, is for the TC to provide a reasonable path for those currently identifying with any of the hardened camps to redirect their negative energy away from argument and toward something more positive: technical work and actual code. Well, I think it's worth pointing out that my transition plan looks somewhat similar to your plan, as far as the jessie release. (Then it starts diverging.) Part of my goal in writing up that plan was, as you say, to try to provide a means for people who are committed to one system or the other to continue to work on what they're passionate about even if it's not chosen as the default init system. Whether they do so or not is up to them, of course, as it should be. However, I don't want to understate the amount of effort required to integrate with the init system across the distribution. I'm less pessimistic than Steve, but he's not wrong that the choice of a default init system will have sweeping consequences for what will work and what won't. This will particularly be the case if that init system supports capabilities beyond the sysvinit set. I do think it would be possible to maintain package compatibility with both systemd and upstart. That was something I was curious about and therefore explicitly tested as part of my evaluation, and was able to do so to my satisfaction. That said, I tackled a fairly simple daemon, and something like NFS support would require people with deep knowledge of each supported init system to maintain that support. I don't think it's a good idea to ask everyone to pursue all paths in parallel. I think Debian does a bit too much of that right now. We should pick a winner that we believe is technically superior and focus the mandatory, archive-wide effort on it. We should then *not get in the way* of people who also want to pursue alternative paths, but not assume that they will necessarily be successful, and not require that everyone get involved in that effort beyond the level of integrating patches that we currently expect for, say, the Hurd port. But, anyway, I don't think our positions are really that different. The main difference is that I think we should pick a default init system for jessie now, and you feel like we should do effectively an archive-wide bake-off and then go with whatever one achieves the best integration. I have to admit to a deep personal dislike of contests like that, since I find them stressful and demotivating and think they make the process of free software development less fun. I'd rather decide on our default and on the mandatory work now, so that everyone knows where they stand, and then let people make their own decisions about what they want to spend time on beyond the required minimum. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Mon, Dec 30, 2013 at 7:16 PM, Vincent Bernat wrote: ❦ 30 décembre 2013 23:31 CET, Michael Gilbert : Doing something like this, the best init system can win based truly on merit (if/when the work gets done), rather than as a fuzzy upfront judgement call. Unfortunately, being the best init is the not only the matter of its maintainers. A good integration implies to modify some packages whose maintainer may be hostile to some init and may consider that their package work well enough to not enable some feature needed by some init. That is a hypothetical, of course, but it is a very real possibility for any path that the TC decides here. However, selectable inits may (possibly counter-intuitively) reduce this likely-hood. Since that maintainer in the way also gets his init of choice, he may be less likely to oppose an alternative that doesn't in any real sense get in his way. However, if/when this does happen, it will be an interesting test of whether the TC has the political will to override an indignant maintainer with their 3-to-1 majority power. Best wishes, Mike -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Mon, Dec 30, 2013 at 7:20 PM, Russ Allbery wrote: Michael Gilbert mgilb...@debian.org writes: On Mon, Dec 30, 2013 at 5:51 PM, Russ Allbery wrote: I believe that we have enough information to make an informed choice already, and that the sides are fairly well-defined and hardened in their opinions. That means that this dispute falls under section 6.1.2 of the constitution: I entirely concur that the social problem resides rightly within the jurisdiction of the TC. With that said, however, it is worth considering whether the role of the TC may be more effective if directed at the root (the social), rather than the branches (the technical choice), of the problem. The key, I think, is for the TC to provide a reasonable path for those currently identifying with any of the hardened camps to redirect their negative energy away from argument and toward something more positive: technical work and actual code. Well, I think it's worth pointing out that my transition plan looks somewhat similar to your plan, as far as the jessie release. (Then it starts diverging.) The key differences are that it is more succinct, roles and requirements are defined, no init system is outright rejected, and the default is selected on demonstrable merit. Part of my goal in writing up that plan was, as you say, to try to provide a means for people who are committed to one system or the other to continue to work on what they're passionate about even if it's not chosen as the default init system. Unfortunately at least two camps will be entirely dejected by any TC mandate here. Whether they do so or not is up to them, of course, as it should be. However, I don't want to understate the amount of effort required to integrate with the init system across the distribution. I'm less pessimistic than Steve, but he's not wrong that the choice of a default init system will have sweeping consequences for what will work and what won't. This will particularly be the case if that init system supports capabilities beyond the sysvinit set. I do think it would be possible to maintain package compatibility with both systemd and upstart. That was something I was curious about and therefore explicitly tested as part of my evaluation, and was able to do so to my satisfaction. That said, I tackled a fairly simple daemon, and something like NFS support would require people with deep knowledge of each supported init system to maintain that support. I don't think it's a good idea to ask everyone to pursue all paths in parallel. I think Debian does a bit too much of that right now. We should pick a winner that we believe is technically superior and focus the mandatory, archive-wide effort on it. We should then *not get in the way* of people who also want to pursue alternative paths, but not assume that they will necessarily be successful, and not require that everyone get involved in that effort beyond the level of integrating patches that we currently expect for, say, the Hurd port. I don But, anyway, I don't think our positions are really that different. The main difference is that I think we should pick a default init system for jessie now, and you feel like we should do effectively an archive-wide bake-off and then go with whatever one achieves the best integration. And Debian ends up with not only apple pie, but pumpkin and blueberry pies, and of in a corner there will be others thinking about cinnamon-spiced apple and blueberry-raspberry and other never-before-seen flavors. What a wonderful bake-off that would be :) Think about how it would go over if the directors of that metaphorical bake-off forced all the participants to produce the exact same pie. No one would really want to participate, and winning would be entirely meaningless. It wouldn't be a bake-off at all. It would be more like a mass-production assembly line. Best wishes, Mike -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
On Mon, Dec 30, 2013 at 08:12:19PM -0500, Michael Gilbert wrote: Part of my goal in writing up that plan was, as you say, to try to provide a means for people who are committed to one system or the other to continue to work on what they're passionate about even if it's not chosen as the default init system. Unfortunately at least two camps will be entirely dejected by any TC mandate here. I don't agree with that conclusion. When it comes to technology choices, you win some and you lose some. If upstart wins, I will be happy. If systemd wins, I will also be happy, because it's long overdue that Debian *make a decision*; and for all that there are aspects of systemd that make me uncomfortable, it will still be far better than the status quo. Your proposal smells like the status quo. Namely, instead of the project making a decision and being able to all pull together in the same direction to provide the best possible OS, we will continue to coast, squandering efforts on preserving users' ability to make choices about things that no user should ever be asked to care about. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Bug#727708: loose ends for init system decision
Steve Langasek vor...@debian.org writes: When it comes to technology choices, you win some and you lose some. If upstart wins, I will be happy. If systemd wins, I will also be happy, because it's long overdue that Debian *make a decision*; and for all that there are aspects of systemd that make me uncomfortable, it will still be far better than the status quo. I concur with this. I freely admit that I fell in love with systemd while investigating both systems, but if we agree to use upstart, I will happily go add upstart configurations to all of my packages and make use of those features. It's still a massive improvement over what we have now. Furthermore, and more philosophically, Debian has to learn how to respect a governance process and make decisions. We spent way too much time and effort not making decisions in this project, which results in big flamewars and discomfort and everyone sniping at each other while little productive happens. Frequently, I think this is like tearing off bandages over the course of months. The cumulative pain is much higher than if we just made a decision and everyone knew where they stood and what reality they needed to adjust to. Also, I too am happy when we can successfully pursue all courses of action at the same time, but at the end of the day, we're trying to create a distribution, and that means integrating components, and that means deciding on integration protocols. It's not useful to try to integrate everything with everything else in every possible way at the same time. You just create an unmanageable combinatoric explosion of possibilities, and then someone who just wants to run a Debian system doesn't know which paths are supported and tested, and which paths have only been touched by one developer if that. We need to pick a winner. That doesn't mean we need to pick losers. It does mean that one solution is going to get better support than everything else, because that's what integration *means*. It doesn't mean, or shouldn't mean, that other solutions are impossible if someone wants to make them work; it just means that we're not going to *require* that they work in order to get one's software into Debian. We probably should have had this discussion for wheezy, to be honest. But regardless, we should pick a default, supported init system for jessie. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727708: loose ends for init system decision
We have been asked to decide the default init system for jessie. As I have just said, my conclusion is that we should choose upstart. However, we also need to decide whether we intend to allow users to choose otherwise. So we need to say what we expect of package maintainers in terms of support for other init system(s). It seems to me that the benefits of upstart integration are such that we should encourage transition to upstart jobs, subject to my concerns about the need for a new readiness protocol. I am not comfortable with the idea of /mandating/ use of upstart. We may discover that despite our adoption, upstart loses the war, or we may find that we want to switch to openrc or another contender. And, given the strong opinions on this topic, I think it is at the very least premature to tell our users and downstreams that they are on their own if they want to use systemd. At the moment, there is no meaningful compatibility layer between all these systems other than sysvinit scripts. This is unfortunate, but given the current state of development of both systems I think this is to be expected. So I'm sorry to say that I think we need to continue to maintain sysvinit scripts in jessie. The alternative would be to permit a package to drop sysvinit support if both systemd and upstart configuration were provided, but I'm not happy at this stage to burn our bridges like that. It would be worth researching some kind of compatibility options. Perhaps simple daemons with very formulaic upstart job files could have synthetic systemd units created. Or something. I think we should give package maintainers some guidance on what kinds of upstart or systemd patches should be accepted. Without this, it's likely we'll find ourselves adjudicating disputes that ought to have been settled in principle much earlier. I think that patches for either should: - use a non-forking startup method - avoid significant amounts of open-coded AF_UNIX socketry - avoid embedded copies of daemon library code - avoid extra build-dependencies (eg on libsystemd) - avoid extra runtime dependencies (eg on deb-systemd-helper) (If the current state of the art in readiness protocols persists that means that upstart patches using raise(SIGSTOP) ought to be accepted, but systemd ones rejected unless they can use socket activation.) We should recommend: - daemon authors and Debian maintainers should prefer command-line options to environment variables - whatever environment variables and fds are used, measures should be taken to avoid them being inherited by daemons' arbitrary children IMO we should treat native non-forking upstart support as we have in the past done for release goals, ie with a low threshold NMU policy of some kind. But as I say I would like to see us try to get agreement or near-agreement on an improved startup protocol. I would suggest that we allow (say) a month for this process, starting now. Before this is settled we shouldn't go around converting a whole bunch of daemons. And, there are two loose ends about systemd in particular: - There doesn't seem to be any Debian policy about how to provide systemd units in Debian; - systemd upstream and the Debian maintainers like to put much of the configuration in /lib; this is controversial and someone needs to decide whether this is appropriate. Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
