Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 05:10:58PM +, Laurent Bercot wrote:
> would it be possible for you to change the name of the project?

What about using "slew.rc" and changing the installation path from
`/etc/s6' to `/etc/slew'?  The change is not exactly trivial but already
much smaller than the `/etc/s6-init' / `/etc/s6-rc' merge I did before
releasing this project, and can be done in a few hours (the project itself
is not complex; the complexity on my part comes from the several machines
where it is delpoyed and has to be migrated :().

(Hint for the etymology: note how "6" is pronounced in Chinese ;)

>  - I now recommend using socklog[1] to implement syslogd.

I see.

> It will likely be called s6-frontend. It will be written in C, and
> provide a user-friendly interface for setting up a complete init
> system, as well as a one-stop-shop "s6" command; the goal is to
> improve usability and simplicity for users who are not longtime
> members of the process supervision gang.

To be honest, I find the idea not very appealing to me.  I did not want
to say about this before, but now I find it necessary before more energy
is spent in goals that I consider less meaningful.  Fortunately, only
`s6-linux-init-maker' is affected by this issue as of now, and I will
use it as an example.

I think the direct downstream of init/rc systems is distributions, not
end users which demand "usability" in the usual sense (actually, one of
the many problems with systemd is that it attempts to bypass distros
and directly provide "unified" features to end users).  After s6/s6-rc
becomes mainstream, `s6-linux-init-maker' will perhaps have, IMHO, an
embarrassing role: most end users would begin with scripts provided by
their distros, which might often be very different from those generated
by `s6-linux-init-maker'.

And the difference is not only in the utilities used, but also in the
functionalities offered: eg. conditional (re-)mounting of `/run'
according to results of `mountpoint -q /run', loading/saving the clock,
saving the "uncaught" logs.  This is honestly not to show off the
features provided by my project, but to show that attempts to encapsule
factors in real-world use cases of an init/rc system into a code
generator can be largely unproductive.

Instead, I think a better way is to provide a full "reference
implementation" of what you have in mind, with the code generation knobs
converted into comments (cf. the `devtmpfs' line in `rc.boot' in my
project) or notes in the documentation.  Let distros (and some
adventurous users), i.e. those who understand their requirements for
init/rc systems better, decide how to customise their systems: after
all, text editors are the ultimate code generators for plain text files.
If it is agreed that the Unix philosophy is essentially about minimising
the total complexity of a system, I guess people would consider this to
be the more Unix-ish way.

> However, if decisions about "where to put stuff" are easy enough
> to take, decisions about "what init scripts should I provide, what
> external software should I assume is present on the system" are
> definitely not.

> But there is a large gray area here: what is "reusable policy" (RP)
> and what is "distribution-specific" (DS)?

First of all, I would like to note that the files in my project are not
to be used exactly as is; instead, they are intended as "reasonable
approximations" considering a majority of use cases.  I currently use
Alpine and Void on my servers and desktops, and I deviate from the
published version of my project, more or less, on each and every
machine, and their versions differ from each other.  I consider this
phenomenon normal and healthy.

Considering the adoption of systemd, I think most longrun definitions
are reusable, at least in an approximate sense; oneshots can be more
volatile, but most are still reusable with a reasonable amount of
distro-specific customisation.  I do not consider package dependencies
to be a big problem: distros (and those adventurous users) will
naturally handle them when customising the init/rc system.  No better
way has been proposed to my knowledge: `s6-linux-init-maker' basically
solves it by depending on skaware, and systemd "solves" it by using its
own extremely bloated implementations.

> If it involves choosing between several external software packages
> that provide equivalent functionality, is it okay to hardcode a
> dependency, or should we provide flexibility (with a complexity cost)?

Regarding networking, I consider netifrc to be, though bloated, a
successful example.  As I mentioned, the functionalities of netifrc can
in principle be implemented using preprocessors in much cleaner ways,
and I guess the complexity cost that comes with the flexibility would
not be too big in this case.

Of course, this means you probably cannot use C exclusively.  If Unix
used a small language with S-expressions that can be both compiled and
interpreted, and with garbage collection enabled a

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Laurent Bercot]

s6.rc [1] is an attempt to bridge the gap between the elegant 
foundation

provided by s6/s6-rc and an init/rc system that implements the main
functionalities beneficial for distributions. s6.rc features a
preprocessor that generates source directories for use with s6-rc from
given templates. The preprocessing procedure is composed of multiple
tiny passes, which makes the preprocessor not only clean and powerful,
but also extensible. Using the preprocessor, s6.rc supports instanced
supervision, optional dependencies, in-place `up'/`down' scripts,
automatic connection between services and loggers, as well as
package-specific passes that can be plugged into the preprocessing
phase.


 Impressive work! Congratulations, Casper :)



[1] .


 I think the correct URL is: https://gitlab.com/CasperVector/s6-dot-rc

 I'm feeling bad asking about this seeing the amount of work you've put
into it, and I hope there's an easy way to achieve it, but: would it
be possible for you to change the name of the project?

 Rationale:

 - I feel that "s6.rc" is very easy to confuse with "s6-rc"; not when
it's pronounced, but when it's written. The s6 ecosystem is confusing
enough as is for non-specialists; I think I should write a general
documentation page explaining what package does what (and an index
saying which package the s6-foobar binary belongs to, for every value
of foobar). If s6 is to become popular, the confusion needs to be
reduced, not entertained; and it would be easier if software such
as s6.rc would have a more distinctive name.
 For instance, "anopa" does a good job of being pretty distinctive. :)

 - I obviously have no way of enforcing this, but I would very much
like to keep control of the "s6" prefix, for clarity: every package
starting with "s6" should come from skarnet.org and have a place in
the integrated s6 ecosystem. (Not to say that s6.rc does not have its
place, far from it, but I'd like people to be able to easily make the
distinction between "core" and "community".) So you're obviously
welcome to use "s6" as *part* of the name of your package, but I'd
appreciate it if the name didn't *start* with "s6".

 A few miscellaneous notes:

 - I now recommend using socklog[1] to implement syslogd. This is
not as elegant as having one process per syslog connection, but
socklog binds to a datagram socket, so it can be used with musl.

 Having one stream per syslog client is a good thing per se because
it obsoletes the need to identify the client in every log record;
but the killer advantage would be to do away with system-wide
regular expressions for log filtering, and that's not something
we have yet. Even when you pipe "s6-ipcserver ucspilogd" into
s6-log, your s6-log script is the same for every client, so the
regexes need to be global. A real improvement would be to have
a different log script for every client connection, so the log
filtering would really be local; but I haven't yet thought about a way
to design such an architecture. That's the main reason why I haven't
much pushed for SOCK_STREAM syslog() in musl; if we can come up with a
syslogd scheme that works without any global regexes, then we'll
have a real case for SOCK_STREAM adoption. Until then, socklog works.

[1]: http://smarden.org/socklog/

 - The next piece of the s6 ecosystem that I plan to write is precisely
an integration layer for all the various existing parts: s6-linux-init,
s6, s6-rc. It will likely be called s6-frontend. It will be written
in C, and provide a user-friendly interface for setting up a complete
init system, as well as a one-stop-shop "s6" command; the goal is to
improve usability and simplicity for users who are not longtime
members of the process supervision gang.
 When I went and asked distributions what functionality they needed
in order to use s6 as their default init system, this is by far the
thing they mentioned the most. So, it's in the plans. Unfortunately,
it will be some time before I can get to it, because I need to work
on money-making projects first. There is a possibility that work on
s6-frontend will be funded, but in any case it wouldn't be before one
year or two, so don't hold your breath. In the meantime, I'm very
happy that initiatives like s6.rc come to light.

 - Everyone knows that I insist on the separation between mechanism
and policy, that I try to write software implementing mechanism, and
that I believe that policy is the realm of distributions. Unfortunately,
the success of a few pieces of software that embed a lot of policy -
systemd for instance, but also OpenRC to some extent - has shown that
this distinction isn't always a key factor in success, and on the
contrary, making distributors' jobs easier by embedding policy could
be an advantage. As such, s6-frontend will include a lot of policy
(where to put s6 service directories, how to organize them, where to
put s6-rc source and compiled databases, where the main scandir is,
etc.) However, 

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 09:23:34PM +0800, Casper Ti. Vector wrote:
> [1] .

Should be .  The project used
`/etc/s6-init' / `/etc/s6-rc', and `/etc/s6/lib/s6.rc' used to be
`/etc/s6-rc/lib/s6-rc.rc', hence the old name.  Many thank to mobinmob
for notifying me of this :)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

[Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

s6.rc [1] is an attempt to bridge the gap between the elegant foundation
provided by s6/s6-rc and an init/rc system that implements the main
functionalities beneficial for distributions.  s6.rc features a
preprocessor that generates source directories for use with s6-rc from
given templates.  The preprocessing procedure is composed of multiple
tiny passes, which makes the preprocessor not only clean and powerful,
but also extensible.  Using the preprocessor, s6.rc supports instanced
supervision, optional dependencies, in-place `up'/`down' scripts,
automatic connection between services and loggers, as well as
package-specific passes that can be plugged into the preprocessing
phase.

[1] .

The init part of s6.rc is, in many aspects, similar to what one would
expect from s6-linux-init.  The stage 1/2/3 scripts shipped with s6.rc
additionally support features like setting the time using hwclock(8) or
other means before exec()ing `s6-svscan' (cf. the documentation for the
rationale), duplicating (while writing to `/run/uncaught-logs') verbose
`s6-rc' output to `/dev/console', automatic reboot under certain
circumstances (eg. when required by fsck(8)), and automatically saving
`/run/uncaught-logs/current' on shutdown.

The attached tarball is an s6.rc-based example setup tailored for Alpine
3.7.x; a VM image of the setup is available at [2].  Due to a lack of
expressiveness in execline, most scripts in s6.rc are written for the
Byron Rakitzis implementation [3] of the rc(1) shell from Plan 9, and
static-linked binaries for x86 [4] and x86_64 [5] are available for
download.  See the attachments for SHA512 checksums (signed using my
OpenPGP public key) of the tarball, the VM image and the rc(1) binaries.

[2] .
[3] .
[4] .
[5] .

Of course, this is just the beginning; a lot of work has to be done:

* Although s6.rc can be easily configured to be rock solid, it is fairly
  fragile to PEBKACs: eg. if you accidentally delete `s6.rc' and then
  reboot the system, the system can get into quite serious troubles.
  Sprinkling related scripts with a lot of "if something is unset then
  default to blah" clauses can avoid the problem, but seems to make
  the code bloated.  Is there a better solution?

* Many more service definitions and ancillary files are necessary for
  s6.rc to be useful for the general public instead of just a few users,
  so any contribution / suggestion is welcome.  BTW, I do not consider
  myself to be a really good documentation writer, so please tell me if
  you have a good idea on improving the s6.rc documentation.

* In principle, it is possible to implement something like Gentoo's
  netifrc [6] using s6.rc's preprocessor, and I believe it would be much
  cleaner than netifrc.  However, since I am not exactly familiar with
  the networking stuffs, perhaps it had better be done by someone else.

[6] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



alpine-s6rc-conf.tgz
Description: GNU Unix tar archive


SHA512SUMS.asc
Description: PGP signature