Re: logging services with shell interaction

[Casper Ti. Vector]

On Fri, Jun 23, 2023 at 01:48:53PM +0200, Ben Franksen wrote:
> thanks for the heads-up; this is certainly an interesting project, but for
> me to start playing with it only makes sense if and when it has matured to
> the point where there is a minimum of documentation (-h/--help or something
> like that) and ideally some sort of revision control, too. I may be (barely)
> able to debug such low-level C code if I notice it misbehaving but to
> reverse-engineer what it is supposed to do is beyond my abilities.

The source code pasted above are indeed of a preview nature; the more
formal documentation will probably be written in the internal testing
here.  recordln works quite like recordio and the command line usage
are identical; the former is just more line-oriented.  The command line
usage of iotrap and ipctee are printed when `-h' is given; below is a
brief summary of them.

iotrap works like the trap program from execline, but the signals
currently cannot be customised.  When iotrap receives a terminating
signal, the spawned subprocess is sent an EOF; after this, if the
subprocess does not exit before the timeout tunable by the `-t' option,
the previous signal is forwarded.

ipctee listens to a pair of input and output sockets, the first accepts
at most 1 connection and the latter multiple connections.  (The input
socket is for the IOC; the output socket is for "procServ clients".)
For each connected client, the bytes it writes are forwarded to all
other clients connected at the time.

(You may realise ipctee is essentially a "chatting server" for all
connected clients, and it may seem that ipctee can be furtherly
simplified by eliminating the input socket and treating all clients
fully equal.  However, for certain use cases, we may disallow writes
from the output socket; this is why there is a `-r' option for the
"readonly" mode, and why the program is called "ipctee" not "ipcchat".)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2024.09.30)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: A program that can get exactly the log of a supervised process?

[Casper Ti. Vector]

On Fri, Jun 23, 2023 at 03:39:39AM +0800, Casper Ti. Vector wrote:
> +  if (!argv[0] || !argv[argc1 + 1]) strerr_dief1x (100, "empty program");
>argv[argc1] = 0;

Well, these two lines should be swapped.  Sorry for the spam :(

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2024.09.30)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: A program that can get exactly the log of a supervised process?

[Casper Ti. Vector]

On Fri, Jun 23, 2023 at 01:44:19AM +0800, Casper Ti. Vector wrote:
> The source code of the program, logtee (licence: CC0), is available at:
> <https://cpaste.org/?fa30831511a456b7=#ECwUd1YaVQBLUokynQbRYZq5wvBvXXeXo3bQoeL2rL4L>

Sorry, patch (the first hunk is cosmetic):

--- logtee.c2023-06-23 03:33:57.233531347 +0800
+++ logtee.c2023-06-23 03:33:05.693672295 +0800
@@ -29,7 +29,7 @@
 *len = 0; return 1;
   }
   if (iopause_g (x, 2 - !x[1].fd, 0) < 0) strerr_diefu1sys (111, "iopause_g");
-  for (i = 0; i < 2; ++i) if (x[i].fd && x[i].revents) {
+  for (i = 0; i < 2; ++i) if (x[i].fd > 0 && x[i].revents) {
 r = fd_write (x[i].fd, buf + pos[i], *len - pos[i]);
 if (r < 0) {
   if (!i) return 0;
@@ -57,8 +57,9 @@
 int main (int argc, char const **argv, char const *const *envp) {
   int pir[2], fdc, fdw, argc1;
   PROG = "logtee"; ++argv; --argc;
-  if ((argc1 = el_semicolon (argv)) >= argc)
-{ strerr_dief1x (100, "unterminated block"); }
+  argc1 = el_semicolon (argv);
+  if (argc1 >= argc) strerr_dief1x (100, "unterminated block");
+  if (!argv[0] || !argv[argc1 + 1]) strerr_dief1x (100, "empty program");
   argv[argc1] = 0;
   if (pipe (pir) < 0) strerr_diefu1sys (111, "pipe");
   if ((fdc = selfpipe_init ()) < 0) strerr_diefu1sys (111, "selfpipe_init");

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2024.09.30)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: A program that can get exactly the log of a supervised process?

[Casper Ti. Vector]

On Sun, Oct 24, 2021 at 12:03:01AM +0800, Casper Ti. Vector wrote:
> Any idea on how the log "teeing" may be done cleanly (and portably
> if possible; something akin to `tail -f' seems unsuitable because of
> potential log rotation), and perhaps any flaw or redundancy in the
> design above?

The recordio program is extremely enlightening here:

* Instead of trying to communicate with the logger, our program can
  directly sit between the service and the logging pipe.
  (This also solves the lifespan issue cleanly, saving lots of code;
  the latter is apart from the lots of code to talk with the logger.)

* When forking, we may let the parent (instead of the child) exec()s
  into the service, so our program does not need to forward signals.
  (But the loggrep subprocess from our program may die after the
  supervisor receives SIGCHLD from the service; therefore in order
  not to interfere with the next run of the service, we should ensure
  the loggrep program would exit on EOF and does not produce spurious
  outputs on its stdout.)

The source code of the program, logtee (licence: CC0), is available at:
<https://cpaste.org/?fa30831511a456b7=#ECwUd1YaVQBLUokynQbRYZq5wvBvXXeXo3bQoeL2rL4L>
It may be tested with (logtee is itself init-agnostic; writing an
adapter to, for instance, s6, should be an easy exercise in the
manipulation of argv):


#!/bin/execlineb -P
fdmove -c 3 1
logtee {
  fdmove 1 3 
  if { timeout 3 grep -q "bytes from" }
  echo
}
fdclose 3
fdmove -c 2 1
ping -i 10 example.com


-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2024.09.30)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: logging services with shell interaction

[Casper Ti. Vector]

On Thu, Oct 21, 2021 at 02:01:29AM +0800, Casper Ti. Vector wrote:
> As has been said by Laurent, in the presence of a supervision system
> with reliable logging and proper rotation, what `procServ' mainly does
> can be done better by something like `socat' which wraps something like
> `recordio', which in turn wraps the actual service process (EPICS IOC).
> The devil is in the details: most importantly, when the service is to
> be stopped, the ideal situation is that the actual service process gets
> killed, leading to the graceful exit of `recordio' and then `socat'.

It is found that socat does not do I/O fan-in/fan-out with multiple
clients; it also assumes the `exec:'-ed subprocess is constantly present
(i.e. it does not handle IOC restarting).  So I have written a dedicated
program, ipctee (see below for link to source code), that does this.
I have also written a program, iotrap, that after receiving a
terminating signal, first closes the stdin of its children in the hope
that the latter exits cleanly, and after a tunable delay forwards the
signal.  This way IOCs are allowed to really run their clean-up code,
instead of just being killed instantly by the signal.

> So the two wrapping programs need to propagate the killing signal, and
> then exit after waiting for the subprocess; since `procServ' defaults
> to kill the subprocess using SIGKILL, `recordio' also needs to translate
> the signal if this is to be emulated.  `socat' does this correctly when
> the `sighup'/`sigint'/`sigquit' options are given for `exec' addresses,
> but its manual page does not state about SIGTERM.  `recordio' does not
> seem to propagate (let alone translate) the signal; additionally, its
> output format (which is after all mainly used for debugging) feels too
> low-level to me, and perhaps needs to be adjusted.

Closer inspection of recordio revealed that it was designed in a smarter
way: after forking, the parent exec()s into the intended program, and
the children is what actually does the work of I/O forwarding.  This way
recordio (the children) does not need to forward signals.  Based on it,
I have written a program, recordln, that performs more line-oriented
recording: line fragments (without the line terminator) that go through
the same fd consecutively are joined before being copied to stderr.

> At the facility where I am from, we use CentOS 7 and unsupervised
> procServ (triple shame for a systemd opponent, s6 enthusiast and
> minimalist :(), because we have not yet been bitten by log rotation
> problems.  It also takes quite an amount of code to implement the
> dynamic management of user supervision trees for IOCs, in addition
> to the adjustments needed for `recordio'.  To make the situation even
> worse, we are also using procServControl; anyway, I still hope we can
> get rid of procServ entirely someday.

Source code for the programs above are available (licence: CC0) at
<https://cpaste.org/?fa30831511a456b7=#ECwUd1YaVQBLUokynQbRYZq5wvBvXXeXo3bQoeL2rL4L>
These programs can be tested with (in three different terminals):
$ ipctee /tmp/in.sock /tmp/out.sock
$ socat unix-connect:/tmp/in.sock exec:'recordln iotrap /bin/sh',sigint,sigquit
$ socat unix-connect:/tmp/out.sock -
Please feel free to tell me in case you find any defect in the code.
The dynamic management of IOC servicedirs is being developed, and will
be tested internally here before a paper gets submitted somewhere.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2024.09.30)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: logging services with shell interaction

[Casper Ti. Vector]

On Sun, Oct 24, 2021 at 10:36:06PM +0200, Ben Franksen wrote:
> I am looking forward to it. You may want to post a link when it's
> done, here or on the EPICS mailing list.

The paper has been published on JSR, and is now available at

(arxiv:2204.08434).  Half of the paper is spent on (forgive my
bluntness) updating certain EPICS-related practices from 1990s to
~2010, which may be quite underwhelming to readers of this mailing
list.  However, I do find one aspect of the paper potentially useful,
and not only so in connection to EPICS: systematic efforts to minimise
the complexity in configuring a perhaps large system composed of often
specialised hardware and software.

Inspired by theories like the Kolmogorov complexity, we can ask: to
what limit can we reduce the amount of code and manual operations in
building a ready system (X-ray beamline, computer cluster, ...) from
commodity hardware, so that the total workload is minimised?  I would
like to note that the idea was not born from vacuum, as similar ideas
can be seen, for instance, from Guix SD's whole-system configuration
mechanism.  My idea actually originated independently from something
like 
(dating back to ~2011), but I guess there must be many other people
who have developed similar ideas.

BTW, as the review process was too slow, the codebase of our packaging
system (see  for a fully
open-source edition) has evolved quite a little after submission of the
original manuscript.  One change of perhaps general interest is a small
inheritance system for RPM specs, obviously motivated by its Gentoo
counterpart; I am mildly confident that the repository in its current
state is capable of being a easy-to-use yet maintainable workalike of
the NSLS-II repository.  Another paper by us and published on JSR is
at 
(arxiv:2203.17236); readers of this list may find it of some interest
in implementing GUIs based on AutoCAD-like "command injection", and may
find the idea of an EPG immediately familiar after the discussion adove.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [announce] skarnet.org Winter 2021-2022 release

[Casper Ti. Vector]

On Wed, Dec 22, 2021 at 08:48:26AM +, Laurent Bercot wrote:
>  I want to keep the "stages" framing for s6-linux-init, because I think
> it is useful: these are qualitatively different parts of the init
> process. (There's a "stage 3" and a "stage 4" as well, at shutdown
> time: they're handled by the s6-linux-init-shutdownd daemon.) The
> framing is actually *more* useful here than in runit, where "stages"
> are only sequence points and the only real hardcoded meaning is that
> the system shuts down when /etc/runit/3 exits.
 
I would like to note that the old-fashioned way to shut down -- letting
`s6-svscan' exec() into a "stage-3" script, which performs the halting
procedure -- is still available from slew and legacy s6-linux-init.
I personally find it simpler to understand, in the cost of a lack of
full compatibility with `shutdown' without relying on some `atd':
.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: A program that can get exactly the log of a supervised process?

[Casper Ti. Vector]

On Mon, Oct 25, 2021 at 12:37:41PM +, Laurent Bercot wrote:
> Another question is how to piggyback loggrep into the notification
> mechanism: if loggrep is tied to the logger and not to the service,
> it doesn't have native access to the notification pipe. That means a
> specific mechanism is needed to give it cross-service access.

As has been sketched in previous mails, make loggrep a double-forking
chainloader, where the grandparent blocks (before closing the
notification fd and then chaining) until the grandchild is ready to
accept logs (after connecting to the logger and spawning the "grep";
the latter perhaps being an awk script in most cases)?

> But it's probably the least bad way to do it, [...]

I fullly agree.  This is the crux of the problem.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: A program that can get exactly the log of a supervised process?

[Casper Ti. Vector]

On Sun, Oct 24, 2021 at 06:20:53AM +, Laurent Bercot wrote:
>  So basically, either loggrep is a simple tee-like program but you
> weaken the supervision properties of the service, or the functionality
> needs to be embedded in the supervision architecture, with loggrep
> being a consumer for service and a producer for logger (which is
> easy with s6-rc but not so much with pure s6) and loggrep always
> watching the state of service (which is not so easy with s6-rc, where
> you don't know the full path to another service directory).

Well, I do realise the lifespan issue of the loggrep program, which is
why I asked the question in the first place.  But I really never thought
of directly inserting loggrep into the logging chain as a new node;
instead, what I have thought is making loggrep a program "attachable" to
the logger.  That is, the logger is extended with a listener which at
least one client can connect to, and which upon connection tees the log
to the client.  I do not know whether you have similar ideas.

Additionally, I have just noticed a race condition in the origin design
in my question.  If loggrep is double-forked by an external program
(execline or so), the daemon may become ready before loggrep is ready to
receive logs; a better way is to let loggrep do the double fork itself,
after the teeing pipe is ready.  The downside is that errors deemed
tolerable by the daemon but fatal by the user would then need to be
explicitly handled by loggrep; but this is probably already over-design,
so perhaps it just does not matter...

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: logging services with shell interaction

[Casper Ti. Vector]

On Sat, Oct 23, 2021 at 05:48:23PM +0200, Ben Franksen wrote:
> Interesting, I didn't know you are from the accelerator community!

(Actually I have only been in this field for 2.5 years...)

> I agree. BTW, another detail is the special handling of certain control
> characters by procServ: ^X to restart the child, ^T to toggle auto-restart,
> and the possibility to disable some others like ^C and especially ^D; which
> is not only convenient but also avoids accidental restarts (people are used
> to ^D meaning "exit the shell").

These functionalities would need to be (and would perhaps have better
been) done outside of the `socat'/`recordio' pair, as separate commands
(like `s6-svc -k ...' or `touch .../down') or wrappers.  `socat' simply
exits upon ^D/^C by default, so the IOC would not be hurt; I find this
enough to prevent most user errors, therefore more filtering of control
characters seems unnecessary.

> Our approach uses a somewhat hybrid mixture of several components. Since the
> OS is Debian we use systemd service units, one for each IOC. They are
> executing `/usr/bin/unshare -u sethostname %i runuser -u ioc -- softIOC-run
> %i` which fakes the host name to trick EPICS' Channel Access "Security" into
> the proper behavior, and then drops privileges. softIOC-run is the script of
> which I posted a simplified version, with the pipeline between procServ and
> multilog. Despite the disadvantages explained by Laurent, so far this works
> pretty well (I have never yet observed multilog to crash or otherwise
> misbehave). Finally, the configuration for all IOCs (name, which host do
> they run on, path to the startup script) all reside in a small database and
> there are scripts to automatically install everything, including automatic
> enabling and disabling of the service units.

Frankly I find the above a little over-complicated, even discounting the
part about CA security which we do not yet involve.  I think you might
be going to find our paper (after publication; it is to be submitted the
next week) interesting in simplifying IOC management.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

A program that can get exactly the log of a supervised process?

[Casper Ti. Vector]

Many daemons output some lines in their logs (here assumed to have been
redirected to logging pipes in some supervision framework) that can be
used as a kind of markers for readiness.  So if we can implement the
following program, we will able to trivially add readiness support to
these daemons without patching their source code, while reducing the
daemon-specific code (the pattern recogniser) to a minimum:

* The program is double-forked from the ./run script of a service, and
  also watches for the death of the daemon: in case the daemon dies
  before logging the readiness marker (and optionally, if a specified
  timeout is reached), this program dies as well, killing its subprocess
  (the pattern recogniser, discussed below).  To maximise portability
  across supervision frameworks, the death notification can be
  implemented with a pipe that can be written to by an ancillary
  program ran from the ./finish script.

* The program somehow (directly or indirectly) gets a copy of the
  log from the reading side of the logging pipe, and feeds it into a
  specified pattern recogniser, which should exit 0 upon finding the
  readiness marker.  A special exit value of the subprocess can be
  considered a notification for fatal error erroneously ignored by the
  daemon, leading to the program exiting some special value which can
  be handled by a `s6-svc -t' invocation.  In order to avoid potential
  feedback loops, the subprocess should be killed (and waited for)
  before the program outputs any warning; the subprocess should be
  responsible for keeping itself quiet when running normally.

Any idea on how the log "teeing" may be done cleanly (and portably
if possible; something akin to `tail -f' seems unsuitable because of
potential log rotation), and perhaps any flaw or redundancy in the
design above?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: logging services with shell interaction

[Casper Ti. Vector]

On Wed, Oct 20, 2021 at 09:53:58AM +0200, Ben Franksen wrote:
> Interesting, I didn't know about recordio, will take a look.

Hello from a fellow sufferer from EPICS.  (If you see a paper on some
synchrotron-related journal in a few months that mentions "automation
of automation", it will be from me, albeit not using a pseudonym.
Another shameless plug: .)

As has been said by Laurent, in the presence of a supervision system
with reliable logging and proper rotation, what `procServ' mainly does
can be done better by something like `socat' which wraps something like
`recordio', which in turn wraps the actual service process (EPICS IOC).
The devil is in the details: most importantly, when the service is to
be stopped, the ideal situation is that the actual service process gets
killed, leading to the graceful exit of `recordio' and then `socat'.

So the two wrapping programs need to propagate the killing signal, and
then exit after waiting for the subprocess; since `procServ' defaults
to kill the subprocess using SIGKILL, `recordio' also needs to translate
the signal if this is to be emulated.  `socat' does this correctly when
the `sighup'/`sigint'/`sigquit' options are given for `exec' addresses,
but its manual page does not state about SIGTERM.  `recordio' does not
seem to propagate (let alone translate) the signal; additionally, its
output format (which is after all mainly used for debugging) feels too
low-level to me, and perhaps needs to be adjusted.

At the facility where I am from, we use CentOS 7 and unsupervised
procServ (triple shame for a systemd opponent, s6 enthusiast and
minimalist :(), because we have not yet been bitten by log rotation
problems.  It also takes quite an amount of code to implement the
dynamic management of user supervision trees for IOCs, in addition
to the adjustments needed for `recordio'.  To make the situation even
worse, we are also using procServControl; anyway, I still hope we can
get rid of procServ entirely someday.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Path monitoring support in s6-services

[Casper Ti. Vector]

On Wed, Feb 17, 2021 at 01:08:44PM +0530, billa chaitanya wrote:
> I am trying to start a service when a file/path is
> modified/touched/created.Do we have any mechanism in s6 that supports
> enabling a service up on monitoring a path?

inotifyd (or something similar) + s6-svc (or s6-rc)?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Mon, Feb 15, 2021 at 02:54:52PM +, Laurent Bercot wrote:
>  The options! The options need to be all compatible. :) And for
> "shutdown", they would never implement a wrapper themselves, I would
> have to do it for them - which is exactly what I did, although it's
> a C program that actually implements shutdown, not a wrapper around an
> atd program I can't assume will be present on the system.

OK, now I understand their excuse.  Nevertheless I still do not think
all these necessarily require something like `shutdownd'; even in the
absence of `atd', chainloading a backgrounding timer for `shutdown' is
not a big exercise with execline (which is perhaps exactly what you have
already done in `s6-linux-init-maker').

>  What army? By the time the final kill happens, the service manager
> has brought everything down, and shutdownd has cleaned up the scandir,
> only leaving it with what *should* be restarted. You seem to think
> I haven't given these basic things the two minutes of attention they
> deserve.

Sorry then, I did not see that in the documentation; now the scandir
cleanup contributes some additional complexity.  Since the mechanism
behind `shutdownd' does not seem to be adequately explained at least to
me, here I explicitly do not conclude this addition is worthy or not.

>  Conceptually, the "old-fashioned" approach may be simpler, yes.
> Implementationally, I disagree that it is, and I'll give you a very
> simple example to illustrate it, but it's not the only thing that
> implementations must pay attention to, there are a few other quirks
> that I've stumbled upon and that disappear when s6-svscan remains
> pid 1 until the very end. [...] after ["wait { }"], you need to make
> sure to unmount filesystems immediately [...]

This is not exactly what older s6-linux-init actually do, which has
been mimicked by slew.  As long as the procedure between `wait { }' and
`umount' does not produce orphans, the `umount' will be fine.  I have
noticed you saying "a shell does not give ordering guarantees when it
gets a SIGCHLD", but it seems to me that the no-orphan requirement can
be verified by ensuring no commands involved gets backgrounded.  Of
course, feel free to correct that; more importantly, may I request you
to list the quirks you have encountered?  Only by that may we really see
how much the remaining `s6-svscan' brings, in comparison with how much
it takes (see my paragraph above).

> If your shutdown sequence is e.g. written in Lisp, and your Lisp
> interpreter handles pid 1 duties correctly, okay, that's fair, but
> that's *two* programs that need to do it, when one would be enough. [...]
>  The fundamental difference is that the current s6-linux-init hardcodes
> a lot of things in stage 1, purposefully. Yes, it is less flexible -
> though you *still* have a stage 1 hook if you really need it - but the
> whole point is to make stage 1 entirely turnkey and foolproof [...]

When mentioning Lisp, I did not mean to imply Lisp interpreters, but
optimising Lisp compilers, which blur the border between scripts and
compiled programs (cf. `fdclose' and `fd_close()').  But you have said
the problem is not about scripting, so we do not disagree on this; with
this background, I do not quite understand your emphasis on stage 1 in
s6-linux-init -- do you mean somewhere that it prepares for `shutdownd'?

>  The "screwdriver and duct tape" feel does not come from the fact that
> those are scripts; it comes from the fact that the scripts run in a less
> forgiving environment where they have to provide the necessary guarantees
> themselves, as opposed to keeping using the framework that has been
> running for the whole lifetime of the system and that is still valid and
> helpful, even though for once you have to interact with it and tell it
> to stop supervising some services because we're shutting down - which is
> the exact kind of situation the supervision API was made for.

Now that scripting does not seem to be a major problem (which falsifies
my previous judgement that it was; sorry for that), the only crucial
issue is the costs and benefits of the supervision tree on halting.
So may I again request you to spare some time to explain the detailed
workflow behind `shutdownd', and the actual quirks that a remaining
`s6-svscan' helps to solve?  Perhaps current s6-linux-init and older
s6-linux-init (with derivatives like slew) are just software that suit
different niches (eg. sysvinit/systemd-minded audience vs. those who
accept daemontools-ish software well), which would be perfectly fine.

>  I also disagree that the script approach is shorter and/or clearer.
> It may be clearer to people who read a script better than a doc page
> (or C code), but I don't think it should matter as long as the doc is
> accurate; if it's not, that's what should be fixed. And the source code
> may be shorter with a scripted stage 1, for sure, but the code paths
> taken by the CPU are way shorter with the C version, and make 

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

(I am extremely sorry for delaying this mail so much.  I have just done
two major refactoring/overhaul projects in this vacation around the
Spring Festival, and still have one remaining.  These projects are a
part of my formal occupation, but I would not have much low-distraction
time best for this kind of work apart from vacations.  By the way, Happy
Chinese "Niu" Year.)

On Fri, Jan 29, 2021 at 03:48:09PM +, Laurent Bercot wrote:
>  Bear in mind that my eventual goal for s6 is distro adoption. And
> distro maintainers will find any and every excuse to reject it.
> Having a "shutdown" command that works exactly like sysvinit's
> shutdown is essential, because it deals with a major objection, which
> is incompatibility and user-unfriendliness.

I do not really understand their excuse here.  CLI incompatibility is
trivially solvable by creating links (or so) for `halt' / `poweroff' /
`reboot', and even the `shutdown' command can be a wrapper for an `atd'
based mechanism.  In case they complain about the implementation of the
CLI, the actual interface to `shutdownd' is not that similar to the
`telinit' interface (at least to the one I think it is) either.

>  The *absence* of a supervision tree after stage 2 is precisely what
> requires careful handling, and runit only works because Linux has
> that peculiarity that kill -9 -1 does not kill the emitter!
>  Having a supervision tree in stage 3 actually *helps* with the
> late shutdown procedure: shutdownd dies right after the kill (which
> would make it usable even on a system without the Linux specialcase)
> and is restarted by the supervisor for stage 4.

If I understand it correctly, letting `s6-svscan' exec() stage 3 also
achieves immunity to `kill -KILL -1'.  I also find this "old-fashioned"
approach conceptually and implementationally simpler than an army of
`s6-supervise' restarting only to be killed again, and a `shutdownd'
restarting to execute the halting procedure (see some kind of "state"
here?  Functional programmers do not hate it for nothing).  I know this
seems less recoverable than the `shutdownd' approach, but does that
count as a reason strong enough to warrant the latter approach, if the
halting procedure has already been distilled to its bare essentials
and is virtually immune to all non-fatal problems (that is, excluding
something as severe as the absence of a `reboot -f' implementation)?

> [...] More seriously, you're being unfair, because you're not locked
> in at all. You can use the new s6-linux-init and *still* do everything
> you were doing before: [...]
>  Besides, when systemd advocates paint sysv-rc shell scripts as
> "duct tape", they're *right*. sysv-rc (and OpenRC) scripts are loaded
> with boilerplate that only exists to compensate for the lack of a
> supervision infrastructure, and systemd, like any supervision system,
> does away with that. systemd has 99 problems, but rightly calling out
> oversized script scaffoldings ain't one. Its disingenuousness lies in
> pretending that an overengineered, opaque, all-encompassing, unescapable
> framework is better than the duct tape; and I think you'll find that
> s6-linux-init isn't quite the monster you seem to believe it is.

What I intend to express is that unconditionally correlating "a bunch
of [...] scripts" to "a 'screwdriver and duct tape' feel" is a typical
systemd fallacy.  You seemed to be confusing "scripts containing lots of
boilerplate" with "scripts that are minimised and clear".

>  So basically, all you're complaining about is that s6-linux-init-maker
> is not generating your preferred run-image layout out-of-the-box
> anymore. Well, you're an advanced user, you know what you are doing;
> the knobs and levers are *still all there*. The only binary that
> kinda hardcodes things is s6-linux-init itself, and if you give it a
> try, I'm pretty sure you'll like it, because there was never any reason
> to modify the core of stage 1 in the first place and what it does is
> what any kind of stage 1 needs to do, no matter what language it's
> written in.

According to Guillermo's observation about the behavioural similarity
between slew's `rc.boot'/`rc.halt' and the current mechanism with
s6-linux-init, if I understand the big picture correctly enough, the
fundamental difference between the approaches might be the difference in
languages (to avoid further digression, here I expressly avoid talking
about Lisp ;) and the attendant difference in dependencies.  Speaking of
the latter, I do not find declaring dependence on things like `rc' and
BusyBox really a problem to any packager of systemd.  Speaking of the
former, the "old-fashioned" approach is obviously more flexible; I have
also said that it is probably shorter and perhaps clearer.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Fri, Jan 29, 2021 at 09:57:43AM +, Laurent Bercot wrote:
>  It may cost more *to you*, but there is real and significant value
> in following existing interfaces that people are familiar with. Being
> able to just use "reboot" instead of the, uh, slightly less intuitive
> "s6-svscanctl -6 /run/service" to reboot your machine, is one fewer
> obstacle on the way to mainstream s6 adoption.

But even `s6-reboot' from older s6-linux-init, or `busybox reboot'
with slew can already do that...

>  Additionally, and maybe more to your liking, there are also technical
> benefits to never killing s6-svscan. Being able to assume that a
> supervision tree will be operational at all times, including during
> shutdown (and even in stage 4!), is really comfortable, it cuts down
> on a lot of specialcasing, it makes shutdown procedures recoverable,
> integration into various configurations easier (I'm thinking
> containers with or without a catch-all logger, for instance), and

There is some non-trivial trade-off: in short, the existence of the
supervision tree after stage 2 is by itself a kind of "special case"
(eg. search for "careful handling" in [1]).  I am also thinking about
an application scenario, where a supervision tree with a new s6 version
replaces the active tree with an old version.  This is somewhat silly:
it can be a little useful in case of major version bump, but is probably
better solved by complete reboot to completely get rid of all old things
(s6 or not, updated together) in the memory.

[1] .

> all-in-all has just less of a "screwdriver and duct tape" feel than
> a bunch of execline (or rc ;)) scripts.

I am very sorry, but I do feel a strong smell of systemd mindset here :(

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Thu, Jan 28, 2021 at 10:41:24PM -0300, Guillermo wrote:
> Out of curiosity, do you have a reason for wanting to keep the
> "old-fashioned way"? Is it a goal of your project to depend on s6 and
> s6-rc, but not current s6-linux-init? It seems to me that doing so
> would be easier. It even looks like you could use the current
> /etc/slew/init/rc.{init,fin} scripts (perhaps with minor adjustments)
> as s6-linux-init's rc.init and rc.shutdown for slew, respectively.

Not using s6-linux-init has never been an explicit goal, but using
static scripts was a natural choice when s6-linux-init only provided
`s6-linux-init-maker', which produced scripts that were not that
flexible.

Currently I do not understand the `s6-linux-init-shutdown(d)' way
well, so the old-fashioned way is retained at least for now, given its
simplicity in implementation and seemingly better flexibility.  Frankly
it is my intuition that the new way costs more than the old way, but
does not provide that much in return.  (Feel free to prove me wrong.)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Fri, Jan 29, 2021 at 12:07:11AM +, Laurent Bercot wrote:
>  I may change it back, but I don't think the current state is broken,
> because you're not supposed to access git.skarnet.org via HTTP(S)! :P

Actually I do visit the CGit web interface fairly often, using it as
a poor man's GitHub workalike :)  Perhaps I need to batch change all
 references in the UP2020 document to
...

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Thu, Jan 28, 2021 at 05:21:59PM +, Laurent Bercot wrote:
>  There is no really good solution, and I prefer a short, sharp pain
> (when things break) followed by relief (when they're fixed) to a long
> dull ache (maintaining compat code).

I see.  I personally prefer to retain compat code if said code is so
small that it can hardly be incorrect, especially when the breakage
(like kernel panics) can be very severe.  Arguably a major stylistic
difference.

>  You seem to have found the proper way of managing this with SIG files,
> but just in case: "s6-svscanctl -tb" will net you the old behaviour.

Now I see; thanks.  I also realised that the revised `s6-svc -X'
proposal would result in wrong behaviour when there exists a `./finish'
script, because the supervisor would exit early (and prematurely).

BTW,  seems to be returning empty HTTP replies
now; both  and  work as
expected though.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Thu, Jan 28, 2021 at 07:09:08PM +0800, Casper Ti. Vector wrote:
> Moreover, `.s6-svscan/finish' (linked to `rc.halt') will still need its
> $1 set to `reboot', `halt' or `poweroff' by `s6-svscan' on exec().

I did not realise the great simplification to the command line options
of `s6-svscanctl' would not have been possible if s6-svscan(ctl) need
to, for example, know about halt, poweroff and reboot.  Here I retract
the quoted statement; instead, I will rework the mechanism around
`.s6-svscan/SIG*' in slew, and yet attempt to make the behaviour
mostly backwards compatible.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

On Thu, Jan 28, 2021 at 06:08:36PM +0800, Casper Ti. Vector wrote:
> then move the `s6-svc -X' invocation from `rc.halt' into `rc.fin'

The `s6-svc -a' invocation in `rc.halt' needs to be moved accordingly.
Moreover, `.s6-svscan/finish' (linked to `rc.halt') will still need its
$1 set to `reboot', `halt' or `poweroff' by `s6-svscan' on exec().

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Some suggestions on old-fashioned usage with s6 2.10.x

[Casper Ti. Vector]

I did not actively follow the recent evolution of s6, and have just been
bitten badly by s6 2.10.x on my Alpine servers (where slew [1] is used
of course) when it comes along with other updates.

[1] 

First, kernel panic on booting.  With some tentative echo(1) invocations
(with I/O redirections to /dev/console when necessary) and messing with
console resolution (so I could see the outputs before the panic), I
found the problem occurred with `s6-svscan' exiting because of the
legacy `-s' option in [2].  The fix itself is trivial, but it would be
better if we kept the option supported for a transition period, and that
only removed it from the manual pages while urging users to get rid of
it.  After all, in this case, silently ignoring `-s' is behaviourly
similar to (if not perfectly compatible with) old `s6-svscan'.

[2] 

Second, `s6-svscan' now waits for its `s6-supervise' children to exit
before exec()ing `.s6-svscan/finish', so it hangs forever (save for
magic SysRq) due to the catch-all logger on halting.  I do know that the
recommended way to shut down is to use `s6-linux-init-shutdown', but
it will be nice if the old-fashioned way (with stage 1 and stage 3 as
static scripts) is supported as well after minimal modifications to both
s6 and (for instance) slew.  I also understand that `s6-svc -X' has been
removed, and that the invocation in [3] would no longer work anyway
because [3] is exec()ed by `s6-svscan'.  However, I think the following
way is practical yet minimal: introduce an option (perhaps still `-X')
of `s6-svc', but that tells `s6-supervise' to exit normally *upon
receiving SIGTERM or SIGHUP* (this is where the behaviour differs from
the old `s6-svc -X') without waiting for the children to exit; then move
the `s6-svc -X' invocation from `rc.halt' into `rc.fin' (where `s6-rc -d
change all' is also spawn).

[3] 

Any suggestions?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6-supervise: use of nosetsid

[Casper Ti. Vector]

On Thu, Dec 03, 2020 at 05:21:44PM +, Laurent Bercot wrote:
> https://github.com/dubiousjim/dcron/blob/master/main.c#L272
Thanks for the explanation; I will report it to the upstream.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6-supervise: use of nosetsid

[Casper Ti. Vector]

On Thu, Dec 03, 2020 at 04:46:58PM +, Laurent Bercot wrote:
>  Hence, my question to users: do you have a *valid* reason to use
> nosetsid files in your service directories? Are there use cases for
> nosetsid that I have not thought about, and that would make using s6
> impractical if the functionality were to be removed?

dcron crashes without it, which is why [1] exists.  The log says
`setpgid: Operation not permitted'.  I have not investigated further.

[1] 


-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: External health Check Process

[Casper Ti. Vector]

On Thu, Oct 22, 2020 at 03:34:37PM +, Laurent Bercot wrote:
>  I'm pretty sure that people in the community already have run script
> models for healthchecker services, if they could contribute them it
> would be awesome ;)

Just in case anyone finds this a useful prototype:

Mainly used here as of now:


-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update early logger logging path after remounting root as rw

[Casper Ti. Vector]

On Sat, Sep 05, 2020 at 11:46:44AM -0400, Rio Liu wrote:
> I'm configuring my linux to use s6-rc. It works fairly well so far. One
> thing I want to improve though, is that the early logger logs to
> /run/uncaught-logs. It's nice to have a log file during early boot and it
> helped my debugging easier. But it would be cool to able to change the
> location to a permanent location like /var/log after the root has been
> remounted as read-write.
> 
> Is it possible to update the log path of early/catch-all s6-log process to
> a new location, and perhaps copying the early logs there as well? Or if
> not, is it possible to spawn a new s6-log process that acts as a catch-all
> logger?

Your requirements of saving logs from /run/uncaught-logs and (mentioned
later) batch-producing `producer-for'/`consumer-for' files can be
automated using slew: .  (Other
encapsulations of s6/s6-rc also exist, but I am the author of slew ;)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [request for review] Port of s6 documentation to mdoc(7)

[Casper Ti. Vector]

On Tue, Sep 01, 2020 at 10:11:14AM +, Laurent Bercot wrote:
>  I'm totally willing to use a HTML schema we can negotiate, to write
> future documentation in. What I don't want to do is:
>  - Touch existing documentation, unless I have to rewrite the content of
> a page for some reason. Of course, if the conversion work is done by
> somebody else, I have no objection to upstreaming the new documents.

Of course it will be done by someone else :)

>  - Add heavy dependencies to the skarnet.org Makefiles.

Alternatively, if you find it acceptable, we can have an independent
project that hosts the code transformers.  What you need to care about
then is writing in the specified schema and merging documentation
patches (in case of schema violation).  The first patches will be fairly
large for obvious reasons, but I can separate them into relatively
easily-to-verify (small or generated by sed-like automation)
transformation passes, in the style of [1].

[1] 
(See the last sentence in the "About" section of README).

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [request for review] Port of s6 documentation to mdoc(7)

[Casper Ti. Vector]

On Tue, Sep 01, 2020 at 08:02:34PM +1000, Alexis wrote:
> That involves ~70 documents. Do you need all of them, or can i just provide
> a diff of a few examples? i don't currently have the HTML sources locally;
> i'd have to download them all.

Preferably some representative samples, thanks.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [request for review] Port of s6 documentation to mdoc(7)

[Casper Ti. Vector]

On Tue, Sep 01, 2020 at 07:03:36PM +1000, Alexis wrote:
> On the basis of my current experiences, it would be No Small Task to convert
> the current, presentationally-based, HTML documentation to markup that's
> sufficiently semantic to enable it to be mechanically converted to
> mdoc/roff. i estimate i've so far spent at least 30 hours over the last
> couple of weeks adding the relevant semantic (and occasionally
> presentational) macros, as well as modifying the page structure to match
> mdoc conventions; and i'm still not finished. Myself, i'm not willing to
> repeat this task again, as i have other volunteer and life commitments.

May I request a diff (via PM) between your attempt and the current HTMLs?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [request for review] Port of s6 documentation to mdoc(7)

[Casper Ti. Vector]

On Mon, Aug 31, 2020 at 08:51:34PM +, Laurent Bercot wrote:
>  - Unless, of course, someone comes up with the perfect solution (adding
> a DocBook dependency is not a perfect solution, and neither is
> generating HTML from mandoc), in which case, obviously, they would have
> the time and willingness to do all the necessary work themself, and in
> doing so, actually meaningfully contribute to the community instead of
> only adding their drop to the useless sea of It's Easy You Just Have To
> Do This, Just Saying In Case You Had Not Considered.

Sorry if this adds to your boredom in this subject, but I wonder whether
you find the following acceptable:

* We negotiate a HTML schema your documentation can be written in, which
  is based on current documentation; existing HTML documentation will be
  converted to the schema, with minimised changes.  (The structuredness
  of HTML helps; now you also see why knowing some Lisp is beneficial :)

* I provide a Python script that validates the schema (and also helps
  to convert existing documentation) of your HTML documentation, and
  transforms it into an IR suitable for downstream conversion.  I am
  quite comfortable with writing code transformations, but it may take
  some time -- about 1 months in this case, since I have my daily job.

* Those who are fluent in roff-related formats (I *do* like manpages,
  but I cannot write them, at least for now) write a program that
  transforms the IR into one pre-chosen format (we do not like multiple
  standards differing from each other subtlely, right?).

This way, the changes to the current documentation will be minimised;
the downside is a soft dependency on Python, which is only necessary
when the amount of modification to the HTML is so big that you cannot
manually compute corresponding modification to the manpages.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Logging in a web server context

[Casper Ti. Vector]

On Sat, Jun 13, 2020 at 12:50:50PM +0200, Carl Winbäck wrote:
> I gather from it that what I want is currently not possible without
> s6-rc. Is that correct?

I do not think so :)  In a nutshell, your service needs to connect its
stdout and stderr to a pair of different loggers; that can be done by
fd-holding and chainloading, and is how s6-rc internally implements its
piping (well, funneling) mechanism.  However, s6-rc only connects the
stdout, so the fd-holder needs an amount of additional configuration
whether you use s6-rc or not; that amount will be a bit big without
s6-rc, but it is surely doable and even scriptable.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Logging in a web server context

[Casper Ti. Vector]

On Sat, Jun 13, 2020 at 10:41:24AM +0200, Carl Winbäck wrote:
> Is it possible when using s6-svscan/s6-supervise to somehow arrange so
> that a daemon’s stdout is sent to one logdir and stderr to another
> logdir?

See ?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Readiness notification exemplars

[Casper Ti. Vector]

On Wed, Apr 01, 2020 at 08:23:26AM -0500, Brett Neumeier wrote:
> I am curious, does anyone on this list know of examples of such daemons? I
> am considering creating and submitting patches for some daemon programs
> that I use that do *not* support this mechanism as yet, and am curious if
> it is as simple as it looks like it should be.

s6-log for example?  Not claimed to be a daemon, but generally used as one:
.
(Search for `notif' in the file; yes, it is indeed very simple to implement.)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: runit SIGPWR support

[Casper Ti. Vector]

On Fri, Feb 14, 2020 at 05:06:40PM +0300, innerspacepilot wrote:
> If you read it carefully, there is NO signal for shutdown.
> Or lxd should also create file /etc/runit/stopit and set permissions for
> that?

Here runit provided a small machanism, and you also need a policy, like
those from Void or Gentoo.  Someone can provide a container template
with a given policy, and then the user does not need to care about
creating it by himself.  The same goes for many other init systems.

> Also what complexity you are talking about? runit.c is only 346 lines long.

Note the word "coupling" I used:
.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: runit SIGPWR support

[Casper Ti. Vector]

On Fri, Feb 14, 2020 at 11:46:16AM +0100, Jeff wrote:
> what should SIGPWR mean to a Linux init ?
> i would suggest: halt and power down the system ASAP.

Init signal semantics is already a mess:
.
When thinking of standardisation, we need to beware of this:
.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: runit SIGPWR support

[Casper Ti. Vector]

On Fri, Feb 14, 2020 at 04:39:28PM +0300, innerspacepilot wrote:
> You mean that adding few lines of code in one place is worse than many
> users of many distros must configure their containers?
> I can configure that myself, but I don't want every user of runit driven
> container to walk this path. Is it necessary?
> It adds complexity to users, and that means users will choose other
> distros which just work.

The burden is just a few lines of code anyway, and user only need to
learn it once.  It is also easily doable with some templating mechanism.

> Also there is a huge lack of documentation about it on the net,
> especially on signals that runit accepts.

.
Took me less than one minute.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: runit SIGPWR support

[Casper Ti. Vector]

On Wed, Feb 12, 2020 at 05:25:56PM +0300, innerspacepilot wrote:
> Why not just make runit systems run inside containers out of the box?
> We are talking about one/two lines of code.

Likewise `lxc.signal.halt' only needs one/two lines of code.  It is also
a interface with well-defined semantics.  Modifying runit source code
touches the implementation, with all the potential coupling between
modules and/or between submodules.  So the increase in total complexity
of the system is almost certainly smaller when using `lxc.signal.halt'.

> Why can't we be just a little bit more friendly to each other?

Let's see some more examples.  Some daemons (eg. mysqld and php-fpm)
use a signal other than SIGTERM for graceful shutdown, and s6 uses a
`down-signal' file (similar to `lxc.signal.halt') for these daemons.
Instead of modifying the daemons, we configure s6 specially; likewise,
here I would definitely prefer `lxc.signal.halt'.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: mdevd / udevd issues, and the issue of "reverse" dependencies

[Casper Ti. Vector]

On Mon, Feb 10, 2020 at 08:02:44PM +, Laurent Bercot wrote:
> There is no fundamental reason why it doesn't. inotify works on tmpfs;
> you don't need to poll for the existence of /run/udev/control, you can
> inotifywait for its appearance.

Thanks.  I just did not take inotify into consideration.

> That is true, but I shamelessly chalk that up to the shell's
> limitations: the shell can't create anonymous pipes outside of a
> pipeline, and it can't make a pipeline without forking both the reader
> and the writer. It's a terrible tool when you need semi-serious
> plumbing work.

Now I daydream again about the scsh-like mini-language I want :)

> It would be quite difficult to add a "udevadm settle" to mdevd. [...]
> However, depending on the kind of device you're waiting for, it may be
> possible to avoid the race for that device. I needed to wait for a
> network interface to appear after the coldplug, so I wrote a tool that
> does just that: bcnm-waitif, in the bcnm package. (Documentation coming
> soon.) So at least network interfaces are covered now.

Here I am more concerned about the coldplugging of disks, so that
loggers other than the catch-all can be started.  I guess Alpine folks
would feel at home with similar requirements in `nlplug-findfs'...

> I don't understand why you'd prefer the original way. The natural and
> normal way of proceeding is 1. start the daemon that processes the
> uevents, 2. trigger the initial hardware scan that produces a big
> batch of uevents. You don't need a temporary devd when you can just
> start the real one; if the interfaces around the existing devd
> implementations make it difficult to start properly, that's what should
> be fixed.

Frankly I was just attempting to find an excuse to introduce my idea of
"reverse" dependencies.  The most important (and perhaps only) advantage
of the original way is that the devd has its own logger; but as we have
noticed, even the bloated udevd usually plays nice with the catch-all.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

mdevd / udevd issues, and the issue of "reverse" dependencies

[Casper Ti. Vector]

I recently added service definitions for mdevd into slew, during which
I switched from a `devd' longrun (which has a dedicated logger, and so
requires a R/W filesystem) that depends on a `devices' oneshot (starting
a temporary devd process to coldplug basic devices) plus a `devd'
longrun to a single `devd' longrun (which starts early, and just logs to
the catch-all logger) followed by a `devices' oneshot (which just do the
coldplug).

This has obvious benefits, at least for now.  udevd does not have
a readiness notification mechanism (polling for the existence of
/run/udev/control surely does not count), and s6's fd-based mechanism
does not integrate well with the shell (at least I have not come up
with a way that avoids the hassle of setting up a temporary fifo and
dancing with foreground and "background" processes that play with the
fifo).  So the `devices' oneshot is destined to be complex if "up" is
not to be identified with "ready".  (Another issue, currently unsolved,
is that mdevd does not have a "udevadm settle" equivalent, so that in
theory we are not sure the basic devices are fully coldplugged when
`mdevd-coldplug' exits.)

In the above, you have seen that I switched from `devd' depending on
`devices' to the converse.  Suppose the problem of handling readiness in
the shell could be done in a "magically simple" way, I would personally
prefer the original way.  However, I would like the switch between mdev,
mdevd, udev etc to be painless, so `devd' is not necessarily present;
instead, I want `devices.mdevd' / `devices.udevd' to automatically drag
`devd' into the dependency tree, even if it is `devd' that depends on
`devices' instead of the converse.  This request is admittedly based on
a fictional premise, but the supposed relation between `devices' and
`devd' is similar to that between `wpa_supplicant' and `wpa_cli', and
the latter is not fictional.

I do not find this kind of "reverse" dependencies trivially doable in
preprocessors (like slew's `prep').  The strightforward way to make
service `srv1' reverse-depend on `srv2' is to create a bundle that
contains both `srv1' and `srv2', and then to make all that depend
on `srv1' actually depend on the bundle; but how should we name the
services?  To minimise rewriting of config files, we would naturally
considering renaming the original `srv1' to something else and let the
bundle take the name.  However, the scripts in the original `srv1' may
change its behaviour based on the service name (eg. `getty.tty1' and
`dhcpc.eth0'), so some careful planning are needed to prevent the
renaming procedure from crashing with this mechanism.

Any thoughts on these issues?  Thank you in advance.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Jan 05, 2020 at 02:31:52PM +0800, Casper Ti. Vector wrote:
> I admit that I am quite bad at quarreling with inherently malicious
> people on social networks, and now explicitly request you to help
> spreading the post -- Twitter, Medium, your private circle, whatever
> suitable -- and direct any useful feedback to me.  Please, and thanks.

Additionally, someone noted [1] that the r/linux moderator which banned
me may be LP himself.  It is my opinion that in case of that being true,
we should spread the correct information more actively, or otherwise we
will also have to dispel a lot of specially crafted misinformation.

[1] <https://www.reddit.com/r/initFreedom/comments/ejkd7r/_/fd6co72/>.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Jan 04, 2020 at 08:25:12PM +0200, fungal-net wrote:
> For the first time in 2.5yrs of blogging anti-systemd propaganda ONE
> article  has 10 time more visits in less than 24hrs than the whole blog
> (nearly 300 articles) has for a whole day.  So someone is interested.

(Somewhat) back to the topic.  I am also pretty sure that my Gentoo
Forums post will be interesting to those who are willing to read
technical analyses (i.e. not systemd fanboys), and that some people
would not like to see the awareness of that post growing quickly.
I admit that I am quite bad at quarreling with inherently malicious
people on social networks, and now explicitly request you to help
spreading the post -- Twitter, Medium, your private circle, whatever
suitable -- and direct any useful feedback to me.  Please, and thanks.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Jan 04, 2020 at 08:25:12PM +0200, fungal-net wrote:
> I wouldn't mind reposting this for you if you had asked me yesterday, or
> about 18 hours ago, as I have an older decorated account.

Many thanks, but I am afraid the submission would still not have met
their high "quality" standard [1], which requires someone to find my
post valid submit it at r/linux "naturally".  Even if you had reposted
it without my request, they would still have been able to assert the
submission was not "natural" because you and me are on the same mailing
list and could communicate privately -- what a coincidence!

[1] .

(And now let's stop the whining here; we are undeniably wandering OT.)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Fri, Dec 27, 2019 at 11:37:19PM +0800, Casper Ti. Vector wrote:
> Now I know the reason.  "Your account is too young.  Please wait
> at least 5 days to begin posting." --- /u/AutoModerator

Another try:
<https://www.reddit.com/r/linux/comments/ejk2tz/>.
Given the result, it seems that attempts at posting to r/linux would be
futile anyway; however, someone did this:
<https://www.reddit.com/r/initFreedom/comments/ejkd7r/>.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Dec 28, 2019 at 03:37:35PM +0200, Alex Suykov wrote:
> The reason I think it's mostly useless is because the only use case
> for cgroup supervision is supervising double-forking daemons, which
> is not a very smart thing to do. A much better approach is to get rid
> of double-forking and then just directly supervise the resulting long
> running process.
> I can't think of any other cases where it would be useful.

Another use case, as I outlined in the Gentoo Forums post, is to handle
orphans of badly written programs.  I remember seeing someone mentioning
certain GNOME programs behaving this way, which seems to be a reason
systemd began to kill nohup'ed processes.  The only program I have
witnessed to behave this way is cjdroute, the service program of cjdns,
which is part of why I dislike the current implementation of the latter
(as I mentioned in the UP2020 document).

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Mailing list archives?

[Casper Ti. Vector]

On Sat, Dec 28, 2019 at 11:55:31AM +0100, Carl Winbäck wrote:
> Are there any archives of the supervision list available online?
See ?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Fri, Dec 27, 2019 at 04:29:13PM -0500, Steve Litt wrote:
> What is slew?

>From the Gentoo Forums post: "[the example] also uses slew, a flexible
framework that thinly (~500 core SLOC, including ~200 for core service
definitions) wraps around s6/s6-rc to offer some features commonly
requested by distributions".

And from the mail [1] on this list where the name "slew" was given for
the first time: "hint for the etymology: note how "6" is pronounced in
Chinese ;)"

[1] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Dec 28, 2019 at 04:24:40AM +0200, Alex Suykov wrote:
> I don't think a tool like this has any actual uses, other than in
> arguments with systemd fans, but I guess that alone could justify
> its existance.

Thanks.  It's brilliant!

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Fri, Dec 27, 2019 at 09:54:11PM +0800, Casper Ti. Vector wrote:
> <https://www.reddit.com/r/linux/comments/egb4wp/>.
> It seems to be downvoted, which may be unsurprising given the generic
> sentiment toward systemd in r/linux.  Anyway, as of now I cannot load
> the post page (perhaps because of the Great Firewall, but somehow I can
> partially load the r/linux subreddit page), so I am not sure.

Now I know the reason.  "Your account is too young.  Please wait
at least 5 days to begin posting." --- /u/AutoModerator

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Thu, Dec 26, 2019 at 09:57:35PM -0500, Steve Litt wrote:
> Very, very nice! You should publicize this.

.
It seems to be downvoted, which may be unsurprising given the generic
sentiment toward systemd in r/linux.  Anyway, as of now I cannot load
the post page (perhaps because of the Great Firewall, but somehow I can
partially load the r/linux subreddit page), so I am not sure.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Fri, Dec 27, 2019 at 12:32:27PM +, Laurent Bercot wrote:
> Is there real pressure to have this?

AFAIK, the only pressure is from systemd fanboys.  But this is indeed
a biggest criticism from them; we would be able to save quite a lot of
flamewars if the feature was simply there.  Nevertheless I understand
the feature will be, frankly, a vase.

> The problem with such a "babysitter" is that it would need to forward
> signals, much like execline's trap program. It's ugly, and I'd rather
> have people not do that any more than strictly necessary.

We will also need to handle disgusting PID files for double-forking
services.  And in order to be safe in case the service crashes before
the PID file is created, we will perhaps need some kind of startup
deadline.  A big can of worms.

> As for cgroups-related chainloaders, I could probably write some in
> s6-linux-utils, but wasn't the cgroups interface designed to make
> sure those operations are trivial to implement as small scripts?

Well, this is a good idea.  I can even provide such library scripts in
slew, but the libraries will be `rc'-specific, and not used in the
traditional (exec()-based) sense of chainloading.

> I don't, for several reasons, one of which is that Google's homemade
> supervisor (which is... not great) is called "babysitter", and it
> triggers cringey memories.

:)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Thu, Dec 26, 2019 at 07:17:02PM +, Laurent Bercot wrote:
> That is awesome, and you are doing very important work.

Thanks :)

> Could somebody volunteer to track down all the similar documents we have,
> and make a list of links on a "meta" page?

I also wonder if someone on this mailing list is interested in actually
implementing a cgroup-based babysitter as is outlined in the post,
perhaps packaged together with standalone workalikes of the cgroup
chainloaders (`create-control-group' etc) from nosh?  I am still quite
a newbie to actual system programming in C...

[And I really like using the word "babysit" here, which comes with a
nice degree of derogatoriness without being excessive.]

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Fri, Dec 27, 2019 at 02:09:48AM +0100, Oliver Schad wrote:
> OMG - how much work was that? Great!

The post is mostly based on materials from the UP2020 document, but
optimised for length and with information added here and there; the
"Linux cgroups" and "Myths" parts are largely new materials.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Nov 17, 2019 at 02:26:44PM +0800, Casper Ti. Vector wrote:
> If you are a regular non-anonymous Slashdot reader, I will be very
> grateful if you can vote my recent comment up:
> <https://slashdot.org/comments.pl?sid=15221854=59420196>
> 
> [I know it is not quite decent to rally for votes in this situation, but
> I consider it some kind of "necessary evil" to raise the (long overdue)
> public awareness of daemontools-like init/rc systems.]

I have written a dedicated "s6/s6-rc vs systemd" post on the Gentoo
forums, which I believe have included convincing responses to most
common arguments by systemd proponents:
<https://forums.gentoo.org/viewtopic-t-1105854.html>.

In addition to providing arguments suitable as stock replies to systemd
proponents, that post also contains steps to build a small example
system based on s6/s6-rc/slew, which updates the Ubuntu example in
<https://skarnet.org/cgi-bin/archive.cgi?2:mss:2110>,
and may be useful to people interested in distro packaging of slew.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Oct 13, 2019 at 09:57:18PM +0800, Casper Ti. Vector wrote:
> Sorry, but there are currently no figures due to a limited time budget;
> I will probably make a plan about adding pictures and then implement it,
> perhaps before the end of this year if time permits.  As a further note,
> I am far from as good a teacher as V. I. Arnold or Dan Friedman (cf.
> the end of Section 16), and soon after actually beginning to write the
> document I found it really hard to accommodate for people with varied
> levels of backgrounds.  This is quite serious a barrier for newbies, and
> what has already been done is surely not quite enough; I will rethink
> about this often.

v0.1.3 of the document has been released, including multiple new
illustrations, a "How to read this document" section, and a rewritten
introduction to the notion of cohesion in Section 03.  I hope these
changes can make the document more accessible, and please feel free
to tell me (probably privately) if you have further suggestions.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Casper Ti. Vector]

On Sun, Dec 01, 2019 at 09:47:52PM -0500, Steve Litt wrote:
> Would it be acceptable to you and them to put the binaries in /bin/s6
> and then very early in the boot add /bin/s6 to the path? This isn't a
> lot different from what djb did with /command, except it's not off the
> root, which everyone seems to hate.

Or can we consider the Plan 9 style [1], which searches all relative
paths (in the broad sense, i.e. all which do not begin with `/', `./' or
`../') from $PATH, so we can install chainloaders into /bin/s6 and then
use `s6/cd' to run `/bin/s6/cd' in execline?

[1] 

I fully understand that this convention is not followed in most Unix
shells (except for rc(1) which is from Plan 9, and perhaps some others),
but execline is not a shell, and we can additionally symlink /bin/s6/*
into /bin for backward compatibility.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Nov 30, 2019 at 04:01:40PM +0100, Jeff wrote:
> a useful command interpreter should provide some builtins IMO.
> this is much more efficient and avoids excessive exec chaining
> (analoguous to single combined utils for several tasks vs the
> "one task one tool" approach). there might be a very good reason
> shells provide builtin "cd", "umask", "(u)limit" etc ...
> 
> i dunno if such builtins would be possible with execline, too.

See also this design:
.
(BTW, that post contributed to the formation of the UP2020 document.)

As was noted by Laurent, language flamewars are off-topic here.
However, I guess discussions on interesting ideas like nosh builtins
are still within the margin of acceptability, as long as we focus on
novel yet feasible chainloading implementions instead of blanket
declarations like "language X is better/worse than language Y".

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Nov 30, 2019 at 01:29:35PM +, Laurent Bercot wrote:
> [...] Here, I'd like to hear *less* about systemd,
> and more about better designs.

I do not mean to bad-mouth nosh, but I find it really necessary to note
that after skimming through `move-to-control-group.cpp', I feel quite
concerned about the coding style of nosh -- the kind of style that
significantly affects maintainability, which is largely independent of
the language used.

To put it bluntly, I think The nosh codebase is undeniably and
dramatically better than the systemd codebase, but obviously inferior to
codebase of most other daemontools-ish software we are fairly familiar
with.  I really find certain aspects of nosh enlightening, eg. the way
"builtins" are supported by the `nosh' interpreter [1], so I intend this
comment to be not pure blaming, but an (not so humble) appeal to push
nosh closer to perfection.

[1] 

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Mon, Nov 25, 2019 at 10:52:02AM +0800, Casper Ti. Vector wrote:
> Macros and/or helper functions (again cf. [1]; they can be factored into
> a mini-library in nosh) can also be used to reduce boilerplate like
> > const int error(errno);
> > std::fprintf(stderr, ..., std::strerror(error));
> > throw EXIT_FAILURE;
> which can be easily observed after the attached patch is applied.

The first chunk in the patch is incorrect, and the new code should be
(in other words, swap the `ENOENT == error' and the `const int' lines)
> if (!self_cgroup) {
> if (ENOENT == error) return;// This is what we'll see on a BSD.
> const int error(errno);
> std::fprintf(stderr, "%s: FATAL: %s: %s\n", prog, "/proc/self/cgroup", 
> std::strerror(error));
> throw EXIT_FAILURE;
> }
I am very sorry for that.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Nov 24, 2019 at 05:13:15PM -0300, Guillermo wrote:
> Those are just two if statements 'sharing' a body for brevity. That
> deals with errors in the openat() and subsequent write() calls, for
> the file that controls cgroup membership. By displaying a message
> constructed in the same way in both cases, and then throwing an
> exception. *Shrug*

That particular piece of code
> if (0 > cgroup_procs_fd.get()) {
> procs_file_error: ...
> }
> if (0 > write(cgroup_procs_fd.get(), "0\n", 2)) goto procs_file_error;
seems equivalent to
> if (0 > cgroup_procs_fd.get() ||
> 0 > write(cgroup_procs_fd.get(), "0\n", 2)) {
> ...
> }

If the error handling branches that need reuse become more complex, the
following way can also be considered (cf. [1]):
> ...
> if (...) goto err;
> ...
> if (...) goto err;
> ...
> return;
> err:
> ...
> return;

Macros and/or helper functions (again cf. [1]; they can be factored into
a mini-library in nosh) can also be used to reduce boilerplate like
> const int error(errno);
> std::fprintf(stderr, ..., std::strerror(error));
> throw EXIT_FAILURE;
which can be easily observed after the attached patch is applied.

BTW, it seems that the value of errno is passed to std::strerror()
before anything can change the errno, which implies that `const int
error(errno);' can be left out and `errno' can be directly used as the
argument of std::strerror().

[1] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

--- move-to-control-group.cpp	2018-09-14 21:48:55.0 +0800
+++ move-to-control-group.new.cpp	2019-11-25 10:50:20.518459398 +0800
@@ -50,9 +50,8 @@
 	next_prog = arg0_of(args);
 
 	FileStar self_cgroup(open_my_control_group_info("/proc/self/cgroup"));
-	if (!self_cgroup) {
+	if (!self_cgroup && ENOENT != errno) {  // `ENOENT == error' is what we'll see on a BSD.
 		const int error(errno);
-		if (ENOENT == error) return;	// This is what we'll see on a BSD.
 		std::fprintf(stderr, "%s: FATAL: %s: %s\n", prog, "/proc/self/cgroup", std::strerror(error));
 		throw EXIT_FAILURE;
 	}
@@ -73,20 +72,16 @@
 
 	current = prefix + current;
 
-	if (0 > mkdirat(AT_FDCWD, current.c_str(), 0755)) {
+	if (0 > mkdirat(AT_FDCWD, current.c_str(), 0755) && EEXIST != error) {
 		const int error(errno);
-		if (EEXIST != error) {
-			std::fprintf(stderr, "%s: FATAL: %s: %s\n", prog, current.c_str(), std::strerror(error));
-			throw EXIT_FAILURE;
-		}
+		std::fprintf(stderr, "%s: FATAL: %s: %s\n", prog, current.c_str(), std::strerror(error));
+		throw EXIT_FAILURE;
 	}
 
 	const FileDescriptorOwner cgroup_procs_fd(open_appendexisting_at(AT_FDCWD, (current + "/cgroup.procs").c_str()));
-	if (0 > cgroup_procs_fd.get()) {
-procs_file_error:
+	if (0 > cgroup_procs_fd.get() || 0 > write(cgroup_procs_fd.get(), "0\n", 2)) {
 		const int error(errno);
 		std::fprintf(stderr, "%s: FATAL: %s%s: %s\n", prog, current.c_str(), "/cgroup.procs", std::strerror(error));
 		throw EXIT_FAILURE;
 	}
-	if (0 > write(cgroup_procs_fd.get(), "0\n", 2)) goto procs_file_error;
 }

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Nov 17, 2019 at 02:26:44PM +0800, Casper Ti. Vector wrote:
> If you are a regular non-anonymous Slashdot reader, I will be very
> grateful if you can vote my recent comment up:
> <https://slashdot.org/comments.pl?sid=15221854=59420196>

A system proponent gave a remark about nosh's `move-to-control-group',
which appears, to some degree, justified:
<https://slashdot.org/comments.pl?sid=15221854=59422200>
Perhaps Jonathan can somehow make the code look better?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Oct 13, 2019 at 01:37:43AM +0800, Casper Ti. Vector wrote:
> [...] Consequently I think that, with an appropriate amount
> of publicity for the document, much more people would be willing
> to keep an eye on, migrate to, or even help develop daemontools-ish
> systems, potentially creating a mini-avalanche effect or even resulting
> in a turning point in the "init wars". [...]

If you are a regular non-anonymous Slashdot reader, I will be very
grateful if you can vote my recent comment up:
<https://slashdot.org/comments.pl?sid=15221854=59420196>

[I know it is not quite decent to rally for votes in this situation, but
I consider it some kind of "necessary evil" to raise the (long overdue)
public awareness of daemontools-like init/rc systems.]

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Mon, Oct 28, 2019 at 08:34:31AM -0700, Avery Payne wrote:
> For those readers that meet the following critieria:
> - Are unfortunate enough to only speak a single language, English;
> - And simply want to read an English version of the document;
> - Are (un)fortunately running a current Debian installation with missing
>   Latex dependencies;

My apologies for that.  If you do not mind missing latest changes (they
are predominantly cosmetics; significant changes are usually followed
by new releases), precompiled PDFs for the tagged releases are always
available at .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Tue, Oct 22, 2019 at 12:54:17PM -0400, Steve Litt wrote:
> What URL is the best one for us to publicize?

" for PDFs,
 for the source code"?
(I do not want to clutter the git repo with PDF files...)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Oct 13, 2019 at 01:37:43AM +0800, Casper Ti. Vector wrote:
> [...]  Consequently I think that, with an appropriate amount
> of publicity for the document, much more people would be willing
> to keep an eye on, migrate to, or even help develop daemontools-ish
> systems, potentially creating a mini-avalanche effect or even resulting
> in a turning point in the "init wars".  However, the influx of people
> into our circle will also result in a lot of noise (especially the noise
> from some ill-intentioned systemd proponents) and a lot of additional
> technical workload, so I request Laurent for a decision on whether to
> publicise the document; and provided he agrees, I request you to help
> spread it to appropriate places.

Through private mail with Laurent, I have confirmed that he agrees to
the publicising (not just publication -- it has been on GitLab/GitHub
for some time!) of this document.  So please help me spread the
information to where it should be, and direct any interesting criticism
to me if you find it necessary.  Thanks in advance.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

(Removed the skaware list from To:, for previously noted reason.)

On Sun, Oct 13, 2019 at 10:21:13AM +0200, Oliver Schad wrote:
> thank you for the effort putting things together. I was asking myself
> some questions. What is the target group? What is the exact purpose of
> that document?

As was hinted in the Foreword, the document is a compendium, but I
attempted to keep reasonable separation of concerns between the parts.
So the first part is about UP2020 (including the systemd rants, which
are the most important reason I can and need to ask for advices here),
the second part about my attempt to more systematically explore the
social values of the Unix philosophy (from the UP2020 perspective),
and the third part about the "Unix/Lisp reconcilliation" proposal.

Different passages are aimed at various groups of people, eg.:
* The document as a whole might be interesting for Unix addicts who
  happen to also appreciate the UP.
* The first part would probably be useful for people entangled in the
  "init wars", particularly for those who have strong opinions.
* The third part might be of some value to Unix developers interested in
  programming languages, as well as certain PL researchers.
* In general, footnotes are less essential contents, and often require
  deeper understanding of the subject matter.

> For systemd I have a more practical approach to discuss:
> 
> 1) how many config statements are there?
> 2) how many cases exist, which you have to work around (practical
> setups, where a config statement is missing or do the wrong thing)
> 3) how many bugs/feature requests are opened over time and how long does
> it take to solve them?
> 4) how big is the memory footprint and for which systems this is too
> much?
> 5) how many lines of code?
> 
> So you would have metrics - especially if you compare them to other
> solutions. If you want to have more food, make more metrics (call graph
> complexity or whatever). But there are simple metrics, which shows the
> result(!) of the design. Talking about the design itself is really a
> personal opinion otherwise and very lengthy and needs a lot of
> knowledge to follow.

I see.  Some of these metrics can be found from citations ([55] for (3),
[132] for (4), both as of v0.1.1) or in an easily verifiable way ((1)
and (5); tarball size of systemd + dependencies vs. tarball size of s6 +
friends + dependencies specific to them might be an easy indicator for
(5)).  Most arguments I made are admittedly qualitative, but they are
(if not, please tell me :) either backed up by citations or self-evident
to people who know a little about systemd.  On the other hand, I find
convincing quantitative evidences for most arguments much harder to
find (which is why statistics is an independent research field :);
nevertheless, the requirement about "know a little about systemd" is
indeed tricky, which is part of why I asked here.

> For the philosophy itself there are some parts missing in my opinion:
> what does that really mean what you're talking about in practical
> solution?
> 
> Is there a practical approach anywhere, interface definition,
> architecture? You describe a few patterns ok - but they are really
> common. I don't get really, which people would help this document.
> 
> Maybe that thing is missing: if somebody would like to build a modern
> UNIX: what are practical steps to achieve it?
> 
> Which tools, which interfaces (kernel, userland) are needed?

Before the publication of the document there was a alpha phase of it
in a subgroup of my local LUG (cf. the Afterword), and somebody asked
something similar.  My current opinion is that the UP2020 summary
(minimising the cognitive complexity of the system while almost
satisfying the requirements) can be used to compare *existing* systems
and *practical/semi-practical* proposals, but does not directly dictate
specific technical routes to a Unix-ish solution.

Metaphorically, UP2020 to practical ways to Unix-ish solutions is like
the principle of least action to the calculus of variations, except that
practical Unix-ish ways cannot be precisely formulated like the calculus
of variations is.  Instead, we have to foster programmers' (imprecise
but productive in another way) creativity, and the ways to foster
creativity were outlined in Section 11-12.

> BTW: I can't really see images inside the PDF

Sorry, but there are currently no figures due to a limited time budget;
I will probably make a plan about adding pictures and then implement it,
perhaps before the end of this year if time permits.  As a further note,
I am far from as good a teacher as V. I. Arnold or Dan Friedman (cf.
the end of Section 16), and soon after actually beginning to write the
document I found it really hard to accommodate for people with varied
levels of backgrounds.  This is quite serious a barrier for newbies, and
what has already been done is surely not quite enough; I will rethink
about this often.

-- 
My current OpenPGP key:

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sat, Oct 12, 2019 at 02:58:59PM -0400, Steve Litt wrote:
> [...]
> http://troubleshooters.com/linux/systemd/lol_systemd.htm
> [...]

Well, I knew that, and cited your diagrams ([15] and [18] in the
document as of v0.1.1).  I believe those who told me the point about
cohesion and coupling in the systemd architecture may seem unfounded
to *people who have very little prior knowledge about systemd* are
well-intentioned and are not fanboys.  Thank you anyway for the
expressive diagrams and past discussions on related topics!

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: The "Unix Philosophy 2020" document

[Casper Ti. Vector]

(I guess discussions about this document is probably destined to be
off-topic on the skaware list, so further public mail in this thread
will only be posted to the supervision list; sorry for the disturbance.)

On Fri, Sep 27, 2019 at 04:38:16PM +0800, Casper Ti. Vector wrote:
> Although the contents of the document are quite related to our mailing
> lists, I do not think Laurent (by the way, I am sorry he might really
> disagree with me on many points in the third part) would like to see too
> much off-topic discussion on these lists.  So please send comments to me
> via private mail or GitLab/GitHub issues if they are unrelated to
> supervision/skaware; I will be especially interested in comments about:

I received comments and suggestions from multiple people, and would like
to express my sincere gratitude to these people.  Most of the issues
involved either are quickly resolvable, or require more time to address
but do not affect the main ideas expressed.  However, there is one issue
that I definitely need to ask for your suggestions on to resolve, and
the issue is about systemd: multiple people told me that they felt
uncomfortable about the recurring (but each time on a different aspect,
obviously) examples about systemd.

Before asking specifically for what I need, please allow me to briefly
explain why the document is in its current shape.  As can be seen from
the Afterword and Footnote 44 (as of v0.1.1), this document originated
from my reaction to the systemd fiasco; and as can be inferred from
Section 11, I find it impossible to discuss UP2020 without major
involvement with systemd.  So I already intended to blame systemd when
the document only existed in my imagination, and this intention is not
unjustified; but once systemd is involved, any argument must be backed
with enough evidences, hence the current shape of the document.

However, people told me that the document is not quite accessible to
those who know really little about systemd: one example is they do not
even know much about how the modules are organised in systemd, so the
claim that the systemd architecture has how cohesion and high coupling
may seem unfounded; because of this, I request your recommendation for
an accessible and not-too-boastful introduction to systemd suitable for
citation in the document.  Additionally, although there do not yet seem
to be other major technical faults in the recurring systemd examples,
they might really appear unpleasant for some readers, so I also request
your advices on how to reduce the "rantiness" of the document (eg. how
certain parts can be rephrased, or certain inessential examples be
removed/replaced) without harming its technical correctness.

A point to note is that I tried to choose a small yet most touted subset
of systemd features, and then to analyse how these features can be done
using s6 and friends, which I find a most efficient way to understand
their nature.  From what I know, there have been few systematic analyses
of systemd from the viewpoint of the daemontools-ish design, so I
believe that these technical arguments, in combination with UP2020, can
be much more convincing than other arguments available in showing why
systemd is bad.  Consequently I think that, with an appropriate amount
of publicity for the document, much more people would be willing
to keep an eye on, migrate to, or even help develop daemontools-ish
systems, potentially creating a mini-avalanche effect or even resulting
in a turning point in the "init wars".  However, the influx of people
into our circle will also result in a lot of noise (especially the noise
from some ill-intentioned systemd proponents) and a lot of additional
technical workload, so I request Laurent for a decision on whether to
publicise the document; and provided he agrees, I request you to help
spread it to appropriate places.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Sun, Mar 17, 2019 at 09:25:32PM +0800, Casper Ti. Vector wrote:
> Since the first announcement [1] of slew [2], a few people expressed
> interest in the project, but I have received little feedback regarding
> its technical contents.  Therefore although I have successfully deployed
> slew on a few real-life systems, it is still quite a slowly moving
> personal hobby project.  However, there are a few changes which I think
> might be interesting to some people here, which is briefly summarised in
> this mail, 5 days before the project's one-year anniversary.

To facilitate distro packaging, I changed the slew repository structure,
so that all files specific to a package are put in the corresponding
subdirectory, eg. (with whitespace squeezed in the output):
> % ls pkg/wpa_supplicant/*
> pkg/wpa_supplicant/base: wpacli. wpasup.
> pkg/wpa_supplicant/lib: prep
> pkg/wpa_supplicant/misc: wpa_cli.rc
> % ls pkg/dhcpcd/*
> pkg/dhcpcd/base: dhcpcd.
> pkg/dhcpcd/lib: fn
Please see the updated manual for details:
<https://gitlab.com/CasperVector/slew/blob/master/Manual>.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

The "Unix Philosophy 2020" document

[Casper Ti. Vector]

On Sun, Sep 01, 2019 at 05:11:57PM +0800, Casper Ti. Vector wrote:
> I roughly see.  Other programs, like date(1) and conky, do seem to
> output the correct time after resuming, but I (at least recently, and
> you will know why in roughly two to four weeks) really do not have time
> to investigate how that is done.

I finally finished the task I had been working on for almost two months,
and the result is at <https://gitlab.com/CasperVector/up2020/releases>
(cf. the afterword and the first part in the document for its relevance
to the skarnet mailing lists).  The first part may serve as a nice
introduction to s6, but will perhaps be a little banal to people that
are already conversant; these people are advised to read the third part.

Although the contents of the document are quite related to our mailing
lists, I do not think Laurent (by the way, I am sorry he might really
disagree with me on many points in the third part) would like to see too
much off-topic discussion on these lists.  So please send comments to me
via private mail or GitLab/GitHub issues if they are unrelated to
supervision/skaware; I will be especially interested in comments about:

* Logical fallacies or incorrect evidences (particularly in the first
  part); platitudes or bloated contents.
* Citations that should be added or removed; obvious mismatch between
  the Chinese and English versions (if you happen to understand both).
* Spelling or grammatical mistakes; obscure or wordy passages; aesthetic
  issues with typesetting.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6-linux-init: Actions after unmounting filesystems

[Casper Ti. Vector]

On Sun, Aug 18, 2019 at 06:26:05AM +, Laurent Bercot wrote:
> The umount command basically performs a "umount -a". I can add a list
> of filesystems that should be kept, but it's more ad-hoc code. It's
> ugly.

I see; slew always does it this way:
.
[BTW, I really dislike the slow (because bloated?) frontend of GitLab.]
And since it is just a script, the user certainly has the final say.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6-linux-init: Actions after unmounting filesystems

[Casper Ti. Vector]

On Sat, Aug 17, 2019 at 11:01:35PM +, Laurent Bercot wrote:
> The asymmetry of mounting and unmounting filesystems is really a pain
> in the ass for the design of an init/shutdown sequence. I wanted to
> keep the shutdown as quick, minimal, and reliable as possible, but it
> seems there's no way to do so with those snowflake filesystems. :/

Sorry if I read this thread too hastily, but why not just keep /proc etc
mounted, as was seemingly the way with s6-linux-init <=v0.4.x.x (and
therefore slew)?  Since the asymmetry is by nature, simply respecting it
appears to be one minimalist way.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 05:10:58PM +, Laurent Bercot wrote:
>  Having one stream per syslog client is a good thing per se because
> it obsoletes the need to identify the client in every log record;
> but the killer advantage would be to do away with system-wide
> regular expressions for log filtering, and that's not something
> we have yet. Even when you pipe "s6-ipcserver ucspilogd" into
> s6-log, your s6-log script is the same for every client, so the
> regexes need to be global. A real improvement would be to have
> a different log script for every client connection, so the log
> filtering would really be local; but I haven't yet thought about a way
> to design such an architecture. That's the main reason why I haven't
> much pushed for SOCK_STREAM syslog() in musl; if we can come up with a
> syslogd scheme that works without any global regexes, then we'll
> have a real case for SOCK_STREAM adoption. Until then, socklog works.
 
Assuming the number of syslog logging scripts is fairly small (a few for
daemons in an anticipated list, and perhaps one for the rest; I think
this scheme is actually already in use by most syslog users), what about
setting up a group of s6-log consumer services, and use a chainloading
program (akin to s6-tcpserver-access) with s6-ipcserver to dynamically
decide which consumer to connect to (by interacting with s6rc-fdholder)?

(I think this scheme, with some variations, can also be usable with
services that produce multiple output streams, like Apache which is
recently discussed on this mailing list.  This would be particularly
easy if these services can be configured to log to /proc/self/fd/[num]:
then the user can simply use s6-fdholder-retrieve to chainload the
daemons for the services.)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: race condition in killall

[Casper Ti. Vector]

On Sun, May 05, 2019 at 03:55:51AM +0200, sysi...@yandex.com wrote:
> since they do more work to select processes and hence need more time
> when iterating the PID dirs in the procfs?  though i doubt they use
> any matching at all when tasked with killing all processes and
> probably behave like the killall5 utility in this situation.

The original snippet on gitlab-2.asag.io used `pkill -SIG .'; and it
seemed that they had really encounterred escaped processes, because they
said:
> the "sync" was added just in case mounting read-only doesn't work

> OpenRC also provides a tool for that task btw:
> /libexec/rc/bin/kill_all
> it uses the kvm method to find running processes on the BSDs and the
> procfs on Linux.

Or use chainloading and `kill -9 -1': both are simple and portable.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Wed, Mar 20, 2019 at 01:14:39PM +0800, Casper Ti. Vector wrote:
> Fixed (provided that sysvinit killall(8) is included in the container)
> in commit 3f246b20 the day before yesterday :)

I forgot to note that sending SIGHUP is unnecessary, and `rc.halt' did
this previously because of my misunderstanding of how the catch-all
logger was stopped.  `s6-svc -X /run/service/s6-svscan.log' lets the
corresponding `s6-supervise' process immediately close its fds 0/1/2 and
wait to exit after the catch-all logger exits, so the logger's input is
only connected to outputs of unlogged services (except for itself) not
stopped in `rc.fin' (which should not exist if `s6-rc -d change ...'
succeeded) and all stray processes that has inherited fds from parents
that directly wrote to the catch-all logger.

If no such processes remained at this moment, the SIGHUP (intended to
stop the catch-all logger because it is started with the `-b' option of
`s6-log') would have no effect; if they existed, sending SIGHUP
immediately after SIGTERM/SIGCONT would probably result in lost logs for
the final outputs of them, and simply letting the impending SIGKILL kill
the logger would be better.  Additionally, I have just realised that
these final logs were not considered by the current version of
`rc.halt', and then pushed commit 5911f892 to fix this.

> something like
> : /bin/importas PIA PIA
> : /usr/bin/env -i PATH=/usr/sbin:/usr/bin:/sbin:/bin PIA=${PIA}

Note that here correctness of the environment is ensured whether $PIA is
non-empty, empty or unset, because in the last case `importas' would
simply delete `PIA=${PIA}' from the command line to be exec()ed.  (Kudos
to Laurent, of course :)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Tue, Mar 19, 2019 at 04:58:53PM +0100, Oliver Schad wrote:
> Exactly - it doesn't make sense for us to have for every service it's
> own logging user. So I defined a common log user.

Using separate logging users is mainly due to security reasons: though
s6-log is very reliable, reasonably more privilege separation still seem
desirable; see the example service definitions shipped with s6/s6-rc, as
well as qmail's practice of privilege separation.  And it is also easy
to do this: just put `useradd'/`adduser' invocations in post-install
hooks of the slew-related packages.

On the other hand, if you really do not want to use separate logging
users, at least do not use `nobody' (sorry for not considering this
yesterday; I have just pushed commit d5cb508b to correct this): nowadays
too many programs use `nobody' as a privilege separation user (I know
the OpenBSD people consciously avoid this, but many Linux distro
developers do not seem aware), so all these service would be in danger
if `nobody' gets compromised, especially considering that these services
are usually much easier targets for attacks than s6-log is.

> https://gitlab-2.asag.io/snippets/9

`cd /etc/slew/db' seems unnecessary as the source filename does not need
to be resolvable in $CWD of ln(1).

`s6-rc-update /etc/slew/db/compiled' is intentionally not invoked in
`lib/build.rc' because in certain conditions (eg. when some services
are renamed), the old-named service would be stopped and then the
new-named service would be started.  More seriously, since the effects
of `up' scripts for certain oneshots are reversed not in the
corresponding `down' scripts (eg. `base/fstab/pre/up' is reversed in
`init/rc.halt'), they are often not reentrant, so renaming of their
dependencies and naiively invoking `s6-rc-update /etc/slew/db/compiled'
may result in service outage that must be fixed manually.

The `-f' option of `s6-rc-update' is intended exactly for this scenario,
and the intended way to use `lib/build.rc' is:
: # Perhaps write a conversion file `conv', according to the output.
: /etc/slew/lib/build.rc main
: s6-rc-update [-f conv] /etc/slew/db/compiled
: rm -rf /etc/slew/db/old.main /etc/slew/prep.main

> killall5 didn't work at all

Fixed (provided that sysvinit killall(8) is included in the container)
in commit 3f246b20 the day before yesterday :)

> the "sync" was added just in case mounting read-only doesn't work

pkill(1), killall(1) and killall5(8) all retrieve a process list and
kill them one by one, instead of calling kill(-1, signal), so a race
condition can happen thats let some process escape the final SIGKILL.
Since pkill(1) and killall(1) use regex matching, the probability for
the race can be significantly larger.

To be 100% sure no process (except for PID 1) escapes that signal, you
can use `s6-nuke' from s6-portable-utils.  `kill -signal -- -1' should
theoretically do similar things, but kill(1) from coreutils and busybox
do not seem to behave in this way.

> emtpyenv kills the container environment, which is needed

You can replace
: /bin/emptyenv
: /bin/export PATH /usr/sbin:/usr/bin:/sbin:/bin
with something like
: /bin/importas PIA PIA
: /usr/bin/env -i PATH=/usr/sbin:/usr/bin:/sbin:/bin PIA=${PIA}

> clock adjustment doesn't work inside a container.

Then it is simpler to set `clock=()' in /etc/slew/lib/slew.rc :)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Tue, Mar 19, 2019 at 08:42:39PM +0800, Casper Ti. Vector wrote:
> * The most important ancillary files are preprocessing passes like
>   `misc/openvpn/70-openvpn.rc', which should of course be installed into
>   /etc/slew/lib/prep.  They are not directly put into `lib/prep' because
>   unlike extra service definitions in `base', extra preprocessing passes
>   results in actual overhead when `lib/prep.rc' is run.  (The user can
>   disable preprocessing passes by removing the `x' bit from the
>   corresponding files, cf. `lib/prep.rc').

A more important reason for not directly putting extra preprocessing
passes into `lib/prep': although I have always been carefully preventing
these passes from conflicting with each other, I cannot be sure that
conflicts would always be avoidable, especially with certain distro
packages that are mutually exclusive.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Sun, Mar 17, 2019 at 03:30:02PM +0100, Oliver Schad wrote:
> https://gitlab-2.asag.io/snippets/7

A closer look at this snippet reveals that most changes therein are:
1. Customisations of `s6-log.rc', probably modifying the logging user.
2. Addition of unshipped services (eg. postfix).
3. Deletion of unused services (eg. dcron and busybox cron).
4. Other regular customisations similar to what I showed yesterday.

Regarding 1, if you (as I guessed) want to just use `nobody' for all
logs (which is, BTW, strongly discouraged!), I have just pushed commit
a1ffc647 so you can simply delete `s6-log.rc' of services for this
purpose, provided that you do not plan to change the $args or $logd of
these services.

Regarding 2 and 3, and noticing the presence of `db' and `prep.main'
(BTW, `prep.main' and `db/old.main' are supposed to be deleted after
successful `lib/build.rc'/`s6-rc-update' invocations), it is likely that
you directly cloned slew's git repository into /etc.  I personally think
this makes updating complicated; and since git ignores ownership,
permission bits (except for the `x' bit) and empty directories, the
structure of /etc/slew would drift from the expected status in certain
conditions.

Instead, I think a better way to distribute slew is to break it into
multiple packages for the intended distro, with the packaging script(s)
essentially performing the jobs of yesterday's `slew-build.sh' and
`ubuntu-conf.sh':
* A "base" package: including the `init' / `run' directories, absolutely
  essential services in `base', and a small `main' config somewhat like
  the shipped one).
* Multiple packages for other services (eg. OpenVPN and wpa_supplicant):
  each including the necessary service definitions in `base', and
  corresponding ancillary files in `misc'.
* The most important ancillary files are preprocessing passes like
  `misc/openvpn/70-openvpn.rc', which should of course be installed into
  /etc/slew/lib/prep.  They are not directly put into `lib/prep' because
  unlike extra service definitions in `base', extra preprocessing passes
  results in actual overhead when `lib/prep.rc' is run.  (The user can
  disable preprocessing passes by removing the `x' bit from the
  corresponding files, cf. `lib/prep.rc').
* Patches like `misc/thinkfan/thinkfan-0.9.3-fglog.patch' are intended
  to be applied to distro packages to increase their compliance with
  s6's way of longrun management (usually about logging), and therefore
  can be omitted from the service packages.  Other ancillary files are
  intended to be installed into locations outside of /etc/slew, like
  `misc/wpa_supplicant/wpa_cli.rc' should be installed into
  /etc/wpa_supplicant, in accordance with `base/wpacli./run'.
Perhaps these intentions were not as clear as I thought they should
implicitly (inferred from the codebase) be; I am sorry for that.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Mon, Mar 18, 2019 at 10:44:43PM +0800, Casper Ti. Vector wrote:
> * Transfer the `pkgs' directory (with its contents, all produced in the
>   step above) to the Ubuntu VM, run (as root) attached `slew-build.sh'
>   in the directory where `pkgs' reside.

Here I actually meant `ubuntu-conf.sh' instead of `slew-build.sh'.
Additionally, one thing `ubuntu-conf.sh' did not say clearly is that
Devuan's binary package for eudev should be downloaded *into the `pkgs'
directory*.  And in case you like smaller files, `ska-build.sh' makes
unstripped binaries; stripping them reduces the total size of execline,
s6 and s6-rc binary packages from ~5.5M to ~1M.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Update on the progress of slew development

[Casper Ti. Vector]

On Sun, Mar 17, 2019 at 03:30:02PM +0100, Oliver Schad wrote:
> So in the end Slew was great to understand, how s6 could be integrated
> as a pattern. But the units/scripts itself didn't work for us.

I personally use Alpine for servers and Void for desktops, and so did
not know what problems distributer might encounter in Debian/Ubuntu.  So
first of all I need to thank you for attempting to use slew on real-life
systems, which is exactly how the slew codebase can evolve to suit more
application scenarios.

> https://gitlab-2.asag.io/snippets/7

I constructed a slew-managed Ubuntu system with only essential services,
udhcpc on eth0, and sshd, reproducible with the following steps:
* Install Ubuntu on a VM with `ubuntu-18.04.2-server-amd64.iso'.
  (Using the US keymap, and with SSH server enabled).
* Build static execline, s6 and s6-rc using attached `ska-build.sh' (as
  root), and tailor slew for Ubuntu using attached `slew-build.sh'.
  (Better done on an Alpine VM because Ubuntu does not use musl.)
* Transfer the `pkgs' directory (with its contents, all produced in the
  step above) to the Ubuntu VM, run (as root) attached `slew-build.sh'
  in the directory where `pkgs' reside.

I personally find the changes fairly minor, except for these issues:
* Debian/Ubuntu do not package eudev, so I used `/sbin/udevd' from
  Devuan as a workaround; to ensure basic safety, you definitely need to
  package this yourself for your customised Debian/Ubuntu systems.
* One other nuisance is that while the slew-managed system uses ~32M
  memory after booting, the dracut-generated initramfs barely loads even
  with 256M, which is an important reason for avoiding Ubuntu.

> So may I ask directly: is the plan to provide scripts/units for
> everyone, which works almost out of the box?

Linux distros are too diverse for slew to fully accomodate, but slew
has been designed from the beginning with flexibility in mind: once you
successfully customise it for the expected average case of a distro, the
user-level customisations would be fairly easy.  And as you can see from
the attached scripts, the distro-level customisations are, while perhaps
non-trivial, quite manageable.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



slew-ubuntu.tgz
Description: GNU Unix tar archive

Update on the progress of slew development

[Casper Ti. Vector]

Since the first announcement [1] of slew [2], a few people expressed
interest in the project, but I have received little feedback regarding
its technical contents.  Therefore although I have successfully deployed
slew on a few real-life systems, it is still quite a slowly moving
personal hobby project.  However, there are a few changes which I think
might be interesting to some people here, which is briefly summarised in
this mail, 5 days before the project's one-year anniversary.

[1] 
.
[2] .

Previously, slew would only save /run/uncaught-logs/current to
/var/log/init (so rotated log files would be ignored) on shutdown (so
no saved logs if the system crashed).  Now the log-saving mechanism is
implemented in an s6-log rotation processor (`init/save_log.rc'), which
would do the task with best effort (if /var/log/init is unwritable for
too long and the catch-all logger is fed with a large stream, the head
of the stream might be discarded anyway).  The `local' oneshot and
`init/rc.halt' would trigger the mechanism by sending the logger
SIGALRM; currently, the remaining issue is that s6-log would not run
the processor upon SIGALRM if `current' is empty, so a temporary write
failure in /var/log/init plus an unfortunate amount of log (no new bytes
after the rotated and unsaved log) would result in discarded logs.

Other noteworthy changes:
* The issue about slew's fault tolerane mentioned in the original
  announcement has been largely solved.
* "Methods" for polymorphic services are supported: see `lib/fn' and
  `misc/wpa_supplicant/wpa_cli.rc' for an example.
* Information can be passed through the kernel command line to slew: see
  `lib/kcmd.rc', and `init/{load,save}_clock.rc' for an example.

Finally, as some people strongly complained [3] about this issue, I
would like to ask for your opinions about the naming convention: what do
you prefer, more "standardised" names like `wpa_supplicant.wlan0.log' or
easier-to-type names like `wpasup.wlan0.log'?  I can switch to a new
convention if you overwhelmingly support it or if I find a very
convincing argument for it, but I need to be really sure that I would
*not* need to change the convention *more than once*.

[3] .

Suggestions and questions are welcome.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Sun, Mar 25, 2018 at 06:38:33AM +, Laurent Bercot wrote:
> A major selling point of the s6 and s6-rc formats is that they're easy
> to autogenerate, and a frontend would be proof of that. We claim we're
> technically better than everyone else, and that our paradigm is more
> flexible than others - well there's the opportunity to prove it.
Fine if you think so.  Religions can really be annoying and wasteful.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 09:20:58PM +0800, Casper Ti. Vector wrote:
> Now I understand.  What a pity for distro developers / users and us.
On a second thought, what about (at least a attempt at) solving the
human (political) problems by human means (propaganda, but of the
factually correct type)?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 03:05:53PM +0200, Alex Suykov wrote:
> The reaction to the slew manual I'm afraid will likely be along the
> lines of "that's all cool and stuff but how do I actually run this?".

I again confess that I am not good at writing tutorials; the current
manual is really more suitable for those who know the backgrounds of
s6/s6-rc.  For a reference system, see the attached tarball and online
VM image from the first message in this thread; the only difference
between its system and slew is that all references to `/etc/s6' /
`/etc/s6/lib/s6.rc' should now be `/etc/slew' /
`/etc/slew/lib/slew.rc'.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

slew: renamed from "s6.rc"

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 12:00:22PM +0800, Casper Ti. Vector wrote:
> What about using "slew.rc" and changing the installation path from
> `/etc/s6' to `/etc/slew'?  The change is not exactly trivial but already
> much smaller than the `/etc/s6-init' / `/etc/s6-rc' merge I did before
> releasing this project, and can be done in a few hours (the project itself
> is not complex; the complexity on my part comes from the several machines
> where it is delpoyed and has to be migrated :().

Done; new project URL: <https://gitlab.com/CasperVector/slew>.
(You will be redirected if you visit the `s6-dot-rc' URL.)

The changes turned out to be very simple: batch changing `/etc/s6' to
`/etc/slew' (including the symlinks in `run/service/.s6-svscan'),
`s6.rc' (filename) to `slew.rc', and `s6.rc' (project name) to `slew';
the rest were cosmetics.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 05:10:58PM +, Laurent Bercot wrote:
> would it be possible for you to change the name of the project?

What about using "slew.rc" and changing the installation path from
`/etc/s6' to `/etc/slew'?  The change is not exactly trivial but already
much smaller than the `/etc/s6-init' / `/etc/s6-rc' merge I did before
releasing this project, and can be done in a few hours (the project itself
is not complex; the complexity on my part comes from the several machines
where it is delpoyed and has to be migrated :().

(Hint for the etymology: note how "6" is pronounced in Chinese ;)

>  - I now recommend using socklog[1] to implement syslogd.

I see.

> It will likely be called s6-frontend. It will be written in C, and
> provide a user-friendly interface for setting up a complete init
> system, as well as a one-stop-shop "s6" command; the goal is to
> improve usability and simplicity for users who are not longtime
> members of the process supervision gang.

To be honest, I find the idea not very appealing to me.  I did not want
to say about this before, but now I find it necessary before more energy
is spent in goals that I consider less meaningful.  Fortunately, only
`s6-linux-init-maker' is affected by this issue as of now, and I will
use it as an example.

I think the direct downstream of init/rc systems is distributions, not
end users which demand "usability" in the usual sense (actually, one of
the many problems with systemd is that it attempts to bypass distros
and directly provide "unified" features to end users).  After s6/s6-rc
becomes mainstream, `s6-linux-init-maker' will perhaps have, IMHO, an
embarrassing role: most end users would begin with scripts provided by
their distros, which might often be very different from those generated
by `s6-linux-init-maker'.

And the difference is not only in the utilities used, but also in the
functionalities offered: eg. conditional (re-)mounting of `/run'
according to results of `mountpoint -q /run', loading/saving the clock,
saving the "uncaught" logs.  This is honestly not to show off the
features provided by my project, but to show that attempts to encapsule
factors in real-world use cases of an init/rc system into a code
generator can be largely unproductive.

Instead, I think a better way is to provide a full "reference
implementation" of what you have in mind, with the code generation knobs
converted into comments (cf. the `devtmpfs' line in `rc.boot' in my
project) or notes in the documentation.  Let distros (and some
adventurous users), i.e. those who understand their requirements for
init/rc systems better, decide how to customise their systems: after
all, text editors are the ultimate code generators for plain text files.
If it is agreed that the Unix philosophy is essentially about minimising
the total complexity of a system, I guess people would consider this to
be the more Unix-ish way.

> However, if decisions about "where to put stuff" are easy enough
> to take, decisions about "what init scripts should I provide, what
> external software should I assume is present on the system" are
> definitely not.

> But there is a large gray area here: what is "reusable policy" (RP)
> and what is "distribution-specific" (DS)?

First of all, I would like to note that the files in my project are not
to be used exactly as is; instead, they are intended as "reasonable
approximations" considering a majority of use cases.  I currently use
Alpine and Void on my servers and desktops, and I deviate from the
published version of my project, more or less, on each and every
machine, and their versions differ from each other.  I consider this
phenomenon normal and healthy.

Considering the adoption of systemd, I think most longrun definitions
are reusable, at least in an approximate sense; oneshots can be more
volatile, but most are still reusable with a reasonable amount of
distro-specific customisation.  I do not consider package dependencies
to be a big problem: distros (and those adventurous users) will
naturally handle them when customising the init/rc system.  No better
way has been proposed to my knowledge: `s6-linux-init-maker' basically
solves it by depending on skaware, and systemd "solves" it by using its
own extremely bloated implementations.

> If it involves choosing between several external software packages
> that provide equivalent functionality, is it okay to hardcode a
> dependency, or should we provide flexibility (with a complexity cost)?

Regarding networking, I consider netifrc to be, though bloated, a
successful example.  As I mentioned, the functionalities of netifrc can
in principle be implemented using preprocessors in much cleaner ways,
and I guess the complexity cost that comes with the flexibility would
not be too big in this case.

Of course, this means you probably cannot use C exclusively.  If Unix
used a small language with S-expressions that can be both compiled and
interpreted, and with garbage collection enabled 

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 09:23:34PM +0800, Casper Ti. Vector wrote:
> [1] <https://gitlab.com/CasperVector/s6rc-dotrc>.

Should be <https://gitlab.com/CasperVector/s6-dot-rc>.  The project used
`/etc/s6-init' / `/etc/s6-rc', and `/etc/s6/lib/s6.rc' used to be
`/etc/s6-rc/lib/s6-rc.rc', hence the old name.  Many thank to mobinmob
for notifying me of this :)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

[Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

s6.rc [1] is an attempt to bridge the gap between the elegant foundation
provided by s6/s6-rc and an init/rc system that implements the main
functionalities beneficial for distributions.  s6.rc features a
preprocessor that generates source directories for use with s6-rc from
given templates.  The preprocessing procedure is composed of multiple
tiny passes, which makes the preprocessor not only clean and powerful,
but also extensible.  Using the preprocessor, s6.rc supports instanced
supervision, optional dependencies, in-place `up'/`down' scripts,
automatic connection between services and loggers, as well as
package-specific passes that can be plugged into the preprocessing
phase.

[1] .

The init part of s6.rc is, in many aspects, similar to what one would
expect from s6-linux-init.  The stage 1/2/3 scripts shipped with s6.rc
additionally support features like setting the time using hwclock(8) or
other means before exec()ing `s6-svscan' (cf. the documentation for the
rationale), duplicating (while writing to `/run/uncaught-logs') verbose
`s6-rc' output to `/dev/console', automatic reboot under certain
circumstances (eg. when required by fsck(8)), and automatically saving
`/run/uncaught-logs/current' on shutdown.

The attached tarball is an s6.rc-based example setup tailored for Alpine
3.7.x; a VM image of the setup is available at [2].  Due to a lack of
expressiveness in execline, most scripts in s6.rc are written for the
Byron Rakitzis implementation [3] of the rc(1) shell from Plan 9, and
static-linked binaries for x86 [4] and x86_64 [5] are available for
download.  See the attachments for SHA512 checksums (signed using my
OpenPGP public key) of the tarball, the VM image and the rc(1) binaries.

[2] .
[3] .
[4] .
[5] .

Of course, this is just the beginning; a lot of work has to be done:

* Although s6.rc can be easily configured to be rock solid, it is fairly
  fragile to PEBKACs: eg. if you accidentally delete `s6.rc' and then
  reboot the system, the system can get into quite serious troubles.
  Sprinkling related scripts with a lot of "if something is unset then
  default to blah" clauses can avoid the problem, but seems to make
  the code bloated.  Is there a better solution?

* Many more service definitions and ancillary files are necessary for
  s6.rc to be useful for the general public instead of just a few users,
  so any contribution / suggestion is welcome.  BTW, I do not consider
  myself to be a really good documentation writer, so please tell me if
  you have a good idea on improving the s6.rc documentation.

* In principle, it is possible to implement something like Gentoo's
  netifrc [6] using s6.rc's preprocessor, and I believe it would be much
  cleaner than netifrc.  However, since I am not exactly familiar with
  the networking stuffs, perhaps it had better be done by someone else.

[6] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



alpine-s6rc-conf.tgz
Description: GNU Unix tar archive


SHA512SUMS.asc
Description: PGP signature

Re: Why is /etc/sv/alsa/supervise a symlink to /run/runit/supervise.alsa

[Casper Ti. Vector]

Perhaps runit just does not care, just like what s6 seems to do?

On Sun, Oct 29, 2017 at 05:35:22AM -0400, Steve Litt wrote:
> Are these symlinks a Void Linux implementation thing, or are they
> specified in runit itself?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 as a systemd alternative

[Casper Ti. Vector]

(Normally Jonathan would be replying to this point, but I still do not
see him in this thread, so I rashly take this job ;)

On Mon, Jun 26, 2017 at 05:05:15PM +0200, Istvan Szukacs wrote:
> I understand that service files are much better that shell scripts

Actually they are not.  This statement might appear logical when you see
[1] or some similar comparisons made by systemd proponents; however,
there are some problems with such comparisons, and here are the ones
that come to my mind at this time:
* The script on the left is a script for sysv-rc; sysv-rc is,
  admittedly, a terrible system.  rc scripts for some other traditional
  rc systems are much cleaner, eg. that for openbsd [2], or that for
  openrc [3].
* Traditional rc systems still deal with double-forking daemons, using
  pid-files, which is an awkward mechanism [4].  By abandoning double
  fork, process state handling (and daemon logging) would be much more
  straightforward [5].
* systemd unit files may seem a little more descriptive (and verbose ;)
  than the scripts in [5].  However, by chainloading, you can use
  noncommutative operator combinations (like passing some information in
  some environment variables and then cleaning up the environment after
  use of this information in some initialisation), which is difficult to
  achieve with systemd unit files.  Implementing the ordering of
  operators by ordering of lines in the unit file would probably result
  in reinventing execline / nosh, in an ugly way (actually they already
  partially reinvented the shell: note the `ExecStart*' fields in the
  unit file specification).
* systemd implements the mechanism for process state manipulation in
  pid-1, which leads to considerable safety/security implications.  By
  chainloading, all of these can be factored into the run script,
  without increasing complexity of the user interface.

[1] .
[2] .
[3] 
.
[4] .
[5] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On possibly "finer" dependencies in s6-rc

[Casper Ti. Vector]

Well, I found this to be fairly over-simplified.  If all members of the
disjunction are longruns, using s6-svwait is OK; however this is not
directly extensible to oneshots and bundles.  Nevertheless, I think the
idea of using a separate command to wait for the disjunction is still
architecturally promising: no change in the s6-rc *command*, just one
more internal command in the s6-rc *suite*; and as with s6-svwait, we
can inspect the procedure of state transition by simply listing the
processes.

On Tue, May 02, 2017 at 11:33:56AM +0800, Casper Ti. Vector wrote:
> perhaps no change is need after all in s6-rc.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On possibly "finer" dependencies in s6-rc

[Casper Ti. Vector]

That would be much cleaner.  Thanks :)

On Tue, May 02, 2017 at 02:40:56PM +, Laurent Bercot wrote:
>   Probably not. If anything, I'll think about the use of s6-svwait > oneshots
> a bit more, and if I assess it's the correct, or as correct as it gets,
> way to implement disjunctions, I'll add a mechanism in s6-rc-compile to
> encapsulate it and auto-generate hidden oneshot services - because
> those "utility" services should not be managed by users - they should
> be part of the magic instead.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Customise shutdown signal at the s6-rc level?

[Casper Ti. Vector]

On Tue, May 02, 2017 at 08:51:19AM +, Laurent Bercot wrote:
>   If I were to work on a more official, better integrated solution, I would
> do it at the s6-supervise level. I would not implement custom control
> scripts, for the reasons indicated in the above link, but it would
> probably be possible to implement a safer solution, such as reading a file
> containing the name of the signal to send when s6-svc -d is called.

I see.  Now I also think that using a `shutdown-signal' file seems to be
the least intrusive way.  Considering the hangup problem, I think the
format of the file can be described as something like
> signal_1 timeout_1 signal_2 timeout_2 ... signal_n [timeout_n]
where the last timeout, when present, indicates that SIGKILL shall be
sent if that timeout elapses; the default is obviously
> SIGTERM

>   Is there a real, important demand for this? I'd rather not do it and fix
> daemons that don't use SIGTERM to shutdown instead...

I think the problem is not daemons that don't use SIGTERM for shutdown,
but daemons that use SIGINT/SIGQUIT/SIGTERM for subtly different
flavours of shutdown.  PostgreSQL does this, as well as nginx and
php-fpm at least, but the signal semantics is again different in each
case.

Even if the use of SIGTERM is somehow unified to mean "fast but clean
shutdown", the sysadmin might for some reason consider "wait for all
clients to disconnect and then shut down cleanly" to be more
appropriate for his use case.  Since we aim to provide mechanisms
instead of policies (when reasonable), I think adding this feature is
acceptable.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On possibly "finer" dependencies in s6-rc

[Casper Ti. Vector]

Yes, I think this is the correct behaviour: if the user does not want
it, the warnings can be somehow filtered; on the other hand, there would
not be a trivial way to know such failures if the user wants it but
there is no warning in the first place.

By the way, if this change is done in s6-svwait, perhaps the s6-rc FAQ
can mention that dynamic (i.e. without using s6-rc-update manually)
virtual dependencies can be implemented using s6-svwait oneshots?

On Tue, May 02, 2017 at 08:59:05AM +, Laurent Bercot wrote:
>   Does a warning message to stderr when this happens sound appropriate
> to you?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Customise shutdown signal at the s6-rc level?

[Casper Ti. Vector]

While it is unwise to transform shutdown signals in s6-supervise [1], I
think the customisation is achievable in s6-rc by adding a file in the
service definition directory which specifies the signal (defaults to
SIGTERM) to send to the daemon process.  Any comments on this?

[1] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On possibly "finer" dependencies in s6-rc

[Casper Ti. Vector]

I just realised that we can set up a oneshot for the disjunction with
its dependencies being the intersection of the dependencies of all
services in the disjunction, with the `up' and `down' being something
like `s6-svwait -o -[ud] srv1 srv2 ...' ([ud]: `u' for `up', `d' for
`down'), so perhaps no change is need after all in s6-rc.

Nevertheless, s6-svwait -o seems to also fail if some failure occurs
before the wanted state is reached, eg. one `dnscrypt-proxy' instance
reporting permanent failure before another one comes up successfully.
So we might need to add an option to ignore such failures.

On Tue, Feb 14, 2017 at 09:31:47PM +, Laurent Bercot wrote:
>   If that kind of setup can be generalized, I'm not opposed to adding a
> "failover instance manager" set of programs to s6, because I believe it's
> reasonably in scope. I would much rather do that than add disjunction
> management to s6-rc.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On the feasibility of a shell that includes execline features

[Casper Ti. Vector]

Sorry for bumping this dormant thread, but I found scsh [1] really
interesting (possibly even more so than rc(1)) in this aspect: written
as a Scheme-embedded shell, it can make full use of the Scheme language,
which is both elegant and expressive (unrivaled by any other programming
language I know IMHO); using its syscall interfaces, chainloading would
be fairly easy to implement.

[1] <https://scsh.net/about/about.html>.

There are, inevitably, downsides:
* As of its latest release (v0.6.7 from 2006), scsh does not have a good
  interactive mode.
* scsh is, arguably, not very lightweight:
> -rw-rw-r-- 1 portage portage 9.0M Sep 15  2016 bash-4.4.tar.gz
> -rw-rw-r-- 1 portage portage 211K May 14  2015 rc-1.7.4.tar.gz
> -rw-rw-r-- 1 portage portage 4.2M Aug 31  2006 scsh-0.6.7.tar.gz
  However, considering the power of scsh as a language ("scsh is a Scheme
  system designed for writing useful standalone Unix programs and shell
  scripts: it spans a wide range of application, from 'script'
  applications usually handled with perl or sh, to more standard system
  applications usually written in C" [2]), I think the size of its
  codebase is justified.

[2] <https://scsh.net/docu/html/man-Z-H-2.html#node_chap_1>.

On Sun, Aug 21, 2016 at 11:00:43PM +0800, Casper Ti. Vector wrote:
> What do we need
> ---
> 
> Unfortunately, I am not a system programmer, and do not my current time
> schedule allow me to spend enough time to systematically learn it; but I
> think that a language that combines the advantages of shell and execline
> is not only a concept, but also a feasible and rewarding goal, which is
> worth a Unix programmer's efforts.  Thus I really wish somebody that is
> interested in process supervision and has the resource to try to realise
> this concept, probably by incorporating execline utilities into rc(1).

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On possibly "finer" dependencies in s6-rc

[Casper Ti. Vector]

What about a `dnsmasq' service depending on serveral `dnscrypt-proxy'
instances for failover, and can start when any of the instances become
ready?

On Tue, Sep 29, 2015 at 06:00:40PM +0200, Laurent Bercot wrote:
>   On a theoretical level, I tend to agree; and I will definitely
> think about handling virtual services in s6-rc if a real-world case
> happens.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [PATCH] supervise: set down (D) even on LASTFINISH

[Casper Ti. Vector]

Seems that the patch uses 4-space tabs while skaware uses 2-space tabs,
and that discrepancy was not addressed when the patch was applied...

On Sat, Jan 28, 2017 at 02:15:31PM +, Laurent Bercot wrote:
> Applied, thanks!

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 talk at FOSDEM 2017

[Casper Ti. Vector]

Now I see.  Thanks :)

On Thu, Jan 05, 2017 at 10:59:28AM +, Laurent Bercot wrote:
>   Please bear in mind that I did not intend these slides to be published
> as is, I only linked them to show FOSDEM organizers that there was
> material to work with - and they ended up on the public event page, oh well.
> But I know they need to be reworked - and as is they probably have very
> little to do with what's going to end up in the 15 minute condensed talk.
> So please don't focus too hard on those slides, they're not really what
> I'm working with at the moment.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 talk at FOSDEM 2017

[Casper Ti. Vector]

* p. 12: after skimming inittab(5), I am inclined to agree that the time
  to start `respawn' processes are undocumented.  But I do not think
  they are started in parallel with `wait' processes: whether on Gentoo
  or Alpine, I always find gettys to be started after `openrc default'.

* pp. 40-41: I think it is advisable to somehow emphasise chainloading.
  At the very least, a mention that resource control and (well, if
  someone really wants it :) cgroup support can be easily implemented
  with chainloading will be very instructive.

On Wed, Jan 04, 2017 at 11:38:10PM +, Laurent Bercot wrote:
> https://fosdem.org/2017/schedule/event/s6_supervision/

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C