Davin McCall's comparison of service and system management

[Jonathan de Boyne Pollard]

https://github.com/davmac314/dinit/blob/master/doc/COMPARISON#L71

It's a shame that this gets a number of things wrong about several of the
systems.

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Guillermo:
Nope, it's a 4.4-series kernel. 


I've wangled a later kernel out of Debian 8 backports.  (-:  I'm going 
to have to write v1 detection for you, then.  Alright.  1.31 is going to 
be slightly delayed.

Re: Adding capability control into the `run' script comparison page

[Guillermo]

2017-01-13 6:00 GMT-03:00 Jonathan de Boyne Pollard :
>
> Guillermo:
>>
>> (/sys/fs/cgroup itself is a tmpfs on my machine)
>
> Does your kernel support version 2 cgroupfs?

Nope, it's a 4.4-series kernel. I usually stick to longterm
maintainance releases, and I don't think there's any such realease
with cgroups v2 support yet. Also, not all cgroups v1 resource
controllers are available in cgroups v2, it seems:



"Due to the lack of consensus in the kernel community, the CPU
controller support on the unified control group hierarchy requires
out-of-tree kernel patches"? Hmmm :/

G.

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Jonathan de Boyne Pollard:
To anyone running the service manager and bundles from nosh version 
1.28 or later on Linux:  You are encouraged to look at your control 
group hierarchy, with a tool like "systemd-cgls /", with the "cgroup" 
field of the ps command, or by simply listing your /sys/fs/cgroup/ 
hierarchy.  You are in for an interesting surprise.


There are more interesting surprises in the same vein in 1.31.  I've put 
a sneak peak of the 1.31 Guide up for you.

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Guillermo:

(/sys/fs/cgroup itself is a tmpfs on my machine)


Does your kernel support version 2 cgroupfs?

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Guillermo:
I suppose the interesting suprise is that as consequence, when a 
service definition gets 'imported' to nosh from a unit file (and this 
covers pretty much everything in the nosh-bundles* binary 
packages),the corresponding service gets placed in a cgroup of its own 
when launched by nosh's service manager:


This also happens in per-user service management.

/service-manager.slice/user-services@.service:
└─user-services@jdebp.service
  ├─31423 per-user-manager
  ├─per-user-manager-log.slice
  │ └─31426 cyclog --max-file-size 32768 --max-total-size 1048576 .
  └─service-manager.slice
├─31427 service-manager
├─gvfs-daemon.service
│ └─9305 /usr/lib/gvfs/gvfsd
├─at-spi-dbus-bus.service
│ ├─9276 /usr/lib/at-spi2-core/at-spi-bus-launcher
│ ├─9290 /usr/bin/dbus-daemon --config-file=/etc/at-spi2/accessibility.co...
│ └─9293 /usr/lib/at-spi2-core/at-spi2-registryd --use-gnome-session
├─dbus-servers-log.service
│ └─30089 cyclog jdebp/dbus-servers/
├─pulseaudio.service
│ └─3625 pulseaudio --exit-idle-time=-1
├─mpd.service
│ └─31592 mpd --no-daemon --stderr --stdout --verbose ./mpd.conf
├─emacs.service
│ ├─6088 strace -f emacs --daemon
│ └─6094 emacs --daemon
├─cyclog@.service
│ ├─cyclog@pulseaudio.service
│ │ └─20138 cyclog jdebp/pulseaudio/
│ └─cyclog@mpd.service
│   └─18028 cyclog jdebp/mpd/
├─simple-servers-log.service
│ └─6070 cyclog jdebp/simple-servers/
└─gnome-terminal-server.service
  ├─9408 /usr/lib/gnome-terminal/gnome-terminal-server
  ├─9411 gnome-pty-helper
  ├─9412 zsh
  └─9451 sleep 6000

Re: Adding capability control into the `run' script comparison page

[Guillermo]

2016-12-07 6:26 GMT-03:00 Jean Louis:
>
> On Wed, Dec 07, 2016 at 09:14:00AM +, Jonathan de Boyne Pollard wrote:
>> [...]
>> To anyone running the service manager and bundles from nosh version 1.28 or
>> later on Linux:  You are encouraged to look at your control group hierarchy,
>> with a tool like "systemd-cgls /", with the "cgroup" field of the ps
>> command, or by simply listing your /sys/fs/cgroup/ hierarchy.  You are in
>> for an interesting surprise.
> [...]
>
> Please be more detailed, I am interested, but due to not using
> systemd, I ahve nothing in /sys/fs/cgroup
>
> What is the interesting surprise, explained?

nosh 1.28 and later includes a move-to-control-group chainloading
utility [1], and its systemd unit file conversion tool inserts a
move-to-control-group invocation in the resulting bundle directory's
scripts.

I suppose the interesting suprise is that as consequence, when a
service definition gets 'imported' to nosh from a unit file (and this
covers pretty much everything in the nosh-bundles* binary packages),
the corresponding service gets placed in a cgroup of its own when
launched by nosh's service manager:

test$ cat my-test-svc.service

[Unit]
Description=My test service

[Service]
User=test
ExecStart=/home/test/my-test-svc-binary
RestartSec=1

test$ system-control convert-systemd-units --no-systemd-quirks
./my-test-svc.service
test$ ls -l my-test-svc/*

my-test-svc/after:
total 0
lrwxrwxrwx 1 test test 34 Dec  7 22:25 basic ->
/etc/service-bundles/targets/basic

my-test-svc/before:
total 0
lrwxrwxrwx 1 test test 37 Dec  7 22:25 shutdown ->
/etc/service-bundles/targets/shutdown

my-test-svc/conflicts:
total 0

my-test-svc/required-by:
total 0

my-test-svc/service:
total 16
-rwxr-xr-x 1 test test 105 Dec  7 22:25 restart
-rwxr-xr-x 1 test test 183 Dec  7 22:25 run
-rwxr-xr-x 1 test test  66 Dec  7 22:25 start
-rwxr-xr-x 1 test test  65 Dec  7 22:25 stop

my-test-svc/stopped-by:
total 0
lrwxrwxrwx 1 test test 37 Dec  7 22:25 shutdown ->
/etc/service-bundles/targets/shutdown

my-test-svc/wanted-by:
total 0

my-test-svc/wants:
total 0
lrwxrwxrwx 1 test test 34 Dec  7 22:25 basic ->
/etc/service-bundles/targets/basic

test$ system-control cat ./my-test-svc

start:#!/bin/nosh
start:#Start file generated from ./my-test-svc.service
start:true
stop:#!/bin/nosh
stop:#Stop file generated from ./my-test-svc.service
stop:true
run:#!/bin/nosh
run:#Run file generated from ./my-test-svc.service
run:#My test service
run:move-to-control-group my-test-svc.service
run:setuidgid test
run:/home/test/my-test-svc-binary
restart:#!/bin/sh
restart:#Restart file generated from ./my-test-svc.service
restart:sleep 1
restart:exec true # ignore script arguments

root# mkdir /sys/fs/cgroup/systemd
root# mount -t cgroup -o rw,nosuid,nodev,noexec,name=systemd systemd
/sys/fs/cgroup/systemd
root# system-control start ./my-test-svc

test$ service-status my-test-svc

my-test-svc:
 State   : running since 2016-12-07 22:35:37 -0300; 7m 48s ago
 Main PID: 1499
  Started: exit 0
 Config  : enabled

test$ ps -eo pid,ppid,euser,cgroup,args

  PID  PPID EUSERCGROUP  COMMAND
 [...]
 1446 1 root -   service-manager
 1499  1446 test 6:name=systemd:/my-test-svc /home/test/my-test-svc-binary

Process 'my-test-svc-binary' is in cgroup 'my-test-svc.service' and
its parent process (PID 1446) is a process supervisor, as expected.

test$ ls -l /sys/fs/cgroup/systemd

total 0
drwxr-xr-x 2 root root 0 Dec  7 22:13 basic.target
-rw-r--r-- 1 root root 0 Dec  7 22:06 cgroup.clone_children
-rw-r--r-- 1 root root 0 Dec  7 22:06 cgroup.procs
-r--r--r-- 1 root root 0 Dec  7 22:06 cgroup.sane_behavior
drwxr-xr-x 2 root root 0 Dec  7 22:13 local-fs-pre.target
drwxr-xr-x 2 root root 0 Dec  7 22:13 local-fs.target
drwxr-xr-x 2 root root 0 Dec  7 22:35 my-test-svc.service
-rw-r--r-- 1 root root 0 Dec  7 22:06 notify_on_release
-rw-r--r-- 1 root root 0 Dec  7 22:06 release_agent
drwxr-xr-x 2 root root 0 Dec  7 22:13 swapauto.target
drwxr-xr-x 2 root root 0 Dec  7 22:13 swaplate.target
drwxr-xr-x 2 root root 0 Dec  7 22:13 sysinit.target
-rw-r--r-- 1 root root 0 Dec  7 22:06 tasks

(/sys/fs/cgroup itself is a tmpfs on my machine)

G.

[1] http://jdebp.eu/Softwares/nosh/guide/move-to-control-group.html

Re: Adding capability control into the `run' script comparison page

[Jean Louis]

On Wed, Dec 07, 2016 at 09:14:00AM +, Jonathan de Boyne Pollard wrote:
> Casper Ti. Vector:
> 
> > But I do think the capability argument has its validity: chainloading
> > is, at this time, not well known to normal users, which is why many
> > systemd supporters compulsorily identify cgroup support with systemd
> > with few people opposing. Therefore I suggest to add some examples of
> > capacility control (eg. one example for ulimit, plus one example for
> > cgroup) into the comparison page, or an independent page.
> > 
> Such "systemd supporters" don't actually know systemd.
> 
> * http://jdebp.eu./FGA/linux-control-groups-are-not-jobs.html
> 
> To anyone running the service manager and bundles from nosh version 1.28 or
> later on Linux:  You are encouraged to look at your control group hierarchy,
> with a tool like "systemd-cgls /", with the "cgroup" field of the ps
> command, or by simply listing your /sys/fs/cgroup/ hierarchy.  You are in
> for an interesting surprise.

I have ready your article, and understand it.

Please be more detailed, I am interested, but due to not using
systemd, I ahve nothing in /sys/fs/cgroup

What is the interesting surprise, explained?

Jean

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Casper Ti. Vector:

But I do think the capability argument has its validity: chainloading 
is, at this time, not well known to normal users, which is why many 
systemd supporters compulsorily identify cgroup support with systemd 
with few people opposing. Therefore I suggest to add some examples of 
capacility control (eg. one example for ulimit, plus one example for 
cgroup) into the comparison page, or an independent page.



Such "systemd supporters" don't actually know systemd.

* http://jdebp.eu./FGA/linux-control-groups-are-not-jobs.html

To anyone running the service manager and bundles from nosh version 1.28 
or later on Linux:  You are encouraged to look at your control group 
hierarchy, with a tool like "systemd-cgls /", with the "cgroup" field of 
the ps command, or by simply listing your /sys/fs/cgroup/ hierarchy.  
You are in for an interesting surprise.

Re: Adding capability control into the `run' script comparison page

[Casper Ti. Vector]

Many thanks :)

On Tue, Dec 06, 2016 at 12:53:14AM +, Jonathan de Boyne Pollard wrote:
> * http://jdebp.eu./Softwares/nosh/guide.html

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Casper Ti. Vector:

the docs are in tarballs on jdebp.eu


* http://jdebp.eu./Softwares/nosh/guide.html

Re: Adding capability control into the `run' script comparison page

[Casper Ti. Vector]

Sorry, my fault.  I read the page in a hurry, and thought the page did
not contain ulimit when the reply said capability control was not
involved in your page.  Impatience is really a sin :(

Nevertheless, if you do plan to create a separate page for cgroup
support, I think a brief introduction of chainloading with reference to
already employed capability control chainloaders (ulimit, user/group...)
in the init script comparison page would, to some extent, prepare the
impatient reader for the contents to come.

On Mon, Dec 05, 2016 at 09:31:20AM +, Jonathan de Boyne Pollard wrote:
> An irony here is that the page *already contains* two entire sets of 
> examples that set memory resource limits, using daemontools, 
> daemontools-encore, freedt, perp, s6, and nosh tools.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: Adding capability control into the `run' script comparison page

[Jonathan de Boyne Pollard]

Casper Ti. Vector:

one example for ulimit


An irony here is that the page *already contains* two entire sets of 
examples that set memory resource limits, using daemontools, 
daemontools-encore, freedt, perp, s6, and nosh tools.

Adding capability control into the `run' script comparison page

[Casper Ti. Vector]

nosh's `run' script comparison page [1] is a very excellent refutation
against the "non-systemd init scripts are full of boilerplate" argument.
Nevertheless, one systemd supporter argues [2] that the page is not
representative for the "advanced" features, including service dependency
and capability control (I consider cgroup support as part of this).  The
dependencies argument is easily solved using one link to the
documentation of the `dependencies' file in s6 (I know nosh supports it,
but the docs are in tarballs on jdebp.eu and has no direct link).

But I do think the capability argument has its validity: chainloading
is, at this time, not well known to normal users, which is why many
systemd supporters compulsorily identify cgroup support with systemd
with few people opposing.  Therefore I suggest to add some examples of
capacility control (eg. one example for ulimit, plus one example for
cgroup) into the comparison page, or an independent page.

(To be clear, I understand that some systemd supporters are just trolls
and would try anything possible to create the illusion that we are
incorrect; here I just want to help spread our information for those
that might convert.)

[1] <https://jdebp.eu/FGA/
 run-scripts-and-service-units-side-by-side.html>
[2] <https://linux.slashdot.org/comments.pl?sid=9952639&cid=53418483>

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 09:29:15 +0200
Laurent Bercot  wrote:

>   In the meantime, if you don't want to get your hands dirty,
> you can still use s6-svscan/s6-supervise as a process supervision
> system without trying to make it run as an init system, just as you
> can use runsvdir/runsv as a process supervision system without using
> the runit binary. That is real modularity, that is the main reason
> why I believe runit and s6 are better designed than
> other-init-software-out-there, and it would be *trivial* to port your
> "suckless init on Plop" setup from daemontools(-encore) to runit or
> s6, even if you don't use every feature those packages provide.

You could be right about that. I just finished doing the same setup,
except instead of Suckless->daemontools-encore, I did
Felker->s6-svscan. Once I cleaned up all my transposition errors, and
modified LittKit to work with S6's command names and s6-svstat's
output, it worked like a charm, and it appears to work just like
Felker->daemontools-encore.

Thanks,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

RE: comparison

[James Powell]

You really do have to cater to get a foothold in the door any more.

Sent from my Windows Phone

From: Laurent Bercot<mailto:ska-supervis...@skarnet.org>
Sent: ‎6/‎16/‎2015 4:24 PM
To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
Subject: Re: comparison

On 16/06/2015 22:32, post-sysv wrote:
> Soon systemd arrives with its promise of being a unified userspace
> toolkit that systems developers can supposedly just plug in and
> integrate without hassle to get X, Y and Z advantages. No more
> writing initscripts, no more setting policy because systemd will do
> as much as it can for you. A lazy package maintainer's dream,
> ostensibly.
>
> Everyone hops on the bandwagon and there you are. Now we get to hear
> about how systemd solves so many long-standing problems with the
> distributions, but I can't shake the feeling that many of them were
> self-inflicted through indifference and/or incompetence.

  I think you nailed it exactly. Inertia is the most potent force in
the universe.
  Basically, to get supervision suites - or anything else - adopted by
distributions, the only necessary thing is to do all their work for
them.

--
  Laurent

Re: comparison

[Laurent Bercot]

On 16/06/2015 22:32, post-sysv wrote:

Soon systemd arrives with its promise of being a unified userspace
toolkit that systems developers can supposedly just plug in and
integrate without hassle to get X, Y and Z advantages. No more
writing initscripts, no more setting policy because systemd will do
as much as it can for you. A lazy package maintainer's dream,
ostensibly.

Everyone hops on the bandwagon and there you are. Now we get to hear
about how systemd solves so many long-standing problems with the
distributions, but I can't shake the feeling that many of them were
self-inflicted through indifference and/or incompetence.


 I think you nailed it exactly. Inertia is the most potent force in
the universe.
 Basically, to get supervision suites - or anything else - adopted by
distributions, the only necessary thing is to do all their work for
them.

--
 Laurent

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 14:12:48 -0700
Avery Payne  wrote:

> 
> 
> On 6/16/2015 1:32 PM, post-sysv wrote:
> > Soon systemd arrives with its promise of being a unified userspace 
> > toolkit that systems developers can supposedly just plug in and 
> > integrate without hassle to get X, Y and Z advantages. No more
> > writing initscripts, no more setting policy because systemd will do
> > as much as it can for you. A lazy package maintainer's dream,
> > ostensibly.
> 
> That last sentence is telling.  It's also why a master catalog of 
> settings is so badly needed.
> 
> In my not very humble opinion, we really need a single point of 
> reference, driven by the community, shared and sharable, and publicly 
> visible.  I could envision something like a single website which
> would collect settings and store them, and if you needed settings, it
> would build all of the envdir files and download them in one giant
> dollop, probably a tarball.  Unpack the tarball and all of the envdir
> settings are there, waiting to be used.  You could even be "fancy"
> and track option flags through various daemon revisions, so that if
> you have an older service running, you tell it "I have older version
> x.y.z" and you get the "correct" flags and not the "current" ones.

I must be too cynical. I see that after the above described collection,
website, and envdirs (does that mean service directories) is somewhat
operational, a well funded "Open Source" vendor will flood it with
wheelbarrows of cash and a parade of developers, and that nice, simple
collection and web app becomes unfathomable (and has a 20K word terms
and conditions). But that's why they have paid support, right?

I'll now go back to talking about technology.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Avery Payne]

On 6/16/2015 1:32 PM, post-sysv wrote:
Soon systemd arrives with its promise of being a unified userspace 
toolkit that systems developers can supposedly just plug in and 
integrate without hassle to get X, Y and Z advantages. No more writing 
initscripts, no more setting policy because systemd will do as much as 
it can for you. A lazy package maintainer's dream, ostensibly.


That last sentence is telling.  It's also why a master catalog of 
settings is so badly needed.


In my not very humble opinion, we really need a single point of 
reference, driven by the community, shared and sharable, and publicly 
visible.  I could envision something like a single website which would 
collect settings and store them, and if you needed settings, it would 
build all of the envdir files and download them in one giant dollop, 
probably a tarball.  Unpack the tarball and all of the envdir settings 
are there, waiting to be used.  You could even be "fancy" and track 
option flags through various daemon revisions, so that if you have an 
older service running, you tell it "I have older version x.y.z" and you 
get the "correct" flags and not the "current" ones.


This service would not only act as a repository but it would also be a 
clearinghouse, a place where distributions could come and take what is 
needed.




Everyone hops on the bandwagon and there you are. Now we get to hear 
about how systemd solves so many long-standing problems with the 
distributions, but I can't shake the feeling that many of them were 
self-inflicted through indifference and/or incompetence.


After learning so much from my own odyssey, I'd like to be a little more 
forgiving and chalk it up to inexperience.  I have this scene running 
through my head as you describe the vendors looking for software nearly 
20 years ago: "We need an init...and something to start services...look, 
there's this Sys-V thingy lying around, grab it and make it work!"

RE: comparison

[James Powell]

Too true. Systemd offered the pot of gold as long as you handed it the keys to 
the kingdom.

Runit still requires work, but less work if a package for scripts exists. The 
most arbitrary need is the Stage scripts, which generally can utilize a general 
script setup, or translated init shell scripts.

I've worked with Runit enough to see that many distributions wanted 
ready-to-use, not more hackfests to redo run script are run script.

The problem is, Runit solved every problem with system management and service 
supervision before systemd, but because few were willing to create scripts, it 
fell off the radar. Runit is a very complete init+supervision suite in all 
respects and regards, but it needed scripts.

And believe me, after you debug a script about 10 times and sshd still refuses 
to stay in the up state and available, it makes you want to rip hair out.

Sent from my Windows Phone

From: post-sysv<mailto:boycottsyst...@openmailbox.org>
Sent: ‎6/‎16/‎2015 1:35 PM
To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
Subject: Re: comparison

Implying that the distributions would have transitioned from System
V-style to daemontools-style mechanisms?

Strongly doubt it.

For all the noise and controversy that's been happening over PID1, I
always got the impression that most distros back in the day simply
didn't care. They happily piled on hack after hack to their messy
sysvinit setups in the form of dependency header parsers (as
standardized by LSB and done through insserv(8), asynchronous executors
(startpar), generic tools like start-stop-daemon(8) to get around the
fact that they never wrote a common initscript library, symlinking
/bin/sh to dash to improve startup speeds, etc.

There were alternatives like initng (which Ubuntu for some reason
skipped over and settled on Upstart instead with its awkward event
model), depinit, minit, eINIT, daemontools derivatives and others for
quite a while, but mainstream distros by and large never expressed much
interest in them. Instead, they drowned themselves in quick fixes until
the boot process unsurprisingly became difficult to maintain.

Soon systemd arrives with its promise of being a unified userspace
toolkit that systems developers can supposedly just plug in and
integrate without hassle to get X, Y and Z advantages. No more writing
initscripts, no more setting policy because systemd will do as much as
it can for you. A lazy package maintainer's dream, ostensibly.

Everyone hops on the bandwagon and there you are. Now we get to hear
about how systemd solves so many long-standing problems with the
distributions, but I can't shake the feeling that many of them were
self-inflicted through indifference and/or incompetence.

On 06/16/2015 04:09 PM, James Powell wrote:
> And supervision-scripts has been that generic profile that can be used in 99% 
> of situations. Sadly, Avery, I wish we could have had your work about 8 years 
> ago. The UNIX world might have been a vastly different place.

Re: comparison

[Steve Litt]

That's an understatement!

And, as the old saying goes, the best time to plant a tree is 20 years
ago. The second best time is today.

Thanks Avery!

SteveT



On Tue, 16 Jun 2015 13:09:32 -0700
James Powell  wrote:

> And supervision-scripts has been that generic profile that can be
> used in 99% of situations. Sadly, Avery, I wish we could have had
> your work about 8 years ago. The UNIX world might have been a vastly
> different place.
> 
> Sent from my Windows Phone
> 
> From: Avery Payne<mailto:avery.p.pa...@gmail.com>
> Sent: ‎6/‎16/‎2015 11:26 AM
> To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
> Subject: Re: comparison
> 
> On 6/16/2015 5:22 AM, James Powell wrote:
> > Very true, but something always seems to say something along the
> > lines of "if we had done #2 years ago, we might have avoided a huge
> > mess that now exists".
> Agreed.
> > The same applies to init systems. If there are ready to use feet
> > wetting, taste testing scripts ready to go, the job of importing
> > things just gets easier on the distribution.
> Also agreed.  Actually, there's some discussion on the mailing list
> from a few months back about this.
> > 
> > From: Steve Litt<mailto:sl...@troubleshooters.com>
> > Sent: ‎6/‎16/‎2015 4:45 AM
> > To:
> > supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
> > Subject: Re: comparison
> >
> > On Tue, 16 Jun 2015 04:05:29 -0700
> > James Powell  wrote:
> >
> >> I agree Laurent. Though, even though complete init+supervision
> >> systems like Runit exist, it's been nearly impossible to get a
> >> foothold with any alternatives to sysvinit and systemd
> >> effectively. I think one of the major setbacks has been the lack
> >> of ready-to-use script sets, like those included with OpenRC,
> >> various rehashes of sysvinit and bsdinit scripts, and systemd
> >> units just aren't there ready to go.
> The true problem is that each daemon needs its own special environment
> variables, command flags, and other gobbledygook that is specific to
> getting it up and running, and a master catalog of all settings
> doesn't exist.  Compounding that is the normal and inevitable need
> for each supervision author to do their own thing, in their own way,
> so tools get renamed, flags get mapped, return codes aren't
> consistent.  That's just the framework, we haven't talked about run
> scripts yet.  Who wants to write hundreds of scripts?  Each
> hand-cobbled script is an error-prone task, and that implies the
> potential for hundreds of errors, bugs, strange behaviors, etc.
> 
> This is the _entire_ reason for supervision-scripts.  It was meant to
> be a generic "one size fits most" solution to providing prefabricated
> run scripts, easing or removing the burden for package maintainers,
> system builders, etc.  All of the renaming and flags and options and
> environment settings and other things are abstracted away as variables
> that are correctly set for whatever environment you have.  With all of
> that out of the way, it becomes much easier to actually write scripts
> to launch things under multiple environments.  A single master script
> handles it all, reduces debugging, and can be easily swapped out to
> support chainload launchers from s6 and nosh.
> 
> The opposite end of this is Laurent's proposal to compile the scripts
> so they are "built into existence".  If I'm understanding / imagining
> this correctly, this would take all of the settings and with a
> makefile "bake" each script into existence with all of the steps and
> settings needed.  It would in effect provide the same thing I am
> doing but it would make it static to the environment. There's nothing
> wrong with the approach, and the end result is the same.
> 
> The only difference between Laurent's approach and mine, is that
> Laurent's would need to "re-bake" your scripts if your framework
> changes; in my project, you simply run a single script and all of the
> needed settings change on the fly.  I'm not sure of the pros/cons to
> either approach, as I would hazard a guess that any system switching
> between frameworks may also require a reboot if a new init is desired.
> 
> Here's the rub: in both cases, the settings for each
> service/daemon/whatever are key to getting things running.  Again, we
> come back to the idea of a "master catalog of settings".  If it
> existed, then half of the problem would be resolved.  There are lots
> of 

Re: comparison

[post-sysv]

Implying that the distributions would have transitioned from System 
V-style to daemontools-style mechanisms?


Strongly doubt it.

For all the noise and controversy that's been happening over PID1, I 
always got the impression that most distros back in the day simply 
didn't care. They happily piled on hack after hack to their messy 
sysvinit setups in the form of dependency header parsers (as 
standardized by LSB and done through insserv(8), asynchronous executors 
(startpar), generic tools like start-stop-daemon(8) to get around the 
fact that they never wrote a common initscript library, symlinking 
/bin/sh to dash to improve startup speeds, etc.


There were alternatives like initng (which Ubuntu for some reason 
skipped over and settled on Upstart instead with its awkward event 
model), depinit, minit, eINIT, daemontools derivatives and others for 
quite a while, but mainstream distros by and large never expressed much 
interest in them. Instead, they drowned themselves in quick fixes until 
the boot process unsurprisingly became difficult to maintain.


Soon systemd arrives with its promise of being a unified userspace 
toolkit that systems developers can supposedly just plug in and 
integrate without hassle to get X, Y and Z advantages. No more writing 
initscripts, no more setting policy because systemd will do as much as 
it can for you. A lazy package maintainer's dream, ostensibly.


Everyone hops on the bandwagon and there you are. Now we get to hear 
about how systemd solves so many long-standing problems with the 
distributions, but I can't shake the feeling that many of them were 
self-inflicted through indifference and/or incompetence.


On 06/16/2015 04:09 PM, James Powell wrote:

And supervision-scripts has been that generic profile that can be used in 99% 
of situations. Sadly, Avery, I wish we could have had your work about 8 years 
ago. The UNIX world might have been a vastly different place.

RE: comparison

[James Powell]

And supervision-scripts has been that generic profile that can be used in 99% 
of situations. Sadly, Avery, I wish we could have had your work about 8 years 
ago. The UNIX world might have been a vastly different place.

Sent from my Windows Phone

From: Avery Payne<mailto:avery.p.pa...@gmail.com>
Sent: ‎6/‎16/‎2015 11:26 AM
To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
Subject: Re: comparison

On 6/16/2015 5:22 AM, James Powell wrote:
> Very true, but something always seems to say something along the lines of "if 
> we had done #2 years ago, we might have avoided a huge mess that now exists".
Agreed.
> The same applies to init systems. If there are ready to use feet wetting, 
> taste testing scripts ready to go, the job of importing things just gets 
> easier on the distribution.
Also agreed.  Actually, there's some discussion on the mailing list from
a few months back about this.
> 
> From: Steve Litt<mailto:sl...@troubleshooters.com>
> Sent: ‎6/‎16/‎2015 4:45 AM
> To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
> Subject: Re: comparison
>
> On Tue, 16 Jun 2015 04:05:29 -0700
> James Powell  wrote:
>
>> I agree Laurent. Though, even though complete init+supervision
>> systems like Runit exist, it's been nearly impossible to get a
>> foothold with any alternatives to sysvinit and systemd effectively. I
>> think one of the major setbacks has been the lack of ready-to-use
>> script sets, like those included with OpenRC, various rehashes of
>> sysvinit and bsdinit scripts, and systemd units just aren't there
>> ready to go.
The true problem is that each daemon needs its own special environment
variables, command flags, and other gobbledygook that is specific to
getting it up and running, and a master catalog of all settings doesn't
exist.  Compounding that is the normal and inevitable need for each
supervision author to do their own thing, in their own way, so tools get
renamed, flags get mapped, return codes aren't consistent.  That's just
the framework, we haven't talked about run scripts yet.  Who wants to
write hundreds of scripts?  Each hand-cobbled script is an error-prone
task, and that implies the potential for hundreds of errors, bugs,
strange behaviors, etc.

This is the _entire_ reason for supervision-scripts.  It was meant to be
a generic "one size fits most" solution to providing prefabricated run
scripts, easing or removing the burden for package maintainers, system
builders, etc.  All of the renaming and flags and options and
environment settings and other things are abstracted away as variables
that are correctly set for whatever environment you have.  With all of
that out of the way, it becomes much easier to actually write scripts to
launch things under multiple environments.  A single master script
handles it all, reduces debugging, and can be easily swapped out to
support chainload launchers from s6 and nosh.

The opposite end of this is Laurent's proposal to compile the scripts so
they are "built into existence".  If I'm understanding / imagining this
correctly, this would take all of the settings and with a makefile
"bake" each script into existence with all of the steps and settings
needed.  It would in effect provide the same thing I am doing but it
would make it static to the environment. There's nothing wrong with the
approach, and the end result is the same.

The only difference between Laurent's approach and mine, is that
Laurent's would need to "re-bake" your scripts if your framework
changes; in my project, you simply run a single script and all of the
needed settings change on the fly.  I'm not sure of the pros/cons to
either approach, as I would hazard a guess that any system switching
between frameworks may also require a reboot if a new init is desired.

Here's the rub: in both cases, the settings for each
service/daemon/whatever are key to getting things running.  Again, we
come back to the idea of a "master catalog of settings".  If it existed,
then half of the problem would be resolved.  There are lots of examples
out there, but, they're not all in one place.

So I try to toil over supervision-scripts when I get time, and make that
catalog.  Even if people don't like what I'm doing with the master run
script itself, that doesn't matter.  *What matters is that I've managed
to capture the settings for the various daemons, along with some
annotations*.  Because I took the time to support envdir, and the
settings for each daemon are stored in this format, those settings can
be extracted and used elsewhere.  I'm slowly creating that "master
catalog" in a plaintext format that can be read and processed easily.
This is the real, hidden value of supervision-scripts.

By the way, I'm going to bite the bullet and switch off of MPL 2.0 soon,
probably by month-end.

Re: comparison

[Avery Payne]

On 6/16/2015 5:22 AM, James Powell wrote:

Very true, but something always seems to say something along the lines of "if we had 
done #2 years ago, we might have avoided a huge mess that now exists".

Agreed.

The same applies to init systems. If there are ready to use feet wetting, taste 
testing scripts ready to go, the job of importing things just gets easier on 
the distribution.
Also agreed.  Actually, there's some discussion on the mailing list from 
a few months back about this.


From: Steve Litt<mailto:sl...@troubleshooters.com>
Sent: ‎6/‎16/‎2015 4:45 AM
To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
Subject: Re: comparison

On Tue, 16 Jun 2015 04:05:29 -0700
James Powell  wrote:


I agree Laurent. Though, even though complete init+supervision
systems like Runit exist, it's been nearly impossible to get a
foothold with any alternatives to sysvinit and systemd effectively. I
think one of the major setbacks has been the lack of ready-to-use
script sets, like those included with OpenRC, various rehashes of
sysvinit and bsdinit scripts, and systemd units just aren't there
ready to go.
The true problem is that each daemon needs its own special environment 
variables, command flags, and other gobbledygook that is specific to 
getting it up and running, and a master catalog of all settings doesn't 
exist.  Compounding that is the normal and inevitable need for each 
supervision author to do their own thing, in their own way, so tools get 
renamed, flags get mapped, return codes aren't consistent.  That's just 
the framework, we haven't talked about run scripts yet.  Who wants to 
write hundreds of scripts?  Each hand-cobbled script is an error-prone 
task, and that implies the potential for hundreds of errors, bugs, 
strange behaviors, etc.


This is the _entire_ reason for supervision-scripts.  It was meant to be 
a generic "one size fits most" solution to providing prefabricated run 
scripts, easing or removing the burden for package maintainers, system 
builders, etc.  All of the renaming and flags and options and 
environment settings and other things are abstracted away as variables 
that are correctly set for whatever environment you have.  With all of 
that out of the way, it becomes much easier to actually write scripts to 
launch things under multiple environments.  A single master script 
handles it all, reduces debugging, and can be easily swapped out to 
support chainload launchers from s6 and nosh.


The opposite end of this is Laurent's proposal to compile the scripts so 
they are "built into existence".  If I'm understanding / imagining this 
correctly, this would take all of the settings and with a makefile 
"bake" each script into existence with all of the steps and settings 
needed.  It would in effect provide the same thing I am doing but it 
would make it static to the environment. There's nothing wrong with the 
approach, and the end result is the same.


The only difference between Laurent's approach and mine, is that 
Laurent's would need to "re-bake" your scripts if your framework 
changes; in my project, you simply run a single script and all of the 
needed settings change on the fly.  I'm not sure of the pros/cons to 
either approach, as I would hazard a guess that any system switching 
between frameworks may also require a reboot if a new init is desired.


Here's the rub: in both cases, the settings for each 
service/daemon/whatever are key to getting things running.  Again, we 
come back to the idea of a "master catalog of settings".  If it existed, 
then half of the problem would be resolved.  There are lots of examples 
out there, but, they're not all in one place.


So I try to toil over supervision-scripts when I get time, and make that 
catalog.  Even if people don't like what I'm doing with the master run 
script itself, that doesn't matter.  *What matters is that I've managed 
to capture the settings for the various daemons, along with some 
annotations*.  Because I took the time to support envdir, and the 
settings for each daemon are stored in this format, those settings can 
be extracted and used elsewhere.  I'm slowly creating that "master 
catalog" in a plaintext format that can be read and processed easily.  
This is the real, hidden value of supervision-scripts.


By the way, I'm going to bite the bullet and switch off of MPL 2.0 soon, 
probably by month-end.

Re: comparison

[Avery Payne]

On 6/15/2015 9:00 PM, Colin Booth wrote:

I only know s6 and runit well enough to comment on for the most part but
filling in some blanks on your matrix:

Updated, thanks for the help.  As I said, it's a start.  It'll need some 
time to improve.  I mostly needed it for the project, to help me keep 
the mapping of "what tool does what action" straight so I can move 
forward.  I'd like to add some of the missing "specialty" tools that s6 
and nosh provides, and see if there are equivalent mappings elsewhere.  
Also, as new scripting frameworks are discovered, I'll add them as well.

RE: comparison

[James Powell]

Very true, but something always seems to say something along the lines of "if 
we had done #2 years ago, we might have avoided a huge mess that now exists".

Runit could have been the successor to sysvinit years ago, but like anything, 
unless there is something tangible to import with less work, rather than more 
work, things don't go into usage.

To put it another way, if CUPS had no drivers how useful would it be?

The same applies to init systems. If there are ready to use feet wetting, taste 
testing scripts ready to go, the job of importing things just gets easier on 
the distribution.

I often think and wonder if this was part of the allure of systemd, and I'm 
possibly correct.

Sent from my Windows Phone

From: Steve Litt<mailto:sl...@troubleshooters.com>
Sent: ‎6/‎16/‎2015 4:45 AM
To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
Subject: Re: comparison

On Tue, 16 Jun 2015 04:05:29 -0700
James Powell  wrote:

> I agree Laurent. Though, even though complete init+supervision
> systems like Runit exist, it's been nearly impossible to get a
> foothold with any alternatives to sysvinit and systemd effectively. I
> think one of the major setbacks has been the lack of ready-to-use
> script sets, like those included with OpenRC, various rehashes of
> sysvinit and bsdinit scripts, and systemd units just aren't there
> ready to go.
>
> Testing and trying to debug in house scripts is a pain and to be
> honest stalled our work with LFS a while back.
>
> Runit is one of the most complete alternatives out there, but if
> scripts are what is holding things back, why has this never been
> accurately addressed?
>
> -Jim

I think part of the difficulty of writing run scripts is there are two
different kinds of runscripts:

1) Simple system specific custom made run script.

2) Works everywhere, regardless of software constellation, one size
   fits all run script.

Distros and packages and "upstreams" make #2, which are very, very,
very difficult. In my Suckless Init plus daemontools-encore adventure,
I had a 200+ line "one size fits all" sysvinit init script degenerate
into a less than 20 line system specific daemontools-encore run script.

Personally, if I were a Linux distribution script maker, at the very
least I would assume I'm working with the Linux kernel and the "stuff"
provided by the distro release for which the script was made. That
would cut down on a lot of the tl;dr cotton candy.

SteveT

Steve Litt
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 04:05:29 -0700
James Powell  wrote:

> I agree Laurent. Though, even though complete init+supervision
> systems like Runit exist, it's been nearly impossible to get a
> foothold with any alternatives to sysvinit and systemd effectively. I
> think one of the major setbacks has been the lack of ready-to-use
> script sets, like those included with OpenRC, various rehashes of
> sysvinit and bsdinit scripts, and systemd units just aren't there
> ready to go.
> 
> Testing and trying to debug in house scripts is a pain and to be
> honest stalled our work with LFS a while back.
> 
> Runit is one of the most complete alternatives out there, but if
> scripts are what is holding things back, why has this never been
> accurately addressed?
> 
> -Jim

I think part of the difficulty of writing run scripts is there are two
different kinds of runscripts:

1) Simple system specific custom made run script.

2) Works everywhere, regardless of software constellation, one size
   fits all run script.

Distros and packages and "upstreams" make #2, which are very, very,
very difficult. In my Suckless Init plus daemontools-encore adventure,
I had a 200+ line "one size fits all" sysvinit init script degenerate
into a less than 20 line system specific daemontools-encore run script.

Personally, if I were a Linux distribution script maker, at the very
least I would assume I'm working with the Linux kernel and the "stuff"
provided by the distro release for which the script was made. That
would cut down on a lot of the tl;dr cotton candy.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 09:29:15 +0200
Laurent Bercot  wrote:

> On 16/06/2015 04:54, Steve Litt wrote:
> > One thing I can tell you is that daemontools and daemontools-encore
> > were never intended to be init systems, whereas I'm pretty sure that
> > runit, s6 and nosh intended to be part or all of an init system.
> 
>   You keep saying that, but at least in the case of runit and s6, it
> is inaccurate.
> 
>   runit and s6 are process supervision systems, just like daemontools,
> and can be used with any init system. There are documentation pages
> that prove it:
>http://smarden.org/runit/useinit.html
>http://skarnet.org/software/s6/s6-svscan-not-1.html

Yes but:

http://smarden.org/runit/replaceinit.html#sysv
http://skarnet.org/software/s6/s6-svscan-1.html

There's no way I've found to do
init=/any/file/of/daemontools(-encore)

> 
>   That's one thing. When you say "s6 is more complex than
> daemontools", it's only more complex because there are more binaries,
> and more features, 

That's what I meant. s6 is a superset of daemontools, making it a
little harder *for me, personally* to find my way around. Also, IIRC I
couldn't compile s6 the last time, but the last time I knew nothing
about the whole idea of init, so let me suspend saying it's more
complicated until I've logged some time actually using s6.

[snip]

>   Now, the thing is, when you have a process supervision system, it
> makes your stock init redundant. init has two jobs:
>   A. reap zombies

Do you include "listen for and handle relevant signals" in "reap
zombies"?

>   B. maintain at least one other process alive, so that if everything
> on the machine is killed (save init itself), it is still possible to
> recover and avoid a hard and dirty reboot.
> 
>   Note that suckless init, or Rich Felker's suggested init in the
> otherwise excellent http://ewontfix.com/14/ article, do not perform
> B, and so, are *incorrect*. The error is minute, and probably
> inconsequential in any real life situation, but it is still there;
> and if you want the smallest possible pid 1 that will keep your
> machine usable under the most dire circumstances, you should not use
> suckless init, you should use runit.

Yes. Someday I might change Suckless Init so that it *respawns*
(supervises, whatever you want to call it) /bin/rc.init. I would also
need a way that such a modified Suckless Init can pass along the
information that this isn't the first time this boot, in case there are
tasks I don't want to redo in /bin/rc.init.
 
[snip]
> And neither I
> nor Gerrit did it first; the first one was Paul Jarc, who pioneered
> the setup using... daemontools. http://code.dogmap.org/svscan-1/

I'd forgotten about that article. Last time I read it, I didn't
understand it at all. This time, when I read it, it reminded me that my
simple /bin/rc.shutdown stage 3 means I'm short circuiting the logs
during final shutdown.

> 
>   Yes, daemontools was never intended to be an init system, but even
> back then there was interest in running it as such, and Paul's
> experiments are what started it all.
> 
>   The problem with init is that it's not only stage 2, and stage 1
> and 3 are heavily system-dependent; so making svscan work as init is
> possible, but requires good hacking skills.

Also, even after you get into stage 2, those "services" that do system
initialization require hacking skills.

> 
>   Gerrit recognized that, so he basically forfeited the idea -
> instead of using runsvdir as process 1, he created the minimal amount
> of pid 1 code necessary to handle stages 1 and 3, as well as any
> stage 2 (and runsvdir is the obvious choice for the stage 2 manager).
> runit is a basic process supervision system (runsv/runsvdir) with a
> simple init wrapper (runit).
> 
>   I did not want to give up on the idea; I designed s6-svscan so that
> it would be easier to run as pid 1 than daemontools' svscan, and
> managed to get something working reproducibly and in an almost
> automatable way, but as you experienced, the setup still requires
> some hacking skills, and I need more time to work around the
> non-portability issues and come up with a full turnkey init.
> 
>   In the meantime, if you don't want to get your hands dirty,
> you can still use s6-svscan/s6-supervise as a process supervision
> system without trying to make it run as an init system, just as you
> can use runsvdir/runsv as a process supervision system without using
> the runit binary. That is real modularity, that is the main reason
> why I believe runit and s6 are better designed than
> other-init-software-out-there, and it would be *trivial* to port your
> "suckless init on Plop" setup from daemontools(-encore) to runit or
> s6, even if you don't use every feature those packages provide.


I'll be doing an s6 install in the next few weeks.

Thanks,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

RE: comparison

[James Powell]

I agree Laurent. Though, even though complete init+supervision systems like 
Runit exist, it's been nearly impossible to get a foothold with any 
alternatives to sysvinit and systemd effectively. I think one of the major 
setbacks has been the lack of ready-to-use script sets, like those included 
with OpenRC, various rehashes of sysvinit and bsdinit scripts, and systemd 
units just aren't there ready to go.

Testing and trying to debug in house scripts is a pain and to be honest stalled 
our work with LFS a while back.

Runit is one of the most complete alternatives out there, but if scripts are 
what is holding things back, why has this never been accurately addressed?

-Jim

Sent from my Windows Phone

From: Laurent Bercot<mailto:ska-supervis...@skarnet.org>
Sent: ‎6/‎16/‎2015 12:29 AM
To: supervision@list.skarnet.org<mailto:supervision@list.skarnet.org>
Subject: Re: comparison

On 16/06/2015 04:54, Steve Litt wrote:
> One thing I can tell you is that daemontools and daemontools-encore
> were never intended to be init systems, whereas I'm pretty sure that
> runit, s6 and nosh intended to be part or all of an init system.

  You keep saying that, but at least in the case of runit and s6, it is
inaccurate.

  runit and s6 are process supervision systems, just like daemontools,
and can be used with any init system. There are documentation pages
that prove it:
   http://smarden.org/runit/useinit.html
   http://skarnet.org/software/s6/s6-svscan-not-1.html

  That's one thing. When you say "s6 is more complex than daemontools",
it's only more complex because there are more binaries, and more
features, but you can use s6 exactly as you use daemontools - just add
s6- prefixes to the daemontools command names and it will pretty much
work the same. You can tell suckless init, or whatever pid 1 you fancy,
to run s6-svscan, or runsvdir, exactly the same way you're telling it
to run svscan.
  (The main difference is that daemontools comes with a svscanboot script
that uses the "readproctitle" logging hack, and runit and s6 don't.
As far as s6 is concerned: readproctitle felt too much like a cop-out to
me, and I wanted a proper logging system for logs falling through
s6-svscan's stderr, which is why I'm using a fifo as shown in the
examples/s6-svscanboot script and the
examples/ROOT/img/tmpfs/service/s6-svscan-log service directory.)

  Now, the thing is, when you have a process supervision system, it makes
your stock init redundant. init has two jobs:
  A. reap zombies
  B. maintain at least one other process alive, so that if everything on the
machine is killed (save init itself), it is still possible to recover and
avoid a hard and dirty reboot.

  Note that suckless init, or Rich Felker's suggested init in the otherwise
excellent http://ewontfix.com/14/ article, do not perform B, and so, are
*incorrect*. The error is minute, and probably inconsequential in any real
life situation, but it is still there; and if you want the smallest possible
pid 1 that will keep your machine usable under the most dire circumstances,
you should not use suckless init, you should use runit.

  Process supervision systems automatically perform B, that's their very job;
and svscan/runsvdir/s6-svscan automatically reap every zombie they get
because they have to collect their dead supervisor processes. So they
basically do everything init has to do in stage 2! It's then a natural idea
to try and use them as pid 1. And neither I nor Gerrit did it first; the
first one was Paul Jarc, who pioneered the setup using... daemontools.
   http://code.dogmap.org/svscan-1/

  Yes, daemontools was never intended to be an init system, but even back then
there was interest in running it as such, and Paul's experiments are what
started it all.

  The problem with init is that it's not only stage 2, and stage 1 and 3 are
heavily system-dependent; so making svscan work as init is possible, but
requires good hacking skills.

  Gerrit recognized that, so he basically forfeited the idea - instead of
using runsvdir as process 1, he created the minimal amount of pid 1 code
necessary to handle stages 1 and 3, as well as any stage 2 (and runsvdir
is the obvious choice for the stage 2 manager). runit is a basic process
supervision system (runsv/runsvdir) with a simple init wrapper (runit).

  I did not want to give up on the idea; I designed s6-svscan so that it
would be easier to run as pid 1 than daemontools' svscan, and managed to
get something working reproducibly and in an almost automatable way, but as
you experienced, the setup still requires some hacking skills, and I need
more time to work around the non-portability issues and come up with a full
turnkey init.

  In the meantime, if you don't want to get your hands dirty,
you can still use s6-svscan/s6-supervise as a process supervision system
without 

Re: comparison

[Laurent Bercot]

On 16/06/2015 04:54, Steve Litt wrote:

One thing I can tell you is that daemontools and daemontools-encore
were never intended to be init systems, whereas I'm pretty sure that
runit, s6 and nosh intended to be part or all of an init system.


 You keep saying that, but at least in the case of runit and s6, it is
inaccurate.

 runit and s6 are process supervision systems, just like daemontools,
and can be used with any init system. There are documentation pages
that prove it:
  http://smarden.org/runit/useinit.html
  http://skarnet.org/software/s6/s6-svscan-not-1.html

 That's one thing. When you say "s6 is more complex than daemontools",
it's only more complex because there are more binaries, and more
features, but you can use s6 exactly as you use daemontools - just add
s6- prefixes to the daemontools command names and it will pretty much
work the same. You can tell suckless init, or whatever pid 1 you fancy,
to run s6-svscan, or runsvdir, exactly the same way you're telling it
to run svscan.
 (The main difference is that daemontools comes with a svscanboot script
that uses the "readproctitle" logging hack, and runit and s6 don't.
As far as s6 is concerned: readproctitle felt too much like a cop-out to
me, and I wanted a proper logging system for logs falling through
s6-svscan's stderr, which is why I'm using a fifo as shown in the
examples/s6-svscanboot script and the
examples/ROOT/img/tmpfs/service/s6-svscan-log service directory.)

 Now, the thing is, when you have a process supervision system, it makes
your stock init redundant. init has two jobs:
 A. reap zombies
 B. maintain at least one other process alive, so that if everything on the
machine is killed (save init itself), it is still possible to recover and
avoid a hard and dirty reboot.

 Note that suckless init, or Rich Felker's suggested init in the otherwise
excellent http://ewontfix.com/14/ article, do not perform B, and so, are
*incorrect*. The error is minute, and probably inconsequential in any real
life situation, but it is still there; and if you want the smallest possible
pid 1 that will keep your machine usable under the most dire circumstances,
you should not use suckless init, you should use runit.

 Process supervision systems automatically perform B, that's their very job;
and svscan/runsvdir/s6-svscan automatically reap every zombie they get
because they have to collect their dead supervisor processes. So they
basically do everything init has to do in stage 2! It's then a natural idea
to try and use them as pid 1. And neither I nor Gerrit did it first; the
first one was Paul Jarc, who pioneered the setup using... daemontools.
  http://code.dogmap.org/svscan-1/

 Yes, daemontools was never intended to be an init system, but even back then
there was interest in running it as such, and Paul's experiments are what
started it all.

 The problem with init is that it's not only stage 2, and stage 1 and 3 are
heavily system-dependent; so making svscan work as init is possible, but
requires good hacking skills.

 Gerrit recognized that, so he basically forfeited the idea - instead of
using runsvdir as process 1, he created the minimal amount of pid 1 code
necessary to handle stages 1 and 3, as well as any stage 2 (and runsvdir
is the obvious choice for the stage 2 manager). runit is a basic process
supervision system (runsv/runsvdir) with a simple init wrapper (runit).

 I did not want to give up on the idea; I designed s6-svscan so that it
would be easier to run as pid 1 than daemontools' svscan, and managed to
get something working reproducibly and in an almost automatable way, but as
you experienced, the setup still requires some hacking skills, and I need
more time to work around the non-portability issues and come up with a full
turnkey init.

 In the meantime, if you don't want to get your hands dirty,
you can still use s6-svscan/s6-supervise as a process supervision system
without trying to make it run as an init system, just as you can use
runsvdir/runsv as a process supervision system without using the runit
binary. That is real modularity, that is the main reason why I believe
runit and s6 are better designed than other-init-software-out-there, and
it would be *trivial* to port your "suckless init on Plop" setup from
daemontools(-encore) to runit or s6, even if you don't use every feature
those packages provide.

--
 Laurent

Re: comparison

[Colin Booth]

I only know s6 and runit well enough to comment on for the most part but
filling in some blanks on your matrix:

Daemontools is public domain, freedt is the OpenBSD license (which doesn't
appear to currently exist according to the OpenBSD licensing page). I'd
treat it like the ISC license. Daemontools-encore is pretty ISC-like as
well.

s6 builds with musl and should build with any libc. Ditto runit.

s6 has a handful of example scripts that can be used as part of a simple
system setup or as templates.

Runit and s6 both have a svscanboot analog though at least in the case of
s6 it's buried in the examples directories. In runit's case you just use
the stage2 script. Both nominally exec into the real supervisor root,
though in s6's case it also shuffles around some file descriptors in order
to use a real logger and to guarantee that no logs get lost.

'sv status $dir > /dev/null' is clunky but functionally equivalent to svok.

I'm assuming by launcher you mean run script language. AFAIK all
supervisors can use scripts in any language as long as they are named "run"
and are executable, though why anyone would use a script written in python
or perl (for example) as a run script I don't know.

runsvdir does readproctitle-style self-renaming. s6-svscan runs an instance
of s6-log to handle all catch-all logging.

runit has svlogd as a multilog analog though the logging script is written
differently.

Cheers!

Re: comparison

[Steve Litt]

On Mon, 15 Jun 2015 17:37:45 -0700
Buck Evan  wrote:

> Is there any resource that compares the capabilities of daemontools,
> daemontools-encore, runit, s6, and friends?

One more thing: It's been my subjective impression that
daemontools-encore is a lot easier to install, set up, and keep running
than original daemontools.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Steve Litt]

On Mon, 15 Jun 2015 17:37:45 -0700
Buck Evan  wrote:

> Is there any resource that compares the capabilities of daemontools,
> daemontools-encore, runit, s6, and friends?

One thing I can tell you is that daemontools and daemontools-encore
were never intended to be init systems, whereas I'm pretty sure that
runit, s6 and nosh intended to be part or all of an init system. In
fact, I used runit as a complete init system, and it worked well.

Regardless of the intent of daemontools and daemontools-encore, I just
this week used daemontools to do almost all the system initialization
for Plop Linux, and all the process management. PID1 was just an 89
line C program that forked /bin/rc.init, then wait() spun killing
zombies and handling relevant signals. /bin/rc.init set the computer up
enough that daemontools-encore would work, and then exec'ed
daemontools-encore.

HTH,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Avery Payne]

I'm working on something similar, but you're asking for capabilities, 
and most of what I have is a mapping.  I've tried to include a few 
critical links in the comparison for the various homepages, licenses, 
source code, etc.  It's incomplete for now, but it's a start.


https://bitbucket.org/avery_payne/supervision-scripts/wiki/comparison



On 6/15/2015 5:37 PM, Buck Evan wrote:

Is there any resource that compares the capabilities of daemontools,
daemontools-encore, runit, s6, and friends?

comparison

[Buck Evan]

Is there any resource that compares the capabilities of daemontools,
daemontools-encore, runit, s6, and friends?