Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 09:29:15 +0200
Laurent Bercot  wrote:

>   In the meantime, if you don't want to get your hands dirty,
> you can still use s6-svscan/s6-supervise as a process supervision
> system without trying to make it run as an init system, just as you
> can use runsvdir/runsv as a process supervision system without using
> the runit binary. That is real modularity, that is the main reason
> why I believe runit and s6 are better designed than
> other-init-software-out-there, and it would be *trivial* to port your
> "suckless init on Plop" setup from daemontools(-encore) to runit or
> s6, even if you don't use every feature those packages provide.

You could be right about that. I just finished doing the same setup,
except instead of Suckless->daemontools-encore, I did
Felker->s6-svscan. Once I cleaned up all my transposition errors, and
modified LittKit to work with S6's command names and s6-svstat's
output, it worked like a charm, and it appears to work just like
Felker->daemontools-encore.

Thanks,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: s6 ordering and run-once?

[Colin Booth]

On Tue, Jun 16, 2015 at 5:58 PM, Steve Litt  wrote:
> On Wed, 17 Jun 2015 01:07:01 +0200
> Laurent Bercot  wrote:
>
>> On 16/06/2015 23:33, Steve Litt wrote:
>> > Hi all,
>
>> > Does anyone know how to do a run-once service without putting an
>> > infinite sleep loop at the end?
>>
>>   Oneshots are not meant to be supervised. Don't create a service
>> directory for a oneshot. You'd run your oneshots exactly as you'd do
>> with runit or daemontools: in a separate script.
>>   Intermixing oneshots and longruns is complex, it requires real
>> dependency management, which is more than a process supervision suite
>> does - supervision only handles longruns.
>
> What do you do if a oneshot requires that a longrun is already running?
>
Is your oneshot script idempotent and safe to tie together into the
longrun? If so and if you're using s6 you can fork out to the other
script, block on a fifo in the services event/ directory with
s6-svwait, and then continue on its way when s6-notifywhenup sends a
U. It does require that your script be safe to run multiple times
since it'll fire every time the service transitions from down to up
but it's reasonably safe, is polling free, and works out of the box.
It also tightly couples the longrun and the oneshot together at the
run script level, so is only suited for custom jobs.

Cheers!
-- 
"If the doors of perception were cleansed every thing would appear to
man as it is, infinite. For man has closed himself up, till he sees
all things thru' narrow chinks of his cavern."
  --  William Blake

Re: s6 ordering and run-once?

[Wayne Marshall]

On Tue, 16 Jun 2015 20:58:35 -0400
Steve Litt  wrote:

> > > Does anyone know how to do a run-once service without putting an
> > > infinite sleep loop at the end?

In perp you would simply touch(1) a file named "flag.once" in the
service definition directory.  See the section STARTUP MODIFICATION in
the perpd(8) man page for the full story:

http://b0llix.net/perp/site.cgi?page=perpd.8

> 
> What do you do if a oneshot requires that a longrun is already
> running?
>

In perp you would simply use the perpok(8) utility to test for the
required service, and use the runpause(8) utility to hack oneshot into
a persistent process (_not_ using the "flag.once" flagfile in this
case).

Here is a complete perp runscript example for oneshot, as would be found
in the file /etc/perp/oneshot/rc.main:

#!/bin/sh
exec 2>&1

if test $1 = 'start' ; then
  if ! perpok -u3 longrun ; then
echo "dependency failure: longrun not running..."
exit 1
  else
echo "starting oneshot..."
/usr/bin/oneshot
exec runpause -L 'oneshot' 0 /bin/true
  fi
fi

if test $1 = 'reset' ; then
  echo "resetting oneshot..."
fi

### eof 


The relevant manual pages:

perpok(8):
http://b0llix.net/perp/site.cgi?page=perpok.8

runpause(8):
http://b0llix.net/perp/site.cgi?page=runpause.8


Wayne
http://b0llix.net/perp/
 

Re: First thoughts on s6

[Colin Booth]

On Tue, Jun 16, 2015 at 5:44 PM, Steve Litt  wrote:
> Hi all,
>
> I'd suggest a ./configure option called --urandom-instead, because this
> isn't the first time I've seen a /dev/random act like this, and
> probably most people don't want to pull the strongarm I did (I wouldn't
> either if it were a real machine).
>
Use --enable-force-devr and/or fix the entropy seed into the vm.
There's a bunch of other options for configure, and they are different
between skalibs and the rest of the packages. I'm sure there's
differences within the non-skalibs packages as well but I've never
bothered with them.
>
> * cat /service/tty5/log/main/current | tai64nlocal
>
> * cat /service/tty5/log/main/current | tai64n
>
> The former command worked as expected and printed out /mm/dd plus
> time timestamps. The latter command appeared to print out two different
> tai64n timestamps, both fairly close to each other but different.
>
Read the html documentation for those two commands, they are opposites.
>
> I think the following additions would make the s6 website a lot easier
> to use, and a lot easier for somebody to begin using s6:
>
> * FAQ
> * A page with simple files, including run and log/run for a tty.
>
> Here are the run files I used for tty5:
>
> For the service:
>
> #!/bin/sh
> exec 2>&1
> echo Starting tty5 >&2
> exec /sbin/getty tty5
>
Doesn't work on BSD. getty lives in /usr/libexec/ and at least by
default includes some extra type data. Also may want to set a baud
rate. You also don't need to redirect the echo since stderr and stdout
are going to the same place already.

This not working is part of the curse of the maintainer and why I
think a big collection of definitions that isn't tied to individual
packages is a mistake. In this case for the examples, Laurent went
with assuming getty is in the path (though it'll still fail on BSD
since /usr/libexec isn't in anyones path by default).
>
> For the tty5 log:
> #!/bin/sh
> exec s6-log t ./main
>
> They seemed to work well. Anyway, so far so good.
>

Cheers!

Re: s6 ordering and run-once?

[Laurent Bercot]

On 17/06/2015 02:58, Steve Litt wrote:

What do you do if a oneshot requires that a longrun is already running?


 That is exactly the problem of mixed dependencies.
 The right answer is anopa (or s6-rc when it's released).
 Apart from that, I don't have any more of an answer than runit or
daemontools do.

--
 Laurent

Re: First thoughts on s6

[Laurent Bercot]

On 17/06/2015 02:44, Steve Litt wrote:

Plop Linux has one of those /dev/random generators that just stops
after a few characters. This caused the ./configure on skalibs to lock
up while checking for /dev/random (thank you very much for the clear
messages). In fact, /dev/random on this VM locks up dd too. I
strongarmed the problem by removing /dev/random and making a hard link
called /dev/random to /dev/urandom.


 http://skarnet.org/software/skalibs/flags.html#forcedevr

 The --enable-force-devr option is also mentioned in ./configure --help.
I expect people to at least read ./configure --help ! :P



My tty5 takes 1 second from Ctrl+D til the time the next login prompt
is printed. Is this a feature, or a bug?


 Feature. There's always a 1-second delay before a service respawns.
Other people have also expressed dislike, but I really want to make
sure services never respawn too fast. I have an idea to get the best
of both worlds, but haven't gotten around to implementing it yet.

 

The following two commands worked *very* differently:

* cat /service/tty5/log/main/current | tai64nlocal

* cat /service/tty5/log/main/current | tai64n


 Um, yes? s6-tai64n prepends every line with a timestamp,
whereas s6-tai64nlocal converts TAI64N timestamps into local time
ISO 8601 timestamps.
 Your logs have already been timestamped by s6-log. If you
filter them through s6-tai64nlocal, the existing timestamps will be
converted, and that is what you are witnessing.
 If you filter your logs through s6-tai64n, though, what will
happen is that a new, additional timestamp will be prepended to
every line. This is also what you are witnessing, and probably not
what you want, but then why are you doing this? XD



I think the following additions would make the s6 website a lot easier
to use, and a lot easier for somebody to begin using s6:

* FAQ


 Can you give me an example of questions you had that were not
covered by the overview ?



* A page with simple files, including run and log/run for a tty.


 Yeah. Remember that s6 aims to be system-agnostic, though, and
it's *very* easy to fall into linuxisms as soon as you start
doing that kind of thing. I'm not sure your getty run script works
under BSD, for instance. :)

--
 Laurent

Re: s6 ordering and run-once?

[Steve Litt]

On Wed, 17 Jun 2015 01:07:01 +0200
Laurent Bercot  wrote:

> On 16/06/2015 23:33, Steve Litt wrote:
> > Hi all,

> > Does anyone know how to do a run-once service without putting an
> > infinite sleep loop at the end?
> 
>   Oneshots are not meant to be supervised. Don't create a service
> directory for a oneshot. You'd run your oneshots exactly as you'd do
> with runit or daemontools: in a separate script.
>   Intermixing oneshots and longruns is complex, it requires real
> dependency management, which is more than a process supervision suite
> does - supervision only handles longruns.

What do you do if a oneshot requires that a longrun is already running?


SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

First thoughts on s6

[Steve Litt]

Hi all,

I have a proof of concept s6 running, *supervision only*, on a Plop
Linux VM. It is started by sysvinit (inittab), and supervises tty5 just
fine, including logging (which is how I found out my run command
originally had bad syntax).

Here are some first impressions...

Plop Linux has one of those /dev/random generators that just stops
after a few characters. This caused the ./configure on skalibs to lock
up while checking for /dev/random (thank you very much for the clear
messages). In fact, /dev/random on this VM locks up dd too. I
strongarmed the problem by removing /dev/random and making a hard link
called /dev/random to /dev/urandom. What the heck, it's an experimental
VM I can throw away, so any side effects this caused probably won't
cause me a whole lot of trouble.

I'd suggest a ./configure option called --urandom-instead, because this
isn't the first time I've seen a /dev/random act like this, and
probably most people don't want to pull the strongarm I did (I wouldn't
either if it were a real machine).

My tty5 takes 1 second from Ctrl+D til the time the next login prompt
is printed. Is this a feature, or a bug?

The following two commands worked *very* differently:

* cat /service/tty5/log/main/current | tai64nlocal

* cat /service/tty5/log/main/current | tai64n

The former command worked as expected and printed out /mm/dd plus
time timestamps. The latter command appeared to print out two different
tai64n timestamps, both fairly close to each other but different.

I think the following additions would make the s6 website a lot easier
to use, and a lot easier for somebody to begin using s6:

* FAQ
* A page with simple files, including run and log/run for a tty.

Here are the run files I used for tty5:

For the service:

#!/bin/sh
exec 2>&1
echo Starting tty5 >&2
exec /sbin/getty tty5


For the tty5 log:
#!/bin/sh
exec s6-log t ./main

They seemed to work well. Anyway, so far so good.

Thanks,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: Towards a clearinghouse

[Buck Evan]

Avery, Laurent:

This pycon lightning talk gave some really good, quick tips that I think
you may like: https://youtu.be/JVVMMULwR4s?t=29m19s

On Tue, Jun 16, 2015 at 4:17 PM, Laurent Bercot  wrote:

> On 17/06/2015 00:08, Buck Evan wrote:
>
>> Avery: copy the stylesheet from cr.yp.to. It seems to be the cool thing
>> recently.
>>
>
>  Eh, it was never intended to be cool. It was more like
> "default HTML without stylesheets is readable and it's all I need,
> I don't want to learn CSS or invest time to make it look pretty"
> and I think that was pretty much DJB's approach too.
>
>  If someone suggests some nice, clean CSS that makes the pages
> look better without modifying the HTML and without taking away from
> clarity and emphasis on the content, I'll take it.
>
> --
>  Laurent
>

RE: comparison

[James Powell]

You really do have to cater to get a foothold in the door any more.

Sent from my Windows Phone

From: Laurent Bercot
Sent: ‎6/‎16/‎2015 4:24 PM
To: supervision@list.skarnet.org
Subject: Re: comparison

On 16/06/2015 22:32, post-sysv wrote:
> Soon systemd arrives with its promise of being a unified userspace
> toolkit that systems developers can supposedly just plug in and
> integrate without hassle to get X, Y and Z advantages. No more
> writing initscripts, no more setting policy because systemd will do
> as much as it can for you. A lazy package maintainer's dream,
> ostensibly.
>
> Everyone hops on the bandwagon and there you are. Now we get to hear
> about how systemd solves so many long-standing problems with the
> distributions, but I can't shake the feeling that many of them were
> self-inflicted through indifference and/or incompetence.

  I think you nailed it exactly. Inertia is the most potent force in
the universe.
  Basically, to get supervision suites - or anything else - adopted by
distributions, the only necessary thing is to do all their work for
them.

--
  Laurent

Re: comparison

[Laurent Bercot]

On 16/06/2015 22:32, post-sysv wrote:

Soon systemd arrives with its promise of being a unified userspace
toolkit that systems developers can supposedly just plug in and
integrate without hassle to get X, Y and Z advantages. No more
writing initscripts, no more setting policy because systemd will do
as much as it can for you. A lazy package maintainer's dream,
ostensibly.

Everyone hops on the bandwagon and there you are. Now we get to hear
about how systemd solves so many long-standing problems with the
distributions, but I can't shake the feeling that many of them were
self-inflicted through indifference and/or incompetence.


 I think you nailed it exactly. Inertia is the most potent force in
the universe.
 Basically, to get supervision suites - or anything else - adopted by
distributions, the only necessary thing is to do all their work for
them.

--
 Laurent

Re: Towards a clearinghouse

[Laurent Bercot]

On 17/06/2015 00:08, Buck Evan wrote:

Avery: copy the stylesheet from cr.yp.to. It seems to be the cool thing
recently.


 Eh, it was never intended to be cool. It was more like
"default HTML without stylesheets is readable and it's all I need,
I don't want to learn CSS or invest time to make it look pretty"
and I think that was pretty much DJB's approach too.

 If someone suggests some nice, clean CSS that makes the pages
look better without modifying the HTML and without taking away from
clarity and emphasis on the content, I'll take it.

--
 Laurent

Re: s6 ordering and run-once?

[Laurent Bercot]

On 16/06/2015 23:33, Steve Litt wrote:

Hi all,

I'm studying the s6 website. The only reference I found to controlling
startup order was a sentence about creating symlinks one by one. I
found a sentence about run-once types of things (I think the example
was bringing up the network), but then it didn't tell me *how* to run a
run-once.

Does anyone know if there are ways to order beyond one symlink at a
time or s6-svwait?


 Nope, not yet. For now, you'd do just as you do with daemontools or
runit: just start everything, and they'll eventually get there. :)
It is possible to start services one by one, but by hand in a script,
and I haven't met anyone who bothers to do that. I certainly don't.
But there are tools for dependency management, see below.



Does anyone know how to do a run-once service without putting an
infinite sleep loop at the end?


 Oneshots are not meant to be supervised. Don't create a service
directory for a oneshot. You'd run your oneshots exactly as you'd do
with runit or daemontools: in a separate script.
 Intermixing oneshots and longruns is complex, it requires real
dependency management, which is more than a process supervision suite
does - supervision only handles longruns.

 There is a dependency manager for s6: http://jjacky.com/anopa/
I also have a dependency manager slowly brewing. Those things are
another world, doing them right is extremely tricky.



Also, are there any flag files, other than "down", of which I should be
aware?


 The complete list is at http://skarnet.org/software/s6/servicedir.html

 Good luck!

--
 Laurent

Re: Towards a clearinghouse

[Buck Evan]

Avery: copy the stylesheet from cr.yp.to. It seems to be the cool thing
recently.

Towards a clearinghouse

[Avery Payne]

On Jun 16, 2015 2:39 PM, "Steve Litt"  wrote:
>
> On Tue, 16 Jun 2015 14:12:48 -0700
> Avery Payne  wrote:
> >
> > In my not very humble opinion, we really need a single point of
> > reference, driven by the community, shared and sharable, and publicly
> > visible.  I could envision something like a single website which
> > would collect settings and store them, and if you needed settings, it
> > would build all of the envdir files and download them in one giant
> > dollop, probably a tarball.  Unpack the tarball and all of the envdir
> > settings are there, waiting to be used.  You could even be "fancy"
> > and track option flags through various daemon revisions, so that if
> > you have an older service running, you tell it "I have older version
> > x.y.z" and you get the "correct" flags and not the "current" ones.
>
> I must be too cynical. I see that after the above described collection,
> website, and envdirs (does that mean service directories) is somewhat
> operational, a well funded "Open Source" vendor will flood it with
> wheelbarrows of cash and a parade of developers, and that nice, simple
> collection and web app becomes unfathomable (and has a 20K word terms
> and conditions).

I think this concept could be made to fly on a shoestring budget.  While
part of me enjoys the illusion of "build it and they will come" the more
rational part of me is well aware that it will probably see less than 1,000
hits a month, 98% from crawlers updating their cache.

With regard to the perinnial issue of "buckets of money sway and manipulate
projects" the answer is simple, Don't Do That (tm) and it will be fine.
This would be a community project with public visibility, and any vendor
attempts to strong-arm it would be spotted fairly quickly.

We need to set the cadence and tempo here and I think having a central
resource would help.   I'm about this close > < to buying a cheap domain
name and then putting it up myself.  Except I know zero about website
design and anything I put up would be ugly.

I'm going to give it some more thought.  I can't promise anything beyond
that.

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 14:12:48 -0700
Avery Payne  wrote:

> 
> 
> On 6/16/2015 1:32 PM, post-sysv wrote:
> > Soon systemd arrives with its promise of being a unified userspace 
> > toolkit that systems developers can supposedly just plug in and 
> > integrate without hassle to get X, Y and Z advantages. No more
> > writing initscripts, no more setting policy because systemd will do
> > as much as it can for you. A lazy package maintainer's dream,
> > ostensibly.
> 
> That last sentence is telling.  It's also why a master catalog of 
> settings is so badly needed.
> 
> In my not very humble opinion, we really need a single point of 
> reference, driven by the community, shared and sharable, and publicly 
> visible.  I could envision something like a single website which
> would collect settings and store them, and if you needed settings, it
> would build all of the envdir files and download them in one giant
> dollop, probably a tarball.  Unpack the tarball and all of the envdir
> settings are there, waiting to be used.  You could even be "fancy"
> and track option flags through various daemon revisions, so that if
> you have an older service running, you tell it "I have older version
> x.y.z" and you get the "correct" flags and not the "current" ones.

I must be too cynical. I see that after the above described collection,
website, and envdirs (does that mean service directories) is somewhat
operational, a well funded "Open Source" vendor will flood it with
wheelbarrows of cash and a parade of developers, and that nice, simple
collection and web app becomes unfathomable (and has a 20K word terms
and conditions). But that's why they have paid support, right?

I'll now go back to talking about technology.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

s6 ordering and run-once?

[Steve Litt]

Hi all,

I'm studying the s6 website. The only reference I found to controlling
startup order was a sentence about creating symlinks one by one. I
found a sentence about run-once types of things (I think the example
was bringing up the network), but then it didn't tell me *how* to run a
run-once.

Does anyone know if there are ways to order beyond one symlink at a
time or s6-svwait?

Does anyone know how to do a run-once service without putting an
infinite sleep loop at the end?

Also, are there any flag files, other than "down", of which I should be
aware?

Thanks,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Avery Payne]

On 6/16/2015 1:32 PM, post-sysv wrote:
Soon systemd arrives with its promise of being a unified userspace 
toolkit that systems developers can supposedly just plug in and 
integrate without hassle to get X, Y and Z advantages. No more writing 
initscripts, no more setting policy because systemd will do as much as 
it can for you. A lazy package maintainer's dream, ostensibly.


That last sentence is telling.  It's also why a master catalog of 
settings is so badly needed.


In my not very humble opinion, we really need a single point of 
reference, driven by the community, shared and sharable, and publicly 
visible.  I could envision something like a single website which would 
collect settings and store them, and if you needed settings, it would 
build all of the envdir files and download them in one giant dollop, 
probably a tarball.  Unpack the tarball and all of the envdir settings 
are there, waiting to be used.  You could even be "fancy" and track 
option flags through various daemon revisions, so that if you have an 
older service running, you tell it "I have older version x.y.z" and you 
get the "correct" flags and not the "current" ones.


This service would not only act as a repository but it would also be a 
clearinghouse, a place where distributions could come and take what is 
needed.




Everyone hops on the bandwagon and there you are. Now we get to hear 
about how systemd solves so many long-standing problems with the 
distributions, but I can't shake the feeling that many of them were 
self-inflicted through indifference and/or incompetence.


After learning so much from my own odyssey, I'd like to be a little more 
forgiving and chalk it up to inexperience.  I have this scene running 
through my head as you describe the vendors looking for software nearly 
20 years ago: "We need an init...and something to start services...look, 
there's this Sys-V thingy lying around, grab it and make it work!"

RE: comparison

[James Powell]

Too true. Systemd offered the pot of gold as long as you handed it the keys to 
the kingdom.

Runit still requires work, but less work if a package for scripts exists. The 
most arbitrary need is the Stage scripts, which generally can utilize a general 
script setup, or translated init shell scripts.

I've worked with Runit enough to see that many distributions wanted 
ready-to-use, not more hackfests to redo run script are run script.

The problem is, Runit solved every problem with system management and service 
supervision before systemd, but because few were willing to create scripts, it 
fell off the radar. Runit is a very complete init+supervision suite in all 
respects and regards, but it needed scripts.

And believe me, after you debug a script about 10 times and sshd still refuses 
to stay in the up state and available, it makes you want to rip hair out.

Sent from my Windows Phone

From: post-sysv
Sent: ‎6/‎16/‎2015 1:35 PM
To: supervision@list.skarnet.org
Subject: Re: comparison

Implying that the distributions would have transitioned from System
V-style to daemontools-style mechanisms?

Strongly doubt it.

For all the noise and controversy that's been happening over PID1, I
always got the impression that most distros back in the day simply
didn't care. They happily piled on hack after hack to their messy
sysvinit setups in the form of dependency header parsers (as
standardized by LSB and done through insserv(8), asynchronous executors
(startpar), generic tools like start-stop-daemon(8) to get around the
fact that they never wrote a common initscript library, symlinking
/bin/sh to dash to improve startup speeds, etc.

There were alternatives like initng (which Ubuntu for some reason
skipped over and settled on Upstart instead with its awkward event
model), depinit, minit, eINIT, daemontools derivatives and others for
quite a while, but mainstream distros by and large never expressed much
interest in them. Instead, they drowned themselves in quick fixes until
the boot process unsurprisingly became difficult to maintain.

Soon systemd arrives with its promise of being a unified userspace
toolkit that systems developers can supposedly just plug in and
integrate without hassle to get X, Y and Z advantages. No more writing
initscripts, no more setting policy because systemd will do as much as
it can for you. A lazy package maintainer's dream, ostensibly.

Everyone hops on the bandwagon and there you are. Now we get to hear
about how systemd solves so many long-standing problems with the
distributions, but I can't shake the feeling that many of them were
self-inflicted through indifference and/or incompetence.

On 06/16/2015 04:09 PM, James Powell wrote:
> And supervision-scripts has been that generic profile that can be used in 99% 
> of situations. Sadly, Avery, I wish we could have had your work about 8 years 
> ago. The UNIX world might have been a vastly different place.

Re: comparison

[Steve Litt]

That's an understatement!

And, as the old saying goes, the best time to plant a tree is 20 years
ago. The second best time is today.

Thanks Avery!

SteveT



On Tue, 16 Jun 2015 13:09:32 -0700
James Powell  wrote:

> And supervision-scripts has been that generic profile that can be
> used in 99% of situations. Sadly, Avery, I wish we could have had
> your work about 8 years ago. The UNIX world might have been a vastly
> different place.
> 
> Sent from my Windows Phone
> 
> From: Avery Payne
> Sent: ‎6/‎16/‎2015 11:26 AM
> To: supervision@list.skarnet.org
> Subject: Re: comparison
> 
> On 6/16/2015 5:22 AM, James Powell wrote:
> > Very true, but something always seems to say something along the
> > lines of "if we had done #2 years ago, we might have avoided a huge
> > mess that now exists".
> Agreed.
> > The same applies to init systems. If there are ready to use feet
> > wetting, taste testing scripts ready to go, the job of importing
> > things just gets easier on the distribution.
> Also agreed.  Actually, there's some discussion on the mailing list
> from a few months back about this.
> > 
> > From: Steve Litt
> > Sent: ‎6/‎16/‎2015 4:45 AM
> > To:
> > supervision@list.skarnet.org
> > Subject: Re: comparison
> >
> > On Tue, 16 Jun 2015 04:05:29 -0700
> > James Powell  wrote:
> >
> >> I agree Laurent. Though, even though complete init+supervision
> >> systems like Runit exist, it's been nearly impossible to get a
> >> foothold with any alternatives to sysvinit and systemd
> >> effectively. I think one of the major setbacks has been the lack
> >> of ready-to-use script sets, like those included with OpenRC,
> >> various rehashes of sysvinit and bsdinit scripts, and systemd
> >> units just aren't there ready to go.
> The true problem is that each daemon needs its own special environment
> variables, command flags, and other gobbledygook that is specific to
> getting it up and running, and a master catalog of all settings
> doesn't exist.  Compounding that is the normal and inevitable need
> for each supervision author to do their own thing, in their own way,
> so tools get renamed, flags get mapped, return codes aren't
> consistent.  That's just the framework, we haven't talked about run
> scripts yet.  Who wants to write hundreds of scripts?  Each
> hand-cobbled script is an error-prone task, and that implies the
> potential for hundreds of errors, bugs, strange behaviors, etc.
> 
> This is the _entire_ reason for supervision-scripts.  It was meant to
> be a generic "one size fits most" solution to providing prefabricated
> run scripts, easing or removing the burden for package maintainers,
> system builders, etc.  All of the renaming and flags and options and
> environment settings and other things are abstracted away as variables
> that are correctly set for whatever environment you have.  With all of
> that out of the way, it becomes much easier to actually write scripts
> to launch things under multiple environments.  A single master script
> handles it all, reduces debugging, and can be easily swapped out to
> support chainload launchers from s6 and nosh.
> 
> The opposite end of this is Laurent's proposal to compile the scripts
> so they are "built into existence".  If I'm understanding / imagining
> this correctly, this would take all of the settings and with a
> makefile "bake" each script into existence with all of the steps and
> settings needed.  It would in effect provide the same thing I am
> doing but it would make it static to the environment. There's nothing
> wrong with the approach, and the end result is the same.
> 
> The only difference between Laurent's approach and mine, is that
> Laurent's would need to "re-bake" your scripts if your framework
> changes; in my project, you simply run a single script and all of the
> needed settings change on the fly.  I'm not sure of the pros/cons to
> either approach, as I would hazard a guess that any system switching
> between frameworks may also require a reboot if a new init is desired.
> 
> Here's the rub: in both cases, the settings for each
> service/daemon/whatever are key to getting things running.  Again, we
> come back to the idea of a "master catalog of settings".  If it
> existed, then half of the problem would be resolved.  There are lots
> of examples out there, but, they're not all in one place.
> 
> So I try to toil over supervision-scripts when I get time, and make
> that catalog.  Even if people don't like what I'm doing with the
> master run script itself, that doesn't matter.  *What matters is that
> I've managed to capture the settings for the various daemons, along
> with some annotations*.  Because I took the time to support envdir,
> and the settings for each daemon are stored in this format, those
> settings can be extr

Re: comparison

[post-sysv]

Implying that the distributions would have transitioned from System 
V-style to daemontools-style mechanisms?


Strongly doubt it.

For all the noise and controversy that's been happening over PID1, I 
always got the impression that most distros back in the day simply 
didn't care. They happily piled on hack after hack to their messy 
sysvinit setups in the form of dependency header parsers (as 
standardized by LSB and done through insserv(8), asynchronous executors 
(startpar), generic tools like start-stop-daemon(8) to get around the 
fact that they never wrote a common initscript library, symlinking 
/bin/sh to dash to improve startup speeds, etc.


There were alternatives like initng (which Ubuntu for some reason 
skipped over and settled on Upstart instead with its awkward event 
model), depinit, minit, eINIT, daemontools derivatives and others for 
quite a while, but mainstream distros by and large never expressed much 
interest in them. Instead, they drowned themselves in quick fixes until 
the boot process unsurprisingly became difficult to maintain.


Soon systemd arrives with its promise of being a unified userspace 
toolkit that systems developers can supposedly just plug in and 
integrate without hassle to get X, Y and Z advantages. No more writing 
initscripts, no more setting policy because systemd will do as much as 
it can for you. A lazy package maintainer's dream, ostensibly.


Everyone hops on the bandwagon and there you are. Now we get to hear 
about how systemd solves so many long-standing problems with the 
distributions, but I can't shake the feeling that many of them were 
self-inflicted through indifference and/or incompetence.


On 06/16/2015 04:09 PM, James Powell wrote:

And supervision-scripts has been that generic profile that can be used in 99% 
of situations. Sadly, Avery, I wish we could have had your work about 8 years 
ago. The UNIX world might have been a vastly different place.

RE: comparison

[James Powell]

And supervision-scripts has been that generic profile that can be used in 99% 
of situations. Sadly, Avery, I wish we could have had your work about 8 years 
ago. The UNIX world might have been a vastly different place.

Sent from my Windows Phone

From: Avery Payne
Sent: ‎6/‎16/‎2015 11:26 AM
To: supervision@list.skarnet.org
Subject: Re: comparison

On 6/16/2015 5:22 AM, James Powell wrote:
> Very true, but something always seems to say something along the lines of "if 
> we had done #2 years ago, we might have avoided a huge mess that now exists".
Agreed.
> The same applies to init systems. If there are ready to use feet wetting, 
> taste testing scripts ready to go, the job of importing things just gets 
> easier on the distribution.
Also agreed.  Actually, there's some discussion on the mailing list from
a few months back about this.
> 
> From: Steve Litt
> Sent: ‎6/‎16/‎2015 4:45 AM
> To: supervision@list.skarnet.org
> Subject: Re: comparison
>
> On Tue, 16 Jun 2015 04:05:29 -0700
> James Powell  wrote:
>
>> I agree Laurent. Though, even though complete init+supervision
>> systems like Runit exist, it's been nearly impossible to get a
>> foothold with any alternatives to sysvinit and systemd effectively. I
>> think one of the major setbacks has been the lack of ready-to-use
>> script sets, like those included with OpenRC, various rehashes of
>> sysvinit and bsdinit scripts, and systemd units just aren't there
>> ready to go.
The true problem is that each daemon needs its own special environment
variables, command flags, and other gobbledygook that is specific to
getting it up and running, and a master catalog of all settings doesn't
exist.  Compounding that is the normal and inevitable need for each
supervision author to do their own thing, in their own way, so tools get
renamed, flags get mapped, return codes aren't consistent.  That's just
the framework, we haven't talked about run scripts yet.  Who wants to
write hundreds of scripts?  Each hand-cobbled script is an error-prone
task, and that implies the potential for hundreds of errors, bugs,
strange behaviors, etc.

This is the _entire_ reason for supervision-scripts.  It was meant to be
a generic "one size fits most" solution to providing prefabricated run
scripts, easing or removing the burden for package maintainers, system
builders, etc.  All of the renaming and flags and options and
environment settings and other things are abstracted away as variables
that are correctly set for whatever environment you have.  With all of
that out of the way, it becomes much easier to actually write scripts to
launch things under multiple environments.  A single master script
handles it all, reduces debugging, and can be easily swapped out to
support chainload launchers from s6 and nosh.

The opposite end of this is Laurent's proposal to compile the scripts so
they are "built into existence".  If I'm understanding / imagining this
correctly, this would take all of the settings and with a makefile
"bake" each script into existence with all of the steps and settings
needed.  It would in effect provide the same thing I am doing but it
would make it static to the environment. There's nothing wrong with the
approach, and the end result is the same.

The only difference between Laurent's approach and mine, is that
Laurent's would need to "re-bake" your scripts if your framework
changes; in my project, you simply run a single script and all of the
needed settings change on the fly.  I'm not sure of the pros/cons to
either approach, as I would hazard a guess that any system switching
between frameworks may also require a reboot if a new init is desired.

Here's the rub: in both cases, the settings for each
service/daemon/whatever are key to getting things running.  Again, we
come back to the idea of a "master catalog of settings".  If it existed,
then half of the problem would be resolved.  There are lots of examples
out there, but, they're not all in one place.

So I try to toil over supervision-scripts when I get time, and make that
catalog.  Even if people don't like what I'm doing with the master run
script itself, that doesn't matter.  *What matters is that I've managed
to capture the settings for the various daemons, along with some
annotations*.  Because I took the time to support envdir, and the
settings for each daemon are stored in this format, those settings can
be extracted and used elsewhere.  I'm slowly creating that "master
catalog" in a plaintext format that can be read and processed easily.
This is the real, hidden value of supervision-scripts.

By the way, I'm going to bite the bullet and switch off of MPL 2.0 soon,
probably by month-end.

Re: Readiness notification for systemd

[Laurent Bercot]

On 16/06/2015 20:40, Avery Payne wrote:

Logging generally (but not always) implies calling printf() with a
newline at some point.

What if we could come up with a simple standard that extends your
newline concept into the logging output?  A newline itself may be
emitted as part of a startup string sent to a log, so I can't be
assured that a daemon is really ready.  But what if we could ensure
that a universally agreed pattern were present in the log? Something
like "Ready.\n" when the daemon is up.  We literally could watch the
stdout/stderr for this pattern and would solve the entire "readiness
notification" problem in one go.


 Well, I really don't like that approach because it mixes a control flow
and a data flow; and when you do that, bad things tend to happen.
 Data flows are big beasts that 1. require efficiency, and 2. will really
mess you up if you take just one wrong step. Data analysis is really a
job for application programs; for system software, I'd rather not get in
the way, and just let logs flow.

 (If you add a log filter just for readiness notification, you can't
remove that filter afterwards. You have to keep filtering the log 100%
of the time. It's possible - on the skaware mailing-list, Patrick needed
such a setup - but I wouldn't want it to be a standard, it's too kludgy
and inefficient.)

--
 Laurent

Re: Readiness notification for systemd

[Avery Payne]

On 6/13/2015 11:48 AM, Laurent Bercot wrote:

It's
a wrapper for daemons using the simple "write a newline" readiness
notification mechanism advertised by s6, which converts that
notification to the sd_notify format.


This had me tossing around some ideas yesterday while I was headed home.

Most (but not all) daemons provide logging.

Logging generally (but not always) implies calling printf() with a 
newline at some point.


What if we could come up with a simple standard that extends your 
newline concept into the logging output?  A newline itself may be 
emitted as part of a startup string sent to a log, so I can't be assured 
that a daemon is really ready.  But what if we could ensure that a 
universally agreed pattern were present in the log? Something like 
"Ready.\n" when the daemon is up.  We literally could watch the 
stdout/stderr for this pattern and would solve the entire "readiness 
notification" problem in one go.


It would be an enormous problem politically because most daemon authors 
aren't going to add the 3-4 lines of code needed to support this.


But I could dream...

Re: comparison

[Avery Payne]

On 6/16/2015 5:22 AM, James Powell wrote:

Very true, but something always seems to say something along the lines of "if we had 
done #2 years ago, we might have avoided a huge mess that now exists".

Agreed.

The same applies to init systems. If there are ready to use feet wetting, taste 
testing scripts ready to go, the job of importing things just gets easier on 
the distribution.
Also agreed.  Actually, there's some discussion on the mailing list from 
a few months back about this.


From: Steve Litt
Sent: ‎6/‎16/‎2015 4:45 AM
To: supervision@list.skarnet.org
Subject: Re: comparison

On Tue, 16 Jun 2015 04:05:29 -0700
James Powell  wrote:


I agree Laurent. Though, even though complete init+supervision
systems like Runit exist, it's been nearly impossible to get a
foothold with any alternatives to sysvinit and systemd effectively. I
think one of the major setbacks has been the lack of ready-to-use
script sets, like those included with OpenRC, various rehashes of
sysvinit and bsdinit scripts, and systemd units just aren't there
ready to go.
The true problem is that each daemon needs its own special environment 
variables, command flags, and other gobbledygook that is specific to 
getting it up and running, and a master catalog of all settings doesn't 
exist.  Compounding that is the normal and inevitable need for each 
supervision author to do their own thing, in their own way, so tools get 
renamed, flags get mapped, return codes aren't consistent.  That's just 
the framework, we haven't talked about run scripts yet.  Who wants to 
write hundreds of scripts?  Each hand-cobbled script is an error-prone 
task, and that implies the potential for hundreds of errors, bugs, 
strange behaviors, etc.


This is the _entire_ reason for supervision-scripts.  It was meant to be 
a generic "one size fits most" solution to providing prefabricated run 
scripts, easing or removing the burden for package maintainers, system 
builders, etc.  All of the renaming and flags and options and 
environment settings and other things are abstracted away as variables 
that are correctly set for whatever environment you have.  With all of 
that out of the way, it becomes much easier to actually write scripts to 
launch things under multiple environments.  A single master script 
handles it all, reduces debugging, and can be easily swapped out to 
support chainload launchers from s6 and nosh.


The opposite end of this is Laurent's proposal to compile the scripts so 
they are "built into existence".  If I'm understanding / imagining this 
correctly, this would take all of the settings and with a makefile 
"bake" each script into existence with all of the steps and settings 
needed.  It would in effect provide the same thing I am doing but it 
would make it static to the environment. There's nothing wrong with the 
approach, and the end result is the same.


The only difference between Laurent's approach and mine, is that 
Laurent's would need to "re-bake" your scripts if your framework 
changes; in my project, you simply run a single script and all of the 
needed settings change on the fly.  I'm not sure of the pros/cons to 
either approach, as I would hazard a guess that any system switching 
between frameworks may also require a reboot if a new init is desired.


Here's the rub: in both cases, the settings for each 
service/daemon/whatever are key to getting things running.  Again, we 
come back to the idea of a "master catalog of settings".  If it existed, 
then half of the problem would be resolved.  There are lots of examples 
out there, but, they're not all in one place.


So I try to toil over supervision-scripts when I get time, and make that 
catalog.  Even if people don't like what I'm doing with the master run 
script itself, that doesn't matter.  *What matters is that I've managed 
to capture the settings for the various daemons, along with some 
annotations*.  Because I took the time to support envdir, and the 
settings for each daemon are stored in this format, those settings can 
be extracted and used elsewhere.  I'm slowly creating that "master 
catalog" in a plaintext format that can be read and processed easily.  
This is the real, hidden value of supervision-scripts.


By the way, I'm going to bite the bullet and switch off of MPL 2.0 soon, 
probably by month-end.

Re: patch: sv check should wait when svrun is not ready

[Avery Payne]

I'm not the maintainer of any C code, anywhere.  While I do host a 
mirror or two on bitbucket, I only do humble scripts, sorry.  Gerrit is 
around, he's just a bit elusive.


On 6/16/2015 9:37 AM, Buck Evan wrote:

I'd still like to get this merged.

Avery: are you the current maintainer?
I haven't seen Gerrit Pape on the list.

On Tue, Feb 17, 2015 at 4:49 PM, Buck Evan > wrote:


On Tue, Feb 17, 2015 at 4:20 PM, Avery Payne
mailto:avery.p.pa...@gmail.com>> wrote:
>
> On 2/17/2015 11:02 AM, Buck Evan wrote:
>>
>> I think there's only three cases here:
>>
>>  1. Users that would have gotten immediate failure, and no
amount of
>> spinning would help. These users will see their error delayed
by $SVWAIT
>> seconds, but no other difference.
>>  2. Users that would have gotten immediate failure, but could
have gotten
>> a success within $SVWAIT seconds. All of these users will of
course be glad
>> of the change.
>>  3. Users that would not have gotten immediate failure. None of
these
>> users will see the slightest change in behavior.
>>
>> Do you have a particular scenario in mind when you mention
"breaking lots
>> of existing installations elsewhere due to a default behavior
change"? I
>> don't see that there is any case this change would break.


Thanks for the thoughtful reply Avery. My background is also
"maintaining business software", although putting it in those terms
gives me horrific visions of java servlets and soap protocols.

> I have to look at it from a viewpoint of "what is everything
else in the system expecting when this code is called".  This
means thinking in terms of code-as-API, so that calls elsewhere
don't break.

As a matter of API, sv-check does sometimes take up to $SVWAIT
seconds to fail.
Any caller to sv-check will be expecting this (strictly limited)
delay, in the exceptional case.
My patch just extends this existing, documented behavior to the
special case of "unable to open supervise/ok".
The API is unchanged, just the amount of time to return the result
is changed.

> This happens because the use of "sv check (child)" follows the
convention of "check, and either succeed fast or fail fast", ...

Either you're confused about what sv-check does, or I'm confused about
what you're saying.
sv-check generaly doesn't fail fast (except in the special case I'm
trying to make no longer fail fast -- svrun is not started).
Generally it will spin for $SVWAIT seconds before failing.

> Without that fast-fail, the logged hint never occurs; the
sysadmin now has to figure out which of three possible services in
a dependency chain are causing the hang.

Even if I put the above issue aside aside, you wouldn't get a hang,
you'd get the failure message you're familiar with, just several
seconds (default: 7) later. The sysadmin wouldn't search any more than
previously. He would however find that the system fails less often,
since it has that 7 seconds of tolerance now. This is how sv-check
behaves already when a ./check script exits nonzero.


> While this is
> implemented differently from other installations, there are
known cases
> similar to what I am doing, where people have ./run scripts like
this:
>
> #!/bin/sh
> sv check child-service || exit 1
> exec parent-service

This would still work just fine, just strictly more often.

Re: comparison

[Avery Payne]

On 6/15/2015 9:00 PM, Colin Booth wrote:

I only know s6 and runit well enough to comment on for the most part but
filling in some blanks on your matrix:

Updated, thanks for the help.  As I said, it's a start.  It'll need some 
time to improve.  I mostly needed it for the project, to help me keep 
the mapping of "what tool does what action" straight so I can move 
forward.  I'd like to add some of the missing "specialty" tools that s6 
and nosh provides, and see if there are equivalent mappings elsewhere.  
Also, as new scripting frameworks are discovered, I'll add them as well.

Re: patch: sv check should wait when svrun is not ready

[Buck Evan]

I'd still like to get this merged.

Avery: are you the current maintainer?
I haven't seen Gerrit Pape on the list.

On Tue, Feb 17, 2015 at 4:49 PM, Buck Evan  wrote:

> On Tue, Feb 17, 2015 at 4:20 PM, Avery Payne 
> wrote:
> >
> > On 2/17/2015 11:02 AM, Buck Evan wrote:
> >>
> >> I think there's only three cases here:
> >>
> >>  1. Users that would have gotten immediate failure, and no amount of
> >> spinning would help. These users will see their error delayed by $SVWAIT
> >> seconds, but no other difference.
> >>  2. Users that would have gotten immediate failure, but could have
> gotten
> >> a success within $SVWAIT seconds. All of these users will of course be
> glad
> >> of the change.
> >>  3. Users that would not have gotten immediate failure. None of these
> >> users will see the slightest change in behavior.
> >>
> >> Do you have a particular scenario in mind when you mention "breaking
> lots
> >> of existing installations elsewhere due to a default behavior change"? I
> >> don't see that there is any case this change would break.
> 
>
> Thanks for the thoughtful reply Avery. My background is also
> "maintaining business software", although putting it in those terms
> gives me horrific visions of java servlets and soap protocols.
>
> > I have to look at it from a viewpoint of "what is everything else in the
> system expecting when this code is called".  This means thinking in terms
> of code-as-API, so that calls elsewhere don't break.
>
> As a matter of API, sv-check does sometimes take up to $SVWAIT seconds to
> fail.
> Any caller to sv-check will be expecting this (strictly limited)
> delay, in the exceptional case.
> My patch just extends this existing, documented behavior to the
> special case of "unable to open supervise/ok".
> The API is unchanged, just the amount of time to return the result is
> changed.
>
> > This happens because the use of "sv check (child)" follows the
> convention of "check, and either succeed fast or fail fast", ...
>
> Either you're confused about what sv-check does, or I'm confused about
> what you're saying.
> sv-check generaly doesn't fail fast (except in the special case I'm
> trying to make no longer fail fast -- svrun is not started).
> Generally it will spin for $SVWAIT seconds before failing.
>
> > Without that fast-fail, the logged hint never occurs; the sysadmin now
> has to figure out which of three possible services in a dependency chain
> are causing the hang.
>
> Even if I put the above issue aside aside, you wouldn't get a hang,
> you'd get the failure message you're familiar with, just several
> seconds (default: 7) later. The sysadmin wouldn't search any more than
> previously. He would however find that the system fails less often,
> since it has that 7 seconds of tolerance now. This is how sv-check
> behaves already when a ./check script exits nonzero.
>
>
> > While this is
> > implemented differently from other installations, there are known cases
> > similar to what I am doing, where people have ./run scripts like this:
> >
> > #!/bin/sh
> > sv check child-service || exit 1
> > exec parent-service
>
> This would still work just fine, just strictly more often.
>

RE: comparison

[James Powell]

Very true, but something always seems to say something along the lines of "if 
we had done #2 years ago, we might have avoided a huge mess that now exists".

Runit could have been the successor to sysvinit years ago, but like anything, 
unless there is something tangible to import with less work, rather than more 
work, things don't go into usage.

To put it another way, if CUPS had no drivers how useful would it be?

The same applies to init systems. If there are ready to use feet wetting, taste 
testing scripts ready to go, the job of importing things just gets easier on 
the distribution.

I often think and wonder if this was part of the allure of systemd, and I'm 
possibly correct.

Sent from my Windows Phone

From: Steve Litt
Sent: ‎6/‎16/‎2015 4:45 AM
To: supervision@list.skarnet.org
Subject: Re: comparison

On Tue, 16 Jun 2015 04:05:29 -0700
James Powell  wrote:

> I agree Laurent. Though, even though complete init+supervision
> systems like Runit exist, it's been nearly impossible to get a
> foothold with any alternatives to sysvinit and systemd effectively. I
> think one of the major setbacks has been the lack of ready-to-use
> script sets, like those included with OpenRC, various rehashes of
> sysvinit and bsdinit scripts, and systemd units just aren't there
> ready to go.
>
> Testing and trying to debug in house scripts is a pain and to be
> honest stalled our work with LFS a while back.
>
> Runit is one of the most complete alternatives out there, but if
> scripts are what is holding things back, why has this never been
> accurately addressed?
>
> -Jim

I think part of the difficulty of writing run scripts is there are two
different kinds of runscripts:

1) Simple system specific custom made run script.

2) Works everywhere, regardless of software constellation, one size
   fits all run script.

Distros and packages and "upstreams" make #2, which are very, very,
very difficult. In my Suckless Init plus daemontools-encore adventure,
I had a 200+ line "one size fits all" sysvinit init script degenerate
into a less than 20 line system specific daemontools-encore run script.

Personally, if I were a Linux distribution script maker, at the very
least I would assume I'm working with the Linux kernel and the "stuff"
provided by the distro release for which the script was made. That
would cut down on a lot of the tl;dr cotton candy.

SteveT

Steve Litt
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 04:05:29 -0700
James Powell  wrote:

> I agree Laurent. Though, even though complete init+supervision
> systems like Runit exist, it's been nearly impossible to get a
> foothold with any alternatives to sysvinit and systemd effectively. I
> think one of the major setbacks has been the lack of ready-to-use
> script sets, like those included with OpenRC, various rehashes of
> sysvinit and bsdinit scripts, and systemd units just aren't there
> ready to go.
> 
> Testing and trying to debug in house scripts is a pain and to be
> honest stalled our work with LFS a while back.
> 
> Runit is one of the most complete alternatives out there, but if
> scripts are what is holding things back, why has this never been
> accurately addressed?
> 
> -Jim

I think part of the difficulty of writing run scripts is there are two
different kinds of runscripts:

1) Simple system specific custom made run script.

2) Works everywhere, regardless of software constellation, one size
   fits all run script.

Distros and packages and "upstreams" make #2, which are very, very,
very difficult. In my Suckless Init plus daemontools-encore adventure,
I had a 200+ line "one size fits all" sysvinit init script degenerate
into a less than 20 line system specific daemontools-encore run script.

Personally, if I were a Linux distribution script maker, at the very
least I would assume I'm working with the Linux kernel and the "stuff"
provided by the distro release for which the script was made. That
would cut down on a lot of the tl;dr cotton candy.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: comparison

[Steve Litt]

On Tue, 16 Jun 2015 09:29:15 +0200
Laurent Bercot  wrote:

> On 16/06/2015 04:54, Steve Litt wrote:
> > One thing I can tell you is that daemontools and daemontools-encore
> > were never intended to be init systems, whereas I'm pretty sure that
> > runit, s6 and nosh intended to be part or all of an init system.
> 
>   You keep saying that, but at least in the case of runit and s6, it
> is inaccurate.
> 
>   runit and s6 are process supervision systems, just like daemontools,
> and can be used with any init system. There are documentation pages
> that prove it:
>http://smarden.org/runit/useinit.html
>http://skarnet.org/software/s6/s6-svscan-not-1.html

Yes but:

http://smarden.org/runit/replaceinit.html#sysv
http://skarnet.org/software/s6/s6-svscan-1.html

There's no way I've found to do
init=/any/file/of/daemontools(-encore)

> 
>   That's one thing. When you say "s6 is more complex than
> daemontools", it's only more complex because there are more binaries,
> and more features, 

That's what I meant. s6 is a superset of daemontools, making it a
little harder *for me, personally* to find my way around. Also, IIRC I
couldn't compile s6 the last time, but the last time I knew nothing
about the whole idea of init, so let me suspend saying it's more
complicated until I've logged some time actually using s6.

[snip]

>   Now, the thing is, when you have a process supervision system, it
> makes your stock init redundant. init has two jobs:
>   A. reap zombies

Do you include "listen for and handle relevant signals" in "reap
zombies"?

>   B. maintain at least one other process alive, so that if everything
> on the machine is killed (save init itself), it is still possible to
> recover and avoid a hard and dirty reboot.
> 
>   Note that suckless init, or Rich Felker's suggested init in the
> otherwise excellent http://ewontfix.com/14/ article, do not perform
> B, and so, are *incorrect*. The error is minute, and probably
> inconsequential in any real life situation, but it is still there;
> and if you want the smallest possible pid 1 that will keep your
> machine usable under the most dire circumstances, you should not use
> suckless init, you should use runit.

Yes. Someday I might change Suckless Init so that it *respawns*
(supervises, whatever you want to call it) /bin/rc.init. I would also
need a way that such a modified Suckless Init can pass along the
information that this isn't the first time this boot, in case there are
tasks I don't want to redo in /bin/rc.init.
 
[snip]
> And neither I
> nor Gerrit did it first; the first one was Paul Jarc, who pioneered
> the setup using... daemontools. http://code.dogmap.org/svscan-1/

I'd forgotten about that article. Last time I read it, I didn't
understand it at all. This time, when I read it, it reminded me that my
simple /bin/rc.shutdown stage 3 means I'm short circuiting the logs
during final shutdown.

> 
>   Yes, daemontools was never intended to be an init system, but even
> back then there was interest in running it as such, and Paul's
> experiments are what started it all.
> 
>   The problem with init is that it's not only stage 2, and stage 1
> and 3 are heavily system-dependent; so making svscan work as init is
> possible, but requires good hacking skills.

Also, even after you get into stage 2, those "services" that do system
initialization require hacking skills.

> 
>   Gerrit recognized that, so he basically forfeited the idea -
> instead of using runsvdir as process 1, he created the minimal amount
> of pid 1 code necessary to handle stages 1 and 3, as well as any
> stage 2 (and runsvdir is the obvious choice for the stage 2 manager).
> runit is a basic process supervision system (runsv/runsvdir) with a
> simple init wrapper (runit).
> 
>   I did not want to give up on the idea; I designed s6-svscan so that
> it would be easier to run as pid 1 than daemontools' svscan, and
> managed to get something working reproducibly and in an almost
> automatable way, but as you experienced, the setup still requires
> some hacking skills, and I need more time to work around the
> non-portability issues and come up with a full turnkey init.
> 
>   In the meantime, if you don't want to get your hands dirty,
> you can still use s6-svscan/s6-supervise as a process supervision
> system without trying to make it run as an init system, just as you
> can use runsvdir/runsv as a process supervision system without using
> the runit binary. That is real modularity, that is the main reason
> why I believe runit and s6 are better designed than
> other-init-software-out-there, and it would be *trivial* to port your
> "suckless init on Plop" setup from daemontools(-encore) to runit or
> s6, even if you don't use every feature those packages provide.


I'll be doing an s6 install in the next few weeks.

Thanks,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

RE: comparison

[James Powell]

I agree Laurent. Though, even though complete init+supervision systems like 
Runit exist, it's been nearly impossible to get a foothold with any 
alternatives to sysvinit and systemd effectively. I think one of the major 
setbacks has been the lack of ready-to-use script sets, like those included 
with OpenRC, various rehashes of sysvinit and bsdinit scripts, and systemd 
units just aren't there ready to go.

Testing and trying to debug in house scripts is a pain and to be honest stalled 
our work with LFS a while back.

Runit is one of the most complete alternatives out there, but if scripts are 
what is holding things back, why has this never been accurately addressed?

-Jim

Sent from my Windows Phone

From: Laurent Bercot
Sent: ‎6/‎16/‎2015 12:29 AM
To: supervision@list.skarnet.org
Subject: Re: comparison

On 16/06/2015 04:54, Steve Litt wrote:
> One thing I can tell you is that daemontools and daemontools-encore
> were never intended to be init systems, whereas I'm pretty sure that
> runit, s6 and nosh intended to be part or all of an init system.

  You keep saying that, but at least in the case of runit and s6, it is
inaccurate.

  runit and s6 are process supervision systems, just like daemontools,
and can be used with any init system. There are documentation pages
that prove it:
   http://smarden.org/runit/useinit.html
   http://skarnet.org/software/s6/s6-svscan-not-1.html

  That's one thing. When you say "s6 is more complex than daemontools",
it's only more complex because there are more binaries, and more
features, but you can use s6 exactly as you use daemontools - just add
s6- prefixes to the daemontools command names and it will pretty much
work the same. You can tell suckless init, or whatever pid 1 you fancy,
to run s6-svscan, or runsvdir, exactly the same way you're telling it
to run svscan.
  (The main difference is that daemontools comes with a svscanboot script
that uses the "readproctitle" logging hack, and runit and s6 don't.
As far as s6 is concerned: readproctitle felt too much like a cop-out to
me, and I wanted a proper logging system for logs falling through
s6-svscan's stderr, which is why I'm using a fifo as shown in the
examples/s6-svscanboot script and the
examples/ROOT/img/tmpfs/service/s6-svscan-log service directory.)

  Now, the thing is, when you have a process supervision system, it makes
your stock init redundant. init has two jobs:
  A. reap zombies
  B. maintain at least one other process alive, so that if everything on the
machine is killed (save init itself), it is still possible to recover and
avoid a hard and dirty reboot.

  Note that suckless init, or Rich Felker's suggested init in the otherwise
excellent http://ewontfix.com/14/ article, do not perform B, and so, are
*incorrect*. The error is minute, and probably inconsequential in any real
life situation, but it is still there; and if you want the smallest possible
pid 1 that will keep your machine usable under the most dire circumstances,
you should not use suckless init, you should use runit.

  Process supervision systems automatically perform B, that's their very job;
and svscan/runsvdir/s6-svscan automatically reap every zombie they get
because they have to collect their dead supervisor processes. So they
basically do everything init has to do in stage 2! It's then a natural idea
to try and use them as pid 1. And neither I nor Gerrit did it first; the
first one was Paul Jarc, who pioneered the setup using... daemontools.
   http://code.dogmap.org/svscan-1/

  Yes, daemontools was never intended to be an init system, but even back then
there was interest in running it as such, and Paul's experiments are what
started it all.

  The problem with init is that it's not only stage 2, and stage 1 and 3 are
heavily system-dependent; so making svscan work as init is possible, but
requires good hacking skills.

  Gerrit recognized that, so he basically forfeited the idea - instead of
using runsvdir as process 1, he created the minimal amount of pid 1 code
necessary to handle stages 1 and 3, as well as any stage 2 (and runsvdir
is the obvious choice for the stage 2 manager). runit is a basic process
supervision system (runsv/runsvdir) with a simple init wrapper (runit).

  I did not want to give up on the idea; I designed s6-svscan so that it
would be easier to run as pid 1 than daemontools' svscan, and managed to
get something working reproducibly and in an almost automatable way, but as
you experienced, the setup still requires some hacking skills, and I need
more time to work around the non-portability issues and come up with a full
turnkey init.

  In the meantime, if you don't want to get your hands dirty,
you can still use s6-svscan/s6-supervise as a process supervision system
without trying to make it run as an init system, just as you can use
runsvdir/runsv as a process supervision system without using t

Re: comparison

[Laurent Bercot]

On 16/06/2015 04:54, Steve Litt wrote:

One thing I can tell you is that daemontools and daemontools-encore
were never intended to be init systems, whereas I'm pretty sure that
runit, s6 and nosh intended to be part or all of an init system.


 You keep saying that, but at least in the case of runit and s6, it is
inaccurate.

 runit and s6 are process supervision systems, just like daemontools,
and can be used with any init system. There are documentation pages
that prove it:
  http://smarden.org/runit/useinit.html
  http://skarnet.org/software/s6/s6-svscan-not-1.html

 That's one thing. When you say "s6 is more complex than daemontools",
it's only more complex because there are more binaries, and more
features, but you can use s6 exactly as you use daemontools - just add
s6- prefixes to the daemontools command names and it will pretty much
work the same. You can tell suckless init, or whatever pid 1 you fancy,
to run s6-svscan, or runsvdir, exactly the same way you're telling it
to run svscan.
 (The main difference is that daemontools comes with a svscanboot script
that uses the "readproctitle" logging hack, and runit and s6 don't.
As far as s6 is concerned: readproctitle felt too much like a cop-out to
me, and I wanted a proper logging system for logs falling through
s6-svscan's stderr, which is why I'm using a fifo as shown in the
examples/s6-svscanboot script and the
examples/ROOT/img/tmpfs/service/s6-svscan-log service directory.)

 Now, the thing is, when you have a process supervision system, it makes
your stock init redundant. init has two jobs:
 A. reap zombies
 B. maintain at least one other process alive, so that if everything on the
machine is killed (save init itself), it is still possible to recover and
avoid a hard and dirty reboot.

 Note that suckless init, or Rich Felker's suggested init in the otherwise
excellent http://ewontfix.com/14/ article, do not perform B, and so, are
*incorrect*. The error is minute, and probably inconsequential in any real
life situation, but it is still there; and if you want the smallest possible
pid 1 that will keep your machine usable under the most dire circumstances,
you should not use suckless init, you should use runit.

 Process supervision systems automatically perform B, that's their very job;
and svscan/runsvdir/s6-svscan automatically reap every zombie they get
because they have to collect their dead supervisor processes. So they
basically do everything init has to do in stage 2! It's then a natural idea
to try and use them as pid 1. And neither I nor Gerrit did it first; the
first one was Paul Jarc, who pioneered the setup using... daemontools.
  http://code.dogmap.org/svscan-1/

 Yes, daemontools was never intended to be an init system, but even back then
there was interest in running it as such, and Paul's experiments are what
started it all.

 The problem with init is that it's not only stage 2, and stage 1 and 3 are
heavily system-dependent; so making svscan work as init is possible, but
requires good hacking skills.

 Gerrit recognized that, so he basically forfeited the idea - instead of
using runsvdir as process 1, he created the minimal amount of pid 1 code
necessary to handle stages 1 and 3, as well as any stage 2 (and runsvdir
is the obvious choice for the stage 2 manager). runit is a basic process
supervision system (runsv/runsvdir) with a simple init wrapper (runit).

 I did not want to give up on the idea; I designed s6-svscan so that it
would be easier to run as pid 1 than daemontools' svscan, and managed to
get something working reproducibly and in an almost automatable way, but as
you experienced, the setup still requires some hacking skills, and I need
more time to work around the non-portability issues and come up with a full
turnkey init.

 In the meantime, if you don't want to get your hands dirty,
you can still use s6-svscan/s6-supervise as a process supervision system
without trying to make it run as an init system, just as you can use
runsvdir/runsv as a process supervision system without using the runit
binary. That is real modularity, that is the main reason why I believe
runit and s6 are better designed than other-init-software-out-there, and
it would be *trivial* to port your "suckless init on Plop" setup from
daemontools(-encore) to runit or s6, even if you don't use every feature
those packages provide.

--
 Laurent