Re: Debianization

2018-07-05 Thread Avery Payne
I've been living with rc-shim for several months now, and while it isn't
perfect, it's certainly taken a lot of the pain out of migration while
using SysV rc scripts.  I'm planning on switching off of runit this summer
and moving to s6, so this is both a "proof in the pudding" moment for me
because the shim is supposedly supervisor-agnostic.  There are two basic
issues that I've yet to solve that aren't cosmetic in nature:

1. Debian's updates will overwrite /etc/init.d/{name-of-service-script}
with a new copy each time a service is updated.  It's annoying but easily
fixed, simply "head -n25 /etc/init.d/{script}" and copy the header comments
into a new shim, and then copy the modified shim over the init script.
2. The shim doesn't detect service flapping.  This nasty condition happens
when the service starts, runs for 1 second, the shim wakes from sleep and
detects the service is OK (because the supervisor task reports back that it
is OK), and the shim exits with a successful return code; but the service
crashes a second later, and the supervisor then restarts it over and over
again.  You're left thinking there is a good start, but the service is
flapping like a wounded moth.  Ugly all the way around, and I don't have a
good fix yet.

Both of these are temporary problems when viewed as a stepping stone to a
pure s6 system, so it's not too big of a concern.  The real problem remains
the same for Debian - writing the hundreds of run script definitions to
cover all packages.

On Sat, Apr 21, 2018 at 1:19 PM, Didier Kryn  wrote:

>
> Whether starting from Alpine or Devuan, at least you don't have to
> fight against Systemd which has put metastases pretty much everywhere in
> the system. You start from sysvinit and just replace the rc scripts even
> with keeping sysv's init proper, at the beginning. A big part of the work
> is packaging and transitionning flawlessly from one system to the other
> when installing/removing the packages.
>
> Didier
>
>


Re: Debianization

2018-04-21 Thread Didier Kryn

Le 21/04/2018 à 18:19, Guillaume Perréal a écrit :

Le 21/04/2018 à 12:28, Didier Kryn a écrit :
    May I humbly ask why you don't start with Devuan (which is Debian 
stripped out of systemd)? Of course it is less challenging than doing 
the job on Debian, but it would be wholehartedly applauded and tested 
by the Devuan community, while you shouldn't expect much recognition 
from Debian.


            Didier
I have tried to boot an Alpine Linux with s6-init and the set of of 
the services I wanted and it is quite a lot a work. I think writing a 
full replacement with shims and "translators" is an order of magnitude 
higher.


Guillaume.


    I'm a follower of musl and a supporter of static 
linking. I started to build my own musl-based Linux before Alpine was 
available and eventually achieved to build a functional native 
development platform. But it is too much work and I plan to try Alpine 
pretty soon :-)


    Whether starting from Alpine or Devuan, at least you don't have to 
fight against Systemd which has put metastases pretty much everywhere in 
the system. You start from sysvinit and just replace the rc scripts even 
with keeping sysv's init proper, at the beginning. A big part of the 
work is packaging and transitionning flawlessly from one system to the 
other when installing/removing the packages.


        Didier





Re: Debianization

2018-04-21 Thread Avery Payne
On Sat, Apr 21, 2018 at 4:27 AM, SZÉPE Viktor  wrote:

>
> Actually my first step of setting up a production Debian-based server is
> reverting back to SysVinit
> https://github.com/szepeviktor/debian-server-tools/blob/
> master/debian-setup.sh#L88-L90
> Currently I haven't reached a single boot, so it is a task for me to
> migrate the above listed services to *native* s6-rc.
>
> Are you looking for run files?  There are several stashed away in various
repositories. Are you looking to have run files for every Debian service
possible?  That is a large task - on Debian 7 there were ~ 900 packages
that would leave rc scripts in /etc/init.d that were not one-shots, so you
have your work cut out.  I'm sure the list has grown since then.

I humbly suggest the following:

1. convert to SysV, and start out using stock rc scripts inside /etc/init.d
2. create Debian packages for s6, and install from the packages.
3. determine where you are locating the directories "service", "svc", and
"svcdef" (or whatever you are calling them) and try to comply with Debian's
rules.  For Debian, I suggest you create the directories /etc/svc and
/etc/svcdef, and then make a symlink from /service pointing to /etc/svc,
which I'm sure is not 100% "Debian acceptable" but is probably the right
thing to do.
4. decide on a logging layout.  You can do anything you want, but to keep
things similar, I suggest that all services that don't do their own logging
receive a s6-log instance, using the T option instead of t (since Debian
doesn't support TAI64N), and that your s6-log output be directed to a file
inside of a directory named /var/log/{service name}.
5. decide on a strategy to replace syslog.  Maybe something that provides
/dev/log.  Not 100% needed, but nice to have.
6. decide on a strategy for udev.  You will need it to run at startup,
prior to switching run levels; I suggest a one-shot that sets up all of the
device names at startup, and a separate longrun that supervises udevd in
normal use.  Not optimal, but it should suffice.
7. decide on a strategy for dbus.  You can easily make a service definition
for it, but you'll have the unpleasant discovery of finding consolekit,
policykit, rtkit, and others being unsupervised. (some day I will peer into
the bowels of dbus and see if there is a way to make it play nice...)
8. decide on a strategy for pulseaudio, which is a worse pain than udev and
dbus together.  You'll have the fun choices like "it runs system wide but
desktop audio is buggy or fails", or "I don't supervise it but I have
desktop audio", or my favorite, "just suffer without it and install ALSA".
9. start writing run scripts.  Look around github for little repos here and
there, and look at what people are doing.  You can contact people in
private if you get stuck.  Be sure to bundle the scripts into a package
that can be re-installed.  Perhaps name it s6-debian-runscripts or
something similar.
10. if you are willing to put up with beta-grade software while you
transition, use rc-shim to replace the scripts in /etc/init.d one by one,
so you can cleanly boot with SysV AND have s6 supervising your services.  I
will warn you that it still has some rough edges, but it should be "good
enough" for the time being.  If you do decide to use rc-shim, you will want
to make a small modification that copies /etc/svcdef/{servicename} to
/etc/svc/{servicename} when it starts a service for the first time...
11. once you have run scripts for everything, you can make the move to a
pure s6 based system, and you don't need the rc-shim anymore, or SysV.  Yay!
12. please share your s6-debian-runscripts package, so that all Debian and
Devuan users can move to s6 as well. :)

That's quite the list to do, and I wish you the best of luck.


Re: Debianization

2018-04-21 Thread Guillaume Perréal

Le 21/04/2018 à 12:28, Didier Kryn a écrit :
    May I humbly ask why you don't start with Devuan (which is Debian 
stripped out of systemd)? Of course it is less challenging than doing 
the job on Debian, but it would be wholehartedly applauded and tested 
by the Devuan community, while you shouldn't expect much recognition 
from Debian.


            Didier
I have tried to boot an Alpine Linux with s6-init and the set of of the 
services I wanted and it is quite a lot a work. I think writing a full 
replacement with shims and "translators" is an order of magnitude higher.


Guillaume.


Re: Debianization

2018-04-21 Thread SZÉPE Viktor

Idézem/Quoting Didier Kryn :


Le 20/04/2018 à 07:48, SZÉPE Viktor a écrit :

Idézem/Quoting Colin Booth :


On Thu, Apr 19, 2018 at 11:10:33PM +, Laurent Bercot wrote:

Have any started (thinking about) packaging skalibs, execline and s6
the Debian policy-obeying way?


 What does that mean? If it's just FHS, skarnet.org follows FHS by
default, so there's no problem here. Are there other constraints
restricting what a Debian package can do?

It's technically a pile of things:
libs/skalibs (the .so)
lib-devel/skalivs-dev (the .a for compiling against
lang/execline
lib/libexecline (the .so)
lib-devel/libexecline-dev (the .a again)
and so on. I have sort-of complaint packages that I used to get my build
chain working for a project at work, but correctly separating stuff isa
chunk of work. Doable, just obnoxious.


In a previous message:


... into both .deb's and .rpm's using fpm without much fuss.


That much fuss includes 859 lintian tags e.g. gcc hardening, rc  
scripts for essential Debian packages, a systemd->s6 migration  
utility etc.

https://lintian.debian.org/tags.html

So basically I type apt-get install s6 and it makes systemd  
disappear, and all these basic services work after reboot:


cgroupfs-mount
console-setup.sh
cron
dbus
hwclock.sh
keyboard-setup.sh
kmod
networking
procps
rsync
rsyslog
ssh
udev

Is there a documentation how to turn e.g. 'networking' into an s6 rc?


SZÉPE Viktor, honlap üzemeltetés / Running your application
https://github.com/szepeviktor/debian-server-tools/blob/master/CV.md



    May I humbly ask why you don't start with Devuan (which is  
Debian stripped out of systemd)? Of course it is less challenging  
than doing the job on Debian, but it would be wholehartedly  
applauded and tested by the Devuan community, while you shouldn't  
expect much recognition from Debian.


            Didier


Thank you for the advise!
Actually my first step of setting up a production Debian-based server  
is reverting back to SysVinit

https://github.com/szepeviktor/debian-server-tools/blob/master/debian-setup.sh#L88-L90
Currently I haven't reached a single boot, so it is a task for me to  
migrate the above listed services to *native* s6-rc.




SZÉPE Viktor, honlap üzemeltetés / Running your application
https://github.com/szepeviktor/debian-server-tools/blob/master/CV.md
--
ügyelet/hotline: +36-20-4242498  s...@szepe.net  skype: szepe.viktor
Budapest, III. kerület







Re: Debianization

2018-04-21 Thread Didier Kryn

Le 20/04/2018 à 07:48, SZÉPE Viktor a écrit :

Idézem/Quoting Colin Booth :


On Thu, Apr 19, 2018 at 11:10:33PM +, Laurent Bercot wrote:

> Have any started (thinking about) packaging skalibs, execline and s6
> the Debian policy-obeying way?

 What does that mean? If it's just FHS, skarnet.org follows FHS by
default, so there's no problem here. Are there other constraints
restricting what a Debian package can do?

It's technically a pile of things:
libs/skalibs (the .so)
lib-devel/skalivs-dev (the .a for compiling against
lang/execline
lib/libexecline (the .so)
lib-devel/libexecline-dev (the .a again)
and so on. I have sort-of complaint packages that I used to get my build
chain working for a project at work, but correctly separating stuff isa
chunk of work. Doable, just obnoxious.


In a previous message:


... into both .deb's and .rpm's using fpm without much fuss.


That much fuss includes 859 lintian tags e.g. gcc hardening, rc 
scripts for essential Debian packages, a systemd->s6 migration utility 
etc.

https://lintian.debian.org/tags.html

So basically I type apt-get install s6 and it makes systemd disappear, 
and all these basic services work after reboot:


cgroupfs-mount
console-setup.sh
cron
dbus
hwclock.sh
keyboard-setup.sh
kmod
networking
procps
rsync
rsyslog
ssh
udev

Is there a documentation how to turn e.g. 'networking' into an s6 rc?


SZÉPE Viktor, honlap üzemeltetés / Running your application
https://github.com/szepeviktor/debian-server-tools/blob/master/CV.md



    May I humbly ask why you don't start with Devuan (which is Debian 
stripped out of systemd)? Of course it is less challenging than doing 
the job on Debian, but it would be wholehartedly applauded and tested by 
the Devuan community, while you shouldn't expect much recognition from 
Debian.


            Didier





Re: Debianization

2018-04-20 Thread Colin Booth
On Fri, Apr 20, 2018 at 07:48:23AM +0200, SZÉPE Viktor wrote:
> > ... into both .deb's and .rpm's using fpm without much fuss.
> 
> That much fuss includes 859 lintian tags e.g. gcc hardening, rc scripts for
> essential Debian packages, a systemd->s6 migration utility etc.
> https://lintian.debian.org/tags.html
That's a different story...
> 
> So basically I type apt-get install s6 and it makes systemd disappear, and
> all these basic services work after reboot:
> 
> cgroupfs-mount
> console-setup.sh
> cron
> dbus
> hwclock.sh
> keyboard-setup.sh
> kmod
> networking
> procps
> rsync
> rsyslog
> ssh
> udev
Ah, you're talking about an s6-init package that pre-depends on s6-rc,
s6, and execline (at a minimum). No, nothing exists like that currently.
A handful of us (myself included) run debian systems under s6/s6-rc but
as far as the packaging system is concerned they are still running
sysvinit. Doing a full strip out is a pile of work, not to mention a lot
of equivs stuff to keep from breaking a mountain of packages.
> 
> Is there a documentation how to turn e.g. 'networking' into an s6 rc?
> 
The simplest way is to have an s6-rc oneshot that calls
'/etc/init.d/networking start' and then adjust the dependencies to put
it in the right place in the boot order. The other option is to rewrite
that script in execline. The rest of the rcS stuff is similar. 

A word of warning for removing sysvinit or systemd packages entirely: a
lot (all?) of the packages for daemons expect one of those inits to be
present. This expectation comes in the form of post-install hooks and
directory assumptions (something with an init script will expect
/etc/init.d to exist, a guarantee that can't be made if sysvinit isn't
present). A shim package covers that need, but that's more plumbing
for an early-stage project.

As I see it, this is what's needed for a full cutover: an s6-init
package, an s6-systemd-shim package, and s6-rc. s6-init has the
stage1/2/3 stuff, and the single-user mode initalization oneshots.
s6-systemd-shim has the directories, a unit-file translation/compilation
layer (does not exist yet), and translation scripts to mask systemd
commands and emit s6-rc commands in their place. And s6-rc owns the
s6-rc suite, /etc/s6-rc, and anything else belonging to the
infrastructure around managing your source and compileds. Beyond that,
everything either already exists in debian, or is in the dependency
chain for s6-rc (s6, execline, maybe the s6-*-utils, etc).
> 
> SZÉPE Viktor, honlap üzemeltetés / Running your application
> https://github.com/szepeviktor/debian-server-tools/blob/master/CV.md
Cheers!
-- 
Colin Booth


Re: Debianization

2018-04-19 Thread SZÉPE Viktor

Idézem/Quoting Colin Booth :


On Thu, Apr 19, 2018 at 11:10:33PM +, Laurent Bercot wrote:

> Have any started (thinking about) packaging skalibs, execline and s6
> the Debian policy-obeying way?

 What does that mean? If it's just FHS, skarnet.org follows FHS by
default, so there's no problem here. Are there other constraints
restricting what a Debian package can do?

It's technically a pile of things:
libs/skalibs (the .so)
lib-devel/skalivs-dev (the .a for compiling against
lang/execline
lib/libexecline (the .so)
lib-devel/libexecline-dev (the .a again)
and so on. I have sort-of complaint packages that I used to get my build
chain working for a project at work, but correctly separating stuff is a
chunk of work. Doable, just obnoxious.


In a previous message:


... into both .deb's and .rpm's using fpm without much fuss.


That much fuss includes 859 lintian tags e.g. gcc hardening, rc  
scripts for essential Debian packages, a systemd->s6 migration utility  
etc.

https://lintian.debian.org/tags.html

So basically I type apt-get install s6 and it makes systemd disappear,  
and all these basic services work after reboot:


cgroupfs-mount
console-setup.sh
cron
dbus
hwclock.sh
keyboard-setup.sh
kmod
networking
procps
rsync
rsyslog
ssh
udev

Is there a documentation how to turn e.g. 'networking' into an s6 rc?


SZÉPE Viktor, honlap üzemeltetés / Running your application
https://github.com/szepeviktor/debian-server-tools/blob/master/CV.md
--
ügyelet/hotline: +36-20-4242498  s...@szepe.net  skype: szepe.viktor
Budapest, III. kerület