Re: ipc in process #1

2019-05-10 Thread Laurent Bercot




IMO process #1 should be solely signal driven.
i guess there is no need for further ipc by other means,
escpecially for those that require rw fs access to work
(eg fifos, unix sockets).


The devil is in the details here. Theoretically, yes, you're right.
In practice, only using signals to control pid 1 is rather limiting,
and the choice to do so or not to do so is essentially decided by
other factors such as the existing interfaces you want to follow.

For instance, if you want to implement the stage 3 procedure in
pid 1, and you want to follow the LSB interface for the "shutdown"
binary - all of which is the case for sysvinit - then signals are
not enough: you need to be able to convey more information to pid 1
than a few signals can.

But yes, limiting what pid 1 does is reducing its attack surface,
which is a good idea in general, and it is totally possible to
design a correct pid 1 that is only controlled by signals.



apart from that process #1 has IMO only the additional duty of
leaving stage 2 and entering staqe 3 when requested.


And reaping zombies, and supervising at least one process. :)



for those who still need more than what is provided by signals
i would recommend using abstract unix domain sockets (Linux only)
or SysV message queues (the latter works even on OpenBSD) since
those ipc mechanisms can work properly without rw fs access.


Have you tried working with SysV message queues before recommending
them? Because my recommendation would be the exact opposite. Don't
ever use SysV message queues if you can avoid it. The API is very
clumsy, and does not mix with event loops, so it constrains your
programming model immensely - you're practically forced to use threads.
And that's a can of worms you certainly don't want to open in pid 1.

Abstract sockets are cool - the only issue is that they're not
portable, which would limit your init system to Linux only. If you're
going for a minimal init, it's a shame not to make it portable.

Really, the argument for an ipc mechanism that does not require a
rw fs is a weak one. Every modern Unix can mount a RAM filesystem
nowadays, and it is basically essential to do so if you want early
logs. Having no logs at all until you can mount a rw fs is a big
no-no, and being unable to log what your init system does *at all*,
unless you're willing to store everything in RAM until a rw fs
comes up, is worse. An early tmpfs technically stores things in RAM
too, but is much, *much* cleaner, and doesn't need ad-hoc code or
weird convolutions to make it work.

Just use an early tmpfs and use whatever IPC you want that uses a
rendez-vous point in that tmpfs to communicate with process 1.
But just say no to message queues.

--
Laurent



Re: s6-log problem with +regex

2019-05-10 Thread Laurent Bercot




However without any control directive, the result is:
s6-log: usage: s6-log [ -d notif ] [ -q | -v ] [ -b ] [ -p ] [ -t ] [ -e
] [ -l linelimit ] logging_script

Though running s6-log without a control directive is probably a little
silly, perhaps the requirement to have one may be worthwhile mentioning
in the doc.


 Again, I cannot reproduce that, either on Linux or on FreeBSD.
Running s6-log without a control directive works as intended for me.
Can you please paste the exact command line you're running that causes
the issue for you ?



Aside: I had orginally placed
ErrorLog "|/usr/local/bin/s6-log -b n32 s5 S700
/var/log/httpd-error T !'/usr/bin/xz -7q' /var/log/httpd-error"
into apache24 which worked well in testing (one httpd), but of course in
production there are lots of httpd that do NOT use the parent for
logging errors, so locking is a problem.


Locking won't be a problem unless your services are logging lines
that are longer than (at least) 4 kB. For lines that are shorter than
4 kB, writing/reading a line through a pipe will be done atomically.
In a normal Apache logging configuration, lines won't be too long,
so you'll be fine.



Because I have three websites (3x error files, 3x access files) I was
looking at using 6 pipelines into two s6-log processes and regex's to
route the content. (hence my original example).  Is this a good use of
resources or better to pipeline (funnel) to their own s6-log?


 It's entirely your choice. The s6-log process doesn't take a lot of
resources on its own, so my default choice would be to use a s6-log
process per log stream - because it's always easier to merge logs
than it is to separate them. If your priority is to use the least
amount of CPU, or if you're not sure, definitely use more s6-log
processes and less regex matching. But if your priority is to use as
little RAM as possible, you'll probably get slightly better results
by funneling several log streams into one s6-log process and using
some regex matching. I have not profiled this, though.

--
 Laurent