Re: GNU Emacs now runs in foreground

[Jonathan de Boyne Pollard]

Jonathan de Boyne Pollard:
One of the interesting developments over the past couple of decades is 
how much the world has been influenced to come around to the 
daemontools way of doing this. I've observed before, elsewhere, the 
number of daemon programs, especially in the BSD worlds, that have in 
that time gained -F/--foreground or similar options that tell them not 
to do a whole bunch of this unnecessary stuff.


Martin "eto" Misuth:
I think this is bit related to daemon(8) program which acts as poor 
man's runsv, with all the "advandages" of pid files.


It isn't, though.  Several of the manual pages that I've seen explicitly 
mention daemontools.  Here's winbindd's manual page, for example:



-F

If specified, this parameter causes the main winbindd process to not 
daemonize, i.e. double-fork and disassociate with the terminal. Child 
processes are still created as normal to service each connection 
request, but the main process does not exit. This operation mode is 
suitable for running winbindd under process supervisors such as 
supervise and svscan from Daniel J. Bernstein's daemontools package, 
or the AIX process monitor.

services that need DISPLAY

[Jonathan de Boyne Pollard]

Daniel Kahn Gillmor:
Yet surely there are some user-wide services that don't need DISPLAY 
at all, and would be happy to run per-user?


GNOME Terminal isn't one of them.  Witness the behaviour of 
gnome-terminal-server run as a service if it doesn't have a DISPLAY 
environment variable:



@4000584830f10ae09b9b Unable to init server: Could not connect: Connection 
refused
@4000584830f10ae0dbb8 Failed to parse arguments: Cannot open display:

Re: GNU Emacs now runs in foreground

[Martin "eto" Misuth]

> On Thu, 8 Dec 2016 08:33:22 +
> Jonathan de Boyne Pollard  wrote:
>
> Martin "eto" Misuth:
> > On my presonal box "user level" s6 /services subtrees are in 
> > `.config/s6/host`   
> 
> For comparison: ~/.config/service-bundles/services/ and 
> ~/.config/service-bundles/targets/

Well what I like most is that I am in control, and I decide trees locations,
yet I can easily "merge trees" with system level supervisors.

Not just that, I can assemble "application subtrees" as well, that themselves
are "supervisable". 

Most importantly when paired with some scripting language, I can actually make
things that run on both systemdized linux machines, non systemdized linux
ones and on several other OSes.

> On Thu, 8 Dec 2016 08:33:22 +
> Jonathan de Boyne Pollard  wrote:
> 
> Martin "eto" Misuth:
> > #!/bin/sh
> > exec mpd --no-daemon /usr/home/eto/.config/mpd/mpd.conf  
> 
> Tip: In the daemontools world all services have ther own service 
> directory, at minimum; and this directory is the working directory of 
> the service.  You can put an mpd.conf that is particular to the service 
> in the service directory itself.
>
> On Thu, 08 Dec 2016 09:55:11 +
> "Laurent Bercot"  wrote:
>   Additionally, most Unix distributions traditionally keep their config
> files in /etc, and the change to a supervision system is already a big
> shock to them - the shock can be made easier to overcome if we don't
> change all the habits at once. :)
> 
>   Anyway, I don't think we should be arguing one way or the other;
> policy decisions are best left to sysadmins and distributions.

I use  with my own scripts, but for things having
local /etc, /usr/local/etc alternatives I prefer those.

> 
> Martin "eto" Misuth:
> > Getting tmux "supervised" was bit tricky as it double forks the daemon 
> > if not running, and is thus is "escaping under init" under normal 
> > conditions.  
> 
> Note: This can be easily handled on FreeBSD and TrueOS.  See NOTE_TRACK 
> in EVFILT_PROC for kevent().

I saw that filter, and tried to find examples, but my google-fu is not that good
these days of dumbed down search engines. The sad thing is that it is kqueue
only. Although kqueue is wonderful, it's BSDs only :). And although I would love
to have only BSDs on servers, I still have to have "exit path" into linux world.

But still kqueue is very nice indeed.

With other person on irc, we noticed there are some gotchas to subreaping,
especially reparented zombies don't sometimes (never?) generate SIGCHLD. 
I think that might be the reason why my PID1 s6-svscan on FreeBSD is
accumulating zombies sometimes (seems like it is affected by dead descendants of
ssh and my experiments).

More over although procctl() on BSDs allows one to enumerate whole subreaper
tree and even send signals atomically, it's not that ideal.

"No children" case detection with "nonblocking wait'n'see ECHILD" seems pretty
sufficient to me, at least for that for that one case. But i belive smething
better could be done.

Anyway as you are probably much closer to FreeBSD team than I am, what do 
you think are chances of subreaping capable BSDs to gain new kqueue note
NOTE_REPARENTED in EVFILT_PROC filter? 

This would allow one to simply "sleep" indefinitely in core of event loop until
reparented process is acquired as child, and add it to "wait_for" list for 
example.

Currently, I am noticing, that if I want to be 100% sure, that I "eat" also
reparented zombies, I need to wake up periodically.

> Martin "eto" Misuth:
> > It would be great if tmux project allowed to spawn raw user level 
> > daemon by explicit command,  
> 
> One of the interesting developments over the past couple of decades is 
> how much the world has been influenced to come around to the daemontools 
> way of doing this.  I've observed before, elsewhere, the number of 
> daemon programs, especially in the BSD worlds, that have in that time 
> gained -F/--foreground or similar options that tell them not to do a 
> whole bunch of this unnecessary stuff.

I think this is bit related to daemon(8) program which acts as poor man's runsv,
with all the "advandages" of pid files.
 
> It's in a way somewhat saddening to see the new option to emacs that 
> started this thread.  It's welcome; but the sad part is that it's a 
> decade and a half behind quite a lot of others.  For some systems like 
> emacs the process has been glacially slow.  MySQL is another. 
> daemontools users were asking about letting daemontools do the service 
> management and doing away with mysqld_safe in 2002.  It took thirteen 
> years for it to happen ... in MariaDB.  Oracle, where software goes to 
> die, may or may not have done the same for MySQL. PostgreSQL lags 
> further behind.

I am not so sure about PostgreSQL my experience differs a bit:

 "By default postgres starts in the foreground and prints log messages to 

Re: subreapers

[Laurent Bercot]

 Jonathan, since you have implemented that mechanism, could you please
shed some light on the goal of subreapers? What is the point? Your post
in the stackexchange forum talks about the history and the mechanism of
subreapers, but remains silent on why we should have that functionality
in the first place.

 I for one have never understood the need for subreapers, and I would
like your opinion.

--
 Laurent

passing the listening socket as an open file descriptor

[Jonathan de Boyne Pollard]

Martin "eto" Misuth:


I personally am not so "hot" about this listening socket passing stuffs.


When you've had to deal with tens if not hundreds of different ways of 
saying "listen on this IP address and port", you'll come around to the 
idea of having a single tool that does this one job universally.  (-:

subreapers

[Jonathan de Boyne Pollard]

Martin "eto" Misuth:
I think Mr Jonathan de Boyne Pollard might be cooking, or even already 
has, something similar in nosh. 


Long since.  (-:  It was in version 1.0 .

Martin "eto" Misuth:
at some point I was interested in digging out whether systemd had 
"subreapers" at it's disposal, and why it didn't use them 


It has.  The systemd people were responsible for getting this mechanism 
into Linux in the first place.


* http://unix.stackexchange.com/a/177361/5132

Martin "eto" Misuth:
- observing behaviour of reparenting under init for thousands of times 
in htop


Now observe it under not init and not htop.  (-:

Here's the nosh service manager's subreaper mechanism in action. Process 
31427 is a per-user service manager, which is a subreaper.  Process 9408 
is GNOME Terminal and process 9412 is a Z Shell running connected to 
that terminal.  In the shell, I ran (sleep 6000&) which resulted in an 
orphaned sleep process, process 9451.  As you can see, that process has 
been reparented to the nearest subreaper.


per-user-manage(31423)-+-cyclog(31426)
   `-service-manager(31427)-+-3(6088)
|-4(31592)-+-{decoder}(31598)
|  |-{io}(31596)
|  |-{output:default 
}(31599)
|  `-{player}(31597)

|-at-spi-bus-laun(9276)-+-dbus-daemon(9290)
|   
|-{dconf worker}(9277)
|   
|-{gdbus}(9279)
|   
`-{gmain}(9291)

|-at-spi2-registr(9293)---{gdbus}(9296)
|-cyclog(6070)
|-cyclog(18028)
|-cyclog(20138)
|-cyclog(30089)
|-emacs(6094)---{gmain}(6095)

|-gnome-terminal-(9408)-+-gnome-pty-helpe(9411)
|   
|-zsh(9412)---pstree(9482)
|   
|-{dconf worker}(9409)
|   
|-{gdbus}(9410)
|   
`-{gmain}(9413)
|-gvfsd(9305)---{gdbus}(9306)

|-pulseaudio(3625)---{null-sink}(3626)
`-sleep(9451)

Re: GNU Emacs now runs in foreground

[Jonathan de Boyne Pollard]

Martin "eto" Misuth:
On my presonal box "user level" s6 /services subtrees are in 
`.config/s6/host` 


For comparison: ~/.config/service-bundles/services/ and 
~/.config/service-bundles/targets/


Martin "eto" Misuth:

#!/bin/sh
exec mpd --no-daemon /usr/home/eto/.config/mpd/mpd.conf


Tip: In the daemontools world all services have ther own service 
directory, at minimum; and this directory is the working directory of 
the service.  You can put an mpd.conf that is particular to the service 
in the service directory itself.


Martin "eto" Misuth:
Getting tmux "supervised" was bit tricky as it double forks the daemon 
if not running, and is thus is "escaping under init" under normal 
conditions.


Note: This can be easily handled on FreeBSD and TrueOS.  See NOTE_TRACK 
in EVFILT_PROC for kevent().


Martin "eto" Misuth:
It would be great if tmux project allowed to spawn raw user level 
daemon by explicit command,


One of the interesting developments over the past couple of decades is 
how much the world has been influenced to come around to the daemontools 
way of doing this.  I've observed before, elsewhere, the number of 
daemon programs, especially in the BSD worlds, that have in that time 
gained -F/--foreground or similar options that tell them not to do a 
whole bunch of this unnecessary stuff.


It's in a way somewhat saddening to see the new option to emacs that 
started this thread.  It's welcome; but the sad part is that it's a 
decade and a half behind quite a lot of others.  For some systems like 
emacs the process has been glacially slow.  MySQL is another. 
daemontools users were asking about letting daemontools do the service 
management and doing away with mysqld_safe in 2002.  It took thirteen 
years for it to happen ... in MariaDB.  Oracle, where software goes to 
die, may or may not have done the same for MySQL. PostgreSQL lags 
further behind.


* http://jdebp.eu./Softwares/nosh/mariadb-and-mysql.html#Prompt