Re: [systemd-devel] Service Type for Tomcat

[Lennart Poettering]

On Wed, 19.04.17 20:32, Andrei Borzenkov (arvidj...@gmail.com) wrote:

> 18.04.2017 21:35, Jonathan de Boyne Pollard пишет:
> > "Igal @ Lucee.org" :
> >> Examples I see online use forking [...]
> > 
> > ... because they are bad examples.  Read
> > http://jdebp.eu./FGA/systemd-house-of-horror/tomcat.html .
> 
> Service type simple is the worst possible type as it does not provide
> for any synchronization between started process and dependent services.
> So examples that recommend forking are correct (as long as forking is
> implemented correctly). If you have choice between working forking and
> simple, forking is definitely preferred.

I can only second that. The various types in order of preference:

1) Type=notify — if the daemon supports that this is almost always the
   best option, as no forking and no messy PID files are involved, and
   proper ready synchronization is supported. Of course, the daemons
   need to support this method explicitly. Supporting this is easy
   though: you can either use libsystemd's sd_notify() function or any
   similar library for your language of choice. In fact, the
   underlying protocol is so simple and generic, that it is trivial to
   reimplement without involving any external library altogether:
   simply send a single AF_UNIX/SOCK_DGRAM datagram to the socket set
   in $NOTIFY_SOCKET containing a newline-separated text string and
   you are done. (There's one trick though: if $NOTIFY_SOCKET starts
   with an '@' character, it refers to a socket in the Linux AF_UNIX
   'abstract' namespace, which essentially means you need to replace
   it with a NUL char when binding).

2) Type=forking — this is the traditional UNIX way. It requires
   double-fork()ing in order to daemonize. If your daemon consists of
   multiple processes your daemon also needs to write out a PID file
   (and you need to tell systemd about it via PIDFile=), otherwise
   systemd can't know which the 'main' process of your daemon
   is. Double-fork()ing correctly is hard, and traditionally people
   haven't been very good at implementing this correctly. Well-known
   mistakes are: people only fork() once, not twice, which means the
   process won't be reparented to PID 1 correctly — systemd is very
   forgiving on this one though. More problematic is if daemons exit
   in the parent process before the grandchild properly completed
   initialization and established all listening sockets and
   suchlike. For broken daemons like that startup synchronization won't
   work, as systemd will continue starting the next daemons at a time
   where these daemons haven't finished start-up yet. Note that not
   implementing this correctly not only is broken on systemd but also
   on all other init systems, including traditional SysV. Note that
   Type=forking is hard to use in programming environments where
   fork() is not available. Specifically, Go, Java, and many other
   higher-level programming languages don't really support fork()
   without execve(), i.e. they don't permit the language runtime being
   duplicated without all associated threads.

3) Type=simple — in this mode, systemd won't do any synchronization,
   and just fork off your daemon and immediately proceed with the
   next. This is a great option for daemons which are unlikely to fail
   or where failure shall not be propagated to any depending
   services. This is a particularly good option for socket activated
   services, as for them all communication channels are already
   established before the service starts, and synchronization is hence
   redundant. Type=simple is not a useful option however, if your
   daemon has a startup phase where it establishes communication
   channels or if it can likely fail during its startup phase and that
   fact should propagated to depending services.

Under the assumption that Tomcat does have communication channels
(most non-trivial software has) and there are depending services that
need to wait for them to be established, and given that Tomcat appears
to be written in Java, I am pretty sure Type=notify is the best
option, but it does require sending out that notification message with
sd_notify() or something equivalent.

Given that systemd is pretty universally adopted in bigger
distributions, in particular commercial ones these days, it should not
be impossible to convince the Tomcat developers to add native support
for the sd_notify() message. In particular as support for it is
entirely transparent: if $NOTIFY_SOCKET is set, send that one ready
message to it, otherwise don't. This will make things work nicely with
systemd, and won't affect systems not using it.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Service Type for Tomcat

[Jonathan de Boyne Pollard]

"Igal @ Lucee.org" :

> Examples I see online use forking [...]

Jonathan de Boyne Pollard:

> ... because they are bad examples. 
> Read http://jdebp.eu./FGA/systemd-house-of-horror/tomcat.html .

Andrei Borzenkov:

> Service type simple is the worst possible type as it does not provide 
> for any synchronization between started process and dependent 
> services.  So examples that recommend forking are correct (as long 
> as forking is implemented correctly).

... which it isn't.  Read
http://jdebp.eu./FGA/systemd-house-of-horror/tomcat.html again, properly this
time, and then read
http://jdebp.eu./FGA/unix-daemon-readiness-protocol-problems.html#NooneSpeaksForking
.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Service Type for Tomcat

[Andrei Borzenkov]

18.04.2017 21:35, Jonathan de Boyne Pollard пишет:
> "Igal @ Lucee.org" :
>> Examples I see online use forking [...]
> 
> ... because they are bad examples.  Read
> http://jdebp.eu./FGA/systemd-house-of-horror/tomcat.html .

Service type simple is the worst possible type as it does not provide
for any synchronization between started process and dependent services.
So examples that recommend forking are correct (as long as forking is
implemented correctly). If you have choice between working forking and
simple, forking is definitely preferred.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Service Type for Tomcat

[Jonathan de Boyne Pollard]

"Igal @ Lucee.org" :
> Examples I see online use forking [...]

... because they are bad examples.  Read
http://jdebp.eu./FGA/systemd-house-of-horror/tomcat.html .
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Service Type for Tomcat

[Mantas Mikulėnas]

On Tue, Apr 18, 2017 at 8:44 PM, Igal @ Lucee.org  wrote:

> I've read about the difference between "forking" and "notify", but am not
> sure how it really applies in real life.
>
> Can someone tell me what would be the consequences of setting Tomcat (or
> any Java-based service, for that matter) to Type=notify instead of
> Type=forking?  Examples I see online use forking but I'm not sure that
> that's the right way to go.
>
Both types depend on the service process actually sending the correct kind
of notification once it's ready to work. (This helps systemd avoid starting
dependent services too early.) If the notification is not received in ~90
seconds, systemd considers the service failed to start and kills all
remaining processes.

For example, with Type=forking, systemd expects the main process to "fork
into background" (daemonize) once it's ready. Similarly, if you use
Type=notify, systemd will wait for "READY=1" to be sent over a Unix socket.

Tomcat does not seem to have any code that would implement Type=notify, so
if you try to use it, your service will spend its first minute in the
'activating' stage, and get immediately killed afterwards.

-- 
Mantas Mikulėnas 
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Service Type for Tomcat

[Igal @ Lucee.org]

On 4/18/2017 10:58 AM, Mantas Mikulėnas wrote:
For example, with Type=forking, systemd expects the main process to 
"fork into background" (daemonize) once it's ready. Similarly, if you 
use Type=notify, systemd will wait for "READY=1" to be sent over a 
Unix socket.


Tomcat does not seem to have any code that would implement 
Type=notify, so if you try to use it, your service will spend its 
first minute in the 'activating' stage, and get immediately killed 
afterwards.


Thank you for the explanation :)
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Service Type for Tomcat

[Tomasz Torcz]

On Tue, Apr 18, 2017 at 10:44:59AM -0700, Igal @ Lucee.org wrote:
> I've read about the difference between "forking" and "notify", but am not
> sure how it really applies in real life.
> 
> Can someone tell me what would be the consequences of setting Tomcat (or any
> Java-based service, for that matter) to Type=notify instead of Type=forking?
> Examples I see online use forking but I'm not sure that that's the right way
> to go.

 For “notify”, application has to have explicit support for systemd.  It has to
send READY=1 notification using socket. Usually, when application authors 
implement
type=notify support, they ship example unit, so you don't have to guess.
 Implementing Type=notify is very simple, see https://github.com/faljse/SDNotify
You can ask Tomcat developers to implement it.
 Also, if you use wrong type, your application may not fail instantly. Sometimes
minute and a half passes before it fails. See 
https://enotty.pipebreaker.pl/2014/10/08/failure-modes-of-incorrect-type-in-systemd-units/

-- 
Tomasz Torcz   72->|   80->|
xmpp: zdzich...@chrome.pl  72->|   80->|

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel