Re: [DRE-maint] unicorn: native systemd service
Christos Trochalakis yati...@ideopolis.gr wrote:
On Sat, Jun 27, 2015 at 03:05:01AM +, Eric Wong wrote:
Is it possible to make systemd fire up two unicorn masters?
That would be a nice feature to have with socket activation.
That would be great! I am a bit surprised that specifying multiple
services ('Service=' option) is not supported.
Apparently I was wrong. I missed the 'Sockets=' option for Service units.
You can specify a list of socket units to be inherited (if active) when
the service is started. Note that this is not the reverse of Sockets'
'Service=' option: the service is not auto-started when there is an
incoming connnection on the relevant socket, you have to use 'Service='
for socket activation.
Here is a test config for 2 preforked unicorn masters using service
templates (I am using the latest unicorn with the systemd patch):
== /srv/uni/config.ru ==
puts initializing for 5sec
sleep 5
app = proc do |env|
[
200,
{ 'Content-Type' = 'text/plain' },
[ppid:#{Process.ppid}\n]
]
end
run app
== /srv/uni/unicorn.conf.rb ==
worker_processes 2
working_directory /srv/uni
# Keep in sync with uni.socket file
listen 9000
listen 9001
== /etc/systemd/system/uni@.service ==
[Unit]
Description=Unicorn Server %i
Wants=uni.socket
After=uni.socket
[Service]
ExecStart=/usr/local/bin/unicorn -c /srv/uni/unicorn.conf.rb -d
Sockets=uni.socket
KillSignal=SIGQUIT
[Install]
WantedBy=multi-user.target
== /etc/systemd/system/uni.socket ==
[Unit]
Description=Unicorn Sockets
[Socket]
ListenStream=0.0.0.0:9000
ListenStream=0.0.0.0:9001
Service=uni@1.service
[Install]
WantedBy=sockets.target
== end ==
systemctl daemon-reload
systemctl enable uni.socket
systemctl start uni.socket
systemctl enable uni@1 uni@2 # Make them start at boot
systemctl start uni@1 uni@2
Now you can start and stop uni@1 and uni@2 with zero latency and without losing
a connection. So we have a happy ending after all :)
While searching systemd's mailing list, I found a thread discussing about a
'Distribute=' option to sockets a few years back[0]. There is also a relevant
item in systemd's TODO list.
[0] http://thread.gmane.org/gmane.comp.sysutils.systemd.devel/14242/focus=14255
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Christos Trochalakis yati...@ideopolis.gr wrote:
On Sat, Jun 27, 2015 at 03:05:01AM +, Eric Wong wrote:
Christos Trochalakis yati...@ideopolis.gr wrote:
KillMode=mixed
I don't think KillMode=mixed is necessary, here. systemd can send
SIGQUIT to workers.
Perhaps there is a race here, if a worker receives SIGQUIT first, and the
master respawns a new worker before receiving/handling its own SIGQUIT.
This is definitely a longshot, and will probably never happen, but
even then every process will eventually quit gracefully.
Right. This race is possible, but workers will automatically die if
they don't have a master.
Easier just to kill the master, of course.
Is it possible to make systemd fire up two unicorn masters?
That would be a nice feature to have with socket activation.
That would be great! I am a bit surprised that specifying multiple
services ('Service=' option) is not supported.
Can you get this feature added to systemd?
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On Sat, Jun 27, 2015 at 03:05:01AM +, Eric Wong wrote:
Christos Trochalakis yati...@ideopolis.gr wrote:
On Thu, Jun 25, 2015 at 11:26:26PM +, Eric Wong wrote:
With socket activation, you should just be able to kill unicorn using
SIGQUIT (just master, or even all workers) and restart without ever
dropping a connection. I do NOT suggest using SIGTERM for unicorn,
since that'll cause the master to kill all workers ASAP.
Yes, you are right socket activation is also an option! I have made some
experiments with a simple rack app to test it.
systemd uses the LISTEN_FDS env variable that is an integer indicating the
number of inherited file descriptors. Those FDs have consecutive numbers
starting from `SD_LISTEN_FDS_START` which is `3` (man sd_listen_fds).
So for example if LISTEN_FDS=2, UNICORN_FD should be 3,4. I used a
simple wrapper script for that. Here is the full configuration:
OK, I'll probably add LISTEN_FDS and LISTEN_PID support to unicorn
directly so the wrapper is unnecessary.
snip
KillMode=mixed
I don't think KillMode=mixed is necessary, here. systemd can send
SIGQUIT to workers.
Perhaps there is a race here, if a worker receives SIGQUIT first, and the
master respawns a new worker before receiving/handling its own SIGQUIT.
This is definitely a longshot, and will probably never happen, but
even then every process will eventually quit gracefully.
snip
TCP socket options are not applied by unicorn on inherited sockets (TCPSocket
=== sock is false). systemd socket files have support for most options now but
we might want unicorn to `setsockopt` them as well. For example,
'DeferAcceptSec', 'KeepAliveIntervalSec', 'NoDelay' are supported since v216, so
they are not available in jessie (v215).
They are now :)
http://bogomips.org/unicorn-public/m/1435373879-4299-1-git-send-emai...@80x24.org.txt
We don't have KeepAliveIntervalSec
(aka TCP_KEEPINTVL) since it's not-portable and probably over
Are you sure about that?
I added 'KeepAliveIntervalSec' by mistake. `KeepAlive`, a supported
option by unicorn, is included in jessie (systemd v215). Either way with
your last patch those options are enforced by unicorn.
socket activation is a really interesting setup, but personally I would not run
it with a large application. Clients would have to wait for the new master to
be up and running before a reply is returned, and that could take tenths of
seconds. The USR2 rexec solves that problem since both old and new workers are
accepting on the socket and we can kill the old ones when we are ready. In that
case the PIDFile trick is handy to support zero downtime restarts with no
latency.
Is it possible to make systemd fire up two unicorn masters?
That would be a nice feature to have with socket activation.
That would be great! I am a bit surprised that specifying multiple
services ('Service=' option) is not supported.
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Hello all, I have recently migrated our main ruby application to systemd implementing zero downtime upgrades. systemd doesn't like replacing the binary on the fly. There is one exception to this, services with PIDFile. When PIDFile is set, systemd reads it when the main process exits and replaces the main process. nginx also had this issue a few months ago [0]. So, in order to support zero-downtime upgrades we have to use a pid file. [0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=728015#15 We also have to use `KillMode=mixed`. That way SIGTERM will only be send to the main process (unicorn master) and not the workers and let unicorn master deal with stopping its children. If the stop timeout passes SIGKILL will be sent to all processes. It is not that usual to add custom actions. I've seen many init.d s scripts and only few had such things. I think systemd was designed using different philosophy. I don't know the history but it could be that limited systemd interface to services is quite intentional. Frankly it is annoying when init.d script does too much but we can avoid this by offering an additional utility to do soft restart and you can call it from init.d script as well if you wish. Perhaps it will be best to introduce a new binary that user could run on upgrade to do zero downtime restart (we can run this utility from postinst). What do you think? I also agree with that, we could introduce a `unicornctl` helper script supporting that kind of things, like `unicornctl upgrade` or `unicornctl rotate`. That helper shouldn't care if this it runs on a systemd or sysvinit system. On more minor note regarding the proposed patch: You can safely use `update-rc.d` in maintainer scripts, no need to use $(which update-rc.d). Quoting https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html Before installation is started, the package management system checks to see if the programs ldconfig, start-stop-daemon, and update-rc.d can be found via the PATH environment variable. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Christos Trochalakis yati...@ideopolis.gr wrote: On Thu, Jun 25, 2015 at 11:26:26PM +, Eric Wong wrote: With socket activation, you should just be able to kill unicorn using SIGQUIT (just master, or even all workers) and restart without ever dropping a connection. I do NOT suggest using SIGTERM for unicorn, since that'll cause the master to kill all workers ASAP. Yes, you are right socket activation is also an option! I have made some experiments with a simple rack app to test it. systemd uses the LISTEN_FDS env variable that is an integer indicating the number of inherited file descriptors. Those FDs have consecutive numbers starting from `SD_LISTEN_FDS_START` which is `3` (man sd_listen_fds). So for example if LISTEN_FDS=2, UNICORN_FD should be 3,4. I used a simple wrapper script for that. Here is the full configuration: OK, I'll probably add LISTEN_FDS and LISTEN_PID support to unicorn directly so the wrapper is unnecessary. snip KillMode=mixed I don't think KillMode=mixed is necessary, here. systemd can send SIGQUIT to workers. snip TCP socket options are not applied by unicorn on inherited sockets (TCPSocket === sock is false). systemd socket files have support for most options now but we might want unicorn to `setsockopt` them as well. For example, 'DeferAcceptSec', 'KeepAliveIntervalSec', 'NoDelay' are supported since v216, so they are not available in jessie (v215). They are now :) http://bogomips.org/unicorn-public/m/1435373879-4299-1-git-send-emai...@80x24.org.txt We don't have KeepAliveIntervalSec (aka TCP_KEEPINTVL) since it's not-portable and probably over Are you sure about that? socket activation is a really interesting setup, but personally I would not run it with a large application. Clients would have to wait for the new master to be up and running before a reply is returned, and that could take tenths of seconds. The USR2 rexec solves that problem since both old and new workers are accepting on the socket and we can kill the old ones when we are ready. In that case the PIDFile trick is handy to support zero downtime restarts with no latency. Is it possible to make systemd fire up two unicorn masters? That would be a nice feature to have with socket activation. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
+Cc unicorn-public list Christos Trochalakis chris...@skroutz.gr wrote: Hello all, I have recently migrated our main ruby application to systemd implementing zero downtime upgrades. systemd doesn't like replacing the binary on the fly. There is one exception to this, services with PIDFile. When PIDFile is set, systemd reads it when the main process exits and replaces the main process. nginx also had this issue a few months ago [0]. So, in order to support zero-downtime upgrades we have to use a pid file. I don't think so. You should be able to bind the listen socket in systemd and rely on the socket activation features (setting the UNICORN_FD environment variable to the created FD). You still need to have the matching listen directive in the unicorn config file so unicorn does not close it. With socket activation, you should just be able to kill unicorn using SIGQUIT (just master, or even all workers) and restart without ever dropping a connection. I do NOT suggest using SIGTERM for unicorn, since that'll cause the master to kill all workers ASAP. If the master dies before the workers (e.g. from SIGKILL), systemd ought to just ignore the workers. I don't actually know the systemd config at all, but anybody familiar with systemd socket activation should have no problems setting the UNICORN_FD env. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Dmitry Smirnov only...@debian.org wrote: On Wed, 24 Jun 2015 23:26:49 Eric Wong wrote: Dmitry: unicorn upstream here, is there anything in unicorn itself can do to make systemd integration easier? Thank you very much for keeping an eye on us and for all your help. I'm not sure if we need anything special -- let me experiment more with soft restart and maybe I'll come back to you with questions. :) No problem! I basically live and breathe plain-text email, so Debian is a good fit for me :) I haven't had much interest or time to dig into systemd, but I'm willing to put some effort into making unicorn work more smoothly with systemd (and any other Free Software process managers/init systems). Great. I'm entertaining idea of modifying init scripts to make soft restart the default behaviour. Is there are any risks or drawbacks of always attempting soft restart instead of normal restart? What do you think? Probably not... I almost regret making SIGTERM behave like nginx instead of making SIGTERM identical to SIGQUIT. OTOH, matching nginx might make learning to use nginx (after learning unicorn signals) easier. I've certainly never used SIGTERM on unicorn or nginx myself. Occasionally I've managed to break the RubyVM enough to require SIGKILL, but I think using SIGTERM is unnecessary other than as training wheels for nginx. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On Thu, Jun 25, 2015 at 11:26:26PM +, Eric Wong wrote:
+Cc unicorn-public list
Christos Trochalakis chris...@skroutz.gr wrote:
Hello all,
I have recently migrated our main ruby application to systemd implementing zero
downtime upgrades.
systemd doesn't like replacing the binary on the fly. There is one exception to
this, services with PIDFile. When PIDFile is set, systemd reads it when the
main process exits and replaces the main process. nginx also had this issue a
few months ago [0].
So, in order to support zero-downtime upgrades we have to use a pid file.
I don't think so. You should be able to bind the listen socket in
systemd and rely on the socket activation features (setting the
UNICORN_FD environment variable to the created FD).
You still need to have the matching listen directive in the unicorn
config file so unicorn does not close it.
With socket activation, you should just be able to kill unicorn using
SIGQUIT (just master, or even all workers) and restart without ever
dropping a connection. I do NOT suggest using SIGTERM for unicorn,
since that'll cause the master to kill all workers ASAP.
Yes, you are right socket activation is also an option! I have made some
experiments with a simple rack app to test it.
systemd uses the LISTEN_FDS env variable that is an integer indicating the
number of inherited file descriptors. Those FDs have consecutive numbers
starting from `SD_LISTEN_FDS_START` which is `3` (man sd_listen_fds).
So for example if LISTEN_FDS=2, UNICORN_FD should be 3,4. I used a
simple wrapper script for that. Here is the full configuration:
$ tail -n+1 /srv/uni/* /etc/systemd/system/uni.*
== /srv/uni/config.ru ==
app = proc do |env|
sleep 5
[
200,
{ 'Content-Type' = 'text/plain' },
[Socket Activated!\n, pid:#{$$}\n, ppid:#{Process.ppid}\n]
]
end
run app
== /srv/uni/unicorn.conf.rb ==
worker_processes 2
working_directory /srv/uni
# Keep in sync with uni.socket
listen 9000, :tcp_nopush = true
listen 9001, :tcp_nopush = true
== /srv/uni/wrapper ==
#!/bin/bash
[ -z $LISTEN_FDS ] exec $@
UNICORN_FD=
for fd in `seq 3 $(($LISTEN_FDS+2))`; do
UNICORN_FD=${UNICORN_FD}${fd},
done
export UNICORN_FD
echo wrapped fds: ${UNICORN_FD}
exec $@
== /etc/systemd/system/uni.service ==
[Unit]
Description=Unicorn Server
[Service]
ExecStart=/srv/uni/wrapper /usr/bin/unicorn -c /srv/uni/unicorn.conf.rb -d
KillSignal=SIGQUIT
KillMode=mixed
== /etc/systemd/system/uni.socket ==
[Unit]
Description=Unicorn Socket
[Socket]
ListenStream=0.0.0.0:9000
ListenStream=0.0.0.0:9001
[Install]
WantedBy=sockets.target
Make sure to activate the systemd units:
chmod +x /srv/uni/wrapper
systemdctl daemon-reload
systemctl enable uni.socket
systemctl start uni.socket
The application sleeps for 5secs before replying.
I run the following commands from 3 different terminals:
$ curl localhost:9000 [blocked for 5sec]
# systemctl stop uni.service [issues sigquit on the running unicorn, killing
the 2nd worker and waiting the 1st to finish]
$ curl localhost:9000 [blocked since there are no more workers to accept right
now]
After the first request is served, unicorn dies and systemd respawns a new
master.
The second request is accepted by the new master (notice the different ppid).
Some notes:
TCP socket options are not applied by unicorn on inherited sockets (TCPSocket
=== sock is false). systemd socket files have support for most options now but
we might want unicorn to `setsockopt` them as well. For example,
'DeferAcceptSec', 'KeepAliveIntervalSec', 'NoDelay' are supported since v216, so
they are not available in jessie (v215).
socket activation is a really interesting setup, but personally I would not run
it with a large application. Clients would have to wait for the new master to
be up and running before a reply is returned, and that could take tenths of
seconds. The USR2 rexec solves that problem since both old and new workers are
accepting on the socket and we can kill the old ones when we are ready. In that
case the PIDFile trick is handy to support zero downtime restarts with no
latency.
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On Fri, 26 Jun 2015 00:34:36 Hleb Valoshka wrote: So if it still supports sysv scripts and native services does not provide additional functions why to add useless native support which breaks enhanced sysv features? Because native support works better than SysV emulation. Because supporting both systemd and SysV is better. Because systemd do not allow to invoke non- standard commands in SysV. interface to services is quite intentional. Frankly it is annoying when init.d script does too much but we can avoid this by offering an additional utility to do soft restart and you can call it from init.d script as well if you wish. WE DO NOT NEED ADDITIONAL UTILITIES. DIXI. Init script is well known interface between user and daemon. No need to break with additional utilities. Additional utility is good because under systemd you can not use custom commands of the SysV script -- disregarding of availability of the native .service file. Postinst do not allow user to invoke soft restart manually. Utility won't break anything and you can call it from custom command of your init script if you insist. Shell script could be common and independent from init system. It should be easier to test and debug as well... I don't think that unicornctl upgrade is more simple that invoke-rc.d unicorn upgrade. It is not more complex either but more flexible. You have to get over it. It was debated enough and Technical Committee has decided. You still can use init.d if you wish, can you? I don't want somebody to make sysv useless, cutting away features of init scripts As I've said, I'm not against custom command of the SysV init script. However utility allows to avoid duplication of effort (i.e. implement soft restart only once) as well as it allows systemd users to benefit from soft restart. just because redhat's crapd does not supports them (but supports a lot of useless crap). Please cut your negativity and remain polite and professional. I suppose that you are from RF, according to your surname. It doesn't matter anyway. Please refrain from making assumptions. It shouldn't matter where I'm from (I'm from Australia FWIW). I find it disturbing that you are trying to make connections between my origin and my views. In any case drawing any conclusions from surname--origin--beliefs is far fetched and unnecessary. What is your point anyway? SW can't be evil and I didn't call crapd as such. Only people can be, and those who made crapd default in Debian are definitely made evil thing to Debian and Free Software community. Decision was well debated and thought. IMHO so far there were more good than harm. There is no point arguing about it here. Please be polite and professional. Let's not go there. As maintainer you can't care only about those users who happened to be using Unicorn on your favourite init system. That kind hostility is harmful. Please do not discriminate. Your last words sounds like manipulation. No manipulation intended. -- Cheers, Dmitry Smirnov. --- The more false we destroy the more room there will be for the true. -- Robert G. Ingersoll, 1902 signature.asc Description: This is a digitally signed message part. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On 6/25/15, Dmitry Smirnov only...@debian.org wrote:
Not sure... I know systemd can invoke init.d script but only standard
{start|
stop|restart|status} functions and only with the absence of native .service
file.
So if it still supports sysv scripts and native services does not
provide additional functions why to add useless native support which
breaks enhanced sysv features?
interface to services is quite intentional. Frankly it is annoying when
init.d
script does too much but we can avoid this by offering an additional utility
to do soft restart and you can call it from init.d script as well if you
wish.
WE DO NOT NEED ADDITIONAL UTILITIES. DIXI. Init script is well known
interface between user and daemon. No need to break with additional
utilities.
Shell script could be common and independent from init system. It should be
easier to test and debug as well...
I don't think that unicornctl upgrade is more simple that invoke-rc.d
unicorn upgrade.
You have to get over it. It was debated enough and Technical Committee has
decided. You still can use init.d if you wish, can you?
I don't want somebody to make sysv useless, cutting away features of
init scripts just because redhat's crapd does not supports them (but
supports a lot of useless crap).
, it quite similar to the last
marriage in Chechnya (old police bastard and young girl), so I don't
want to support this.
I do not know this story but from the way you've described it I'm sure that
I suppose that you are from RF, according to your surname. It doesn't
matter anyway.
your analogy is very inaccurate and unfair. I'm worried about such attitude
of
yours. There are things we may not like. For instance I do not like GNOME
(so
naturally I do not use it) but I don't need to demonise it (e.g. declare it
Evil etc.) to justify my preference.
SW can't be evil and I didn't call crapd as such. Only people can be,
and those who made crapd default in Debian are definitely made evil
thing to Debian and Free Software community.
Let's not go there. As maintainer you can't care only about those users who
happened to be using Unicorn on your favourite init system. That kind
hostility is harmful. Please do not discriminate.
Your last words sounds like manipulation.
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Hi Christos, On Thu, 25 Jun 2015 11:50:57 Christos Trochalakis wrote: I have recently migrated our main ruby application to systemd implementing zero downtime upgrades. [...] Thank you for a very useful hints. I'll update .service file as soon as I try soft restart under systemd. On more minor note regarding the proposed patch: You can safely use `update-rc.d` in maintainer scripts, no need to use $(which update-rc.d). Thanks. I wasn't sure about that and so far I merely fixed the check which is probably redundant... -- All the best, Dmitry Smirnov. --- Platitude: an idea (a) that is admitted to be true by everyone, and (b) that is not true. -- H. L. Mencken signature.asc Description: This is a digitally signed message part. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Hi Eric, On Wed, 24 Jun 2015 23:26:49 Eric Wong wrote: Dmitry: unicorn upstream here, is there anything in unicorn itself can do to make systemd integration easier? Thank you very much for keeping an eye on us and for all your help. I'm not sure if we need anything special -- let me experiment more with soft restart and maybe I'll come back to you with questions. :) I haven't had much interest or time to dig into systemd, but I'm willing to put some effort into making unicorn work more smoothly with systemd (and any other Free Software process managers/init systems). Great. I'm entertaining idea of modifying init scripts to make soft restart the default behaviour. Is there are any risks or drawbacks of always attempting soft restart instead of normal restart? What do you think? -- Cheers, Dmitry Smirnov. --- It is no use saying, 'We are doing our best.' You have got to succeed in doing what is necessary. -- Winston Churchill signature.asc Description: This is a digitally signed message part. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
Adding unicorn-pub...@bogomips.org to Cc: Those of you who haven't followed along on the unic...@packages.debian.org list can catch up here: http://lists.alioth.debian.org/pipermail/pkg-ruby-extras-maintainers/2015-June/024539.html Dmitry: unicorn upstream here, is there anything in unicorn itself can do to make systemd integration easier? I haven't had much interest or time to dig into systemd, but I'm willing to put some effort into making unicorn work more smoothly with systemd (and any other Free Software process managers/init systems). So far unicorn.git has documented the UNICORN_FD environment variable in the manpage in commit 548e1e67d314f6ebd17df37ece0ee20632462f6f [1] and support SIGWINCH (kill children) in commit a6077391bb62d0b13016084b0eea36b987afe8f0 [2] I think that should be sufficient, but maybe there's more. Thanks. [1] http://bogomips.org/unicorn.git/patch/?id=548e1e67 [2] http://bogomips.org/unicorn.git/patch/?id=a6077391 ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On Wed, 24 Jun 2015 12:02:46 Hleb Valoshka wrote:
On 6/23/15, Dmitry Smirnov only...@debian.org wrote:
It seems that --update-rcd-params=disable (or remove) is the solution.
I tried that but it does not work and produces error on installation of
the package...
I'll test it too when I have more spare time, and if it's broken then
I'll fill bug against dh.
I any case code that I've committed successfully disables service on first
install. We just need to implement soft restart. I'll try to draft an utility
as soon as I'll manage to allocate some time for it.
And of course I don't want to lose features just because of systemd.
Right but at least we need to make sure that custom restart code is
systemd-aware...
Does debian's systemd package has invoke-rc.d? OpenRC has.
Not sure... I know systemd can invoke init.d script but only standard {start|
stop|restart|status} functions and only with the absence of native .service
file.
It's usual way to add custom actions to init script besides usual
start, stop, reload, restart. Because init script is a interface
between user (sysadmin) and daemon. I don't wonder that lennarts don't
know this. For example, apache's init even can start/stop cache
cleaner.
It is not that usual to add custom actions. I've seen many init.d s scripts
and only few had such things. I think systemd was designed using different
philosophy. I don't know the history but it could be that limited systemd
interface to services is quite intentional. Frankly it is annoying when init.d
script does too much but we can avoid this by offering an additional utility
to do soft restart and you can call it from init.d script as well if you wish.
Perhaps it will be best to introduce a new binary that user could run on
upgrade to do zero downtime restart (we can run this utility from
postinst).
What do you think?
I thought about shell library sourced by both. But then decided to
move to init script may be first idea was better.
Shell script could be common and independent from init system. It should be
easier to test and debug as well...
It is our problem because like it or not Debian is now married to
systemd.
I dislike the way it was married
You have to get over it. It was debated enough and Technical Committee has
decided. You still can use init.d if you wish, can you?
, it quite similar to the last
marriage in Chechnya (old police bastard and young girl), so I don't
want to support this.
I do not know this story but from the way you've described it I'm sure that
your analogy is very inaccurate and unfair. I'm worried about such attitude of
yours. There are things we may not like. For instance I do not like GNOME (so
naturally I do not use it) but I don't need to demonise it (e.g. declare it
Evil etc.) to justify my preference.
Remember that it is not users' fault to use default init system so let's
try to offer a solution for them.
User can always apt-get install better init.
Let's not go there. As maintainer you can't care only about those users who
happened to be using Unicorn on your favourite init system. That kind
hostility is harmful. Please do not discriminate.
--
Cheers,
Dmitry Smirnov.
---
Democracy is a pathetic belief in the collective wisdom of individual
ignorance.
-- H. L. Mencken
signature.asc
Description: This is a digitally signed message part.
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On 6/22/15, Dmitry Smirnov only...@debian.org wrote:
Perhaps it might be better to make sure that daemon is not enabled after
first install of the package?
It sounds reasonably.
I realised that dh_installinit --no-start still enables automatic start
of
daemon so I've added workaround to postinst to disable daemon on first
install.
It seems that --update-rcd-params=disable (or remove) is the solution.
Also I dropped old service start/stop/restart code from {postinst|prerm} in
favour of debhelper scripts. I believe custom code was redundant and not
aware
of systemd so IMHO it is the best to remove it. I understand this approach
might be a bit intrusive so please revert if you prefer to keep old
behaviour.
I don't want to lose transparent upgrade feature and it's not possible
with default code from dh_installinit.
And of course I don't want to lose features just because of systemd.
I think that debhelper is already handles service restart gracefully so
transparent upgrade code might be just redundant.
No. Transparent upgrades are supported only by nginx and unicorn
afaik, so I don't think that dh is aware of it. It can just restart
service.
I believe it should not be in init.d script because init.d is better stay
simple and don't do unexpected things.
No. How will you behave after security upgrade of ruby or redmine?
Restart unicorn dropping connections?
Also I'd like to keep SysV and Systemd
scripts close to each other so service restart would behave more or less
similar disregarding of the init system.
If systemd does not allows custom actions it's systemd's problem.
* Should-Start: mysql looks as dirty hack but it looks reasonable,
It is not a dirty hack. It is a legitimate SysV facility to advise on
desirable order of start. See more in
I mean knowledge of existence of particular db.
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On Mon, 22 Jun 2015 19:45:23 Hleb Valoshka wrote: It seems that --update-rcd-params=disable (or remove) is the solution. I tried that but it does not work and produces error on installation of the package... I don't want to lose transparent upgrade feature and it's not possible with default code from dh_installinit. Maybe we can recommend needrestart package (if it can take care of restart for us)? O, I've just learned that transparent upgrade should send a custom signal to unicorn process... And of course I don't want to lose features just because of systemd. Right but at least we need to make sure that custom restart code is systemd- aware... I think that debhelper is already handles service restart gracefully so transparent upgrade code might be just redundant. No. Transparent upgrades are supported only by nginx and unicorn afaik, so I don't think that dh is aware of it. It can just restart service. I'm not sure I understand transparent upgrade. Is it zero downtime restart achieved by sending SIGUSR2 to the unicorn process and then sending QUIT signal again after delay? It should be safe to do so from postinst. Looks like we need to follow Procedure to replace a running unicorn executable described here: http://unicorn.bogomips.org/SIGNALS.html Besides old postinst was starting unicorn unconditionally and I'd like to avoid that. I believe it should not be in init.d script because init.d is better stay simple and don't do unexpected things. No. How will you behave after security upgrade of ruby or redmine? Restart unicorn dropping connections? Suppose you modified init.d script -- you will have to either create a new interface (e.g. non-standard zero-restart command) or to modify behaviour of restart which make the latter do unexpected things. Either way if you take care of init script it will not work for systemd users (and they are the majority). Perhaps it will be best to introduce a new binary that user could run on upgrade to do zero downtime restart (we can run this utility from postinst). What do you think? Also I'd like to keep SysV and Systemd scripts close to each other so service restart would behave more or less similar disregarding of the init system. If systemd does not allows custom actions it's systemd's problem. It is our problem because like it or not Debian is now married to systemd. Remember that it is not users' fault to use default init system so let's try to offer a solution for them. * Should-Start: mysql looks as dirty hack but it looks reasonable, It is not a dirty hack. It is a legitimate SysV facility to advise on desirable order of start. See more in I mean knowledge of existence of particular db. Technically it is not a knowledge but an advisory info on desirable order of start to init system... Looks nice to me. :) -- All the best, Dmitry Smirnov. --- You have to start with the truth. The truth is the only way that we can get anywhere. Because any decision-making that is based upon lies or ignorance can't lead to a good conclusion. -- Julian Assange, 2010 signature.asc Description: This is a digitally signed message part. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On 6/19/15, Dmitry Smirnov only...@debian.org wrote: I took liberty to commit native systemd service to unicorn repository. Thanks! I hope it works properly (I can't test as I'm not a systemd user). But I have several comments regarding your changes: * You have removed CONFIGURED variable, that's not good because unicorn may be installed as a dependency for Rainbows (or any other unicorn-based server) and in this case it sh'ld not be started (of course user can disable it with update-rc.d but it's not a good idea). So I'll return it back. * I want to move transparent upgrade code from post-install hook to init-script because it may be useful not only on upgrade of unicorn package but with ruby and application upgrades as well. I hope invoke-rc.d unicorn upgrade || invoke-rc.d unicorn restart will be enough in case of systemd too. * Should-Start: mysql looks as dirty hack but it looks reasonable, I'll rather add postgres there too. Your other changes look reasonable, thanks. I shall be happy to answer all your questions and hopefully help little more with Unicorn maintenance, if time allows. Unicorn does not require a lot of maintenance efforts, but all that systemd stuff does. Please let me know what you think. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
Re: [DRE-maint] unicorn: native systemd service
On Sun, 21 Jun 2015 18:19:44 Hleb Valoshka wrote:
I hope it works properly (I can't test as I'm not a systemd
user).
I've been using unicorn.service for some weeks and it works for me although my
local .service file also creates directory for .sock file and set permissions
on it...
But I have several comments regarding your changes:
* You have removed CONFIGURED variable, that's not good because
unicorn may be installed as a dependency for Rainbows (or any other
unicorn-based server) and in this case it sh'ld not be started (of
course user can disable it with update-rc.d but it's not a good idea).
As far as I understand current situation, .default file should not control
activation of daemon because it is redundant (duplicates SysV functionality)
and not standard among system daemons. Now when systemd is introduced,
.default file have more potential to interfere and increase complexity.
I believe it will be better to manually activate service with update-rc.d
unicorn enable when needed. This is also beneficial as it is makes init
systems behaviour closer to each other.
So I'll return it back.
Perhaps it might be better to make sure that daemon is not enabled after first
install of the package?
I realised that dh_installinit --no-start still enables automatic start of
daemon so I've added workaround to postinst to disable daemon on first
install. On systemd side automatic enable of service is already prevented by
dh_systemd_enable --no-enable override.
Also I dropped old service start/stop/restart code from {postinst|prerm} in
favour of debhelper scripts. I believe custom code was redundant and not aware
of systemd so IMHO it is the best to remove it. I understand this approach
might be a bit intrusive so please revert if you prefer to keep old behaviour.
I shall be happy to discuss more and answer your questions.
* I want to move transparent upgrade code from post-install hook to
init-script because it may be useful not only on upgrade of unicorn
package but with ruby and application upgrades as well. I hope
invoke-rc.d unicorn upgrade || invoke-rc.d unicorn restart will be
enough in case of systemd too.
I think that debhelper is already handles service restart gracefully so
transparent upgrade code might be just redundant.
I believe it should not be in init.d script because init.d is better stay
simple and don't do unexpected things. Also I'd like to keep SysV and Systemd
scripts close to each other so service restart would behave more or less
similar disregarding of the init system.
* Should-Start: mysql looks as dirty hack but it looks reasonable,
It is not a dirty hack. It is a legitimate SysV facility to advise on
desirable order of start. See more in
https://wiki.debian.org/LSBInitScripts
I think it makes sense because many Ruby apps depend on database while
database do not depend on Ruby services. I've added this because I've seen
failures when Unicorn start my Redmine before MySQL and Redmine@unicor fails
to start on reboot of the server because database is not ready yet.
I'll rather add postgres there too.
Good idea. :)
Your other changes look reasonable, thanks.
Thank you, I'm glad to read that. :)
I shall be happy to answer all your questions and hopefully help little
more with Unicorn maintenance, if time allows.
Unicorn does not require a lot of maintenance efforts, but all that
systemd stuff does.
Not really. It's just introduction of service is a bit tricky due to clashes
with .default file but I think I've managed to solve that. We might also need
a NEWS file to notify about changes...
--
All the best,
Dmitry Smirnov.
---
What can be asserted without proof can be dismissed without proof.
-- Christopher Hitchens, 2004
signature.asc
Description: This is a digitally signed message part.
___
Pkg-ruby-extras-maintainers mailing list
Pkg-ruby-extras-maintainers@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
[DRE-maint] unicorn: native systemd service
Dear Hleb, Thank you for maintaining unicorn in Debian. I took liberty to commit native systemd service to unicorn repository. I shall be happy to answer all your questions and hopefully help little more with Unicorn maintenance, if time allows. Please let me know what you think. -- Regards, Dmitry Smirnov. --- It is impossible to imagine Goethe or Beethoven being good at billiards or golf. -- H. L. Mencken signature.asc Description: This is a digitally signed message part. ___ Pkg-ruby-extras-maintainers mailing list Pkg-ruby-extras-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-ruby-extras-maintainers
