Re: [systemd-devel] SysVInit service migration to systemd
On Jun 29, 2015 16:58, Lesley Kimmel ljkimme...@hotmail.com wrote: Jonathan; Thanks for the background and information. Since you clearly seem to have a grasp of systemd please humour me with a few more questions (some of them slightly ignorant): a) Why are PID bad? b) Why are lock files bad? c) If a/b are so bad why did they persist for so many years in SysVInit? init itself *doesn't need* either, though it only supports non-forking services, and in practice the only service it ever manages is agetty. (And sometimes xdm.) But the init.d scripts, they're just she'll scripts which aren't capable of any persistent monitoring. They cannot rely on pid1 to do any monitoring. So pidfiles and lockfiles are the only (or at least the simplest) way for these scripts keep state from init.d/foo start across to init.d/foo stop. Personally I would say they persisted in sysvinit because they pretty much /define/ the sysvinit design (i.e. pid1 that does almost nothing, and short-lived external scripts). Though there's also openrc, which also runs on top of sysv pid1, but that's really all I know about it. I wonder how it tracks processes. I heard it uses cgroups? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] SysVInit service migration to systemd
В Mon, 29 Jun 2015 08:58:11 -0500 Lesley Kimmel ljkimme...@hotmail.com пишет: Jonathan; Thanks for the background and information. Since you clearly seem to have a grasp of systemd please humour me with a few more questions (some of them slightly ignorant): a) Why are PID bad? PID files as the primary way to track running service are too unreliable - it is easy for a service to die without removing PID file. PID file as a mean to track service startup the way systemd does it is IMHO just fine. Actually it is the only way for a service to announce its readiness without adding explicit systemd support (Type=notify). b) Why are lock files bad? c) If a/b are so bad why did they persist for so many years in SysVInit? d) Generically, how would you prescribe to use systemd to start Java processes (for Java application servers) that are typically started from a set of relatively complex scripts that are used to set up the environment before launching the Java process? It seems that you are advocating to call, as directly as possible, the target service/daemon. However, some things don't seem so straight-forward. Thanks again! Lesley Kimmel, RHCE Unix/Linux Systems Engineer Date: Sun, 28 Jun 2015 14:29:40 +0100 From: j.deboynepollard-newsgro...@ntlworld.com To: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] SysVInit service migration to systemd Lesley Kimmel: I've been working with RHEL5/6 for the past several years and have developed many init scripts/services which generally use lock files and PID files to allow for tracking of the service status. We are moving to RHEL7 (systemd) in the near future and I am looking for instruction or tutorials on how to effectively migrate these scripts to work with systemd. [...] It looks like I may be able to use the 'forking' type with the 'pidfile' parameter to somewhat mimic what the scripts to today. You don't do such migration. You understand how your daemons are started, and you write service (and other) units to describe that. You do not start with the assumption that you migrate straight from one to the other, particularly if your existing mechanisms involve the dangerous and rickety things (e.g. PID files) that proper service management is designed to avoid in the first place, or things such as subsystem lock files which proper service management has no need of by its very nature. Type=forking specifies a quite specific readiness protocol that your daemon has to enact, lest it be regarded as failed. It's isn't a catch-all for anything that might fork in any possible way. And service managers know, by dint of simply remembering, what processes they started and whether they've already started them. I've been collecting case studies of people who have migrated to systemd exceedingly badly, and constructed some quite horrendous systems, because they've done so without either consideration of, or understanding of, how their sytems actually work. Here's one candidate that I have yet to add, because I'm hoping that now they've been told that they are going wrong they will correct themselves, whose errors are good to learn from. There's a computer game called ARK: Survival Evolved. It's daemon is a program named ShooterGameServer. To run this program continually as a daemon, someone named Federico Zivolo and a couple of other unidentified people came up with the somewhat bizarre idea of running it under screen, and using screen's commands to send its pseudo-terminal an interrupt character, in order to send a SIGINT to the daemon when it came time to shut it down. They wrapped this up in a System 5 rc script, taking the conventional start and stop verbs, named arkmanager. Its prevent-multiple-instances system was not lock files, but grepping the process table. Wrapped around this they put another System 5 rc script, named arkdaemon, which also took the conventional start and stop verbs, and which treated the wrapper arkmanager script as the daemon, recording the process ID of the shell interpreter for the arkmanager script in a PID file, as if it were the actual daemon's process ID. It also did various other bad things that proper service managers eliminate, including grepping the process table (again), abusing su to drop privileges, using arbitrary 5 second sleeps to make the timing almost work, and (sic) hardwired ECMA-48 SGR sequences to change the colour of output that isn't going to a terminal in the first place. Then they wrote a systemd service unit file, arkdeamon.service (sic), that did this: ExecStart=/etc/init.d/arkdaemon start ExecStop=/etc/init.d/arkdaemon stop Type=forking A couple of days ago, I pointed out the errors of even starting down this route to them, and wrote a systemd unit file for them
Re: [systemd-devel] SysVInit service migration to systemd
Am 29.06.2015 um 15:58 schrieb Lesley Kimmel: Jonathan; Thanks for the background and information. Since you clearly seem to have a grasp of systemd please humour me with a few more questions (some of them slightly ignorant): a) Why are PID bad? what are they good for when the supervisor knows the PID to monitor? b) Why are lock files bad? what are they good for when the supervisor knows the state of each service c) If a/b are so bad why did they persist for so many years in SysVInit? because SysVInit did have no other way to know the PID or if a service is running and things like lockfiles are far away from safe when the application crashs and don't remove them - often enough that a restart of services failed because of that d) Generically, how would you prescribe to use systemd to start Java processes (for Java application servers) that are typically started from a set of relatively complex scripts that are used to set up the environment before launching the Java process? It seems that you are advocating to call, as directly as possible, the target service/daemon. However, some things don't seem so straight-forward. most complexity in that scripts is because the way SysVInit worked a daemon should read each configuration files and not need more than the argument of the config file as param signature.asc Description: OpenPGP digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] SysVInit service migration to systemd
On Mon, Jun 29, 2015 at 10:58 AM, Lesley Kimmel ljkimme...@hotmail.com wrote: Jonathan; Thanks for the background and information. Since you clearly seem to have a grasp of systemd please humour me with a few more questions (some of them slightly ignorant): a) Why are PID bad? Because they pretend to work but they really don't. This is because only a tiny portion of software implements pid file creation correctly, this is in part due to the lack of a FREEBSD-like pidfile_*() interface that at least tries to be correct. b) Why are lock files bad? Mostly because at least till the *very recent* advent of File-private POSIX locks (un-POSIX locks) the OS facilities were terrible. c) If a/b are so bad why did they persist for so many years in SysVInit? Because sysvinit is unable to track processes, in that case you need at least to know what is the PID of the deamon, in order to be able to kill it: ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] SysVInit service migration to systemd
Lesley Kimmel: I've been working with RHEL5/6 for the past several years and have developed many init scripts/services which generally use lock files and PID files to allow for tracking of the service status. We are moving to RHEL7 (systemd) in the near future and I am looking for instruction or tutorials on how to effectively migrate these scripts to work with systemd. [...] It looks like I may be able to use the 'forking' type with the 'pidfile' parameter to somewhat mimic what the scripts to today. You don't do such migration. You understand how your daemons are started, and you write service (and other) units to describe that. You do not start with the assumption that you migrate straight from one to the other, particularly if your existing mechanisms involve the dangerous and rickety things (e.g. PID files) that proper service management is designed to avoid in the first place, or things such as subsystem lock files which proper service management has no need of by its very nature. Type=forking specifies a quite specific readiness protocol that your daemon has to enact, lest it be regarded as failed. It's isn't a catch-all for anything that might fork in any possible way. And service managers know, by dint of simply remembering, what processes they started and whether they've already started them. I've been collecting case studies of people who have migrated to systemd exceedingly badly, and constructed some quite horrendous systems, because they've done so without either consideration of, or understanding of, how their sytems actually work. Here's one candidate that I have yet to add, because I'm hoping that now they've been told that they are going wrong they will correct themselves, whose errors are good to learn from. There's a computer game called ARK: Survival Evolved. It's daemon is a program named ShooterGameServer. To run this program continually as a daemon, someone named Federico Zivolo and a couple of other unidentified people came up with the somewhat bizarre idea of running it under screen, and using screen's commands to send its pseudo-terminal an interrupt character, in order to send a SIGINT to the daemon when it came time to shut it down. They wrapped this up in a System 5 rc script, taking the conventional start and stop verbs, named arkmanager. Its prevent-multiple-instances system was not lock files, but grepping the process table. Wrapped around this they put another System 5 rc script, named arkdaemon, which also took the conventional start and stop verbs, and which treated the wrapper arkmanager script as the daemon, recording the process ID of the shell interpreter for the arkmanager script in a PID file, as if it were the actual daemon's process ID. It also did various other bad things that proper service managers eliminate, including grepping the process table (again), abusing su to drop privileges, using arbitrary 5 second sleeps to make the timing almost work, and (sic) hardwired ECMA-48 SGR sequences to change the colour of output that isn't going to a terminal in the first place. Then they wrote a systemd service unit file, arkdeamon.service (sic), that did this: ExecStart=/etc/init.d/arkdaemon start ExecStop=/etc/init.d/arkdaemon stop Type=forking A couple of days ago, I pointed out the errors of even starting down this route to them, and wrote a systemd unit file for them that actually starts the daemon directly from systemd; no PID files, abuses of su, racy grepping of the process table, abritrary sleeps, or incorrect forking readiness protocol necessary. Set the account with User= ; set the environment with Environment= ; and run the program with ExecStart=/home/steam/ARK/ShooterGame/Binaries/Linux/ShooterGameServer plus some options. Other stuff, like software upgrades, one then builds on top of the service management layer, not beneath it. I also pointed out to them that I had been beaten in this regard by some 4 days by the (again unfortunately unidentified) people who wrote http://ark.gamepedia.com/Dedicated_Server_Setup#Automatic_Startup , whose service unit was pretty much the same as the one that I had independently written (as one would expect, indeed). If you understand how your daemon is run in the first place, then you end up with what I and the gamepedia people did. If you just think that you wrap what you already have, using forking to imperfectly paper over the cracks, you oftentimes end up with an utterly horrendous mess. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] SysVInit service migration to systemd
Lesley Kimmel píše v Pá 26. 06. 2015 v 08:15 -0500: Hi all; I've been working with RHEL5/6 for the past several years and have developed many init scripts/services which generally use lock files and PID files to allow for tracking of the service status. We are moving to RHEL7 (systemd) in the near future and I am looking for instruction or tutorials on how to effectively migrate these scripts to work with systemd. http://0pointer.de/blog/projects/systemd-for-admins-3.html I found https://wiki.archlinux.org/index.php/Systemd#Service_types which seems somewhat promising but it is fairly high-level. It looks like I may be able to use the 'forking' type with the 'pidfile' parameter to somewhat mimic what the scripts to today. However, I have a couple of questions: -How does systemd track whether it should be stopping a service at shutdown (analogous to the /var/lock/subsys files in SysVInit)? Systemd tracks the services using cgroups, so it knows that the service is running and needs to be stopped. -Are there merits to using the notify or dbus types? If so does anyone know of a tutorial I could use to get to that point? (FYI, I'm not a developer but I learn complicated things quickly). -If the current service logs to a custom, dedicated log is there a way to get systemd to provide the same type of output that it does for the built-in system services or must I make some modifications to integrate with journald? Look at the Administrators Blog Series and Developers Series https://wiki.freedesktop.org/www/Software/systemd/ Thanks ahead of time, Lesley Kimmel, RHCE Linux/Unix Systems Engineer ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel Regards Lukas ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] SysVInit service migration to systemd
В Fri, 26 Jun 2015 10:02:41 -0500 Lesley Kimmel ljkimme...@hotmail.com пишет: Thanks for the information. I've seen that blog before. Unfortunately, it only describes a starting a service that already has a good level of integration with some of the underlying infrastructure of systemd (e.g. dbus). Let me be a little more specific about what I'm trying to accomplish and see if anyone has any thoughts on how systemd could help (or impede me). I have a collection of servers hosting many processes such as Apache HTTPD, a database, and Java application servers. Using init these servers would: a) need to be started in a specific order and b) take a long time to start. To improve both of these scenarios I created a Python service which took an XML configuration file describing the dependencies of the various components. The Python service is started by init and forks so as to not stop the boot process. The forked process then does some basic dependency checking (including remote tests, mostly telnet or pings) before starting local services using init scripts that are not configured to be started by init. That sounds exactly like what pacemaker does. I'm wondering, with systemd, if this Python control daemon would be required at all. Does systemd have the ability to check the status of remote servers? No. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] SysVInit service migration to systemd
On Fri, Jun 26, 2015 at 4:15 PM, Lesley Kimmel ljkimme...@hotmail.com wrote: Hi all; I've been working with RHEL5/6 for the past several years and have developed many init scripts/services which generally use lock files and PID files to allow for tracking of the service status. We are moving to RHEL7 (systemd) in the near future and I am looking for instruction or tutorials on how to effectively migrate these scripts to work with systemd. I found https://wiki.archlinux.org/index.php/Systemd#Service_types which seems somewhat promising but it is fairly high-level. It looks like I may be able to use the 'forking' type with the 'pidfile' parameter to somewhat mimic what the scripts to today. However, I have a couple of questions: Usually systemd detects the main process even without PIDFile, using cgroups to keep track. -How does systemd track whether it should be stopping a service at shutdown (analogous to the /var/lock/subsys files in SysVInit)? All service control requests go through the always-running init process, so it can just keep track of service status internally without having to write it out. [This also means services run in a clean environment and can't ask for interactive input when starting. Upstart and even /etc/inittab also work the same way.] Systemd also monitors whether the service processes are actually running – by putting each into a separate cgroup – so it generally avoids issues such as stale pidfiles. -Are there merits to using the notify or dbus types? If so does anyone know of a tutorial I could use to get to that point? (FYI, I'm not a developer but I learn complicated things quickly). Similar to Type=forking, they both let systemd know whether the service is ready or still initializing (which can't be done with the default Type=simple). Type=dbus might be best for creating DBus-activatable services compatible with regular dbus-daemon, since it has mostly the same expectations as dbus-daemon: the service doesn't fork, and is considered ready as soon as it finally claims the configured BusName. Type=notify is somewhat more lightweight – the service just needs to send READY=1 over a Unix socket. (The service can also show some additional information in `systemctl status` – e.g. lldpd could send STATUS=No neighbors). libsystemd has sd_notify(3) http://www.freedesktop.org/software/systemd/man/sd_notify.html for this, though it could be done using regular socket API. I suppose the same notify socket could be exposed to all other service types. I don't know why it isn't yet. -If the current service logs to a custom, dedicated log is there a way to get systemd to provide the same type of output that it does for the built-in system services or must I make some modifications to integrate with journald? `systemctl status` only reads from the journal. The journal, however, also receives messages sent using the traditional syslog API or written to the kernel log – in addition to its native sd_journal_send(3) http://www.freedesktop.org/software/systemd/man/sd_journal_send.html – so that already covers most services. You could also use rsyslogd to feed messages from custom text logs into the journal, using its omjournal module. -- Mantas Mikulėnas graw...@gmail.com ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] SysVInit service migration to systemd
Hi all; I've been working with RHEL5/6 for the past several years and have developed many init scripts/services which generally use lock files and PID files to allow for tracking of the service status. We are moving to RHEL7 (systemd) in the near future and I am looking for instruction or tutorials on how to effectively migrate these scripts to work with systemd. I found https://wiki.archlinux.org/index.php/Systemd#Service_types which seems somewhat promising but it is fairly high-level. It looks like I may be able to use the 'forking' type with the 'pidfile' parameter to somewhat mimic what the scripts to today. However, I have a couple of questions: -How does systemd track whether it should be stopping a service at shutdown (analogous to the /var/lock/subsys files in SysVInit)? -Are there merits to using the notify or dbus types? If so does anyone know of a tutorial I could use to get to that point? (FYI, I'm not a developer but I learn complicated things quickly). -If the current service logs to a custom, dedicated log is there a way to get systemd to provide the same type of output that it does for the built-in system services or must I make some modifications to integrate with journald? Thanks ahead of time, Lesley Kimmel, RHCE Linux/Unix Systems Engineer ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
