Re: [systemd-devel] Need advice on daemon's architecture
On Sun, 03.11.13 10:40, Peter Lemenkov (lemen...@gmail.com) wrote: Hello All! I'm working on a system service which uses systemd intensively. Right now it's socket-activated, with main service of type simple. I recently added support for querying and publishing some internals via D-Bus, so it has a D-Bus name now. Does it add anything if I change type of a main service to dbus thus allowing systemd to know for sure if my service is fully initialized? Type=notify or Type=dbus are usually the best choices and is what I'd recommend. The former requires you to use sd_notify(), the latter requires you to make your service a bus service. Type=forking is the nastiest choice. Type=simple is an OK choice too, but doesn't pass as much state information to systemd as the other options. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
On Sun, 03.11.13 13:42, Hoyer, Marko (ADITG/SW2) (mho...@de.adit-jv.com) wrote: If you are using systemd intensively, then you may want to use Type=notify. With type=dbus, systemd will consider things ready when you take the name on the bus, but this might not actually be the last thing you do to initialise your daemon (although if this is the only interface for clients this is not a problem!). You still might want to use sd_notify() instead. This can also pass through some internal performance info to systemd which will show up on systemctl status output which is kinda nice. Col Isn't the classical Linux way an option too? Well, it is, but it is hard to get right and a lot of redundant code involved. - the daemon does its initialization with the calling thread - once it is done with the initialization, it forks off a process that goes on with the daemons work (the main loop probably) No, this is wrong. You really shouldn't do initialization in one process and then run the daemon from a forked off second one. A lot of (library) code is not happy with being initialized in one process and being used in another forked off one. For example, any code involving threads is generally allergic to this, since background threads cease to exist in the forked off child. This is nearly impossible to work around [1]. Or code involving file locks or even a lot of socket code doesn't like it either. In general you should not assume that any library can handle it, except for those cases where the library author explicitly tells you that it is safe. Actually, all systemd libraries (like libsystemd-journal or libsystem-bus) and a couple of my other libraries (like libcanberra) explicitly check for the PID changing and refuse operation in such cases, simply because the effects of fork()ing are so nasty. Or to be more explicit: if you initialize a sd_journal or sd_bus object in one process and then try to execute any operation on it in a forked off child process you will get -ECHILD returned which is how we indicate this misuage. So, what is the right way to do this? Fork off early, do the initialization in the child, and signal the parent that you are done via a pipe, so that the parent exits only after the child is done. This is explained in daemon(7). Or even better: don't bother at all, write your services without forking, and use Type=dbus or Type=notify instead. Lennart [1] Yeah, and don't tell me pthread_atfork() is the solution for this. It's not. It makes things even worse, since there's no defined execution order defined for it. If you claim pthread_atfork() was a solution and not a problem in itself, then you obviously have never used it, or only in trivial programs. -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
On 04/11/13 14:42, Lennart Poettering wrote: A lot of (library) code is not happy with being initialized in one process and being used in another forked off one. For what it's worth, fork(3posix) also notes this: * A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe opera‐ tions until such time as one of the exec functions is called. See Linux signal(7) for a list of async-signal-safe operations: it's not as long a list as you might hope, and mostly contains syscalls. In particular, malloc() is not on the list, which rules out a lot of library code... S ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
On Mon, 2013-11-04 at 14:57 +, Simon McVittie wrote: See Linux signal(7) for a list of async-signal-safe operations: it's not as long a list as you might hope, and mostly contains syscalls. In particular, malloc() is not on the list, which rules out a lot of library code... Given however that systemd is Linux/glibc-equivalent only, you can rely on malloc() working after fork() if you're writing a daemon which only runs on systemd systems. See: https://bugzilla.gnome.org/show_bug.cgi?id=659326#c24 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
-Original Message- From: Lennart Poettering [mailto:lenn...@poettering.net] Sent: Monday, November 04, 2013 3:42 PM To: Hoyer, Marko (ADITG/SW2) Cc: Colin Guthrie; Peter Lemenkov; systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Need advice on daemon's architecture On Sun, 03.11.13 13:42, Hoyer, Marko (ADITG/SW2) (mho...@de.adit-jv.com) wrote: If you are using systemd intensively, then you may want to use Type=notify. With type=dbus, systemd will consider things ready when you take the name on the bus, but this might not actually be the last thing you do to initialise your daemon (although if this is the only interface for clients this is not a problem!). You still might want to use sd_notify() instead. This can also pass through some internal performance info to systemd which will show up on systemctl status output which is kinda nice. Col Isn't the classical Linux way an option too? Well, it is, but it is hard to get right and a lot of redundant code involved. - the daemon does its initialization with the calling thread - once it is done with the initialization, it forks off a process that goes on with the daemons work (the main loop probably) No, this is wrong. You really shouldn't do initialization in one process and then run the daemon from a forked off second one. A lot of (library) code is not happy with being initialized in one process and being used in another forked off one. For example, any code involving threads is generally allergic to this, since background threads cease to exist in the forked off child. This is nearly impossible to work around [1]. Or code involving file locks or even a lot of socket code doesn't like it either. In general you should not assume that any library can handle it, except for those cases where the library author explicitly tells you that it is safe. Actually, all systemd libraries (like libsystemd-journal or libsystem-bus) and a couple of my other libraries (like libcanberra) explicitly check for the PID changing and refuse operation in such cases, simply because the effects of fork()ing are so nasty. Or to be more explicit: if you initialize a sd_journal or sd_bus object in one process and then try to execute any operation on it in a forked off child process you will get -ECHILD returned which is how we indicate this misuage. So, what is the right way to do this? Fork off early, do the initialization in the child, and signal the parent that you are done via a pipe, so that the parent exits only after the child is done. This is explained in daemon(7). Or even better: don't bother at all, write your services without forking, and use Type=dbus or Type=notify instead. Lennart [1] Yeah, and don't tell me pthread_atfork() is the solution for this. It's not. It makes things even worse, since there's no defined execution order defined for it. If you claim pthread_atfork() was a solution and not a problem in itself, then you obviously have never used it, or only in trivial programs. -- Lennart Poettering, Red Hat Thx for the comprehensive answer. My daemon is quite simple, threads are only used as kind of workers which are started from the daemon process out of the main loop. So I guess: no problem at this end. I'm additionally polling a udevd socket opened via libudev, which actually is initialized from the calling process by now. Additionally, I'm working on the connection to a control tool, which I plan to realize via sockets as well. So I plan to open a listener socket during the initialization. Maybe I'll actually switch to the sd_notify() way until someone appears who is not systemd and wants to use my daemon ... Best regards Marko Hoyer Software Group II (ADITG/SW2) Tel. +49 5121 49 6948 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Need advice on daemon's architecture
Hello All! I'm working on a system service which uses systemd intensively. Right now it's socket-activated, with main service of type simple. I recently added support for querying and publishing some internals via D-Bus, so it has a D-Bus name now. Does it add anything if I change type of a main service to dbus thus allowing systemd to know for sure if my service is fully initialized? -- With best regards, Peter Lemenkov. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
On Sat, Nov 2, 2013 at 11:40 PM, Peter Lemenkov lemen...@gmail.com wrote: Does it add anything if I change type of a main service to dbus thus allowing systemd to know for sure if my service is fully initialized? Yes. Changing to Type=dbus will cause systemd to only consider the service fully started after there's a listener for whatever's specified with BusName= [1]. [1] http://www.freedesktop.org/software/systemd/man/systemd.service.html#Type= ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
'Twas brillig, and Peter Lemenkov at 03/11/13 06:40 did gyre and gimble: Hello All! I'm working on a system service which uses systemd intensively. Right now it's socket-activated, with main service of type simple. I recently added support for querying and publishing some internals via D-Bus, so it has a D-Bus name now. Does it add anything if I change type of a main service to dbus thus allowing systemd to know for sure if my service is fully initialized? If you are using systemd intensively, then you may want to use Type=notify. With type=dbus, systemd will consider things ready when you take the name on the bus, but this might not actually be the last thing you do to initialise your daemon (although if this is the only interface for clients this is not a problem!). You still might want to use sd_notify() instead. This can also pass through some internal performance info to systemd which will show up on systemctl status output which is kinda nice. Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
-Original Message- From: systemd-devel-boun...@lists.freedesktop.org [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Colin Guthrie Sent: Sunday, November 03, 2013 12:54 PM To: Peter Lemenkov; systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Need advice on daemon's architecture 'Twas brillig, and Peter Lemenkov at 03/11/13 06:40 did gyre and gimble: Hello All! I'm working on a system service which uses systemd intensively. Right now it's socket-activated, with main service of type simple. I recently added support for querying and publishing some internals via D-Bus, so it has a D-Bus name now. Does it add anything if I change type of a main service to dbus thus allowing systemd to know for sure if my service is fully initialized? If you are using systemd intensively, then you may want to use Type=notify. With type=dbus, systemd will consider things ready when you take the name on the bus, but this might not actually be the last thing you do to initialise your daemon (although if this is the only interface for clients this is not a problem!). You still might want to use sd_notify() instead. This can also pass through some internal performance info to systemd which will show up on systemctl status output which is kinda nice. Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel Isn't the classical Linux way an option to? - the daemon does its initialization with the calling thread - once it is done with the initialization, it forks off a process that goes on with the daemons work (the main loop probably) - the calling thread returns, which signals systemd that the daemon is up now Type=forking must be defined in the .service to support this architecture. Are there any drawbacks with this solution? I'm just asking because I'm working at the moment on a daemon that is going exactly this way ... Best regards Marko Hoyer Software Group II (ADITG/SW2) Tel. +49 5121 49 6948 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
On Sunday 2013-11-03 14:42, Hoyer, Marko (ADITG/SW2) wrote: Isn't the classical Linux way an option to? - the daemon does its initialization with the calling thread - once it is done with the initialization, it forks off a process that goes on with the daemons work (the main loop probably) - the calling thread returns, which signals systemd that the daemon is up now Are there any drawbacks with this solution? Yes, you forgot the other 12 ugly steps as described in daemon(7). Which is why the classic way is not something worth the time. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
El 03/11/13 10:42, Hoyer, Marko (ADITG/SW2) escribió: Isn't the classical Linux way an option to? - the daemon does its initialization with the calling thread - once it is done with the initialization, it forks off a process that goes on with the daemons work (the main loop probably) - the calling thread returns, which signals systemd that the daemon is up now Type=forking must be defined in the .service to support this architecture. Are there any drawbacks with this solution? Yes, having reviewed dozens of daemons for migration to systemd, I can assure yours will also be missing something of the required initialization sequence (see daemon(7) ) or doing it wrong, as the number of daemons that do this routines correctly is almost non-existent. For new daemons, please use Type=notify. -- Judging by their response, the meanest thing you can do to people on the Internet is to give them really good software for free. - Anil Dash ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
-Original Message- From: systemd-devel-boun...@lists.freedesktop.org [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Cristian Rodríguez Sent: Sunday, November 03, 2013 3:25 PM To: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Need advice on daemon's architecture El 03/11/13 10:42, Hoyer, Marko (ADITG/SW2) escribió: Isn't the classical Linux way an option to? - the daemon does its initialization with the calling thread - once it is done with the initialization, it forks off a process that goes on with the daemons work (the main loop probably) - the calling thread returns, which signals systemd that the daemon is up now Type=forking must be defined in the .service to support this architecture. Are there any drawbacks with this solution? Yes, having reviewed dozens of daemons for migration to systemd, I can assure yours will also be missing something of the required initialization sequence (see daemon(7) ) or doing it wrong, as the number of daemons that do this routines correctly is almost non-existent. For new daemons, please use Type=notify. -- Judging by their response, the meanest thing you can do to people on the Internet is to give them really good software for free. - Anil Dash ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel Thx for the fast feedback. Good hint with the man page, I'll have a more detailed look on the page. I think when you need to stay a bit independent from systemd and don't have a dbus interface which can be used for synchronization there is probably no other way then the classical one. But in case I'm starting in such a well prepared environment like the one provided by a systemd service, I hopefully will not run into any troubles even if my daemon is missing something of the required initialization sequence or doing it wrong ;) Best regards Marko Hoyer Software Group II (ADITG/SW2) Tel. +49 5121 49 6948 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need advice on daemon's architecture
On Sun, Nov 03, 2013 at 06:18:39PM +, Hoyer, Marko (ADITG/SW2) wrote: Thx for the fast feedback. Good hint with the man page, I'll have a more detailed look on the page. I think when you need to stay a bit independent from systemd and don't have a dbus interface which can be used for synchronization there is probably no other way then the classical one. But in case I'm starting in such a well prepared environment like the one provided by a systemd service, I hopefully will not run into any troubles even if my daemon is missing something of the required initialization sequence or doing it wrong ;) Actually, the opposite is true. Systemd is more demanding, primarily because everything is faster, and if there's a race condition during daemon startup, it's more likely to matter with systemd. So, having a well prepared environment doesn't mean the startup procedure can be sloppy. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel