Re: [systemd-devel] runtime directories for services vs. tmpfiles
Le mardi 16 juillet 2013 à 17:59 +0100, Colin Guthrie a écrit : 'Twas brillig, and Kay Sievers at 16/07/13 17:24 did gyre and gimble: On Tue, Jul 16, 2013 at 6:01 PM, Lennart Poettering lenn...@poettering.net wrote: Hmm, I'd like such an automatism, but I'd really prefer if we could come up with some scheme to automatically determine all tmpfiles snippets in the package and apply them all automatically. But I am not sure how that could be done with current RPM. In fact, I'd actually like to do the same for the %systemd_post macros, and suchlike, where people tend to be pretty bad at always listing all unit files correctly. Actually rereading this bit above, I can see the desire for automated snippets, but are there not some cases where you would want to avoid calling %systemd_post macros for some shipped units? e.g. they may only be used internally by others (Requires=, Also= and similar). I think such automation might not be good. (just look at the NFS units which I think we've finally cleaned up a bit in Mageia - enough to actually work at least). Hmm, an rpmlint check for stuff like this maybe is the first step? At least people who care and look at that would fix their stuff? Misc was looking at doing most of that. Not sure what you mean about naming tho'... do you just mean the folder name (etc vs usr) or just not picking very nice unit names? For the folder, I put rpmlint checks in Mageia a while ago to ban units, tmpfiles and udev rules from shipping in /etc. And another to do with non-ghost files in /run or /var/run. I had it in my head that I'd spoken to Misc about this at the time I wrote them but now I have a paranoid fear that maybe I didn't... They are simple enough anyway, so if they are not upstream and have instead been reimplemented then it's no great loss, but apologies if forgot to ping you at the time Misc (and I've seen you twice in person since then too!!) Patches here. Can't check if it's upstream yet as rpmlint.zarb.org is down for now... We are now on sf.net. And someone already contacted me to get the patches, I said i was ok on the principle, but from the code point of view, it could have been refactored ( but I didn't do it yet ); IE having a structure like : ['regexp','name-of-exception','message'], have it exposed in the configuration and have a generic module using this, so every distribution could add them. IIRC, Suse do have this kind of patchs, but I didn't merged them yet. -- Michael Scherer ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Thu, Jul 18, 2013 at 4:50 AM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Thu, Jul 18, 2013 at 02:51:09AM +0200, Lennart Poettering wrote: [snip repoquery magic] This means: ~81% of the packages have been converted from sysv to systemd. And ~10% of the converted packages make use of tmpfiles. Now, my rpm/yum-fu is a bit too limited to easily figure out what precisely they use of the tmpfiles functionality. So, as the person who originally porposed adding tmpfiles to units, I'm now more or less convinced that it's too late and not worth it. Since the tmpfiles snippets have been written, and a big chunk is done, it would be too confusing to add duplicate functionality. People pointed out problems with the limited RuntimeDirectory=/RuntimeDirectoryMode= proposal, and Lennart and Kay are against the original idea to add tmpfiles snippets directly. The unproven benefits are not worth it, and we certainly have other things waiting on the table. So let's close this thread. Sounds good to me. Thanks, Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On 07/18/2013 12:51 AM, Lennart Poettering wrote: Fedora is not completely converted, but here are some stats. $ repoquery --whatprovides '/etc/rc.d/init.d/*' --qf %{name} | sort -u | egrep -v '(-sysvinit|-initscript|-sysv)$' | wc -l 139 Well even that number is not accurate since we have several migrated components being stuck in bugzilla and not being packaged and shipped which is the biggest problem we are faced with in the migration process ( and have been faced with from the start ) as in not the actual migration but to get maintainers of those components, package and ship the units ( and quite few of those remaining ones seem to unmaintained ). If everything goes as planned we should finish migrating those this release cycle ( F20 ). JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 07:41:08PM +, Jóhann B. Guðmundsson wrote: On 07/16/2013 07:34 PM, Kay Sievers wrote: On Tue, Jul 16, 2013 at 9:28 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 07/16/2013 06:26 PM, Lennart Poettering wrote: I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= What exactly are we trying to solve here which should not be fixed upstream in the daemons themselves? Stuff that uses User= cannot be fixed in the daemon itself, at the time it runs, systemd itself has already taken away the privileges to create these directories. We increase unit complexity with each option we add so I would suggest unless it's absolutely necessary to just stick with just tmpfiles to handle this. I second Jóhann. As a packager, I never had problems with separate configuration of runtime dir. Additionally, runtime dirs hardly need customizations by admin. It's not worth exposing them in unit file. -- Tomasz Torcz Morality must always be based on practicality. xmpp: zdzich...@chrome.pl-- Baron Vladimir Harkonnen ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On 07/16/2013 09:28 PM, Jóhann B. Guðmundsson wrote: On 07/16/2013 06:26 PM, Lennart Poettering wrote: I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= What exactly are we trying to solve here which should not be fixed upstream in the daemons themselves? If RuntimeDirectory would support variable substitutions, this feature could not be provided with systemd-tmpfiles. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
2013/7/16 Lennart Poettering lenn...@poettering.net: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= If RuntimeDirectory= is set we'd create it and chown() it to the UID/GID set with User= and Group=. We'd apply the mode specified in RuntimeDirectoryMode= to it. What about daemons which drop privileges on their own? Shouldn't we provide a directive to set the directory owner/group? If this scheme is not flexible enough to cover the vast majority of all cases (for services), then I fear we'd end up half of the services using RuntimeDirectory, the other half a tmpfile. And that imho would be even more confusing. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
2013/7/17 Harald Hoyer harald.ho...@gmail.com: If RuntimeDirectory would support variable substitutions, this feature could not be provided with systemd-tmpfiles. That would indeed be a nice feature. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
2013/7/17 Michael Biebl mbi...@gmail.com: If this scheme is not flexible enough to cover the vast majority of all cases (for services), then I fear we'd end up half of the services using RuntimeDirectory, the other half a tmpfile. And that imho would be even more confusing. Could we have some stats from Fedora and/or Arch which have done a (complete) migration to systemd, about how many services/packages currently use a tmpfile and what they use in the tmpfile, i.e. if the proposed scheme from Lennart would suffice say for 90+ % of them Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Wed, 17.07.13 06:56, Andrey Borzenkov (arvidj...@gmail.com) wrote: On Tue, 16.07.13 18:53, Lennart Poettering (lenn...@poettering.net) wrote: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= If RuntimeDirectory= is set we'd create it and chown() it to the UID/GID set with User= and Group=. We'd apply the mode specified in RuntimeDirectoryMode= to it. We'd even allow multiple runtime directories to be specified. What if different directories need different modes? I'm afraid it will become hard to express in unit file. All complex cases would be handled by tmpfiles as before. This is not supposed to be a 100% solution. It would be a 70% solution though, if I look at the tmpfiles snippets I have installed on my local machine. Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Wed, 17.07.13 11:24, Michael Biebl (mbi...@gmail.com) wrote: 2013/7/17 Michael Biebl mbi...@gmail.com: If this scheme is not flexible enough to cover the vast majority of all cases (for services), then I fear we'd end up half of the services using RuntimeDirectory, the other half a tmpfile. And that imho would be even more confusing. Could we have some stats from Fedora and/or Arch which have done a (complete) migration to systemd, about how many services/packages currently use a tmpfile and what they use in the tmpfile, i.e. if the proposed scheme from Lennart would suffice say for 90+ % of them Fedora is not completely converted, but here are some stats. The number of packages with tmpfiles: $ repoquery --whatprovides '/usr/lib/tmpfiles.d/*' --qf %{name} | sort -u | wc -l 59 The full list is here: http://ur1.ca/eonvl The number of packages with service files: $ repoquery --whatprovides '/usr/lib/systemd/system/*' --qf %{name} | sort -u | wc -l 602 The full list is here: http://ur1.ca/eonxq The number of holdouts with init scripts: $ repoquery --whatprovides '/etc/rc.d/init.d/*' --qf %{name} | sort -u | wc -l 167 This is highly misleading however as it is permitted by the fedora policy to continue sysv scripts if this is done in a separate package from the main one. Let's manually sort out these cases: $ repoquery --whatprovides '/etc/rc.d/init.d/*' --qf %{name} | sort -u | egrep -v '(-sysvinit|-initscript|-sysv)$' | wc -l 139 The full list is here: http://ur1.ca/eoo02 This means: ~81% of the packages have been converted from sysv to systemd. And ~10% of the converted packages make use of tmpfiles. Now, my rpm/yum-fu is a bit too limited to easily figure out what precisely they use of the tmpfiles functionality. This is Fedora 19. Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Thu, Jul 18, 2013 at 02:51:09AM +0200, Lennart Poettering wrote: [snip repoquery magic] This means: ~81% of the packages have been converted from sysv to systemd. And ~10% of the converted packages make use of tmpfiles. Now, my rpm/yum-fu is a bit too limited to easily figure out what precisely they use of the tmpfiles functionality. So, as the person who originally porposed adding tmpfiles to units, I'm now more or less convinced that it's too late and not worth it. Since the tmpfiles snippets have been written, and a big chunk is done, it would be too confusing to add duplicate functionality. People pointed out problems with the limited RuntimeDirectory=/RuntimeDirectoryMode= proposal, and Lennart and Kay are against the original idea to add tmpfiles snippets directly. The unproven benefits are not worth it, and we certainly have other things waiting on the table. So let's close this thread. Zbyszek -- they are not broken. they are refucktored -- alxchk ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
'Twas brillig, and Michael Biebl at 16/07/13 02:24 did gyre and gimble: Hi, an interesting issue was raised as part of reviewing a patch for iodione [1], a system service which needs a runtime directory. We thought this might need further dicussion, so reposting the issue to systemd-devel: For system services needing a runtime directory, we basically have two (three) options nowadays 1/ use ExecStartPre=/usr/bin/mkdir /var/run/foo 2/ use a tmpfile snippet (3/ let the daemon create the runtime directory itself) In [1] we recommended the the usage of 2/ over 1/. The main reason for that were, that systemd-tmpfiles properly handles applying security policies, like SELinux labels, and it avoids spawning unnecessary processes (every ExecStartPre=/usr/bin/mkdir is a separate process) Zbyszek is arguing, that splitting the configuration over two files, a tmpfile and a service file, complicates things for the administrator, as things are no longer in a single place. He also argues, that tmpfiles are active, independently of the service, which needs them. Which can lead to unused runtime directories. One suggestion that came up was, to specficy runtime directories in a declarative fashion in the .service file itself, which means it would be only be created if the service itself is enabled. I think this idea warrants further discussion, so I'm posting it here. Hmmm, I can certainly see the argument, but it would be a shame to have multiple ways to achieve the same goal through different places too (it makes the documentation case much more confusing). There is also an argument that on first install, one has to call systemd-tmpfiles --create $BASENAME in a %post script to ensure the necessary dirs are created. If it was all built in to the service this would be fine. So I see a few options. 1. Bake it into systemd unit syntax and handle it internally. 2. Bake it in as a noop directive and have a generator parse it out and then write out /run/tmpfiles.d/ snippets and let systemd-tmpfiles work with that. 3. Keep the status quo. I don't really like option 2 there - seems unnecessary overhead. But then option one (which presumably would see the tmpfiles code being libified and linked to from systemd itself) doesn't seem amazing too. I guess I don't really have a major problem with either 1 or 3 to be honest. I can see arguments for both. Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
Hi Michael, On Tue, Jul 16, 2013 at 3:24 AM, Michael Biebl mbi...@gmail.com wrote: an interesting issue was raised as part of reviewing a patch for iodione [1], a system service which needs a runtime directory. We thought this might need further dicussion, so reposting the issue to systemd-devel: For system services needing a runtime directory, we basically have two (three) options nowadays 1/ use ExecStartPre=/usr/bin/mkdir /var/run/foo 2/ use a tmpfile snippet (3/ let the daemon create the runtime directory itself) In [1] we recommended the the usage of 1/ over 2/. The main reason for that were, that systemd-tmpfiles properly handles applying security policies, like SELinux labels, and it avoids spawning unnecessary processes (every ExecStartPre=/usr/bin/mkdir is a separate process) I think the current way of doing this (letting tmpfiles or the daemon itself handle it) is for the best, for the reasons you mention. Also because it means we only have one tool to create folders during early boot. Zbyszek is arguing, that splitting the configuration over two files, a tmpfile and a service file, complicates things for the administrator, as things are no longer in a single place. Hm, that is a fair point, but we also have the dual argument: if we add a new place to configure runtime dirs without removing the old (which I suppose we can't), the admin will anyway have to look in both places to figure out what is going on... He also argues, that tmpfiles are active, independently of the service, which needs them. Which can lead to unused runtime directories. I suppose the concern here is speed? If it can be shown that there is a real speed advantage of not creating unneeded directories I guess that's something worth discussing (I must admit that would surprise me though). -t ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
'Twas brillig, and Lennart Poettering at 16/07/13 16:00 did gyre and gimble: Something I'd love to see though is if we could make it easier to apply tmpfiles stuff automatically on package installation. More specifically, I'd like an RPM macro to be added that handles this, and which is sufficient for packagers to call for their tmpfiles. ALl in order to ensure that the user can invoke the services right after package installation without requiring a reboot. I've gone around the block a few times on this. 1. We used file triggers (which we've supported for a while with various patches to rpm but which think are only just now coming to fedora via official support in upstream rpm - I forget the details tho', so could be wrong), but this was a bad idea as these triggers ran too late after %post. As %post might need these folders to start/restart the daemon, I had to abandon that. 2. I used to recommend just running systemd-tmpfiles --create but this meant that the /run/nologin file was created which meant you couldn't login... ugg. Don't do that in any macro please :D 3. Finally I settled on: %_tmpfilesdir /usr/lib/tmpfiles.d %_tmpfilescreate() /usr/bin/systemd-tmpfiles --create %{1}.conf \ %{nil} In spec files I just put %_tmpfilescreate %{name} in %post in 99% of cases and a custom basename for when the file name does not match the package name in some rare cases. I'm sure the macro could be tweaked to put %{name} in magically if no argument is supplied.. HTHs (and if you have no preference as to the name, that you can use the one I already use ;) Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, 16.07.13 16:50, Colin Guthrie (gm...@colin.guthr.ie) wrote: 3. Finally I settled on: %_tmpfilesdir /usr/lib/tmpfiles.d %_tmpfilescreate() /usr/bin/systemd-tmpfiles --create %{1}.conf \ %{nil} This looks pretty close to what I'd like to see in place. Note that %_tmpfilesdir is already set in macros.systemd as we ship it upstream. My suggestion would be this however: _tmpfiles_create() \ /usr/bin/systemd-tmpfiles --create %{1} /dev/null 21 || : \ %{nil} I thik that people should specify file names rather than package names (just to keep this closer to the underlying command and to make the multiple drop-in case less exotic, and since we require the suffixe for the unit macros too) and ignore errors. Also, add an underscore since our other macros use the same naming style. In spec files I just put %_tmpfilescreate %{name} in %post in 99% of cases and a custom basename for when the file name does not match the package name in some rare cases. I'm sure the macro could be tweaked to put %{name} in magically if no argument is supplied.. Hmm, I'd like such an automatism, but I'd really prefer if we could come up with some scheme to automatically determine all tmpfiles snippets in the package and apply them all automatically. But I am not sure how that could be done with current RPM. In fact, I'd actually like to do the same for the %systemd_post macros, and suchlike, where people tend to be pretty bad at always listing all unit files correctly. Anyway, does that RPM macro sound good to you? If so, let's add this to macros.systemd and I'll ask our Fedora guys to add this to the guidelines. Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
'Twas brillig, and Lennart Poettering at 16/07/13 17:01 did gyre and gimble: Anyway, does that RPM macro sound good to you? Sure, seems close enough :) I can do a mass update to all our packages anyway so the slight change in syntax isn't a problem. Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 6:01 PM, Lennart Poettering lenn...@poettering.net wrote: Hmm, I'd like such an automatism, but I'd really prefer if we could come up with some scheme to automatically determine all tmpfiles snippets in the package and apply them all automatically. But I am not sure how that could be done with current RPM. In fact, I'd actually like to do the same for the %systemd_post macros, and suchlike, where people tend to be pretty bad at always listing all unit files correctly. Hmm, an rpmlint check for stuff like this maybe is the first step? At least people who care and look at that would fix their stuff? Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 05:08:12PM +0100, Colin Guthrie wrote: 'Twas brillig, and Lennart Poettering at 16/07/13 17:01 did gyre and gimble: Anyway, does that RPM macro sound good to you? Sure, seems close enough :) I can do a mass update to all our packages anyway so the slight change in syntax isn't a problem. Hm, can we take a step back for a moment? It seems that the rpm macros are a fairly complicated solution, and they also don't carry over into debian or arch. User mode sessions also will not work with rpm macros. I'm sure we can get them to work, but I'd like to explore the idea of moving that into unit files first. I am not too concerned about unused runtime directories. After all this is not something that would (or even could) grow without bounds. There will never be more than O(n) runtime directores for n being the number of packages you have installed (or had installed). Sure, but this n can be around a couple hundred I'd image when debian is fully converted and installed. For efficiency this is probably unmesurable, but as an admin I'd prefer not to have hundreds of empty dirs in /run. We will always need setting up of tmpfiles during boot, as many clients expect certain directories to be existant in /run without any specific daemon running. Thus, we couldn't move this stuff into service files --- we could only allow both. Yes, I'm not arguing for removing boot time tmpfiles. However, I am not convinced that allowing tmpfiles to be denoted in unit files would be a good thing though, because it sounds unnecessary. It really sounds as if the daemons which need that should rather create the directories on their own before making use of them. This should generally be the most robust solution. And in such a case there's no need to denote anything about this in the unit files at all... Well, we seem to be using the tmpfiles mechanism quite a lot. And there are good reasons for that: we get a lot of flexibility with a powerful syntax, and actually there's significant downsides to putting all this logic in daemons, e.g. support for various xattr mechanisms. We want to simplify daemons. Also, as you mentioned, some stuff runs unpriviledged. Also, we could grow a mechanism to *remove* dirs after the daemon is stopped. Putting it in rpm specific mechanism removes this possibility. I mean, this is the general recommendation anyway: if possible just fix the daemon to create the dirs as it needs it. Use tmpfiles only if that's not feasible, maybe because the daemon never runs privileged code or because the dirs need to exist at boot already. Something I'd love to see though is if we could make it easier to apply tmpfiles stuff automatically on package installation. More specifically, I'd like an RPM macro to be added that handles this, and which is sufficient for packagers to call for their tmpfiles. ALl in order to ensure that the user can invoke the services right after package installation without requiring a reboot. Zbyszek -- they are not broken. they are refucktored -- alxchk ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 6:34 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: Hm, can we take a step back for a moment? It seems that the rpm macros are a fairly complicated solution, and they also don't carry over into debian or arch. User mode sessions also will not work with rpm macros. For what it is worth we have been calling systemd-tmpfiles --create $pkgname.conf in post-install for a long time in Arch (since the beginning I think), and it doesn't seem to be a big problem. However, if you have some idea for simplifying it somehow, I'd be all for that of course :-) Maybe establish an implicit link between foo.service and tmpfiles.d/foo.conf, which would allow calling systemd-tmpfiles --create/--clean before/after start/stop? Not sure if it is really worth it, and not sure there won't be regressions though... -t ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, 16.07.13 18:24, Kay Sievers (k...@vrfy.org) wrote: On Tue, Jul 16, 2013 at 6:01 PM, Lennart Poettering lenn...@poettering.net wrote: Hmm, I'd like such an automatism, but I'd really prefer if we could come up with some scheme to automatically determine all tmpfiles snippets in the package and apply them all automatically. But I am not sure how that could be done with current RPM. In fact, I'd actually like to do the same for the %systemd_post macros, and suchlike, where people tend to be pretty bad at always listing all unit files correctly. Hmm, an rpmlint check for stuff like this maybe is the first step? At least people who care and look at that would fix their stuff? Makes sense. Added to the TODO list for now. Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, 16.07.13 17:08, Colin Guthrie (gm...@colin.guthr.ie) wrote: 'Twas brillig, and Lennart Poettering at 16/07/13 17:01 did gyre and gimble: Anyway, does that RPM macro sound good to you? Sure, seems close enough :) I can do a mass update to all our packages anyway so the slight change in syntax isn't a problem. As mentioned, I commited this now, since we need this in any way. Could you check if this works for you, please? Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, 16.07.13 18:34, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote: On Tue, Jul 16, 2013 at 05:08:12PM +0100, Colin Guthrie wrote: 'Twas brillig, and Lennart Poettering at 16/07/13 17:01 did gyre and gimble: Anyway, does that RPM macro sound good to you? Sure, seems close enough :) I can do a mass update to all our packages anyway so the slight change in syntax isn't a problem. Hm, can we take a step back for a moment? It seems that the rpm macros are a fairly complicated solution, and they also don't carry over into debian or arch. User mode sessions also will not work with rpm macros. Well, it's easy to come up with similar macros for debhelper or arch... User session don't really need this, as there as user daemons usually create the dirs they need on their own and there are no privileges involved which might forbid this. I have commite the RPM macro fro now, since either way we will need it for the packages where the dirs need to exist all the time. I am not too concerned about unused runtime directories. After all this is not something that would (or even could) grow without bounds. There will never be more than O(n) runtime directores for n being the number of packages you have installed (or had installed). Sure, but this n can be around a couple hundred I'd image when debian is fully converted and installed. For efficiency this is probably unmesurable, but as an admin I'd prefer not to have hundreds of empty dirs in /run. On my (pretty much fully converted) Fedora I currently have 20 tmpfiles snippets around. I doubt on an everage Debian machine this would grow much larger. May 40 or so, but that's still not much. However, I am not convinced that allowing tmpfiles to be denoted in unit files would be a good thing though, because it sounds unnecessary. It really sounds as if the daemons which need that should rather create the directories on their own before making use of them. This should generally be the most robust solution. And in such a case there's no need to denote anything about this in the unit files at all... Well, we seem to be using the tmpfiles mechanism quite a lot. And there are good reasons for that: we get a lot of flexibility with a powerful syntax, and actually there's significant downsides to putting all this logic in daemons, e.g. support for various xattr mechanisms. We want to simplify daemons. Also, as you mentioned, some stuff runs unpriviledged. I want daemons to be robust. So if possible I think daemons should do these things on their own. This also has benefits for systems like selinux where the label for files can be determined using the normal security transitions, rather then databse checks like we currently need to do it... Also, we could grow a mechanism to *remove* dirs after the daemon is stopped. Putting it in rpm specific mechanism removes this possibility. I can see how this could be nice but it kinda reminds me of the situation regarding removing UIDs from /etc/passwd after package deinstallation, which is soemthing that is simply very hard to ever get rid, and which is why Fedora is not doing this at all... Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
'Twas brillig, and Kay Sievers at 16/07/13 17:24 did gyre and gimble: On Tue, Jul 16, 2013 at 6:01 PM, Lennart Poettering lenn...@poettering.net wrote: Hmm, I'd like such an automatism, but I'd really prefer if we could come up with some scheme to automatically determine all tmpfiles snippets in the package and apply them all automatically. But I am not sure how that could be done with current RPM. In fact, I'd actually like to do the same for the %systemd_post macros, and suchlike, where people tend to be pretty bad at always listing all unit files correctly. Actually rereading this bit above, I can see the desire for automated snippets, but are there not some cases where you would want to avoid calling %systemd_post macros for some shipped units? e.g. they may only be used internally by others (Requires=, Also= and similar). I think such automation might not be good. (just look at the NFS units which I think we've finally cleaned up a bit in Mageia - enough to actually work at least). Hmm, an rpmlint check for stuff like this maybe is the first step? At least people who care and look at that would fix their stuff? Misc was looking at doing most of that. Not sure what you mean about naming tho'... do you just mean the folder name (etc vs usr) or just not picking very nice unit names? For the folder, I put rpmlint checks in Mageia a while ago to ban units, tmpfiles and udev rules from shipping in /etc. And another to do with non-ghost files in /run or /var/run. I had it in my head that I'd spoken to Misc about this at the time I wrote them but now I have a paranoid fear that maybe I didn't... They are simple enough anyway, so if they are not upstream and have instead been reimplemented then it's no great loss, but apologies if forgot to ping you at the time Misc (and I've seen you twice in person since then too!!) Patches here. Can't check if it's upstream yet as rpmlint.zarb.org is down for now... http://svnweb.mageia.org/packages/cauldron/rpmlint/current/SOURCES/ Col -- Colin Guthrie colin(at)mageia.org http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On 07/16/2013 04:53 PM, Lennart Poettering wrote: On my (pretty much fully converted) Fedora I currently have 20 tmpfiles snippets around. I doubt on an everage Debian machine this would grow much larger. May 40 or so, but that's still not much. Well we have only migrated what 400 components out of ca 550 Debian has twice that much ( around 1100 - 1200 I think ) but yes that number seems to fairly accurate which means it might actually be better pay off to simple fix those 20 daemons that require the tmpfiles in the first place. JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, 16.07.13 18:53, Lennart Poettering (lenn...@poettering.net) wrote: I can do a mass update to all our packages anyway so the slight change in syntax isn't a problem. Hm, can we take a step back for a moment? It seems that the rpm macros are a fairly complicated solution, and they also don't carry over into debian or arch. User mode sessions also will not work with rpm macros. Well, it's easy to come up with similar macros for debhelper or arch... User session don't really need this, as there as user daemons usually create the dirs they need on their own and there are no privileges involved which might forbid this. I have commite the RPM macro fro now, since either way we will need it for the packages where the dirs need to exist all the time. I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= If RuntimeDirectory= is set we'd create it and chown() it to the UID/GID set with User= and Group=. We'd apply the mode specified in RuntimeDirectoryMode= to it. We'd even allow multiple runtime directories to be specified. I want this to be specific to the purpose of running daemons so I would even go es far as generating a parse error if the specified directory is not within /run. We wouldn't do any of the fancier things that tmpfiles does, because we want people to use tmpfiles for that. For example, if automatic clean-up needs to take place this information needs to be in tmpfiles, not in a service unit. We would hook this up with the same logic that currently handles PrivateTmp= and remove the the runtime directory when the service goes down. With this in place we could probably cover 60% of the current users of tmpfiles or so. Putting this all together we'd recommend: a) in the best case make your daemon create all runtime dirs as necessary b) if that's not possible (for example, due to rpivs), then use RuntimeDirectory= instead. c) If that's not enough (for example, because you need your dirs around even if the daemon is not running) , then drop in a tmpfiles snippet instead, and hook it in via the new RPM macro (or counterparts for non-RPM distributions) so that it is executed at package installation time More specific to Fedora, there's still the question whether these dirs in /run should be tracked by RPM or not. For this I think I am leaning towards: d) List the runtime dirs you have in the RPM spec file, but %ghost them. This is nice so that the directory can be easily traced back to the package that created them. Opinions? Suggestions? Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, 16.07.13 17:59, Colin Guthrie (co...@mageia.org) wrote: 'Twas brillig, and Kay Sievers at 16/07/13 17:24 did gyre and gimble: On Tue, Jul 16, 2013 at 6:01 PM, Lennart Poettering lenn...@poettering.net wrote: Hmm, I'd like such an automatism, but I'd really prefer if we could come up with some scheme to automatically determine all tmpfiles snippets in the package and apply them all automatically. But I am not sure how that could be done with current RPM. In fact, I'd actually like to do the same for the %systemd_post macros, and suchlike, where people tend to be pretty bad at always listing all unit files correctly. Actually rereading this bit above, I can see the desire for automated snippets, but are there not some cases where you would want to avoid calling %systemd_post macros for some shipped units? e.g. they may only be used internally by others (Requires=, Also= and similar). I think such automation might not be good. (just look at the NFS units which I think we've finally cleaned up a bit in Mageia - enough to actually work at least). No. You really want the macros for all units, always. The macros use the preset information to enable units if that's what the vendor or admin wants. This preset information should be what decides what is enabled and what is not, not the packager who accidently forgot to invoke systemctl preset on one of his units. The unit names also matter to shutdown running units before uninstalling them. For that too you really need the list of all units the package installs and not just the ones the package might have though about listing. So yeah, we want that the scritps process *all* units the package installs, not just some subset. Only that guarantees correct behaviour regarding presets and regarding stopping things when we uninstall. Of coruse, this is not obvious to people, which is why many packages don't get this right. With the automatic logic we'd avoid this confusion. Hmm, an rpmlint check for stuff like this maybe is the first step? At least people who care and look at that would fix their stuff? Misc was looking at doing most of that. Not sure what you mean about naming tho'... do you just mean the folder name (etc vs usr) or just not picking very nice unit names? The rpmlint should just check that all units a package ships are listed as arguments for the RPM macro. For the folder, I put rpmlint checks in Mageia a while ago to ban units, tmpfiles and udev rules from shipping in /etc. That's a very good choice. And another to do with non-ghost files in /run or /var/run. Also a good choice! Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 08:26:23PM +0200, Lennart Poettering wrote: On Tue, 16.07.13 18:53, Lennart Poettering (lenn...@poettering.net) wrote: I can do a mass update to all our packages anyway so the slight change in syntax isn't a problem. Hm, can we take a step back for a moment? It seems that the rpm macros are a fairly complicated solution, and they also don't carry over into debian or arch. User mode sessions also will not work with rpm macros. Well, it's easy to come up with similar macros for debhelper or arch... User session don't really need this, as there as user daemons usually create the dirs they need on their own and there are no privileges involved which might forbid this. I have commite the RPM macro fro now, since either way we will need it for the packages where the dirs need to exist all the time. I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= If RuntimeDirectory= is set we'd create it and chown() it to the UID/GID set with User= and Group=. We'd apply the mode specified in RuntimeDirectoryMode= to it. There are daemons which do, in order: 1) start as root 2) open some root-only resources, like network sockets 3) change to less priviledge user 4) if all OK, write PID file or some state in /run 5) do the work Those daemons can't have User= specified, because 2) would fail. And because they must start as root, systemd can't know what chown() to do on runtime dir. So what's the solution? a) declare those daemons incompatible with RuntimeDirectory* stuff - nothing changes, they continue to use separate tmpfile .conf b) fix the daemons? -- Tomasz Torcz God, root, what's the difference? xmpp: zdzich...@chrome.pl God is more forgiving. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 9:17 PM, Tomasz Torcz to...@pipebreaker.pl wrote: If RuntimeDirectory= is set we'd create it and chown() it to the UID/GID set with User= and Group=. We'd apply the mode specified in RuntimeDirectoryMode= to it. There are daemons which do, in order: 1) start as root 2) open some root-only resources, like network sockets 3) change to less priviledge user 4) if all OK, write PID file or some state in /run 5) do the work Those daemons can't have User= specified, because 2) would fail. And because they must start as root, systemd can't know what chown() to do on runtime dir. So what's the solution? a) declare those daemons incompatible with RuntimeDirectory* stuff - nothing changes, they continue to use separate tmpfile .conf b) fix the daemons? B, sure, when it's private to the daemon and it run as root, it should prepare its own environment itslef. It's just weird to assume someone else does that for the daemon. And it's just slightly less broken to use a tmpfiles to work around such a broken daemon init. :) Stuff is different though if the directory is some kind of an API, then tmpfiles are sure fine, and creating them at boot is the right thing. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On 07/16/2013 06:26 PM, Lennart Poettering wrote: I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= What exactly are we trying to solve here which should not be fixed upstream in the daemons themselves? JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 07:28:12PM +, Jóhann B. Guðmundsson wrote: On 07/16/2013 06:26 PM, Lennart Poettering wrote: I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= What exactly are we trying to solve here which should not be fixed upstream in the daemons themselves? It was written in the part you removed: Putting this all together we'd recommend: a) in the best case make your daemon create all runtime dirs as necessary b) if that's not possible (for example, due to privs), then use RuntimeDirectory= instead. ... Zbyszek -- they are not broken. they are refucktored -- alxchk ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On Tue, Jul 16, 2013 at 9:28 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 07/16/2013 06:26 PM, Lennart Poettering wrote: I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= What exactly are we trying to solve here which should not be fixed upstream in the daemons themselves? Stuff that uses User= cannot be fixed in the daemon itself, at the time it runs, systemd itself has already taken away the privileges to create these directories. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
On 07/16/2013 07:34 PM, Kay Sievers wrote: On Tue, Jul 16, 2013 at 9:28 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 07/16/2013 06:26 PM, Lennart Poettering wrote: I discussed this a bit more with Kay on the phone. Here's what we'd propose: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= What exactly are we trying to solve here which should not be fixed upstream in the daemons themselves? Stuff that uses User= cannot be fixed in the daemon itself, at the time it runs, systemd itself has already taken away the privileges to create these directories. We increase unit complexity with each option we add so I would suggest unless it's absolutely necessary to just stick with just tmpfiles to handle this. JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
2013/7/16 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: On Tue, Jul 16, 2013 at 05:08:12PM +0100, Colin Guthrie wrote: I am not too concerned about unused runtime directories. After all this is not something that would (or even could) grow without bounds. There will never be more than O(n) runtime directores for n being the number of packages you have installed (or had installed). Sure, but this n can be around a couple hundred I'd image when debian is fully converted and installed. For efficiency this is probably unmesurable, but as an admin I'd prefer not to have hundreds of empty dirs in /run. That seems to be misconception: Even when all 1200+ packages currently shipping SysV init scripts, have native systemd support, this doesn't mean that all those 1200 packages are actually installed. I think it's even less of an issue for Debian, as we usually only install stuff when needed and not in a disabled state. I expect that a typical desktop installation has around 30 system services, and I'm pretty sure it's rather uncommon to have more then 5 disabled system services. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
And to follow up on that: 2013/7/16 Michael Biebl mbi...@gmail.com: I think it's even less of an issue for Debian, as we usually only install stuff when needed and not in a disabled state. So the unused directories in /run are something I don't expect to be an issue for Debian. As for tmpfiles support: In Debian we already have pretty nice support and debhelper (dh_installinit) scans the installed files and creates postinst snippet like if [ -d /run/systemd/system ] ; then systemd-tmpfiles --create /usr/lib/tmpfiles.d/xconsole.conf /dev/null || true fi which is run before the daemon is started. So, also not an issue for Debian. That leaves the only point: the configuration is split between two files. Tbh, I'm not too thrilled duplicating part of the tmpfiles functionality in .service files itself. What we could do though, similar to Documentation, reference a tmpfile which belongs to the service. So when you run systemctl status foo.service, it would show the corresponding file. This would it more discoverable for the admin and he'd see which file to modify. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] runtime directories for services vs. tmpfiles
В Tue, 16 Jul 2013 20:26:23 +0200 Lennart Poettering lenn...@poettering.net пишет: On Tue, 16.07.13 18:53, Lennart Poettering (lenn...@poettering.net) wrote: I'd be very conservative regarding adding full tmpfiles support into unit files directly. Instead, I'd suggest adding two very minimal, very specific new unit file settings: RuntimeDirectory= RuntimeDirectoyMode= If RuntimeDirectory= is set we'd create it and chown() it to the UID/GID set with User= and Group=. We'd apply the mode specified in RuntimeDirectoryMode= to it. We'd even allow multiple runtime directories to be specified. What if different directories need different modes? I'm afraid it will become hard to express in unit file. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel