Re: [systemd-devel] rpcbind static, want enabled
Andrei Borzenkov composed on 2014-10-21 11:29 (UTC+0400): Felix Miata wrote: I have 27 Fedora 21 22 installations to real hardware, all originating via HTTP process. Half work as expected. Those that do have NetworkManager not installed, and have rpcbind.service enabled Those that misbehave by having a more than a minute delay on attempt to mount nfs via fstab entries, of which half do and half do not have NetworkManager installed, have rpcbind.service static then it is socket activated, you need to enable rpcbind.socket. But every one of the failing installations already had: rpcbind.socket enabled same as the working installations. With chkconfig and sysvinit it was a simple matter to enable rpcbind. With systemd evolved to v216 it seems I should be able to 'systemctl enable rpcbind.service', but that changes nothing on these installations. What's the obstacle here? If the explanation is in man systemctl or man rpcbind, I'm not finding it. Lots more blame, unit status and logs at: http://fm.no-ip.com/Tmp/Linux/F/NFS/ Also Fedora mailing list thread: https://lists.fedoraproject.org/pipermail/test/2014-May/121461.html -- The wise are known for their understanding, and pleasant words are persuasive. Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Fast User Switching does not seem to work
Hi On Tue, Oct 21, 2014 at 1:35 AM, Lennart Poettering lenn...@poettering.net wrote: On Thu, 09.10.14 19:44, constantine (costas.magn...@gmail.com) wrote: Hello all! I am not sure this is the appropriate mailing list, and I have also posted to intel-...@lists.freedesktop.org (without any solution) and arch-gene...@archlinux.org Please indicate if you recommend me posting the problem to another list. I am encountering a peculiar problem with Fast User Switching or Kernel Mode Setting and I do not know how to troubleshoot it. Suppose the following tty2,3 or VTs: 1. Ctrl+Alt+F2:logged in user in console (no X) 2. Ctrl+Alt+F3:logged in user in console (no X) 3. Ctrl+Alt+F4:logged in user in Desktop Environment (X) 4. Ctrl+Alt+F5:logged in user in Desktop Environment (X) From 1 to 2, or 2 to 1 there is no delay. From 1,2 to 3, or 1,2 to 4 there is no delay. From 3,4 to 1,2 there **is** delay. From 3 to 4, or 4 to 3, there **is**delay. In words: when I run two (or more) Virtual Terminals switching between them (Ctrl+Alt+F4-5, etc.) takes 2+ seconds. (The VTs are either simply initiated with startx from tty2 and tty3 and having only TWM, xorg-clock and xterm as their open windows OR as different displays with KDisplay Manager (kdm) in :0 and :1) I am using an intel graphics driver so I suppose KMS should be enabled by default and this seems to be the case, but there still are delays swithing between VTs. I have followed the instructions at http://carlosbecker.com/posts/how-to-enable-kms-on-i915-graphiccard-archlinux/ https://wiki.archlinux.org/index.php/kernel_mode_setting https://wiki.archlinux.org/index.php/Intel_graphics after a fresh Arch Linux installation, but I am still experiencing the problem. I am certain it is not a hardware related issue, because I can fast switch in ubuntu, but not in Arch linux. My Arch uses systemd and Ubuntu upstart. Last, I am aware of the systemd's on-demand feature of spawning terminals but I think it is not related to this, since I think I correctly pre-spawned them. How can I make switching between VTs as instantaneous as possible? Well, the only component that subscribes to VT switching is logind really, but it actually shouldn't interfere with it. VT switching on linux is synchronous, hence I figure that the X server is blocking things when you switch VT. It might be worth using tools like strace to figure out what it blocks on. It might also be sueful to check X's logs to see if you see any messages about timeouts or so. Not sure what else I can recommend. This is expected behavior. X.org runs hardware specific drivers which often use different modes than any generic KMS users. Changing into such modes is usually fast and properly supported, but leaving them (and restoring a generic mode) requires a full modeset, which can take up to 2s. One example: to improve 3D rendering performance, you wanna render into tiled-buffers, therefore, those are also used for scan-out. Generic user-space usually uses linear buffers and thus requires a pipe-line change. Some hardware cannot perform such changes without a full modeset. Furthermore, some Xorg drivers reset their state on VT-leave events. This requires extra time on behalf of Xorg and is usually redundant. Thanks David ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] rpcbind static, want enabled
On Tue, Oct 21, 2014 at 11:58 AM, Felix Miata mrma...@earthlink.net wrote: Andrei Borzenkov composed on 2014-10-21 11:29 (UTC+0400): Felix Miata wrote: I have 27 Fedora 21 22 installations to real hardware, all originating via HTTP process. Half work as expected. Those that do have NetworkManager not installed, and have rpcbind.service enabled Those that misbehave by having a more than a minute delay on attempt to mount nfs via fstab entries, of which half do and half do not have NetworkManager installed, have rpcbind.service static then it is socket activated, you need to enable rpcbind.socket. But every one of the failing installations already had: rpcbind.socket enabled same as the working installations. With chkconfig and sysvinit it was a simple matter to enable rpcbind. With systemd evolved to v216 it seems I should be able to 'systemctl enable rpcbind.service', but that changes nothing on these installations. What's the obstacle here? If the explanation is in man systemctl or man rpcbind, I'm not finding it. Lots more blame, unit status and logs at: http://fm.no-ip.com/Tmp/Linux/F/NFS/ Also Fedora mailing list thread: https://lists.fedoraproject.org/pipermail/test/2014-May/121461.html May 29 00:40:00 g5eas rpc.statd[692]: Version 1.3.0 starting May 29 00:40:00 g5eas systemd[1]: Started RPC bind service. It looks like rpcbind is correctly attempted to be started when statd tries to contact it so socket activation appears to work as far as systemd is concerned. Then for some reasons rpcbind fails to process request from statd. Enabling more debugging on rpcbind and/or statd may explain why registration fails. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] swap: introduce Discard property
Lennart Poettering lenn...@poettering.net writes: On Mon, 20.10.14 11:08, Karel Zak (k...@redhat.com) wrote: On Fri, Oct 03, 2014 at 07:16:55AM +0200, Jan Synacek wrote: Karel Zak k...@redhat.com writes: Karel, any chance you can add a -o option to swapon? No problem, added to TODO. I'll implement it next week. Implemented, it's in util-linux git tree, will be in v2.26. Would you please let me know when that patch makes it to the package in rawhide? I would then fix the code I wrote to support it. Thank you. Probably in December. Jan, any chance you could rework the current systemd code to simply hackishly accept Options=discard instead of Discard=yes for the discard code? Then, later on, when swapon learns -o we can pass the string 1:1 over. For now we'd just check if Options= is set to the precise string discard, and generate an error for anything else. That way we could make the discard functionality available now, but keep compatibility with the future mkswap? Yeah, I'll look into it. -- Jan Synacek Software Engineer, Red Hat signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Sonntag, 21. September 2014, 15:31:15 schrieb Martin Steigerwald: Hello! I know this is a daring post. I just have one question. In the light of http://boycottsystemd.org/ http://uselessd.darknedgy.net/ developing some systemd compatible services for BSD: http://undeadly.org/cgi?action=articlesid=20140915064856 in the light of Debian Bug report logs - #727708 tech-ctte: Decide which init system to default to in Debian. https://bugs.debian.org/727708 Debian Bug report logs - #746715 the foreseeable outcome of the TC vote on init systems https://bugs.debian.org/bug=746715 in the light of the ongoing discussions on linux-kernel, debian-devel, debian- user and other mailing lists more than some dozens threads meanwhile: In my long years of using Debian and also doing some packages for it in the last years I never saw that any introduced changed caused a serious we may need to fork like announcement like this: http://debianfork.org/ Also the upcoming GR decision about whether to require that packages may not depend on PID 1 it refers to. I think I never saw that kind of uproar in Debian. In my view its worrying and damaging. And I think it points at a real problem. There is a dire problem with the acceptance of systemd in Debian (and not only there). In Debian this is not only with users, but also with developers. Wherever I look systemd triggers a split into pro and contra. Even here at work. I still use it, I test it, it seems to work. But I am concerned. Some co-workers dislike it highly. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Hi Rob, Am Montag, 6. Oktober 2014, 14:56:22 schrieb Rob Owens: - Original Message - From: Martin Steigerwald mar...@lichtvoll.de Heck, I started a thread here and then didn´t manage to take time to carefully read it and reply here and there as I see fit. But I challenged people on debian-user mailing list to constructively voice their concerns upstream, and even pointed them to this mailing list. As far as I saw *no one* of the posters in debian-user took up on that challenge. Which I view as a pity. Cause now actually you invited constructive feedback. I wonder whether I may forward your answer to debian-user so they see your statement of inviting constructive feedback. I am here from debian-user, due to Martin's suggestion. So now that he's calling me out, I guess I'll post my questions :) For the record, I'm a sysadmin and not a developer. I imagine my questions and opinions will reflect that. Thank you for voicing your concerns as one of the many users who where not so hesitant to voice their concerns on the debian-user mailing list. Here the feedback I read over and over again is that you and RedHat basically forced the systemd decision onto other distributions. While I do not see how you actually can be powerful enough to do that, as we live in a free will zone. I do see tendencies that more and more stuff *depends* on systemd cause it needs features only available there. On of the most talked on things on debian-user is the logind thing. GNOME actually depends on it, as far as I know. While KDE in Debian still uses ConsoleKit, as it seems to me when looking at the process list and finding: On Debian, I came across an unusual dependency. Installing a cd burner (brasero) required me to change my init system to systemd. Sounds kind of ridiculous, I think. The dependency chain went like this: brasero - gvfs - gvfs-daemons - udisks2 - libpam-systemd - systemd-sysv I really find this kind of chain quite ridiculous. But if its breakable at least as Martin told… still. I think its important to make sure that installing brasero does not accidentally switch a sysvinit system to systemd. As it would be the least a user would expect here and from common sense it does not even make sense. It may make sense technically as explained, but from a user and sysadmin point of view it does not make any sense at all and is quite disruptive. Ciao, -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to use cgroups for Tracker?
On 20/10/14 21:12, Lennart Poettering wrote: On Tue, 14.10.14 15:35, Martyn Russell (mar...@lanedo.com) wrote: Hej Lennart, I am not entirely sure what cgroups would really give you that sched_setscheduler(), ioprio_set(), setrlimit() wouldn't give you It's another approach, more of a sandbox in my mind than a priority system (which the above APIs are really - possibly with the exception of setrlimit()). I will get on to APIs in a bit... Tracker has been used on embedded platforms in the past with its own cgroups because it was more effective than the other APIs. It was recently suggested again in the bug above. anyway, after all tracker is only a single process, no? No, there are several: 1. tracker-store (handles DB writes mainly and ontology / DB schema maintenance). 2. tracker-miner-fs (handles finding and sending the primary data we found to the store to be added to the DB. Primary data is purely information about files, like size, name, etc. no file type specific metadata, like song duration). 3. tracker-extract (handles file type metadata extraction - e.g. the song duration). 4. Other data-miners (tracker-miner-apps, etc) Typically, the data miners use the ioprio_set(), sched_setscheduler() and tracker-extract (ONLY) used to use setrlimit() until about 2 weeks ago where we removed it - it's produces more problems than it solves. The store only uses ioprio_set() - due to all the disk writes we perform there. We don't set the scheduling here because apps on the desktop are using the store too and it should be responsive, so instead we try to make the processes feeding the store more idle or lower priority to drip feed instead. Moreover tracker is unpriviliged and runs in user context (not system context), right? In that case access to cgroupfs is not available, you have to go through systemd's per-user APIs. However, currently moving user sessions to system is not complete, hence I fear its to early to make this change. Also, at least initially, even if we'd establish systemd for users widely I have doubts we'll support much more than CPUShares= for unpriviliged users. All completely true. Though my suggestion was to try to install a cgroup by the packager that could be generally used by Tracker. I don't see Tracker needing to change the details of the group. libcgroup is obsolete btw. And while this is currently not strictly enforced there is supposed to be only one writer to the cgroup tree, and on most systems that is systemd. Interesting, I didn't know that. I can scrap that branch I started then :) Anyway, what precisely are you trying to do? Even using the kernel APIs, we still get bug reports about crazy CPU use and/or disk use. Since employing sched_setscheduler(), I think the situation is much better, but some people really want Tracker to a) only perform when they're away from their computer OR b) be completely unnoticeable. Now, we can do a) without cgroups, but I believe b) would be better done using cgroups. Why don't the kernel API calls pointed out above not suffice? I think it depends on the API and who you talk to. For me, the APIs work quite well. However, we still get bug reports. I find this quite hard to quantify personally because the filesystems, hardware and version of Tracker all come into play and can make quite some difference. The one API that doesn't really work for us is setrlimit(), mainly because we have to guess the memory threshold (which we never get right) and we get a lot of SIGABRTs that get reported as bugs. I suppose we could catch SIGABRT and exit gracefully, but lately, we've agreed (as a team) that if an extractor or library we depend on uses 2Gb of memory and brings a smaller system to its knees, it's a bug and we should just fix the extractor/library, not try to compensate for it. Sadly, there are always these sort of bugs and it's precisely why tracker-extract is a separate process. I suppose what we're left with is cgroups that would really bottleneck the process (tracker-extract is the main one I am thinking of here) in terms of memory and disk use. Thanks Lennart, any suggestions you have would be appreciated :) -- Regards, Martyn ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
On Tue, 21.10.14 03:01, Ronny Chevalier (chevalier.ro...@gmail.com) wrote:
+static int get_editors(char ***editors) {
+char **tmp_editors = strv_new(nano, vim, vi, NULL);
Please avoid calling functions and declaring variables in one line.
Also, there's an OOM check missing for this line.
+char *editor;
+
+/* SYSTEMD_EDITOR takes precedence over EDITOR which takes
precedence over VISUAL
+ * If neither SYSTEMD_EDITOR nor EDITOR nor VISUAL are present,
+ * we try to execute well known editors
+ */
+editor = getenv(SYSTEMD_EDITOR);
+if (!editor)
+editor = getenv(EDITOR);
+if (!editor)
+editor = getenv(VISUAL);
+
+if (editor) {
+int r;
+
+editor = strdup(editor);
+if (!editor)
+return log_oom();
+
+r = strv_consume_prepend(tmp_editors, editor);
+if (r 0)
+return log_oom();
+}
+
+*editors = tmp_editors;
+
+return 0;
+}
Hmm, I don't like this bit. Instead of figuring out the editor in one
step, and then executing it, I'd really prefer if we'd try to execute
the editors one-by-one until we find one that works. When we invoke
the pager we follow a similar logic.
Actually that's what I do. I fill a strv of editors and try to execute
each one of them, until one works. But we fail if the error is
different than No such file or directory.
This function (get_editors) is used in run_editor, where I do a
STRV_FOREACH in the child to try to execute every editor, with the one
comming from SYSTEMD_EDITOR or EDITOR or VISUAL, first.
Oh, well, indeed. Hmm, so I figure I'd just prefer if we wouldn't have
to allocate memory for this. or in other words, I'd like to see an alg
like this:
if (fork() == 0) {
...
e = getenv(SYSTEMD_EDITOR);
if (!e)
e = getenv(EDITOR);
if (!e)
e = getenv(VISUAL);
if (e)
execlp(e, ...);
execlp(nano, ...);
execlp(vim, ...);
execlp(vi, ...);
...
}
That way we wouldn't have to allocate an strv, and just open code the
search logic. This would then mirror nicely how the pager logic works...
Or maybe, you meant that if the one coming from SYSTEMD_EDITOR failed,
we should try to execute EDITOR, then VISUAL, too? Because I think we
should do that, but with the current code we don't.
I think it's OK checking only the first of SYSTEMD_EDITOR, EDITOR,
VISUAL that exists, and then proceed with nano/vim/vi
+if (arg_transport != BUS_TRANSPORT_LOCAL) {
+log_error(Cannot remotely edit units);
+return -EINVAL;
+}
+
+if (arg_runtime) {
+log_error(Cannot edit runtime units);
+return -EINVAL;
+}
Hmm, why not support this? And imply --runtime or so?
You are right, I forgot to ask a question about this one, sorry.
If there is a unit in /etc/systemd/system/ and someone wants to edit
this unit temporarily it will run systemctl --runtime myunit. The
problem is the unit will be copied in the /run/ directory but this
directory do not have precedence over /etc/systemd/, so when this unit
will be restarted, it won't take into account the one in /run/, right?
(My understanding of this come from src/shared/path-lookup.c)
Indeed. We should probably print an error in this case and refuse
operation. However we should support adding a --runtime .d/ snippet if
a unit file is in /etc, because that will actually work fine.
Furthermore, if I recall correctly the FragmentPath of a runtime
unit is empty?
No, it should be set properly like in all other cases really.
So, if I'm wrong and systemctl understands that when we do restart
with --runtime it means to execute the one from /run/, well I can
certainly add support for this.
If I'm a right, I do not see the point because it will not be easy to
understand for a user that it can temporarily edit a unit if it's in
/usr/lib/systemd/ but not in /etc/.
Yeah it is suprising but I think we could explain it nicely in an
error.
I'd merge your patch with or without this btw, I think it would just
be nice to support --runtime to the level we can support it, but it's
certainly just the cherry on top.
Lennart
--
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Tue, Oct 21, 2014 at 10:53 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Am Montag, 6. Oktober 2014, 21:53:04 schrieb Zbigniew Jędrzejewski-Szmek: Systemd-shim provides some functionality that systemd-sysv provides, and allows admins to use init systems other than systemd while still installing things like brasero. I think this is a great thing, except I wonder why the systemd project didn't separate this functionality from the init system in the first place. Systemd-shim is a duplication of effort. Not only that, but it must time its releases with the releases of systemd-sysv. That's no big deal for Debian's stable release, but it can be problematic in Debian's unstable and testing branches. Because it's additional work and complications. Apparently providing systemd functionality without systemd running is not a trivial undertaking, as the lag between systemd and systemd-shim releases shows. I could spend my time for open source development on a (technically) pointless (from my vantage point) split, but I prefer to improve systemd instead. Debian as a project already paid significant to support systemd as an alternative, when systemd version was held back for a year waiting for systemd-shim to catch up. This is certainly not the last time. [snip] I think exactly this kind of attitude is what triggers most of the polarity around systemd. Its like We don´t force systemd on anyone, but we provide as lots of functionality tightly coupled with it and if you do not implement it elsewhere, i.e. catch up, you need systemd anyway. While I believe it is *extra* work to separate functionality, I think it is *important* work. It would raise the acceptance of systemd, it would reduce the polarity it triggers. And it would make it more interoperable. Right now people are duplicating systemd stuff in several other areas. The reluctance to adopt systemd with certain distributions creates friction. Other implementations need to catch up and at any point in time may not be compatible. So, aside from it being additional work, is there any *solid* or even *unavoidable* technical reason to couple functionality that tightly? Wherever I look free software projects do great extra work to modularize and separate out functionality that can be separate. For a reason. See KDE community for example. They spend years of development work into separating things out into separate packages and have a clear ruling on what may depend on what. There are other examples for sure, OpenStack for example, while I do not yet know it in detail consists of a ton of separate packages in Debian. So or so… I think its this kind of attitude that triggers most of the polarity and split. Most of the modules in systemd do not depend on eachother. However, logind does depend on the cgroups dbus API of systemd (PID1). In the past logind interacted with the cgroups fs directly, but this had to be changed as the kernel is moving to a model where only one process may be in charge of cgroups. On systemd systems that one process is PID1 (why that is so has been discussed elsewhere so won't repeat that), so anything needing to deal with cgroups needs to go through the systemd API. I hope that explains why logind has to depend on systemd (or at least something implementing the systemd API). Cheers, Tom ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Dienstag, 7. Oktober 2014, 23:40:45 schrieb Uoti Urpala: On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote: My question really isn't why are the Debian dependencies the way they are. I understand that. I was trying to highlight the strange situation of a desktop application requiring a particular init system. I *think* this is a result of the init portion of systemd being bundled together with the logind portion of systemd. To me (unfamiliar with the systemd code) those two functions seem distinct enough to merit being separate. Debian can't easily separate what the systemd devs have developed as a single binary, so we end up with these strange dependency chains. Single binary is false of course. Logind is developed as a separate program, which is why systemd-shim is possible at all. AFAIK the actual relevant dependencies go as follows: First, there's a good reason why logind requires cgroup functionality. And there's a good reason why cgroup functionality is best implemented together with init (see http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/ for more info). So it's not quite directly logind has to depend on systemd as init, but logind has to depend on the system having cgroup support, and there's no equally good cgroup support available for inits other than systemd. It is possible to provide the relevant cgroup interfaces in or on top of another init, as the systemd-sysv + cgmanager combination attempts to do. But it is not trivial to do, as bugs and implementation delays with that combo have shown, and it's quite likely that there will be more problems in the future. It's not a particularly good idea to use the less-tested and less-reliable systemd-shim instead of the more reliable systemd. Thus the overall result is that yes, it does make sense to switch machines to systemd when you add certain functionality, even if that functionality does not appear to be directly tied to the init system at first glance. I can´t point my finger at it why… but I really feel uncomfortable with this tight coupling. I think that the systemd-shim + cgmanager combination may have bugs does not *prove* that such a think isn´t workable. But due to the design choice of you systemd developers to have it all in one people who do believe that separating different functionality is for the long term benefit of interoperability and avoiding what I´d call vendor lock in, although I clearly see that systemd upstream community isn´t a single vendor as in one company who produces a comercial product. In other areas where the Linux kernel requires user space helpers things initially have been separated. The whole do it all in once hald daemon was split into udev, libudev, udisks and upower… yet now, systemd and udev is in one upstream again. What speaks against putting the cgroup functionality and other stuff from systemd into shared libraries with clearly defined APIs – which are documented and where the documentation is the reference, not the source code – and allowing it to be used elsewhere? Then systemd may use it as PID 1, but if someother wants to use it in own project, can use it as well. I consider cgroups as part of the kernel API and I highly dislike the battle on which of the available solutions will get control over it. Actually I still think the API is broke if it cannot allow for mutiple processes accessing it. I don´t know an easy way to fix it, but I think such a kind of API as kernel interface… anyone can read a file, mount a filesystem, open a network socket, set a nice value depending on permissions but when it comes to control groups it is down to one to rule them all. I can´t help it, but this just seems utterly broke to me. I can´t help it but I don´t consider this to be a sane operating system API. I never thought much about my init system until recently. I never really had any complaints with SysV init, although I do recognize that systemd provides real improvements for some use cases. So for me as a sysadmin the wisest thing to do is stick with what I already know, as long as it's working well enough (and for me, it is). The issue with I should be able to stay with sysvinit because it worked fine for me is that keeping sysvinit working is COSTLY. The reason sysvinit used to mostly work was not that it would have been a reliable system - it mostly worked because people kept using a lot of effort to work around and paper over various issues that kept popping up. And there's no justification to keep up that effort for the minority who wants to stay with sysvinit. So, you can keep running your old systems unchanged if you want, but you shouldn't expect to be able to upgrade them or install new software without switching to systemd. What you write here basically comes down to a forced adoption. At least this is how I receive what you write. I know understand much more clearly on why
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
On Sat, 18.10.14 18:30, Ronny Chevalier (chevalier.ro...@gmail.com) wrote:
Three more suggestions after thinking about this a bit more...
+static int unit_file_drop_in(const char *unit_name, const char *config_home,
char **new_path) {
+char *tmp_path;
+int r;
+
+assert(unit_name);
+assert(new_path);
+
+switch (arg_scope) {
+case UNIT_FILE_SYSTEM:
+tmp_path = strjoin(SYSTEM_CONFIG_UNIT_PATH, /,
unit_name, .d/amendments.conf, NULL);
+break;
+case UNIT_FILE_GLOBAL:
+tmp_path = strjoin(USER_CONFIG_UNIT_PATH, /,
unit_name, .d/amendments.conf, NULL);
+break;
+case UNIT_FILE_USER:
+assert(config_home);
+tmp_path = strjoin(config_home, /, unit_name,
.d/amendments.conf, NULL);
+break;
+default:
+assert_not_reached(Invalid scope);
+}
+if (!tmp_path)
+return log_oom();
+
+r = mkdir_parents(tmp_path, 0755);
+if (r 0) {
+log_error(Failed to create directories for %s: %s,
tmp_path, strerror(-r));
+free(tmp_path);
+return r;
+}
+
+*new_path = tmp_path;
+
+return 0;
I think we should really take some inspiration here from how git
prepares the editor when asking for commit messages: adding a really
useful commented help text to the end of the new file to edit would be
really awesome. More specifically, I think we should include an output
equivalent to systemctl cat's add the end, commented and prefixed
with some clear marker that we can then use to remove the bits again
before saving things.
+if (arg_scope == UNIT_FILE_USER
+|| startswith(fragment_path, SYSTEM_CONFIG_UNIT_PATH)
+|| startswith(fragment_path, USER_CONFIG_UNIT_PATH)) {
Please use path_startswith() for this.
+if (access(tmp_path, F_OK) == 0) {
+char response;
+
+r = ask_char(response, yn, %s already exists, are you
sure to overwrite it with %s? [(y)es, (n)o] , tmp_path, fragment_path);
+if (r 0) {
+free(tmp_path);
+return r;
+}
Hmm, when precisely is this actually triggered? I mean, if the file
already exists in /etc we should just edit it there...
Lennart
--
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Tue, 21.10.14 10:53, Martin Steigerwald (mar...@lichtvoll.de) wrote: So, aside from it being additional work, is there any *solid* or even *unavoidable* technical reason to couple functionality that tightly? Yes, there always is. For logind for example we need to be able to group the processes of a session, so that we can keep track of them, list them, kill them, get notifications about them, and so on. For that we need the scope concept of PID1. That's why logind talks to PID 1. You know, it really annoys me if you imply that we just made these choices because we are assholes. We use the APIs we use because we need their technical functionality. It's that simple. It would be great if you'd grant us the benefit of the doubt at least, instead of implying anything else. Wherever I look free software projects do great extra work to modularize and separate out functionality that can be separate. For a reason. See KDE community for example. They spend years of development work into separating things out into separate packages and have a clear ruling on what may depend on what. There are other examples for sure, OpenStack for example, while I do not yet know it in detail consists of a ton of separate packages in Debian. Well, we are not KDE, and not OpenStack. We provide a basic toolbox to build an OS from. Compare us with Busybox if you must. I don't hear you complaining about busybox all the time! So or so… I think its this kind of attitude that triggers most of the polarity and split. Well, our priority is to solve technical problems in a way we perceive elegant and minimal. Your priority appears to be appeasing people who prefer religious reasoning over technical reasoning. I am pretty sure you cannot appease those, and we will not compromise our technical necessesities for that. Anyway, can we please end this discussion on this ML please? Please continue this somewhere else, this ML is really for technical discussions. Sorry, Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Mittwoch, 8. Oktober 2014, 10:54:00 schrieb Lennart Poettering: On Tue, 07.10.14 23:40, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote: My question really isn't why are the Debian dependencies the way they are. I understand that. I was trying to highlight the strange situation of a desktop application requiring a particular init system. I *think* this is a result of the init portion of systemd being bundled together with the logind portion of systemd. To me (unfamiliar with the systemd code) those two functions seem distinct enough to merit being separate. Debian can't easily separate what the systemd devs have developed as a single binary, so we end up with these strange dependency chains. Single binary is false of course. Logind is developed as a separate program, which is why systemd-shim is possible at all. AFAIK the actual relevant dependencies go as follows: First, there's a good reason why logind requires cgroup functionality. And there's a good reason why cgroup functionality is best implemented together with init (see http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/ for more info). So it's not quite directly logind has to depend on systemd as init, but logind has to depend on the system having cgroup support, and there's no equally good cgroup support available for inits other than systemd. It is possible to provide the relevant cgroup interfaces in or on top of another init, as the systemd-sysv + cgmanager combination attempts to do. But it is not trivial to do, as bugs and implementation delays with that combo have shown, and it's quite likely that there will be more problems in the future. It's not a particularly good idea to use the less-tested and less-reliable systemd-shim instead of the more reliable systemd. Thus the overall result is that yes, it does make sense to switch machines to systemd when you add certain functionality, even if that functionality does not appear to be directly tied to the init system at first glance. Also note that the systemd concepts logind makes use of are also exported in its own API. The scopes and slices that logind uses are exposed on its bus objects, and they are used by tools like loginctl to do their work. The scopes and slices concept does not exist elsewhere, and there's nothing comparable around, so even if we wanted we couldn't make logind work on anything else. Then why in the first hand are the scopes and slices concepts within systemd *tightly* coupled when it is functionality that makes sense to be utilitizes in a component that provides a different functionality. I wonder what functionality systemd provides right now in one more or less tightly coupled package: - early boot process - managing kernel drivers and device files (via coupled udev) - resource control for process groups - logging - core dumping - network while I think a distro-unified way to approaching network that replaces the current distro specific ways like ifupdown and so on, why does it have to be coupled with systemd? - cron, although that at least in Debian is separate as systemd-cron Thats rather much. Really rather much. Way more than traditonal sysvinit covered. What of this actually *needs* to be coupled with systemd? What of this needs to be coupled tightly to Linux kernel? systemd is driving a road where its all of this functionality by default is only available for Linux and where upstream said they just won´t accept patches – is that still true? – for portability. systemd provides more and more functionality that desktops like to use, that other tools like to use. When Microsoft back then did something like this it was called Embrace, Extend and Extinguish¹… a) Embrace: systemd provides functionality that is mostly compatible or similar with what sysvinit and other components provided b) Extend: systemd extends on that functionality and makes it more and more difficult for desktops and apps to ignore that kind of functionality offers c) Extinguish: The scope of systemd becomes so broad, the functionality it offers in a – admittedly – convenient package to often needed and used, that other ways of providing the functionality are extinguished. Really… it matches quite closely. I don´t think this was your plan. But… when looking at the currently visible outcome this is quite exactly what is happening here. So if you can follow this just a tiny bit: Do you start to understand why systemd triggers the uproar, polarity and split so obvious that one needs to have hands before both eyes for not seeing it? So as I see it: there are *real* concerns. And if systemd is meant to be a tool for users and admins, I *highly* and *strongly* recommend that you as systemd closely look at these concerns. Unless you are willing to trigger the polarity, the split and the uproar that it
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Tue, Oct 21, 2014 at 11:08 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Then systemd may use it as PID 1, but if someother wants to use it in own project, can use it as well. I consider cgroups as part of the kernel API and I highly dislike the battle on which of the available solutions will get control over it. Actually I still think the API is broke if it cannot allow for mutiple processes accessing it. I don´t know an easy way to fix it, but I think such a kind of API as kernel interface… anyone can read a file, mount a filesystem, open a network socket, set a nice value depending on permissions but when it comes to control groups it is down to one to rule them all. I can´t help it, but this just seems utterly broke to me. I can´t help it but I don´t consider this to be a sane operating system API. Note that the maintainers of the kernel-side cgroup API (primarily Tejun Heo, AFAIK) consider the current interface insane. In the future, the kernel will expect a single userspace process to control a single hierarchy. Systemd has already been adapted to provide this schema using the current API. See http://lists.freedesktop.org/archives/systemd-devel/2013-June/011521.html . ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Dienstag, 21. Oktober 2014, 11:07:01 schrieben Sie: On Tue, Oct 21, 2014 at 10:53 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Am Montag, 6. Oktober 2014, 21:53:04 schrieb Zbigniew Jędrzejewski-Szmek: Systemd-shim provides some functionality that systemd-sysv provides, and allows admins to use init systems other than systemd while still installing things like brasero. I think this is a great thing, except I wonder why the systemd project didn't separate this functionality from the init system in the first place. Systemd-shim is a duplication of effort. Not only that, but it must time its releases with the releases of systemd-sysv. That's no big deal for Debian's stable release, but it can be problematic in Debian's unstable and testing branches. Because it's additional work and complications. Apparently providing systemd functionality without systemd running is not a trivial undertaking, as the lag between systemd and systemd-shim releases shows. I could spend my time for open source development on a (technically) pointless (from my vantage point) split, but I prefer to improve systemd instead. Debian as a project already paid significant to support systemd as an alternative, when systemd version was held back for a year waiting for systemd-shim to catch up. This is certainly not the last time. [snip] I think exactly this kind of attitude is what triggers most of the polarity around systemd. Its like We don´t force systemd on anyone, but we provide as lots of functionality tightly coupled with it and if you do not implement it elsewhere, i.e. catch up, you need systemd anyway. While I believe it is *extra* work to separate functionality, I think it is *important* work. It would raise the acceptance of systemd, it would reduce the polarity it triggers. And it would make it more interoperable. Right now people are duplicating systemd stuff in several other areas. The reluctance to adopt systemd with certain distributions creates friction. Other implementations need to catch up and at any point in time may not be compatible. So, aside from it being additional work, is there any *solid* or even *unavoidable* technical reason to couple functionality that tightly? Wherever I look free software projects do great extra work to modularize and separate out functionality that can be separate. For a reason. See KDE community for example. They spend years of development work into separating things out into separate packages and have a clear ruling on what may depend on what. There are other examples for sure, OpenStack for example, while I do not yet know it in detail consists of a ton of separate packages in Debian. So or so… I think its this kind of attitude that triggers most of the polarity and split. Most of the modules in systemd do not depend on eachother. However, logind does depend on the cgroups dbus API of systemd (PID1). In the past logind interacted with the cgroups fs directly, but this had to be changed as the kernel is moving to a model where only one process may be in charge of cgroups. On systemd systems that one process is PID1 (why that is so has been discussed elsewhere so won't repeat that), so anything needing to deal with cgroups needs to go through the systemd API. I hope that explains why logind has to depend on systemd (or at least something implementing the systemd API). Thanks, Tom. As explained in another post – I didn´t read all the new answers… but it makes sense to do so – actually I feel uncomfortable about the new cgroup API. Its a kernel API. The Linux Kernel is a kernel supporting preemptive multitasking. Yet the new cgroup API requires or at least strongly recommends that only one process uses it. And… additionally to that that is a change that has been proposed by developers that to me oppinion are quite near or even working for systemd upstream. To me a single-process-usable API in the kernel seems broke. It may be hard to get the locking right… but still… I think the current cgroup API is still as broke as it can get. The unified hierarchy approach makes some sense to me, but requiring that only one process modifies it, instead of providing an API mutiple processes can use just seems plain broke to me. And I know, thats probably something to raise on LKML. And I may be inclined to do so. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Tue, 21.10.14 11:08, Martin Steigerwald (mar...@lichtvoll.de) wrote: systemd as init, but logind has to depend on the system having cgroup support, and there's no equally good cgroup support available for inits other than systemd. It is possible to provide the relevant cgroup interfaces in or on top of another init, as the systemd-sysv + cgmanager combination attempts to do. But it is not trivial to do, as bugs and implementation delays with that combo have shown, and it's quite likely that there will be more problems in the future. It's not a particularly good idea to use the less-tested and less-reliable systemd-shim instead of the more reliable systemd. Thus the overall result is that yes, it does make sense to switch machines to systemd when you add certain functionality, even if that functionality does not appear to be directly tied to the init system at first glance. I can´t point my finger at it why… but I really feel uncomfortable with this tight coupling. I think that the systemd-shim + cgmanager combination may have bugs does not *prove* that such a think isn´t workable. Well, you can do tons of things, that simple fact doesn't really make all of them a good idea. PID1 is supposed to start and maintain services. In our eyes this is best done with cgroups, because that allows us to keep track of the service processes, label them, list them, kill them, get notifications about them. All this functionality is implicitly done by the kernel for us, we just need to make use of it, with a few mkdir()s and open()+write()s, awesome! That's why we use it. Now, of course you can split that out in multiple daemons, but then you have to do IPC between these daemons. That's already bad enough, for complexity and stability reasons. However, what's worse is that it creates a chicken and egg problem: if PID forks of a service like cgmanager in order to *manage* a service like cgmanager, then you have a cyclic dependency. You can of course hackishly work around this, but this would mean to exclude cgmanager from service supervision. And moreover: for what even? For executing a few mkdir()s and write()s out-of-process? Really? Hence: while these things are certainly possible I am strongly of the opinion that they would be a systematically bad design, and we will hence not do such a fuckup in systemd. Sorry. But due to the design choice of you systemd developers to have it all in one people who do believe that separating different functionality is for the long term benefit of interoperability and avoiding what I´d call vendor lock in, although I clearly see that systemd upstream community isn´t a single vendor as in one company who produces a comercial product. We are open source. There is no such thing as vendor lockin. You are just fudding around now. I really don't appreciate what you are writing here. This has been discussed a million times before. If you want to discuss this again then find another forum. In other areas where the Linux kernel requires user space helpers things initially have been separated. The whole do it all in once hald daemon was split into udev, libudev, udisks and upower… yet now, systemd and udev is in one upstream again. I think you misunderstand the history of hal really. What speaks against putting the cgroup functionality and other stuff from systemd into shared libraries with clearly defined APIs – which are documented and where the documentation is the reference, not the source code – and allowing it to be used elsewhere? See above. Our APIs are actually pretty well documented, in documentation, not source code. You seem to imply otherwise. What do you want more documentation for? Can you be more explicit please? Then systemd may use it as PID 1, but if someother wants to use it in own project, can use it as well. I consider cgroups as part of the kernel API and I highly dislike the battle on which of the available solutions will get control over it. Actually I still think the API is broke if it cannot allow for mutiple processes accessing it. I don´t know an easy way to fix it, but I think such a kind of API as kernel interface… anyone can read a file, mount a filesystem, open a network socket, set a nice value depending on permissions but when it comes to control groups it is down to one to rule them all. I can´t help it, but this just seems utterly broke to me. Well, if you think the cgroup interface is broken, please bring this up with the kernel folks not us. I can´t help it but I don´t consider this to be a sane operating system API. Maybe instead of criticising the kernel folks and us for technical choices you should really get your hands dirty and try to make it better? The issue with I should be able to stay with sysvinit because it worked fine for me is that keeping sysvinit working is COSTLY. The reason sysvinit used to mostly work was not that it would have been a
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Dienstag, 21. Oktober 2014, 11:26:16 schrieben Sie: On Tue, Oct 21, 2014 at 11:08 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Then systemd may use it as PID 1, but if someother wants to use it in own project, can use it as well. I consider cgroups as part of the kernel API and I highly dislike the battle on which of the available solutions will get control over it. Actually I still think the API is broke if it cannot allow for mutiple processes accessing it. I don´t know an easy way to fix it, but I think such a kind of API as kernel interface… anyone can read a file, mount a filesystem, open a network socket, set a nice value depending on permissions but when it comes to control groups it is down to one to rule them all. I can´t help it, but this just seems utterly broke to me. I can´t help it but I don´t consider this to be a sane operating system API. Note that the maintainers of the kernel-side cgroup API (primarily Tejun Heo, AFAIK) consider the current interface insane. In the future, the kernel will expect a single userspace process to control a single hierarchy. Systemd has already been adapted to provide this schema using the current API. See http://lists.freedesktop.org/archives/systemd-devel/2013-June/011521.html Well I concur with that. I think the mutiple hierarchy API is inconsistent and insane. I see the point of having a unified hierarchy. But I never understand the requirement of one process controlling it and then having a battle over which process it will be. Whereever else we had something like this, it was tightly reduced to one functionality and available separate. Like udev, but still tightly coupled with the kernel and free to anyone to use – *just* this part of the functionality. Just like the Performance Events with the perf tools. Or iproute. Or you name it: Everyone providing one functionality. And… in its part being replacable. But with systemd it appears to be an all or nothing approach. I don´t only get control groups, but everything else… or else I need to duplicate the functionality myself. That just doesn´t make any sense to me. For udev I had the impression it was some quite approved part of the Linux kernel userspace utility. So while I think the old mutiple hierarchy API is broke, and heck I teached it in my Linux Performance analysis and tuning courses… I do think the new API is broke as well. Sometimes I think better ditch it all and redo it from scratch. Anyone can open a file, mount a fs, set a nice value, but only one process can do resource control of process groups via cgroups. This doesn´t make any sense to me. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Tue, Oct 21, 2014 at 11:35 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Am Dienstag, 21. Oktober 2014, 11:07:01 schrieben Sie: On Tue, Oct 21, 2014 at 10:53 AM, Martin Steigerwald mar...@lichtvoll.de wrote: Am Montag, 6. Oktober 2014, 21:53:04 schrieb Zbigniew Jędrzejewski-Szmek: Systemd-shim provides some functionality that systemd-sysv provides, and allows admins to use init systems other than systemd while still installing things like brasero. I think this is a great thing, except I wonder why the systemd project didn't separate this functionality from the init system in the first place. Systemd-shim is a duplication of effort. Not only that, but it must time its releases with the releases of systemd-sysv. That's no big deal for Debian's stable release, but it can be problematic in Debian's unstable and testing branches. Because it's additional work and complications. Apparently providing systemd functionality without systemd running is not a trivial undertaking, as the lag between systemd and systemd-shim releases shows. I could spend my time for open source development on a (technically) pointless (from my vantage point) split, but I prefer to improve systemd instead. Debian as a project already paid significant to support systemd as an alternative, when systemd version was held back for a year waiting for systemd-shim to catch up. This is certainly not the last time. [snip] I think exactly this kind of attitude is what triggers most of the polarity around systemd. Its like We don´t force systemd on anyone, but we provide as lots of functionality tightly coupled with it and if you do not implement it elsewhere, i.e. catch up, you need systemd anyway. While I believe it is *extra* work to separate functionality, I think it is *important* work. It would raise the acceptance of systemd, it would reduce the polarity it triggers. And it would make it more interoperable. Right now people are duplicating systemd stuff in several other areas. The reluctance to adopt systemd with certain distributions creates friction. Other implementations need to catch up and at any point in time may not be compatible. So, aside from it being additional work, is there any *solid* or even *unavoidable* technical reason to couple functionality that tightly? Wherever I look free software projects do great extra work to modularize and separate out functionality that can be separate. For a reason. See KDE community for example. They spend years of development work into separating things out into separate packages and have a clear ruling on what may depend on what. There are other examples for sure, OpenStack for example, while I do not yet know it in detail consists of a ton of separate packages in Debian. So or so… I think its this kind of attitude that triggers most of the polarity and split. Most of the modules in systemd do not depend on eachother. However, logind does depend on the cgroups dbus API of systemd (PID1). In the past logind interacted with the cgroups fs directly, but this had to be changed as the kernel is moving to a model where only one process may be in charge of cgroups. On systemd systems that one process is PID1 (why that is so has been discussed elsewhere so won't repeat that), so anything needing to deal with cgroups needs to go through the systemd API. I hope that explains why logind has to depend on systemd (or at least something implementing the systemd API). Thanks, Tom. As explained in another post – I didn´t read all the new answers… but it makes sense to do so – actually I feel uncomfortable about the new cgroup API. Its a kernel API. The Linux Kernel is a kernel supporting preemptive multitasking. Yet the new cgroup API requires or at least strongly recommends that only one process uses it. And… additionally to that that is a change that has been proposed by developers that to me oppinion are quite near or even working for systemd upstream. To me a single-process-usable API in the kernel seems broke. It may be hard to get the locking right… but still… I think the current cgroup API is still as broke as it can get. The unified hierarchy approach makes some sense to me, but requiring that only one process modifies it, instead of providing an API mutiple processes can use just seems plain broke to me. And I know, thats probably something to raise on LKML. And I may be inclined to do so. The systemd project does not have much (any?) influence over what interfaces the kernel provides, so I suggest you take your concerns up with them. That said, you will likely get further with technical arguments, and preferably an alternate implementation, than just claiming you have a bad feeling. Cheers, Tom ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Dienstag, 21. Oktober 2014, 11:21:36 schrieb Lennart Poettering: On Tue, 21.10.14 10:53, Martin Steigerwald (mar...@lichtvoll.de) wrote: So, aside from it being additional work, is there any *solid* or even *unavoidable* technical reason to couple functionality that tightly? Yes, there always is. For logind for example we need to be able to group the processes of a session, so that we can keep track of them, list them, kill them, get notifications about them, and so on. For that we need the scope concept of PID1. That's why logind talks to PID 1. You know, it really annoys me if you imply that we just made these choices because we are assholes. We use the APIs we use because we need their technical functionality. It's that simple. It would be great if you'd grant us the benefit of the doubt at least, instead of implying anything else. Lennart, I didn´t imply that, and I didn´t say that. Wherever I look free software projects do great extra work to modularize and separate out functionality that can be separate. For a reason. See KDE community for example. They spend years of development work into separating things out into separate packages and have a clear ruling on what may depend on what. There are other examples for sure, OpenStack for example, while I do not yet know it in detail consists of a ton of separate packages in Debian. Well, we are not KDE, and not OpenStack. We provide a basic toolbox to build an OS from. Compare us with Busybox if you must. I don't hear you complaining about busybox all the time! Lennart, I get the impression you feel being accused. Yet I tried honestly to keep my mails to be polite and respectful. I tried to discuss about systemd and attitudes, not about persons. Busybox is highly more optional than systemd. I can use bash and coreutils, or mksh and BSD commands or whatnot. So or so… I think its this kind of attitude that triggers most of the polarity and split. Well, our priority is to solve technical problems in a way we perceive elegant and minimal. Your priority appears to be appeasing people who prefer religious reasoning over technical reasoning. I am pretty sure you cannot appease those, and we will not compromise our technical necessesities for that. Anyway, can we please end this discussion on this ML please? Please continue this somewhere else, this ML is really for technical discussions. Sorry, Well… actually I tried to discuss the concerns I and others have openly. I went through the hassle to provide the feedback where it matters… upstream… instead of joining the flamefests elsewhere or calling you names… … but it seems to me you are so sensitive to feedback that I don´t have the impression you even relate to the concerns I voiced here, which in part is a summary of the part of concerns I read elsewhere I see as being founded. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Tue, 21.10.14 11:47, Martin Steigerwald (mar...@lichtvoll.de) wrote: So or so… I think its this kind of attitude that triggers most of the polarity and split. Well, our priority is to solve technical problems in a way we perceive elegant and minimal. Your priority appears to be appeasing people who prefer religious reasoning over technical reasoning. I am pretty sure you cannot appease those, and we will not compromise our technical necessesities for that. Anyway, can we please end this discussion on this ML please? Please continue this somewhere else, this ML is really for technical discussions. Sorry, Well… actually I tried to discuss the concerns I and others have openly. I went through the hassle to provide the feedback where it matters… upstream… instead of joining the flamefests elsewhere or calling you names… … but it seems to me you are so sensitive to feedback that I don´t have the impression you even relate to the concerns I voiced here, which in part is a summary of the part of concerns I read elsewhere I see as being founded. We are always interested in technical feedback. We are not very interested in FUD mails that tell us how we'd force people, how we'd behave like microsoft and so on. That's not useful, that's pretty much only hurtful. Sorry, Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Dienstag, 21. Oktober 2014, 11:52:50 schrieb Lennart Poettering: When Microsoft back then did something like this it was called Embrace, Extend and Extinguish¹… Oh come on. You are just being a dick now. For now just this: Thats a personal accusation. I didn´t attack you personally. Please refrain from attacking me personally. Thats not the way of discussing I am willing to lower my communication into. I shared my thoughts and observations… thats all. To me what I perceive has similarities regarding the outcome. I didn´t say that it was your intention to create this. I specifically wrote: I don´t think this was your plan. But… when looking at the currently visible outcome this is quite exactly what is happening here. So I didn´t say anything about your intention. And how could I? I don´t know them. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Am Dienstag, 21. Oktober 2014, 11:59:09 schrieb Lennart Poettering: On Tue, 21.10.14 11:47, Martin Steigerwald (mar...@lichtvoll.de) wrote: So or so… I think its this kind of attitude that triggers most of the polarity and split. Well, our priority is to solve technical problems in a way we perceive elegant and minimal. Your priority appears to be appeasing people who prefer religious reasoning over technical reasoning. I am pretty sure you cannot appease those, and we will not compromise our technical necessesities for that. Anyway, can we please end this discussion on this ML please? Please continue this somewhere else, this ML is really for technical discussions. Sorry, Well… actually I tried to discuss the concerns I and others have openly. I went through the hassle to provide the feedback where it matters… upstream… instead of joining the flamefests elsewhere or calling you names… … but it seems to me you are so sensitive to feedback that I don´t have the impression you even relate to the concerns I voiced here, which in part is a summary of the part of concerns I read elsewhere I see as being founded. We are always interested in technical feedback. We are not very interested in FUD mails that tell us how we'd force people, how we'd behave like microsoft and so on. That's not useful, that's pretty much only hurtful. Sorry, I didn´t intend to hurt you. I am sorry if you used what I wrote to hurt yourself. I just compared regarding the outcome. And to me there are similarities regarding the outcome. I never said you intended this. I never said you behaved intently in this way. That is *where* I differ from any of the personal accusation stuff. I am not assuming bad intent. Not at all. But I see similarities in the outcome. -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to use cgroups for Tracker?
On Tue, 21.10.14 09:53, Martyn Russell (mar...@lanedo.com) wrote: On 20/10/14 21:12, Lennart Poettering wrote: On Tue, 14.10.14 15:35, Martyn Russell (mar...@lanedo.com) wrote: Hej Lennart, I am not entirely sure what cgroups would really give you that sched_setscheduler(), ioprio_set(), setrlimit() wouldn't give you It's another approach, more of a sandbox in my mind than a priority system (which the above APIs are really - possibly with the exception of setrlimit()). I will get on to APIs in a bit... Tracker has been used on embedded platforms in the past with its own cgroups because it was more effective than the other APIs. It was recently suggested again in the bug above. What functionality of cgroups were you precisely using? Anyway, what precisely are you trying to do? Even using the kernel APIs, we still get bug reports about crazy CPU use and/or disk use. Since employing sched_setscheduler(), I think the situation What precisely are you setting with sched_setscheulder() and ioprio_set()? is much better, but some people really want Tracker to a) only perform when they're away from their computer OR b) be completely unnoticeable. Now, we can do a) without cgroups, but I believe b) would be better done using cgroups. Well, it will always be noticable since it generates IO. It will always show up in top/iotop hence. If you want it to not interfere with other things beind done on the system use ioprio_set/sched_setscheduler to mark things as batch jobs. If you want to put hard bandwitdh limits on things, then I#d really advise not to, because you effectively just prolong with that what you want to do, keep the hw full ypowered on for longer that way. Hard bandwidth limits are something for accounting models, for implementing business plans, but I think it would be wrong to use them for hiding things in top/iotop. Why don't the kernel API calls pointed out above not suffice? I think it depends on the API and who you talk to. For me, the APIs work quite well. However, we still get bug reports. I find this quite hard to quantify personally because the filesystems, hardware and version of Tracker all come into play and can make quite some difference. What precisely are the bug reports about? The one API that doesn't really work for us is setrlimit(), mainly because we have to guess the memory threshold (which we never get right) and we get a lot of SIGABRTs that get reported as bugs. I suppose we could catch SIGABRT and exit gracefully, but lately, we've agreed (as a team) that if an extractor or library we depend on uses 2Gb of memory and brings a smaller system to its knees, it's a bug and we should just fix the extractor/library, not try to compensate for it. Sadly, there are always these sort of bugs and it's precisely why tracker-extract is a separate process. What functionality of setrlimit() are you precisely using? Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
Am 2014-10-20 17:05, schrieb Lennart Poettering:
I am sorry, but this is nothing we want to support. Monopolizing the
OS in /usr is what makes ProtectSystem= work. If you split things up
into many dirs then you will simply not get the same level of
protection. We will not try to list every possible dirs that the OS
might be split up to in systemd.
Why wouldn't you get the same level of protection, even if it is split
up? I mean, we are talking about /bin, /sbin, /lib and possibly some
pseudo-multiarch-variants for /lib - it's not like there are any other
directories, the /usr-merge is about moving /bin, /sbin and /lib to
/usr/$dir. I don't see that split-/usr with these additional
directories
protected would be inherently more insecure.
Note that your patch is likely to break systems that have the dirs
you
list as symlinks (which all systems that have /usr merged have).
But on those systems, systemd is going to be compiled without
HAVE_SPLIT_USR, right?
Also note that it hardcodes x86_64 peculiarities in an
arch-independent way, which looks pretty wrong too.
That is definitely a good point. Also note that /lib32 is not included
in the patch...
We are fine with supporting HAVE_SPLIT_USR work to the level where
things generally work, but given that ProtectSystem= is only an extra
layer of protection where nothing breaks if it doesn't fully protect
systems that haven't done the usr-merge I think applying this patch
is not useful.
But computer security often works as defence in depth. Of course
ProtectSystem is an additional security measure, and protecting /usr
alone is obviously still better than protecting nothing at all, but if
it is not much work to provide a slightly higher level of protection -
why not?
That all said, to add something constructive to this: If I put my
administrator hat on, I think I can come up with something that might
make all parties happy:
To start with, ProtectSystem does not include /opt. Now I don't suggest
adding that to the list: distributions by default do not install
anything there, therefore a vanilla installation of a reasonable
distribution will not be affected by this. And there might be software
which requires system services to be able to write to /opt (for
example,
think of some proprietary software that creates logfiles underneath
/opt
that you want to rotate, but want the logrotate-job to use
ProtectSystem, since it's run as root), so it's probably a good default
not to include it.
However, as an administrator, if I know what software is installed
underneath /opt, and that there is no need for services to write to it,
I might actually want to include it. Of course, I could add drop-ins
for
all services that use ProtectSystem to also have
ReadOnlyDirectories=/InaccessibleDirectories=, but that would mean as
an
administrator I have to track these services - and if an update
suddenly
changes service files to use ProtectSystem= for increased protection,
I'd likely miss that and not be able to add these directives.
Additionally, not all custom installations follow the FHS and put
everything in /opt. I've seen computing clusters where software was
installed centrally and mounted per NFS to some other top-level
directory that is not /opt and /usr.
Therefore, may I suggest to make this configurable in
/etc/systemd/system.conf:
- have an option such as ProtectSystemDirectories= that is a list
like the other *Directories= options for units
- set the default to /usr, -/boot (like currently)
- distributions can customize that to fit their needs (depending on
/lib{32,64}, split- or no-split-/usr, etc.)
- administrators may themselves add /opt and other directories if
needed
- still hard-code /etc for ProtectSystem=full
If you're willing to accept a patch for this, I'd provide one.
Christian
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
On Tue, 21.10.14 14:03, Christian Seiler (christ...@iwakd.de) wrote:
Am 2014-10-20 17:05, schrieb Lennart Poettering:
I am sorry, but this is nothing we want to support. Monopolizing the
OS in /usr is what makes ProtectSystem= work. If you split things up
into many dirs then you will simply not get the same level of
protection. We will not try to list every possible dirs that the OS
might be split up to in systemd.
Why wouldn't you get the same level of protection, even if it is split
up? I mean, we are talking about /bin, /sbin, /lib and possibly some
pseudo-multiarch-variants for /lib - it's not like there are any other
directories, the /usr-merge is about moving /bin, /sbin and /lib to
/usr/$dir. I don't see that split-/usr with these additional directories
protected would be inherently more insecure.
Well, for starters the bin/sbin merge is orthogonal to the /usr
merge. arch did both for example, fedora only one, debian none. gentoo
did a really weird usr merge where they didn't merge but just moved
all things breaking things all over the place. Suse also did a weird
usr merge. Moreover the lib dirs are different for all archs.
I really don't want to be in the business of guessing and
distuingishing the many different ways how distros applied
that. Instead we do the minimally useful stuff that works in full on
the usr-merged distros, but only provides limited protection on the
ones which did not merge this. But that's OK, this is just an
additional line of protection.
Note that your patch is likely to break systems that have the dirs you
list as symlinks (which all systems that have /usr merged have).
But on those systems, systemd is going to be compiled without
HAVE_SPLIT_USR, right?
No, not necessarily. see above.
We are fine with supporting HAVE_SPLIT_USR work to the level where
things generally work, but given that ProtectSystem= is only an extra
layer of protection where nothing breaks if it doesn't fully protect
systems that haven't done the usr-merge I think applying this patch
is not useful.
But computer security often works as defence in depth. Of course
ProtectSystem is an additional security measure, and protecting /usr
alone is obviously still better than protecting nothing at all, but if
it is not much work to provide a slightly higher level of protection -
why not?
because we want to keep our code minimal, and nice, and simple, and
easily explainable.
Mounting just /usr read-only is something that is minimal, nice,
simple and explainable. It is everywhere the same, and gets a good
message across regarding that upstream devs really should monopolize
their vendor code in /usr, and so on.
That all said, to add something constructive to this: If I put my
administrator hat on, I think I can come up with something that might
make all parties happy:
To start with, ProtectSystem does not include /opt. Now I don't suggest
adding that to the list: distributions by default do not install
anything there, therefore a vanilla installation of a reasonable
distribution will not be affected by this. And there might be software
which requires system services to be able to write to /opt (for example,
think of some proprietary software that creates logfiles underneath /opt
that you want to rotate, but want the logrotate-job to use
ProtectSystem, since it's run as root), so it's probably a good default
not to include it.
ProtectedSystem= is really supposed to be something that has very well
defined behaviour and only does something on the dirs we
*sufficiently* know that nothing breaks. We explicitly make no
assumptions on /opt because nobody knows right now what it is supposed
to be... Same for /usr, /bin, /sbin, and the other stuff Martin#s
patch added: we cannot make assumptions about it, it might be (and is
in real life) set up in different ways, and we don't want to be in
that game. However, what we do know is that /usr is something we can
mount read-only on all systems.
Therefore, may I suggest to make this configurable in
/etc/systemd/system.conf:
- have an option such as ProtectSystemDirectories= that is a list
like the other *Directories= options for units
- set the default to /usr, -/boot (like currently)
- distributions can customize that to fit their needs (depending on
/lib{32,64}, split- or no-split-/usr, etc.)
- administrators may themselves add /opt and other directories if
needed
- still hard-code /etc for ProtectSystem=full
If you're willing to accept a patch for this, I'd provide one.
I really disagree that this would be a good idea. We should give clear
guidelines how things should be set up here to take full benefit of
the functionality. Because this is about an agreement between the OS
people and the upstream developers of packages to run on the OS. We
want to make sure they can make the assumptions everywhere, which are
not configurable and behave differently everywhere. For example, I
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
On 21/10/14 13:03, Christian Seiler wrote: That is definitely a good point. Also note that /lib32 is not included in the patch... lib64 is part of the Linux/x86_64 platform ABI (the exact path /lib64/ld-linux-x86-64.so.2 is hard-coded into every Linux/x86_64 executable) so it cannot be considered legacy. Is there a Linux ABI where /lib32 has the same status? If not, then its priority is rather lower. Debian x86_64 (amd64) systems with bi-arch i386 binaries (ia32-libs) are the only instance of /lib32 that I can think of, and that setup is superseded by multiarch these days. S ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
Am 21.10.2014 um 14:38 schrieb Simon McVittie: On 21/10/14 13:03, Christian Seiler wrote: That is definitely a good point. Also note that /lib32 is not included in the patch... lib64 is part of the Linux/x86_64 platform ABI (the exact path /lib64/ld-linux-x86-64.so.2 is hard-coded into every Linux/x86_64 executable) so it cannot be considered legacy. Is there a Linux ABI where /lib32 has the same status? If not, then its priority is rather lower it don't matter after UsrMove: lib64 - usr/lib64 so the protection of /usr works as expected if some disto invents other dirs - the same logic if some distro don't follow that structure - no protection they can do the following in their unit-files ProtectSystem is nothing more than a nice addition ReadOnlyDirectories works even on old systemd versions ReadOnlyDirectories=/etc ReadOnlyDirectories=/usr ReadOnlyDirectories=/lib64 ReadOnlyDirectories=/lib32 signature.asc Description: OpenPGP digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to use cgroups for Tracker?
On 21/10/14 12:21, Lennart Poettering wrote: On Tue, 21.10.14 12:03, Martyn Russell (mar...@lanedo.com) wrote: What precisely are you setting with sched_setscheulder() and ioprio_set()? https://git.gnome.org/browse/tracker/tree/src/libtracker-common/tracker-sched.c#n29 and https://git.gnome.org/browse/tracker/tree/src/libtracker-common/tracker-ioprio.c#n131 Looks OK. is much better, but some people really want Tracker to a) only perform when they're away from their computer OR b) be completely unnoticeable. Now, we can do a) without cgroups, but I believe b) would be better done using cgroups. Well, it will always be noticable since it generates IO. It will always show up in top/iotop hence. If you want it to not interfere with other things beind done on the system use ioprio_set/sched_setscheduler to mark things as batch jobs. You're right. When we get reports of 100% CPU use I am not so concerned if the system is not doing anything else - but I suspect what leads people to find out about the 100% situation is related to their UI not responding. Hmm. I would always have assumed that tracker is strictly IO-bound, not CPU-bound, hence 100% sounds suspicious to me. What precisely is tracker doing there that it needs to crunch that much data? Just extracting some meta-data from a set of files doesn't sound like something CPU intensive. Tracker does quite a number of things that could require some level of processing power. Just some off the top of my head: - Parsing words any any language in large quantities of text - Unaccenting words - Unicode normalization - Case folding - Stemming and more. It also depends on which process or binary you're talking about, but extractors (like the one using poppler for PDFs) can easily require a LOT of processing power to handle complex PDFs. We only care about the text usually, but that's not always under our control unless we write our own extractor. On top of all this, you have a SPARQL engine which translates to SQL and then has to perform some DB operation - which adds to the processing requirement. It might be worth measuring again to see where this processing power is going. Why don't the kernel API calls pointed out above not suffice? I think it depends on the API and who you talk to. For me, the APIs work quite well. However, we still get bug reports. I find this quite hard to quantify personally because the filesystems, hardware and version of Tracker all come into play and can make quite some difference. What precisely are the bug reports about? Often 100% CPU use. Sometimes high memory use (but that's mostly related to tracker-extract). Disk use occasionally when someone is indexing a crazy data set. The latest bug report: https://bugzilla.gnome.org/show_bug.cgi?id=676713 Well, looking at that bug it appears to me that this is caused because you try to use inotify for something it shouldn't be used for: to recursively watch an entire directory subtree. If you fake recursive fs watching by recursively adding all subdirs to the inotify watch then of course you eat a lot of CPU. In our experience, watching a tree and seeing changes in that tree through inotify is not the expensive part (unless you're currency is FDs). It does depend on what operations are taking place. The bug report above is certainly asking for trouble :) but it's just an example. The appropriate fix for this is to not make use of inotify this way, which might mean fixing the kernel to provide recursive subscription to fs events for unpriviliged processes. Sorry if that's disappointing, but no amount of cgriups can work around that. Not at all. Actually, I really would like something like that in the kernel and user space has been asking for a while :) Don't try to work around limitations of kernel APIs by implementing inherently not scalabale algorithms in userspace. I mean, you implemented something that scales O(n) with n the numbers of dirs. That's what you need to fix, there's no way around that. Just looking for magic wands in cgroups and scheduling facilities to make an algorithm that fundamentally scales badly acceptable is not going to work. OK. Could I ask one more favour from you Lennart, could you possibly reply on the bug report where your fellow RedHat-ers :) suggest using cgroups? https://bugzilla.gnome.org/show_bug.cgi?id=737663#c6 The one API that doesn't really work for us is setrlimit(), mainly because we have to guess the memory threshold (which we never get right) and we get a lot of SIGABRTs that get reported as bugs. I suppose we could catch SIGABRT and exit gracefully, but lately, we've agreed (as a team) that if an extractor or library we depend on uses 2Gb of memory and brings a smaller system to its knees, it's a bug and we should just fix the extractor/library, not try to compensate for it. Sadly, there are always these sort of bugs and it's precisely why tracker-extract is a separate
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
Am 2014-10-21 14:28, schrieb Lennart Poettering: We explicitly make no assumptions on /opt because nobody knows right now what it is supposed to be... Sure, I wasn't disputing that point. Same for /usr, /bin, /sbin, and the other stuff Martin#s patch added: we cannot make assumptions about it, it might be (and is in real life) set up in different ways, and we don't want to be in that game. That's why I didn't suggest that you (as upstream) should be in that game, but that distributions and administrators should be able to do so themselves. Therefore, may I suggest to make this configurable in /etc/systemd/system.conf: [...] If you're willing to accept a patch for this, I'd provide one. I really disagree that this would be a good idea. We should give clear guidelines how things should be set up here to take full benefit of the functionality. Because this is about an agreement between the OS people and the upstream developers of packages to run on the OS. We want to make sure they can make the assumptions everywhere, which are not configurable and behave differently everywhere. For example, I really want that let's say apache sets ProtectSystem= and can be sure it will just not break things on our OSes. And because of that its impact should be only on the safe subset, and it should be the same on all installations, and not be subject to configuration. Debian's systemd package currently includes a variant of Martin's patch that does include additional directories. So your point that ProtectSystem= does the same thing on every distro is already not true. Of course, if you make it configurable, people can shoot themselves in the foot. But you already have a ton of global options in system.conf that can break a lot of software if people do stupid stuff with it: - set a global CapabilityBoundingSet= that's very restrictive - either the system doesn't boot at all because some essential stuff is completely missing - or it boots but some services don't work because they rely on certain things - set SystemCallArchitectures=native when non-native software is installed - set DefaultEnvironment=LD_TRACE_LOADED_OBJECTS=1 or the such - set DefaultTimeoutStartSec=1 to break any unit that takes longer than 1 second to start but doesn't set an own timeout because it assumes the default timeout is sane - DefaultLimitCPU=1 - ... Also, to go to your apache example: it's not clear that ProtectSystem= just making /usr readonly doesn't break things: I have seen DocumentRoots beneath /usr in the wild (/usr/local/www or the such), with people running dynamic webapps that had to write into that tree. If you then upgrade from a package version that did not include ProtectSystem= (perhaps because it only included a SysV init script) to a package version that does include ProtectSystem=, things will break. I actually agree with your sentiment of having an agreement between upstream developers and the core OS - I just think I would like to interpret the matter a little differently: To me, ProtectSystem= is supposed to be a protection of all the files a) installed on a system (not created by the user) and b) not subject to modification by typical services. (i.e. not a cache / status file / ...) For distros with /usr-move, this falls back to /usr and /boot (and /etc if =full). For other distros, there may be a few additional directories. And on a custom installation, it may include additional directories, such as /opt. If I am an upstream developer and ship a unit file with ProtectSystem=full, my expectations are that normal operation on directories that are supposed to contain data that is not put there at installation (such as /var, /tmp, /home, /srv, /run, ...) remain accessible, but that systemd will provide an extra layer of security around the rest of the installed system. As an upstream developer, I don't know where the distro the user is using decided to install stuff, whether it's in /usr or directly in /bin or wherever. And I don't really care about that and I don't WANT to care about that. If I really only want /usr to be read-only, I could just add ReadOnlyDirectories=/usr to the unit file and be done with it. But I don't want to care about distro details as an upstream developer, I want the setting to just work[tm] and do the right thing[tm]. The fact that it is a generically named option makes me actually expect an abstraction of distro differences. On the other hand, if I put my administrator hat on, as I said in the last mail, I will know what directories may be present additionally that could also fall under that umbrella. And if something breaks because I put in a stupid setting, that's my bad. By all means, put a big fat warning in the docs that this setting is dangerous to fiddle around with. I do see a good bit of orthogonality here: - Upstream developers can clearly expect that /usr-_type_ (!) stuff is protected
Re: [systemd-devel] [PATCH] shared: fix cppcheck warnings
Hi On Mon, Oct 20, 2014 at 10:06 PM, Boris Egorov wrote: Oh, you are right. This should probably be left as is. It should be reported to cppcheck http://sourceforge.net/p/cppcheck/wiki/Home/ Preferably with a small test case. Ignoring false positives makes it harder to figure out when there is an actual issue. Rahul ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 00/26] hashmap rewrite
On 10/20/2014 08:23 PM, Lennart Poettering wrote: On Thu, 16.10.14 09:50, Michal Schmidt (mschm...@redhat.com) wrote: Key changes that affect other code: - Sets and Hashmaps do not remember the insertion order anymore. They can still be iterated with *_FOREACH* or *_first*, but the order of entries is undefined. Hmm, this means the iteration logic will have to iteratively skip over empty buckets to be able to enumerate? Yes, to find occupied buckets, it does a sequential scan over the array of DIB values. That should be pretty fast. I figure that isn't too bad if we scale the number of buckets by the size of the hashmap anyway. Yes, though we never scale back down when many entries get removed from a hashmap (unless ALL of them get removed). It does not seem to be needed often. - There is a new type LinkedHashmap to use in the few cases where insertion order matters. The cases that I believe need it are converted in this patch series. Sounds good. Don't like the name though. I think OrderedHashmap would be better, since for the user of the data structure the different is all about the order, right? I saw the name LinkedHashmap used for this thing in Java/Android, so I went with that. I can rename it, no problem. Michal ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 21/26] hashmap: rewrite the implementation
On 10/20/2014 08:42 PM, Lennart Poettering wrote:
On Thu, 16.10.14 09:50, Michal Schmidt (mschm...@redhat.com) wrote:
+enum {
+HASHMAP_TYPE_PLAIN,
+HASHMAP_TYPE_LINKED,
+HASHMAP_TYPE_SET,
+__HASHMAP_TYPE_COUNT
+};
Why is this enum anonymous? Wouldn't it be nicer to give it a name? We
only use it internally only anyway...
So far we called the final counting entry of such enums _FOOBAR_MAX,
not __FOOBAR_COUNT, please stick to this naming scheme... (also iirc
the C standard reserves the double _ prefix for the language
itself. The kernel ignores that C standard issue so far, but we should
probably honour it.
OK, I'll improve these.
+uint8_t type:2; /* HASHMAP_TYPE_* */
Should probably be a named enum (see above)
[...]
+uint8_t has_indirect:1; /* whether indirect storage is
used */
Should really be a bool, no?
OK.
+static struct HashmapBase *__hashmap_new(const struct hash_ops *hash_ops,
uint8_t type HASHMAP_DEBUG_PARAMS) {
+HashmapBase *h;
+
+h = malloc0(all_hashmap_sizes[type]);
+if (!h)
+return NULL;
+
Hmm, a malloc() for each hashmap we allocate? Isn#t that prohibitively
expensive given how slow malloc() is? We have soo many hashmaps!
Is it really that slow? A small test program that does malloc(48) ten
million times (no freeing) runs for about 0.3 seconds on my laptop.
systemd allocates about 2000 hashmaps here. So these would take about
60 microseconds to malloc. Maybe the test program has it too easy due
to having to deal with no heap fragmentation. But then I'd expect the
allocations this small to be helped by glibc's fastbins...
I guess I'll take some measurements under qemu with no KVM :-)
+Set *_set_new(const struct hash_ops *hash_ops HASHMAP_DEBUG_PARAMS) {
+return (Set*) __hashmap_new(hash_ops, HASHMAP_TYPE_SET
HASHMAP_DEBUG_PASS_ARGS);
+}
Not liking the double underscores... I know this is kernel style, but it's
still in conflict with ISO C...
Can we maybe name this internal_xyz or so?
Will do so.
Michal
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH] util: introduce sethostname_idempotent
Function queries system hostname and applies changes only when necessary. Also,
migrate all client of sethostname to sethostname_idempotent while at it.
---
src/core/hostname-setup.c | 2 +-
src/hostname/hostnamed.c | 2 +-
src/nspawn/nspawn.c | 2 +-
src/shared/util.c | 20
src/shared/util.h | 2 ++
5 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/src/core/hostname-setup.c b/src/core/hostname-setup.c
index 8aa1cff..57baa79 100644
--- a/src/core/hostname-setup.c
+++ b/src/core/hostname-setup.c
@@ -82,7 +82,7 @@ int hostname_setup(void) {
hn = localhost;
}
-if (sethostname(hn, strlen(hn)) 0) {
+if (sethostname_idempotent(hn) 0) {
log_warning(Failed to set hostname to %s: %m, hn);
return -errno;
}
diff --git a/src/hostname/hostnamed.c b/src/hostname/hostnamed.c
index b6b5d52..3e5d434 100644
--- a/src/hostname/hostnamed.c
+++ b/src/hostname/hostnamed.c
@@ -287,7 +287,7 @@ static int context_update_kernel_hostname(Context *c) {
else
hn = localhost;
-if (sethostname(hn, strlen(hn)) 0)
+if (sethostname_idempotent(hn) 0)
return -errno;
return 0;
diff --git a/src/nspawn/nspawn.c b/src/nspawn/nspawn.c
index c567c8d..b6d9bc6 100644
--- a/src/nspawn/nspawn.c
+++ b/src/nspawn/nspawn.c
@@ -1289,7 +1289,7 @@ static int setup_hostname(void) {
if (arg_share_system)
return 0;
-if (sethostname(arg_machine, strlen(arg_machine)) 0)
+if (sethostname_idempotent(arg_machine) 0)
return -errno;
return 0;
diff --git a/src/shared/util.c b/src/shared/util.c
index 5f6249e..cb3bd2f 100644
--- a/src/shared/util.c
+++ b/src/shared/util.c
@@ -7164,3 +7164,23 @@ int free_and_strdup(char **p, const char *s) {
return 0;
}
+
+int sethostname_idempotent(const char *s) {
+int r = 0;
+char buf[HOST_NAME_MAX + 1] = {};
+
+assert(s);
+
+r = gethostname(buf, sizeof(buf));
+if (r 0)
+return -errno;
+
+if (streq(buf, s))
+return 0;
+
+r = sethostname(buf, strlen(buf));
+if (r 0)
+return -errno;
+
+return 0;
+}
diff --git a/src/shared/util.h b/src/shared/util.h
index 21a90a4..10cdc7b 100644
--- a/src/shared/util.h
+++ b/src/shared/util.h
@@ -996,3 +996,5 @@ int unquote_first_word(const char **p, char **ret);
int unquote_many_words(const char **p, ...) _sentinel_;
int free_and_strdup(char **p, const char *s);
+
+int sethostname_idempotent(const char *s);
--
1.8.3.1
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Unicode support in console after boot
On Tue, Oct 14, 2014 at 09:04:56AM +0200, Jan Synacek wrote: Michal Sekletar msekl...@redhat.com writes: On Mon, Oct 13, 2014 at 09:36:16AM +0200, Jan Synacek wrote: Hello, currently, unicode characters are not correctly displayed in the console. After login, when I run /usr/bin/unicode_start, unicode works fine. I tried to create a service file that runs this script, linking tty to stdout and stderr, but that didn't work. Is there a way how to turn on unicode support in console after boot using a service file? Or any other type of unit? Or is this something that has to be patched in the source (logind perhaps?)? Please try editing /usr/lib/systemd/system/systemd-vconsole-setup.service and remove RemainAfterExit=yes, then regenerate your initramfs image by running dracut command. Add back RemainAfterExit=yes to service file. Reboot. Yep, this helped. Could you please explain why? Also, I believe this should be fixed in all Fedora versions. I must admit I'm not sure why this workaround works. Maybe there is some race condition with some kernel initialization or settings get unapplied because of switch-root. Also, if we go with workaround in Fedora I think dracut needs to fixed to include version on unit file *without* RemainAfterExit=yes. Michal Michal Cheers, -- Jan Synacek Software Engineer, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- Jan Synacek Software Engineer, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] util: introduce sethostname_idempotent
On Tue, 21.10.14 18:32, Michal Sekletar (msekl...@redhat.com) wrote:
Function queries system hostname and applies changes only when necessary.
Also,
migrate all client of sethostname to sethostname_idempotent while at
it.
Looks pretty good!
+int sethostname_idempotent(const char *s) {
+int r = 0;
Initializing this to 0 is not really necessary as you override it anyway.
+char buf[HOST_NAME_MAX + 1] = {};
+
+assert(s);
+
+r = gethostname(buf, sizeof(buf));
+if (r 0)
+return -errno;
+
+if (streq(buf, s))
+return 0;
+
+r = sethostname(buf, strlen(buf));
+if (r 0)
+return -errno;
+
+return 0;
Not that it would really matter here, but given that we return an int
here anyway we might as well return 1 if we made any change, so that a
future caller might use the return code as an indication whether the
host name was changed or not returning negative would then
indicate error, returning 0 would mean no change and returning
postiive would mean a change was made.
+}
diff --git a/src/shared/util.h b/src/shared/util.h
index 21a90a4..10cdc7b 100644
--- a/src/shared/util.h
+++ b/src/shared/util.h
@@ -996,3 +996,5 @@ int unquote_first_word(const char **p, char **ret);
int unquote_many_words(const char **p, ...) _sentinel_;
int free_and_strdup(char **p, const char *s);
+
+int sethostname_idempotent(const char *s);
Go ahead and commit. Ideally with those two nitpicks fixed, but even
if you don't it's OK.
Lennart
--
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Unicode support in console after boot
On Tuesday 21 October 2014 at 19:03:17, Michal Sekletar wrote: On Tue, Oct 14, 2014 at 09:04:56AM +0200, Jan Synacek wrote: Michal Sekletar msekl...@redhat.com writes: On Mon, Oct 13, 2014 at 09:36:16AM +0200, Jan Synacek wrote: Hello, currently, unicode characters are not correctly displayed in the console. After login, when I run /usr/bin/unicode_start, unicode works fine. I tried to create a service file that runs this script, linking tty to stdout and stderr, but that didn't work. Is there a way how to turn on unicode support in console after boot using a service file? Or any other type of unit? Or is this something that has to be patched in the source (logind perhaps?)? Please try editing /usr/lib/systemd/system/systemd-vconsole-setup.service and remove RemainAfterExit=yes, then regenerate your initramfs image by running dracut command. Add back RemainAfterExit=yes to service file. Reboot. Yep, this helped. Could you please explain why? Also, I believe this should be fixed in all Fedora versions. I must admit I'm not sure why this workaround works. Maybe there is some race condition with some kernel initialization or settings get unapplied because of switch-root. Also, if we go with workaround in Fedora I think dracut needs to fixed to include version on unit file *without* RemainAfterExit=yes. IIUC, this makes unit to be re-run outside of initramfs, so the VT is set up twice, second time after switching the framebuffer driver. And the latter condition is not mandated by anything, it's just a coincidence... -- Ivan Shapovalov / intelfx / signature.asc Description: This is a digitally signed message part. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to use cgroups for Tracker?
On Tue, 21.10.14 14:48, Martyn Russell (mar...@lanedo.com) wrote: Hmm. I would always have assumed that tracker is strictly IO-bound, not CPU-bound, hence 100% sounds suspicious to me. What precisely is tracker doing there that it needs to crunch that much data? Just extracting some meta-data from a set of files doesn't sound like something CPU intensive. Tracker does quite a number of things that could require some level of processing power. Just some off the top of my head: - Parsing words any any language in large quantities of text - Unaccenting words - Unicode normalization - Case folding - Stemming and more. That all doesn't sound too excessive in CPU except if you index whole encyclopedias... But in this case I'd really time bound things: stop processing each file after 500ms or so of time spent on them. It also depends on which process or binary you're talking about, but extractors (like the one using poppler for PDFs) can easily require a LOT of processing power to handle complex PDFs. We only care about the text usually, but that's not always under our control unless we write our own extractor. Well, it certainly sounds like a great chance to work together with the poppler folks to figure out a way to only hand you the text. But either way, given the variable quality of the extractors it really sounds as if you want to indivudally run them out-of-process and then kill after a fixed time limit of 500ms and continue with the next one. Well, looking at that bug it appears to me that this is caused because you try to use inotify for something it shouldn't be used for: to recursively watch an entire directory subtree. If you fake recursive fs watching by recursively adding all subdirs to the inotify watch then of course you eat a lot of CPU. In our experience, watching a tree and seeing changes in that tree through inotify is not the expensive part (unless you're currency is FDs). It does depend on what operations are taking place. Well, the bug report you linked suggests it is an inotify add loop that is the culprit here... The appropriate fix for this is to not make use of inotify this way, which might mean fixing the kernel to provide recursive subscription to fs events for unpriviliged processes. Sorry if that's disappointing, but no amount of cgriups can work around that. Not at all. Actually, I really would like something like that in the kernel and user space has been asking for a while :) Well, I think I said this before: it's quite possible that the Linux kernel is not quite ready for something like Tracker. Apparently nobody is working to make it ready for Tracker though. So either you have to do the work yourself or find somebody to fix the missing bits (which would be to fix fanotify for unpriviliged clients). Don't try to work around limitations of kernel APIs by implementing inherently not scalabale algorithms in userspace. I mean, you implemented something that scales O(n) with n the numbers of dirs. That's what you need to fix, there's no way around that. Just looking for magic wands in cgroups and scheduling facilities to make an algorithm that fundamentally scales badly acceptable is not going to work. OK. Could I ask one more favour from you Lennart, could you possibly reply on the bug report where your fellow RedHat-ers :) suggest using cgroups? https://bugzilla.gnome.org/show_bug.cgi?id=737663#c6 Well, just link this thread there really, I think that should be enough... This is misleading, as RLIMIT_AS and RLIMIT_DATA limit address space, not actual memory usage. In particularly limiting RLIMIT_AS like this is actively wrong as this just breaks mmap(). I mean, this is the beauty of modern memory management: you can set up mappings, and they are relatively cheap and only are backed by real RAM as soon as you access them. really, you should not limit RLIMIT_AS, it's not for what you think it is. When would you use this functionality? I struggle to see a use case. Well RLIMIT_AS is certainly not useful for your purpose, that's true. As it turns out both RLIMIT_DATA and RLIMIT_RSS are NOPs these days. Which basically means, that the memory cgroup controller is the only technology that would come close, but there you'd still have a problem to find a value to initialize its limit too (also, as mentioned, this isn't open to unpriviliged processes just now). Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 00/26] hashmap rewrite
On Tue, 21.10.14 17:36, Michal Schmidt (mschm...@redhat.com) wrote: On 10/20/2014 08:23 PM, Lennart Poettering wrote: On Thu, 16.10.14 09:50, Michal Schmidt (mschm...@redhat.com) wrote: Key changes that affect other code: - Sets and Hashmaps do not remember the insertion order anymore. They can still be iterated with *_FOREACH* or *_first*, but the order of entries is undefined. Hmm, this means the iteration logic will have to iteratively skip over empty buckets to be able to enumerate? Yes, to find occupied buckets, it does a sequential scan over the array of DIB values. That should be pretty fast. I figure that isn't too bad if we scale the number of buckets by the size of the hashmap anyway. Yes, though we never scale back down when many entries get removed from a hashmap (unless ALL of them get removed). It does not seem to be needed often. - There is a new type LinkedHashmap to use in the few cases where insertion order matters. The cases that I believe need it are converted in this patch series. Sounds good. Don't like the name though. I think OrderedHashmap would be better, since for the user of the data structure the different is all about the order, right? I saw the name LinkedHashmap used for this thing in Java/Android, so I went with that. I can rename it, no problem. OrderedHashmap sounds more descriptive to me... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 21/26] hashmap: rewrite the implementation
On Tue, 21.10.14 17:50, Michal Schmidt (mschm...@redhat.com) wrote:
+static struct HashmapBase *__hashmap_new(const struct hash_ops *hash_ops,
uint8_t type HASHMAP_DEBUG_PARAMS) {
+HashmapBase *h;
+
+h = malloc0(all_hashmap_sizes[type]);
+if (!h)
+return NULL;
+
Hmm, a malloc() for each hashmap we allocate? Isn#t that prohibitively
expensive given how slow malloc() is? We have soo many hashmaps!
Is it really that slow? A small test program that does malloc(48) ten
million times (no freeing) runs for about 0.3 seconds on my laptop.
systemd allocates about 2000 hashmaps here. So these would take about
60 microseconds to malloc. Maybe the test program has it too easy due
to having to deal with no heap fragmentation. But then I'd expect the
allocations this small to be helped by glibc's fastbins...
I guess I'll take some measurements under qemu with no KVM :-)
Hmm, I did my testing back in callgrind back when I optimized systemd
a bit, and malloc() was by far the most expensive call of all. After
adding the allocation cache for hashmap items it was much better, but
still at the top of things. I then added an allocation cache also for
the hashmap heads themselves and everything appeared good enough to
me, and malloc() was not the most expensive call anymore (and the
allocation cache was neither...)
Would be interesting to run systemd in callgrind again with a
non-trivial number of units and see what costs these days. My last
callgrind optimizations I did years back now, things might have
changed quite a bit.
Lennart
--
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
On Tue, 21.10.14 15:57, Christian Seiler (christ...@iwakd.de) wrote: Am 2014-10-21 14:28, schrieb Lennart Poettering: We explicitly make no assumptions on /opt because nobody knows right now what it is supposed to be... Sure, I wasn't disputing that point. Same for /usr, /bin, /sbin, and the other stuff Martin#s patch added: we cannot make assumptions about it, it might be (and is in real life) set up in different ways, and we don't want to be in that game. That's why I didn't suggest that you (as upstream) should be in that game, but that distributions and administrators should be able to do so themselves. Will distros always are in charge, as they can for example merge Martin's patch downstream. (Though I'd be careful.) Therefore, may I suggest to make this configurable in /etc/systemd/system.conf: [...] If you're willing to accept a patch for this, I'd provide one. I really disagree that this would be a good idea. We should give clear guidelines how things should be set up here to take full benefit of the functionality. Because this is about an agreement between the OS people and the upstream developers of packages to run on the OS. We want to make sure they can make the assumptions everywhere, which are not configurable and behave differently everywhere. For example, I really want that let's say apache sets ProtectSystem= and can be sure it will just not break things on our OSes. And because of that its impact should be only on the safe subset, and it should be the same on all installations, and not be subject to configuration. Debian's systemd package currently includes a variant of Martin's patch that does include additional directories. So your point that ProtectSystem= does the same thing on every distro is already not true. Which ones precisely? Of course, if you make it configurable, people can shoot themselves in the foot. But you already have a ton of global options in system.conf that can break a lot of software if people do stupid stuff with it: - set a global CapabilityBoundingSet= that's very restrictive - either the system doesn't boot at all because some essential stuff is completely missing - or it boots but some services don't work because they rely on certain things Well, it's a different thing really, I think. Finding resources in the file system and access to certain anyway priviliged facilities are not the same thing. - set SystemCallArchitectures=native when non-native software is installed Same here. In this case the software will totally fail to run, the failure is early and immediate. - set DefaultEnvironment=LD_TRACE_LOADED_OBJECTS=1 or the such Well, environment variables is something completely out of our reach, we are not the ones defining them. - set DefaultTimeoutStartSec=1 to break any unit that takes longer than 1 second to start but doesn't set an own timeout because it assumes the default timeout is sane - DefaultLimitCPU=1 - ... Well, I am not against making certain parameters configurable. I am simply saying that I don't want to make configurable how the base OS is laid out. Also, to go to your apache example: it's not clear that ProtectSystem= just making /usr readonly doesn't break things: I have seen DocumentRoots beneath /usr in the wild (/usr/local/www or the such), with people running dynamic webapps that had to write into that tree. If you then upgrade from a package version that did not include ProtectSystem= (perhaps because it only included a SysV init script) to a package version that does include ProtectSystem=, things will break. Well, sure everything new we introduce can break things. No doubt. But that's really why I wrote sufficiently sure instead of absolutely in my earlier mail. I actually agree with your sentiment of having an agreement between upstream developers and the core OS - I just think I would like to interpret the matter a little differently: To me, ProtectSystem= is supposed to be a protection of all the files a) installed on a system (not created by the user) and b) not subject to modification by typical services. (i.e. not a cache / status file / ...) For distros with /usr-move, this falls back to /usr and /boot (and /etc if =full). For other distros, there may be a few additional directories. And on a custom installation, it may include additional directories, such as /opt. Again, I disagree with this definition. Consider looking at this from an upstream perspective: if you are writing a unit file and really want to lock things down, then you want to enable everything where you understand the effect and what it entails. You also want to arrange your package in a way that fits this nicely. We achieve that now with the option, because we can explain in one or two sentences what this is about and what upstream has to do to make this work nicely. But sticking one configuration layer on
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
On Tue, 21.10.14 13:38, Simon McVittie (simon.mcvit...@collabora.co.uk) wrote: On 21/10/14 13:03, Christian Seiler wrote: That is definitely a good point. Also note that /lib32 is not included in the patch... lib64 is part of the Linux/x86_64 platform ABI (the exact path /lib64/ld-linux-x86-64.so.2 is hard-coded into every Linux/x86_64 executable) so it cannot be considered legacy. Is there a Linux ABI where /lib32 has the same status? If not, then its priority is rather lower. Debian x86_64 (amd64) systems with bi-arch i386 binaries (ia32-libs) are the only instance of /lib32 that I can think of, and that setup is superseded by multiarch these days. Well, on some distros lib64 is a symlink on others it isn't. Doesn't Debian have /lib/arch or so with /lib64 just a symlink to the right subdir? Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
On 21/10/14 19:18, Lennart Poettering wrote: Well, on some distros lib64 is a symlink on others it isn't. Doesn't Debian have /lib/arch or so with /lib64 just a symlink to the right subdir? My Debian laptop has /lib64 as a real directory, containing a ld-linux-x86-64.so.2 symlink into /lib/multiarch tuple. I suspect this might be partly because Debian packages containing other files or symlinks in /lib64 have existed in the past (e.g. to support biarch compilers), and if any of those packages have lingered, dpkg is not going to be happy to replace a non-empty directory with a symlink. Being able to mount something read-only over /lib64 and /lib also seems necessary from the ProtectSystem point of view, if you want ProtectSystem to be a security measure and not just a guard against accidents, since those two strings are part of the portable ABI for Linux binaries on various architectures[1]. If a service can overwrite one of those symlinks with an attacker-chosen value, then it's game over the next time a binary with the relevant PT_INTERP tag is executed. It looks as though I was wrong about lib32 not being necessary, it's in that list too (albeit only for mips and tilegx); so is /libx32. S [1] https://sourceware.org/glibc/wiki/ABIList ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] [PATCH] cgroup: don't trim cgroup trees created by someone else
On Fri, 19.09.14 17:14, Michal Sekletar (msekl...@redhat.com) wrote: Heya, In cases when there is a cgroup tree in a controller hierarchy which was not created by us, but it looks like it was (i.e. cgroup path is the same as the one in systemd's named hierarchy) we shouldn't delete it. So, this is definitely not upstream material, as we need to move things towards the direction of systemd being the only writer. Downstream it's a different story though, possibly. In general we really need to make sure that we clean up things as we go, and that in particular when we move cgroups or turn off or on specific cgroup bits which might have the effect of turning off/on specific controllers for it. Note that cgroup membership for controllers is actually propagated both sidewides and towards the root of a tree. This means means systemd ends up adding removing units to controllers depending on configuration of units that are not directly related to it. Due to this we it's really dynamic when we add and remove cgroup controller memberships and we must agressively clean up memberships. I do see the usecase though for those projects. I'd probably suggest not to merge it for RHEL either. But instead I'd propose a different solution for software: make sure sure to always move a PID into a cgroup as soon as it is created, so that its removal is blocked. Or in other words: right before you need a cgroup to add yourself to it, create it, and expect that it might go away while you are not using it. To deal with the possible race condition of creating a cgroup which is immediately cleaned up by somebody else, try doing this in a loop, until you succeeded. That way things can always stay nicely cleaned up and things are robust towards other processing playing around in the cgroup tree. Hope that makes sense? Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Apply ProtectSystem to non-merged /usr directories
On Tue, 21.10.14 21:22, Christian Seiler (christ...@iwakd.de) wrote: Am 21.10.2014 20:09, schrieb Lennart Poettering: Debian's systemd package currently includes a variant of Martin's patch that does include additional directories. So your point that ProtectSystem= does the same thing on every distro is already not true. Which ones precisely? Here: http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Include-additional-directories-in-ProtectSystem.patch (included in debian/patches/series under the heading 'Debian-specific patches') Ah, well, at least they should make the lib64 thing arch dependent. Otherwise, since you are obviously dead-set against this, I'm not going to argue this further (I don't think I will be able convince you), even though I disagree with a lot of your points of view in your reply. Unless you'd be interested in my response regardless, then I'd be happy to actually write it up. Yeah, I think it will be hard to convince me otherwise. Sorry! THis is one thing to agree to disagree on I figure. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Unicode support in console after boot
On Tue, Oct 21, 2014 at 09:39:46PM +0400, Ivan Shapovalov wrote: On Tuesday 21 October 2014 at 19:03:17, Michal Sekletar wrote: On Tue, Oct 14, 2014 at 09:04:56AM +0200, Jan Synacek wrote: Michal Sekletar msekl...@redhat.com writes: On Mon, Oct 13, 2014 at 09:36:16AM +0200, Jan Synacek wrote: Hello, currently, unicode characters are not correctly displayed in the console. After login, when I run /usr/bin/unicode_start, unicode works fine. I tried to create a service file that runs this script, linking tty to stdout and stderr, but that didn't work. Is there a way how to turn on unicode support in console after boot using a service file? Or any other type of unit? Or is this something that has to be patched in the source (logind perhaps?)? Please try editing /usr/lib/systemd/system/systemd-vconsole-setup.service and remove RemainAfterExit=yes, then regenerate your initramfs image by running dracut command. Add back RemainAfterExit=yes to service file. Reboot. Yep, this helped. Could you please explain why? Also, I believe this should be fixed in all Fedora versions. I must admit I'm not sure why this workaround works. Maybe there is some race condition with some kernel initialization or settings get unapplied because of switch-root. Also, if we go with workaround in Fedora I think dracut needs to fixed to include version on unit file *without* RemainAfterExit=yes. IIUC, this makes unit to be re-run outside of initramfs, so the VT is set up twice, second time after switching the framebuffer driver. And the latter condition is not mandated by anything, it's just a coincidence... I guess that what you are saying pretty much summarizes the situation here. Can we make something about framebuffer driver, thus settings applied in initramfs are not thrashed? Michal ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [systemd-commits] src/libsystemd
On Tue, 21.10.14 13:16, Daniel Mack (zon...@kemper.freedesktop.org) wrote:
src/libsystemd/sd-bus/bus-kernel.c | 16 +---
src/libsystemd/sd-bus/kdbus.h |3 ++-
2 files changed, 15 insertions(+), 4 deletions(-)
New commits:
commit 03785ad0e51b061efb9f9b3f2e328685f9a866aa
Author: Daniel Mack dan...@zonque.org
Date: Tue Oct 21 22:14:03 2014 +0200
sd-bus: sync kdbus.h (API change: switch to absolute timeouts)
kdbus_msg.timeout_ns now takes an absolute value, based on
CLOCK_MONOTONIC,
in order to eventually support automatically restarted syscalls.
Signed-off-by: Daniel Mack dan...@zonque.org
diff --git a/src/libsystemd/sd-bus/bus-kernel.c
b/src/libsystemd/sd-bus/bus-kernel.c
index 6b5a3d3..136c90f 100644
--- a/src/libsystemd/sd-bus/bus-kernel.c
+++ b/src/libsystemd/sd-bus/bus-kernel.c
@@ -270,10 +270,20 @@ static int bus_message_setup_kmsg(sd_bus *b,
sd_bus_message *m) {
m-kdbus-cookie = (uint64_t) m-header-serial;
m-kdbus-priority = m-priority;
-if (m-header-flags BUS_MESSAGE_NO_REPLY_EXPECTED)
+if (m-header-flags BUS_MESSAGE_NO_REPLY_EXPECTED) {
m-kdbus-cookie_reply = m-reply_cookie;
-else
-m-kdbus-timeout_ns = m-timeout * NSEC_PER_USEC;
+} else {
+struct timespec now;
+
+r = clock_gettime(CLOCK_MONOTONIC_COARSE, now);
It sounds OK to enclose this in assert( == 0) really. That's what we
do in now() at least. If gettint the time fails we are massively
fucked, and we can just assert.
+if (r 0) {
+r = -errno;
+goto fail;
+}
+
+m-kdbus-timeout_ns = now.tv_sec * NSEC_PER_SEC +
now.tv_nsec +
+ m-timeout * NSEC_PER_USEC;
I wonder if it's worth adding timspec_load_ns() or so, similar to
timespec_load() but for nsec_t instead of usec_t for this.
Lennart
--
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
2014-10-21 11:01 GMT+02:00 Lennart Poettering lenn...@poettering.net:
On Tue, 21.10.14 03:01, Ronny Chevalier (chevalier.ro...@gmail.com) wrote:
+static int get_editors(char ***editors) {
+char **tmp_editors = strv_new(nano, vim, vi, NULL);
Please avoid calling functions and declaring variables in one line.
Also, there's an OOM check missing for this line.
+char *editor;
+
+/* SYSTEMD_EDITOR takes precedence over EDITOR which takes
precedence over VISUAL
+ * If neither SYSTEMD_EDITOR nor EDITOR nor VISUAL are present,
+ * we try to execute well known editors
+ */
+editor = getenv(SYSTEMD_EDITOR);
+if (!editor)
+editor = getenv(EDITOR);
+if (!editor)
+editor = getenv(VISUAL);
+
+if (editor) {
+int r;
+
+editor = strdup(editor);
+if (!editor)
+return log_oom();
+
+r = strv_consume_prepend(tmp_editors, editor);
+if (r 0)
+return log_oom();
+}
+
+*editors = tmp_editors;
+
+return 0;
+}
Hmm, I don't like this bit. Instead of figuring out the editor in one
step, and then executing it, I'd really prefer if we'd try to execute
the editors one-by-one until we find one that works. When we invoke
the pager we follow a similar logic.
Actually that's what I do. I fill a strv of editors and try to execute
each one of them, until one works. But we fail if the error is
different than No such file or directory.
This function (get_editors) is used in run_editor, where I do a
STRV_FOREACH in the child to try to execute every editor, with the one
comming from SYSTEMD_EDITOR or EDITOR or VISUAL, first.
Oh, well, indeed. Hmm, so I figure I'd just prefer if we wouldn't have
to allocate memory for this. or in other words, I'd like to see an alg
like this:
if (fork() == 0) {
...
e = getenv(SYSTEMD_EDITOR);
if (!e)
e = getenv(EDITOR);
if (!e)
e = getenv(VISUAL);
if (e)
execlp(e, ...);
execlp(nano, ...);
execlp(vim, ...);
execlp(vi, ...);
...
}
That way we wouldn't have to allocate an strv, and just open code the
search logic. This would then mirror nicely how the pager logic works...
I agree but I can't (or I don't know how to do this), because we need
to give the paths of the units to execvp, and since the first argument
must be the name of the editor, I need to add it to the strv. The
original strv is a strv of units path, so I copy it and prepend the
name of the editor.
We can do this for the pager because there is no arguments to give,
here we have a list of paths.
Or maybe, you meant that if the one coming from SYSTEMD_EDITOR failed,
we should try to execute EDITOR, then VISUAL, too? Because I think we
should do that, but with the current code we don't.
I think it's OK checking only the first of SYSTEMD_EDITOR, EDITOR,
VISUAL that exists, and then proceed with nano/vim/vi
Ok
+if (arg_transport != BUS_TRANSPORT_LOCAL) {
+log_error(Cannot remotely edit units);
+return -EINVAL;
+}
+
+if (arg_runtime) {
+log_error(Cannot edit runtime units);
+return -EINVAL;
+}
Hmm, why not support this? And imply --runtime or so?
You are right, I forgot to ask a question about this one, sorry.
If there is a unit in /etc/systemd/system/ and someone wants to edit
this unit temporarily it will run systemctl --runtime myunit. The
problem is the unit will be copied in the /run/ directory but this
directory do not have precedence over /etc/systemd/, so when this unit
will be restarted, it won't take into account the one in /run/, right?
(My understanding of this come from src/shared/path-lookup.c)
Indeed. We should probably print an error in this case and refuse
operation. However we should support adding a --runtime .d/ snippet if
a unit file is in /etc, because that will actually work fine.
I think you meant /usr/.
Ok
Furthermore, if I recall correctly the FragmentPath of a runtime
unit is empty?
No, it should be set properly like in all other cases really.
So, if I'm wrong and systemctl understands that when we do restart
with --runtime it means to execute the one from /run/, well I can
certainly add support for this.
If I'm a right, I do not see the point because it will not be easy to
understand for a user that it can temporarily edit a unit if it's in
/usr/lib/systemd/ but not in /etc/.
Yeah it is suprising but I think we could explain it nicely in an
error.
I'd merge your patch with or without this btw, I think it would just
be nice to support --runtime to the level we can support it, but
Re: [systemd-devel] Unicode support in console after boot
On Tuesday 21 October 2014 at 21:57:09, Michal Sekletar wrote: On Tue, Oct 21, 2014 at 09:39:46PM +0400, Ivan Shapovalov wrote: On Tuesday 21 October 2014 at 19:03:17, Michal Sekletar wrote: On Tue, Oct 14, 2014 at 09:04:56AM +0200, Jan Synacek wrote: Michal Sekletar msekl...@redhat.com writes: On Mon, Oct 13, 2014 at 09:36:16AM +0200, Jan Synacek wrote: Hello, currently, unicode characters are not correctly displayed in the console. After login, when I run /usr/bin/unicode_start, unicode works fine. I tried to create a service file that runs this script, linking tty to stdout and stderr, but that didn't work. Is there a way how to turn on unicode support in console after boot using a service file? Or any other type of unit? Or is this something that has to be patched in the source (logind perhaps?)? Please try editing /usr/lib/systemd/system/systemd-vconsole-setup.service and remove RemainAfterExit=yes, then regenerate your initramfs image by running dracut command. Add back RemainAfterExit=yes to service file. Reboot. Yep, this helped. Could you please explain why? Also, I believe this should be fixed in all Fedora versions. I must admit I'm not sure why this workaround works. Maybe there is some race condition with some kernel initialization or settings get unapplied because of switch-root. Also, if we go with workaround in Fedora I think dracut needs to fixed to include version on unit file *without* RemainAfterExit=yes. IIUC, this makes unit to be re-run outside of initramfs, so the VT is set up twice, second time after switching the framebuffer driver. And the latter condition is not mandated by anything, it's just a coincidence... I guess that what you are saying pretty much summarizes the situation here. Can we make something about framebuffer driver, thus settings applied in initramfs are not thrashed? I don't know if there is a valid solution. Maybe someone familiar with the VT subsystem knows... Personally I've worked around this problem as follows: the fb driver (i915) is included into the initramfs, while systemd-vconsole-setup is ran only in the real root. -- Ivan Shapovalov / intelfx / signature.asc Description: This is a digitally signed message part. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
2014-10-21 11:09 GMT+02:00 Lennart Poettering lenn...@poettering.net:
On Sat, 18.10.14 18:30, Ronny Chevalier (chevalier.ro...@gmail.com) wrote:
Three more suggestions after thinking about this a bit more...
+static int unit_file_drop_in(const char *unit_name, const char
*config_home, char **new_path) {
+char *tmp_path;
+int r;
+
+assert(unit_name);
+assert(new_path);
+
+switch (arg_scope) {
+case UNIT_FILE_SYSTEM:
+tmp_path = strjoin(SYSTEM_CONFIG_UNIT_PATH, /,
unit_name, .d/amendments.conf, NULL);
+break;
+case UNIT_FILE_GLOBAL:
+tmp_path = strjoin(USER_CONFIG_UNIT_PATH, /,
unit_name, .d/amendments.conf, NULL);
+break;
+case UNIT_FILE_USER:
+assert(config_home);
+tmp_path = strjoin(config_home, /, unit_name,
.d/amendments.conf, NULL);
+break;
+default:
+assert_not_reached(Invalid scope);
+}
+if (!tmp_path)
+return log_oom();
+
+r = mkdir_parents(tmp_path, 0755);
+if (r 0) {
+log_error(Failed to create directories for %s: %s,
tmp_path, strerror(-r));
+free(tmp_path);
+return r;
+}
+
+*new_path = tmp_path;
+
+return 0;
I think we should really take some inspiration here from how git
prepares the editor when asking for commit messages: adding a really
useful commented help text to the end of the new file to edit would be
really awesome. More specifically, I think we should include an output
equivalent to systemctl cat's add the end, commented and prefixed
with some clear marker that we can then use to remove the bits again
before saving things.
Yeah It seems nice.
Do you prefer that I add this in the correction of this patch, or with
follow-up patches ?
+if (arg_scope == UNIT_FILE_USER
+|| startswith(fragment_path, SYSTEM_CONFIG_UNIT_PATH)
+|| startswith(fragment_path, USER_CONFIG_UNIT_PATH)) {
Please use path_startswith() for this.
Oh I didn't know this one, thanks.
+if (access(tmp_path, F_OK) == 0) {
+char response;
+
+r = ask_char(response, yn, %s already exists, are you
sure to overwrite it with %s? [(y)es, (n)o] , tmp_path, fragment_path);
+if (r 0) {
+free(tmp_path);
+return r;
+}
Hmm, when precisely is this actually triggered? I mean, if the file
already exists in /etc we should just edit it there...
There is a specific case when this can happen.
Imagine you have a unit foobar with
FragmentPath=/usr/lib/systemd/system/foobar.service, and you added
manually /etc/systemd/system/foobar.service without doing
daemon-reload. When avoid_bus() == false, we will get the /usr/***
path, so we need to be sure that the user don't want to overwrite the
one in /etc/***.
We can avoid this by doing daemon-reload before editing, but I'm not
sure this is wanted.
Lennart
--
Lennart Poettering, Red Hat
Thanks for the review!
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Martin Steigerwald wrote on 21/10/14 10:25: Am Mittwoch, 8. Oktober 2014, 10:54:00 schrieb Lennart Poettering: On Tue, 07.10.14 23:40, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote: My question really isn't why are the Debian dependencies the way they are. I understand that. I was trying to highlight the strange situation of a desktop application requiring a particular init system. I *think* this is a result of the init portion of systemd being bundled together with the logind portion of systemd. To me (unfamiliar with the systemd code) those two functions seem distinct enough to merit being separate. Debian can't easily separate what the systemd devs have developed as a single binary, so we end up with these strange dependency chains. Single binary is false of course. Logind is developed as a separate program, which is why systemd-shim is possible at all. AFAIK the actual relevant dependencies go as follows: First, there's a good reason why logind requires cgroup functionality. And there's a good reason why cgroup functionality is best implemented together with init (see http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/ for more info). So it's not quite directly logind has to depend on systemd as init, but logind has to depend on the system having cgroup support, and there's no equally good cgroup support available for inits other than systemd. It is possible to provide the relevant cgroup interfaces in or on top of another init, as the systemd-sysv + cgmanager combination attempts to do. But it is not trivial to do, as bugs and implementation delays with that combo have shown, and it's quite likely that there will be more problems in the future. It's not a particularly good idea to use the less-tested and less-reliable systemd-shim instead of the more reliable systemd. Thus the overall result is that yes, it does make sense to switch machines to systemd when you add certain functionality, even if that functionality does not appear to be directly tied to the init system at first glance. Also note that the systemd concepts logind makes use of are also exported in its own API. The scopes and slices that logind uses are exposed on its bus objects, and they are used by tools like loginctl to do their work. The scopes and slices concept does not exist elsewhere, and there's nothing comparable around, so even if we wanted we couldn't make logind work on anything else. Then why in the first hand are the scopes and slices concepts within systemd *tightly* coupled when it is functionality that makes sense to be utilitizes in a component that provides a different functionality. I wonder what functionality systemd provides right now in one more or less tightly coupled package: - early boot process PID 1 - managing kernel drivers and device files (via coupled udev) Not PID 1 - resource control for process groups PID 1. - logging Not PID 1 - core dumping Not PID 1 - network while I think a distro-unified way to approaching network that replaces the current distro specific ways like ifupdown and so on, why does it have to be coupled with systemd? Not PID 1 - cron, although that at least in Debian is separate as systemd-cron Partly PID 1 Thats rather much. Really rather much. Way more than traditonal sysvinit covered. This is because traditional sysvinit covered PID 1 and performing it's job (if you consider e.g. killall5 and friends). You seem to be conflating systemd as PID 1 and systemd as a project. These are two completely different things. There are many related things here and a lot of the sub components obviously use a lot of the same functionality and utility functions. This is achieved via a shared library with a private API. The only reason that all these separate parts are developed as part of the systemd project is that it's *easier*. It's just the easiest way to develop. An alternative would be to make the utility functions API stable and export the shared library publicly and give API guarantees, but that puts a lot of pressure and it's a difficult thing to provide and it has long term maintenance overhead. This is something that *costs*. It costs in time/man hours and therefore it carries real overheads. Doing this for the convenience of splitting things out is simply not worth it - especially so as the main people working on these things are the same people. It also increases the test matrix. If logind v300 has to work with libsystemd v300 and all future versions then the testing side of things increases in complexity *massively*. Again this causes problems that translate to time and effort of developers that could better be allocated to building a better overall set of building blocks. The other option would be to just split out the git repos but keep the API a private one - with releases still in lockstep. This is ultimately
[systemd-devel] [PATCH, v3 1/3] tests: added tests for unit_file_get_{state, list}
This test constructs different unit file states and checks the output of unit_file_get_state and unit_file_get_list for each. This test characterizes the current output of the master branch in preparation for a patch which improves the performance of unit state detection in the face of thousands of units. --- .gitignore | 1 + Makefile.am| 44 ++- src/test/test-enabled.c| 141 + .../etc/systemd/system/masked.service | 1 + .../etc/systemd/system/maskedstatic.service| 1 + .../etc/systemd/system/some.target | 11 ++ .../system/some.target.wants/aliased.service | 1 + .../system/some.target.wants/also_masked.service | 1 + .../system/some.target.wants/another.service | 1 + .../system/some.target.wants/different.service | 1 + .../system/some.target.wants/masked.service| 1 + .../some.target.wants/templating@four.service | 1 + .../some.target.wants/templating@one.service | 1 + .../some.target.wants/templating@three.service | 1 + .../some.target.wants/templating@two.service | 1 + .../run/systemd/system/maskedruntime.service | 1 + .../run/systemd/system/maskedruntimestatic.service | 1 + .../run/systemd/system/other.target| 14 ++ .../system/other.target.wants/runtime.service | 1 + .../usr/lib/systemd/system/another.service | 9 ++ .../usr/lib/systemd/system/disabled.service| 9 ++ .../usr/lib/systemd/system/invalid.service | 1 + .../usr/lib/systemd/system/masked.service | 9 ++ .../usr/lib/systemd/system/maskedruntime.service | 9 ++ .../lib/systemd/system/maskedruntimestatic.service | 6 + .../usr/lib/systemd/system/maskedstatic.service| 6 + .../usr/lib/systemd/system/runtime.service | 9 ++ .../usr/lib/systemd/system/static.service | 6 + .../usr/lib/systemd/system/templating@.service | 9 ++ .../lib/systemd/system/templating@three.service| 9 ++ .../usr/lib/systemd/system/templating@two.service | 9 ++ .../usr/lib/systemd/system/unique.service | 9 ++ 32 files changed, 322 insertions(+), 3 deletions(-) create mode 100644 src/test/test-enabled.c create mode 12 test/test-enabled-root/etc/systemd/system/masked.service create mode 12 test/test-enabled-root/etc/systemd/system/maskedstatic.service create mode 100644 test/test-enabled-root/etc/systemd/system/some.target create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/aliased.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/also_masked.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/another.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/different.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/masked.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/templating@four.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/templating@one.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/templating@three.service create mode 12 test/test-enabled-root/etc/systemd/system/some.target.wants/templating@two.service create mode 12 test/test-enabled-root/run/systemd/system/maskedruntime.service create mode 12 test/test-enabled-root/run/systemd/system/maskedruntimestatic.service create mode 100644 test/test-enabled-root/run/systemd/system/other.target create mode 12 test/test-enabled-root/run/systemd/system/other.target.wants/runtime.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/another.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/disabled.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/invalid.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/masked.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/maskedruntime.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/maskedruntimestatic.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/maskedstatic.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/runtime.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/static.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/templating@.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/templating@three.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/templating@two.service create mode 100644 test/test-enabled-root/usr/lib/systemd/system/unique.service diff --git
[systemd-devel] [PATCH, v3 3/3] find_symlinks: adds a cache of enabled unit symbolic link state
The current find_symlinks_fd code traverses the config directories
duplicatively. This is a performance problem if 1000s of units are
being controlled. This patch adds a hashmap cache of symbolic link
state which is filled in one traversal and then consulted as needed to
prevent re-traversal.
The enabled_context cache lives in the manager struct. Initially the
cache is empty and is filled on first use. A pointer to the cache is
passed to all routines which can call find_symlinks_fd. If a NULL is
passed to find_symlinks_fd a temporary cache is created and used for
the single call.
The cache has two levels, the first is keyed by config_path and the
second by the names and paths of the units.
The cache contains both forward and reverse mappings; from symbolic
link name to target and from target to symbolic link name.
The cache contains entries for both the basename of the unit and the
full path of the link name and target.
The test-enabled patch (previously submitted) checks that the results
of unit_file_get_state and unit_file_get_list do not change as a
result of this cache. This patch presumes the test-enabled patch has
already been applied.
The test-manyunits patch (previously submitted) checks that the
performance of unit_file_get_list is acceptable in the face of
thousands of units. This patch presumes the test-manyunits patch has
already been applied.
---
src/core/dbus-manager.c | 6 +-
src/core/manager.c| 6 +
src/core/manager.h| 2 +
src/core/unit.c | 2 +-
src/shared/install.c | 283 ++
src/shared/install.h | 18 ++-
src/systemctl/systemctl.c | 6 +-
src/test/test-enabled.c | 24 ++--
src/test/test-install.c | 87 +++---
src/test/test-manyunits.c | 11 +-
src/test/test-unit-file.c | 2 +-
11 files changed, 339 insertions(+), 108 deletions(-)
diff --git a/src/core/dbus-manager.c b/src/core/dbus-manager.c
index 57db1c9..0ce41b0 100644
--- a/src/core/dbus-manager.c
+++ b/src/core/dbus-manager.c
@@ -1403,7 +1403,7 @@ static int method_list_unit_files(sd_bus *bus,
sd_bus_message *message, void *us
if (!h)
return -ENOMEM;
-r = unit_file_get_list(m-running_as == SYSTEMD_SYSTEM ?
UNIT_FILE_SYSTEM : UNIT_FILE_USER, NULL, h);
+r = unit_file_get_list(m-running_as == SYSTEMD_SYSTEM ?
UNIT_FILE_SYSTEM : UNIT_FILE_USER, NULL, h, m-enabled);
if (r 0)
goto fail;
@@ -1454,7 +1454,7 @@ static int method_get_unit_file_state(sd_bus *bus,
sd_bus_message *message, void
scope = m-running_as == SYSTEMD_SYSTEM ? UNIT_FILE_SYSTEM :
UNIT_FILE_USER;
-state = unit_file_get_state(scope, NULL, name);
+state = unit_file_get_state(scope, NULL, name, m-enabled);
if (state 0)
return state;
@@ -1843,7 +1843,7 @@ static int method_add_dependency_unit_files(sd_bus *bus,
sd_bus_message *message
scope = m-running_as == SYSTEMD_SYSTEM ? UNIT_FILE_SYSTEM :
UNIT_FILE_USER;
-r = unit_file_add_dependency(scope, runtime, NULL, l, target, dep,
force, changes, n_changes);
+r = unit_file_add_dependency(scope, runtime, NULL, l, target, dep,
force, NULL, changes, n_changes);
if (r 0)
return r;
diff --git a/src/core/manager.c b/src/core/manager.c
index 1a5d252..1eb9683 100644
--- a/src/core/manager.c
+++ b/src/core/manager.c
@@ -464,6 +464,10 @@ int manager_new(SystemdRunningAs running_as, bool
test_run, Manager **_m) {
if (r 0)
goto fail;
+m-enabled = enabled_context_new();
+if (!m-enabled)
+goto fail;
+
r = set_ensure_allocated(m-startup_units, NULL);
if (r 0)
goto fail;
@@ -821,6 +825,8 @@ void manager_free(Manager *m) {
hashmap_free(m-watch_pids2);
hashmap_free(m-watch_bus);
+enabled_context_free(m-enabled);
+
set_free(m-startup_units);
set_free(m-failed_units);
diff --git a/src/core/manager.h b/src/core/manager.h
index 8e3c146..3f54fe0 100644
--- a/src/core/manager.h
+++ b/src/core/manager.h
@@ -72,6 +72,7 @@ typedef enum ManagerExitCode {
#include unit-name.h
#include exit-status.h
#include show-status.h
+#include install.h
#include failure-action.h
struct Manager {
@@ -82,6 +83,7 @@ struct Manager {
/* Active jobs and units */
Hashmap *units; /* name string = Unit object n:1 */
Hashmap *jobs; /* job id = Job object 1:1 */
+EnabledContext *enabled; /* name string = is enabled */
/* To make it easy to iterate through the units of a specific
* type we maintain a per type linked list */
diff --git a/src/core/unit.c b/src/core/unit.c
index 0389e6e..3e29944 100644
--- a/src/core/unit.c
+++ b/src/core/unit.c
@@ -2941,7 +2941,7 @@ UnitFileState unit_get_unit_file_state(Unit *u) {
if (u-unit_file_state 0
[systemd-devel] [PATCH, v3 2/3] tests: unit_file_get_list performance with many units
This test temporarily creates several thousand unit files and checks
the performance of unit_file_get_list.
This test is currently added to manual_tests only since it does not
pass.
This test does pass if the subsequent enabled unit cache patch is
applied.
---
.gitignore| 1 +
Makefile.am | 13 -
src/test/test-manyunits.c | 133 ++
3 files changed, 146 insertions(+), 1 deletion(-)
create mode 100644 src/test/test-manyunits.c
diff --git a/.gitignore b/.gitignore
index 97b2b2b..7d5c97c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -207,6 +207,7 @@
/test-login-tables
/test-loopback
/test-machine-tables
+/test-manyunits
/test-mmap-cache
/test-namespace
/test-network
diff --git a/Makefile.am b/Makefile.am
index 46d532c..12cae02 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1306,7 +1306,8 @@ manual_tests += \
test-install \
test-watchdog \
test-log \
- test-ipcrm
+ test-ipcrm \
+ test-manyunits
if HAVE_KMOD
manual_tests += \
@@ -1819,6 +1820,16 @@ test_enabled_LDADD = \
libsystemd-shared.la \
libsystemd-internal.la
+test_manyunits_SOURCES = \
+ src/test/test-manyunits.c
+
+test_manyunits_LDADD = \
+ libsystemd-core.la \
+ libsystemd-units.la \
+ libsystemd-label.la \
+ libsystemd-shared.la \
+ libsystemd-internal.la
+
test_watchdog_SOURCES = \
src/test/test-watchdog.c
diff --git a/src/test/test-manyunits.c b/src/test/test-manyunits.c
new file mode 100644
index 000..e76f04c
--- /dev/null
+++ b/src/test/test-manyunits.c
@@ -0,0 +1,133 @@
+/*-*- Mode: C; c-basic-offset: 8; indent-tabs-mode: nil -*-*/
+
+/***
+ This file is part of systemd.
+
+ Copyright 2014 Pantheon, Inc.
+
+ systemd is free software; you can redistribute it and/or modify it
+ under the terms of the GNU Lesser General Public License as published by
+ the Free Software Foundation; either version 2.1 of the License, or
+ (at your option) any later version.
+
+ systemd is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with systemd; If not, see http://www.gnu.org/licenses/.
+***/
+
+#include errno.h
+#include fcntl.h
+#include stdio.h
+#include string.h
+#include sys/param.h
+#include unistd.h
+#include time.h
+
+#include manager.h
+#include macro.h
+#include util.h
+
+static const int NUNITS = 3000;
+
+static const char *root_dir;
+
+/* Cleanup the created unit files if we fail an assertion. */
+#define assert_se_cleanup(expr) \
+do {\
+if (_unlikely_(!(expr))) { \
+cleanup_manyunits();\
+log_assert_failed(#expr, __FILE__, __LINE__,\
+ __PRETTY_FUNCTION__); \
+} \
+} while (false) \
+
+static const char *many_path(int unitnum) {
+static char path[PATH_MAX];
+snprintf(path, PATH_MAX, %s/%s/many-%06d.service,
+ root_dir, usr/lib/systemd/system, unitnum);
+return path;
+}
+
+static const char *link_path(int unitnum) {
+static char path[PATH_MAX];
+snprintf(path, PATH_MAX, %s/%s/many-%06d.service,
+ root_dir, etc/systemd/system/some.target.wants, unitnum);
+return path;
+}
+
+static const char *another_path(void) {
+static char path[PATH_MAX];
+snprintf(path, PATH_MAX, %s/%s/another.service,
+ root_dir, usr/lib/systemd/system);
+return path;
+}
+
+
+static void cleanup_manyunits(void) {
+int unitnum;
+
+fprintf(stderr, removing %d unit files\n, NUNITS);
+
+for (unitnum = 0; unitnum NUNITS; ++unitnum) {
+unlink(link_path(unitnum));
+unlink(many_path(unitnum));
+}
+}
+
+static void setup_manyunits(void) {
+int unitnum;
+const char *another;
+
+another = another_path();
+
+fprintf(stderr, creating %d unit files\n, NUNITS);
+
+for (unitnum = 0; unitnum NUNITS; ++unitnum) {
+assert_se_cleanup(link(another, many_path(unitnum)) == 0);
+assert_se_cleanup(symlink(many_path(unitnum),
+ link_path(unitnum)) == 0);
+}
+}
+
+static void test_manyunits(void) {
+time_t t0, t1;
+int r = 0;
+int count = 0;
+Hashmap *h;
+UnitFileList *p;
+Iterator
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
2014-10-21 0:51 GMT+02:00 Lennart Poettering lenn...@poettering.net:
On Sat, 18.10.14 18:30, Ronny Chevalier (chevalier.ro...@gmail.com) wrote:
Looks pretty good. A few comments.
+
+static int unit_file_copy_if_needed(const char *unit_name, const char
*fragment_path, char **new_path) {
+char *tmp_path;
+int r;
+
+assert(fragment_path);
+assert(unit_name);
+assert(new_path);
+
+/* If it's a unit for the --user scope there is no need to copy it,
it's already in the right directory.
+ * Same if this is --system/--global scope and the file is in
{SYSTEM,USER}_CONFIG_UNIT_PATH
+ */
+if (arg_scope == UNIT_FILE_USER
+|| startswith(fragment_path, SYSTEM_CONFIG_UNIT_PATH)
+|| startswith(fragment_path, USER_CONFIG_UNIT_PATH)) {
+*new_path = strdup(fragment_path);
+if (!*new_path)
+return log_oom();
+return 0;
+}
+
+switch (arg_scope) {
+case UNIT_FILE_SYSTEM:
+tmp_path = strjoin(SYSTEM_CONFIG_UNIT_PATH, /,
unit_name, NULL);
+break;
+case UNIT_FILE_GLOBAL:
+tmp_path = strjoin(USER_CONFIG_UNIT_PATH, /,
unit_name, NULL);
+break;
+default:
+assert_not_reached(Invalid scope);
+}
+if (!tmp_path)
+return log_oom();
+
+if (access(tmp_path, F_OK) == 0) {
+char response;
+
+r = ask_char(response, yn, %s already exists, are you
sure to overwrite it with %s? [(y)es, (n)o] , tmp_path, fragment_path);
+if (r 0) {
+free(tmp_path);
+return r;
+}
+if (response != 'y') {
+log_warning(%s ignored, unit_name);
+free(tmp_path);
+return -1;
+}
+}
+
+r = mkdir_parents(tmp_path, 0755);
+if (r 0) {
+log_error(Failed to create directories for %s: %s,
tmp_path, strerror(-r));
+free(tmp_path);
+return r;
+}
+
+r = copy_file(fragment_path, tmp_path, 0, 0644);
+if (r 0) {
+log_error(Failed to copy %s to %s: %s, fragment_path,
tmp_path, strerror(-r));
+free(tmp_path);
+return r;
+}
+
+*new_path = tmp_path;
+
+return 0;
+}
+
+static int get_editors(char ***editors) {
+char **tmp_editors = strv_new(nano, vim, vi, NULL);
Please avoid calling functions and declaring variables in one line.
Also, there's an OOM check missing for this line.
+char *editor;
+
+/* SYSTEMD_EDITOR takes precedence over EDITOR which takes
precedence over VISUAL
+ * If neither SYSTEMD_EDITOR nor EDITOR nor VISUAL are present,
+ * we try to execute well known editors
+ */
+editor = getenv(SYSTEMD_EDITOR);
+if (!editor)
+editor = getenv(EDITOR);
+if (!editor)
+editor = getenv(VISUAL);
+
+if (editor) {
+int r;
+
+editor = strdup(editor);
+if (!editor)
+return log_oom();
+
+r = strv_consume_prepend(tmp_editors, editor);
+if (r 0)
+return log_oom();
+}
+
+*editors = tmp_editors;
+
+return 0;
+}
Hmm, I don't like this bit. Instead of figuring out the editor in one
step, and then executing it, I'd really prefer if we'd try to execute
the editors one-by-one until we find one that works. When we invoke
the pager we follow a similar logic.
+
+static int run_editor(char **paths) {
+pid_t pid;
+siginfo_t status;
+int r;
+
+assert(paths);
+
+pid = fork();
+if (pid 0) {
+log_error(Failed to fork: %m);
+return -errno;
+}
+
+if (pid == 0) {
+_cleanup_strv_free_ char **editors = NULL;
+char *editor;
+char **p;
+
+r = get_editors(editors);
+if (r 0) {
+_exit(EXIT_FAILURE);
+}
Nitpick: please no {} for single-line if blocks.
Oh and about this one, since I saw you mentioning it a couple of times
in other patches sent to the ML, maybe you should add this to the
CODING_STYLE ? I don't see any mention of this in it currently.
+
+STRV_FOREACH(p, editors) {
+_cleanup_strv_free_ char **args = NULL;
+
+editor = strdup(*p);
+
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
On Wed, Oct 22, 2014 at 12:29:15AM +0200, Ronny Chevalier wrote:
2014-10-21 0:51 GMT+02:00 Lennart Poettering lenn...@poettering.net:
On Sat, 18.10.14 18:30, Ronny Chevalier (chevalier.ro...@gmail.com) wrote:
Looks pretty good. A few comments.
+
+static int unit_file_copy_if_needed(const char *unit_name, const char
*fragment_path, char **new_path) {
+char *tmp_path;
+int r;
+
+assert(fragment_path);
+assert(unit_name);
+assert(new_path);
+
+/* If it's a unit for the --user scope there is no need to copy
it, it's already in the right directory.
+ * Same if this is --system/--global scope and the file is in
{SYSTEM,USER}_CONFIG_UNIT_PATH
+ */
+if (arg_scope == UNIT_FILE_USER
+|| startswith(fragment_path, SYSTEM_CONFIG_UNIT_PATH)
+|| startswith(fragment_path, USER_CONFIG_UNIT_PATH)) {
+*new_path = strdup(fragment_path);
+if (!*new_path)
+return log_oom();
+return 0;
+}
+
+switch (arg_scope) {
+case UNIT_FILE_SYSTEM:
+tmp_path = strjoin(SYSTEM_CONFIG_UNIT_PATH, /,
unit_name, NULL);
+break;
+case UNIT_FILE_GLOBAL:
+tmp_path = strjoin(USER_CONFIG_UNIT_PATH, /,
unit_name, NULL);
+break;
+default:
+assert_not_reached(Invalid scope);
+}
+if (!tmp_path)
+return log_oom();
+
+if (access(tmp_path, F_OK) == 0) {
+char response;
+
+r = ask_char(response, yn, %s already exists, are you
sure to overwrite it with %s? [(y)es, (n)o] , tmp_path, fragment_path);
+if (r 0) {
+free(tmp_path);
+return r;
+}
+if (response != 'y') {
+log_warning(%s ignored, unit_name);
+free(tmp_path);
+return -1;
+}
+}
+
+r = mkdir_parents(tmp_path, 0755);
+if (r 0) {
+log_error(Failed to create directories for %s: %s,
tmp_path, strerror(-r));
+free(tmp_path);
+return r;
+}
+
+r = copy_file(fragment_path, tmp_path, 0, 0644);
+if (r 0) {
+log_error(Failed to copy %s to %s: %s, fragment_path,
tmp_path, strerror(-r));
+free(tmp_path);
+return r;
+}
+
+*new_path = tmp_path;
+
+return 0;
+}
+
+static int get_editors(char ***editors) {
+char **tmp_editors = strv_new(nano, vim, vi, NULL);
Please avoid calling functions and declaring variables in one line.
Also, there's an OOM check missing for this line.
+char *editor;
+
+/* SYSTEMD_EDITOR takes precedence over EDITOR which takes
precedence over VISUAL
+ * If neither SYSTEMD_EDITOR nor EDITOR nor VISUAL are present,
+ * we try to execute well known editors
+ */
+editor = getenv(SYSTEMD_EDITOR);
+if (!editor)
+editor = getenv(EDITOR);
+if (!editor)
+editor = getenv(VISUAL);
+
+if (editor) {
+int r;
+
+editor = strdup(editor);
+if (!editor)
+return log_oom();
+
+r = strv_consume_prepend(tmp_editors, editor);
+if (r 0)
+return log_oom();
+}
+
+*editors = tmp_editors;
+
+return 0;
+}
Hmm, I don't like this bit. Instead of figuring out the editor in one
step, and then executing it, I'd really prefer if we'd try to execute
the editors one-by-one until we find one that works. When we invoke
the pager we follow a similar logic.
+
+static int run_editor(char **paths) {
+pid_t pid;
+siginfo_t status;
+int r;
+
+assert(paths);
+
+pid = fork();
+if (pid 0) {
+log_error(Failed to fork: %m);
+return -errno;
+}
+
+if (pid == 0) {
+_cleanup_strv_free_ char **editors = NULL;
+char *editor;
+char **p;
+
+r = get_editors(editors);
+if (r 0) {
+_exit(EXIT_FAILURE);
+}
Nitpick: please no {} for single-line if blocks.
Oh and about this one, since I saw you mentioning it a couple of times
in other patches sent to the ML, maybe you should add this to the
CODING_STYLE ? I don't see any
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
On Sat, Oct 18, 2014 at 06:30:02PM +0200, Ronny Chevalier wrote: It helps editing units by either creating a drop-in file, like /etc/systemd/system/my.service.d/amendments.conf, or by copying the original unit from /usr/lib/systemd/ to /etc/systemd/ if the --full option is specified. Then it invokes an editor to the related files and daemon-reload is invoked when the editor exited successfully. Hm, this sequence doesn't sound right. A temporary file should be created, edited, and then atomically put in place, iff the editor exits successfully. I think we should follow in the footsteps of git here... and abort if the editor exits with an error. I'm not sure abou the name 'amendments.conf'. Wouldn't 'local.conf' be more idiomatic, and also easier to type? See https://bugzilla.redhat.com/show_bug.cgi?id=906824 --- TODO | 2 - man/journalctl.xml| 6 +- man/less-variables.xml| 40 +++-- man/localectl.xml | 6 +- man/loginctl.xml | 6 +- man/machinectl.xml| 6 +- man/systemctl.xml | 49 +- man/systemd-analyze.xml | 6 +- man/timedatectl.xml | 6 +- src/systemctl/systemctl.c | 394 +- 10 files changed, 488 insertions(+), 33 deletions(-) There's no need to mangle all the xml files. It is possible to include specific parts of a file. See how standard-options.xml incorporated whole, and sometimes just specific parts using xi:include href=standard-options.xml xpointer=... /. -xi:include href=less-variables.xml / +refsect1 +titleEnvironment/title + +xi:include href=less-variables.xml / +/refsect1 +varlistentry + termcommandedit replaceableNAME/replaceable.../command/term + + listitem +paraEdit one or more unit files, as specified on the command +line./para This wording is misleading, because the unit file actually will not be *edited*, but extended in the normal case where --full is not used. I'm missing an explanatory sentence here, something like An editor will be launched to edit a drop-in snippet (or a whole replacement file if --full is used), to extend or override the specified unit. Then the next paragraph about --system/--user/--global will be more natural. + +paraDepending on whether option--system/option (the default), +option--user/option, or option--global/option is specified, +this create a drop-in file for each units either for the system, +for the calling user or for all futures logins of all users. Then +the editor is invoked on them (see section Environment below)./para + +paraIf option--full/option is specified, this will copy the original +units instead of creating drop-in files./para + +paraAfter the units have been edited, the systemd configuration is +reloaded (in a way that is equivalent to commanddaemon-reload/command), +but it does not restart or reload the units./para + +paraNote that this command cannot be used with option--runtime/option or +to remotely edit units./para + /listitem +/varlistentry /variablelist /refsect2 +variablelist class='environment-variables' + varlistentry +termvarname$SYSTEMD_EDITOR/varname/term + +listitemparaEditor to use when editing units; overrides +varname$EDITOR/varname and varname$VISUAL/varname. If neither +varname$SYSTEMD_EDITOR/varname nor varname$EDITOR/varname nor +varname$VISUAL/varname are present or if it is set to an empty +string or if their execution failed, systemctl will try to execute well +known editors in this order: + citerefentryrefentrytitlenano/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevim/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevi/refentrytitlemanvolnum1/manvolnum/citerefentry. +/para/listitem + /varlistentry +/variablelist +xi:include href=less-variables.xml / + /refsect1 refsect1 titleSee Also/title @@ -1572,7 +1617,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket systemd-udevd.service citerefentryrefentrytitlesystemd.resource-management/refentrytitlemanvolnum5/manvolnum/citerefentry, citerefentryrefentrytitlesystemd.special/refentrytitlemanvolnum7/manvolnum/citerefentry, citerefentry project='man-pages'refentrytitlewall/refentrytitlemanvolnum1/manvolnum/citerefentry, - citerefentryrefentrytitlesystemd.preset/refentrytitlemanvolnum5/manvolnum/citerefentry + citerefentryrefentrytitlesystemd.preset/refentrytitlemanvolnum5/manvolnum/citerefentry,
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
Colin, I had the feeling that is a bad idea to read your mail before I go to sleep. But I was interested in what you have to say since you made quite an effort in your reply to me. And now I can´t sleep since my head if full of thoughts and I am full of emotions as well. With that I perceive starts an answer on a technical matter ends with what I received as a dire personal attack: I.e. calling me names. I received this twice on this mailing list. Once from Lennart (being a dick now) and now from you calling me a troll. I will make an effort to reply to your mail and then most likely unsubscribe, cause for me I feel like being in an hostile environment. Am Dienstag, 21. Oktober 2014, 22:14:48 schrieb Colin Guthrie: Martin Steigerwald wrote on 21/10/14 10:25: Am Mittwoch, 8. Oktober 2014, 10:54:00 schrieb Lennart Poettering: On Tue, 07.10.14 23:40, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote: My question really isn't why are the Debian dependencies the way they are. I understand that. I was trying to highlight the strange situation of a desktop application requiring a particular init system. I *think* this is a result of the init portion of systemd being bundled together with the logind portion of systemd. To me (unfamiliar with the systemd code) those two functions seem distinct enough to merit being separate. Debian can't easily separate what the systemd devs have developed as a single binary, so we end up with these strange dependency chains. Single binary is false of course. Logind is developed as a separate program, which is why systemd-shim is possible at all. AFAIK the actual relevant dependencies go as follows: First, there's a good reason why logind requires cgroup functionality. And there's a good reason why cgroup functionality is best implemented together with init (see http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInterface/ for more info). So it's not quite directly logind has to depend on systemd as init, but logind has to depend on the system having cgroup support, and there's no equally good cgroup support available for inits other than systemd. It is possible to provide the relevant cgroup interfaces in or on top of another init, as the systemd-sysv + cgmanager combination attempts to do. But it is not trivial to do, as bugs and implementation delays with that combo have shown, and it's quite likely that there will be more problems in the future. It's not a particularly good idea to use the less-tested and less-reliable systemd-shim instead of the more reliable systemd. Thus the overall result is that yes, it does make sense to switch machines to systemd when you add certain functionality, even if that functionality does not appear to be directly tied to the init system at first glance. Also note that the systemd concepts logind makes use of are also exported in its own API. The scopes and slices that logind uses are exposed on its bus objects, and they are used by tools like loginctl to do their work. The scopes and slices concept does not exist elsewhere, and there's nothing comparable around, so even if we wanted we couldn't make logind work on anything else. Then why in the first hand are the scopes and slices concepts within systemd *tightly* coupled when it is functionality that makes sense to be utilitizes in a component that provides a different functionality. I wonder what functionality systemd provides right now in one more or less tightly coupled package: - early boot process PID 1 - managing kernel drivers and device files (via coupled udev) Not PID 1 - resource control for process groups PID 1. - logging Not PID 1 - core dumping Not PID 1 - network while I think a distro-unified way to approaching network that replaces the current distro specific ways like ifupdown and so on, why does it have to be coupled with systemd? Not PID 1 - cron, although that at least in Debian is separate as systemd-cron Partly PID 1 Thats rather much. Really rather much. Way more than traditonal sysvinit covered. This is because traditional sysvinit covered PID 1 and performing it's job (if you consider e.g. killall5 and friends). You seem to be conflating systemd as PID 1 and systemd as a project. These are two completely different things. No. I am concerned about both. The functionality that is stuffed inside PID 1 which is more than 1,3 MiB and also sports user session functionality. And what is coupled inside on project, more or less tight. There are many related things here and a lot of the sub components obviously use a lot of the same functionality and utility functions. This is achieved via a shared library with a private API. The only reason that all these separate parts are developed as part of the systemd project is
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
2014-10-22 1:48 GMT+02:00 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: On Sat, Oct 18, 2014 at 06:30:02PM +0200, Ronny Chevalier wrote: It helps editing units by either creating a drop-in file, like /etc/systemd/system/my.service.d/amendments.conf, or by copying the original unit from /usr/lib/systemd/ to /etc/systemd/ if the --full option is specified. Then it invokes an editor to the related files and daemon-reload is invoked when the editor exited successfully. Hm, this sequence doesn't sound right. A temporary file should be created, edited, and then atomically put in place, iff the editor exits successfully. I think we should follow in the footsteps of git here... and abort if the editor exits with an error. You are right, I will rework it this way. I'm not sure abou the name 'amendments.conf'. Wouldn't 'local.conf' be more idiomatic, and also easier to type? Ok See https://bugzilla.redhat.com/show_bug.cgi?id=906824 --- TODO | 2 - man/journalctl.xml| 6 +- man/less-variables.xml| 40 +++-- man/localectl.xml | 6 +- man/loginctl.xml | 6 +- man/machinectl.xml| 6 +- man/systemctl.xml | 49 +- man/systemd-analyze.xml | 6 +- man/timedatectl.xml | 6 +- src/systemctl/systemctl.c | 394 +- 10 files changed, 488 insertions(+), 33 deletions(-) There's no need to mangle all the xml files. It is possible to include specific parts of a file. See how standard-options.xml incorporated whole, and sometimes just specific parts using xi:include href=standard-options.xml xpointer=... /. Ok I will look into this. -xi:include href=less-variables.xml / +refsect1 +titleEnvironment/title + +xi:include href=less-variables.xml / +/refsect1 +varlistentry + termcommandedit replaceableNAME/replaceable.../command/term + + listitem +paraEdit one or more unit files, as specified on the command +line./para This wording is misleading, because the unit file actually will not be *edited*, but extended in the normal case where --full is not used. I'm missing an explanatory sentence here, something like An editor will be launched to edit a drop-in snippet (or a whole replacement file if --full is used), to extend or override the specified unit. Then the next paragraph about --system/--user/--global will be more natural. You are right, it's better this way. + +paraDepending on whether option--system/option (the default), +option--user/option, or option--global/option is specified, +this create a drop-in file for each units either for the system, +for the calling user or for all futures logins of all users. Then +the editor is invoked on them (see section Environment below)./para + +paraIf option--full/option is specified, this will copy the original +units instead of creating drop-in files./para + +paraAfter the units have been edited, the systemd configuration is +reloaded (in a way that is equivalent to commanddaemon-reload/command), +but it does not restart or reload the units./para + +paraNote that this command cannot be used with option--runtime/option or +to remotely edit units./para + /listitem +/varlistentry /variablelist /refsect2 +variablelist class='environment-variables' + varlistentry +termvarname$SYSTEMD_EDITOR/varname/term + +listitemparaEditor to use when editing units; overrides +varname$EDITOR/varname and varname$VISUAL/varname. If neither +varname$SYSTEMD_EDITOR/varname nor varname$EDITOR/varname nor +varname$VISUAL/varname are present or if it is set to an empty +string or if their execution failed, systemctl will try to execute well +known editors in this order: + citerefentryrefentrytitlenano/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevim/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevi/refentrytitlemanvolnum1/manvolnum/citerefentry. +/para/listitem + /varlistentry +/variablelist +xi:include href=less-variables.xml / + /refsect1 refsect1 titleSee Also/title @@ -1572,7 +1617,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket systemd-udevd.service citerefentryrefentrytitlesystemd.resource-management/refentrytitlemanvolnum5/manvolnum/citerefentry, citerefentryrefentrytitlesystemd.special/refentrytitlemanvolnum7/manvolnum/citerefentry, citerefentry project='man-pages'refentrytitlewall/refentrytitlemanvolnum1/manvolnum/citerefentry, -
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
2014-10-22 2:13 GMT+02:00 Ronny Chevalier chevalier.ro...@gmail.com: 2014-10-22 1:48 GMT+02:00 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: On Sat, Oct 18, 2014 at 06:30:02PM +0200, Ronny Chevalier wrote: It helps editing units by either creating a drop-in file, like /etc/systemd/system/my.service.d/amendments.conf, or by copying the original unit from /usr/lib/systemd/ to /etc/systemd/ if the --full option is specified. Then it invokes an editor to the related files and daemon-reload is invoked when the editor exited successfully. Hm, this sequence doesn't sound right. A temporary file should be created, edited, and then atomically put in place, iff the editor exits successfully. I think we should follow in the footsteps of git here... and abort if the editor exits with an error. You are right, I will rework it this way. I'm not sure abou the name 'amendments.conf'. Wouldn't 'local.conf' be more idiomatic, and also easier to type? Ok See https://bugzilla.redhat.com/show_bug.cgi?id=906824 --- TODO | 2 - man/journalctl.xml| 6 +- man/less-variables.xml| 40 +++-- man/localectl.xml | 6 +- man/loginctl.xml | 6 +- man/machinectl.xml| 6 +- man/systemctl.xml | 49 +- man/systemd-analyze.xml | 6 +- man/timedatectl.xml | 6 +- src/systemctl/systemctl.c | 394 +- 10 files changed, 488 insertions(+), 33 deletions(-) There's no need to mangle all the xml files. It is possible to include specific parts of a file. See how standard-options.xml incorporated whole, and sometimes just specific parts using xi:include href=standard-options.xml xpointer=... /. Ok I will look into this. -xi:include href=less-variables.xml / +refsect1 +titleEnvironment/title + +xi:include href=less-variables.xml / +/refsect1 +varlistentry + termcommandedit replaceableNAME/replaceable.../command/term + + listitem +paraEdit one or more unit files, as specified on the command +line./para This wording is misleading, because the unit file actually will not be *edited*, but extended in the normal case where --full is not used. I'm missing an explanatory sentence here, something like An editor will be launched to edit a drop-in snippet (or a whole replacement file if --full is used), to extend or override the specified unit. Then the next paragraph about --system/--user/--global will be more natural. You are right, it's better this way. + +paraDepending on whether option--system/option (the default), +option--user/option, or option--global/option is specified, +this create a drop-in file for each units either for the system, +for the calling user or for all futures logins of all users. Then +the editor is invoked on them (see section Environment below)./para + +paraIf option--full/option is specified, this will copy the original +units instead of creating drop-in files./para + +paraAfter the units have been edited, the systemd configuration is +reloaded (in a way that is equivalent to commanddaemon-reload/command), +but it does not restart or reload the units./para + +paraNote that this command cannot be used with option--runtime/option or +to remotely edit units./para + /listitem +/varlistentry /variablelist /refsect2 +variablelist class='environment-variables' + varlistentry +termvarname$SYSTEMD_EDITOR/varname/term + +listitemparaEditor to use when editing units; overrides +varname$EDITOR/varname and varname$VISUAL/varname. If neither +varname$SYSTEMD_EDITOR/varname nor varname$EDITOR/varname nor +varname$VISUAL/varname are present or if it is set to an empty +string or if their execution failed, systemctl will try to execute well +known editors in this order: + citerefentryrefentrytitlenano/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevim/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevi/refentrytitlemanvolnum1/manvolnum/citerefentry. +/para/listitem + /varlistentry +/variablelist +xi:include href=less-variables.xml / + /refsect1 refsect1 titleSee Also/title @@ -1572,7 +1617,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket systemd-udevd.service citerefentryrefentrytitlesystemd.resource-management/refentrytitlemanvolnum5/manvolnum/citerefentry, citerefentryrefentrytitlesystemd.special/refentrytitlemanvolnum7/manvolnum/citerefentry, citerefentry
Re: [systemd-devel] [PATCH v3] systemctl: add edit verb
On Wed, Oct 22, 2014 at 02:42:13AM +0200, Ronny Chevalier wrote: 2014-10-22 2:13 GMT+02:00 Ronny Chevalier chevalier.ro...@gmail.com: 2014-10-22 1:48 GMT+02:00 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: On Sat, Oct 18, 2014 at 06:30:02PM +0200, Ronny Chevalier wrote: It helps editing units by either creating a drop-in file, like /etc/systemd/system/my.service.d/amendments.conf, or by copying the original unit from /usr/lib/systemd/ to /etc/systemd/ if the --full option is specified. Then it invokes an editor to the related files and daemon-reload is invoked when the editor exited successfully. Hm, this sequence doesn't sound right. A temporary file should be created, edited, and then atomically put in place, iff the editor exits successfully. I think we should follow in the footsteps of git here... and abort if the editor exits with an error. You are right, I will rework it this way. I'm not sure abou the name 'amendments.conf'. Wouldn't 'local.conf' be more idiomatic, and also easier to type? Ok See https://bugzilla.redhat.com/show_bug.cgi?id=906824 --- TODO | 2 - man/journalctl.xml| 6 +- man/less-variables.xml| 40 +++-- man/localectl.xml | 6 +- man/loginctl.xml | 6 +- man/machinectl.xml| 6 +- man/systemctl.xml | 49 +- man/systemd-analyze.xml | 6 +- man/timedatectl.xml | 6 +- src/systemctl/systemctl.c | 394 +- 10 files changed, 488 insertions(+), 33 deletions(-) There's no need to mangle all the xml files. It is possible to include specific parts of a file. See how standard-options.xml incorporated whole, and sometimes just specific parts using xi:include href=standard-options.xml xpointer=... /. Ok I will look into this. -xi:include href=less-variables.xml / +refsect1 +titleEnvironment/title + +xi:include href=less-variables.xml / +/refsect1 +varlistentry + termcommandedit replaceableNAME/replaceable.../command/term + + listitem +paraEdit one or more unit files, as specified on the command +line./para This wording is misleading, because the unit file actually will not be *edited*, but extended in the normal case where --full is not used. I'm missing an explanatory sentence here, something like An editor will be launched to edit a drop-in snippet (or a whole replacement file if --full is used), to extend or override the specified unit. Then the next paragraph about --system/--user/--global will be more natural. You are right, it's better this way. + +paraDepending on whether option--system/option (the default), +option--user/option, or option--global/option is specified, +this create a drop-in file for each units either for the system, +for the calling user or for all futures logins of all users. Then +the editor is invoked on them (see section Environment below)./para + +paraIf option--full/option is specified, this will copy the original +units instead of creating drop-in files./para + +paraAfter the units have been edited, the systemd configuration is +reloaded (in a way that is equivalent to commanddaemon-reload/command), +but it does not restart or reload the units./para + +paraNote that this command cannot be used with option--runtime/option or +to remotely edit units./para + /listitem +/varlistentry /variablelist /refsect2 +variablelist class='environment-variables' + varlistentry +termvarname$SYSTEMD_EDITOR/varname/term + +listitemparaEditor to use when editing units; overrides +varname$EDITOR/varname and varname$VISUAL/varname. If neither +varname$SYSTEMD_EDITOR/varname nor varname$EDITOR/varname nor +varname$VISUAL/varname are present or if it is set to an empty +string or if their execution failed, systemctl will try to execute well +known editors in this order: + citerefentryrefentrytitlenano/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevim/refentrytitlemanvolnum1/manvolnum/citerefentry, + citerefentryrefentrytitlevi/refentrytitlemanvolnum1/manvolnum/citerefentry. +/para/listitem + /varlistentry +/variablelist +xi:include href=less-variables.xml / + /refsect1 refsect1 titleSee Also/title @@ -1572,7 +1617,7 @@ kobject-uevent 1 systemd-udevd-kernel.socket systemd-udevd.service
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
On Wed, 2014-10-22 at 02:13 +0200, Martin Steigerwald wrote: With that I perceive starts an answer on a technical matter ends with what I received as a dire personal attack: I.e. calling me names. I think it was a mostly justified criticism of your posting style here. I will make an effort to reply to your mail and then most likely unsubscribe, cause for me I feel like being in an hostile environment. If you post such strongly worded criticism of people's work (which I don't consider really justified criticism either) then IMO you should tolerate that level negative response without starting to complain about hostile environment. Upstream systemd has a very high development speed. Which you may view as good. And heck, yes, it has its advantages I agree. But to me it also seems that this speed partly come due to what you wrote above as the easy way of developing things. And that easy way to develop things, I argue now, makes it more difficult for people wanting to port to different platforms, people only wanting a subset of systemd and people who want to adapt systemd. Those latter are much smaller groups than the number of people who just need a well-working init system for their Linux machine. It wouldn't make sense to sacrifice the functionality of init just to make porting easier. systemd provides more and more functionality that desktops like to use, that other tools like to use. When Microsoft back then did something like this it was called Embrace, Extend and Extinguish¹… Really… it matches quite closely. Oh come on! This is just an attack and FUD. You make repeated claims of coming in good faith etc. and seem to dismiss any technical defence being made with vague references and then you bring out a aggressive and argumentative statement like the above. That is the impression you get. I think I replied to technical arguments as well. The above does not match the definition of Embrace, extend and extinguish (see for example the Wikipedia definition at http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish ). It's a lot more specific than just a product manager to push competing ones out of the market, and pretty much requires intentional malice to apply. IMO it was quite accurate to call your claim attack/FUD. What I tried to achieve is to describe and interpret what I see regarding the state of systemd as I understand it now, and granted my understanding may not be complete, sure, and also describe and interpret behavior I have seen. And also summarize some of this from the feedback I read elsewhere. What I didn´t try to achieve was attacking persons. Yet, I interpret your reaction to me as if I attacked you. So I am obviously not producing the outcome I wanted to produce. And thats one reason why I think I will stop doing what I am doing after this mail and unsubscribe from this list for a while. Actually I think I made my point. I tried to channel some of the dire concerns and uproar and polarity and split tendencies upstream. I see this happening to my beloved distribution Debian and I am not happy about it. The systemd debates and polarity within Debian I consider as being harmful. And it was my intention to address some of this upstream in order to discuss what can be done to first *understand* why it triggers this polarity and what can be done to address this. Maybe your *intention* was to address reasons for controversy in a constructive manner, but I do not think you succeeded very well. Several of your points had already been made by others before - many, many times. You bring up little that systemd authors would not have already addressed before. Things like your Embrace, extend and extinguish comparison above are attacks with little constructive content. And when presented with technical justification to develop certain things in the same project, or keep certain functionality in PID 1, you seem to largely ignore it. Yes, it is a tradeoff, and you can always find some negative side. But you won't achieve anything by ignoring the answers and talking about the negative sides, if you can't make a better argument why the tradeoff would be wrong overall. Of course this criticism is listened to and often actions are taken because of it, but what do you expect the outcome to be? Do you expect all the repos to be split up? Stable APIs exported and supported? What outcome do you actually *want* here? You seem to be doing lots of complaining, but very little in the actual suggesting what could be done different that has not already been addressed technically. You may disagree about that end decision but that's just the way it is sometimes? The people doing the work ultimately get to make the decisions. Yeah, thats the do-ocracy aspect of things. Still if what I do again and again and again triggers much of polarity and resistance, I´d ask myself whats going
Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance
The system project seems to have made great strides to the point that now there are definite philosophical differences. The people on both sides have their own opinions of what the direction should be given how much progress has occurred. I think this would a very good time for a fork of the project. The systemd developers have rightly pointed out that this is free ware and if there is disagreement as has happened in the past, fork the project. The people who disagree with the current direction have a solid base to build on, fork the project take it in the direction you feel it should go. It will be interesting to see how the two sides progress. -Original Message- From: systemd-devel [mailto:systemd-devel-boun...@lists.freedesktop.org] On Behalf Of Martin Steigerwald Sent: Tuesday, October 21, 2014 7:13 PM To: Colin Guthrie Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] I wonder… why systemd provokes this amount of polarity and resistance Colin, I had the feeling that is a bad idea to read your mail before I go to sleep. But I was interested in what you have to say since you made quite an effort in your reply to me. And now I can´t sleep since my head if full of thoughts and I am full of emotions as well. With that I perceive starts an answer on a technical matter ends with what I received as a dire personal attack: I.e. calling me names. I received this twice on this mailing list. Once from Lennart (being a dick now) and now from you calling me a troll. I will make an effort to reply to your mail and then most likely unsubscribe, cause for me I feel like being in an hostile environment. Am Dienstag, 21. Oktober 2014, 22:14:48 schrieb Colin Guthrie: Martin Steigerwald wrote on 21/10/14 10:25: Am Mittwoch, 8. Oktober 2014, 10:54:00 schrieb Lennart Poettering: On Tue, 07.10.14 23:40, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: On Tue, 2014-10-07 at 14:15 -0400, Rob Owens wrote: My question really isn't why are the Debian dependencies the way they are. I understand that. I was trying to highlight the strange situation of a desktop application requiring a particular init system. I *think* this is a result of the init portion of systemd being bundled together with the logind portion of systemd. To me (unfamiliar with the systemd code) those two functions seem distinct enough to merit being separate. Debian can't easily separate what the systemd devs have developed as a single binary, so we end up with these strange dependency chains. Single binary is false of course. Logind is developed as a separate program, which is why systemd-shim is possible at all. AFAIK the actual relevant dependencies go as follows: First, there's a good reason why logind requires cgroup functionality. And there's a good reason why cgroup functionality is best implemented together with init (see http://www.freedesktop.org/wiki/Software/systemd/ControlGroupInter face/ for more info). So it's not quite directly logind has to depend on systemd as init, but logind has to depend on the system having cgroup support, and there's no equally good cgroup support available for inits other than systemd. It is possible to provide the relevant cgroup interfaces in or on top of another init, as the systemd-sysv + cgmanager combination attempts to do. But it is not trivial to do, as bugs and implementation delays with that combo have shown, and it's quite likely that there will be more problems in the future. It's not a particularly good idea to use the less-tested and less-reliable systemd-shim instead of the more reliable systemd. Thus the overall result is that yes, it does make sense to switch machines to systemd when you add certain functionality, even if that functionality does not appear to be directly tied to the init system at first glance. Also note that the systemd concepts logind makes use of are also exported in its own API. The scopes and slices that logind uses are exposed on its bus objects, and they are used by tools like loginctl to do their work. The scopes and slices concept does not exist elsewhere, and there's nothing comparable around, so even if we wanted we couldn't make logind work on anything else. Then why in the first hand are the scopes and slices concepts within systemd *tightly* coupled when it is functionality that makes sense to be utilitizes in a component that provides a different functionality. I wonder what functionality systemd provides right now in one more or less tightly coupled package: - early boot process PID 1 - managing kernel drivers and device files (via coupled udev) Not PID 1 - resource control for process groups PID 1. - logging Not PID 1 - core dumping Not PID 1 - network while I think a distro-unified way to approaching network that replaces the current
