Re: [systemd-devel] Improving module loading
Hi, -Original Message- From: systemd-devel [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Umut Tezduyar Lindskog Sent: Tuesday, December 16, 2014 4:55 PM To: systemd-devel@lists.freedesktop.org Subject: [systemd-devel] Improving module loading Hi, Is there a reason why systemd-modules-load is loading modules sequentially? Few things can happen simultaneously like resolving the symbols etc. Seems like modules_mutex is common on module loads which gets locked up on few occasions throughout the execution of sys_init_module. We are actually doing this (in embedded systems which need to be up very fast with limited resources) and gained a lot. Mainly, IO and CPU can be better utilized by loading modules in parallel (one module is loaded while another one probes for hardware or is doing memory initialization). The other thought is, what is the preferred way of loading modules when they are needed. Do they have to be loaded on ExecStartPre= or as a separate service which has ExecStart that uses kmod to load them? Wouldn't it be useful to have something like ExecStartModule=? I had such a discussion earlier with some of the systemd guys. My intention was to introduce an additional unit for module loading for exactly the reason you mentioned. The following (reasonable) outcome was: - It is dangerous to load kernel modules from PID 1 since module loading can get stuck - Since modules are actually loaded with the thread that calls the syscall, systemd would need additional threads - Multi Threading is not really aimed in systemd for stability reasons The probably safest way to do what you intended is to use an additional process to load your modules, which could be easily done by using ExecStartPre= in a service file. We are doing it exactly this way not with kmod but with a tool that loads modules in parallel. Btw: Be careful with synchronization. We found that lots of kernel modules are exporting device nodes in the background (alsa, some graphics driver, ...). With the proceeding mentioned above, you are moving the kernel module loading and the actual use of the driver interface very close together in time. This might lead to race conditions. It is even worse when you need to access sys attributes, which are exported by some drivers even after the device is already available and uevents have been sent out. For such modules, there actually is no other way for synchronization but waiting for the attributes to appear. Umut ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel Best regards Marko Hoyer Software Group II (ADITG/SW2) Tel. +49 5121 49 6948 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH] man (5) machine-info: add hostnamed chassis type embedded
man machine-info lacks hostnamed chassis type embedded as introduced in 218. The following lines should fix this. --- man/machine-info.xml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/man/machine-info.xml b/man/machine-info.xml index c654daa..9bf2bcb 100644 --- a/man/machine-info.xml +++ b/man/machine-info.xml @@ -139,7 +139,8 @@ literalserver/literal, literaltablet/literal, literalhandset/literal, - literalwatch/literal, as well as + literalwatch/literal and + literalembedded/literal as well as the special chassis types literalvm/literal and literalcontainer/literal for -- 2.2.1 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Improving module loading
On 16 Dec 2014 17:21, Umut Tezduyar Lindskog u...@tezduyar.com wrote: On Tue, Dec 16, 2014 at 4:59 PM, Tom Gundersen t...@jklm.no wrote: On Tue, Dec 16, 2014 at 4:54 PM, Umut Tezduyar Lindskog u...@tezduyar.com wrote: The other thought is, what is the preferred way of loading modules when they are needed. Rely on kernel autoloading. Not all modules support that yet, but most do. What do you have in mind? We have some modules that we don't need them to be loaded so early. We much prefer them to be loaded when they are needed. For example we don't need to load the SD driver module until the service that uses SD driver is starting. With this idea in mind I started some investigation. Then I realized that our CPU utilization is not that high during module loading and I blame it to the sequential loading of modules. I am thinking this can be improved on systemd-modules-load side. We can probably improve the module loading by making it use worker processes similar to how udev works. In principle this could cause problems with things making assumptions on the order of module loading, so that is something to keep in mind. That said, note that most modules will be loaded by udev which already does it in parallel... Do they have to be loaded on ExecStartPre= or as a separate service which has ExecStart that uses kmod to load them? Wouldn't it be useful to have something like ExecStartModule=? I'd much rather we improve the autoloading support... My understanding is autoloading support is loading a module if the hardware is available. That, or for non-hardware modules when the functionally is first used (networking, filesystems, ...). What I am after is though loading the module when they are needed. This sounds really fragile to me (having to encode this dependency everywhere rather than just always assume the functionality is available). Cheers, Tom ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [PATCH 2/2] verbs: allow matching by prefix
Be nice to command line users and allow matching prefixes of verbs, similar to the way getopt_long operates. If the prefix cannot be resolved to a singular verb, log the ambiguity and return an error. --- I'm not thrilled about adding this to every manpage out of concern that it might get out of sync, but I don't really see a better way. man/bootctl.xml| 3 ++- man/busctl.xml | 3 ++- man/coredumpctl.xml| 3 ++- man/hostnamectl.xml| 3 ++- man/kernel-install.xml | 3 ++- man/localectl.xml | 3 ++- man/loginctl.xml | 3 ++- man/systemctl.xml | 3 ++- man/telinit.xml| 3 ++- src/shared/verbs.c | 58 ++ src/test/test-verbs.c | 9 11 files changed, 71 insertions(+), 23 deletions(-) diff --git a/man/bootctl.xml b/man/bootctl.xml index 5254022..93363bd 100644 --- a/man/bootctl.xml +++ b/man/bootctl.xml @@ -79,7 +79,8 @@ xi:include href=standard-options.xml xpointer=version / /variablelist -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry diff --git a/man/busctl.xml b/man/busctl.xml index 285725e..a215a7e 100644 --- a/man/busctl.xml +++ b/man/busctl.xml @@ -260,7 +260,8 @@ along with systemd; If not, see http://www.gnu.org/licenses/. refsect1 titleCommands/title -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry diff --git a/man/coredumpctl.xml b/man/coredumpctl.xml index ed84621..28cce85 100644 --- a/man/coredumpctl.xml +++ b/man/coredumpctl.xml @@ -111,7 +111,8 @@ /variablelist -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry diff --git a/man/hostnamectl.xml b/man/hostnamectl.xml index de15402..51cb995 100644 --- a/man/hostnamectl.xml +++ b/man/hostnamectl.xml @@ -135,7 +135,8 @@ xi:include href=standard-options.xml xpointer=version / /variablelist -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry diff --git a/man/kernel-install.xml b/man/kernel-install.xml index d4d0180..aa6aca2 100644 --- a/man/kernel-install.xml +++ b/man/kernel-install.xml @@ -79,7 +79,8 @@ along with systemd; If not, see http://www.gnu.org/licenses/. refsect1 titleCommands/title -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry termcommandadd replaceableKERNEL-VERSION/replaceable replaceableKERNEL-IMAGE/replaceable/command/term diff --git a/man/localectl.xml b/man/localectl.xml index b472b6b..8f5b442 100644 --- a/man/localectl.xml +++ b/man/localectl.xml @@ -113,7 +113,8 @@ xi:include href=standard-options.xml xpointer=no-pager / /variablelist -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry diff --git a/man/loginctl.xml b/man/loginctl.xml index 749db92..f2162ac 100644 --- a/man/loginctl.xml +++ b/man/loginctl.xml @@ -165,7 +165,8 @@ xi:include href=standard-options.xml xpointer=no-pager / /variablelist -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para variablelist varlistentry diff --git a/man/systemctl.xml b/man/systemctl.xml index d1991e0..91fcba3 100644 --- a/man/systemctl.xml +++ b/man/systemctl.xml @@ -557,7 +557,8 @@ along with systemd; If not, see http://www.gnu.org/licenses/. refsect1 titleCommands/title -paraThe following commands are understood:/para +paraCommands can be specified by exact match or unambiguous prefix match. +The following commands are understood:/para
[systemd-devel] [PATCH 1/2] test-verbs: add unit tests for verbs minilib
---
Makefile.am | 9 +-
src/test/test-verbs.c | 78 +++
2 files changed, 86 insertions(+), 1 deletion(-)
create mode 100644 src/test/test-verbs.c
diff --git a/Makefile.am b/Makefile.am
index baa1398..db7dd46 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1384,7 +1384,8 @@ tests += \
test-locale-util \
test-execute \
test-copy \
- test-cap-list
+ test-cap-list \
+ test-verbs
EXTRA_DIST += \
test/a.service \
@@ -1651,6 +1652,12 @@ test_tmpfiles_LDADD = \
test_namespace_SOURCES = \
src/test/test-namespace.c
+test_verbs_SOURCES = \
+ src/test/test-verbs.c
+
+test_verbs_LDADD = \
+ libsystemd-shared.la
+
test_namespace_LDADD = \
libsystemd-core.la
diff --git a/src/test/test-verbs.c b/src/test/test-verbs.c
new file mode 100644
index 000..0fcdd9e
--- /dev/null
+++ b/src/test/test-verbs.c
@@ -0,0 +1,78 @@
+/***
+ This file is part of systemd.
+
+ Copyright 2014 systemd developers
+
+ systemd is free software; you can redistribute it and/or modify it
+ under the terms of the GNU Lesser General Public License as published by
+ the Free Software Foundation; either version 2.1 of the License, or
+ (at your option) any later version.
+
+ systemd is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with systemd; If not, see http://www.gnu.org/licenses/.
+***/
+
+#include macro.h
+#include strv.h
+#include verbs.h
+
+static int noop_dispatcher(int argc, char *argv[], void *userdata) {
+return 0;
+}
+
+#define test_dispatch_one(argv, verbs, expected) \
+optind = 0; \
+assert_se(dispatch_verb(strv_length(argv), argv, verbs, NULL) ==
expected);
+
+static void test_verbs(void) {
+static const Verb verbs[] = {
+{ help,VERB_ANY, VERB_ANY, 0,
noop_dispatcher },
+{ list-images, VERB_ANY, 1,0,
noop_dispatcher },
+{ list,VERB_ANY, 2,VERB_DEFAULT,
noop_dispatcher },
+{ status, 2,VERB_ANY, 0,
noop_dispatcher },
+{ show,VERB_ANY, VERB_ANY, 0,
noop_dispatcher },
+{ terminate, 2,VERB_ANY, 0,
noop_dispatcher },
+{ login, 2,2,0,
noop_dispatcher },
+{ copy-to, 3,4,0,
noop_dispatcher },
+{}
+};
+
+/* not found */
+test_dispatch_one(STRV_MAKE(command-not-found), verbs, -EINVAL);
+
+/* found */
+test_dispatch_one(STRV_MAKE(show), verbs, 0);
+
+/* found, too few args */
+test_dispatch_one(STRV_MAKE(copy-to, foo), verbs, -EINVAL);
+
+/* found, meets min args */
+test_dispatch_one(STRV_MAKE(status, foo, bar), verbs, 0);
+
+/* found, too many args */
+test_dispatch_one(STRV_MAKE(copy-to, foo, bar, baz, quux,
qaax), verbs, -EINVAL);
+
+/* no verb, but a default is set */
+test_dispatch_one(STRV_MAKE_EMPTY, verbs, 0);
+}
+
+static void test_verbs_no_default(void) {
+static const Verb verbs[] = {
+{ help, VERB_ANY, VERB_ANY, 0, noop_dispatcher },
+{},
+};
+
+test_dispatch_one(STRV_MAKE(NULL), verbs, -EINVAL);
+}
+
+int main(int argc, char *argv[]) {
+test_verbs();
+test_verbs_no_default();
+
+return 0;
+}
--
2.2.0
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Improving module loading
On Sat, Dec 20, 2014 at 10:45:34AM +, Hoyer, Marko (ADITG/SW2) wrote: Hi, -Original Message- From: systemd-devel [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Umut Tezduyar Lindskog Sent: Tuesday, December 16, 2014 4:55 PM To: systemd-devel@lists.freedesktop.org Subject: [systemd-devel] Improving module loading Hi, Is there a reason why systemd-modules-load is loading modules sequentially? Few things can happen simultaneously like resolving the symbols etc. Seems like modules_mutex is common on module loads which gets locked up on few occasions throughout the execution of sys_init_module. We are actually doing this (in embedded systems which need to be up very fast with limited resources) and gained a lot. Mainly, IO and CPU can be better utilized by loading modules in parallel (one module is loaded while another one probes for hardware or is doing memory initialization). If you have control over your kernel, why not just build the modules into the kernel, then all of this isn't an issue at all and there is no overhead of module loading? greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Improving module loading
On Sat, Dec 20, 2014 at 6:10 PM, Greg KH gre...@linuxfoundation.org wrote: On Sat, Dec 20, 2014 at 10:45:34AM +, Hoyer, Marko (ADITG/SW2) wrote: Hi, -Original Message- From: systemd-devel [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Umut Tezduyar Lindskog Sent: Tuesday, December 16, 2014 4:55 PM To: systemd-devel@lists.freedesktop.org Subject: [systemd-devel] Improving module loading Hi, Is there a reason why systemd-modules-load is loading modules sequentially? Few things can happen simultaneously like resolving the symbols etc. Seems like modules_mutex is common on module loads which gets locked up on few occasions throughout the execution of sys_init_module. We are actually doing this (in embedded systems which need to be up very fast with limited resources) and gained a lot. Mainly, IO and CPU can be better utilized by loading modules in parallel (one module is loaded while another one probes for hardware or is doing memory initialization). If you have control over your kernel, why not just build the modules into the kernel, then all of this isn't an issue at all and there is no overhead of module loading? For us, licenses are the problem. Umut greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Improving module loading
Hi Marko, Thank you very much for your feedback! On Sat, Dec 20, 2014 at 11:45 AM, Hoyer, Marko (ADITG/SW2) mho...@de.adit-jv.com wrote: Hi, -Original Message- From: systemd-devel [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Umut Tezduyar Lindskog Sent: Tuesday, December 16, 2014 4:55 PM To: systemd-devel@lists.freedesktop.org Subject: [systemd-devel] Improving module loading Hi, Is there a reason why systemd-modules-load is loading modules sequentially? Few things can happen simultaneously like resolving the symbols etc. Seems like modules_mutex is common on module loads which gets locked up on few occasions throughout the execution of sys_init_module. We are actually doing this (in embedded systems which need to be up very fast with limited resources) and gained a lot. Mainly, IO and CPU can be better utilized by loading modules in parallel (one module is loaded while another one probes for hardware or is doing memory initialization). The other thought is, what is the preferred way of loading modules when they are needed. Do they have to be loaded on ExecStartPre= or as a separate service which has ExecStart that uses kmod to load them? Wouldn't it be useful to have something like ExecStartModule=? I had such a discussion earlier with some of the systemd guys. My intention was to introduce an additional unit for module loading for exactly the reason you mentioned. The following (reasonable) outcome was: Do you have links for the discussions, I cannot find them. systemd already has a service that loads the modules. - It is dangerous to load kernel modules from PID 1 since module loading can get stuck - Since modules are actually loaded with the thread that calls the syscall, systemd would need additional threads - Multi Threading is not really aimed in systemd for stability reasons The probably safest way to do what you intended is to use an additional process to load your modules, which could be easily done by using ExecStartPre= in a service file. We are doing it exactly this way not with kmod but with a tool that loads modules in parallel. Btw: Be careful with synchronization. We found that lots of kernel modules are exporting device nodes in the background (alsa, some graphics driver, ...). With the proceeding mentioned above, you are moving the kernel module loading and the actual use of the driver interface very close together in time. This might lead to race conditions. It is even worse when you need to access sys attributes, which are exported by some drivers even after the device is already available and uevents have been sent out. For such modules, there actually is no other way for synchronization but waiting for the attributes to appear. We are aware of the potential complications and races. But good to be reminded :) Umut Umut ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel Best regards Marko Hoyer Software Group II (ADITG/SW2) Tel. +49 5121 49 6948 ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] How to create a systemd service?
jenia.ivlev jenia.iv...@gmail.com schrieb: I want to start synapse (its a program that allows to run programs more easily) at login using systemd. So I created this systemd-service config file: [Unit] Description=Syanpse start up After=lxdm.service [Service] User=jenia ExecStart=/usr/bin/synapse [Install] WantedBy=multi-user.target I doesnt work neither automatically, nor by calling /usr/lib/systemd/system/synapse.service as so: systemctl start synapse It tells me: ● synapse.service - Syanpse start up Loaded: loaded (/usr/lib/systemd/system/synapse.service; enabled) Active: failed (Result: exit-code) since Tue 2014-12-16 21:23:11 EST; 8min ago Process: 8545 ExecStart=/usr/bin/synapse (code=exited, status=1/FAILURE) Main PID: 8545 (code=exited, status=1/FAILURE) Dec 16 21:23:11 station1 systemd[1]: Started Syanpse start up. Dec 16 21:23:11 station1 synapse[8545]: [02:23:11.039206 Info] Starting up... Dec 16 21:23:11 station1 synapse[8545]: [02:23:11.040620 Gtk-Warning] cannot open display: Dec 16 21:23:11 station1 systemd[1]: synapse.service: main process exited, code=exited, status=1/FAILURE Dec 16 21:23:11 station1 systemd[1]: Unit synapse.service entered failed state. Dec 16 21:23:11 station1 systemd[1]: synapse.service failed. The error say cannot open display. How do I make this work, can someone please tell me? You cannot run synapse as a system service. It's an X client which needs access to your X session after you logged in. I'm not sure if it could be solved by using systemd in --user mode but probably at least not now. There are afforts to use systemd in user mode to spawn the xsession of the user and use it as session manager, i.e. in KDE. But currently this does not work well enough to not be complicated. So I suggest sticking synapse into your session autostart like so (as a user, not root): $ ln -snf $(which synapse) $HOME/.config/autostart/ Then logout and in again. -- Replies to list only preferred. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Improving module loading
On Sat, Dec 20, 2014 at 06:40:49PM +0100, Umut Tezduyar Lindskog wrote: On Sat, Dec 20, 2014 at 6:10 PM, Greg KH gre...@linuxfoundation.org wrote: On Sat, Dec 20, 2014 at 10:45:34AM +, Hoyer, Marko (ADITG/SW2) wrote: Hi, -Original Message- From: systemd-devel [mailto:systemd-devel- boun...@lists.freedesktop.org] On Behalf Of Umut Tezduyar Lindskog Sent: Tuesday, December 16, 2014 4:55 PM To: systemd-devel@lists.freedesktop.org Subject: [systemd-devel] Improving module loading Hi, Is there a reason why systemd-modules-load is loading modules sequentially? Few things can happen simultaneously like resolving the symbols etc. Seems like modules_mutex is common on module loads which gets locked up on few occasions throughout the execution of sys_init_module. We are actually doing this (in embedded systems which need to be up very fast with limited resources) and gained a lot. Mainly, IO and CPU can be better utilized by loading modules in parallel (one module is loaded while another one probes for hardware or is doing memory initialization). If you have control over your kernel, why not just build the modules into the kernel, then all of this isn't an issue at all and there is no overhead of module loading? For us, licenses are the problem. Then you are on your own, and be prepared to deal with the legal issues involved. No sympathy here. greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 1/2] test-verbs: add unit tests for verbs minilib
Hi, On Sat, Dec 20, 2014 at 8:19 AM, Dave Reisner dreis...@archlinux.org wrote: --- Makefile.am | 9 +- src/test/test-verbs.c | 78 +++ 2 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 src/test/test-verbs.c diff --git a/Makefile.am b/Makefile.am index baa1398..db7dd46 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1384,7 +1384,8 @@ tests += \ test-locale-util \ test-execute \ test-copy \ - test-cap-list + test-cap-list \ + test-verbs Make sure you also add /test-verbs to the top level .gitignore. Cheers, Filipe ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
