> -----Original Message-----
> From: Lucas De Marchi [mailto:lucas.de.mar...@gmail.com]
> Sent: Monday, December 22, 2014 7:00 PM
> To: Lennart Poettering
> Cc: Hoyer, Marko (ADITG/SW2); systemd-devel@lists.freedesktop.org
> Subject: Re: [systemd-devel] Improving module loading
> On Mon, Dec 22, 2014 at 1:04 PM, Lennart Poettering
> <lenn...@poettering.net> wrote:
> > On Sat, 20.12.14 10:45, Hoyer, Marko (ADITG/SW2) (mho...@de.adit-
> jv.com) wrote:
> >
> >> I had such a discussion earlier with some of the systemd guys. My
> >> intention was to introduce an additional unit for module loading for
> >> exactly the reason you mentioned. The following (reasonable) outcome
> >> was:
> >>
> >> - It is dangerous to load kernel modules from PID 1 since module
> >>   loading can get stuck
> >
> >> - Since modules are actually loaded with the thread that calls the
> >>   syscall, systemd would need additional threads
> >
> >> - Multi Threading is not really aimed in systemd for stability
> >> reasons
> >>
> >> The probably safest way to do what you intended is to use an
> >> additional process to load your modules, which could be easily done
> >> by using ExecStartPre= in a service file. We are doing it exactly
> >> this way not with kmod but with a tool that loads modules in
> >> parallel.
> >
> > I'd be willing to merge a good patch that beefs up
> > systemd-modules-load to load the specified modules in parallel, with
> > one thread for each.
> >
> > We already have a very limited number of threaded bits in systemd,
> and
> > I figure out would be OK to do that for this too.
> >
> > Please keep the threading minimal though, i.e. one kmod context per
> > thread, so that we need no synchronization and no locking. One thread
> > per module, i.e. no worker thread logic with thread reusing. also,
> > please set a thred name, so that hanging module loading only hang one
> > specific thread and the backtrace shows which module is at fault.
> I'm skeptical you would get any speed up for that. I think it would be
> better to have some numbers shared before merging such a thing.

As I already outlined in my answer to Greg, the parallel loading was not our 
main motivation for inventing something new. We found that for some of our 
modules parallel loading gained us benefit, so we integrated this feature. 
Since we are not using udevd during startup at all, most of our modules are 
loaded manually. I've no idea how things are distributed between 
systemd-modules-load and udevd in conventional Linux desktop or server systems. 
If only a hand full of modules are actually loaded using systemd-modules-load, 
it is probably not worth optimizing at this end.

Has someone concrete numbers how many modules are loaded "by hand" using 
systemd-modules-load in a conventional system?

> If you have 1 context per module/thread you will need to initialize
> each context which is really the most expensive part in userspace,
> particularly if finit_module() is being used (which you should unless
> you have restrictions on the physical size taken by the modules). Bare
> in mind the udev logic has only 1 context, so the initialization is
> amortized among the multiple module load calls.

This does not really meet my experience. Once the kmod binary cache is in the 
VFS page buffer cache, it is really fast getting a new context even in new 
processes. The expensive thing about udev is that it starts very fast forking 
off worker processes. So at least one new context per process is created 
finally too. Additionally, the people who decide to use systemd-modules-load to 
load specific modules have good reasons for that. A prominent one is probably 
that udevd is not working for the respective module because no concrete device 
is coupled with it. I think we do not have so many kernel modules, which need 
to be handled like this which brings us again to the question if it is really 
worth pimping systemd-modules-load. 

> For the "don't load until it's needed" I very much prefer the static
> nodes approach we have. Shouldn't this be used instead of filling
> modules-load-d with lots of entries?

We are not using systemd-modules-load for applying this approach since it is 
trying to load all modules in one shot. We are executing our tool several times 
during startup to get up hardware piece by piece exactly at the point where it 
is needed. The tool is either executed like modprobe or with a configuration 
file containing a set of modules to be loaded in one shot and some other stuff 
needed for synchronization and setup.

> I really miss numbers here and more information on which modules are
> taking long because they are serialized.
> --
> Lucas De Marchi

Best regards

Marko Hoyer
Software Group II (ADITG/SW2)

Tel. +49 5121 49 6948
systemd-devel mailing list

Reply via email to