Re: Ответ: Ответ: Ответ: MODPROBE: next generation
Vladimir, can you try these two patches on top on current svn? In my limited testing they work. This one fixes a small buglet with gpl and strings finding their way into depfile, and also makes depmod usable. depmod is needed for people which build kernel and install modules, then reboot with /lib/modules mounted RO (no chance to generate and store depfile at boot time) -- vda b.patch Description: Binary data ___ busybox mailing list busybox@busybox.net http://busybox.net/cgi-bin/mailman/listinfo/busybox
Re: Ответ: Ответ: Ответ: MODPROBE: next generation
[CC [EMAIL PROTECTED] 2008/7/10 Vladimir Dronnikov [EMAIL PROTECTED]: and also makes depmod usable... depmod should be synonym to rm -f /path/modules.dep.bb; modprobe, IMO. Tell this to somebody who will be bitten by depmod -F System.map -ae 2.6.30 not working as expected during kernel build. See my version. I compiled 9 and a patched svn. handwave +1000 bytes larger than my version. What for...? Did you look thru my version? What are the points not to try it? If /lib/modules are read-only and depfile is not present, it scans entire module directory, including every module's body. Even if it is asked to load a module which can be found by filename. -- vda ___ busybox mailing list busybox@busybox.net http://busybox.net/cgi-bin/mailman/listinfo/busybox
Re: Ответ: Ответ: Ответ: MODPROBE: next generation
THe discussion have to happen on the mailing list! Please CC [EMAIL PROTECTED] 2008/7/10 Vladimir Dronnikov [EMAIL PROTECTED]: I liked the idea of not needing depfile at all and still being reasonably fast. And it's actually YOUR idea - I based my work on your depfile-less patch. :) The only catch is system startup. If 5 more seconds is not of big concern, let us get rid of modules.dep.bb. Again, there exist much more time consuming operations at startup so the delay is not so dramatic! There are people whose target is 1 second boot, total time. Don't have to match that, but keep it in mind as a yardstick. I am planning to make depfile generation optional... This would also solve the problem of modutils to be fool-proof. I know people who constantly try to edit something (like depfile :). But first I want to be sure that full-blown (all options on) version works for desktop users. Understood. Agreed. Just drank a cup of tee and another idea came. If we advance your approach of lazy scanning only needed modules further we could get rid of the problems at all. When we have to snoop into module file we should make _symlinks_ to the module file which are named after all the aliases and symbols it contains. That way the next time when a symbol:pci:foo:bar is queried we have direct answer: use this particular module! We already have to use fnmatch(), why not to use filesystem means directly? The only drawback is again RO filesystem, but a system has to have some RW filesystem, be it tmpfs or the other (/etc, e.g), right? Crazy idea. Might be fun to try. I propose to put links into aliases/ subdir. RO filesystem got to live without depmod info then. User's choice. The drawback is the size. With links, you can't use modules.dep.bb.bz2 trick. -- vda ___ busybox mailing list busybox@busybox.net http://busybox.net/cgi-bin/mailman/listinfo/busybox
Re: Ответ: Ответ: Ответ: MODPROBE: next generation
On Tuesday 08 July 2008 12:16, Vladimir Dronnikov wrote: 3. not read - become a master. open .modules.dep.do-not-edit!!! for exclusive write. all other concurrent modprobes have to wait till the master closes the file. This exclusive write stuff is notorious for breaking on non-local filesystems, NFS, etc... I prefer the scheme where atomicity is achieved by rename(). The spiky edges here are: i) by definition it is mandatory locking that is needed to lock .modules.dep.do-not-edit!!!. That way concurrent modprobes just block in open() and there's no need for ugly workarounds. Do we have mandatory locking? We do. Sometimes :) ii) string xreallocations. but you've introduced xrealloc_vector() which can fit. I've been using my modprobe about a week. When modprove enters alias searching loop, it tries to generate new modprobe.dep.bb for the benefit of other mdprobes, since it is going to scan all modules anyway. It creates modprobe.dep.bb.new with O_EXCL. If that fails, other modprobe is already doing that, so just wait for it to disappear (other modprobe will rename it to modprobe.dep.bb) and use modprobe.dep.bb. O_EXCL is definitely not enough here. All 20 concurrents will generate the full stuff and then the last spawned will win. Let me clarify: file is opened with O_CREAT + O_EXCL _before_ modprobe starts expensive directory scan. If it fails with EEXIST, modprobe understands that it is the second one and it will _not_ do the scan. It will wait for modprobe.dep.bb.new to disappear (first modprobe is expected to do the scan, write out modprobe.dep.bb.new and rename(modprobe.dep.bb.new, modprobe.dep.bb)). At this moment second modprobe opens and reads modprobe.dep.bb. This way, only one modprobe will do the scan. It is semi-trivial to implement that in modprobe-small.c, I will try to find the time in next few days for that. -- vda ___ busybox mailing list busybox@busybox.net http://busybox.net/cgi-bin/mailman/listinfo/busybox
Re: Ответ: Ответ: Ответ: MODPROBE: next generation
2008/7/9 Denys Vlasenko [EMAIL PROTECTED]: On Tuesday 08 July 2008 12:16, Vladimir Dronnikov wrote: 3. not read - become a master. open .modules.dep.do-not-edit!!! for exclusive write. all other concurrent modprobes have to wait till the master closes the file. This exclusive write stuff is notorious for breaking on non-local filesystems, NFS, etc... I prefer the scheme where atomicity is achieved by rename(). The spiky edges here are: i) by definition it is mandatory locking that is needed to lock .modules.dep.do-not-edit!!!. That way concurrent modprobes just block in open() and there's no need for ugly workarounds. Do we have mandatory locking? We do. Sometimes :) ii) string xreallocations. but you've introduced xrealloc_vector() which can fit. I've been using my modprobe about a week. When modprove enters alias searching loop, it tries to generate new modprobe.dep.bb for the benefit of other mdprobes, since it is going to scan all modules anyway. It creates modprobe.dep.bb.new with O_EXCL. If that fails, other modprobe is already doing that, so just wait for it to disappear (other modprobe will rename it to modprobe.dep.bb) and use modprobe.dep.bb. O_EXCL is definitely not enough here. All 20 concurrents will generate the full stuff and then the last spawned will win. Let me clarify: file is opened with O_CREAT + O_EXCL _before_ modprobe starts expensive directory scan. If it fails with EEXIST, modprobe understands that it is the second one and it will _not_ do the scan. It will wait for modprobe.dep.bb.new to disappear (first modprobe is expected to do the scan, write out modprobe.dep.bb.new and rename(modprobe.dep.bb.new, modprobe.dep.bb)). At this moment second modprobe opens and reads modprobe.dep.bb. This way, only one modprobe will do the scan. It is semi-trivial to implement that in modprobe-small.c, I will try to find the time in next few days for that. Vladimir, can you try these two patches on top on current svn? In my limited testing they work. -- vda 9.patch Description: Binary data a.patch Description: Binary data ___ busybox mailing list busybox@busybox.net http://busybox.net/cgi-bin/mailman/listinfo/busybox
Re: Ответ: Ответ: Ответ: MODPROBE: next generation
On Sunday 06 July 2008 13:58, Vladimir Dronnikov wrote: Yes. You see some kind of grepping of all modules is needed at startup, so why not to perform it explicitly? On my system it is about 20 modprobe calls with pci: aliases. The time to resolve them currently is an order of magnitude longer than just to cache the definitions once and then use the cache. I propose the following: modprobe tries to open modprobe.dep.bb and populates modinfo[] array from there (see current code in svn). Otherwise, it proceeds as it currently does. When modprove enters alias searching loop, it tries to generate new modprobe.dep.bb for the benefit of other mdprobes, since it is going to scan all modules anyway. It creates modprobe.dep.bb.new with O_EXCL. If that fails, other modprobe is already doing that, so just wait for it to disappear (other modprobe will rename it to modprobe.dep.bb) and use modprobe.dep.bb. Otherwise (modprobe.dep.bb.new created ok) generate it and rename to modprobe.dep.bb. Is this ok? -- vda ___ busybox mailing list busybox@busybox.net http://busybox.net/cgi-bin/mailman/listinfo/busybox
