[kbuild-devel] Re: External kernel modules, second try
Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes) distributions don't like to install the vmlinux since it's big(ish) and means customers need to download a new vmlinux at each kernel erratum. The same information is btw also present in System.map so imo the real solution is to make modpost use System.map instead ;) modules SUBDIRS=$PWD'' works against a read-only tree. Agree - should be easy to test using a CD. Are there an easy way to mount just a directory structure RO? not without making it a separate partition or loopback device. signature.asc Description: This is a digitally signed message part
[kbuild-devel] Re: External kernel modules, second try
On Sun, Mar 07, 2004 at 05:05:27PM +0100, Sam Ravnborg wrote: On Sun, Mar 07, 2004 at 03:01:31PM +0100, Arjan van de Ven wrote: and it's missing the symbols from module files. sure but the module files are generally installed... I'm aiming for a situation where I can build external modules, using an almost clean kernel src tree. Personally I don't see the point. I'm perfectly happy with the current situation with the exception of not using System.map and co. From a distribution kernel pov; I already ship a subset of files for building modules against (basically include/, the KConfig and makefiles), which only not 100% works because I don't ship vmlinux. pgp0.pgp Description: PGP signature
[kbuild-devel] Re: External kernel modules, second try
On Sun, 2004-03-07 at 14:46, Andreas Gruenbacher wrote: Hello Arjan, On Sun, 2004-03-07 at 14:03, Arjan van de Ven wrote: Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes) distributions don't like to install the vmlinux since it's big(ish) and means customers need to download a new vmlinux at each kernel erratum. The same information is btw also present in System.map so imo the real solution is to make modpost use System.map instead ;) System.map doesn't have the hashes, are you sure ? I could have sworn it had. and it's missing the symbols from module files. sure but the module files are generally installed... Now it would be possible to extract the modver symbols from the installed vmlinux and .ko files when needed, but note that we may be building modules for kernels that are not currently running, and for which those binaries are not even installed. So this sounds like a bad idea. I don't personally care about those; you need SOME stuff to build against obviously, and vmlinux is well over the top I agree that. But assuming the .ko's for the modules are there...you need those to use the kernel anyway. signature.asc Description: This is a digitally signed message part
[kbuild-devel] Re: External kernel modules, second try
On Sun, Mar 07, 2004 at 05:45:22PM +0100, Andreas Gruenbacher wrote: From a distribution kernel pov; I already ship a subset of files for building modules against (basically include/, the KConfig and makefiles), which only not 100% works because I don't ship vmlinux. We have tried that with our latest round of kernels (still 2.4), and the results have been mixed. You need various headers outside include/ for 2.6 has the biggest offender (scsi) fixed. some obscure external modules. Amazingly there are even external modules that make use of kernel C files. I can't imagine that being the case anymore, and for sure it won't be binary only ones ;) ), the default being the running kernel. The Red Hat kernel has had a partial solution for merging autoconf.h. It's a gross hack that we thankfully got rid of finally! pgp0.pgp Description: PGP signature
[kbuild-devel] Re: External kernel modules, second try
On Sun, Mar 07, 2004 at 01:44:58AM +0100, Andreas Gruenbacher wrote: Hello, here is the patch I posted previously that adds support for modversions in external kernel modules that are built outside the main kernel tree (first posting archived here: http://lwn.net/Articles/69148/). Relative to the original version, the attached version also works when compiling with O=. Hi Andreas. I have started to look into this. The changes im Makefile when you use SUBDIRS as a flag does not look pretty. What I have in mind is something like this: - Get rid of current use of SUBDIRS. It is no longer used in any arch Makefiles. - Introduce a KBUILD_EXTMOD variable that is only set when building external modules - Introduce a new method to be used when compiling external modules: make M=dir-to-module - Keeping the SUbDIRS notation for backward compatibility - When using SUBDIRS or M= the 'modules' target is implicit. - make clean and make mrproper/distclen only deletes files in the external module directory (as done in your patch) - Error out if any updadtes are requires in the kernel tree - Find a magic way to include a Kconfig file for the external module - Allow kbuild Makefiles to be named Kbuild, so local stuff can be in a file named Makefile file note: this can be achieved using makefile/Makefile today but it makes sense since there is not much 'Make' syntax left in kbuild makefiles today. Above will not be made in one go. My next step is to make a patch for the first four steps - to see the actual impact on current makefiles. Comments welcome! Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes) The patch also adds a modules_add target that does the equivalent of modules_install for one external module. Looks good. The third change is to remove one instance of temporary file creation inside the main kernel tree while external modules are built. I think there are still other cases where temp files in the kernel tree are used. IMHO they should all go away, so that a ``make -C $KERNEL_SOURCE modules SUBDIRS=$PWD'' works against a read-only tree. Agree - should be easy to test using a CD. Are there an easy way to mount just a directory structure RO? Sam --- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470alloc_id=3638op=click ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: External kernel modules, second try
Hello Arjan, On Sun, 2004-03-07 at 14:03, Arjan van de Ven wrote: Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes) distributions don't like to install the vmlinux since it's big(ish) and means customers need to download a new vmlinux at each kernel erratum. The same information is btw also present in System.map so imo the real solution is to make modpost use System.map instead ;) System.map doesn't have the hashes, and it's missing the symbols from module files. External modules may require symbols that live in another module. There are also a number of other minor differences that could probably be worked around. Now it would be possible to extract the modver symbols from the installed vmlinux and .ko files when needed, but note that we may be building modules for kernels that are not currently running, and for which those binaries are not even installed. So this sounds like a bad idea. Cheers, -- Andreas Gruenbacher [EMAIL PROTECTED] SUSE Labs, SUSE LINUX AG --- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470alloc_id=3638op=click ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: External kernel modules, second try
On Sun, 2004-03-07 at 15:01, Arjan van de Ven wrote: On Sun, 2004-03-07 at 14:46, Andreas Gruenbacher wrote: and it's missing the symbols from module files. sure but the module files are generally installed... Not when building for five different configurations on one machine. (And this is not a theoretical case.) Now it would be possible to extract the modver symbols from the installed vmlinux and .ko files when needed, but note that we may be building modules for kernels that are not currently running, and for which those binaries are not even installed. So this sounds like a bad idea. I don't personally care about those; you need SOME stuff to build against obviously, and vmlinux is well over the top I agree that. But assuming the .ko's for the modules are there...you need those to use the kernel anyway. The .ko's are really not there. And even if they were, I don't think we want to teach modpost to poke around in /lib/modules when it can be avoided easily. Cheers, -- Andreas Gruenbacher [EMAIL PROTECTED] SUSE Labs, SUSE LINUX AG --- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470alloc_id=3638op=click ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: External kernel modules, second try
On Sun, Mar 07, 2004 at 03:01:31PM +0100, Arjan van de Ven wrote: and it's missing the symbols from module files. sure but the module files are generally installed... I'm aiming for a situation where I can build external modules, using an almost clean kernel src tree. Which means that my make clean on steroids patch maybe is a bit too effective. The distintion point could be: make clean leaves only enough to build external modules. Otherwise we need a third (fourth) cleaning target, which is not desireable. Sam --- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470alloc_id=3638op=click ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: External kernel modules, second try
On Sun, Mar 07, 2004 at 02:32:14PM +0100, Andreas Gruenbacher wrote: What I have in mind is something like this: - Get rid of current use of SUBDIRS. It is no longer used in any arch Makefiles. - Introduce a KBUILD_EXTMOD variable that is only set when building external modules - Introduce a new method to be used when compiling external modules: make M=dir-to-module - Keeping the SUbDIRS notation for backward compatibility - When using SUBDIRS or M= the 'modules' target is implicit. Why not keep the SUBDIRS notation for external modules only then? That's what was documented to work since a long time; I see no big benefit in changing it if it can be avoided. Since a long time is from 2.5.5x roughly. The SUBDIRS= notation is not intuitive, and the M= notation follow the other similar options: O=, C=. SUBDIRS will be kept, but warned for in 2.7. Removed when 2.8 opens or similar. So I see no breakage here. - Find a magic way to include a Kconfig file for the external module This is where it gets pretty messy. You would also have a different configuration in the external module. I have no clear idea how that can work reasonably cleanly. If the solution becomes messy then I will just drop it. I do not see a big need here anyway. It will also create troubles: Can we modify .config etc. - Allow kbuild Makefiles to be named Kbuild, so local stuff can be in a file named Makefile file note: this can be achieved using makefile/Makefile today but it makes sense since there is not much 'Make' syntax left in kbuild makefiles today. The Makefile can already include both the kbuild and local stuff (same snippet I sent you in personal communication already): I know - but the incentive here was to seperate stuff out that does not belong in a Kbuild makefile. In 2.7 I may write a simple parser to create one single big Makefile, and then it will be good to have very simple Makefiles only. Now with mainline, when building external modules they will end up not having modversions. This is caused by the way .tmp_versions is handled, and is a real problem. There are two different ways how we are building external modules today: (1) after the kernel source tree was just compiled, so the kernel source tree still contains all the object files, (2) in a separate step, against an almost clean kernel source tree. Almost-clean means the tree contains a set of configuration files, and the modversions dump file. The modversions dump file elegantly solves both cases. You already convinced me about the usefullness of the modversions file. Can you try to make a patch that _only_ incorporate the modversions functionality. This we could ask Andrew to apply when reviewed. See it as first step. Then I will try to work out something for the functionality outlined before. Agree - should be easy to test using a CD. Are there an easy way to mount just a directory structure RO? Not sure what you mean exactly. My main motivation is this: we have a whole bunch of external modules that we build one after the other. Those external modules are notoriously nasty: We had cases where the modules fondled in kernel headers in the original tree. Of course then the next modules would build against a broken tree. To stop and detect such madness, I want to give modules a kernel source tree to which they have no write access, by chowning to a different user. Trees on read-only media are irrelevant IMHO. Actually we agree, I just did not think of chowing the files/dirs to catch the problems. Sam --- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470alloc_id=3638op=click ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: External kernel modules, second try
On Sun, Mar 07, 2004 at 05:45:22PM +0100, Andreas Gruenbacher wrote: All in all, in the end I changed my mind. I now think that it's better to build modules against a clean kernel source tree that additionally has the modversions file copied in. This already works when using O=. With the SUBDIRS= approach, the kernel source tree must include a few compiled files (scripts/ stuff), and it cannot be read-only. I'm still undecided whether it makes sense to disallow the SUBDIRS= approach completely and only allow building with O=. (Note that this doesn't change the modversion dump file argument.) When building with SUBDIRS=, you ideally want a (read-only) kernel source tree that can adapt to different configurations (e.g., by doing like this: make -C $KERNEL_SOURCE modules SUBDIRS=$PWD FLAVOR=bigsmp This is already possible. You can do: make -C $KERNEL_SRC SUBDIRS=$PWD O=output-dir modules or with my proposed syntax: make -C $KERNEL_SRC M=$PWD O=output-dir The files relevant for the module will be located in the $PWD dir, since they use absolute paths. ), the default being the running kernel. I do not want to have potentially distro specific solutions. So it depends if we can find a solution that most will agree on. Sam --- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470alloc_id=3638op=click ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
