On Tue, 29 May 2001, Keith Owens wrote:
> While your rules are good, they only handle the common case of
> compiling a kernel source to a kernel object. That is not really a
> problem, there are several solutions for the common case. I am looking
> at the unusual cases that require explicit user commands. Most are
> $(HOSTCC) commands but some run special scripts, there is no kbuild
> support for automatic rebuild in these cases.
Yes, I know, and you're right. But my point was that a program like your
preprocessor can convert such rules into safe variants automatically. It's
much like what your plan seems to be, i.e. having the user input his rule
and than enhance it. It's only that I think when you convert into a rule
like the above, you can actually still use most of the Makefile's
features, i.e. automatic variables like $@ $? $^ etc still just work.
Well, again, not quite, as I said, because you have to filter-out FORCE,
but it's still way easier then having your preparser substituting all this
at Makefile generation time. It also makes it possible to have pattern
rules like this, where substituting at generation time just doesn't work.
But yes, you're right, what I quoted above is nothing like a complete
solution, to work for the cases you mentioned you still need a
specialized tool. It's just that I think writing this tools is easier the
way I pointed out.
>
> >This means that I even get modversions etc. right.
>
> I doubt it ;). modversions is fundamentally broken as designed and has
> been since day 1. It is absolutely wrong to calculate a hash once at
> the start then never recalculate it. Config and code changes all
> affect the hash but we rely on the user manually running make mrproper,
> make dep to calculate the new hash. Even if you move the hash
> generation into the compile path so it is recalculated automatically,
> you still have the problem of deciding where the changed symbols are
> used. The hash for foo() has changed, where is foo() used?
Well, let me claim that it's not broken by design, at least not where you
say it is. If you start from a fresh tree (and maybe compile without -j),
you'll get a working result.
If you change some config variable, edit a file, whatever and re-make,
then you may get an invalid result, messed up versions etc. However,
that's not broken as designed, it means the dependencies are broken, and
things are remade in the wrong order or not at all. But not a fundamental
problem: The easiest, but of course slowest way, to remove this breakage
is to change the Makefiles to do the equivalent of mrproper before every
recompile. That will obviously work (with backing up .config etc), but of
course I'm not suggesting this. It's just to say that it can be fixed.
Now, I did claim that getting this right is possible, actually, that it
even gets done right automatically if you do dependencies the Right Way.
If you compile without modversions, everything is rather easy. Now you put
on modversions. This will add "-include linux/modversions.h" to your
command line. For everything which has been compiled befored, make will
note that the command line changes, therefore it will rebuild everything.
Also, due to the changed command line, every source file which is compiled
with these flags depends on include/linux/modversions.h (explicitly in the
Makefile).
modversions.h depends on all the included .ver files, which btw. should be
in the same directory as the corresponding export-obj, to remove the flat
namespace there.
The .ver files depend on the export-obj from which they are generated, and
are generated via the usual genksyms rule etc. Of course, they have the
usual dependencies, i.e. command line, and the converted files, plus all
files included by the C preprocessor etc.
So, if you turn on modversions in .config, all these dependencies get
triggered, the .ver files are built, then modversions.h, then basically
everything gets recompiled. Also, all the dependencies during the
intermediate steps are recorded. Now, if you change a config option, this
will trigger rebuilding of the affected .ver files. That means
modversions.h needs to be remade, and eventually, everything which
included the now changed hash will be remade. If your list of export-objs
changes due to editing or config, the Makefile will notice, because
modversions.h depends on export-objs:%.o=.%.dep, modversions.h will be
rebuilt, everything else depends on it, so gets rebuilt.
Well, that means it works, you don't get broken output. But it needs to be
optimized. Particularly, we are rebuilding to many files. In most cases, a
change doesn't affect the hash, so no need to rebuild anything at all.
This optimization was already done in the current Makefiles (don't touch
modversions.h unless necessary).
Still, if a hash changes, actually only the files which depend on the
affected versioned symbol need to be remade. Currently, every file which
uses any versioned symbol depends on all of them. So we rebuild too much.
It's not a correctness issue, but a performance issue which I didn't
address. The best way I can see is to take an approach like autoconf.h,
i.e. check on which symbols we actually depend, and then exchange the
modversions.h dependency with more specific ones. That's doable, at least.
> And don't get me started on the third party developers who tell users
> to copy modversions.h and .config into other directories for external
> compilation. The theory is that modversions and .config are for the
> current kernel but there is _NO_ verification that the files are in
> sync, either with each other or with the kernel.
I don't think I said anything closely related to this problem. If people
don't follow the rules, they should not complain when the result is
broken.
> >Maybe I should mention that I achieve this without relying on any new
> >external tools/parsers/whatever, except for a mkdep replacement, which is
> >basically mec's dancing Makefile's fix_dep program, that handles the
> >splitted autoconf.h :)
>
> I want:
>
> * Support for external source compilation.
> * Add on patch sets.
> * Separate source and object directories.
I believe these are related. They may be partially achieved with even
uglier Makfile hacking than what I'm using now, but I don't think that
it's the right way to go. With respect to add-on patch sets, layering etc.
I'm not sure if it's the right way to go, it really only works when you
have no overlaps between patches, and that's easy enough to handle without
any special tools. For the complicated cases, you better use CVS or
something anyway. But as it basically comes for free with the
source/object separation (which I believe is useful), I see no reason to
object anyway.
> * Detection of all changes, not just the common cases.
That's what made has been invented for. It just needs to know the
dependencies, so you have to provide them. Which may call for help of
external tools, sure.
> * Verification of rules, especially what gets linked into vmlinux.
I'm not sure what you're saying here.
> Now if you can do that without external tools, let me know.
I didn't say so, and I don't think so. But I believe it's good to use the
well-known tools whereever possible, finding the spots where they don't
work well and have additional helpers at these localized points.
So why teach your preprocessor about all of make's automatic variables
when you can just leave them to make itself.
--Kai
_______________________________________________
kbuild-devel mailing list
[EMAIL PROTECTED]
http://lists.sourceforge.net/lists/listinfo/kbuild-devel
