Re: [kbuild-devel] kbuild status
[Brendan J Simon] Is there a site that documents the changes the Kai has made or is making ? Well, there's http://www.??.kernel.org/pub/linux/kernel/v2.5/ChangLog-*, search for kai@. He seems to have started in 2.5.7. Off the top of my head, the main kbuild2.5 features 2.5.40 *doesn't* have would be: * separate source and object trees - but see below * source tree overlays, aka shadow tree support * backward timestamp tracking * nonrecursive make * speed - largely due to the nonrecursive make, I think The backward timestamp tracking was implemented because it is required for proper handling of source overlays. It is useful in its own right, but in the 15+-year history of 'make', most of the world seems to have gotten along fine without it, so it's hard to justify as *necesssary* outside that context. I'm mainly interested in building in seperate directories, building from readonly src directories and having a single makefile so that a complete dependency graph is generated. The separate build directory is something Kai is currently working on, and I'm guessing it will work in another week or two. Most of the infrastructure is in place already. The complete dependency graph, one feature of nonrecursive make, might be necessary for 100% correct and efficient MODVERSIONS handling, but it is not necessary for most other tasks, I believe. It certainly makes it *easier* to implement minimum necessary change build semantics, but I am not convinced it's *necessary* for this. It sounds like politics has killed another great project. Hopefully Kai can do a good job of getting some of great kbuild work into the kernel (in some form). He has been doing an excellent job so far, in my opinion - especially considering the mess the build system was in to begin with. The remaining kbuld2.5 features - source overlays, backward timestamp tracking, and nonrecursive make (and the accompanying speed increase) - are probably impossible with the current architecture. Peter --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] kbuild status
[Brendan J Simon] Thanks for that information. Looks like Kai's work will/may eventually do what I want but when will it be mainstream, that is the question. If by mainstream you mean in 2.4, probably never. There is just no motivation to backport it, and I doubt Marcelo would accept the changes if anyone did. Which, indeed, he shouldn't. The 2.4 makefiles only require GNU Make 3.77, whereas 2.5 now uses certain features introduced in 3.78 and 3.79. It is not acceptible to change the required program list within a stable kernel release cycle. And no, you can't really hack around the 3.79 features Kai is using. At least not easily. Kai and I both tried, somewhere in the 2.4.0testN series, I believe it was I guess my only option is to either keep porting kbuild-2.5 (which seems like a waste of time if it isn't going to become mainstream) or put some other kludges in my build system to work around the deficiencies of the current kernel build system. Unfortunately, I think the latter is probably going to win. You may have some luck backporting the Kai makefiles. Because he took an incremental approach, rather than a rewrite, most of the porting will consist of deleting cruft from makefiles. The hard parts will be backporting (a) the toplevel Makefile, and (b) the arch/* makefiles. Just realise that your porting work will never make mainstream 2.4, because of the Make version requirement. The backward timestamp tracking was implemented because it is required for proper handling of source overlays. It is useful in its own right, but in the 15+-year history of 'make', most of the world seems to have gotten along fine without it, so it's hard to justify as *necesssary* outside that context. What does this do and how is it useful in non shadow tree environments ? The way 'make' determines when a file needs to be rebuilt is by the simple heuristic is this file older than any of its dependencies? This works fine if you can trust your timestamps to increase monotonically. Seems obvious - but there are in fact at least four scenarios where you *don't* have monotonic timestamps: (1) If some files are on an NFS server and others on a local machine (think separate source / object trees), the two system clocks may be far enough from each other that a freshly built object file still shows up as older than the source - or a stale object shows up as newer. (2) You can get similar effects one a single box by using ntpd to skew your local clock. (3) Things like 'tar', plus certain (broken) source management tools, can move a file's timestamp backward. (4) Finally, if you use shadow trees, and you delete a file on a shadow layer - so that a different source file is visible than was visible last time you built the object - the visible timestamp is not indicative of whether the object is up-to-date. Keith's kbuild2.5 fixes all four problems by tracking all source file timestamps; it triggers an object rebuild *whenever* the source file's timestamp changes - whether or not the source is newer than the existing object, and whether its time moved forward or backward. Like I said, this is a feature the world has done without for the past 15 or 20 years, so it's kind of a hard sell - unless you use shadow trees, where it is quite necessary because of case (4). It's quite ironic but I bet Kai ends up with something similar to kbuild-2.5 at the end of the day. I'll bet against you. The two architectures have basically *nothing* in common except their external behavior. Keith runs a fancy preprocessor program that puts together a master makefile out of all the directory makefiles and maintains a state database. Kai just plods along with a recursive make structure with a very sophisticated piece of infrastructure called Rules.make, but doesn't really do any preprocessing or dynamic content. Certainly Kai has studied kbuild2.5, but there really isn't any way to use an incremental approach to produce the kbuild2.5 architecture from the in-kernel kbuild architecture. It's a problem of irreducible complexity, as they say in evolution debates. I know I've stripped back other peoples projects in the past (mainly because I didn't understand them or thought something was irrelevant) only to find me adding back features at a later date and ending up with something similar to what I started with. I hear ya. Remember the 2.1.1xx days? Linus looked at the relatively mature uusb project and basically said Wow, what a load of crap, the usb spec isn't *nearly* this complex. Then he spent a weekend hacking up an extremely simple but working usb keyboard / mouse driver. Fast-forward a few years and look at the kernel usb system now. I have no idea whether it resembles uusb at all, but simple it ain't Peter --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf
Re: [kbuild-devel] kbuild status
On Mon, Oct 07, 2002 at 07:30:18PM -0500, Peter Samuelson wrote: I hear ya. Remember the 2.1.1xx days? Linus looked at the relatively mature uusb project and basically said Wow, what a load of crap, the usb spec isn't *nearly* this complex. Then he spent a weekend hacking up an extremely simple but working usb keyboard / mouse driver. And that was the correct thing to do at the time. A load of people who had previously been just sitting on the sidelines watching the mass of uusb grow, were suddenly able to start to contribute. This raised the level of the code, and again, was an incremental development cycle. Fast-forward a few years and look at the kernel usb system now. I have no idea whether it resembles uusb at all, but simple it ain't That would be interesting to compare, I remember the old code, and know the current code quite well, so my thoughts are probably pretty biased :) thanks, greg k-h (current Linux USB maintainer...) --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: RfC: Don't cd into subdirs during kbuild
Le jeu 03/10/2002 à 04:59, Kai Germaschewski a écrit : Hi, I'd appreciate to get comments on the appended patch. It's mostly cleanups and the like, but the interesting part is the last cset, which is actually fairly small: 14 files changed, 64 insertions(+), 47 deletions(-) The build process remains recursive, but it changes the recursion from make -C subdir to make -f subdir/Makefile Could you do instead: include subdir/Makefile ? This would avoid recursive make, which isn't really a good idea (even if it's used widely). Here is a good agument about that: http://www.cse.iitb.ac.in/~soumen/teach/cs699a1999/make.html --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: RfC: Don't cd into subdirs during kbuild
Le jeu 03/10/2002 à 16:56, Kai Germaschewski a écrit : This would avoid recursive make, which isn't really a good idea (even if it's used widely). Here is a good agument about that: http://www.cse.iitb.ac.in/~soumen/teach/cs699a1999/make.html I think I heard that before, but I would argue that recursive builds if done right are just fine from the correctness point of view. Then I would argue that recursive builds are a tradeoff between speed/maintainability and correctness. Perhaps you should re-read this paper then. Your patches go in the right direction, though. --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: RfC: Don't cd into subdirs during kbuild
On Thu, 3 Oct 2002, Sam Ravnborg wrote: -obj-$(CONFIG_ACPI_INTERPRETER) := $(patsubst %.c,%.o,$(wildcard *.c)) +obj-y := dsfield.o dsmthdat.o dsopcode.o dswexec.o dswscope.o \ +dsmethod.o dsobject.o dsutils.o dswload.o dswstate.o Should that have been: obj-$(CONFIG_ACPI_INTERPRETER) := dsfield.o dsmthdat.o dsopcode.o... Looks wrong to me that you remove the CONFIG_ dependency. Same is true for the rest of this cset. No, that's fine. We only enter this subdirectory if CONFIG_ACPI_INTERPRETER is set, so we do not need to repeat the variable here. A lot of places rely on this behavior to persist, so why not use it? +ifdef list-multi +$(warning kbuild: list-multi ($(list-multi)) is obsolete in 2.5. Please fix!) +endif Since kbuild no longer support list-multi this should be $(error ) Well, since it will still work fine (throwing an additional warning later on), I think a warning is okay here. I should have made the O_TARGET one an error now, though, since that stopped working. Anyway, I'll make both of them an error in a little bit, so... SUBDIRS+= $(patsubst %/,%,$(filter %/, $(init-y) $(init-m))) I prefer first assignment to be := not += This is true for several places including several makefiles as well. Well, really mostly a matter of taste. Using += everywhere has the advantage that you can add another line before that line without changing out. Kinda the same thing as adding a comma after the last element of a struct / enum. -export CPPFLAGS EXPORT_FLAGS NOSTDINC_FLAGS OBJCOPYFLAGS +export CPPFLAGS EXPORT_FLAGS NOSTDINC_FLAGS OBJCOPYFLAGS LDFLAGS Did not see this change justified. The export LDFLAGS just moved to a place where it's more logical. -export NETWORKS DRIVERS LIBS HEAD LDFLAGS MAKEBOOT +$(warning $(SUBDIRS)) Warning shall be deleted Right. I overlooked it first, but it's deleted in a later cset. ifndef O_TARGET ifndef L_TARGET -O_TARGET := built-in.o +O_TARGET := $(obj)/built-in.o +endif endif This change result in ld being called for directories like: $(TOPDIR)/scripts $(TOPDIR)/scripts/lxdialog $(TOPDIR)/Documentation/DocBook If obj-y is empty then do not define O_TARGET? Well, it's rather that I used EXTRA_TARGETS in those subdirs now. You're right that the standard rules do not apply in those dirs, so I'll think of a way to fix it there. Not defining O_TARGET when obj-y is empty is not an option, we rely on that case working elsewhere. Another more general comment. There seem to no consistency in the variables used in the first section of the makefile. There is a mixture of lower and upper case variables: O_TARGET, host-progs etc. This is confusing. Well, the whole thing is moving away from capitalized letters (in particular in the per-subdir Makefiles), as it is moving from old-style to new-style. The only common variables which are capitalized are CFLAGS, CC and the like, and I think they'll stay since that's standard make. kbuild-specific variables should really be basically all lower-case by now, I can only think of L_TARGET as an exception. Furthermore the construct: obj-y := some.o dot.o .o module.o Seems illogical to me. What does obj-y mean to me?? mandatory-objs := some.o dot.o .o module.o No, I think once you've understood obj-$(CONFIG_FOO), the meaning of obj-y is perfectly clear. Giving multiple names to the samt thing is not good, next thing would be people wondering what the difference between obj-y and mandatory-objs is. first_rule: $(if $(KBUILD_BUILTIN),$(O_TARGET) $(L_TARGET) $(EXTRA_TARGETS)) \ Where comes the requirement that EXTRA_TARGETS needs to be buildin? Initially, it was for built-in targets in addition to the standard O_TARGET, like arch/i386/kernel/head.o. I've been abusing it for scripts/, and I shouldn't be doing that. -cmd_link_multi = $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) -r -o $@ $(filter $($(basename $@)-objs),$^) +cmd_link_multi = $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) -r -o $@ $(filter $(addprefix $(obj)/,$($(subst $(obj)/,,$(@:.o=-objs,$^) Keep a variable without obj appended would make this readable I think. I agree that it is not particularly readable, but I'm limited to what make offers. What do you suggest? --Kai --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: RfC: Don't cd into subdirs during kbuild
On Thu, 3 Oct 2002, Sam Ravnborg wrote: On Thu, Oct 03, 2002 at 10:01:20PM +0200, Sam Ravnborg wrote: Now it's testing time.. [...] You must be missing some of the changes (My first push to bkbits was incomplete, since I did inadvertently edit Makefile without checking it out, I do that mistake all the time...). It's fixed in the current repo. --Kai --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: RfC: Don't cd into subdirs during kbuild
On 3 Oct 2002, Xavier Bestel wrote: Could you do instead: include subdir/Makefile ? It's not quite that easy, unfortunately ;( This would avoid recursive make, which isn't really a good idea (even if it's used widely). Here is a good agument about that: http://www.cse.iitb.ac.in/~soumen/teach/cs699a1999/make.html I think I heard that before, but I would argue that recursive builds if done right are just fine from the correctness point of view. --Kai --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
