Re: [kbuild-devel] kbuild status
[Brendan J Simon] Is there a site that documents the changes the Kai has made or is making ? Well, there's http://www.??.kernel.org/pub/linux/kernel/v2.5/ChangLog-*, search for kai@. He seems to have started in 2.5.7. Off the top of my head, the main kbuild2.5 features 2.5.40 *doesn't* have would be: * separate source and object trees - but see below * source tree overlays, aka shadow tree support * backward timestamp tracking * nonrecursive make * speed - largely due to the nonrecursive make, I think The backward timestamp tracking was implemented because it is required for proper handling of source overlays. It is useful in its own right, but in the 15+-year history of 'make', most of the world seems to have gotten along fine without it, so it's hard to justify as *necesssary* outside that context. I'm mainly interested in building in seperate directories, building from readonly src directories and having a single makefile so that a complete dependency graph is generated. The separate build directory is something Kai is currently working on, and I'm guessing it will work in another week or two. Most of the infrastructure is in place already. The complete dependency graph, one feature of nonrecursive make, might be necessary for 100% correct and efficient MODVERSIONS handling, but it is not necessary for most other tasks, I believe. It certainly makes it *easier* to implement minimum necessary change build semantics, but I am not convinced it's *necessary* for this. It sounds like politics has killed another great project. Hopefully Kai can do a good job of getting some of great kbuild work into the kernel (in some form). He has been doing an excellent job so far, in my opinion - especially considering the mess the build system was in to begin with. The remaining kbuld2.5 features - source overlays, backward timestamp tracking, and nonrecursive make (and the accompanying speed increase) - are probably impossible with the current architecture. Peter --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] kbuild status
[Brendan J Simon] Thanks for that information. Looks like Kai's work will/may eventually do what I want but when will it be mainstream, that is the question. If by mainstream you mean in 2.4, probably never. There is just no motivation to backport it, and I doubt Marcelo would accept the changes if anyone did. Which, indeed, he shouldn't. The 2.4 makefiles only require GNU Make 3.77, whereas 2.5 now uses certain features introduced in 3.78 and 3.79. It is not acceptible to change the required program list within a stable kernel release cycle. And no, you can't really hack around the 3.79 features Kai is using. At least not easily. Kai and I both tried, somewhere in the 2.4.0testN series, I believe it was I guess my only option is to either keep porting kbuild-2.5 (which seems like a waste of time if it isn't going to become mainstream) or put some other kludges in my build system to work around the deficiencies of the current kernel build system. Unfortunately, I think the latter is probably going to win. You may have some luck backporting the Kai makefiles. Because he took an incremental approach, rather than a rewrite, most of the porting will consist of deleting cruft from makefiles. The hard parts will be backporting (a) the toplevel Makefile, and (b) the arch/* makefiles. Just realise that your porting work will never make mainstream 2.4, because of the Make version requirement. The backward timestamp tracking was implemented because it is required for proper handling of source overlays. It is useful in its own right, but in the 15+-year history of 'make', most of the world seems to have gotten along fine without it, so it's hard to justify as *necesssary* outside that context. What does this do and how is it useful in non shadow tree environments ? The way 'make' determines when a file needs to be rebuilt is by the simple heuristic is this file older than any of its dependencies? This works fine if you can trust your timestamps to increase monotonically. Seems obvious - but there are in fact at least four scenarios where you *don't* have monotonic timestamps: (1) If some files are on an NFS server and others on a local machine (think separate source / object trees), the two system clocks may be far enough from each other that a freshly built object file still shows up as older than the source - or a stale object shows up as newer. (2) You can get similar effects one a single box by using ntpd to skew your local clock. (3) Things like 'tar', plus certain (broken) source management tools, can move a file's timestamp backward. (4) Finally, if you use shadow trees, and you delete a file on a shadow layer - so that a different source file is visible than was visible last time you built the object - the visible timestamp is not indicative of whether the object is up-to-date. Keith's kbuild2.5 fixes all four problems by tracking all source file timestamps; it triggers an object rebuild *whenever* the source file's timestamp changes - whether or not the source is newer than the existing object, and whether its time moved forward or backward. Like I said, this is a feature the world has done without for the past 15 or 20 years, so it's kind of a hard sell - unless you use shadow trees, where it is quite necessary because of case (4). It's quite ironic but I bet Kai ends up with something similar to kbuild-2.5 at the end of the day. I'll bet against you. The two architectures have basically *nothing* in common except their external behavior. Keith runs a fancy preprocessor program that puts together a master makefile out of all the directory makefiles and maintains a state database. Kai just plods along with a recursive make structure with a very sophisticated piece of infrastructure called Rules.make, but doesn't really do any preprocessing or dynamic content. Certainly Kai has studied kbuild2.5, but there really isn't any way to use an incremental approach to produce the kbuild2.5 architecture from the in-kernel kbuild architecture. It's a problem of irreducible complexity, as they say in evolution debates. I know I've stripped back other peoples projects in the past (mainly because I didn't understand them or thought something was irrelevant) only to find me adding back features at a later date and ending up with something similar to what I started with. I hear ya. Remember the 2.1.1xx days? Linus looked at the relatively mature uusb project and basically said Wow, what a load of crap, the usb spec isn't *nearly* this complex. Then he spent a weekend hacking up an extremely simple but working usb keyboard / mouse driver. Fast-forward a few years and look at the kernel usb system now. I have no idea whether it resembles uusb at all, but simple it ain't Peter --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf
Re: [kbuild-devel] kbuild status
On Mon, Oct 07, 2002 at 07:30:18PM -0500, Peter Samuelson wrote: I hear ya. Remember the 2.1.1xx days? Linus looked at the relatively mature uusb project and basically said Wow, what a load of crap, the usb spec isn't *nearly* this complex. Then he spent a weekend hacking up an extremely simple but working usb keyboard / mouse driver. And that was the correct thing to do at the time. A load of people who had previously been just sitting on the sidelines watching the mass of uusb grow, were suddenly able to start to contribute. This raised the level of the code, and again, was an incremental development cycle. Fast-forward a few years and look at the kernel usb system now. I have no idea whether it resembles uusb at all, but simple it ain't That would be interesting to compare, I remember the old code, and know the current code quite well, so my thoughts are probably pretty biased :) thanks, greg k-h (current Linux USB maintainer...) --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] kbuild status
Hi kbuild team, What is the status of kbuild with respect to getting into the kernel proper ? Is it going to get into the linus kernel source tree ?? The reason I ask is that I require a kernel build system for the 2.4 kernel on a embedded PowerPC architecture. It seems that only the i386 is being actively maintained. I did do some work in porting the kbuild-2.5-1.12 to 2.4.18-pre8 some time ago. It worked but did not work well with my Configuration Management System (Aegis). Aparently kbuild-2.5-3.0 onwards should work with Aegis but I don't really have to time to report everything from scratch again. I can probably generate patches for the work I've done for the embedded boards if anyone is interested ? Unfortunately these patches would not be against the vanilla kernel sources, but against the linuxppc kernel tree (linuxppc_2_4_devel to be procise). So does anyone know if kbuild-2.5 is likely to become mainstream in the near future ? If so, is it going to be backported to 2.4 for all architectures ? If not, is there anything in the vanilla kernel that will build from a a readonly repository ? I hope kbuild-2.5 does have a future because so much hard work has gone into it. It would be a shame to see all the effort wasted. Many thanks, Brendan Simon. --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] kbuild status
[Brendan J Simon] What is the status of kbuild with respect to getting into the kernel proper ? Is it going to get into the linus kernel source tree ?? At this point it doesn't seem likely. Of the rich feature set Keith produced, Kai Germaschewski has managed to replicate the most compelling features (though by no means all!) by incrementally tweaking the Rules.make-based system, so Linus doesn't seem to see the urgency of replacing the whole thing. I hope kbuild-2.5 does have a future because so much hard work has gone into it. It would be a shame to see all the effort wasted. Me too, but Linus has never made any secret of the fact that he doesn't really care much about effort wasted, so long as he gets a kernel he is happy with. Call it what you will, we all know that's how this game is played Peter --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] kbuild status
Peter Samuelson wrote: [Brendan J Simon] What is the status of kbuild with respect to getting into the kernel proper ? Is it going to get into the linus kernel source tree ?? At this point it doesn't seem likely. Of the rich feature set Keith produced, Kai Germaschewski has managed to replicate the most compelling features (though by no means all!) by incrementally tweaking the Rules.make-based system, so Linus doesn't seem to see the urgency of replacing the whole thing. Is there a site that documents the changes the Kai has made or is making ? I'm mainly interested in building in seperate directories, building from readonly src directories and having a single makefile so that a complete dependency graph is generated. I hope kbuild-2.5 does have a future because so much hard work has gone into it. It would be a shame to see all the effort wasted. Me too, but Linus has never made any secret of the fact that he doesn't really care much about effort wasted, so long as he gets a kernel he is happy with. Call it what you will, we all know that's how this game is played Doesn't sound promising. I guess that's why Keith has been quiet on this list in recent times. It sounds like politics has killed another great project. Hopefully Kai can do a good job of getting some of great kbuild work into the kernel (in some form). Cheers, Brendan Simon. --- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
