[kbuild-devel] Re: State of the new config & build system
On Sat, Dec 29, 2001 at 12:27:24PM +1100, Keith Owens wrote: > Dependencies _do_ change when your .config changes, the list of files > that are included varies. 1) "#ifdef CONFIG_FOO #include ..." is usually wrong and a bug. But that is a tangent and I digress. 2) Such changes can be expressed without regenerating all dependencies. > Linus, you have a choice between a known broken build system and a > clean and reliable system, which is slightly slower in mark 1. Please > add kbuild 2.5 to the kernel, Your system is known broken because it is 100% slower. My kernel builds work just fine now, your changes gain me nothing, while COSTING me productivity. I see no gains, only costs, with your kbuild-2.5 system as it exists. Keith the target audience is Linus and Alan and ME etc. We are the kernel hackers that perform kernel -development-. Making end-user builds easier is NOT a primary nor secondary nor tertiary goal here. Make my life easier first. Fuck Aunt Tillie. Aunt Tillie can get her kernels from a vendor. Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Sat, Dec 29, 2001 at 12:26:49PM +1100, Keith Owens wrote: > On Fri, 28 Dec 2001 16:16:03 -0500, > Legacy Fishtank <[EMAIL PROTECTED]> wrote: > >I think one thing to note is that dependencies is that if you are smart > >about it, dependencies -really- do not even change when your .config > >changes. > > > >What about a system where Linus runs "make deps" -once- before he > >releases a tarball. This in turn generates dependency information > >(perhaps not in purely make format) which includes 'ifdef CONFIG_xxx' > >information embedded within. We know that make can support ifeq > >CONFIG_xxx for example... > > Then people apply patches and break. s/break/update dependencies/ I assumed this was blindingly obvious, but I guess not. Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001 14:27:37 -0800 (PST), Linus Torvalds <[EMAIL PROTECTED]> wrote: >Note that I do _not_ want to mess up source files with magic comments. I >absolutely detest those. They only detract from the real job of coding, >and do not make anybody happier. > >We have a hierarchical filesystem. Most drivers already have > > driver.c > driver.h > >(in fact _very_ few drivers are single-file) and some have a subdirectory >of their own. So why not just have > > driver.conf > >and be done with it. No point in messing up the C file with stuff that >doesn't add any information to either the programmer _or_ the compiler. I would love to do that, with each driver/filesystem/... having an associated control file that defined its config options, its help and who to make it. That is, an "Insert New Facility" file, we could call them driver.inf (ducks and runs ;). There is one big problem in the way, makefile order controls link order which controls init order. I have no problem with the link order controlling init order, that is far better than the old Space.c code. I intensely dislike makefile order controlling link order, it results in loss of information, we have makefiles in a specific order with no idea about whether that order is required or is just accidental. IMHO the link order should be divorced from makefile order and made explicit. Then you could have makefile fragments associated with each driver. But the last time I tried to break the dependency between make and link order, Linus shot me down in flames[1], so I have no intention of going there again. As long as you have monolithic makefiles, drivers.conf is going to be problematic. [1] http://marc.theaimsgroup.com/?l=linux-kernel&m=97301359812683&w=2 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Larry McVoy wrote: >On Fri, Dec 28, 2001 at 12:41:48PM +1100, Keith Owens wrote: > >>On Thu, 27 Dec 2001 17:37:39 -0800, >>Larry McVoy <[EMAIL PROTECTED]> wrote: >> >>>A couple of questions: >>> >>>a) will 2.5 be as fast as the current system? Faster? >>> >>At the moment kbuild 2.5 ranges from 10% faster on small builds to 100% >>slower on a full kernel build. >> > >I don't understand why it would be slower. > Thank's go to basically to python and other excessfull overengineering there. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
dammit, didn't hit "reply all" grr On Saturday, December 29, 2001, at 05:02 AM, Linus Torvalds wrote: > My pet peeve is "centralized knowledge". I absolutely detested the first > versions of cml2 for having a single config file, and quite frankly I > don't think Eric has even _yet_ separated things out enough - why does > the > main "rules.cml" file have architecture-specific info, for example? agreed - it's something that really irritates me too. As Linux is running on so many different architectures (some of which are purely virtual, such as Usermode Linux and my whacky idea of running it ontop of MacOS X) so it seems that keeping all the options for architectures separate would make a lot of sense. I've never seen a cross-platform binary kernel (although have had scary dreams of one) > So if somebody really wants to help this, make scripts that generate > config files AND Configure.help files from a distributed set. And once > you > do that, you could even imagine creating the old-style config files > (without the automatic checking and losing some information) from the > information. This shouldn't be too hard should it? In each module directory have a config and Configure.help file, then just find . |grep config and then cat all the files together. If I have some spare time today I'll see if I can hack something up :) -- Stewart Smith [EMAIL PROTECTED] Ph: +61 4 3884 4332 ICQ: 6734154 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> that if there is reusable code in BK, we're willing to let people use > it under whatever license they want. It would be nice if that actually > happened after all the yelling and screaming. mdbm is one I've not seen. The timings I've done are with db2/db3/tdb when I was playing with a fast UDP server that had to do a db lookup per packet. Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Linus Torvalds wrote: >On Fri, 28 Dec 2001, Alan Cox wrote: > >>It would certainly fit nicely with the existing metadata. We already rip out >>code comments via kernel-doc, and extending it to rip out >> >> - Help text >> - Web site >> >... > >No no no. > >The comments can at least be helpful to programmers, whether ripped out or >not. > >Extra stuff is not helpful to anybody, and is just really irritating. I >personally despise source trees that start out with one page of copyright >statement crap, it just detracts from the real _point_ of the .c file, >which is to contain C code. Making it a comment requirement is > > - stupid: > we have a filesystem, guys > Not quite... It is making moving patches through e-mail around easier... > ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 06:05:57PM -0500, Benjamin LaHaise wrote: > On Fri, Dec 28, 2001 at 02:27:37PM -0800, Linus Torvalds wrote: > > and it's readable and probably trivially parseable into both the existing > > format (ie some "find . -name '*.conf'" plus sed-scripts) and into cml2 or > > whatever. > > It's even doable within the .c file (and preferable for small drivers). > Something like: > > /* mydriver.c header blah blah */ > config_requires(CONFIG_INET); > config_option(CONFIG_MY_FAST_CHIP, "Help info for this"); If Linus is willing to buy into "driver.conf" there is no need to stuff things into the source. [my previous post made the mistaken assumption that Linus would not like an additional metadata file like driver.conf] A per-driver metadata file is IMHO clearly the preferred solution. Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 10:02:01AM -0800, Linus Torvalds wrote: > [ Btw, Jeff, any reason why you changed your name to "Legacy Fishtank"? It > took a few mails before I noticed that it also said "garzik" in the > fine print;] Away-from-home account and a long story :) Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 02:27:37PM -0800, Linus Torvalds wrote: > and it's readable and probably trivially parseable into both the existing > format (ie some "find . -name '*.conf'" plus sed-scripts) and into cml2 or > whatever. It's even doable within the .c file (and preferable for small drivers). Something like: /* mydriver.c header blah blah */ config_requires(CONFIG_INET); config_option(CONFIG_MY_FAST_CHIP, "Help info for this"); which gets picked out of the .c files during depend phase, and nullified during compile by means of -Iconfig_system.h would even let us get rid of Makefiles for drivers. Wouldn't being able to just drop a .c file (or a bunch of .c files) into the tree in the right place be great? Eliminating makefiles means eliminating more conflicts, which might mean more time to respond to other issues... -ben -- Fish. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
I think one thing to note is that dependencies is that if you are smart about it, dependencies -really- do not even change when your .config changes. What about a system where Linus runs "make deps" -once- before he releases a tarball. This in turn generates dependency information (perhaps not in purely make format) which includes 'ifdef CONFIG_xxx' information embedded within. We know that make can support ifeq CONFIG_xxx for example... Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 03:45:37PM -0500, Eric S. Raymond wrote: > Legacy Fishtank <[EMAIL PROTECTED]>: > > For single-file drivers, I like Becker's (correct credit?) system... > > about 10 lines of metadata is embedded in a C comment, and it includes > > the Config.in and Configure.help info. > > I proposed implementing something like this about a year ago (to > replace the nasty centralized knowledge in the MAINTAINERS and CREDITS > files) and was shot down. Note I am specifically NOT talking about MAINTAINERS and CREDITS. -PLEASE- don't obscure my point by mentioning them. Dealing with MAINTAINERS and CREDITS in an automated fashion seems more like pointless masturbation to me. If you want to find out who needs to be CC'd on patches, use your brain like I do. Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: your mail
Linus Torvalds wrote: >(Right now you can see this in block_ioctl.c - while only a few of the >ioctl's have been converted, you get the idea. I'm actually surprised that >nobody seems to have commented on that part). > That was just too obvious, at least for me... However I don't see why you just don't start killing of constructs like: swtch (ioctrl) BLASH: BLAHHH: BLASHH: BLAASS: BLAH: default: return -ENOVAL; } There are ton' s of them out there in the block drivers.. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Linus Torvalds wrote: >[ Btw, Jeff, any reason why you changed your name to "Legacy Fishtank"? It > took a few mails before I noticed that it also said "garzik" in the > fine print;] > >One thing that this big flame-war has brought up is that different people >like different things. There may be a simpler solution to this: have the >core dependency files generated from some other file format. > >My pet peeve is "centralized knowledge". I absolutely detested the first >versions of cml2 for having a single config file, and quite frankly I >don't think Eric has even _yet_ separated things out enough - why does the >main "rules.cml" file have architecture-specific info, for example? > >That's a big step backwards as far as I'm concerned - we didn't use to >have those stupid global files, and each architecture could do it's own >config rules. Eric never got the point that to me, modularity is _the_ >most important thing for maintenance. > >Something I also asked for the config system at least a year ago was to >have Configure.help split up. Never happened. It's still one large ugly >file. Driver or architecture maintainers still can't just change _their_ >small fragment, they have to touch a global file that they don't "own". > >So if somebody really wants to help this, make scripts that generate >config files AND Configure.help files from a distributed set. And once you >do that, you could even imagine creating the old-style config files >(without the automatic checking and losing some information) from the >information. > If you go thus far... then I think, that the Configure.help stuff should be embedded inside the driver source code itself. Like for example the postfix MTA code is embedding whole *man* pages there. And *man* pages would be anyway a more appriopriate and classical place where the current Configure.help information should be. Just lift the code over from there (The extraction is even proper awk insead of some perl crap...) and be nearly done. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Larry McVoy writes:
> On Fri, Dec 28, 2001 at 08:42:44PM +1100, Keith Owens wrote:
> > "All" I need to do is have one server process that reads the big list
> > once and the other client processes talk to the server. Much less data
> > involved means faster conversion from absolute to standardized names.
>
> Actually, if you use the mdbm code, you can have a server process which
> reads the data, stashes it in the db, touchs ./i_am_done, and exits.
> "client" processes do a
>
> while (!exists("i_am_done")) usleep(10);
> m = mdbm_open("db", O_RDONLY, 0, 0);
> val = mdbm_fetch_str(m, "key");
> etc.
>
> No sockets, no back and forth, runs at mmap speed.
That sounds like a better approach. I got a bit nervous when Keith
talked about a "server process". Made me think I'm going to have to
install some daemon, or I'm going to have a pile of background
processes being left behind (no matter how careful you are, you always
end up with some "leakage" of stale processes).
Regards,
Richard
Permanent: [EMAIL PROTECTED]
Current: [EMAIL PROTECTED]
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 10:02:01AM -0800, Linus Torvalds wrote: > Something I also asked for the config system at least a year ago was to > have Configure.help split up. Never happened. It's still one large ugly > file. Driver or architecture maintainers still can't just change _their_ > small fragment, they have to touch a global file that they don't "own". > > So if somebody really wants to help this, make scripts that generate > config files AND Configure.help files from a distributed set. And once you > do that, you could even imagine creating the old-style config files > (without the automatic checking and losing some information) from the > information. For single-file drivers, I like Becker's (correct credit?) system... about 10 lines of metadata is embedded in a C comment, and it includes the Config.in and Configure.help info. Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001 14:51:13 -0800, Larry McVoy <[EMAIL PROTECTED]> wrote: >On Fri, Dec 28, 2001 at 09:56:53PM +0100, Kai Germaschewski wrote: >> A couple of months ago, I came up with an alternative to kbuild 2.5. It >> doesn't try to have all the features kbuild 2.5 has, but solves the major >> problems with kbuild 2.4. > >So has anyone looked at this? Is this a viable choice? I've heard nothing >since Kai posted this. Keith? I looked back through the kbuild mail for Kai's suggestions, I may not have them all. RFD: Tracking indirect dependencies [long] We knocked this back and forth for a while. We both agree that extracting dependencies after compile is correct, where we differed was the mechanism. In fact I have currently implemented Kai's approach (lots of little files) as a stepping stone to storing the data in a database. It turns out that one of the reasons that kbuild 2.5 is slow is handling all the little files containing dependency data. [PATCH] removal of list-multi I agree with the patch but that was December 2000, in code freeze, and again in April 2001, AFAICR Linus had said "2.5 soon". This patch is worth resurrecting for 2.4. Auto detection of changed commands/flags That was a decent fix for part of the problem, but it did not address tracking user commands nor host compiles. It did not allow for separate source and object trees, for read only source trees, nor did it handle the more esoteric cases like modules being built from multiple directories. I am not interested in partial fixes, I want the whole kbuild problem list to be cleared. Fixes that only solve part of the problem tend to be filed and ignored. Kai, did I miss any of your patches? ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> Unlike bio, kbuild 2.5 works, it just needs to be a bit faster. Put > kbuild 2.5 in the kernel and I will have a faster version within 2 > weeks. Ok. I was assuming from what you had said that we were talking about months before it got up to a sane speed. If its 2 weeks then I have absolutely no problems with that. Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Sat, 29 Dec 2001 01:53:17 + (GMT), Alan Cox <[EMAIL PROTECTED]> wrote: >> dependency problem, any solution that does not fix _all_ 9 problems in >> http://prdownloads.sourceforge.net/kbuild/kbuild-2.5-history.tar.bz2, >> makefile-2.5_make_dep.html is not a complete fix. > >All well and good but "takes 100% longer" is number 10 on that list which >you missed off, and the same argument holds for that. You are missing the point Alan. * The makefile rules are correct now. * The build is correct now. * kbuild 2.5 is faster on small compiles and much faster on recompiles after small changes. * kbuild 2.5 is slower on large compiles. * The speed problem is fixable, given time. Correctness came first. * I don't have time to keep tracking multiple kernels and architectures _and_ rewrite the core code. * Once kbuild 2.5 is in the kernel I can spend far less time on tracking changes and can redesign the core programs for speed. * It will get faster! Why do you expect a change in a development kernel to be perfect first time? Look at all the bio changes, I just did a full 2.5.1 build and had to disable 87 config options before the kernel would build, and that is ignoring all the warning messages which point to out of date function definitions. Is anybody complaining that bio should have worked first time? Unlike bio, kbuild 2.5 works, it just needs to be a bit faster. Put kbuild 2.5 in the kernel and I will have a faster version within 2 weeks. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> dependency problem, any solution that does not fix _all_ 9 problems in > http://prdownloads.sourceforge.net/kbuild/kbuild-2.5-history.tar.bz2, > makefile-2.5_make_dep.html is not a complete fix. All well and good but "takes 100% longer" is number 10 on that list which you missed off, and the same argument holds for that. > but there is one problem that is inherently unfixable. make dep is a > manual process so it relies on users knowing when they have to rerun > make dep AND THEY DON'T DO IT! Please do not say "I always run make So automate running make dep. > Linus, you have a choice between a known broken build system and a So broken its worked for say 5 years without major problem > ps. I don't want mail discussing individual bug fixes to mkdep. Code > that does not fix _all_ 9 bugs listed in makefile-2.5_make_dep.html > is pointless. And bug number 10 you didnt mention ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001 14:17:24 -0800 (PST), Linus Torvalds <[EMAIL PROTECTED]> wrote: > >On Fri, 28 Dec 2001, Legacy Fishtank wrote: >> >> I think one thing to note is that dependencies is that if you are smart >> about it, dependencies -really- do not even change when your .config >> changes. > >Absolutely. I detest "gcc -MD", exactly because it doesn't get this part >right. "mkdep.c" gets this _right_. Sorry, it does not. Everybody is attacking little bits of the dependency problem, any solution that does not fix _all_ 9 problems in http://prdownloads.sourceforge.net/kbuild/kbuild-2.5-history.tar.bz2, makefile-2.5_make_dep.html is not a complete fix. Yes, some of the problems with mkdep can be fixed in the current design but there is one problem that is inherently unfixable. make dep is a manual process so it relies on users knowing when they have to rerun make dep AND THEY DON'T DO IT! Please do not say "I always run make dep" after a change, I guarantee that you are the exception. Users apply patches and do not run make dep, then wonder why their kernel is broken. Dependencies _do_ change when your .config changes, the list of files that are included varies. gcc -MD gets this exactly right, gcc knows which files it read. mkdep does an incorrect approximation, see tyhe bug list in makefile-2.5_make_dep.html. The errors in mkdep were acceptable as long as only kernel hackers built their own kernels, they could be relied upon to manually run commands when necessary. The target population has changed, more and more beginners are building kernels and too many are getting it wrong. I am aiming at the entire population, not that small subset who have been building kernels since the year dot. Any build system that silently fails when users forget to run a command is a broken system. kbuild 2.5 fixes _all_ 9 problems with mkdep, it also positions us for correct modversion handling. kbuild 2.4 is faster, inaccurate and manual, kbuild 2.5 is slower, accurate and totally automatic. I know how to speed up 2.5. What I don't have is time to rewrite the code for speed, I am too busy tracking kernel changes because kbuild 2.5 is not in the kernel yet. Linus, you have a choice between a known broken build system and a clean and reliable system, which is slightly slower in mark 1. Please add kbuild 2.5 to the kernel, then I will have time to rewrite the core programs for speed. Mark 2 of the core code will be significantly faster. ps. I don't want mail discussing individual bug fixes to mkdep. Code that does not fix _all_ 9 bugs listed in makefile-2.5_make_dep.html is pointless. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001 16:16:03 -0500, Legacy Fishtank <[EMAIL PROTECTED]> wrote: >I think one thing to note is that dependencies is that if you are smart >about it, dependencies -really- do not even change when your .config >changes. > >What about a system where Linus runs "make deps" -once- before he >releases a tarball. This in turn generates dependency information >(perhaps not in purely make format) which includes 'ifdef CONFIG_xxx' >information embedded within. We know that make can support ifeq >CONFIG_xxx for example... Then people apply patches and break. Please read the list of mkdep bugs before suggesting partial fixes. http://prdownloads.sourceforge.net/kbuild/kbuild-2.5-history.tar.bz2, makefile-2.5_make_dep.html. I want a system that fixes _all_ the bugs, not just some of them. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> I also want updates from the dependency back end code, to remove the > phase 5 processing. The "extract dependency" code runs after each > compile step so there can be multiple updates running in parallel. My > gut feeling is that it will be faster to have one database server and > all the back ends talk to that server. Otherwise each compile will > have overhead for lock, open, mmap, update, close, write back, unlock. > A single threading server removes the need for lock/unlock and can sync > the data to disk after n compiles instead of being forced to do it > after every compile. > > If your experience says that doing updates from each compile step > without a server process would not be too slow, let me know. You certainly don't need a server process. And as was pointed out earlier, it's nice not to have them, then you don't have to worry about them still being there. I can write you up a multi writer version using in file locks (which work over NFS, we had do that for BK and I'm pretty sure it is platform independent, I can't break it). We have to do this sort of multi reader/writer crud in BK all the time and have lots of experience with locking, breaking locks, waiting, NFS, etc. Much more experience than we ever wanted :-) You don't need to sync to disk at all, let the data sit in memory, that's why mmap is cool. Give me a spec for what you want, I'll crank out some code. Maybe I'll finally actually be useful to the kernel after all these years... -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
cc: list trimmed.
On Fri, 28 Dec 2001 12:01:04 -0800,
Larry McVoy <[EMAIL PROTECTED]> wrote:
>On Fri, Dec 28, 2001 at 08:42:44PM +1100, Keith Owens wrote:
>> "All" I need to do is have one server process that reads the big list
>> once and the other client processes talk to the server. Much less data
>> involved means faster conversion from absolute to standardized names.
>
>Actually, if you use the mdbm code, you can have a server process which
>reads the data, stashes it in the db, touchs ./i_am_done, and exits.
>"client" processes do a
>
> while (!exists("i_am_done")) usleep(10);
> m = mdbm_open("db", O_RDONLY, 0, 0);
> val = mdbm_fetch_str(m, "key");
> etc.
>
>No sockets, no back and forth, runs at mmap speed.
>
>If you tell me what the data looks like that needs to be in the DB, i.e.,
>how to get it, I'll code you up the "server" side.
I also want updates from the dependency back end code, to remove the
phase 5 processing. The "extract dependency" code runs after each
compile step so there can be multiple updates running in parallel. My
gut feeling is that it will be faster to have one database server and
all the back ends talk to that server. Otherwise each compile will
have overhead for lock, open, mmap, update, close, write back, unlock.
A single threading server removes the need for lock/unlock and can sync
the data to disk after n compiles instead of being forced to do it
after every compile.
If your experience says that doing updates from each compile step
without a server process would not be too slow, let me know.
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Linus Torvalds wrote: > On Fri, 28 Dec 2001, Legacy Fishtank wrote: > > > > I think one thing to note is that dependencies is that if you are smart > > about it, dependencies -really- do not even change when your .config > > changes. > > Absolutely. I detest "gcc -MD", exactly because it doesn't get this part > right. "mkdep.c" gets this _right_. Well, -MD gets this right. The dependencies it generates will cause a recompile when necessary. Unfortunately, though, it's too good, because the dependency on include/linux/autoconf.h will cause lots of unnecessary recompiles. But yes, it seems possible to replace the -MD dependency file, which depends on a specific config, with a generic dependency file, which knows about our #ifdef CONFIG_XXX and translates them to the corresponding ifeq(CONFIG_,) Makefile syntax. It'd make an interesting project, but it effectively means re-implementing a C preprocessor. I don't think you can blame gcc -MD for not knowing about the kernel's CONFIG_ system, though ;-) From --- #ifdef CONFIG_XXX #include #endif #ifdef CONFIG_YYY const int nr = 10; #else const int nr = 100; #endif --- you'd have to generate --- ifeq(CONFIG_XXX,y) DEPS += include/linux/xxx.h endif DEPS += include/config/yyy --- i.e. the include/config trick has to stay any way. I don't think the above is necessary, though, the following does work pretty good (I did it this way, inspired by mec, and I think kbuild-2.5 does it similarly): Generate dependencies for a .o file when compiling it. [ Doing make dep in advance is unnessary. Actually, it's pretty stupid to generate dependencies for *all* possible object files which you are never going to compile (think arch/*). If you don't have the object yet, you don't need to know the dependencies, dependencies only make sense for recompiles. It's also cheaper to generate dependencies during the compile, as you need to read the file anyway. Also, dependencies on generated files cannot be found correctly until these files have been generated. ] The generated dependencies will always include linux/autoconf.h, which is correct, but will cause too many recompiles. So, replace linux/autoconf.h with linux/config/xxx, where xxx are all the config options which appear in all of the files used to build the object file (which is what -MD gave you). The result is still dependencies which are 100% correct. It's that simple. The object file gcc generates depends on the command line and all the files it reads during the compile. Why make it more complex? --Kai ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Alan Cox <[EMAIL PROTECTED]>: > > I'd be happy to take another swing at this problem once the kbuild-2.5/CML2 > > transition is done. But I don't think we should let it block us from > > having the good results we can get from that change. > > It would certainly fit nicely with the existing metadata. We already rip out > code comments via kernel-doc, and extending it to rip out > > - Help text > - Web site > - Version information > - Man page for the driver > - Module options > > etc, shouldn't be too challenging. Ok so kernel-doc is in perl and ugly perl > but if someone hates it enough to rewrite it in python thats a win too 8) I've been thinking about doing that very thing anyway. Part of my master plan to reduce the tree's external dependencies. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond I do not find in orthodox Christianity one redeeming feature. -- Thomas Jefferson ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Martin Dalecki <[EMAIL PROTECTED]>: > >>At the moment kbuild 2.5 ranges from 10% faster on small builds to 100% > >>slower on a full kernel build. > > > >I don't understand why it would be slower. > > > Thank's go to basically to python and other excessfull overengineering > there. Bzzzt! Thank you for playing. kbuild-2.5 doesn't use Python. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond The possession of arms by the people is the ultimate warrant that government governs only with the consent of the governed. -- Jeff Snyder ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Re: State of the new config & build system
Linus Torvalds <[EMAIL PROTECTED]>: > Eric, this is the _wrong_approach_. I want /local/ files, not global ones. I hear you. There are some problems with that, however. First: where should the prompt-string definitions for capability symbols that occur in multiple port trees live? (This is an important question. Right now, most options are low-level and platform-specific, which makes it easy to decide what directory their symbol declaration(s) should live in. But that's not good; there are lots of excellent reasons we want there to be *more* cross-platform capability symbols rather than fewer. So the percentage of "roving" symbols without an obvious home is likely to go up over time.) Second: Forward references, and references across the tree, mean that there is a class of symbols that have theoretically natural home directories but which would have to be declared elsewhere in order to be defined at the point of first reference. (A potential solution to this would be to improve the CML2 compiler's handling of forward references.) Third: I could hack my installer to break Configure.help up into a bunch of little component CML files distributed through the tree... but Configure.help doesn't currently contain any markup that says where to direct each entry to. (The logical time to split up symbols.cml would be immediately after CML2 goes into the tree, because at that point Configure.help won't be an issue any more.) Fourth: There's still the localization issue. If it's your ukase that this is not an important problem, then I'll accept that -- but I haven't heard you say that yet, so I'm not sure you've considered it enough. So, I can and will put this in the transition plan if that's what you direct. But you need to be aware that it's not a snap-of-the-fingers change, and not something best done before CML1 goes away. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond "As to the species of exercise, I advise the gun. While this gives [only] moderate exercise to the body, it gives boldness, enterprise, and independence to the mind. Games played with the ball and others of that nature, are too violent for the body and stamp no character on the mind. Let your gun, therefore, be the constant companion to your walks." -- Thomas Jefferson, writing to his teenaged nephew. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Alan Cox wrote: > > It would certainly fit nicely with the existing metadata. We already rip out > code comments via kernel-doc, and extending it to rip out > > - Help text > - Web site ... No no no. The comments can at least be helpful to programmers, whether ripped out or not. Extra stuff is not helpful to anybody, and is just really irritating. I personally despise source trees that start out with one page of copyright statement crap, it just detracts from the real _point_ of the .c file, which is to contain C code. Making it a comment requirement is - stupid: we have a filesystem, guys - slow: we don't need to parse every C file we encounter when we can just open another file based on filename - nonsensical: many config options are _not_ limited to one C file - hard to parse and read: why limit ourself to C comments, when just keeping the thing logically separated means that we don't have to. Having per-function comment blocks, in contrast, makes sense to have inline: - you read the comment when you read the function - you might even update the comment when you update the function - you have a reasonable 1:1 relationship. _None_ of those are sensible for config file entries. Linus ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> Frankly, I find it very amusing that advocates of i18n efforts tend to > be either British or USAnians. Folks, get real - your languages are > too close to show where the problems are. I can see how doing that > gives you a warm fuzzy feeling, but could you please listen to those > of us who have to deal with the resulting mess for real? The biggest advocates I see are from the Middle-East and Japan. We already have people providing translations for Configure.help in several languages. Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> I'd be happy to take another swing at this problem once the kbuild-2.5/CML2 > transition is done. But I don't think we should let it block us from > having the good results we can get from that change. It would certainly fit nicely with the existing metadata. We already rip out code comments via kernel-doc, and extending it to rip out - Help text - Web site - Version information - Man page for the driver - Module options etc, shouldn't be too challenging. Ok so kernel-doc is in perl and ugly perl but if someone hates it enough to rewrite it in python thats a win too 8) Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 09:56:53PM +0100, Kai Germaschewski wrote: > A couple of months ago, I came up with an alternative to kbuild 2.5. It > doesn't try to have all the features kbuild 2.5 has, but solves the major > problems with kbuild 2.4. So has anyone looked at this? Is this a viable choice? I've heard nothing since Kai posted this. Keith? -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Linus Torvalds <[EMAIL PROTECTED]>: > On Fri, 28 Dec 2001, Eric S. Raymond wrote: > > I'm not certain what you're objecting to, and I want to understand it. > > There are rules that use architecture symbols to suppress things like > > bus types. I presume that's not a problem for you, but tell me if it is. > > It _is_ a problem for me, because I want to do "diffstat" on a patch from > a PPC maintainer, and if I see anything non-PPC, loud ringing > noises go off in my head. I want that diffstat to say _only_ > > arch/ppc/... > include/asm-ppc/... > > and nothing else. That way I know that I don't have to worry. Perhaps we're talking past each other. I don't understand your objection yet, and I want to so I can design (or redesign) to meet it. When I talk about "rules that use architecture symbols to suppress things like bus types" I have in mind things like this: unless X86 suppress dependent MCA EISA unless MIPS32 suppress dependent TC unless (PCI and (X86 or SUPERH)) suppress pci_access unless (ISA or PCI) suppress dependent IDE unless PCI suppress dependent USB HOTPLUG_PCI unless (X86 or ALPHA or MIPS32 or PPC) suppress usb unless (X86 and PCI and EXPERIMENTAL) or PPC or ARM or SPARC suppress dependent IEEE1394 unless (M68K or ALL_PPC) suppress MACINTOSH_DRIVERS unless SPARC suppress dependent FC4 unless ARCH_S390==n suppress buses It seems to me *extremely* unlikely that a typical patch from a PPC maintainer would mess with any of these! They're rules that are likely to be written once at the time a new port is added to the tree and seldom or ever changed afterwards. Thus I really don't think you have to worry about spurious spikes in your diffstat. The root rules.cml file will not change very often -- I know this is true, because I can look at the RCS history since I broke it out in response to your request at the Kernel Summit and *see* that changes have been few and sparse. > In contrast, if it starts talking about Documentation/Configure.help and > the main config file, I start worrying. Rightly so in the latter case. Configure.help patches shouldn't worry you, I don't think. It's not like they can actually break anything. > For example, that MATHEMU thing is just ugly. It was perfectly fine in the > per-architecture version, now it suddenly has magic dependencies just > because different architectures call it different things, and different > architectures have different rules on when it's needed. It sounds to me like you're agreeing that it *shouldn't* be called different things, and thus with my goal of cleaning this mess up the rest of the way. Yes? No? Guidance, please. I am, as ever, willing to meet your concerns. But I have to understand clearly what they are in order to do that. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond "...The Bill of Rights is a literal and absolute document. The First Amendment doesn't say you have a right to speak out unless the government has a 'compelling interest' in censoring the Internet. The Second Amendment doesn't say you have the right to keep and bear arms until some madman plants a bomb. The Fourth Amendment doesn't say you have the right to be secure from search and seizure unless some FBI agent thinks you fit the profile of a terrorist. The government has no right to interfere with any of these freedoms under any circumstances." -- Harry Browne, 1996 USA presidential candidate, Libertarian Party ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Eric S. Raymond wrote: > > OK. Background, for anyone who doesn't know this: the equivalent of > Configure.help in CML2 is the symbols.cml file. It's actually generated > fat CML2 installation time from Configure.help. Oh, crap, _another_ magic global file. Eric, this is the _wrong_approach_. I want /local/ files, not global ones. Linus ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Eric S. Raymond wrote: > Legacy Fishtank <[EMAIL PROTECTED]>: > > Note I am specifically NOT talking about MAINTAINERS and CREDITS. > > -PLEASE- don't obscure my point by mentioning them. > > It's the same problem, Jeff. Same solution, too. It's not. We already have pre-file credits information - people can put stuff in there for their own (C) messages etc. The MAINTAINERS file is a much higher-level thing which is there to be human-readable. Note that I do _not_ want to mess up source files with magic comments. I absolutely detest those. They only detract from the real job of coding, and do not make anybody happier. We have a hierarchical filesystem. Most drivers already have driver.c driver.h (in fact _very_ few drivers are single-file) and some have a subdirectory of their own. So why not just have driver.conf and be done with it. No point in messing up the C file with stuff that doesn't add any information to either the programmer _or_ the compiler. Then you can make the config file _truly_ readable, and make the format something like define tristate CONFIG_SCSI_AIC7XXX: Adaptec AIC7xxx support This driver supports all of Adaptec's PCI based SCSI controllers (not the hardware RAID controllers though) as well as the aic7770 based EISA and VLB SCSI controllers (the 274x and 284x series). This is an Adaptec sponsored driver written by Justin Gibbs. It is intended to replace the previous aic7xxx driver maintained by Doug Ledford since Doug is no longer maintaining that driver. depends on CONFIG_SCSI depends on CONFIG_PCI depends on ... define integer CONFIG_AIC7XXX_CMDS_PER_DEVICE: Maximum number of TCQ commands per device depends on CONFIG_SCSI_AIC7XXX default value 253 define integer CONFIG_AIC7XXX_RESET_DELAY_MS: Initial bus reset delay in milli-seconds depends on CONFIG_SCSI_AIC7XXX default value 15000 and it's readable and probably trivially parseable into both the existing format (ie some "find . -name '*.conf'" plus sed-scripts) and into cml2 or whatever. Linus ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 05:08:40PM -0500, Eric S. Raymond wrote: > What makes megabyte-large blocks of code bad is that they tend to be > tangled messes with unmaintainable reference and control structures. > Configure.help isn't; the entries are atoms that don't interact with > each other. Doesn't this statement miss the point that the configure stuff belongs with the associated source, for many reasons? Reasons being stuff like Linus' desire to see PPC stuff only under the PPC directories, stuff like the desire to have the config stuff as part of the source. The very fact that they are atoms and don't interact with each seems to scream that they should be located next to, or in, the stuff with which they do interact. -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Re: State of the new config & build system
Linus Torvalds <[EMAIL PROTECTED]>:
> So the config file format could be something that includes the docs, and
> you could do something like
>
> find . -name '*.cf' -exec grep '^+' {} \; > Configure.help
>
> for old tools, and nw tools would just automatically get the docs from the
> same place they get the config info.
OK. Background, for anyone who doesn't know this: the equivalent of
Configure.help in CML2 is the symbols.cml file. It's actually generated
fat CML2 installation time from Configure.help.
Here's what a sample entry looks like in Configure.help form:
Support the foo feature
CONFIG_FOO
This is a sample help entry.
Here's the same entry in CML2 format:
FOO "Support the foo feature" text
This is a sample help entry.
.
Now. It would be dead easy to split symbols.cml into bunch of little
files and distribute them through the source tree. It would be just as
easy to write a script to reassemble Configure.help, but there won't be
any reason to do it. Once CML2 goes in, nothing will use Configure.help
> The current Configure.help is 25k _lines_, and over a megabyte in size. I
> would never consider that good taste in programming, why should I consider
> it good in documentation?
Um...because the cases aren't parallel?
What makes megabyte-large blocks of code bad is that they tend to be
tangled messes with unmaintainable reference and control structures.
Configure.help isn't; the entries are atoms that don't interact with
each other.
--
http://www.tuxedo.org/~esr/";>Eric S. Raymond
We shall not cease from exploration, and the end of all our exploring will be
to arrive where we started and know the place for the first time.
-- T.S. Eliot
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Legacy Fishtank wrote: > > I think one thing to note is that dependencies is that if you are smart > about it, dependencies -really- do not even change when your .config > changes. Absolutely. I detest "gcc -MD", exactly because it doesn't get this part right. "mkdep.c" gets this _right_. Linus ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Eric S. Raymond wrote: > > I'm not certain what you're objecting to, and I want to understand it. > There are rules that use architecture symbols to suppress things like > bus types. I presume that's not a problem for you, but tell me if it is. It _is_ a problem for me, because I want to do "diffstat" on a patch from a PPC maintainer, and if I see anything non-PPC, loud ringing noises go off in my head. I want that diffstat to say _only_ arch/ppc/... include/asm-ppc/... and nothing else. That way I know that I don't have to worry. In contrast, if it starts talking about Documentation/Configure.help and the main config file, I start worrying. For example, that MATHEMU thing is just ugly. It was perfectly fine in the per-architecture version, now it suddenly has magic dependencies just because different architectures call it different things, and different architectures have different rules on when it's needed. Linus ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Alan Cox wrote:
>
> > So if somebody really wants to help this, make scripts that generate
> > config files AND Configure.help files from a distributed set. And once you
> > do that, you could even imagine creating the old-style config files
>
> Something like:
>
> find $TOPDIR -name "*.cf" -exec cat {} \; > Configure.help
For old tools..
> or changing the tools to look for
>
> Documentation/Configure/CONFIG_SMALL_BANANA
"small banana"? Freud would go wild.
But no. I don't want it under the Documentation directory: I'd much rather
have them _together_ with the config file.
So the config file format could be something that includes the docs, and
you could do something like
find . -name '*.cf' -exec grep '^+' {} \; > Configure.help
for old tools, and nw tools would just automatically get the docs from the
same place they get the config info.
And there would _never_ be more than a few entries per config file: you
can imagine having a separate config file for PCI 100Mbps ethernet drivers
and one for ISA drivers.
The current Configure.help is 25k _lines_, and over a megabyte in size. I
would never consider that good taste in programming, why should I consider
it good in documentation?
Linus
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Legacy Fishtank <[EMAIL PROTECTED]>: > Note I am specifically NOT talking about MAINTAINERS and CREDITS. > -PLEASE- don't obscure my point by mentioning them. It's the same problem, Jeff. Same solution, too. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond It will be of little avail to the people, that the laws are made by men of their own choice, if the laws be so voluminous that they cannot be read, or so incoherent that they cannot be understood; if they be repealed or revised before they are promulgated, or undergo such incessant changes that no man, who knows what the law is to-day, can guess what it will be to-morrow. Law is defined to be a rule of action; but how can that be a rule, which is little known, and less fixed? -- James Madison, Federalist Papers 62 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Legacy Fishtank <[EMAIL PROTECTED]>: > For single-file drivers, I like Becker's (correct credit?) system... > about 10 lines of metadata is embedded in a C comment, and it includes > the Config.in and Configure.help info. I proposed implementing something like this about a year ago (to replace the nasty centralized knowledge in the MAINTAINERS and CREDITS files) and was shot down. I'd be happy to take another swing at this problem once the kbuild-2.5/CML2 transition is done. But I don't think we should let it block us from having the good results we can get from that change. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond "Those who make peaceful revolution impossible will make violent revolution inevitable." -- John F. Kennedy ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
[So far, I've generally been trying to keep away from the hot topics, but I guess it's about time to make that experience. Also, let me add that my opinion here is of course influenced by the fact that things didn't go the way I would have liked...] On Fri, 28 Dec 2001, Keith Owens wrote: > On Thu, 27 Dec 2001 17:15:45 -0800, > Larry McVoy <[EMAIL PROTECTED]> wrote: > >[talking about kbuild 2.5 speed] > >Then it does seem reasonable to ask that the new one is at least as fast > >as the old one. > > kbuild 2.4 is fast but inaccurate, kbuild 2.5 is slower but accurate. > Pick one. Most problems which exist within kbuild 2.4 are fixable without the elaborate rewrite Keith did. The single biggest problem the current system has is that modversions get screwed up, since dependencies are screwed up, and yes, that's not easily fixable. However, this problem isn't even attacked in kbuild 2.5 AFAIK (I think modversions are simply disabled there). A couple of months ago, I came up with an alternative to kbuild 2.5. It doesn't try to have all the features kbuild 2.5 has, but solves the major problems with kbuild 2.4. It definitely has things in common with kbuild 2.5, it also uses the "non-recursive" approach, i.e. the top level Makefile includes all the others. It also doesn't have "make dep" but builds dependencies with "gcc -MD" plus postprocessing. I'm not claiming it is complete, and it doesn't even try to add the multiple source tree etc. features. Others said one should use proper version management instead, and I agree with that, but that's not the point. Non-complete list of pros/cons: o gets dependencies right, i.e. a new make "whatever" will really rebuild everything which is needed. Even *with* CONFIG_MODVERSIONS turned on. o uses standard tools. I believe people said that one of the advantages of UNIX is that you don't need specialized tools for everything, but combine existing tools to reach your goals. The new kbuild has the disadvantage that most is implemented from scratch, the meat is in C programs which probably nobody apart from Keith is familiar with. My solution used the standard tool for building, i.e. make + standard utilities like sh, sed, grep and the like. I only have one non-standard tool, that postprocesses a dependency list: replace include/linux/autoconf.h with the /include/linux/config/options - this is needed so that a .config change doesn't cause an entire rebuild every time. o It's actually pretty fast. On my laptop, the time to read all the dependencies when doing a "make bzImage modules" is was about 5 seconds with hot caches. That means a make takes about 5 seconds when there's nothing to do - that's good enough IMHO. When doing a full rebuild, the time spent within make is definitely down in the noise, if only a few files get rebuild, it's noticable, but still faster than what the current kbuild system gives. o The Makefiles in the SUBDIRS look basically the same as currently, only a somewhat simpler (no special $(LD) rules for composite objects etc). Keith implemented a whole new language - I supoose most coders are familiar with normal Makefiles, they have yet to learn the new commands in kbuild-2.5 (which, however, is easy, of course) o It's not nearly as feature-rich as Keith's approach is. o Behind the scenes, the code is not exactly clear. make is pretty flexible, but it really needs some hacks to do what's needed. So if someone wants to understand the build system, it takes some effort - same situation as in kbuild-2.4 and -2.5, though. o I had the major problems solved and things worked fine in my tree. However, I discontinued to work on it months ago, as I saw no way this work would ever be useful for other people - maintaining a build system just for personal use is a bit too much effort. I don't claim that my work is superior to kbuild-2.5 or anything. (I still think it may be "good enough", i.e. does solve the current problems - it doesn't add features, though). But I'm dissatified that there never ever was even consideration. When I posted ideas/patches to kbuild-devel, I usually got a response like "this work isn't needed, I developed kbuild-2.5, which will be the solution to all problems in 2.5". I also submitted non-intrusive changes for 2.4, which fixed/simplified things there without breaking anything, but the answer was about the same, "kbuild-2.4 is obsolete, for 2.5 it's irrelevant". Well, 2.4 will be around for some time I guess... When I replied (with technical arguments), I never heard anything back - compare the current thread about just silently dropping mails/patches ;-( That's why I decided to drop out of the kbuild business again. (BTW: note that this was about kbuild-2.5 only - nothing to do with CML2) --Kai ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuil
[kbuild-devel] Re: State of the new config & build system
More numbers. I coded up a little program (included below) which
reads paths from stdin, lstats() them, and builds an MDBM of inode ->
pathname entries. I ran that 10 times on the 2.4 kernel, which had 8679
files matching *.[chSs]. I did a little tuning of page size and inital
DB size (reduces page split costs) and got it down to 105 millisecs from
200, so we're at 12 usecs per item. Then I removed the mdbm_store()
call so I was doing everything except that. That took 7 usecs/item.
Write path summary: the mdbm_store() cost is about 5 usecs/item, which
is about right. To build a DB of the same number of items as source
files in the kernel should cost less than 50 milliseconds for the DB
part of the work. In other words, it's basically free.
OK, on to the read path. I generated the list of inodes as an ascii file
and wrote another program to open the mdbm and fetch each one. Ran that
10 times, it cost 40 milliseconds to look up all the items, so that's
about 4 usecs/item including the read of the data from stdin. That's
slower than I think it should be and I may go look to see what is going
on, but it's plenty fast for the config/build system.
Here's the code. Sorry about the perlisms, wait, no I'm not, I like those,
but it will make you look at it twice before it makes sense.
--
/*
* inode.c - create an MDBM of inode -> path mappings
*/
#include
#include
#include
#include
#include
#include "mdbm.h"
#define unless(x) if (!(x))
#define fnext(buf, f) fgets(buf, sizeof(buf), f)
#define u32 unsigned int
void
chomp(char *s)
{
unless (s && *s) return;
while (*s && (*s != '\n')) s++;
*s = 0;
}
u32
inode(char *path)
{
struct stat sb;
if (lstat(path, &sb)) return (0);
return ((u32)sb.st_ino);
}
int
main()
{
charbuf[1024];
MDBM*m;
datum k, v;
u32 ino;
unlink("ino.mdbm");
unless (m = mdbm_open("ino.mdbm", O_RDWR|O_CREAT, 0644, 4<<10)) {
perror("ino.mdbm");
exit(1);
}
mdbm_pre_split(m, 128);
while (fnext(buf, stdin)) {
chomp(buf);
unless (ino = inode(buf)) {
perror(buf);
continue;
}
printf("%u\n", ino);
k.dptr = (void*)&ino;
k.dsize = sizeof(u32);
v.dptr = buf;
v.dsize = strlen(buf) + 1;
if (mdbm_store(m, k, v, MDBM_INSERT)) {
perror(buf);
exit(1);
}
}
mdbm_close(m);
exit(0);
}
--
/*
* read.c - read items from the mdbm
*/
#include
#include
#include
#include
#include
#include "mdbm.h"
#define unless(x) if (!(x))
#define fnext(buf, f) fgets(buf, sizeof(buf), f)
#define u32 unsigned int
int
main()
{
charbuf[1024];
MDBM*m;
datum k, v;
u32 ino;
unless (m = mdbm_open("ino.mdbm", O_RDONLY, 0644, 0)) {
perror("ino.mdbm");
exit(1);
}
while (fnext(buf, stdin)) {
ino = atoi(buf);
continue;
k.dptr = (void*)&ino;
k.dsize = sizeof(u32);
v = mdbm_fetch(m, k);
unless (v.dsize) {
perror(buf);
exit(1);
}
}
mdbm_close(m);
exit(0);
}
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Alexander Viro <[EMAIL PROTECTED]>: > > I think this is an issue that is rising in importance. I have no problem > > with assuming that kernel hackers are English-literate, but it's no longer > > an assumption we should make about people *building* kernels. I want > > to encourage CML2 and question-string localizations for French. And > > German. And Thai. And Ethiopian. > > You are nuts. OK, you've got these translations. Now what? $FOO adds > a new option. Should he, by any chance, supply all relevant translations > in the same patch? No? No. The usual way to handle this, of course, is to fall back on the English where you don't have translations. Imperfect, but liveable. > Good, so we are going to have them permanently > out of sync. Better yet, option changes its meaning. Now we have > English variant with new semantics and Thai one with the old. Happy, > happy, joy, joy... Which is why there are organized translation groups that do periodic translation updates for software that has registered with them. This doesn't eliminate the problem, but it can keep it within manageable bounds that make having localizations better than not. I deal with this regularly with respect to fetchmail. Anyway, options change semantics only very rarely in the kernel rulebase. Trust me on this as I've been maintaining the CML2 rulebase for 18 months; I have a better handle on the frequency of these events than *anyone* else. You are worrying about a non-problem in this case. > And that's aside of the real problem with "internationalization" - quality > of translations _sucks_. Always. No, not always. I read French, Italian, and Spanish; I can puzzle out technical prose in a couple of other languages. I can read fetchmail's .po files and *see* that they don't suck. > Frankly, I find it very amusing that advocates of i18n efforts tend to > be either British or USAnians. That's not my experience. I've had technical problems with GNU gettext (unrelated to quality of translation) severe enough that I've come very close to dropping localization support twice. The people who plead with me not to drop it have been non-Anglophones. It may be that the reason our experiences have been different is because we focus on different target languages. But I think my experience is an existence proof that there *is* demand for localization and that meeting it can have useful results. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond I do not find in orthodox Christianity one redeeming feature. -- Thomas Jefferson ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] autoconfigure.sh and cmlconfigure.py design philosophy
Giacomo A. Catenazzi <[EMAIL PROTECTED]>: > 1) From CML2 rules I derive all symbols that depends on some rules. >The new tables are included in autoconfigure.sh >This can be done in Makefile, so the symbols are always updated, >and the autoconfigure is still simple shell, so it can be run >to the TARGET machine. >(easy ? python script). Yes, pretty easy. Or you could just hack autoconfigure.sh itself into Python and access the rulebase pickle directly. > 2) I give CML2 some rules, and set the symbols (not already set) >according to these rules. >(complex hack in CML2 ?) Yes, quite complex. The problem is that adding new rules is something you have to invoke the compiler for, and it would involve mutating the rulebase pickle. I don't want to go that route. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary. -- H.L. Mencken ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001, Eric S. Raymond wrote: > The design reason is that having a single file with all the symbol-to-prompt > mappings in it is really helpful when you want to localize the rulebase for > another language. I'm still leaning towards keeping symbols.cml together > just to make it easier for people to do and distribute translations of it. > > I think this is an issue that is rising in importance. I have no problem > with assuming that kernel hackers are English-literate, but it's no longer > an assumption we should make about people *building* kernels. I want > to encourage CML2 and question-string localizations for French. And > German. And Thai. And Ethiopian. You are nuts. OK, you've got these translations. Now what? $FOO adds a new option. Should he, by any chance, supply all relevant translations in the same patch? No? Good, so we are going to have them permanently out of sync. Better yet, option changes its meaning. Now we have English variant with new semantics and Thai one with the old. Happy, happy, joy, joy... And that's aside of the real problem with "internationalization" - quality of translations _sucks_. Always. Yes, USAnian to English is easy. But that's it. I've tried to use LANG=ru_RU.koi8-r. It had lasted a couple of days. You end up reconstructing the English original and translating it to Russian - and boy, does that process annoy... Frankly, I find it very amusing that advocates of i18n efforts tend to be either British or USAnians. Folks, get real - your languages are too close to show where the problems are. I can see how doing that gives you a warm fuzzy feeling, but could you please listen to those of us who have to deal with the resulting mess for real? ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] autoconfigure.sh and cmlconfigure.py design philosophy
"Eric S. Raymond" wrote: > There is a third possibility. autoconfigure.sh is going to pass its > results to cmlconfigure.py in a file with the format of a config.out. > Right now all those results are symbol bindings (FOO=y, BAR=n). > Conceivably autoconfigure.sh could also pass a directive like > "finished PCI" which would tell cmlconfigure.py that all PCI-dependent > symbols not yet set should be set to `n'. > > Either of these alternatives (autoconfigure.sh querying the rulebase > or passing a directive to cmlconfigure.py) could work nicely. Which > is the right thing is really a question of design philosophy -- that > is, what do we *want* the roles of autoconfigure.sh and > cmlconfigure.py to be? And how closely coupled should they be? I already thinked two possibilities (there are similar to your alternatives). 1) From CML2 rules I derive all symbols that depends on some rules. The new tables are included in autoconfigure.sh This can be done in Makefile, so the symbols are always updated, and the autoconfigure is still simple shell, so it can be run to the TARGET machine. (easy ? python script). 2) I give CML2 some rules, and set the symbols (not already set) according to these rules. (complex hack in CML2 ?) These rules IMHO can be more complex. I.e. I think that for PCI I should use the "if PCI and not PCI_HOTPLUG then set no". Then CML2 should ask user if he want HOTPLUG support, and then eventually the PCI_HOTPLUG card. (I should detect the PCI_HOTPLUG, but yet is a edge feature, so I know little about this, but anyway it is a simple example). For the syntax of autoconfig.out, I think it better to hide such rules in meta-commens (e.g. #:RULES ...) I have no preferences. giacomo ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 08:42:44PM +1100, Keith Owens wrote:
> "All" I need to do is have one server process that reads the big list
> once and the other client processes talk to the server. Much less data
> involved means faster conversion from absolute to standardized names.
Actually, if you use the mdbm code, you can have a server process which
reads the data, stashes it in the db, touchs ./i_am_done, and exits.
"client" processes do a
while (!exists("i_am_done")) usleep(10);
m = mdbm_open("db", O_RDONLY, 0, 0);
val = mdbm_fetch_str(m, "key");
etc.
No sockets, no back and forth, runs at mmap speed.
If you tell me what the data looks like that needs to be in the DB, i.e.,
how to get it, I'll code you up the "server" side.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
Linus Torvalds <[EMAIL PROTECTED]>: > My pet peeve is "centralized knowledge". I absolutely detested the first > versions of cml2 for having a single config file, and quite frankly I > don't think Eric has even _yet_ separated things out enough - why does the > main "rules.cml" file have architecture-specific info, for example? I'm not certain what you're objecting to, and I want to understand it. There are rules that use architecture symbols to suppress things like bus types. I presume that's not a problem for you, but tell me if it is. My best guess is that you're objecting to the archihacks and kernelhacking menus, or the architecture-dependent derivations down around line 330. In general what's going on here is actually the beginnings of an attempt to replace architecture-dependent questions with architecture-*independent* questions. It looks kind of ugly right now because it's too early in the game to mess with the config-symbol namespace -- but, for example, I want to merge the MATH_EMULATION and MATHEMU symbols eventually. And there ought to be a generic set of toggles for kernel-debugging that present to the user as cross-platform capabilities rather than platform- specific switches. In those two menus I've gathered together architecture-specific symbols that I think ought to merge into cross-platform capabilities. But I know there is other cruft in there for historical reasons. Since you've brought up the point, I'll do a cleanup pass on these and see how much I can exile to the arch/*/rules.cml files. There isn't really any help for the ceoss-platform derivations. There are exactly four of these. I've worked hard at holding them to a minimum: derive HAVE_DEC_LOCK from (SMP and (ALPHA or X86_CMPXCHG)) or SPARC or PPC derive HIGHMEM from HIGHMEM4G or HIGHMEM64G or SPARC derive MAC_HID from (ALL_PPC and INPUT!=n) or (MAC and INPUT_ADBHID) derive PC_KEYB from ARM_PC_KEYB or MIPS_PC_KEYB If you notice that each right-hand part includes port symbols from at least two different architectures, I think it will be clearer why these are necessary. CML1's way of doing this had the problem that it was hard to know by inspection of the rulebase under what circumstances a given symbol was actually turned on. This is why CML2 has a rule that each symbol is derived (or occurs in a menu) exactly once. With some work I could relax this restriction, but I don't want to -- it's a major factor in keeping the rulebase's complexity down in the range that a human brain can mentally model. > That's a big step backwards as far as I'm concerned - we didn't use to > have those stupid global files, and each architecture could do it's own > config rules. Eric never got the point that to me, modularity is _the_ > most important thing for maintenance. Oh, no, I got that all right. What I have been trying to do is trade off correctly between modularity (which helps maintenance) and the advantages to the configurator *users* of having a global capability namespace, single-apex menu structure, and the symbols-to-prompts mapping in one file. These choices weren't made at random. You don't readily see their advantages because you have a nose-to-the-code, maintainer perspective (quite properly so, in most cases). But in designing the configuration system, simplifying life for *users* is just as important, if not more so. Sometimes this implies not going as far in the direction you favor direction as you might like (monolithic Configure.help is an example). > Something I also asked for the config system at least a year ago was to > have Configure.help split up. Never happened. It's still one large ugly > file. Driver or architecture maintainers still can't just change _their_ > small fragment, they have to touch a global file that they don't "own". Yes, there are two reasons for this. The contingent, historical reason is that I wanted to get Configure.help in good shape before thinking about dispersing it. That work is now done (though you haven't kept up to date with it). The design reason is that having a single file with all the symbol-to-prompt mappings in it is really helpful when you want to localize the rulebase for another language. I'm still leaning towards keeping symbols.cml together just to make it easier for people to do and distribute translations of it. I think this is an issue that is rising in importance. I have no problem with assuming that kernel hackers are English-literate, but it's no longer an assumption we should make about people *building* kernels. I want to encourage CML2 and question-string localizations for French. And German. And Thai. And Ethiopian. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond If I were to select a jack-booted group of fascists who are perhaps as large a danger to American society as I could pick today, I would pick BATF [the Bureau of Alcohol, Tobacco, and Firearms]. -- U.S. Representative John Dingell, 1980 _
[kbuild-devel] Re: State of the new config & build system
> So if somebody really wants to help this, make scripts that generate
> config files AND Configure.help files from a distributed set. And once you
> do that, you could even imagine creating the old-style config files
Something like:
find $TOPDIR -name "*.cf" -exec cat {} \; > Configure.help
or changing the tools to look for
Documentation/Configure/CONFIG_SMALL_BANANA
??
Alan
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
[ Btw, Jeff, any reason why you changed your name to "Legacy Fishtank"? It took a few mails before I noticed that it also said "garzik" in the fine print;] One thing that this big flame-war has brought up is that different people like different things. There may be a simpler solution to this: have the core dependency files generated from some other file format. My pet peeve is "centralized knowledge". I absolutely detested the first versions of cml2 for having a single config file, and quite frankly I don't think Eric has even _yet_ separated things out enough - why does the main "rules.cml" file have architecture-specific info, for example? That's a big step backwards as far as I'm concerned - we didn't use to have those stupid global files, and each architecture could do it's own config rules. Eric never got the point that to me, modularity is _the_ most important thing for maintenance. Something I also asked for the config system at least a year ago was to have Configure.help split up. Never happened. It's still one large ugly file. Driver or architecture maintainers still can't just change _their_ small fragment, they have to touch a global file that they don't "own". So if somebody really wants to help this, make scripts that generate config files AND Configure.help files from a distributed set. And once you do that, you could even imagine creating the old-style config files (without the automatic checking and losing some information) from the information. Linus ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 02:14:37PM +, Alan Cox wrote: > > Ah, OK, I get it. Hey, would it help to have a dbm interface compat > > library which uses mmap instead of building the db in brk() space? > > mmap for db file seems to be slower. I'll need to see some numbers to back up that statement, please. If you look at the graphs produced by LMbench, they tell you exactly what you need to know. It's true that for very small files, 8K and under, using read() to access them is faster than using mmap, due to the extra work of setting up and tearing down the mapping. To quantify this, a 4KB open/read/close is 500MB/sec, but an open/mmap/access/unmap/close is 425MB/sec. By the time we hit 16K, mmap wins by 15% and just gets better from there. And that all assumes you are doing large reads, which in db code you are not. So mmap will look better even on the small files if you are doing little DB style accesses. > For basic db hash usage and raw speed > nothing seems to touch tdb (Tridge's db hack). Taking nothing away from Tridge, I like Tridge, I'd like to see numbers. I'm sure that Tridge's stuff is great, but we were very motivated to go well beyond the normal effort when we wrote this code. A multithreaded version of the code that I sent to Keith was doing 455,000 lookups/second on an 8way 200Mhz R4400 SGI box in 1996. Each lookup was locked. If you assume perfect scaling (it was) and you assume the locks took 0 time (they didn't), that's 1.75 usecs for each lookup. On a machine with horrible memory latency and a large dataset. We designed the MDBM code to be scalable (its 64bit clean), portable (runs on 20+ platforms today), multiplatform (metadata is stored in network byte order on disk), and fast (we knew exactly what the instruction and data cache footprint was for hot cache, and we made sure that you did at most 2 disk accesses, 1 was typical, to get at any item in a cold cache). SGI uses this code for their name server, every process mmaps the name server cache. We use this code all over BitKeeper. > Its also portable code which > is important since the tool has to be built on the compiling host. The code works on Windows, MacOS X, and basically all Unix platforms. Yeah, yeah, I pounding my chest and I'm sorry, but I get beat up all the time that the BK license doesn't let you reuse code and this code is part of BK that we broke out and licensed under the GPL. The point being that if there is reusable code in BK, we're willing to let people use it under whatever license they want. It would be nice if that actually happened after all the yelling and screaming. -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] autoconfigure.sh and cmlconfigure.py design philosophy
Eric S. Raymond wrote: > Conceivably autoconfigure.sh could also pass a directive like > "finished PCI" which would tell cmlconfigure.py that all PCI-dependent > symbols not yet set should be set to `n'. Insofar as I understand the problem, I think this is clearly the best approach, as it minimizes coupling. Autoconfigure knows that what it has learned about the state of the PCI bus is complete, and therefore that no other PCI devices exist. It does not need to know what PCI devices are or are not supported at this moment by the kernel. Allowing it to look inside the kernel rulebase would promote unnecessary coupling. -- Not to perambulate || John Cowan <[EMAIL PROTECTED]> the corridors || http://www.reutershealth.com during the hours of repose || http://www.ccil.org/~cowan in the boots of ascension. \\ Sign in Austrian ski-resort hotel ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> including if you build on one system then try to install via NFS on a > second system. kbuild 2.5 can cope with trees being renamed and trees > having different names on local and NFS mounted systems. That > flexibility comes at a cost. So you've halved performance rather than documented that you have to mount the tree in the space place on every NFS export ? I'm obviously still missing something here. Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] autoconfigure.sh and cmlconfigure.py design philosophy
I have been looking at Giacomo's autoconfigure.sh code and thinking about it should communicate with CML2. I like the concept, and the implementation starting from a table of probe routines calls in a rules file is clever. Unfortunately, as the program is now it yields mostly information about what is present on the system, not information about hardware and features that are known *not* to be present. To reduce the complexity of the user's configuration task to a minimum, autoconfigure.sh should not merely be able to list (for example) all the PCI hardware on the system, it should also be able to assert that the symbols for all *other* PCI hardware should be frozen to `n' so so none of those questions will ever be asked. Similar considerations obtain for any bus with a registry (such as I2O), but PCI is the most important case so I'll keep using it as an example. The design question then becomes this: who knows what the set of all PCI devices is, and computes the complement of the set of detected devices? There are several possible answers to this question. Each has different design implications. autoconfigure.sh itself could compute the set complement. This implies that either (a) autoconfigure.sh itself would have to contain a list of all symbols dependent on PCI or (b) such a list would have to live in the autoconfigure.rules file. Both these design choices would be poor. The major problem is that either would duplicate knowledge already present in the configuration rulebase, probably leading to inconsistencies and certainly creating an unpleasant maintainance task. This leads us directly to the second possibility, that autoconfigure.sh could read the configuration rulebase and use that information to compute the complement set. This could work in one of two ways: either (a) autoconfigure.sh could process the pickled CML2 rulebase itself (in which case it would have to be rewritten at least partly in Python) or (b) autoconfigure.sh could call an agent program to generate the list for it (adding an option to kxref.py to do this would be nearly trivial). There is a third possibility. autoconfigure.sh is going to pass its results to cmlconfigure.py in a file with the format of a config.out. Right now all those results are symbol bindings (FOO=y, BAR=n). Conceivably autoconfigure.sh could also pass a directive like "finished PCI" which would tell cmlconfigure.py that all PCI-dependent symbols not yet set should be set to `n'. Either of these alternatives (autoconfigure.sh querying the rulebase or passing a directive to cmlconfigure.py) could work nicely. Which is the right thing is really a question of design philosophy -- that is, what do we *want* the roles of autoconfigure.sh and cmlconfigure.py to be? And how closely coupled should they be? I don't have strong opinions on either question. But I want to get them on the table so we'll think about them at a design level before rushing to code. -- http://www.tuxedo.org/~esr/";>Eric S. Raymond Rifles, muskets, long-bows and hand-grenades are inherently democratic weapons. A complex weapon makes the strong stronger, while a simple weapon -- so long as there is no answer to it -- gives claws to the weak. -- George Orwell, "You and the Atom Bomb", 1945 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001 14:14:37 + (GMT), Alan Cox <[EMAIL PROTECTED]> wrote: >> Ah, OK, I get it. Hey, would it help to have a dbm interface compat >> library which uses mmap instead of building the db in brk() space? > >mmap for db file seems to be slower. For basic db hash usage and raw speed >nothing seems to touch tdb (Tridge's db hack). Its also portable code which >is important since the tool has to be built on the compiling host. lm sent me the bk mdbm code but I will look at tdb as well. Four acronyms in one sentance, I must be a phb :). >Personally I've always considered make dep good enough. Its trying to solve >the extra .5% that probably can be solved by careful use of make clean when >CML realises a critical rule changed (SMP etc) http://prdownloads.sourceforge.net/kbuild/kbuild-2.5-history.tar.bz2 Especially makefile-2.5_make_dep.html, 9 reasons why make dep is broken as designed. Some are fixable in the current system, others are inherently unfixable. I skipped that page when I did my presentation at the 2.5 developer's conference. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> At the moment kbuild 2.5 ranges from 10% faster on small builds to 100% > slower on a full kernel build. But that is using slow core code which > kbuild 2.5 is ready now. I am not even going to start on the core "Its 100% slower so its ready" I must be missing something here. If its 100% slower its not ready. Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
> Ah, OK, I get it. Hey, would it help to have a dbm interface compat > library which uses mmap instead of building the db in brk() space? mmap for db file seems to be slower. For basic db hash usage and raw speed nothing seems to touch tdb (Tridge's db hack). Its also portable code which is important since the tool has to be built on the compiling host. Personally I've always considered make dep good enough. Its trying to solve the extra .5% that probably can be solved by careful use of make clean when CML realises a critical rule changed (SMP etc) Alan ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, 28 Dec 2001 04:26:48 -0500, Legacy Fishtank <[EMAIL PROTECTED]> wrote: >On Fri, Dec 28, 2001 at 01:54:42AM +0100, Dave Jones wrote: >> How far down the list was "make it not take twice as long >> to build the kernel as kbuild 2.4" ? Keith mentioned O(n^2) >> effects due to each compile operation needing to reload >> the dependancies etc. > >Each compile needs to reload deps??? > >Ug. IMHO if you are doing to shake up the entire build system, you >should Do It Right(tm) and build a -complete- dependency graph -once-. We have one complete dependency graph for the explicit dependencies. What is slow is extracting the implicit dependencies after an object has been compiled, i.e. the files that it includes. Actually extracting the implicit dependencies is fast, converting them to standard names is fast, what is slow is _reading_ the big list that maps from absolute names to standardized names. I need the big list in order to remove absolute names in the dependency trees. kbuild 2.4 forces a complete recompile if you rename a tree, including if you build on one system then try to install via NFS on a second system. kbuild 2.5 can cope with trees being renamed and trees having different names on local and NFS mounted systems. That flexibility comes at a cost. "All" I need to do is have one server process that reads the big list once and the other client processes talk to the server. Much less data involved means faster conversion from absolute to standardized names. ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
[kbuild-devel] Re: State of the new config & build system
On Fri, Dec 28, 2001 at 01:54:42AM +0100, Dave Jones wrote: > How far down the list was "make it not take twice as long > to build the kernel as kbuild 2.4" ? Keith mentioned O(n^2) > effects due to each compile operation needing to reload > the dependancies etc. Each compile needs to reload deps??? Ug. IMHO if you are doing to shake up the entire build system, you should Do It Right(tm) and build a -complete- dependency graph -once-. Jeff ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
