Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Wed, Jun 25, 2008 at 10:50:43AM +0100, James Chapman wrote: > Adrian Bunk wrote: >> On Mon, Jun 23, 2008 at 07:28:09PM +0200, Denys Vlasenko wrote: >>> On Thursday 01 May 2008 12:41, Andi Kleen wrote: > To a large extent, I agree. I certainly don't want to focus solely on > code size; there's a lot more to embedded Linux than that. But it _is_ Not only code size, far more important is dynamic memory consumption. [admittedly we right now lack a good instrumentation framework for this] > There are some cases where we really _do_ want to have CONFIG options, > but I agree that we should keep them to a minimum. And when we _do_ have > CONFIG options, they don't have to litter the actual code with ifdefs. The problem I see is more that really nobody can even compile not alone test all these combinations anymore. Hidding the problem in inlines does not solve that. And no randconfig is not the solution either. >>> Because we allowed kernel to be developed without the requirement that >>> random config should be buildable for release kernels. >>> >>> Had it been a requirement, keeping it in shape wouldn't be >>> too difficult. >>> >>> Sure enough, _now_ fixing kernel to pass such a test on i386 >>> would take several weeks of work at least. But it is doable. >>> ... >> >> On i386 it might even already work today. >> >> But guess how much time it costs to get at least all defconfigs >> compiling on the other 22 architectures. >> >> Even getting allmodconfig/allyesconfig compiling isn't trivial for all >> architectures, and random configurations are _far_ from compiling. > > > > And we are not talking about something to be done once, as soon as you > > leave x86 there are tons of regular breakages. > > Could automated builds and build error reporting be used to help resolve > these problems? > > The good people at Simtec have an automated build report available as an > e-mail digest. I use it to watch for architecture build breakages in > subsystems or drivers that I use or touch. It covers defconfigs of ARM > and MIPS architectures and reports compile errors/warnings, module size, > kernel size etc. If this approach were extended/distributed to cover > more architectures Jan Dittmer has a great page showing the build status and kernel size of the defconfigs of all architectures that is running since 2004 or 2005: http://l4x.org/k/ > and random config builds, developers could with > little effort spot problems and fix them. Hell, it might also encourage > new developers to get involved and contribute. Perhaps in an ideal world... In reality, I'd claim I'm one out of only two people who regularly fix architecture-specific build problems for all architectures. > James Chapman cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Adrian Bunk wrote: On Mon, Jun 23, 2008 at 07:28:09PM +0200, Denys Vlasenko wrote: On Thursday 01 May 2008 12:41, Andi Kleen wrote: To a large extent, I agree. I certainly don't want to focus solely on code size; there's a lot more to embedded Linux than that. But it _is_ Not only code size, far more important is dynamic memory consumption. [admittedly we right now lack a good instrumentation framework for this] There are some cases where we really _do_ want to have CONFIG options, but I agree that we should keep them to a minimum. And when we _do_ have CONFIG options, they don't have to litter the actual code with ifdefs. The problem I see is more that really nobody can even compile not alone test all these combinations anymore. Hidding the problem in inlines does not solve that. And no randconfig is not the solution either. Because we allowed kernel to be developed without the requirement that random config should be buildable for release kernels. Had it been a requirement, keeping it in shape wouldn't be too difficult. Sure enough, _now_ fixing kernel to pass such a test on i386 would take several weeks of work at least. But it is doable. ... On i386 it might even already work today. But guess how much time it costs to get at least all defconfigs compiling on the other 22 architectures. Even getting allmodconfig/allyesconfig compiling isn't trivial for all architectures, and random configurations are _far_ from compiling. > > And we are not talking about something to be done once, as soon as you > leave x86 there are tons of regular breakages. Could automated builds and build error reporting be used to help resolve these problems? The good people at Simtec have an automated build report available as an e-mail digest. I use it to watch for architecture build breakages in subsystems or drivers that I use or touch. It covers defconfigs of ARM and MIPS architectures and reports compile errors/warnings, module size, kernel size etc. If this approach were extended/distributed to cover more architectures and random config builds, developers could with little effort spot problems and fix them. Hell, it might also encourage new developers to get involved and contribute. Here's a link to a recent report for ARM, fyi:- http://lists.simtec.co.uk/pipermail/kautobuild/2008-June/001299.html Plus the fact that you often get into situations where more options mean complex and fragile stuff. Read the Kconfig files under drivers/media/ and check in git all commits to them since 2.6.25 alone, and you'll understand why "add an option for every bit" can result in very high ongoing maintainance work required. Not everything that is technically possible is also maintainable, and maintainability is a very important point in a project with several million lines changing each year. vda cu Adrian -- James Chapman Katalix Systems Ltd http://www.katalix.com Catalysts for your Embedded Linux software development -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Mon, Jun 23, 2008 at 09:12:30PM +0200, Denys Vlasenko wrote: > On Monday 23 June 2008 20:57, Sam Ravnborg wrote: > > > > I agree. And if we do want to pay attention to pure code size, there are > > > > other approaches -- like --gc-sections > > > > > > I have some patches in this area too. Were submitted to Sam > > > but he was too busy it seems. > > > > They were not trivial to apply and so went down on the TODO list. > > I realize that they were not trivial to review, but that > was unavoidable. They were even more not trivial to create. > > > We could try to push the generic and x86 specific .lds stuff via > > the arch maintainers? > > IIRC I splitted the entire GC collection patch in a way > that first patches were doing exactly this easier first part > and I hoped that at least these first patches > will be taken. They were big, but somewhat trivial, > from "it's obviously safe" department. I do not recall anything wrong with the patch-set. > > Had they been applied, now making --gc-sections to work > would be easier. Agreed. I should have asked you to push this via arch maintainers back then. Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Monday 23 June 2008 20:57, Sam Ravnborg wrote: > > > I agree. And if we do want to pay attention to pure code size, there are > > > other approaches -- like --gc-sections > > > > I have some patches in this area too. Were submitted to Sam > > but he was too busy it seems. > > They were not trivial to apply and so went down on the TODO list. I realize that they were not trivial to review, but that was unavoidable. They were even more not trivial to create. > We could try to push the generic and x86 specific .lds stuff via > the arch maintainers? IIRC I splitted the entire GC collection patch in a way that first patches were doing exactly this easier first part and I hoped that at least these first patches will be taken. They were big, but somewhat trivial, from "it's obviously safe" department. Had they been applied, now making --gc-sections to work would be easier. -- vda -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Adrian Bunk wrote: > On Mon, Jun 23, 2008 at 07:28:09PM +0200, Denys Vlasenko wrote: >> Had it been a requirement, keeping it in shape wouldn't be >> too difficult. >> >> Sure enough, _now_ fixing kernel to pass such a test on i386 >> would take several weeks of work at least. But it is doable. >> ... > > On i386 it might even already work today. > > But guess how much time it costs to get at least all defconfigs > compiling on the other 22 architectures. > > Even getting allmodconfig/allyesconfig compiling isn't trivial for all > architectures, and random configurations are _far_ from compiling. > > And we are not talking about something to be done once, as soon as you > leave x86 there are tons of regular breakages. > > Plus the fact that you often get into situations where more options > mean complex and fragile stuff. Read the Kconfig files under > drivers/media/ and check in git all commits to them since 2.6.25 alone, > and you'll understand why "add an option for every bit" can result in > very high ongoing maintainance work required. > > Not everything that is technically possible is also maintainable, and > maintainability is a very important point in a project with several > million lines changing each year. OK sure. Nobody's going to disagree with that. I would, however, disagree with a characterization of Linux-tiny as "adding an option for every bit". Linux-tiny has been around about 5 years now, and if you added the whole thing right now you'd add about 30 config options. If you're worried about this multiplying out of control, let me just say that having to curtail the rate of patch submission by embedded developers has not been our biggest problem. :-) -- Tim = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Mon, Jun 23, 2008 at 07:22:10PM +0200, Denys Vlasenko wrote: > On Wednesday 30 April 2008 21:11, David Woodhouse wrote: > > On Wed, 2008-04-30 at 20:22 +0200, Andi Kleen wrote: > > > David Woodhouse <[EMAIL PROTECTED]> writes: > > > > > > > Andrew Morton has been saying recently that we need an 'embedded > > > > maintainer', to take responsibility for 'embedded issues' in the core > > > > kernel, as well as trying to improve our relationship with those using > > > > the Linux kernel for 'embedded' devices -- who have a reputation of > > > > not working with us very closely; to their detriment as well as our > > > > own. > > > > > > I hope your job description doesn't include adding more and more > > > CONFIGs though. > > > > > > I am sure there are lots of low hanging fruit where memory can be > > > saved and it's a good thing someone cares about that, but please don't > > > focus on the code size only. Or if you work on that don't do it > > > using CONFIG or when you really add a new one find some other > > > that is pointless and remove it first. > > > > > > There are simply already far too many of them and they make the > > > kernel harder and harder to change. > > > > I agree. And if we do want to pay attention to pure code size, there are > > other approaches -- like --gc-sections > > I have some patches in this area too. Were submitted to Sam > but he was too busy it seems. They were not trivial to apply and so went down on the TODO list. We could try to push the generic and x86 specific .lds stuff via the arch maintainers? Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Monday 23 June 2008 19:45, Adrian Bunk wrote: > Plus the fact that you often get into situations where more options > mean complex and fragile stuff. Read the Kconfig files under > drivers/media/ and check in git all commits to them since 2.6.25 alone, > and you'll understand why "add an option for every bit" can result in > very high ongoing maintainance work required. > > Not everything that is technically possible is also maintainable, and > maintainability is a very important point in a project with several > million lines changing each year. Well, I am not (and was not) disputing this. I agree with it. CONFIGs should not be multiplying like rabbits. -- vda -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Mon, Jun 23, 2008 at 07:28:09PM +0200, Denys Vlasenko wrote: > On Thursday 01 May 2008 12:41, Andi Kleen wrote: > > > To a large extent, I agree. I certainly don't want to focus solely on > > > code size; there's a lot more to embedded Linux than that. But it _is_ > > > > Not only code size, far more important is dynamic memory consumption. > > [admittedly we right now lack a good instrumentation framework for this] > > > > > There are some cases where we really _do_ want to have CONFIG options, > > > but I agree that we should keep them to a minimum. And when we _do_ have > > > CONFIG options, they don't have to litter the actual code with ifdefs. > > > > The problem I see is more that really nobody can even compile not > > alone test all these combinations anymore. Hidding the problem in inlines > > does not solve that. And no randconfig is not the solution either. > > Because we allowed kernel to be developed without the requirement that > random config should be buildable for release kernels. > > Had it been a requirement, keeping it in shape wouldn't be > too difficult. > > Sure enough, _now_ fixing kernel to pass such a test on i386 > would take several weeks of work at least. But it is doable. >... On i386 it might even already work today. But guess how much time it costs to get at least all defconfigs compiling on the other 22 architectures. Even getting allmodconfig/allyesconfig compiling isn't trivial for all architectures, and random configurations are _far_ from compiling. And we are not talking about something to be done once, as soon as you leave x86 there are tons of regular breakages. Plus the fact that you often get into situations where more options mean complex and fragile stuff. Read the Kconfig files under drivers/media/ and check in git all commits to them since 2.6.25 alone, and you'll understand why "add an option for every bit" can result in very high ongoing maintainance work required. Not everything that is technically possible is also maintainable, and maintainability is a very important point in a project with several million lines changing each year. > vda cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Thursday 01 May 2008 12:41, Andi Kleen wrote: > > To a large extent, I agree. I certainly don't want to focus solely on > > code size; there's a lot more to embedded Linux than that. But it _is_ > > Not only code size, far more important is dynamic memory consumption. > [admittedly we right now lack a good instrumentation framework for this] > > > There are some cases where we really _do_ want to have CONFIG options, > > but I agree that we should keep them to a minimum. And when we _do_ have > > CONFIG options, they don't have to litter the actual code with ifdefs. > > The problem I see is more that really nobody can even compile not > alone test all these combinations anymore. Hidding the problem in inlines > does not solve that. And no randconfig is not the solution either. Because we allowed kernel to be developed without the requirement that random config should be buildable for release kernels. Had it been a requirement, keeping it in shape wouldn't be too difficult. Sure enough, _now_ fixing kernel to pass such a test on i386 would take several weeks of work at least. But it is doable. I would even volunteer to do it if there are some reasonable chances resulting patches would be viewed as worthwhile for inclusion. I am somewhat tired of killing weeks of my time only to find that my work is deemed "not important enough for inclusion". -- vda -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Wednesday 30 April 2008 21:11, David Woodhouse wrote: > On Wed, 2008-04-30 at 20:22 +0200, Andi Kleen wrote: > > David Woodhouse <[EMAIL PROTECTED]> writes: > > > > > Andrew Morton has been saying recently that we need an 'embedded > > > maintainer', to take responsibility for 'embedded issues' in the core > > > kernel, as well as trying to improve our relationship with those using > > > the Linux kernel for 'embedded' devices -- who have a reputation of > > > not working with us very closely; to their detriment as well as our > > > own. > > > > I hope your job description doesn't include adding more and more > > CONFIGs though. > > > > I am sure there are lots of low hanging fruit where memory can be > > saved and it's a good thing someone cares about that, but please don't > > focus on the code size only. Or if you work on that don't do it > > using CONFIG or when you really add a new one find some other > > that is pointless and remove it first. > > > > There are simply already far too many of them and they make the > > kernel harder and harder to change. > > I agree. And if we do want to pay attention to pure code size, there are > other approaches -- like --gc-sections I have some patches in this area too. Were submitted to Sam but he was too busy it seems. -- vda -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
* Rob Landley <[EMAIL PROTECTED]> schrieb: Hi, > You can't get away from cross compiling whenever you want to bootstrap a new > platform. But cross compiling can be minimized and encapsulated. It can be > a stage you pass through to get it over with and no longer have to deal with > it on the other side, which is the approach I take. That's not enought for me. I want more encapsulation (yes, but buildfarm is also running in a VZ, but still sysroot'ed). For example, I *want* certain tests (like running target code) to fail, since I *do not* trust them. (btw: I also let compiler warnings fail, just to be sure). I, personally, prefer high build-time constraints, because runtime tests are quite limited. > By my count sysroot is the fifth layer of path logic the gcc guys have added > in an attempt to paint over the dry rot. I see this as a quite good separation between running and target system, just one more fence for clear borders. As long as you don't pass absolute search pathes to it, you can be sure that it doesn't mix up target and host. Actually, I never want to crosscompile w/o it. > Personally I use a derivative of the old uClibc wrapper script that rewrites > the command line to start with "--nostdinc --nostdlib" and then builds it > back up again without having any paths in there it shouldn't. Might work, but then you're limited to some specific cases. If you *only* intent bootstrapping of an minimal system, fine, but for my projects too complex to handle. > > #2: fix broken configure.in's (and feed back to upstream or OSS-QM) > > Whack-a-mole. Fun for the whole family. Only problem is, it never stops. Most times, it as only to be done one per package. And it's an clean solution. > > #3: replace libtool by unitool > > Uninstall libtool and don't replace it with anything, it's a NOP on Linux. The problem is: many packages are entirely built upon this. So you'll have to de-libtoolize them. Very much work. As I already had Unitool, I preferred investing just a few hours for creating an generic drop-in replacement for libtool instead of touching each single package. > > Only crap sw looks at /proc at build time. > > Yes, there's *much* crap sw out there :( > > 99% of all the developers out there don't really care about portability, and > never will. Even if you eliminate the windows guys and the people who don't > do C, 90% of the people who are _left_ get to work on the PC first, get it to > work natively on other Linux platforms afterwards. True :( Some packages out there even don't have an clean native build path, eg. requiring parts of the package already built and installed (-> firebird-db) > Cross compiling is a step beyond "portability". They'll _never_ care about > cross compiling. If they get inspired to make it work on MacOS X, then > you'll have to extract the source and _build_ it on MacOS X to make that > work. And 99% of all developers will nod their heads and go "quite right, as > it should be". > > This isn't going to change any time soon. Actually, I don't care much about that. I concentrate on getting those packages I need cleaned-up and crosscompile'able - even if this means going alone and maintaining an own branch. If I sum up all the invested working hours of all the last years on that and substract the total saved time from other projects where I'm reusing this work, I get a positive balance ;-P cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ - -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Thursday 12 June 2008 13:18:07 Enrico Weigelt wrote: > * Rob Landley <[EMAIL PROTECTED]> schrieb: > > Hi, > > > There's also qemu. You can native build under emulation. > > did you ever consider that crosscompiling is not only good for > some other arch, but a few more things ? Sure, such as building a uClibc system on a glibc host, which my _previous_ firmware linux project (http://landley.net/code/firmware/old) was aimed at. That used User Mode Linux instead of qemu, because "fakeroot" wasn't good enough and chroot A) requires the build to run as root, B) sometimes gets a little segfaulty if you build uClibc with newer kernel headers than the kernel in the system you're running on. You can't get away from cross compiling whenever you want to bootstrap a new platform. But cross compiling can be minimized and encapsulated. It can be a stage you pass through to get it over with and no longer have to deal with it on the other side, which is the approach I take. > > In addition, if you have a cross compiler but don't want to spend all > > your time lying to ./configure, preventing gcc from linking against the > > host's zlib or grabbing stuff out of /usr/include that your target hasn't > > got, or > > #1: use a proper (sysroot'ed) toolchain I break everything. (I've broken native toolchains. I just break them _less_.) By my count sysroot is the fifth layer of path logic the gcc guys have added in an attempt to paint over the dry rot. Personally I use a derivative of the old uClibc wrapper script that rewrites the command line to start with "--nostdinc --nostdlib" and then builds it back up again without having any paths in there it shouldn't. > #2: fix broken configure.in's (and feed back to upstream or OSS-QM) Whack-a-mole. Fun for the whole family. Only problem is, it never stops. > #3: replace libtool by unitool Uninstall libtool and don't replace it with anything, it's a NOP on Linux. > > libraries are linked inside the emulator, anything that wants to look > > at /proc or sysinfo does it natively inside the emulator...) > > Only crap sw looks at /proc at build time. > Yes, there's *much* crap sw out there :( 99% of all the developers out there don't really care about portability, and never will. Even if you eliminate the windows guys and the people who don't do C, 90% of the people who are _left_ get to work on the PC first, get it to work natively on other Linux platforms afterwards. Cross compiling is a step beyond "portability". They'll _never_ care about cross compiling. If they get inspired to make it work on MacOS X, then you'll have to extract the source and _build_ it on MacOS X to make that work. And 99% of all developers will nod their heads and go "quite right, as it should be". This isn't going to change any time soon. Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Sunday 15 June 2008 10:39:43 Leon Woestenberg wrote: > Hello all, > > On Thu, Jun 12, 2008 at 2:41 AM, Rob Landley <[EMAIL PROTECTED]> wrote: > > Most packages don't cross compile at all. Debian has somewhere north of > > 30,000 packages. Every project that does large scale cross compiling > > (buildroot, gentoo embedded, timesys making fedora cross compile, etc) > > tends to have about 200 packages that cross compile more or less easily, > > another 400 or so that can be made to cross compile with _lot_ of effort > > and a large enough rock, and then the project stalls at about that size. > > Agreed, OpenEmbedded has a few thousands, but your point is valid. > However, fleeing to target-native compilation is not the way to > improve the situation IMHO. You say it like fleeing is a bad thing. :) I believe building natively under emulation is the Right Thing. Cross compiling has always historically been a transitional step until native compiling became available on the target. When Ken Thompson and Dennis Ritchie were originally creating Unix for the PDP-7, they cross compiled their code from a honking big GE mainframe because that was their only option. One of the first things they wrote was a PDP-7 assembler that ran on the PDP-7. The reason they created the B programming language in the first place was to have a tiny compiler that could run natively on the PDP-7, and when they moved up to a PDP-11 Dennis had more space to work with and expanded B into C. When they severed the mainframe umbilical cord as soon as they were able to get the system self-hosting, it wasn't because the PDP-7 had suddenly become faster than the GE mainframe. Compiling natively where possible has been the normal way to build Unix software ever since. Linux became a real project when Linus stopped needing Minix to cross-compile it. Linus didn't "flee" Minix, he assures us he erased his minix partition purely by accident. :) > Moore's law on hardware also goes for the host, Which is why people no longer regularly write application software in assembly language, because we don't need to do that anymore. The result would be faster, but not better. The rise of scripting languages like Python and javascript that run the source code directly is also related (and if you don't think people don't write complete applications in those you haven't seen any of the google apps). The big push for Java in 1998 could happen because the hardware was now fast enough to run _everything_ under an emulator for a processor that didn't actually exist (until Rockwell built one, anyway). Build environments are now literally thousands of times faster than when I started programming. The first machine I compiled code on was a commodore 64 (1mhz, 8 bits, the compiler was called "blitz" and the best accelerator for it was a book). The slowest machine I ever ran Linux on was a 16 mhz 386sx. According to my blog, I moved from a 166mhz laptop to a 266mhz one on April 13, 2002. I started building entire Linux From Scratch systems on the 166mhz machine, including a ton of optional packages (apache, postgresql, openssh, samba, plus it was based on glibc and coreutils and stuff back then so the build was _slow_), hence the necessity of scripting it and leaving the build to its own devices for a few hours. Even without distcc calling out to the cross compiler, the emulated system running on my laptop is several times faster than the build environment I had 7 years ago (2001), somewhat faster than the one I had 5 years ago (2003), and somewhat slower than the one I had 3 years ago (2005). (That's emulating an x86 build environment on my x86_64 laptop. I didn't _have_ a non-x86 build enviornment 5 years ago for comparison purposes.) > I think the progress is even bigger on big iron. Not that I've noticed, unless by "big iron", you mean "PC clusters". (You can expand laterally if you've got the money for it and your problem distributes well...) > Also, how much of the 3 packages are useful for something like > your own firmware Linux? None of them, because Firmware Linux has a strictly limited agenda: provide a native build environment on every system emulation supported by qemu. That's the 1.0 release criteria. (Some day I may add other emulators like hercules for s390, but the principle's the same.) Once you have the native build environment, you can bootstrap Gentoo, or Debian, or Linux From Scratch, or whatever you like. I've got instructions for some of 'em. The buildroot project fell into the trap of becoming a distro and having to care about the interaction between hundreds of packages. I'm not interested in repeating that mistake. Figuring out what packages will other people might need is something I stopped trying to predict a long time ago. If it exists, somebody wanted it. People want/need the weirdest stuff: the accelerometer in laptops is used for rolling marble games
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Hello all, On Thu, Jun 12, 2008 at 2:41 AM, Rob Landley <[EMAIL PROTECTED]> wrote: > > Most packages don't cross compile at all. Debian has somewhere north of > 30,000 packages. Every project that does large scale cross compiling > (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends > to have about 200 packages that cross compile more or less easily, another > 400 or so that can be made to cross compile with _lot_ of effort and a large > enough rock, and then the project stalls at about that size. > Agreed, OpenEmbedded has a few thousands, but your point is valid. However, fleeing to target-native compilation is not the way to improve the situation IMHO. Moore's law on hardware also goes for the host, I think the progress is even bigger on big iron. Also, how much of the 3 packages are useful for something like your own firmware Linux? > Distcc can take advantage of smp, but that won't help the ./configure stage > and I need to do some work on distcc to teach it to understand more gcc > If you want to build 1000+ packages, you don't need to run configure itself multithreaded. There are enough jobs available to keep 16/32 processors busy (beyond that, you probably end up in inter-package-dependencies stalling the build). This is just a guess from what I see during a multi-threaded bake and multi-threaded make on OpenEmbedded. > However, having one or more full-time engineers devoted to debugging > cross-compile issues is quite a high price to pay too. Moore's law really > doesn't help that one. > How about 30+ volunteers. > I'm not saying either solution is perfect, I'm just saying the "build under > emulation" approach is a viable alternative that gets more attractive as time > passes, both because of ongoing development on emulators and because of > Moore's law on the hardware. > I cannot follow your reasoning - Moore's law will help you more on the big iron side of things. That said, I welcome any effort (such as yours) to help improve the embedded Linux domain, I rather try to fix the cross-compile stuff of the few thousand packages I am interested in. Yes it hurts my brain. Regards, -- Leon -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
James Chapman wrote: David VomLehn wrote: Enrico Weigelt wrote: * Rob Landley <[EMAIL PROTECTED]> schrieb: Cross compiling breaks stuff, yes. Most packages don't cross compile at all. Debian has somewhere north of 30,000 packages. Every project that does large scale cross compiling (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends to have about 200 packages that cross compile more or less easily, another 400 or so that can be made to cross compile with _lot_ of effort and a large enough rock, and then the project stalls at about that size. The problem is: most embedded projects don't make really general-purpose fixes (instead strange things like hacking up autogenerated files), so they can't feed back to upstream. IMHO, a huge waste of working time. Amen, brother. I'm fortunate in that I work for an organization that is quite good about enforcing code reviews, specifically, the QA organization is empowered to reject changes that do not have code review notes. I also have a fairly broad scope, so I'm in on code reviews for a number of open source components. At each such review, one of my criteria is whether the change is suitable for pushing back to the appropriate community. This is not necessarily a short-term way to make friends, but the long-term effects will be good both for the company and for the open source community in general. Now, if we can only get the time to actually push all the backlogged fixes out... Er, is that GPL or LGPL code that you're modifying? If so, you *have* to push those code changes out (make them available to others), whether you think people will be interested or not! I guess I'm making a distinction that wasn't clear. We *have* to make the code available, and I can assure you that Cisco is very aware of our obligations in this area and I spend a fair amount of my time trying to ensure they are met. I used the term "push" to mean getting patches ready, posting them to the appropriate mailing lists, revising them in light of comments, and doing everything else necessary to get them incorporated into the kernel source base. "Pushing" is a lot more work than just making source available, but also yield much more productive long term results for everyone. -- David VomLehn, [EMAIL PROTECTED] The opinions expressed herein are likely mine, but might not be my employer's... - - - - - Cisco- - - - - This e-mail and any attachments may contain information which is confidential, proprietary, privileged or otherwise protected by law. The information is solely intended for the named addressee (or a person responsible for delivering it to the addressee). If you are not the intended recipient of this message, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete it from your computer. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
* Samuel Robb <[EMAIL PROTECTED]> schrieb: > When you have a one- or two-line fix, and face Yet Another round of > finding the right mailing list, identifying the right maintainers, > figuring out the right way to submit a bug and a patch, and then have to > spend the next 3 weeks explaining how no, you're not interested in being > the PPC maintainer for libfoo... is it any wonder that developers (not > to mention their management) eventually just gives up on the idea of > "giving back to the community?" ACK. > One possible solution would be to provide a clearing house for these > sorts of changes, maybe under the auspices of CELF or a similar > organization. Instead of submitting patches to individual projects, > submit them to the clearing house, and let interested individuals either > gather together and push related patches upstream in individual > projects, or give project maintainers a place to go and find embedded > systems patches related to their projects. See my last mail: that's exactly what the oss-qm project is for: http://oss-qm.metux.de/ With OSS-QM and related tools you often even don't need your own patching infrastructure - oss-qm provides complete patches against virtually any package/release. Perhaps a few words on the infrastructure: * single patches are collected per package, each vendor may get his own namespace/subdir, where he can feed in his patches. * the single patches are automatically pulled together based on "listfiles" - per package+release there is an simple text file which just lists the single patches to be pulled together. * each vendor may get it's own namespace for the listfiles. In other words: if you fear somebody else breaks your already tested/approved packages, just use your own (listfile) namespace. BTW: CSDB does a similar thing for retrieving source tarballs. Just query the DB, never ever care about individual URLs in you local build system. cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ - -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
* Jim Freeman <[EMAIL PROTECTED]> schrieb: > And I'm just betting that when he said "push ... fixes ... out" > he meant "work to get them incorporated back upstream", not just > make them available to requesters. Exactly. That's the essence of opensource development: Working together, instead of just taking someone else's work. BTW: everyone here should know that maintaining own branches (this actually happens if you have your own patches not fed back to upstream) is an quite work intensive and sometimes complex task. I doubt anyone here's really eager on doing that ;-P I understand that not everyone can talk with every project's upstream. Simply too much load. And not every project wants to merge in your patches asap (think of how long it took until expat team took in my really trivial $DESTDIR patch ;-o). That's why I founded the OSS-QM project: it a kind of "overlay" which provides fixes for a lot of packages in a strictly normalized namespace (so 100% automatic applying is easy). http://oss-qm.metux.de/ So, eg. if you've made some local changes, please at least feed them to us :) cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ - -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Thu, 2008-06-12 at 16:02 -0600, Jim Freeman wrote: > Most vendors these days have finally gotten the clue that sources/changes > have to be made available to downstream requesters, but far fewer > are sufficiently self-enlightened to figure out that changes need to > be accepted upstream for them to keep flowing back. And to make that > happen, vendors have to take on substantially higher overhead to win > acceptance of patches/changes upstream, an undertaking often sadly > fraught with hassle, uncertainty, and even peril. So they mostly > don't bother. To their (and their customers, and our) long-term > detriment. So - how do you reduce that overhead, then? Keep in mind that while pushing kernel changes upstream is significant, there are other projects as well. Most embedded developers are working not just with the kernel, but with a constellation of packages related to their projects. In order to "push changes upstream", they may end up having to work with several different communities... each with their own model of interaction, their own model of patch submission, and their own release schedules. Figuring out how to deal with just one community (kernel) doesn't help them in dealing with another (say, samba). When you have a one- or two-line fix, and face Yet Another round of finding the right mailing list, identifying the right maintainers, figuring out the right way to submit a bug and a patch, and then have to spend the next 3 weeks explaining how no, you're not interested in being the PPC maintainer for libfoo... is it any wonder that developers (not to mention their management) eventually just gives up on the idea of "giving back to the community?" One possible solution would be to provide a clearing house for these sorts of changes, maybe under the auspices of CELF or a similar organization. Instead of submitting patches to individual projects, submit them to the clearing house, and let interested individuals either gather together and push related patches upstream in individual projects, or give project maintainers a place to go and find embedded systems patches related to their projects. -Samrobb -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Daniel THOMPSON wrote: James Chapman wrote: Mike Frysinger wrote: On Thu, Jun 12, 2008 at 5:53 PM, Tim Bird wrote: Mike Frysinger wrote: Er, is that GPL or LGPL code that you're modifying? If so, you *have* to push those code changes out (make them available to others), whether you think people will be interested or not! umm, not really. only if (1) he gives a binary to someone and (2) they ask him for the source. if he doesnt distribute or no one asks, he doesnt have to do squat. This is closer to correct, but missing some important details. Start the GPL compliance tutorial/flameware in 3, 2, 1... yeah, i really dont think licensing things belong here. sorry for following up. how about this policy: if you want to make a statement, go pay a lawyer. but that statement still shouldnt be made here ;). -mike Sorry, I didn't mean to provoke a GPL flame war. The point I was trying to make (badly as it turns out) is that if a company really wants to see its changes taken upstream, it could simply publish the work on its website and let each relevant community know that it's there. Isn't this a lot of the problem with the way embedded companies and developers interact with upstream. In some cases it is in the embedded developers interests to see their code adopted upstream (i.e. so they don't have to maintain it). Totally agree! And the best chance of having code accepted upstream is to work with the community _while_ developing it, i.e. discussing the code during implementation, rather than presenting it to the community when it's done. All too often, companies get frustrated by feedback from the community because changes are requested to code that the original authors have spent time testing etc. Had early versions been submitted for feedback, changes could be made with less chance of wasted effort. Just tossing some code over the wall will, in almost all circumstances, result in the code being ignored. It depends. But it stands a better chance of being adopted than holding on to the work until someone asks for it. There could be lots of embedded developers out there who would be willing to take some code from cisco, modify it and work with the community to have it adopted upstream. -- James Chapman Katalix Systems Ltd http://www.katalix.com Catalysts for your Embedded Linux software development -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
James Chapman wrote: > Mike Frysinger wrote: >> On Thu, Jun 12, 2008 at 5:53 PM, Tim Bird wrote: >>> Mike Frysinger wrote: > Er, is that GPL or LGPL code that you're modifying? If so, you > *have* to > push those code changes out (make them available to others), > whether you > think people will be interested or not! umm, not really. only if (1) he gives a binary to someone and (2) they ask him for the source. if he doesnt distribute or no one asks, he doesnt have to do squat. >>> This is closer to correct, but missing some important details. >>> >>> Start the GPL compliance tutorial/flameware in 3, 2, 1... >> >> yeah, i really dont think licensing things belong here. sorry for >> following up. >> >> how about this policy: if you want to make a statement, go pay a >> lawyer. but that statement still shouldnt be made here ;). >> -mike > > Sorry, I didn't mean to provoke a GPL flame war. The point I was trying > to make (badly as it turns out) is that if a company really wants to see > its changes taken upstream, it could simply publish the work on its > website and let each relevant community know that it's there. Isn't this a lot of the problem with the way embedded companies and developers interact with upstream. In some cases it is in the embedded developers interests to see their code adopted upstream (i.e. so they don't have to maintain it). Just tossing some code over the wall will, in almost all circumstances, result in the code being ignored. > A diff of > the changes would be ideal. This is above and beyond what they have to > do under GPL terms of course. There's no need for a company to filter > out changes that it thinks others won't be interested in. This is a legal concern, not a practical one. All companies *should* comply with the law. Really it is more useful to convince people that it is in their own self-interest to do more than this. -- Daniel Thompson (STMicroelectronics) <[EMAIL PROTECTED]> 1000 Aztec West, Almondsbury, Bristol, BS32 4SQ. 01454 462659 If a car is a horseless carriage then is a motorcycle a horseless horse? -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Mike Frysinger wrote: On Thu, Jun 12, 2008 at 5:53 PM, Tim Bird wrote: Mike Frysinger wrote: Er, is that GPL or LGPL code that you're modifying? If so, you *have* to push those code changes out (make them available to others), whether you think people will be interested or not! umm, not really. only if (1) he gives a binary to someone and (2) they ask him for the source. if he doesnt distribute or no one asks, he doesnt have to do squat. This is closer to correct, but missing some important details. Start the GPL compliance tutorial/flameware in 3, 2, 1... yeah, i really dont think licensing things belong here. sorry for following up. how about this policy: if you want to make a statement, go pay a lawyer. but that statement still shouldnt be made here ;). -mike Sorry, I didn't mean to provoke a GPL flame war. The point I was trying to make (badly as it turns out) is that if a company really wants to see its changes taken upstream, it could simply publish the work on its website and let each relevant community know that it's there. A diff of the changes would be ideal. This is above and beyond what they have to do under GPL terms of course. There's no need for a company to filter out changes that it thinks others won't be interested in. -- James Chapman Katalix Systems Ltd http://www.katalix.com Catalysts for your Embedded Linux software development -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Thu, Jun 12, 2008 at 05:46:42PM -0400, Mike Frysinger wrote: > On Thu, Jun 12, 2008 at 5:42 PM, James Chapman wrote: > > David VomLehn wrote: > >> Amen, brother. I'm fortunate in that I work for an organization that is > >> quite good about enforcing code reviews, specifically, the QA organization > >> is empowered to reject changes that do not have code review notes. I also > >> have a fairly broad scope, so I'm in on code reviews for a number of open > >> source components. At each such review, one of my criteria is whether the > >> change is suitable for pushing back to the appropriate community. This is > >> not necessarily a short-term way to make friends, but the long-term effects > >> will be good both for the company and for the open source community in > >> general. > >> > >> Now, if we can only get the time to actually push all the backlogged fixes > >> out... > > > > Er, is that GPL or LGPL code that you're modifying? If so, you *have* to > > push those code changes out (make them available to others), whether you > > think people will be interested or not! > > umm, not really. only if (1) he gives a binary to someone and (2) > they ask him for the source. if he doesnt distribute or no one asks, > he doesnt have to do squat. > -mike And I'm just betting that when he said "push ... fixes ... out" he meant "work to get them incorporated back upstream", not just make them available to requesters. Most vendors these days have finally gotten the clue that sources/changes have to be made available to downstream requesters, but far fewer are sufficiently self-enlightened to figure out that changes need to be accepted upstream for them to keep flowing back. And to make that happen, vendors have to take on substantially higher overhead to win acceptance of patches/changes upstream, an undertaking often sadly fraught with hassle, uncertainty, and even peril. So they mostly don't bother. To their (and their customers, and our) long-term detriment. And Cisco has probably learned by now (and by sad experience) to do the Right (tm) thing. ...jfree -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Thu, Jun 12, 2008 at 5:53 PM, Tim Bird wrote: > Mike Frysinger wrote: >>> Er, is that GPL or LGPL code that you're modifying? If so, you *have* to >>> push those code changes out (make them available to others), whether you >>> think people will be interested or not! >> >> umm, not really. only if (1) he gives a binary to someone and (2) >> they ask him for the source. if he doesnt distribute or no one asks, >> he doesnt have to do squat. > > This is closer to correct, but missing some important details. > > Start the GPL compliance tutorial/flameware in 3, 2, 1... yeah, i really dont think licensing things belong here. sorry for following up. how about this policy: if you want to make a statement, go pay a lawyer. but that statement still shouldnt be made here ;). -mike -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Mike Frysinger wrote: >> Er, is that GPL or LGPL code that you're modifying? If so, you *have* to >> push those code changes out (make them available to others), whether you >> think people will be interested or not! > > umm, not really. only if (1) he gives a binary to someone and (2) > they ask him for the source. if he doesnt distribute or no one asks, > he doesnt have to do squat. This is closer to correct, but missing some important details. Start the GPL compliance tutorial/flameware in 3, 2, 1... Luckily this in only on the linux-embedded list. If it were on LKML the fun would really begin. :-) -- Tim = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Thu, Jun 12, 2008 at 5:42 PM, James Chapman wrote: > David VomLehn wrote: >> Amen, brother. I'm fortunate in that I work for an organization that is >> quite good about enforcing code reviews, specifically, the QA organization >> is empowered to reject changes that do not have code review notes. I also >> have a fairly broad scope, so I'm in on code reviews for a number of open >> source components. At each such review, one of my criteria is whether the >> change is suitable for pushing back to the appropriate community. This is >> not necessarily a short-term way to make friends, but the long-term effects >> will be good both for the company and for the open source community in >> general. >> >> Now, if we can only get the time to actually push all the backlogged fixes >> out... > > Er, is that GPL or LGPL code that you're modifying? If so, you *have* to > push those code changes out (make them available to others), whether you > think people will be interested or not! umm, not really. only if (1) he gives a binary to someone and (2) they ask him for the source. if he doesnt distribute or no one asks, he doesnt have to do squat. -mike -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
David VomLehn wrote: Enrico Weigelt wrote: * Rob Landley <[EMAIL PROTECTED]> schrieb: Cross compiling breaks stuff, yes. Most packages don't cross compile at all. Debian has somewhere north of 30,000 packages. Every project that does large scale cross compiling (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends to have about 200 packages that cross compile more or less easily, another 400 or so that can be made to cross compile with _lot_ of effort and a large enough rock, and then the project stalls at about that size. The problem is: most embedded projects don't make really general-purpose fixes (instead strange things like hacking up autogenerated files), so they can't feed back to upstream. IMHO, a huge waste of working time. Amen, brother. I'm fortunate in that I work for an organization that is quite good about enforcing code reviews, specifically, the QA organization is empowered to reject changes that do not have code review notes. I also have a fairly broad scope, so I'm in on code reviews for a number of open source components. At each such review, one of my criteria is whether the change is suitable for pushing back to the appropriate community. This is not necessarily a short-term way to make friends, but the long-term effects will be good both for the company and for the open source community in general. Now, if we can only get the time to actually push all the backlogged fixes out... Er, is that GPL or LGPL code that you're modifying? If so, you *have* to push those code changes out (make them available to others), whether you think people will be interested or not! -- David VomLehn, [EMAIL PROTECTED] The opinions expressed herein are likely mine, but might not be my employer's... -- James Chapman Katalix Systems Ltd http://www.katalix.com Catalysts for your Embedded Linux software development -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Enrico Weigelt wrote: * Rob Landley <[EMAIL PROTECTED]> schrieb: Cross compiling breaks stuff, yes. Most packages don't cross compile at all. Debian has somewhere north of 30,000 packages. Every project that does large scale cross compiling (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends to have about 200 packages that cross compile more or less easily, another 400 or so that can be made to cross compile with _lot_ of effort and a large enough rock, and then the project stalls at about that size. The problem is: most embedded projects don't make really general-purpose fixes (instead strange things like hacking up autogenerated files), so they can't feed back to upstream. IMHO, a huge waste of working time. Amen, brother. I'm fortunate in that I work for an organization that is quite good about enforcing code reviews, specifically, the QA organization is empowered to reject changes that do not have code review notes. I also have a fairly broad scope, so I'm in on code reviews for a number of open source components. At each such review, one of my criteria is whether the change is suitable for pushing back to the appropriate community. This is not necessarily a short-term way to make friends, but the long-term effects will be good both for the company and for the open source community in general. Now, if we can only get the time to actually push all the backlogged fixes out... -- David VomLehn, [EMAIL PROTECTED] The opinions expressed herein are likely mine, but might not be my employer's... -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
* Wolfgang Denk <[EMAIL PROTECTED]> schrieb: > In message <[EMAIL PROTECTED]> you wrote: > > > > Perl never was crosscompile-capable. > > I've rewrote much of the buildscripts, but not finished yet. > > Note that Perl is included with our ELDK, and we do cross-compile it. > It's not exactly trivial, but not really difficult either. URL ? cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ - -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
In message <[EMAIL PROTECTED]> you wrote: > > Perl never was crosscompile-capable. > I've rewrote much of the buildscripts, but not finished yet. Note that Perl is included with our ELDK, and we do cross-compile it. It's not exactly trivial, but not really difficult either. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] I have a theory that it's impossible to prove anything, but I can't prove it. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
* Rob Landley <[EMAIL PROTECTED]> schrieb: > Cross compiling breaks stuff, yes. > > Most packages don't cross compile at all. Debian has somewhere north of > 30,000 packages. Every project that does large scale cross compiling > (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends > to have about 200 packages that cross compile more or less easily, another > 400 or so that can be made to cross compile with _lot_ of effort and a large > enough rock, and then the project stalls at about that size. The problem is: most embedded projects don't make really general-purpose fixes (instead strange things like hacking up autogenerated files), so they can't feed back to upstream. IMHO, a huge waste of working time. Did someone ever hear of the OSS-QM project ? > Some of the other build systems out there hook qemu application emulation up > to the kernel's misc binary support so a ./configure that builds arm > executables can run them. WTF ?! cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ - -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
* Rob Landley <[EMAIL PROTECTED]> schrieb: Hi, > There's also qemu. You can native build under emulation. did you ever consider that crosscompiling is not only good for some other arch, but a few more things ? > In addition, if you have a cross compiler but don't want to spend all your > time lying to ./configure, preventing gcc from linking against the host's > zlib or grabbing stuff out of /usr/include that your target hasn't got, or #1: use a proper (sysroot'ed) toolchain #2: fix broken configure.in's (and feed back to upstream or OSS-QM) #3: replace libtool by unitool > trying to figure out why on EARTH the perl build decided to use x86 signal Perl never was crosscompile-capable. I've rewrote much of the buildscripts, but not finished yet. Just in case that anyone's interested in it, let me know. > libraries are linked inside the emulator, anything that wants to look > at /proc or sysinfo does it natively inside the emulator...) Only crap sw looks at /proc at build time. Yes, there's *much* crap sw out there :( cu -- - Enrico Weigelt== metux IT service - http://www.metux.de/ - Please visit the OpenSource QM Taskforce: http://wiki.metux.de/public/OpenSource_QM_Taskforce Patches / Fixes for a lot dozens of packages in dozens of versions: http://patches.metux.de/ - -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Rob Landley wrote: > Most packages don't cross compile at all. Debian has somewhere north of > 30,000 packages. Every project that does large scale cross compiling > (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends > to have about 200 packages that cross compile more or less easily, another > 400 or so that can be made to cross compile with _lot_ of effort and a large > enough rock, and then the project stalls at about that size. +1. I spent several months fixing up cross-compile issues on Gentoo Embedded a few years ago, for a specific application - only a small subset of packages needed. The majority of packages I needed failed to compile out of the box, one way or another - Glibc dependencies, arch dependencies, scripts which depend on the host environment, or invoke the host compiler, or the host Perl (and then depend on it's byte order). etc. More recently, I've been compiling a build which is _intended_ for cross-compilation - it's an old uClinux kit, patched by a third party. Even that fails to build on newer GNU/Linuxes, as the syntax of GNU Make has changed, and Bash has changed. Also GCC 2.old doesn't compile on current GNU/Linux with GCC 4.new. Fortunately the latter were a few small, easy to fix issues. But I understand now why some find it important to have a replicatable build environment when you need to get an old distribution out of the closet to update firmware for some 5 year old device. Virtual machines ought to be great for that. They are. But even those are surprisingly changable - images that worked on a VM a few years ago no longer work on the current version of the VM host. -- Jamie -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Wednesday 11 June 2008 00:47:36 Greg Ungerer wrote: > Hi Rob, > > Rob Landley wrote: > > On Tuesday 10 June 2008 02:54:32 Sam Ravnborg wrote: > >>> (Maybe I _am_ the only person who still cares about > >>> building on a host without perl. If I wasn't, somebody else would have > >>> acked the patch...) > >> > >> perl is pretty standard > > > > An implementation is not the same thing as a standard. If you mean > > "there is one implementation everybody uses, ala excel and Word, and even > > the Perl guys can't reproduce it from scratch as parrot showed", then > > you're using a different definition of the word "standard" than I am. > > > > Or do you mean it comes preinstalled on most modern systems, the way > > Windows does, and who could object to that? > > > > I know from experience that it's an _amazing_ pain to try to cross > > compile the sucker... > > Ain't that the truth! > > >> and I fail to see the benefits of avoiding it. > >> For embedded development I see even less benefits as I assume > >> any sane embedded development environment are based on a > >> cross-toolchain so you do the build on a high perfomance box. > >> > >> Building everything for my arm board on the arm board would be a disater > >> for example. > > > > I build everything for my arm board natively, on an arm system running > > under qemu, calling out to the cross compiler via distcc to accelerate > > the limited parts of the process that cross compiling doesn't actually > > break. To get to that point, I cross compile just enough to get a native > > development environment, and then I avoid cross compiling from then on > > because it's an enormous source of complexity and random breakage. > > Random breakage? Cross compiling breaks stuff, yes. Most packages don't cross compile at all. Debian has somewhere north of 30,000 packages. Every project that does large scale cross compiling (buildroot, gentoo embedded, timesys making fedora cross compile, etc) tends to have about 200 packages that cross compile more or less easily, another 400 or so that can be made to cross compile with _lot_ of effort and a large enough rock, and then the project stalls at about that size. > > I did this because throwing hardware at the problem is cheaper than > > throwing engineering time at the problem, because Moore's Law is on my > > side, and > > Are you sure about that? > How well does qemu do SMP? At all? QEMU does not currently do SMP at all. There are proposals to make qemu multi-threaded and handle SMP that way, but they're at least a year away from anybody actually trying to implement them. (The big destabilization right now is the new code generator, and that's plenty for the moment. Anybody who wanted to wave money at codesourcery could probably get it implemented on a deadline for their platforms of interest, but in the absence of that it's not really a priority right now.) Distcc can take advantage of smp, but that won't help the ./configure stage and I need to do some work on distcc to teach it to understand more gcc command lines. (For one thing, distcc can't break "compile and link" commands ala "gcc hello.c" into separate "compile" and "link" stages, this it can't distribute those. This takes out pretty much the entire uClibc build, for example. But the gcc build parallelizes quite well.) > Modern x86 and friends are getting most new performance from > more cores. A cross compile today can take advantage of those > for the most part. Your emulated system probably can't. It can if you're using distcc to call out to the cross compiler, which my scripts do (./emulator-build.sh does it with the build/* stuff and ./run-with-distcc.sh does it for the shipped system-image tarballs). Some of the other build systems out there hook qemu application emulation up to the kernel's misc binary support so a ./configure that builds arm executables can run them. (Openembedded, was it? Open moko? Something like that.) But this only solves one of about a dozen major problems cross compiling is prone to. Building natively solves all of 'em, except for speed. > An order of magnitude compile time (or more) for a native build > is quite a high price to pay :-) Yup. Agreed. No argument there. However, having one or more full-time engineers devoted to debugging cross-compile issues is quite a high price to pay too. Moore's law really doesn't help that one. I'm not saying either solution is perfect, I'm just saying the "build under emulation" approach is a viable alternative that gets more attractive as time passes, both because of ongoing development on emulators and because of Moore's law on the hardware. Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Hi Rob, Rob Landley wrote: On Tuesday 10 June 2008 02:54:32 Sam Ravnborg wrote: (Maybe I _am_ the only person who still cares about building on a host without perl. If I wasn't, somebody else would have acked the patch...) perl is pretty standard An implementation is not the same thing as a standard. If you mean "there is one implementation everybody uses, ala excel and Word, and even the Perl guys can't reproduce it from scratch as parrot showed", then you're using a different definition of the word "standard" than I am. Or do you mean it comes preinstalled on most modern systems, the way Windows does, and who could object to that? I know from experience that it's an _amazing_ pain to try to cross compile the sucker... Ain't that the truth! and I fail to see the benefits of avoiding it. For embedded development I see even less benefits as I assume any sane embedded development environment are based on a cross-toolchain so you do the build on a high perfomance box. Building everything for my arm board on the arm board would be a disater for example. I build everything for my arm board natively, on an arm system running under qemu, calling out to the cross compiler via distcc to accelerate the limited parts of the process that cross compiling doesn't actually break. To get to that point, I cross compile just enough to get a native development environment, and then I avoid cross compiling from then on because it's an enormous source of complexity and random breakage. Random breakage? I did this because throwing hardware at the problem is cheaper than throwing engineering time at the problem, because Moore's Law is on my side, and Are you sure about that? How well does qemu do SMP? At all? Modern x86 and friends are getting most new performance from more cores. A cross compile today can take advantage of those for the most part. Your emulated system probably can't. An order of magnitude compile time (or more) for a native build is quite a high price to pay :-) Regards Greg because I find native compiling (where possible) more straightforward and conceptually cleaner. Before 2.6.25, I had a complete self-bootstrapping environment with seven packages (busybox, uClibc, make, gcc, binutils, bash, and linux). Building glibc needs perl, if you're using a uClibc based system you may _never_ need Perl, and could go up through Linux From Scratch and beyond to a fairly powerful system. I can probably natively build perl under the mini-native environment running inside qemu, but it wasn't needed before and there really wasn't a good reason to add it. The task being performed and the build dependency brought in to perform it are hugely disproportionate. Rob -- Greg Ungerer -- Chief Software Dude EMAIL: [EMAIL PROTECTED] Secure Computing CorporationPHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tuesday 10 June 2008 08:47:20 Will Newton wrote: > On Tue, Jun 10, 2008 at 2:33 PM, David Woodhouse <[EMAIL PROTECTED]> wrote: > > On Tue, 2008-06-10 at 14:25 +0100, Will Newton wrote: > >> On Tue, Jun 10, 2008 at 2:12 PM, Jamie Lokier <[EMAIL PROTECTED]> wrote: > >> > Wolfgang Denk wrote: > >> >> Being unable to do this just because we now also would need a native > >> >> Perl is indeed a PITA... > >> > > >> > You can run the Perl bit with "ssh remote perl", and still do the rest > >> > of the compile natively. It's not pretty, but workable. > >> > >> I'm not convinced it matters at all. Self hosting on an embedded > >> architecture is, as has been mentioned, pretty pointless. > >> > >> Using a kernel compile as a test isn't such a great idea. Stress tests > >> of that kind are not particularly useful for pinning down bugs - so > >> your kernel compile failed, what now? Far better to use LTP tests or > >> similar that are designed to be reproduceable and tunable for your > >> system. For example I don't think I'll ever be able to self host a > >> kernel build on a board with only 32Mb of on-board RAM. > > > > Actually, cross-building on NFS does tend to find a _lot_ of issues > > which crop up with board ports; especially PCI arbitration, DMA > > coherency, cache and MMU issues. LTP often doesn't catch the same > > problems. > > It may trigger a number of bugs, I don't disagree, but as a test it is > a blunt instrument. It's likely to be hard to reproduce and have an > inconsistent failure mode. If you're really serious about testing it's > not the best solution. It's like using gcc instead of memtest86 to > test your memory. Eventually it might go wrong but you won't be much > the wiser about why, or have any way to trim your testcase down so you > can run it on an in-circuit emulator or pass it to your silicon > vendor. > > > I agree that it's not so easy on a board with 32Mb of RAM, since that's > > only 4,000,000 bytes -- but 32MiB ought to be _perfectly_ sufficient :) > > I would be surprised if it was possible to compile Linux with gcc 4.2 > with 32MiB of total system memory. Haven't tried, but I generally run emulated builds in 128 megs of ram (on 32 bit hosts), and I use this: # This tells gcc to aggressively garbage collect its internal data # structures. Without this, gcc triggers the OOM killer trying to rebuild # itself in 128 megs of ram, which is the QEMU default size. export CFLAGS="--param ggc-min-expand=0 --param ggc-min-heapsize=8192" Don't do that on a 64 bit host or your build will slow to a crawl, of course. Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, Jun 10, 2008 at 12:50:34PM +0200, Sam Ravnborg wrote: > > > > When did this policy change, so that it's now acceptable to depend on > > Perl, which is roughly equivalent as a tool dependency? > > We have perl as a mandatory part of the kernel build in several places > for various architectures. > And I do not recall anyone submitting a bug that they could not build > a kernel due to the perl dependency. > But I am obviously well aware of that we use it for the time stuff. > And plenty of places in scripts/ have dependencies on either perl or python already (and have for some time). Both are pretty ubiquitous these days, whether people like it or not. There's not much point in trying to keep the build limping along for people who don't want to set up these tools on their platforms, since they're going to lose out on half of the functionality anyways (checkstack, bloat-o-meter, etc.). Building natively is a good stress tester, and does find a lot of bugs. For that same reason, if you can build the kernel, you can build python and perl natively on your platform, too. Some of us have already been doing this for ages and have no idea what the fuss is about. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tuesday 10 June 2008 08:49:32 Wolfgang Denk wrote: > In message <[EMAIL PROTECTED]> you wrote: > > I'm not convinced it matters at all. Self hosting on an embedded > > architecture is, as has been mentioned, pretty pointless. > > YMMV... > > > system. For example I don't think I'll ever be able to self host a > > kernel build on a board with only 32Mb of on-board RAM. > > It depends. We've done this on board with as little as 16 MB RAM. There's also qemu. You can native build under emulation. In addition, if you have a cross compiler but don't want to spend all your time lying to ./configure, preventing gcc from linking against the host's zlib or grabbing stuff out of /usr/include that your target hasn't got, or trying to figure out why on EARTH the perl build decided to use x86 signal numbers when you built it for mips, you can build natively inside the emulator but use distcc to call out to the cross compiler through the virtual network. This uses the compiler for the heavy lifting of compilation _and_nothing_else_. (make runs natively inside the emulator, ./configure runs inside the emulator, headers are #included inside the emulator, libraries are linked inside the emulator, anything that wants to look at /proc or sysinfo does it natively inside the emulator...) And the thing is, QEMU is running on fast cheap x86 hardware with buckets of memory and disk space, and Moore's Law is doubling the power of it every 18 months (whereas it's making a lot of embedded stuff cheaper and have longer battery life instead, at least until we get to fully disposable computers). So yay using x86 as your build envionment, but building natively under emulation is now an alternative to trying to make cross compiling scale. Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tuesday 10 June 2008 02:54:32 Sam Ravnborg wrote: > > (Maybe I _am_ the only person who still cares about > > building on a host without perl. If I wasn't, somebody else would have > > acked the patch...) > > perl is pretty standard An implementation is not the same thing as a standard. If you mean "there is one implementation everybody uses, ala excel and Word, and even the Perl guys can't reproduce it from scratch as parrot showed", then you're using a different definition of the word "standard" than I am. Or do you mean it comes preinstalled on most modern systems, the way Windows does, and who could object to that? I know from experience that it's an _amazing_ pain to try to cross compile the sucker... > and I fail to see the benefits of avoiding it. > For embedded development I see even less benefits as I assume > any sane embedded development environment are based on a > cross-toolchain so you do the build on a high perfomance box. > > Building everything for my arm board on the arm board would be a disater > for example. I build everything for my arm board natively, on an arm system running under qemu, calling out to the cross compiler via distcc to accelerate the limited parts of the process that cross compiling doesn't actually break. To get to that point, I cross compile just enough to get a native development environment, and then I avoid cross compiling from then on because it's an enormous source of complexity and random breakage. I did this because throwing hardware at the problem is cheaper than throwing engineering time at the problem, because Moore's Law is on my side, and because I find native compiling (where possible) more straightforward and conceptually cleaner. Before 2.6.25, I had a complete self-bootstrapping environment with seven packages (busybox, uClibc, make, gcc, binutils, bash, and linux). Building glibc needs perl, if you're using a uClibc based system you may _never_ need Perl, and could go up through Linux From Scratch and beyond to a fairly powerful system. I can probably natively build perl under the mini-native environment running inside qemu, but it wasn't needed before and there really wasn't a good reason to add it. The task being performed and the build dependency brought in to perform it are hugely disproportionate. Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Sam Ravnborg wrote: >> (Maybe I _am_ the only person who still cares about >> building on a host without perl. If I wasn't, somebody else would have >> acked >> the patch...) > > perl is pretty standard and I fail to see the benefits of avoiding it. > For embedded development I see even less benefits as I assume > any sane embedded development environment are based on a > cross-toolchain so you do the build on a high perfomance box. > > Building everything for my arm board on the arm board would be a disater > for example. It is true that perl is pervasive and standard, in most scenarios. However, it is not uncommon in the embedded world for people to strive to containerize the build process for a whole distro (kernel included) in order to reduce problems with configure scripts that (incorrectly) probe the host environment. I know of at least 3 distros or distro-building methods that do this. Working to reduce the complexity of these constrained environments is a valid goal. My own reasons for wanting to avoid using perl, however, aren't based on its availability. I don't want to start a language war, but in general I think adding "knowledge of perl" to the list of things someone needs to understand in order to understand how to build the kernel, would be a bad thing. There's already a huge mountain to climb for someone trying to learn enough to modify the kernel (or it's build system). Adding perl doesn't IMHO contribute enough value to warrant this extension in required knowledge. Also (again IMHO), perl is harder to maintain than other alternatives, if a full-service scripting language is really needed. I don't care so much if perl is used in some non-essential role, like python is used for bloat-o-meter, for example. But it would be nice, IMHO, to minimize the external dependencies for required kernel build infrastructure. Having said all this, I can't object too strenuously, since I have some code, out of tree, that uses perl as a pre-processor. One of the reasons I haven't submitted it is that I find its use of perl somewhat distasteful, for the reasons mentioned above. -- Tim = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Wolfgang Denk wrote: > In message <[EMAIL PROTECTED]> you wrote: > > > > I would be surprised if it was possible to compile Linux with gcc 4.2 > > with 32MiB of total system memory. > > Hint: if memory really gets tight, you can use swap space. Either to > a local drive (either through PCI or PCMCIA/PCCard or USB or FW or > ...), or over the network. This just adds another level of stress > testing to areas in the kernel that are not so well covered by some > other tests. Great, I'm looking forward to your implementation of swap on no-MMU! Thanks, -- Jamie -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, Jun 10, 2008 at 7:53 AM, David Woodhouse <[EMAIL PROTECTED]> wrote: > On Tue, 2008-06-10 at 14:47 +0100, Will Newton wrote: >> On Tue, Jun 10, 2008 at 2:33 PM, David Woodhouse <[EMAIL PROTECTED]> wrote: >> > On Tue, 2008-06-10 at 14:25 +0100, Will Newton wrote: >> >> Using a kernel compile as a test isn't such a great idea. Stress tests >> >> of that kind are not particularly useful for pinning down bugs - so >> >> your kernel compile failed, what now? Far better to use LTP tests or >> >> similar that are designed to be reproduceable and tunable for your >> >> system. For example I don't think I'll ever be able to self host a >> >> kernel build on a board with only 32Mb of on-board RAM. >> > >> > Actually, cross-building on NFS does tend to find a _lot_ of issues >> > which crop up with board ports; especially PCI arbitration, DMA >> > coherency, cache and MMU issues. LTP often doesn't catch the same >> > problems. >> >> It may trigger a number of bugs, I don't disagree, but as a test it is >> a blunt instrument. > > Yes, it's a blunt instrument, but blunt instruments are often effective. > > I disagree with your claim that using it as a test isn't a good idea. > I would, however, grant you that using it as your _only_ test is a bad > idea :) Just to add my voice to the chorus; I fully agree. Brute force testing is useful. It can expose corner cases that haven't been considered in formal test suites. Cheers, g. -- Grant Likely, B.Sc., P.Eng. Secret Lab Technologies Ltd. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
In message <[EMAIL PROTECTED]> you wrote: > > I would be surprised if it was possible to compile Linux with gcc 4.2 > with 32MiB of total system memory. Hint: if memory really gets tight, you can use swap space. Either to a local drive (either through PCI or PCMCIA/PCCard or USB or FW or ...), or over the network. This just adds another level of stress testing to areas in the kernel that are not so well covered by some other tests. Guess who found out that swap on MPC8xx was broken for a long, long time, and how it was detected? Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] You Don't Have To Be 'Damned' To Work Here, But It Helps!!! - Terry Pratchett, _Eric_ -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, 2008-06-10 at 14:47 +0100, Will Newton wrote: > On Tue, Jun 10, 2008 at 2:33 PM, David Woodhouse <[EMAIL PROTECTED]> wrote: > > On Tue, 2008-06-10 at 14:25 +0100, Will Newton wrote: > >> Using a kernel compile as a test isn't such a great idea. Stress tests > >> of that kind are not particularly useful for pinning down bugs - so > >> your kernel compile failed, what now? Far better to use LTP tests or > >> similar that are designed to be reproduceable and tunable for your > >> system. For example I don't think I'll ever be able to self host a > >> kernel build on a board with only 32Mb of on-board RAM. > > > > Actually, cross-building on NFS does tend to find a _lot_ of issues > > which crop up with board ports; especially PCI arbitration, DMA > > coherency, cache and MMU issues. LTP often doesn't catch the same > > problems. > > It may trigger a number of bugs, I don't disagree, but as a test it is > a blunt instrument. Yes, it's a blunt instrument, but blunt instruments are often effective. I disagree with your claim that using it as a test isn't a good idea. I would, however, grant you that using it as your _only_ test is a bad idea :) -- dwmw2 -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
In message <[EMAIL PROTECTED]> you wrote: > > I'm not convinced it matters at all. Self hosting on an embedded > architecture is, as has been mentioned, pretty pointless. YMMV... > system. For example I don't think I'll ever be able to self host a > kernel build on a board with only 32Mb of on-board RAM. It depends. We've done this on board with as little as 16 MB RAM. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] "Confound these ancestors They've stolen our best ideas!" - Ben Jonson -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, Jun 10, 2008 at 2:33 PM, David Woodhouse <[EMAIL PROTECTED]> wrote: > On Tue, 2008-06-10 at 14:25 +0100, Will Newton wrote: >> On Tue, Jun 10, 2008 at 2:12 PM, Jamie Lokier <[EMAIL PROTECTED]> wrote: >> > Wolfgang Denk wrote: >> >> Being unable to do this just because we now also would need a native >> >> Perl is indeed a PITA... >> > >> > You can run the Perl bit with "ssh remote perl", and still do the rest >> > of the compile natively. It's not pretty, but workable. >> >> I'm not convinced it matters at all. Self hosting on an embedded >> architecture is, as has been mentioned, pretty pointless. >> >> Using a kernel compile as a test isn't such a great idea. Stress tests >> of that kind are not particularly useful for pinning down bugs - so >> your kernel compile failed, what now? Far better to use LTP tests or >> similar that are designed to be reproduceable and tunable for your >> system. For example I don't think I'll ever be able to self host a >> kernel build on a board with only 32Mb of on-board RAM. > > Actually, cross-building on NFS does tend to find a _lot_ of issues > which crop up with board ports; especially PCI arbitration, DMA > coherency, cache and MMU issues. LTP often doesn't catch the same > problems. It may trigger a number of bugs, I don't disagree, but as a test it is a blunt instrument. It's likely to be hard to reproduce and have an inconsistent failure mode. If you're really serious about testing it's not the best solution. It's like using gcc instead of memtest86 to test your memory. Eventually it might go wrong but you won't be much the wiser about why, or have any way to trim your testcase down so you can run it on an in-circuit emulator or pass it to your silicon vendor. > I agree that it's not so easy on a board with 32Mb of RAM, since that's > only 4,000,000 bytes -- but 32MiB ought to be _perfectly_ sufficient :) I would be surprised if it was possible to compile Linux with gcc 4.2 with 32MiB of total system memory. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
In message <[EMAIL PROTECTED]> you wrote: > > You can run the Perl bit with "ssh remote perl", and still do the rest > of the compile natively. It's not pretty, but workable. This may or may not work. If, for example, perl is running on a remote little endian host (like a standard x86 PC) and we depend on native bin endian byte order, there may be trouble... Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] Quote from a recent meeting: "We are going to continue having these meetings everyday until I find out why no work is getting done." -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, 2008-06-10 at 14:25 +0100, Will Newton wrote: > On Tue, Jun 10, 2008 at 2:12 PM, Jamie Lokier <[EMAIL PROTECTED]> wrote: > > Wolfgang Denk wrote: > >> Being unable to do this just because we now also would need a native > >> Perl is indeed a PITA... > > > > You can run the Perl bit with "ssh remote perl", and still do the rest > > of the compile natively. It's not pretty, but workable. > > I'm not convinced it matters at all. Self hosting on an embedded > architecture is, as has been mentioned, pretty pointless. > > Using a kernel compile as a test isn't such a great idea. Stress tests > of that kind are not particularly useful for pinning down bugs - so > your kernel compile failed, what now? Far better to use LTP tests or > similar that are designed to be reproduceable and tunable for your > system. For example I don't think I'll ever be able to self host a > kernel build on a board with only 32Mb of on-board RAM. Actually, cross-building on NFS does tend to find a _lot_ of issues which crop up with board ports; especially PCI arbitration, DMA coherency, cache and MMU issues. LTP often doesn't catch the same problems. I agree that it's not so easy on a board with 32Mb of RAM, since that's only 4,000,000 bytes -- but 32MiB ought to be _perfectly_ sufficient :) -- dwmw2 -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, Jun 10, 2008 at 2:12 PM, Jamie Lokier <[EMAIL PROTECTED]> wrote: > Wolfgang Denk wrote: >> Being unable to do this just because we now also would need a native >> Perl is indeed a PITA... > > You can run the Perl bit with "ssh remote perl", and still do the rest > of the compile natively. It's not pretty, but workable. I'm not convinced it matters at all. Self hosting on an embedded architecture is, as has been mentioned, pretty pointless. Using a kernel compile as a test isn't such a great idea. Stress tests of that kind are not particularly useful for pinning down bugs - so your kernel compile failed, what now? Far better to use LTP tests or similar that are designed to be reproduceable and tunable for your system. For example I don't think I'll ever be able to self host a kernel build on a board with only 32Mb of on-board RAM. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Wolfgang Denk wrote: > Being unable to do this just because we now also would need a native > Perl is indeed a PITA... You can run the Perl bit with "ssh remote perl", and still do the rest of the compile natively. It's not pretty, but workable. -- Jamie -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
> > When did this policy change, so that it's now acceptable to depend on > Perl, which is roughly equivalent as a tool dependency? We have perl as a mandatory part of the kernel build in several places for various architectures. And I do not recall anyone submitting a bug that they could not build a kernel due to the perl dependency. But I am obviously well aware of that we use it for the time stuff. For the headers_* targets I will shortly introduce yet another perl dependency - but only if these targets are used - so less of an issue. o We shall try to keep the dependencies low for the common things o but for the more exoctic things a wides dependency is OK. Which is also why I'm happy to apply a patch that remove the mandatory dependency of perl we have today - if and only if that patch meet normal patch acceptance criterias. Rob's initial patch had some issues and neither Rob nor I have fixed these and therefore it has not been applied. Rob seems to put much more into this (private reply accustions etc) for reasons unknown to me. And doing so does not help to get me interested. So try to get the facts correct here - there is noone against removing the mandatory perl dependency. But it is lower on my priority list than many other things which explain why I do not do it myself. But if someone submit a patch to do so then if the patch is OK it will be applied. And if we have a policy that say no-go to perl then it is new to me. I hope one day to rewrite part of kbuild and perl seems to be the best candidate around. But that day may never come. Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Tue, Jun 10, 2008 at 11:20:18AM +0100, Jamie Lokier wrote: > My only comment is I remember when Eric Raymond submitted a smart > config thing (before Kconfig, which copied Eric's best ideas). > > The main objection to Eric's patch was that it was written in Python, > causing kernel builds to depend Python. >... Although the usage of Python was brought as a reason against it the main objection against CML2 was ESRs interaction with the kernel community, like the fact that he was not willing to split his changes into appropriate pieces but only offered one big patch. > -- Jamie cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
My only comment is I remember when Eric Raymond submitted a smart config thing (before Kconfig, which copied Eric's best ideas). The main objection to Eric's patch was that it was written in Python, causing kernel builds to depend Python. When did this policy change, so that it's now acceptable to depend on Perl, which is roughly equivalent as a tool dependency? -- Jamie -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
In message <[EMAIL PROTECTED]> you wrote: > > (Maybe I _am_ the only person who still cares about > > building on a host without perl. If I wasn't, somebody else would have > > acked > > the patch...) > > perl is pretty standard and I fail to see the benefits of avoiding it. > For embedded development I see even less benefits as I assume > any sane embedded development environment are based on a > cross-toolchain so you do the build on a high perfomance box. > > Building everything for my arm board on the arm board would be a disater > for example. Well, compiling the Linux kernel on the native system with the root file system mounted over NFS has always been a really good regression test for us. It exercises a *lot* of kernel code - tasks, memory, network, ... Being unable to do this just because we now also would need a native Perl is indeed a PITA... Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [EMAIL PROTECTED] You don't have to worry about me. I might have been born yesterday... but I stayed up all night. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
> (Maybe I _am_ the only person who still cares about > building on a host without perl. If I wasn't, somebody else would have acked > the patch...) perl is pretty standard and I fail to see the benefits of avoiding it. For embedded development I see even less benefits as I assume any sane embedded development environment are based on a cross-toolchain so you do the build on a high perfomance box. Building everything for my arm board on the arm board would be a disater for example. Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Monday 09 June 2008 23:30:20 Sam Ravnborg wrote: > Rob - please reread the mails. It was your broken mailer or something > that caused it to be off-list. > When I hit reply-to-all in mutt I expect it to be sent to the list - > and in your case it did not not hit the list. Could easily be the case. Kmail's gotten increasingly weird ever since Kontact ate it, and I do tend to break stuff anyway... > > A number of embedded people emailed me about it off list, but nobody ever > > replied to my post _on_ the list to say I wasn't alone in this, or acked > > the patch to say it worked for them, or anything like that. So there was > > a perception of zero support, and I gave up trying to follow up on it for > > 2.6.25. I've got it working for me, and if more perl shows up I'll come > > up with more patches to remove it for my own personal build environment. > > > > I might try to submit again in a few months > > Then please try to address the comments first. The comments I remember (it's been a while) were that it was futile to fight against the addition of perl because A) nobody else was bothered by it, B) plans were in place to add more perl to the build in future so it was only a matter of time anyway, C) some out of tree patch might add an architecture variant that wanted to define arbitrary clock values. My response to C was "I can't find this patch, show me", my response to B) was "I'll address that when it happens", and my response to A) was "yeah, everybody else who seemed to care about this has decided to shut up and bog off, haven't they?" I keep meaning to write a much improved patch that records a comment at the start of the .h file with the complete command line the perl thing was run as, so re-running the header generator is a question of cut and paste and then adding the extra value you want before hitting enter. And then ripping all the logic out of the makefile that tries to rebuild the thing itself because only a patch to Kconfig files can ever require a new value, and that patch should change the .h file so it can build the result. If they forget to check in the .h file, the build should fail noisily. The result is simpler, has many fewer lines, and doesn't need perl to compile (just to add new clock values during development), but since nobody else in the embedded community cared to speak up for it, I kind of lost interest in trying to get it merged. (Maybe I _am_ the only person who still cares about building on a host without perl. If I wasn't, somebody else would have acked the patch...) *shrug* If I go back to revisit this issue I'll dig up the old thread and reread it, but at the moment I have no immediate plans to. Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Mon, Jun 09, 2008 at 10:53:11PM -0500, Rob Landley wrote: > On Monday 09 June 2008 16:27:29 Leon Woestenberg wrote: > > > I submitted a patch to remove the use of perl to build the linux kernel > > > (which HPA added in 2.6.25) not because it affected the result, but > > > because it unnecessarily complicates the build system. (And perl tends > > > to metasticize. > > > > Thanks, I hope it is or gets accepted. > > Nope, Peter Anvin shot it down (saying what I was doing was a strange, purely > academic exercise), and Sam Ravnborg announced his vague intention to rewrite > large chunks of kbuild in perl in the future. (Because nothing > says "maintainability" like perl...) > > For some reason they wanted to mail me about it off-list rather than cc:ing > the list about this plan. Rob - please reread the mails. It was your broken mailer or something that caused it to be off-list. When I hit reply-to-all in mutt I expect it to be sent to the list - and in your case it did not not hit the list. > > A number of embedded people emailed me about it off list, but nobody ever > replied to my post _on_ the list to say I wasn't alone in this, or acked the > patch to say it worked for them, or anything like that. So there was a > perception of zero support, and I gave up trying to follow up on it for > 2.6.25. I've got it working for me, and if more perl shows up I'll come up > with more patches to remove it for my own personal build environment. > > I might try to submit again in a few months Then please try to address the comments first. Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
On Monday 09 June 2008 16:27:29 Leon Woestenberg wrote: > > I submitted a patch to remove the use of perl to build the linux kernel > > (which HPA added in 2.6.25) not because it affected the result, but > > because it unnecessarily complicates the build system. (And perl tends > > to metasticize. > > Thanks, I hope it is or gets accepted. Nope, Peter Anvin shot it down (saying what I was doing was a strange, purely academic exercise), and Sam Ravnborg announced his vague intention to rewrite large chunks of kbuild in perl in the future. (Because nothing says "maintainability" like perl...) For some reason they wanted to mail me about it off-list rather than cc:ing the list about this plan. A number of embedded people emailed me about it off list, but nobody ever replied to my post _on_ the list to say I wasn't alone in this, or acked the patch to say it worked for them, or anything like that. So there was a perception of zero support, and I gave up trying to follow up on it for 2.6.25. I've got it working for me, and if more perl shows up I'll come up with more patches to remove it for my own personal build environment. I might try to submit again in a few months, but in the meantime my patches are at http://landley.net/hg/firmware/file/tip/sources/patches/ (see the linux-*noperl*.patch one). Maybe I'll try linux-kernel again. I submitted my miniconfig patches something like three times. But if the guys on the list didn't want it, and the guys in the embedded world poking me to do something about it weren't willing to even ack the darn patch on the list, and the people who want changes made to it aren't willing to change it from what I'm happy with to what they want and expect me to do it for them when I'm already happy with it... My todo list is long. This goes somewhere after resorting the contents of my bookshelves. I've found that posting to linux-kernel tends to attract "belling the cat" ideas. For example, I use kconfig in toybox and I posted patches to make it easier to use kconfig in other, non-kernel projects. The resulting thread turned into a grandiose plan about a separate kconfig that would be maintained as a spearate project and get installed on your system ala make, for use by any source package that wants that functionality. To which my response was "feel free, I have no interest in doing that, did you want my patches or not?" Improvements in the UI to use miniconfigs were held up by people objecting to how they were generated, which is separate from how they're used. And one of peter's objections to the perl removal patch (which basically checked the precalculated header file into the tree) was that some out of tree mips patch that never got merged might want to let you enter the clock speed as an input field, so you can't possibly just rely on precalculated values and people who want to check in code using a new value checking in an updated header with that value. (Even though that makes every existing architecture happy, last I checked.) I'm a hobbyist. I made it work for me. If nobody else is interested in what I did, I'm ok with that... Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/1] Embedded Maintainer(s), [EMAIL PROTECTED] list
Hello, On Wed, May 28, 2008 at 11:52 PM, Rob Landley <[EMAIL PROTECTED]> wrote: > On Wednesday 30 April 2008 14:11:49 David Woodhouse wrote: >> I agree. And if we do want to pay attention to pure code size, there are >> other approaches -- like --gc-sections and/or building with '--combine >> -fwhole-program' which I was playing with for OLPC a while back. I must >> dust that off now that the GCC fixes should mostly have made it into >> current distributions. > > I like simplicity. I like _simple_. Half my attraction to embedded systems > Same here. > I submitted a patch to remove the use of perl to build the linux kernel (which > HPA added in 2.6.25) not because it affected the result, but because it > unnecessarily complicates the build system. (And perl tends to metasticize. Thanks, I hope it is or gets accepted. In general I think one of the aspects of embedded Linux is about minimizing the amount of bloat dependencies. Especially, when each dependency can explode in a hurd of new dependencies. > "One of my most productive days was throwing away 1000 lines of code." > - Ken Thompson. > How appropriate. Regards, -- Leon -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
