Re: Why is building a cross compiler out-of-the-box always broken?
Stephen, I have been working on the x86_64-pc-mingw32 toolchain with Kai Tietz (Kai is the main person, I am doing much more learning than helping). I put together a built script that requires no tweaking whatsoever of any of the projects incorporated in the toolchain. It is very straightforward. You can check it out here: http://mingw-w64.svn.sourceforge.net/viewvc/*checkout*/mingw-w64/experimental/buildsystem/makebuildroot.sh?revision=56 The project itself is here: https://sourceforge.net/projects/mingw-w64 If you make a new directory (like /tmp/rt) and change into it, then run the script with the single argument build, it will download binutils, gcc, the crt, and the headers, then proceed to compile everything using the most default set of options and create a buildroot starting at the present working directory. The options to binutils are: --prefix=$PF --with-sysroot=$PF --target=x86_64-pc-mingw32 --disable-nls gcc only adds the option --enable-languages=c. The disable-nls option is just there to make the build faster -- it is not needed. with-sysroot has replaced with-headers and with-includes, and it allows for easy creation of buildroots. I set the prefix equal to the same thing because it works and I don't really understand it enough to do differently. The biggest difference between this and normal toolchains is the absence of glibc. Anyway, tell me what you think.
Re: Why is building a cross compiler out-of-the-box always broken?
On Sun, Aug 19, 2007 at 10:09:25AM +0200, Paolo Bonzini wrote:
Thanks for reminding me to ping that patch.
Could you try moving this case statement
+ case $target in
+ i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
+ libgloss_dir=i386
+ ;;
+ m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
+ libgloss_dir=m68hc11
+ ;;
above in another case $target in statement?
I don't understand what you want it to look like. Your explanation sounds
to me like
case $target in
i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
libgloss_dir=i386
;;
m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
libgloss_dir=m68hc11
;;
esac
case $target in
... rest of cases go here ...
*)
libgloss_dir=${cpu}
;;
esac
but then the default of the second case construct will override libgloss_dir
from the first one, if I'm not missing something here.
--
Rask Ingemann Lambertsen
Re: Why is building a cross compiler out-of-the-box always broken?
Rask Ingemann Lambertsen wrote:
On Sun, Aug 19, 2007 at 10:09:25AM +0200, Paolo Bonzini wrote:
Thanks for reminding me to ping that patch.
Could you try moving this case statement
+ case $target in
+ i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
+ libgloss_dir=i386
+ ;;
+ m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
+ libgloss_dir=m68hc11
+ ;;
above in another case $target in statement?
I don't understand what you want it to look like. Your explanation sounds
to me like
case $target in
i[[3456]]86-*-elf* | i[[3456]]86-*-coff*)
libgloss_dir=i386
;;
m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*)
libgloss_dir=m68hc11
;;
esac
Sorry, I meant all the case statement. There are many case $target
statements and I would rather avoid trying to avoid one. Setting
libgloss_dir=${cpu} as a default can be done above the case $target
you choose.
Paolo
Fwd: Re: Why is building a cross compiler out-of-the-box always broken?
(Stephen typoed the gcc address, forwarding) From: Segher Boessenkool [EMAIL PROTECTED] Date: 21 augustus 2007 17:10:30 GMT+02:00 To: Stephen M. Kenton [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Why is building a cross compiler out-of-the-box always broken? I got 17 minimal compilers to build using your help. Upon closer examination I realized that you are not building glibc, which of course if fine for your application testing kernel builds. Half of my targets do not even support glibc :-) When I tried to use those compilers to configure glibc, it complained about needing unwind support. Presumably because of --enable-sjlj-exceptions. Most likely yes. I modified the gcc configure to allow --with-headers to point to the kernel headers and still set inhibit libc, What sets the inhibit_libc -- do you modify configure itself for that, or does some configure option do it? then I removed the --enable-sjlj-exceptions and it built about the same set of compilers as before. Glibc configure died for another reason after that, but that's a different issue. I really think that lumping kernel and libc headers together in the configure when deciding whether to set inhibit_libc is a mistake. It looks one should be independent of the other. Another problem is that GCC should provide some of those headers itself really; certainly the headers for the parts of C that it does implement itself. That's a long-standing issue though, is there any progress on it (or even a plan for fixing it)? Segher
Re: Why is building a cross compiler out-of-the-box always broken?
Segher Boessenkool wrote: The manual explicitly says you need to have target headers. With all those --disable-XXX and especially --enable-sjlj-exceptions it all works fine without, though. If one tries to produce everything in the 'gcc' subdirectory, including 'libgcc', then headers may be needed. But if producing only the 'xgcc', 'cc1', 'collect2' etc binaries for the host is enough, then nothing for the target will be required. Everything is totally dependent on how that cross compiler will be defined Years ago in Berkeley university there was a system called Nachos which required a crosscompiler for DECstation/MIPS but without any DEC's target headers or libraries and even without any 'libgcc'. So if one needs a Nachos-like crosscompiler for some target on one's host, that could always succeed! Only problems totally related to the $host could be seen Meanwhile a cross GCC for AIX, HP-UX, and even for the quite common Mac OS X and for the commercial Enterprise Linux distros and for many other systems could be either impossible or very hard if no free access to the target stuff (copyrighted prebuilt libraries) is provided. And when one thinks a cross GCC being something similar to the native GCC, with all the base C libraries, all the extra termcap, curses, X11 etc. libraries... Or when there is not much or not enough support in the GNU binutils for the target. For instance a cross GCC for the MS's Interix aka Services for Unix would fail with the GNU ld not working for this target :-( So if one defines a cross compiler to include the as, ld etc. binutils and all the target libraries, then there can be very serious problems, neither the GNU binutils, nor the GNU libc are supporting all the targets GCC is seemingly supporting Someone recently mentioned that the GNU binutils shouldn't work for AIX and quite surely there isn't any free access to the AIX C libraries... Quite many crosscompiler builders don't have any clue about what GCC is and what components are belonging to it... Some will always mix producing a cross GCC with producing a self-made Linux distro, where that GCC is only a tool used to produce the final product, Linux :-( These things just should be much more clear in people's minds...
Re: Why is building a cross compiler out-of-the-box always broken?
On Mon, Aug 20, 2007 at 12:06:03PM +0300, Kai Ruottu wrote: If one tries to produce everything in the 'gcc' subdirectory, including 'libgcc', then headers may be needed. But if producing only the 'xgcc', 'cc1', 'collect2' etc binaries for the host is enough, then nothing for the target will be required. This is part of the reason why libgcc no longer lives in the gcc directory. -- Daniel Jacobowitz CodeSourcery
Re: Why is building a cross compiler out-of-the-box always broken?
Thanks for reminding me to ping that patch. Could you try moving this case statement + case $target in + i[[3456]]86-*-elf* | i[[3456]]86-*-coff*) + libgloss_dir=i386 + ;; + m68hc11-*-* | m6811-*-* | m68hc12-*-* | m6812-*-*) + libgloss_dir=m68hc11 + ;; above in another case $target in statement? Thanks, Paolo
Re: Why is building a cross compiler out-of-the-box always broken?
4. Use the minimal cross-compiler produced in step 3 to configure glibc and do make install-headers to get glibc headers. At this point you should be able to delete the minimal cross-compiler, it's job has been done. 5. Make distclean and then configure, make, and install a normal gcc cross-compiler for the target using the glibc headers produced in step 4. 6. Make distclean and then configure, make, and install the complete glibc header and libraries etc. for the target. Sometimes you need to repeat 4,5,6 (but using the full toolchain produced in the previous run, not the minimal one obviously). It's a good sanity check anyway ;-) 1. Seems to be a good reliable source of sanitized headers for recent kernels. I know of no problems with the current release. There still are problems so you're advised to use the separate kernel headers package instead. 2. Seems to build and work out-of-the-box using the headers from step 1. I know of no problems with current release. Binutils doesn't need any kernel headers AFAIK. 3. Figuring out the correct combination of --with-newlib, --with-sysroot, and --with-headers (or --without, or =no etc.) to get inhibit_libc set and then figuring out which of the myriad of --with-this-that-and-the-other options can or must be disabled to get a functional minimal compiler is a real pain in the neck. configure --verbose --help I posted the full set you currently need. Further more, the make file may try (and fail) to make additional things like crt0 in EXTRA_PARTS= Don't use --with-newlib if you don't want to build newlib. Right now it's a question of wtf caused that to die? Yeah, it's a fun voyage, isn't it :-) Also, should the inhibit_libc hack also disable things like the unwind support that requires kernel headers? That not scrictly libc, but --without-headers sort of assume neither are present. Yeah, that's a good point. I use --enable-sjlj-exceptions to avoid the need for those headers, but that's not enough for building ia64-linux (it _is_ enough for all other Linux targets). Some people advocate combining steps 3,4,5 by either cribbing a set of headers from someone elses sysroot, That's the normal procedure for building a cross-toolchain, yes. or by fooling glibc configure in to producing them without having a minimal cross compiler. You should only need this if you're porting to a new target. And then this all is the least of your troubles ;-) However, it's probably not viable for someone trying to build 20 or so tools chains so they can check for kernel build failure tree wide etc. That's exactly what I'm doing. Feel free to take a look at my scripts: git://git.infradead.org/~segher/buildall.git or browse it at: http://git.infradead.org/?p=users/segher/buildall.git And besides, it would sure be nice if gcc would build a cross-compiler out-of-the-box if there is no problem with some other part of the tool chain. That does not seem to be the case for all targets at this point. Bugs happen :-) So, that's the situation as I see it. How should things work in an ideal world and where do we go from here? Or, are other people happy enough with the current ecosystem and think nothing needs to change? It would be nice if things were a bit easier; maybe a single configure flag --from-scratch or something like that. Someone needs to create that though. Segher
Re: Why is building a cross compiler out-of-the-box always broken?
Thanks, I git cloned buildall and it sure looks simple (now that you have made it work) :-! Although, that combination of options was not exactly obvious at first glance. Time to re-read all the config --help stuff. I'll try it on my build system at work tomorrow and see what bites me next. I expect that I am having old nightmares from when I was trying to do this under Solaris and made it harder than I needed to on a Linux host. Apparently, I'm not the only one who got lead astray though. Thank again - Steve Segher Boessenkool wrote: 4. Use the minimal cross-compiler produced in step 3 to configure glibc and do make install-headers to get glibc headers. At this point you should be able to delete the minimal cross-compiler, it's job has been done. 5. Make distclean and then configure, make, and install a normal gcc cross-compiler for the target using the glibc headers produced in step 4. 6. Make distclean and then configure, make, and install the complete glibc header and libraries etc. for the target. Sometimes you need to repeat 4,5,6 (but using the full toolchain produced in the previous run, not the minimal one obviously). It's a good sanity check anyway ;-) 1. Seems to be a good reliable source of sanitized headers for recent kernels. I know of no problems with the current release. There still are problems so you're advised to use the separate kernel headers package instead. 2. Seems to build and work out-of-the-box using the headers from step 1. I know of no problems with current release. Binutils doesn't need any kernel headers AFAIK. 3. Figuring out the correct combination of --with-newlib, --with-sysroot, and --with-headers (or --without, or =no etc.) to get inhibit_libc set and then figuring out which of the myriad of --with-this-that-and-the-other options can or must be disabled to get a functional minimal compiler is a real pain in the neck. configure --verbose --help I posted the full set you currently need. Further more, the make file may try (and fail) to make additional things like crt0 in EXTRA_PARTS= Don't use --with-newlib if you don't want to build newlib. Right now it's a question of wtf caused that to die? Yeah, it's a fun voyage, isn't it :-) Also, should the inhibit_libc hack also disable things like the unwind support that requires kernel headers? That not scrictly libc, but --without-headers sort of assume neither are present. Yeah, that's a good point. I use --enable-sjlj-exceptions to avoid the need for those headers, but that's not enough for building ia64-linux (it _is_ enough for all other Linux targets). Some people advocate combining steps 3,4,5 by either cribbing a set of headers from someone elses sysroot, That's the normal procedure for building a cross-toolchain, yes. or by fooling glibc configure in to producing them without having a minimal cross compiler. You should only need this if you're porting to a new target. And then this all is the least of your troubles ;-) However, it's probably not viable for someone trying to build 20 or so tools chains so they can check for kernel build failure tree wide etc. That's exactly what I'm doing. Feel free to take a look at my scripts: git://git.infradead.org/~segher/buildall.git or browse it at: http://git.infradead.org/?p=users/segher/buildall.git And besides, it would sure be nice if gcc would build a cross-compiler out-of-the-box if there is no problem with some other part of the tool chain. That does not seem to be the case for all targets at this point. Bugs happen :-) So, that's the situation as I see it. How should things work in an ideal world and where do we go from here? Or, are other people happy enough with the current ecosystem and think nothing needs to change? It would be nice if things were a bit easier; maybe a single configure flag --from-scratch or something like that. Someone needs to create that though. Segher
Re: Why is building a cross compiler out-of-the-box always broken?
Thanks, I git cloned buildall and it sure looks simple (now that you have made it work) :-! Good to see it is useful to more people than just me :-) Although, that combination of options was not exactly obvious at first glance. Heh. Almost all of it is just disable what we don't need. OTOH, that's not how I ended up with those options: I added one every time something broke. I'll try it on my build system at work tomorrow and see what bites me next. What target(s) are you building? I expect that I am having old nightmares from when I was trying to do this under Solaris and made it harder than I needed to on a Linux host. Apparently, I'm not the only one who got lead astray though. The manual explicitly says you need to have target headers. With all those --disable-XXX and especially --enable-sjlj-exceptions it all works fine without, though. Let me know how it goes, Segher
Re: Why is building a cross compiler out-of-the-box always broken?
David Daney writes: Stephen M. Kenton wrote: Hello all, . . . I realize that there are various solutions for specific platforms. Dan Kegel's excellent crosstool and the cross-lfs website, . . . So, my open questions to the list are, what is/should be the preferred way to bootstrap a cross compiler/glibc environment? Don't bootstrap. That's right. Unless you're building a new operating system -- in which case building the toolchain is the least of your problems -- you don't need to boostrap. Just point the sysroot at the root filesystem of your target OS when configuring the cross compiler. Andrew.
Re: Why is building a cross compiler out-of-the-box always broken?
Andrew Haley wrote: David Daney writes: Stephen M. Kenton wrote: Hello all, . . . I realize that there are various solutions for specific platforms. Dan Kegel's excellent crosstool and the cross-lfs website, . . . So, my open questions to the list are, what is/should be the preferred way to bootstrap a cross compiler/glibc environment? Don't bootstrap. That's right. Unless you're building a new operating system -- in which case building the toolchain is the least of your problems -- you don't need to boostrap. Just point the sysroot at the root filesystem of your target OS when configuring the cross compiler. It is not always that simple. Sometimes you have everything you need for an existing OS except a sysroot. While it theory one might be able to obtain a suitable pre-built sysroot, often it is easier to bootstrap. Recently I bootstrapped a mips64-linux sysroot. I installed Debian mips which is 32 bits, but the kernel is 64 bits and can handle a mips64-linux userspace also. For me it seemed the easiest path to obtaining a sysroot was to crib off of the cross-lfs instructions. However I will never do it again now that I have a working sysroot. David Daney
Re: Why is building a cross compiler out-of-the-box always broken?
On Friday 17 August 2007 23:56:30 Ian Lance Taylor wrote: Stephen M. Kenton [EMAIL PROTECTED] writes: However, the question remains, why is the problem still there to be circumvented? Is there some secret opposition to easy use of these tools, is there some law of nature that prevents them from building, is there some good technical reason that is hard to implement, or has it just not been a big enough pain for anyone to beat it into submission? The central problem is that gcc, binutils, glibc, and the kernel are all separate projects which are distributed and maintained separately by different people. Thus there are mismatches and confusions and difficulties which result from the different release cycles and different agendas. Thus there is a place in the ecosystem for people to write the scripts needed to smooth over those differences. And indeed that ecosystem is filled by tools like crosstool and buildtool. Or the T2 SDE (http://www.t2-project.org). This is certainly not ideal. But the organizational differences make it quite difficult to fix in any other way. Ian -- René Rebe - ExactCODE GmbH - Europe, Germany, Berlin http://exactcode.de | http://t2-project.org | http://rene.rebe.name
Re: Why is building a cross compiler out-of-the-box always broken?
So, my open questions to the list are, what is/should be the preferred way to bootstrap a cross compiler/glibc environment? This likely isn't the preferred way, but the following builds a cross toolchain for all but a few Linux targets: $SRC/src/configure \ --target=$TARGET --prefix=$PREFIX make make install $SRC/gcc/configure \ --target=$TARGET --enable-targets=all \ --prefix=$PREFIX \ --enable-languages=c --without-headers \ --disable-nls --disable-multilib --disable-threads --disable-shared \ --disable-libmudflap --disable-libssp --disable-libgomp \ --disable-decimal-float \ --enable-sjlj-exceptions make make install (blackfin compiler needs to be configured as bfin-linux-uclibc, cris compiler won't build, h8300 compiler won't build, ia64 compiler won't build -- some embedded targets seem to need unmerged patches to work for building Linux but I don't bother). All normal targets work fine. People on the build farm can find this stuff in gcc13:~segher/build, and installed toolchains (build off mainline) in ~segher/cross. I do some fresh builds almost every day. Segher p.s. Any advice on how to get more targets working would be more than welcome :-)
Re: Why is building a cross compiler out-of-the-box always broken?
On Sat, Aug 18, 2007 at 08:18:08PM +0200, Segher Boessenkool wrote: p.s. Any advice on how to get more targets working would be more than welcome :-) With patch three from bug 32154 URL:http://gcc.gnu.org/bugzilla/attachment.cgi?id=13669, the newlib targets build out of the box in a combined tree: .../configure --target=xxx --with-newlib --enable-sim --disable-gdb --disable-nls --enable-checking=yes,rtl make See also URL:http://gcc.gnu.org/ml/gcc-help/2007-06/msg00230.html. Thanks for reminding me to ping that patch. -- Rask Ingemann Lambertsen
Re: Why is building a cross compiler out-of-the-box always broken?
On 8/17/07, Stephen M. Kenton [EMAIL PROTECTED] wrote: Cross compiling works for me out of the box if done correctly. Yes I have to compile GCC and glibc (or newlib) twice but I don't care as it is all scripted. Thanks, Andrew Pinski
Re: Why is building a cross compiler out-of-the-box always broken?
Stephen M. Kenton wrote: Hello all, . . . I realize that there are various solutions for specific platforms. Dan Kegel's excellent crosstool and the cross-lfs website, . . . So, my open questions to the list are, what is/should be the preferred way to bootstrap a cross compiler/glibc environment? Don't bootstrap. Use one of Kegal's crosstool or the cross-lfs instructions exactly once. Then keep a good build of glibc around for future build cycles. That way you never bootstrap. You just build a cross compiler in the standard manner, which *always* works. David Daney
Re: Why is building a cross compiler out-of-the-box always broken?
Stephen M. Kenton [EMAIL PROTECTED] writes: However, the question remains, why is the problem still there to be circumvented? Is there some secret opposition to easy use of these tools, is there some law of nature that prevents them from building, is there some good technical reason that is hard to implement, or has it just not been a big enough pain for anyone to beat it into submission? The central problem is that gcc, binutils, glibc, and the kernel are all separate projects which are distributed and maintained separately by different people. Thus there are mismatches and confusions and difficulties which result from the different release cycles and different agendas. Thus there is a place in the ecosystem for people to write the scripts needed to smooth over those differences. And indeed that ecosystem is filled by tools like crosstool and buildtool. This is certainly not ideal. But the organizational differences make it quite difficult to fix in any other way. Ian
Re: Why is building a cross compiler out-of-the-box always broken?
Andrew Pinski wrote: On 8/17/07, Stephen M. Kenton [EMAIL PROTECTED] wrote: Cross compiling works for me out of the box if done correctly. Yes I have to compile GCC and glibc (or newlib) twice but I don't care as it is all scripted. Thanks, Andrew Pinski Great! Scripting is wonderful and I don't object to multiple cycles either. Care to hit me with a clue-bat and post your script so I can find what I did wrong? It sound's like other people have out-of-the-box problems too. I'll patch my scripts and turn them loose on ~20 tool chain builds. Steve
