Re: [gentoo-user] The CHOST variable
On 8/2/2011, at 9:55pm, Alan McKinnon wrote: ... If you're a gambling man, play it by the numbers: A re-install for a Gentoo user with a clue is a certain 1 hour of your life tops to get it redone with a recent stage 3, more likely 30 minutes. That will give you a working system albeit one a bit out of date that emerge -avunD world will fix nicely. That makes no sense to me at all. There are stage3s built on a daily basis - they won't be out of date at all. But they don't include system logger, cron, locale, or any of your personalisations. On any kind of complex system it will take me more than a day to set those up - can I be sure that I'll get them right first time, every time? If we're talking about an older system then I need to back up everything so that I can copy files the files I missed from the /etc to the new one. If you're talking about stuff like KDE then that alone is going to take more than an hour to compile. I understood that a backup that was of your own system - that might be a bit out of date but which includes these things - was called a stage4. And in that case you're still faced with the problem that it's probably just as out of date as the install which is proving difficult to update in the first place. My experience of a system which was a year or even 18 months old was that it did have some blockers and some bugs which had been addressed here months before. It did require me to pull some files out of Gentoo's CVS attic and emerge the old ebuilds from my local tree, before they could be updated to current. But I knew that the configuration was sound and that all I had to do was address the blocker and then resume the `emerge -ud world`- the emerging is something that can go on in the background whilst I'm not paying attention. Stroller.
Re: [gentoo-user] The CHOST variable
On 8/2/2011, at 9:11pm, Nils Holland wrote: On 12:41 Tue 08 Feb , Stroller wrote: If my process wasn't clear from my last email: it looks like, following that document, you have to do the whole thing with changed CHOST, *before* making any changes to CFLAGS. It appears like only after you've `emerge -e world` with the new CHOST can you change CFLAGS. Thanks for the hint, right now I've got the i486 stage3 in a chroot and have changed both the CHOST and CFLAGS to i586. If that leads to problems, as you've described, I'll start over with only setting CHOST to i586 at first and then changing CFLAGS. Currently the build process is running fine, however, but I'm mentally prepared for difficults now. ;-) I completed the `emerge -e world` with the new CHOST, ran `lafilefixer --justfixit revdep-rebuild` and then Python and the Perl packages, as listed in the documentation. I'm pretty sure I ran `lafilefixer --justfixit revdep-rebuild` again, finding nothing. Changing CFLAGS=-march=native -O2 -pipe -fomit-frame-pointer failed again on the first package, ncurses. CFLAGS=-mtune=native -O2 -pipe -fomit-frame-pointer seems to be working (now on package 10 of 155). Perhaps I should have tried CFLAGS=-O2 -march=i586 -pipe instead. I don't really know what the -fomit-frame-pointer part does - I imagine someone suggested it, perhaps on here, years ago, and it has got copied from system to system. Stroller.
Re: [gentoo-user] The CHOST variable
On Thu, Feb 10, 2011 at 9:24 AM, Stroller strol...@stellar.eclipse.co.uk wrote: I don't really know what the -fomit-frame-pointer part does - I imagine someone suggested it, perhaps on here, years ago, and it has got copied from system to system. I think it removes your ability to get a stack trace when programs crash, in exchange for potentially more speed. If you're a programmer or filing bug reports on crashing software you should not use it, I don't think. But I could be wrong. :)
Re: [gentoo-user] The CHOST variable
On 10/2/2011, at 4:22pm, Paul Hartman wrote: On Thu, Feb 10, 2011 at 9:24 AM, Stroller strol...@stellar.eclipse.co.uk wrote: I don't really know what the -fomit-frame-pointer part does - I imagine someone suggested it, perhaps on here, years ago, and it has got copied from system to system. I think it removes your ability to get a stack trace when programs crash, in exchange for potentially more speed. If you're a programmer or filing bug reports on crashing software you should not use it, I don't think. But I could be wrong. :) Thanks! :D I appreciate the informative helpful answer. Stroller.
Re: [gentoo-user] The CHOST variable
Apparently, though unproven, at 17:18 on Thursday 10 February 2011, Stroller did opine thusly: On 8/2/2011, at 9:55pm, Alan McKinnon wrote: ... If you're a gambling man, play it by the numbers: A re-install for a Gentoo user with a clue is a certain 1 hour of your life tops to get it redone with a recent stage 3, more likely 30 minutes. That will give you a working system albeit one a bit out of date that emerge -avunD world will fix nicely. That makes no sense to me at all. There are stage3s built on a daily basis - they won't be out of date at all. But they don't include system logger, cron, locale, or any of your personalisations. All I'm saying is it is often better to go the reinstall route which will take a known amount of time for a certain result, rather than the try fix it route which is highly variable as to how long it will take or even if it will work at all. It's assumed that one has backups of world and all configs, or can make backups before starting the reinstall. As with all things YMMV and real circumstance trumps abstract theory every time On any kind of complex system it will take me more than a day to set those up - can I be sure that I'll get them right first time, every time? If we're talking about an older system then I need to back up everything so that I can copy files the files I missed from the /etc to the new one. If you're talking about stuff like KDE then that alone is going to take more than an hour to compile. I understood that a backup that was of your own system - that might be a bit out of date but which includes these things - was called a stage4. And in that case you're still faced with the problem that it's probably just as out of date as the install which is proving difficult to update in the first place. My experience of a system which was a year or even 18 months old was that it did have some blockers and some bugs which had been addressed here months before. It did require me to pull some files out of Gentoo's CVS attic and emerge the old ebuilds from my local tree, before they could be updated to current. But I knew that the configuration was sound and that all I had to do was address the blocker and then resume the `emerge -ud world`- the emerging is something that can go on in the background whilst I'm not paying attention. Stroller. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] The CHOST variable
On 10:22 Thu 10 Feb , Paul Hartman wrote: On Thu, Feb 10, 2011 at 9:24 AM, Stroller strol...@stellar.eclipse.co.uk wrote: I don't really know what the -fomit-frame-pointer part does - I imagine someone suggested it, perhaps on here, years ago, and it has got copied from system to system. I think it removes your ability to get a stack trace when programs crash, in exchange for potentially more speed. If you're a programmer or filing bug reports on crashing software you should not use it, I don't think. But I could be wrong. :) Yep, that option leads to the frame pointer not being saved / restored, making an additional register available (but obviously there are differences between architectures; on some this option has no effect at all). On architectures where it does have an effect, it would make debugging impossible. Well, after having written the last paragraph I've decided to check out the GCC man page for more information and see: According to the GCC man page -fomit-frame-pointer gets enabled by default when using optimization levels -O or higher (including, of course -O2, which in my experience is what most people seem to use). As a matter of fact, I have -O2 but intentionally not set -fomit-frame-pointer, not being aware that with -O2, I automatically get it. ;-) I'll have to check out the Gentoo handbook, but if I remember correctly, it actually suggests using -O2 but better stay away from -fomit-frame-pointer. If it really suggests this, that might be considered a bug I guess. ;-) Greetings, Nils -- Nils Holland * Ti Systems, Wunstorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] The CHOST variable
On 7/2/2011, at 11:03pm, Nils Holland wrote: On 12:24 Mon 07 Feb , Stroller wrote: The closest Gentoo stage was i486, and on such a slow old system it would be nice to squeeze out any extra performance I can. ... So what would probably work and what I'll try in the next days is the following: Unpack i486 tarball, chroot into it, and then manually try to make a i586 tarball out of it. That would probably involve the process described in the Gentoo how to change your CHOST document. If my process wasn't clear from my last email: it looks like, following that document, you have to do the whole thing with changed CHOST, *before* making any changes to CFLAGS. It appears like only after you've `emerge -e world` with the new CHOST can you change CFLAGS. I guess that once I've prepared such an i586 stage3 manually, I can regularely let catalyst handle updating it, just as it does fine with the original i486 and i686 stage3's. I haven't used catalyst at all. You make it sound interesting useful. Stroller.
Re: [gentoo-user] The CHOST variable
On 12:41 Tue 08 Feb , Stroller wrote: If my process wasn't clear from my last email: it looks like, following that document, you have to do the whole thing with changed CHOST, *before* making any changes to CFLAGS. It appears like only after you've `emerge -e world` with the new CHOST can you change CFLAGS. Thanks for the hint, right now I've got the i486 stage3 in a chroot and have changed both the CHOST and CFLAGS to i586. If that leads to problems, as you've described, I'll start over with only setting CHOST to i586 at first and then changing CFLAGS. Currently the build process is running fine, however, but I'm mentally prepared for difficults now. ;-) I guess that once I've prepared such an i586 stage3 manually, I can regularely let catalyst handle updating it, just as it does fine with the original i486 and i686 stage3's. I haven't used catalyst at all. You make it sound interesting useful. I've found it useful in the sense that it has allowed me to download an original Gentoo stage3 way back, and then - say, once a month - I've let catalyst build a new, updated stage3 out of it which I then used for new installations whenever I had to do then. Of course, I could just have grabbed one of the official Gentoo stage3's that get updated weekly, but hey, somehow it just feels cooler to use a stage3 you've rolled yourself. ;-) I'll post again when I have my i586 stage3 available for download. Folks wanting to perform a new Gentoo installation on an i586 kind of machine could then just grab and use that one if they don't want to use Gentoo's i486 stage3 (and stay at i486 or change the CHOST / CFLAGS to i586 after installation themselves). Greetings, Nils -- Nils Holland * Ti Systems, Wunstorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] The CHOST variable
Apparently, though unproven, at 15:33 on Monday 07 February 2011, Neil Bothwick did opine thusly: An `emerge -e world` may break things, but it's not usually that likely to. An emerge -e world is not likely to break things in itself, but the steps that require it, such as changing CHOST, are. The extra steps of a reinstall over trying to fix a machine with a borked binutils/glibc/whatever can be far more time consuming, not to mention frustrating, than a reinstall, and may only be fixed by a reinstall anyway after all that. I'm not an advocate of reinstalling normally, this installation is three years older than the computer running it, but when performing drastic low-level surgery, I believe it should be contemplated. If you're a gambling man, play it by the numbers: A re-install for a Gentoo user with a clue is a certain 1 hour of your life tops to get it redone with a recent stage 3, more likely 30 minutes. That will give you a working system albeit one a bit out of date that emerge -avunD world will fix nicely. Trying to fix the existing system complete with it's borkage is very uncertain. Maybe it's 5 minutes, maybe it's 2 hours, maybe it's three days on this list scratching heads before eventually giving up and re-installing anyway. Personally, I long ago proved I can do #2. I like the certainty from #1. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] The CHOST variable
On 4/2/2011, at 9:31am, Neil Bothwick wrote: ... Any time you consider a process that involves emerge -e world, you should also consider a reinstall. A reinstall will certainly be quicker, the only reason for an in place fix is if you cannot take the machine down for that length of time. I disagree. The `emerge -e world` may take longer, but it will consume less of *my time* overseeing it. Obviously this may not be the case if we're talking about a desktop system in constant use, and the update breaks your web-browser. But in all my use cases I'm going to be able to leave the `emerge -e world` running in the background and just check up on it periodically. A Gentoo reinstall is not so easy as click, click, click and let it run. You should do a full backup first, because you can easily copy /etc over to the new machine and only afterwards realise you forgot to copy /var/bind/ An `emerge -e world` may break things, but it's not usually that likely to. Stroller.
Re: [gentoo-user] The CHOST variable
On 4/2/2011, at 5:58am, Florian Philipp wrote:
...
The warning is actually there to stop users doing stupid things like blindly
trying to convert 32 bit systems to 64 bit. This is how that goes down:
1. Change CHOST
2. emerge -e world
3. ???
4. Fail!
Is the same true for more compatible arches like i486 - i686? I have a
system where I used the wrong stage-3 and now I'm stuck with an i486
CHOST on an Atom netbook where I could use every bit of performance.
I'm in the process of this at the moment.
I have a Cobalt Qube III, which has a K6-2 CPU, and which will never
accommodate any other processor.
The closest Gentoo stage was i486, and on such a slow old system it would be
nice to squeeze out any extra performance I can.
From the i486 stage I used:
# cat /etc/make.conf.catalyst
# These settings were set by the catalyst build script that automatically
# built this stage.
CFLAGS=-O2 -march=i486 -pipe
CXXFLAGS=${CFLAGS}
# WARNING: Changing your CHOST ...
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST=i486-pc-linux-gnu
Following the instructions I changed the CHOST (to i586) and did the `emerge -1
binutils gcc glibc` part.
That went fine, so I changed CFLAGS=-march=native -O2 -pipe
-fomit-frame-pointer, and followed the next steps up to 2.12 - `emerge -e
world`.
This failed about 3 packages in and really messed things up - I quickly
couldn't emerge anything else and running executables failed with a segfault or
something.
I had to pull the drive back out of the Qube and copy most of the i486 stage
back over the fouled-up installation.
This time around, I changed the CHOST again to i586 and ran `emerge -1 binutils
gcc glibc`, then `lafilefixer --justfixit revdep-rebuild` (didn't need to do
anything, it said) then the `emerge -e world` part - but this time still with
the original i486 CFLAGS (as the catalyst above). This is currently at 104 of
145 packages through the emerge process, so seems ok.
Hopefully when that's done I'll be able to change CFLAGS to -march=native and
re-emerge the whole lot again. I'm now a bit nervous of that (I'm wondering if
`-mcpu` or `-mtune` would be more suitable as a first step), so I'll take a
disk image using dd (again) so that I have a restore point, should I need it.
Stroller.
Re: [gentoo-user] The CHOST variable
On Mon, 7 Feb 2011 11:13:27 +, Stroller wrote: Any time you consider a process that involves emerge -e world, you should also consider a reinstall. A reinstall will certainly be quicker, the only reason for an in place fix is if you cannot take the machine down for that length of time. I disagree. The `emerge -e world` may take longer, but it will consume less of *my time* overseeing it. Obviously this may not be the case if we're talking about a desktop system in constant use, and the update breaks your web-browser. But in all my use cases I'm going to be able to leave the `emerge -e world` running in the background and just check up on it periodically. A Gentoo reinstall is not so easy as click, click, click and let it run. To a large extent, it is. you've already partitioned the drive, set up make.conf, created a world file, all of which can be reused. So a reinstall isn't much more than emerge -e world and recompile the kernel after emptying the filesystem. An `emerge -e world` may break things, but it's not usually that likely to. An emerge -e world is not likely to break things in itself, but the steps that require it, such as changing CHOST, are. The extra steps of a reinstall over trying to fix a machine with a borked binutils/glibc/whatever can be far more time consuming, not to mention frustrating, than a reinstall, and may only be fixed by a reinstall anyway after all that. I'm not an advocate of reinstalling normally, this installation is three years older than the computer running it, but when performing drastic low-level surgery, I believe it should be contemplated. -- Neil Bothwick Quality control, n.: Assuring that the quality of a product does not get out of hand and add to the cost of its manufacture or design. signature.asc Description: PGP signature
Re: [gentoo-user] The CHOST variable
On 12:24 Mon 07 Feb , Stroller wrote: The closest Gentoo stage was i486, and on such a slow old system it would be nice to squeeze out any extra performance I can. Well, what I'm currently in the process of trying to do (not because I have an actual need for it, but rather out of experimentation) is taking a Gentoo i486 stage3 tarball and turning it into a i586 one. I realized that on the i686 host I am using, I can just let catalyst loose on an official Gentoo i486 or i686 tarball and, with the appropriate settings, have it build a new and more current tarball out of it. Fine. However, what certainly doesn't work is taking one of these stage3 tarballs (i486 or i686) and have catalyst build an i586 tarball out of that. In my attemts when trying this, catalyst first compiled a new glibc with i586, and the next thing it tried was building a new zlib with using the i586 glibc and the still i486 gcc and other stuff, which resulted in failure (don't have the exact error message anymore, but it was along the lines of the linker complaining about not being able to determine SONAME of libz.so). So what would probably work and what I'll try in the next days is the following: Unpack i486 tarball, chroot into it, and then manually try to make a i586 tarball out of it. That would probably involve the process described in the Gentoo how to change your CHOST document. I guess that once I've prepared such an i586 stage3 manually, I can regularely let catalyst handle updating it, just as it does fine with the original i486 and i686 stage3's. Again, I'm only doing this out of curiosity to see if it works and / or what problems I encounter and to learn some stuff; I'm probably never going to use the resulting stage3 tarball on my own machines, as Gentoo's i486 and i686 ones suit me just fine and I'm not a big performance tuner / tweaker. Still, if I succeed and if anyone is interested, I could certainly make me i586 stage3 available for download. I've also, just for fun, looked at a i386 stage3. However, while that would technically certainly also be possible, it seems that Gentoo's glibc ebuild is specifically set up to bail out when building on i386. That would probably even be fixable by not building with nptl, but that way it would only be possible to built something that would make problems when using the normal portage tree and coming into a situation where it wants to build a new (nptl-enabled) glibc for the first time, not to mention that I don't have a clue what other stuff would break when the user tries to install it from portage on a system that comes with a glibc I've hacked to come without nptl. Greetings, Nils -- Nils Holland * Ti Systems, Wunstorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] The CHOST variable
On 21:21 Fri 04 Feb , Enrico Weigelt wrote: * Nils Holland n...@tisys.org wrote: 1) So a package using the GNU build system determines or is passed (via --host aka. CHOST) a target triplet specifying the system on which the resulting compiled code is supposed to run. What does the package do with that information? Does it only use it to determine what it has to compile (different / special code for different systems / architectures), or does this already have an influence on the optimization of the resulting code for a certain (sub-)architecture? Exactly. Some packages have arch- and subarch-specific optimizations. Those things IMHO should be primarily taken from the target triplet (instead of other ./configure options), as it quite exactly defines what cputype, platform and toolchain type you're using. It's very important for crosscompiling (even some toolchains, eg. gcc., could, and should, be explicitly asked for their target triplet). Hi Enrico, thanks very much for your in-depth explanation in this and also your other mail you've sent in response to my question. That was really helpful, as it cleared up some things I could previously only guess about but was by no means certain about. I've learnt something new (which was my highest priority here), and can even use this knowlede to deduce what makes sense in my current experiments with building my own stage3's and install boot discs via catalyst. So, thanks again for your insightful replies! Greetings, Nils -- Nils Holland * Ti Systems, Wunstorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] The CHOST variable
Florian Philipp wrote: Am 04.02.2011 01:27, schrieb Alan McKinnon: Yes, if you are real smart it can be done. But real smart really does mean real smart i.e. not for the faint of heart and certainly not worth being officially supported. Is the same true for more compatible arches like i486 - i686? I have a system where I used the wrong stage-3 and now I'm stuck with an i486 CHOST on an Atom netbook where I could use every bit of performance. Yep, that's about the same direction I was wondering about when asking my question. Specifically, I've been playing around with catalyst a bit in order to create my own stage3 tarballs and install discs. Works fine. But then I started to wonder about the following: I'd probably like to make my own stage3 and install disc i486 or even i386, so that it works everywhere. Then, on whatever machine I install it, I would probably like to change the CHOST to something suitable for that machine, and that's actually what the comment in /etc/make.conf suggests against doing. As a result, I started to wonder why this is so, and what exactly would happen if done anyway. Of course, this is not really a big problem. I could create multiple stage3's with catalyst whenever I really need one to install on a machine and have that stage3 by of the right type for the machine right from the start. But still, always eager to learn, I thought I'd ask about the background behind all that stuff. ;-) Greetings, Nils
Re: [gentoo-user] The CHOST variable
Apparently, though unproven, at 11:29 on Friday 04 February 2011, Nils Holland did opine thusly: Florian Philipp wrote: Am 04.02.2011 01:27, schrieb Alan McKinnon: Yes, if you are real smart it can be done. But real smart really does mean real smart i.e. not for the faint of heart and certainly not worth being officially supported. Is the same true for more compatible arches like i486 - i686? I have a system where I used the wrong stage-3 and now I'm stuck with an i486 CHOST on an Atom netbook where I could use every bit of performance. Yep, that's about the same direction I was wondering about when asking my question. Specifically, I've been playing around with catalyst a bit in order to create my own stage3 tarballs and install discs. Works fine. But then I started to wonder about the following: I'd probably like to make my own stage3 and install disc i486 or even i386, so that it works everywhere. Then, on whatever machine I install it, I would probably like to change the CHOST to something suitable for that machine, and that's actually what the comment in /etc/make.conf suggests against doing. As a result, I started to wonder why this is so, and what exactly would happen if done anyway. Of course, this is not really a big problem. I could create multiple stage3's with catalyst whenever I really need one to install on a machine and have that stage3 by of the right type for the machine right from the start. But still, always eager to learn, I thought I'd ask about the background behind all that stuff. ;-) Interestingly, Ubuntu has always built for basic arches, and they seem to get away with it. IIRC they are now on i586 but for the longest time used i386. No performance issues. You might want to investigate how they do their builds and see if you can use their tricks. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] The CHOST variable
On Fri, 04 Feb 2011 06:58:27 +0100, Florian Philipp wrote: Is the same true for more compatible arches like i486 - i686? I have a system where I used the wrong stage-3 and now I'm stuck with an i486 CHOST on an Atom netbook where I could use every bit of performance. It's possible, there's even a Gentoo document on it, http://www.gentoo.org/doc/en/change-chost.xml Note the first line states Changing the CHOST is a big issue that can seriously screw up your system Any time you consider a process that involves emerge -e world, you should also consider a reinstall. A reinstall will certainly be quicker, the only reason for an in place fix is if you cannot take the machine down for that length of time. -- Neil Bothwick On the other hand, you have different fingers. signature.asc Description: PGP signature
Re: [gentoo-user] The CHOST variable
Alan McKinnon wrote: Interestingly, Ubuntu has always built for basic arches, and they seem to get away with it. IIRC they are now on i586 but for the longest time used i386. No performance issues. You might want to investigate how they do their builds and see if you can use their tricks. The question is, I guess, if the target host, when of the same arch (i.e. i[3456]86) does actually have any influence on the code that gets generated in terms of performance or ability to run on other sub-arches. This is what I really couldn't find out so far and would find highly interesting to know. For example, why not just go (and stay) with CHOST=i386-pc-linux-gnu and on an i686 machine, set march or mcpu = i686 via CFLAGS if you want to optimize for the particular subarch at hand? Why should it be necessary / what would the (dis)advantages be of of such a setup vs. also having CHOST set to i686-pc-linux-gnu? Concering the Gentoo doc about changing the CHOST that was mentioned: Yep, I've read that. If I understood it correctly, problems when changing CHOST mainly seem to arise because of the way GCC and related basic build utils install themselves (using the host triplet as part of their path or executable name), leaving wrong / messed up references when changing the CHOST. For example, as I've said previously, the CHOST value gets passed to ./configure as --host for each package that gets build. That would make configure try to select a compiler called CHOST-gcc in order to compile the package, i.e. when CHOST is i486-pc-linux-gnu, a compiler called i486-pc-linux-gnu-gcc would be used. Include file directories for glibc and / or glibc itself sems to be selected by a similiar mechanism. All right, no problem, so far this only determines how things are called and where they are located, but are there any other real effects of all this stuff? The Gentoo CHOST document that was mentioned says something about nptl not being available on i386. If true, and if the host variable generally influences the availability of features, things would slowly start to make sens as to why the CHOST might be important. On the other hand, I've read through some documentation of the GNU C library (glibc) and didn't even find anything about nptl not being available on i386, not to mention anything else about different features on different subarches. You see, I'm probably not entirely getting it yet. ;-) Greetings, Nils
Re: [gentoo-user] The CHOST variable
On Fri, Feb 4, 2011 at 3:50 PM, Nils Holland n...@tisys.org wrote: Alan McKinnon wrote: Interestingly, Ubuntu has always built for basic arches, and they seem to get away with it. IIRC they are now on i586 but for the longest time used i386. No performance issues. You might want to investigate how they do their builds and see if you can use their tricks. The question is, I guess, if the target host, when of the same arch (i.e. i[3456]86) does actually have any influence on the code that gets generated in terms of performance or ability to run on other sub-arches. This is what I really couldn't find out so far and would find highly interesting to know. For example, why not just go (and stay) with CHOST=i386-pc-linux-gnu and on an i686 machine, set march or mcpu = i686 via CFLAGS if you want to optimize for the particular subarch at hand? Why should it be necessary / what would the (dis)advantages be of of such a setup vs. also having CHOST set to i686-pc-linux-gnu? Concering the Gentoo doc about changing the CHOST that was mentioned: Yep, I've read that. If I understood it correctly, problems when changing CHOST mainly seem to arise because of the way GCC and related basic build utils install themselves (using the host triplet as part of their path or executable name), leaving wrong / messed up references when changing the CHOST. For example, as I've said previously, the CHOST value gets passed to ./configure as --host for each package that gets build. That would make configure try to select a compiler called CHOST-gcc in order to compile the package, i.e. when CHOST is i486-pc-linux-gnu, a compiler called i486-pc-linux-gnu-gcc would be used. Include file directories for glibc and / or glibc itself sems to be selected by a similiar mechanism. All right, no problem, so far this only determines how things are called and where they are located, but are there any other real effects of all this stuff? The Gentoo CHOST document that was mentioned says something about nptl not being available on i386. If true, and if the host variable generally influences the availability of features, things would slowly start to make sens as to why the CHOST might be important. On the other hand, I've read through some documentation of the GNU C library (glibc) and didn't even find anything about nptl not being available on i386, not to mention anything else about different features on different subarches. You see, I'm probably not entirely getting it yet. ;-) Greetings, Nils Hi, I am not a big guru there, but i have changed CHOST variable successfully few years ago. If I remember, the steps were like that: Change CHOST variable. Bootstrap system (like building from stage 1): # /usr/portage/scripts/bootstrap.sh # emerge -e system # emerge -e world Before gentoo has been providing daily stages, I was installing my systems from stage1. It was a nice learning curve :) -- mv
Re: [gentoo-user] The CHOST variable
* Nils Holland n...@tisys.org wrote: 1) So a package using the GNU build system determines or is passed (via --host aka. CHOST) a target triplet specifying the system on which the resulting compiled code is supposed to run. What does the package do with that information? Does it only use it to determine what it has to compile (different / special code for different systems / architectures), or does this already have an influence on the optimization of the resulting code for a certain (sub-)architecture? Exactly. Some packages have arch- and subarch-specific optimizations. Those things IMHO should be primarily taken from the target triplet (instead of other ./configure options), as it quite exactly defines what cputype, platform and toolchain type you're using. It's very important for crosscompiling (even some toolchains, eg. gcc., could, and should, be explicitly asked for their target triplet). Forthermore: Does configuring a package with, say, --host=i686-pc-linux-gnu already automatically mean that said package would not be able to run on (for example) an i486-pc-linux-gnu host? Maybe. Or maybe it's just quite slow, eg. when it's using certain opcodes that are not supported by the older subarch and handled by the kernel via exceptions (back in i386 times, that was the case for fpops, when no fpu was present). It all depends on how the individual package is actually coded. 2) /etc/make.conf contains a note that one should not change the CHOST lightly (not that I'm planning to) and links to a nice document explaining how it can be done anyway (which, I have to admit, didn't make me any wiser, however). The question is, out of curiosity, why the CHOST should not be changed and what would happen if one did it anyway. I willingly believe that it would lead to problems, but would the actual cause of these problems actually be caused by the configuration of the machine being mixed up (for example, by the GNU build system / autoconf suddenly looking for a compiler or similiar tools / libraries under a path or by a name involving, for example, i486-pc-linux-gnu, which does not automatically exist of the appropriate tools have not been installed accordingly. Or would problems arise because code generated with the new CHOST does no longer fit to code generated with the previous / old CHOST? One of the most evil things that can happen (also w/ certain optimization flags) is a mismatching calling convention between certain libraries and executables. For example, code built for a newer subarch could put more call arguments into registers which the called functions dont expect - stack corruptions will happen. cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weig...@metux.de mobile: +49 151 27565287 icq: 210169427 skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [gentoo-user] The CHOST variable
* Nils Holland n...@tisys.org wrote:
The question is, I guess, if the target host, when of the same arch (i.e.
i[3456]86) does actually have any influence on the code that gets
generated in terms of performance or ability to run on other sub-arches.
This is what I really couldn't find out so far and would find highly
interesting to know.
Yes, both. Passing arguments via register (instead of stack) can be
much faster in some cases, especially when you have a lot of calls
with small parameter sets, eg:
libfoo.c:
int foo(int a, int b)
{
... do-something ...
}
progbar.c:
int main()
{
...
for (int x=0; x1; x++)
foo(x, wurst);
...
}
On older subarchs, main() will have to push x and wurst on stack for
_each_ call, and foo() has to pop them down. With a newer subarch that
has more usable registers, the parameters a and b can directly sit in
registers (probably the counting of x might already happen in the same
one), so the parameters dont have to go through the stack. That all
heavily depends on the target ABI.
For example, why not just go (and stay) with CHOST=i386-pc-linux-gnu and
on an i686 machine, set march or mcpu = i686 via CFLAGS if you want to
optimize for the particular subarch at hand? Why should it be necessary /
what would the (dis)advantages be of of such a setup vs. also having CHOST
set to i686-pc-linux-gnu?
I'm not sure right now if/how -march and -mcpu affect calling convention,
variable alignments, etc (IMHO -march does, while -mcpu doesn't), but
as soon as you change these ABI elements, it can get really dangerous.
Concering the Gentoo doc about changing the CHOST that was mentioned: Yep,
I've read that. If I understood it correctly, problems when changing CHOST
mainly seem to arise because of the way GCC and related basic build utils
install themselves (using the host triplet as part of their path or
executable name), leaving wrong / messed up references when changing the
CHOST.
You're lucky if the compiler fails early this way, so you don't have
the unpleasant experience of sudden runtime breakages ;-p
For example, as I've said previously, the CHOST value gets passed to
./configure as --host for each package that gets build. That would make
configure try to select a compiler called CHOST-gcc in order to compile
... if the toolchain commands aren't passed explicitly ...
(which you *SHOULD* do in any crosscompiling scenario)
the package, i.e. when CHOST is i486-pc-linux-gnu, a compiler called
i486-pc-linux-gnu-gcc would be used. Include file directories for glibc
and / or glibc itself sems to be selected by a similiar mechanism.
AFAIK they're not selected by this parameter (even some illdesigned
configure script might do such crazy things on its own ;-o), but built
into the toolchain commands themselves. You can override/change them,
but then you really should know what you're doing ;-p
The Gentoo CHOST document that was mentioned says something about nptl not
being available on i386. If true, and if the host variable generally
influences the availability of features, things would slowly start to make
sens as to why the CHOST might be important.
Exactly. nptl vs. linuxthreads is exactly an good example for
incompatible ABIs (while API more or less stays the same).
On the other hand, I've read
through some documentation of the GNU C library (glibc) and didn't even
find anything about nptl not being available on i386, not to mention
anything else about different features on different subarches.
I don't know much about their internals, but I can easily imagine
that nptl makes heavy use of newer registers which linuxthreads didn't.
For those things you should ask the gcc and libc folks.
cu
--
--
Enrico Weigelt, metux IT service -- http://www.metux.de/
phone: +49 36207 519931 email: weig...@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
--
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
--
[gentoo-user] The CHOST variable
Hi folks, well, it's not that a certain thing I'm intending to do has pointed me to it, but I've just noticed that something I've taken for granted for years is something I probably fail to understand correctly. And as I'm always eager to learn, I'm wondering if someone can point me in the right direction. I'm talking about the CHOST variable as defined in /etc/make.conf, and what actually lies behind it. Obviously, whatever is specified in make.conf as CHOST is passed to ./configure as --host when emerging packages. Ok, I know several things about that: The contents of the CHOST variable is a target triplet in the form of (for example) i686-pc-linux-gnu. According to the autoconf info documentation, packages using the GNU build environment use this triplet to properly configure the package at hand for the specified target architecture. If no --host is given to configure, it will try to determine the correct triplet for the current system by the rules specified in config.sub. Beside --host, configure also accepts --build and --target, which specify the system on which the package is being configured and built (--build) and the the type of architecture for which a compiler being configured is supposed to be able to generate code (--target, which - I believe - only really seems to make sense when building a compiler like GCC). These can also be influenced via make.conf, where they are called CBUILD and CTARGET. All of this is fine so far and relatively understandable. But a few questions remain: 1) So a package using the GNU build system determines or is passed (via --host aka. CHOST) a target triplet specifying the system on which the resulting compiled code is supposed to run. What does the package do with that information? Does it only use it to determine what it has to compile (different / special code for different systems / architectures), or does this already have an influence on the optimization of the resulting code for a certain (sub-)architecture? Forthermore: Does configuring a package with, say, --host=i686-pc-linux-gnu already automatically mean that said package would not be able to run on (for example) an i486-pc-linux-gnu host? Or can this only be said to be true if the package itself decides to compile different / incompatible code for i686 and i486; in other words: If the package itself does not make any distinction on the CPU subtype, then the result would run everywhere, as --host does not by itself imply any such thing as --march=? 2) /etc/make.conf contains a note that one should not change the CHOST lightly (not that I'm planning to) and links to a nice document explaining how it can be done anyway (which, I have to admit, didn't make me any wiser, however). The question is, out of curiosity, why the CHOST should not be changed and what would happen if one did it anyway. I willingly believe that it would lead to problems, but would the actual cause of these problems actually be caused by the configuration of the machine being mixed up (for example, by the GNU build system / autoconf suddenly looking for a compiler or similiar tools / libraries under a path or by a name involving, for example, i486-pc-linux-gnu, which does not automatically exist of the appropriate tools have not been installed accordingly. Or would problems arise because code generated with the new CHOST does no longer fit to code generated with the previous / old CHOST? Any hints here would be welcome. Again, it's not that I need to know in order to do something, but all of this stuff has always worked so well every time I've built a package, and if feels kind of strange not to know why / how it actually works. As neither the docs of autoconf, binutils nor GCC could properly enlighten me, I thought I'd ask here. ;-) Greetings, Nils -- Nils Holland * Ti Systems, Wunstorf-Luthe (Germany) Powered by GNU/Linux since 1998
Re: [gentoo-user] The CHOST variable
Apparently, though unproven, at 01:43 on Friday 04 February 2011, Nils Holland did opine thusly: I'm not in a position to give a fully definitive answer to 1) ... 2) /etc/make.conf contains a note that one should not change the CHOST lightly (not that I'm planning to) and links to a nice document explaining how it can be done anyway (which, I have to admit, didn't make me any wiser, however). The question is, out of curiosity, why the CHOST should not be changed and what would happen if one did it anyway. I willingly believe that it would lead to problems, but would the actual cause of these problems actually be caused by the configuration of the machine being mixed up (for example, by the GNU build system / autoconf suddenly looking for a compiler or similiar tools / libraries under a path or by a name involving, for example, i486-pc-linux-gnu, which does not automatically exist of the appropriate tools have not been installed accordingly. Or would problems arise because code generated with the new CHOST does no longer fit to code generated with the previous / old CHOST? The warning is actually there to stop users doing stupid things like blindly trying to convert 32 bit systems to 64 bit. This is how that goes down: 1. Change CHOST 2. emerge -e world 3. ??? 4. Fail! Yes, if you are real smart it can be done. But real smart really does mean real smart i.e. not for the faint of heart and certainly not worth being officially supported. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] The CHOST variable
Am 04.02.2011 01:27, schrieb Alan McKinnon: Apparently, though unproven, at 01:43 on Friday 04 February 2011, Nils Holland did opine thusly: I'm not in a position to give a fully definitive answer to 1) ... 2) /etc/make.conf contains a note that one should not change the CHOST lightly (not that I'm planning to) and links to a nice document explaining how it can be done anyway (which, I have to admit, didn't make me any wiser, however). The question is, out of curiosity, why the CHOST should not be changed and what would happen if one did it anyway. I willingly believe that it would lead to problems, but would the actual cause of these problems actually be caused by the configuration of the machine being mixed up (for example, by the GNU build system / autoconf suddenly looking for a compiler or similiar tools / libraries under a path or by a name involving, for example, i486-pc-linux-gnu, which does not automatically exist of the appropriate tools have not been installed accordingly. Or would problems arise because code generated with the new CHOST does no longer fit to code generated with the previous / old CHOST? The warning is actually there to stop users doing stupid things like blindly trying to convert 32 bit systems to 64 bit. This is how that goes down: 1. Change CHOST 2. emerge -e world 3. ??? 4. Fail! Yes, if you are real smart it can be done. But real smart really does mean real smart i.e. not for the faint of heart and certainly not worth being officially supported. Is the same true for more compatible arches like i486 - i686? I have a system where I used the wrong stage-3 and now I'm stuck with an i486 CHOST on an Atom netbook where I could use every bit of performance. Regards, Florian signature.asc Description: OpenPGP digital signature
