Re: cygport cross-compiling beta1
On 7/25/2010 5:01 AM, Yaakov (Cygwin/X) wrote: > On Sat, 2010-07-24 at 00:26 -0400, Charles Wilson wrote: >> What does gentoo do with cross compilers and sysroots? I know >> ebuild/emerge supports them; do they treat them strictly as support for >> cross compiles, or as installable images on the intended $host? I >> suspect the latter... > > Not AFAICS: > > http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=3#doc_chap2 That's sad. It's also limiting. It means that you *can't* -- using gentoo's crossdev framework -- build a -dev package for the $host, that someone ELSE could use for native development ON $host. I fear this would also be a limitation of any cross.cygclass | deploy.cygclass bifurcation we cooked up. > OTOH, cygport is NOT a package manager, it is a build tool which creates > package tarballs, which are easily unpacked on other systems. So I > wouldn't mind making a separate cygclass for that purpose, which would > also allow for $prefix customization. In fact, it's basically ready, > but I'm not sure what to call it[1]. > > [1] http://cygwin.com/ml/cygwin-apps/2010-07/msg00174.html But, could I use $WHATEVER.cygclass to build libfoo for the $host libbar for the $host -- when libbar depends on libfoo? I suspect the only way to make this work would be to have TWO $host copies of libfoo: The deploy-on-$host version, which the end user would need. But also the local $build.cross version, in order to link libbar-for-deploy-on-$host. But then you get into issues with relinking and embedded rpaths (at least for ELF; not so much for PE/COFF). It really seems like TRTTD is to make the two trees one, and *fix* the issues with libtool, pkgconfig, and foo-config. OTOH, we're just a tiny little FOSS project; if the Big Boys haven't solved this by now, and even gentoo, fedora, and debian have all given up on the project...would we be tilting at windmills to try to solve it with our limited resources here? >> We could go that route as well; it just strikes me as a shame to "give >> up" since win32 is /right there/ underneath cygwin all along... >> >> OTOH, even on fedora-x86, it'd be nice to be able to map $sysroot$prefix >> to some specific wine path (identity mount "inside" wine, as $prefix?), >> and "run" the binaries "in place". That would let you, in some cases, >> run package test suites and the like. > > For testing MinGW binaries, because they see only Windows paths, *either > way* you have a problem running in the sysroot IF $prefix et al are used > in the code at runtime (/mingw != C:\mingw) or if the tests pass > Cygwin-native paths to the executable. Yes, I know. But...many times testsuites use only relative paths, which will work fine (as long as you don't use symlinks in your source tree for some reason). It won't ALWAYS work, but at least we wouldn't be deliberately creating MORE breakages, through our choice of cross compile strategies. > If not, either because there is > no runtime data or modules to find, or because it is coded to be > relocatable (at least on Windows, which is not uncommon), then you > should still be able to test-run cross-built programs using sysroot > libraries by appending $sysroot$prefix/bin to path. That might very > well be why Fedora moves libgcc and friends into the sysroot. Yes, probably so. > AFAICS the only thing you lose by configuring --prefix=$sysroot$prefix > is transferability between the sysroot and the host, IOW: Which would seem to me to be a pretty big deal. I mean, what's the point of cross compiling something if you can never use it on the $host? Sure, you could compile all the prereqs for, say, GIMP, to create a suitable compilation environment/sysroot on $build, so that you CAN then finally go ahead an compile GIMP itself for $host. But I'd assume at that point you'd want to copy GIMP over TO the $host and use it. And that means you need -- what? a SECOND version of all of those same prereqs, only this time compiled in a non-sysroot manner, so that the runtime deps of GIMP can also be deployed over on $host? That seems...excessive. (And I'm assuming that all the deps of GIMP are simple; they don't actually have interacting dependencies with each other. That makes things even MORE complicated, and unfortunately my assumption is counterfactual.) > 1) sysroot-configured libraries would not run on the host if runtime > data/module paths are hard-coded; Well, at least .la files can be postprocessed for deployment. No such luck for compiled-in paths. There's a strong feeling among some win32 folks that any "properly" ported app will be $prefix-independent; that it will always deduce its own location and use relative paths exclusively. Nice idea, but the FOSS universe isn't there yet. > 2) sysroot-configured ELF libraries would have an extra RPATH pointing > to the sysroot, which IIUC should be harmless (could this be mitigated > by patching libtool to not use $hardcode_libdir_flag_spec wh
Re: cygport cross-compiling beta1
On 7/22/2010 8:47 PM, Yaakov (Cygwin/X) wrote: > On Tue, 2010-07-20 at 00:53 -0400, Charles Wilson wrote: >> Why shouldn't cygport allow that? > > Thinking about this further, I think the problem is that we're talking > about two entirely different use cases: > > 1) cross-compiling a package to install into the sysroot, to be used > solely on Cygwin for cross-compile other software, which would follow > Cygwin packaging guidelines and use the system-default prefix; > > 2) cross-compiling software for use on another system (along the lines > of canadian-cross.cygclass), which would follow the *host*'s packaging > guidelines, or that of the user (at least wrt prefix). > > These are two completely different scenarios and in order to cover both > of them w/o confusion, I think we should handle them separately, > particularly if (as I suspect) we are forced to prepend $CROSS_SYSROOT > to $prefix for case (1). My early tests with Paolo's libtool patches look promising. You configure with --prefix=$prefix (not sysroot), and libtool takes care of the rest. When libtool detects a cross build sysroot situation, you basically end up with a magic pattern in the deplibs variable of the .la file: (lib2.la depends on lib1.la): dependency_libs=' =/mingw/lib/lib1.la =/mingw/lib/lib1.la' libdir='/mingw/lib' (Also, -L and -R entities are similarly munged, to include the '=' marker for sysroot builds) When linking, libtool doesn't warn about "relocations" because it knows about sysroots. It all seems to work on the $build system as expected. My concern -- which I haven't tested -- is three-fold: 1) what happens when an old libtool sees that '=/path" pattern? Do all clients need to have an updated libtool, in order to be built against the libs even on the $build machine? 2) Can you deploy these .la's on the $host machine for futher, native development? How does libtool deal with the magic '=' symbol, when you DON'T have a sysroot are are building a client using a native compiler, against a lib that was built elsewhere using a cross compiler? 3) So, you build a product that uses libtool modules for dynamic loading, and it needs the .la files so that libltdl can do its thing. Will module .la's have this '=' symbol in them? If so, will that confuse libltdl? More testing to do... > So how about this: leave cross.cygclass to handle case (1), since that > was what it was originally designed for, and make a new cygclass for > case (2), sort of like a generalized canadian-cross.cygclass (which > would be removed). The API would look something like: > > _HOST="i686-pc-linux-gnu" > _PREFIX="/usr/local" # defaults to /mingw or /usr > inherit [...] > > The only question is what to use for ? DEPLOY ? DIST? -- Chuck
Re: cygport cross-compiling beta1
On 7/25/2010 2:00 AM, Yaakov (Cygwin/X) wrote: > On Sat, 2010-07-24 at 10:34 -0400, Charles Wilson wrote: >> Same problem. Using Yaakov's latest pre-built linux cross compiler, and >> the latest linux-glibc src package, I attempted to rebuild. >> >> First I had the same problem where -jN was too aggressive on my quad >> core box, so I backed that down to -j1. But then I had a link error at >> the very end of the build: > > Do you have the linux-i686-kernel-headers installed as well? You do > need those for both building and using glibc. Otherwise, I'm baffled; I > didn't do anything to build glibc beyond what is in my .cygport. I do. I'm stumped. I think the only thing left that might give some clues is a line-by-line comparison of the config.log and build log output, which is why I uploaded them. -- Chuck
Re: cygport cross-compiling beta1
On Sat, 2010-07-24 at 00:26 -0400, Charles Wilson wrote: > What does gentoo do with cross compilers and sysroots? I know > ebuild/emerge supports them; do they treat them strictly as support for > cross compiles, or as installable images on the intended $host? I > suspect the latter... Not AFAICS: http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=3#doc_chap2 > However, I take your point. Looking at the fedora-mingw-cross > distribution, it does appear that they intend to provide a cross > compilation environment *only*; if you intend to package something for > deployment to the $host (e.g. to win32), then that's a separate build > process, using something other than rpm-build. Correct, after all, RPM is, as it name indicates, a package manager, and shipping RPMs for Windows software wouldn't be very helpful (although 7-Zip does open them). OTOH, cygport is NOT a package manager, it is a build tool which creates package tarballs, which are easily unpacked on other systems. So I wouldn't mind making a separate cygclass for that purpose, which would also allow for $prefix customization. In fact, it's basically ready, but I'm not sure what to call it[1]. [1] http://cygwin.com/ml/cygwin-apps/2010-07/msg00174.html > We could go that route as well; it just strikes me as a shame to "give > up" since win32 is /right there/ underneath cygwin all along... > > OTOH, even on fedora-x86, it'd be nice to be able to map $sysroot$prefix > to some specific wine path (identity mount "inside" wine, as $prefix?), > and "run" the binaries "in place". That would let you, in some cases, > run package test suites and the like. For testing MinGW binaries, because they see only Windows paths, *either way* you have a problem running in the sysroot IF $prefix et al are used in the code at runtime (/mingw != C:\mingw) or if the tests pass Cygwin-native paths to the executable. If not, either because there is no runtime data or modules to find, or because it is coded to be relocatable (at least on Windows, which is not uncommon), then you should still be able to test-run cross-built programs using sysroot libraries by appending $sysroot$prefix/bin to path. That might very well be why Fedora moves libgcc and friends into the sysroot. AFAICS the only thing you lose by configuring --prefix=$sysroot$prefix is transferability between the sysroot and the host, IOW: 1) sysroot-configured libraries would not run on the host if runtime data/module paths are hard-coded; 2) sysroot-configured ELF libraries would have an extra RPATH pointing to the sysroot, which IIUC should be harmless (could this be mitigated by patching libtool to not use $hardcode_libdir_flag_spec when $libdir is in $sys_lib_search_path_spec?); 3) host native-built libraries would need $sysroot prepended to paths in their foo-config/.la/.pc files (none of which should affect using host libc's for bootstrapping) to be usable in the sysroot, and vice-versa for sysroot-configured libs to be used on the host. But if you're only using sysroots for libraries, and you treat building-for-host and building-for-sysroot as two different things, then none of that should really matter. > See, my experience with "sys-roots" have been the "place where I install > all the crap I will turn into the system image on embedded-platform-X". > And that includes $sysroot$prefix/bin/busybox, etc. But if you're making a system image, then there will anyways be a lot of stuff in the sysroot that you will not want on an embedded platform (headers, .a and .la libs, pkg-config files, foo-config scripts, extra docs, unused l10n, etc). Gentoo's approach seems to be that you use a different directory for a staging area, copying only what you need from the sysroot (shared libs, essential runtime data) into there. > Right, that's a better approach. However, you'd probably still need to > patch foo's foo-config script itself, so that it also accepts a > --sysroot option, right? Only if you configure foo with --prefix=$prefix, not with --prefix= $sysroot$prefix. > And, the configure.ac for packages that are clients of libfoo, may also > need to be patched to support --with-foo-config=a-path if they don't > already. But again, that would be acceptable upstream. If foo-config is found with AC_(CHECK|PATH)_PROG, which it should be, then you just pass FOO_CONFIG=$CROSS_SYSROOT$prefix/bin/foo-config to configure. A patch to do *that* would almost certainly be accepted and would be simpler than --with-foo-config. Which makes me think that perhaps I should provide definitions for CROSS_SYSROOT_{BIN,INCLUDE,LIB}DIR to make the $prefix handling opaque. > True, but foo-config + sysroot is broken. That's not ideal either. > However: how many packages, that either provide or use foo-config > scripts, are *WE* planning to build/support using the cygwin cross > toolchains, immediately? $ ls -1 /usr/bin/*[-_]config | wc -l 99 Now, seven of those are Cygwin service installation
Re: cygport cross-compiling beta1
On Sat, 2010-07-24 at 10:34 -0400, Charles Wilson wrote: > Same problem. Using Yaakov's latest pre-built linux cross compiler, and > the latest linux-glibc src package, I attempted to rebuild. > > First I had the same problem where -jN was too aggressive on my quad > core box, so I backed that down to -j1. But then I had a link error at > the very end of the build: Do you have the linux-i686-kernel-headers installed as well? You do need those for both building and using glibc. Otherwise, I'm baffled; I didn't do anything to build glibc beyond what is in my .cygport. Yaakov
Re: cygport cross-compiling beta1
On 7/24/2010 12:32 AM, Charles Wilson wrote: > On 7/19/2010 9:49 PM, Yaakov (Cygwin/X) wrote: >> On Sat, 2010-07-17 at 16:26 -0400, Charles Wilson wrote: > > [linux toolchain] > I'll try again, only this time using *your* linux-gcc pre-compiled > compiler package. Same problem. Using Yaakov's latest pre-built linux cross compiler, and the latest linux-glibc src package, I attempted to rebuild. First I had the same problem where -jN was too aggressive on my quad core box, so I backed that down to -j1. But then I had a link error at the very end of the build: i686-pc-linux-gnu-gcc -nostdlib -nostartfiles -r -o /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os '-Wl,-(' /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/dl-allobjs.os /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/rtld-libc.a -lgcc '-Wl,-)' \ -Wl,-Map,/usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os.map i686-pc-linux-gnu-gcc -nostdlib -nostartfiles -shared \ -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -Wl,-z,defs -Wl,--verbose 2>&1 | \ LC_ALL=C \ sed -e '/^=/,/^=/!d;/^=/d'\ -e 's/\. = .* + SIZEOF_HEADERS;/& _begin = . - SIZEOF_HEADERS;/' \ > /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/ld.so.lds i686-pc-linux-gnu-gcc -nostdlib -nostartfiles -shared -o /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/ld.so\ -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -Wl,-z,defs \ /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os -Wl,--version-script=/usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/ld.map \ -Wl,-soname=ld-linux.so.2 -T /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/ld.so.lds /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os: In function `.L125': rtld.c:(.text+0x1149): undefined reference to `__open' rtld.c:(.text+0x1328): undefined reference to `__access' /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os: In function `dl_main': rtld.c:(.text+0x225a): undefined reference to `__access' /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os: In function `lose': dl-load.c:(.text+0x4c28): undefined reference to `__close' /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os: In function `open_verify': dl-load.c:(.text+0x4d92): undefined reference to `__open' dl-load.c:(.text+0x4da6): undefined reference to `rtld_errno' /usr/lib/gcc/i686-pc-linux-gnu/4.5.1/../../../../i686-pc-linux-gnu/bin/ld: /usr/src/devel/cross/linux/linux-i686-glibc-2.11.2-1/build/elf/librtld.os: relocation R_386_GOTOFF against undefined hidden symbol `rtld_errno' can not be used when making a shared object /usr/lib/gcc/i686-pc-linux-gnu/4.5.1/../../../../i686-pc-linux-gnu/bin/ld: final link failed: Bad value collect2: ld returned 1 exit status I've put the config.log and the -compile.log here (even lzma compressed, they are over 100K): http://cygwin.cwilson.fastmail.fm/ITP/logs.lzma -- Chuck
Re: cygport cross-compiling beta1
On 7/19/2010 9:49 PM, Yaakov (Cygwin/X) wrote: > On Sat, 2010-07-17 at 16:26 -0400, Charles Wilson wrote: [linux toolchain] >> I don't think either of these two lines belong in the cygport: >> >> mv ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/lib/libgcc_s.a >> ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/${PV[1]}.*/ >> # mv ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/lib32/libgcc_s.a >> ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/4.6.0/32/ >> >> Because there doesn't seem to be any libgcc_s.a file anywhere in the >> build or install trees: > > You are correct; copy-and-paste errors. still present in latest -src tarball. >> Anyway, building binutils and gcc went mostly ok, but building glibc did >> not. > > I am unable to reproduce this, even after cleaning out my sysroot. > While I was at it, I did push an updated gcc and rebuilt glibc, in case > you want to try again. I d/led the latest linux-gcc-src tarball, and rebuilt it from scratch. Then I d/led the latest glibc src tarball, and...same problem. I'll try again, only this time using *your* linux-gcc pre-compiled compiler package. -- Chuck
Re: cygport cross-compiling beta1
On 7/23/2010 1:54 AM, Yaakov (Cygwin/X) wrote: > On Thu, 2010-07-22 at 22:41 -0400, Charles Wilson wrote: >> OK, so the libtool stuff is still percolating. Paolo just posted his >> patch series earlier today, so I haven't had a chance to test it out >> yet. However, it APPEARS after a cursory glance to work like this: >> >> 1) you configure stuff with --prefix=${CROSS_SYSROOT}${prefix} > > Not according to his sysroot.at (patch 5/8). He adds a new configure > option, so you configure with --with-sysroot=${CROSS_SYSROOT} --prefix= > ${prefix}. Ah, you're right. I'm sure this will make more sense once I get a chance to actually test it. Looks like Sunday afternoon. >> The reason I think this is, I'm concerned about *hardcoding* the >> $sysroot$prefix values into the compiled binaries! (e.g. 'strings >> foo.exe | grep $sysroot' might return non-empty, and that would be bad, >> when foo.exe is deployed on the $host system in $prefix. > > If $sysroot needs to be prepended to $prefix, then that strengthens the > argument for considering cross-compiling-for-sysroot and > cross-compiling-for-host two separate use cases. Ah, but I was wrong, so my incorrect analysis can hardly be said to strengthen anything. What does gentoo do with cross compilers and sysroots? I know ebuild/emerge supports them; do they treat them strictly as support for cross compiles, or as installable images on the intended $host? I suspect the latter... However, I take your point. Looking at the fedora-mingw-cross distribution, it does appear that they intend to provide a cross compilation environment *only*; if you intend to package something for deployment to the $host (e.g. to win32), then that's a separate build process, using something other than rpm-build. We could go that route as well; it just strikes me as a shame to "give up" since win32 is /right there/ underneath cygwin all along... OTOH, even on fedora-x86, it'd be nice to be able to map $sysroot$prefix to some specific wine path (identity mount "inside" wine, as $prefix?), and "run" the binaries "in place". That would let you, in some cases, run package test suites and the like. >> One thing I thought of; if you want to run foo-config scripts from >> /usr/i686-pc-mingw32/sys-root/mingw/bin during configure, then you >> pretty much have to set $PATH so that $sysroot$prefix/bin precedes your >> own /usr/bin -- otherwise, you'd get /usr/bin/foo-config. >> >> But...what if you're creating a REAL, full sysroot. Like, with >> coreutils and everything, over there in $sysroot$prefix/bin. Do you >> REALLY want the cross-$host version of 'uname' to precede your own? > > No, but packages such as coreutils which only provide programs don't > belong in a sysroot. The purpose of a sysroot is for libraries which > you use to cross-compile other software which depend on them. See, my experience with "sys-roots" have been the "place where I install all the crap I will turn into the system image on embedded-platform-X". And that includes $sysroot$prefix/bin/busybox, etc. > But for this very reason, adding $sysroot$prefix/bin to PATH is > unhelpful at best and maybe even dangerous. Well, yeah, that was kinda my point. On cygwin-x-mingw, I might want to do that for limited times and purposes, but not ordinarily. And doing it routinely just so that "configure" will "work" is...fraught with peril. > If a package's configure > depends on a foo-config script, then it should provide a way to specify > the path to that script, by passing either a --with-foo-config=$sysroot > $prefix/bin option or FOO_CONFIG=$sysroot$prefix/bin value to configure. Right, that's a better approach. However, you'd probably still need to patch foo's foo-config script itself, so that it also accepts a --sysroot option, right? Otherwise, even $sysroot$prefix/bin/foo-config --cflags will return something like -I$prefix/include/foo, when you really need -I$sysroot$prefix/include/foo. OTOH, a patch to foo-config to support sysroot is quite likely to be accepted upstream. (I know, you don't like this; see below). And, the configure.ac for packages that are clients of libfoo, may also need to be patched to support --with-foo-config=a-path if they don't already. But again, that would be acceptable upstream. Here's how vlc does it, for dvdnav (where VLC_ADD_CFLAGS and VLC_ADD_LIBS do what you would expect; adds the argument to CFLAGS and LIBS): dnl dnl libdvdnav plugin dnl AC_ARG_ENABLE(dvdnav, [ --enable-dvdnav dvdnav input module (default enabled)]) if test "${enable_dvdnav}" != "no" then dnl prepend -ldvdcss on OS that need it AS_CASE(["${SYS}"], [mingw32|darwin|beos], [VLC_ADD_LIBS([dvdnav], [-ldvdcss])]) DVDNAV_PATH="${PATH}" AC_ARG_WITH(dvdnav-config-path, [ --with-dvdnav-config-path=PATH dvdnav-config path (default search in \$PATH)], [ if test "${with_dvdnav_config_path}" != "no" then DVDNAV_PATH="${with_dvdnav_config_path}:${PATH}"
Re: cygport cross-compiling beta1
On Fri, 2010-07-23 at 16:32 +0100, Dave Korn wrote: > I must be missing something. Shouldn't what's under the sysroot be > basically an exact copy with the same layout as what would be on the host's > own native filesystem? That's what I get from the description of > --with-sysroot at http://gcc.gnu.org/install/configure.html#TOC3. Well, sort of. As far as GCC is concerned, yes: the default search paths become $sysroot$prefix/include and $sysroot$prefix/lib (where prefix=/mingw for *-*-mingw*, and /usr for most other systems), so you can for example take a linux glibc package, unpack it under the linux sysroot, and voila, you're ready to build a linux cross-gcc. (In fact, that's exactly what I did in order to bootstrap.) pkg-config also provides support for sysroots with an environment variable, so a package built with prefix=/usr unpacked under the sysroot will produce -I$sysroot/usr/include/foo CFLAGS (and no -L flags in the usual case where the library is in $sysroot/usr/lib). So ideally, one could cross-configure a package with --prefix=$prefix, build it w/o any hard-coding of the sysroot, then either install with DESTDIR=$D$sysroot and use it in your sysroot, or with DESTDIR=$D and use it on the host. Unfortunately, libtool isn't (yet?) aware of sysroots, so when a package is configured with --prefix=/usr, it hardcodes the libdir as /usr/lib and takes that literally, adding -L/usr/lib flags which cause the build to fail. The other problem is foo-config scripts (installed into $sysroot $prefix/bin), most of which are prone to adding -I/usr/include* and -L/usr/lib flags, which GCC will accept literally, causing it to compile and (try to) link with the native version of the dependency instead of the cross-compiled one in the sysroot. For these reasons, Fedora's mingw packages build with --prefix=$sysroot $prefix[1]. GCC's sysroot still helps: you can #include and link with -lfoo w/o -I and -L, and you can still drop a native-built package into the sysroot (providing it doesn't have foo-config, .la, or .pc files). But it does mean that building for the sysroot and building for usage on the host are two different things. Yaakov [1] https://fedoraproject.org/wiki/Packaging/MinGW
Re: cygport cross-compiling beta1
On 23/07/2010 01:47, Yaakov (Cygwin/X) wrote: > Thinking about this further, I think the problem is that we're talking > about two entirely different use cases: > > 1) cross-compiling a package to install into the sysroot, to be used > solely on Cygwin for cross-compile other software, which would follow > Cygwin packaging guidelines and use the system-default prefix; > > 2) cross-compiling software for use on another system (along the lines > of canadian-cross.cygclass), which would follow the *host*'s packaging > guidelines, or that of the user (at least wrt prefix). > > These are two completely different scenarios I must be missing something. Shouldn't what's under the sysroot be basically an exact copy with the same layout as what would be on the host's own native filesystem? That's what I get from the description of --with-sysroot at http://gcc.gnu.org/install/configure.html#TOC3. cheers, DaveK
Re: cygport cross-compiling beta1
On Thu, 2010-07-22 at 22:41 -0400, Charles Wilson wrote: > OK, so the libtool stuff is still percolating. Paolo just posted his > patch series earlier today, so I haven't had a chance to test it out > yet. However, it APPEARS after a cursory glance to work like this: > > 1) you configure stuff with --prefix=${CROSS_SYSROOT}${prefix} Not according to his sysroot.at (patch 5/8). He adds a new configure option, so you configure with --with-sysroot=${CROSS_SYSROOT} --prefix= ${prefix}. > The reason I think this is, I'm concerned about *hardcoding* the > $sysroot$prefix values into the compiled binaries! (e.g. 'strings > foo.exe | grep $sysroot' might return non-empty, and that would be bad, > when foo.exe is deployed on the $host system in $prefix. If $sysroot needs to be prepended to $prefix, then that strengthens the argument for considering cross-compiling-for-sysroot and cross-compiling-for-host two separate use cases. > One thing I thought of; if you want to run foo-config scripts from > /usr/i686-pc-mingw32/sys-root/mingw/bin during configure, then you > pretty much have to set $PATH so that $sysroot$prefix/bin precedes your > own /usr/bin -- otherwise, you'd get /usr/bin/foo-config. > > But...what if you're creating a REAL, full sysroot. Like, with > coreutils and everything, over there in $sysroot$prefix/bin. Do you > REALLY want the cross-$host version of 'uname' to precede your own? No, but packages such as coreutils which only provide programs don't belong in a sysroot. The purpose of a sysroot is for libraries which you use to cross-compile other software which depend on them. But for this very reason, adding $sysroot$prefix/bin to PATH is unhelpful at best and maybe even dangerous. If a package's configure depends on a foo-config script, then it should provide a way to specify the path to that script, by passing either a --with-foo-config=$sysroot $prefix/bin option or FOO_CONFIG=$sysroot$prefix/bin value to configure. And no, I'm not about to start inventing paths or moving files around either. > I think that's probably easier than patching every package that ships a > foo-config, so that foo-config accepts a -sysroot option. But...a patch > adding a -sysroot option to a foo-config would probably be accepted by > the upstream maintainers of foo. Not necessarily, and requiring a patch for every single package which ships a foo-config is less than ideal. This is, as I noted earlier, the other area which makes configuring without $sysroot in $prefix difficult. They are prone to adding -I and -L flags which will point to the native root instead of the sysroot, which will break the build at link time at least, if not earlier. If the libtool patch is accepted (and that's an if, given upstream's track record on patch review), and this remains the only other issue to using --prefix=$prefix, then perhaps adding a cross_sysrootize() function which will do the trick: src_install() { cd ${B} cyginstall cross_sysrootize ${D}${CROSS_SYSROOT}/usr/bin/foo-config } where cross_sysrootize essentially does: for f in $@; do sed -i -e "s|\([^t]\)$prefix|\1$CROSS_SYSROOT$prefix|g" $f; done (the leading [^t] to avoid paths which somehow already had sys-root) AFAICS from the *-config scripts I have in /usr/bin, that would appear to do the trick w/o breaking anything, but it would need some testing. > Now, as far as using the cross.cygclass to build non-core stuff. Using > my original mingw-{zlib,bzip2,xz,gpg-error,gcrypt} cygports and Yaakov's > versions, I've created local versions of all those packages using the > new tools and the i686-pc-mingw32 compiler. > > It all seemed to go pretty smoothly; my cygports are still uglier than > Yaakov's, but that's partly because I was trying to maintain strict > one-for-one compatibility with the old versions (mgwbz2_1.dll names, and > all). Later on, we might think about breaking backwards compat on those > packages (esp. the DLLs, since nobody uses them yet) and make "ours" > more normal, and the cygports more clean. But not yet. Those "mgw" prefixes need to GO AWAY. Period. NOW is the time to fix these DLL names, when were anyway completely changing mingw support, because they were wrong to begin with. Cross-compiled sysroot libraries must always be compatible with the native versions in name and ABI, otherwise you completely defeat the purpose. > 2) For ALL of the mingw-{zlib,bzip2,xz,gpg-error,gcrypt} packages, I > made sure that each was compiled with -mms-bitfields. All of the > *-mingw32-* packages shipped by mingw.org are built that way, even > though it is not the default for the compiler. If mingw.org is using it for its packages then so must we; that's part of ABI compatibility after all. > I really think it ought to BE the default for *mingw32*, but that train > has long left the station. What about adding it to mingw*-gcc specs? If not, then I'll have to add it automatically to CFLAGS and friends. JonY, what about
gcc-3 -mno-cygwin with sysroot mingw runtime [Was: Re: cygport cross-compiling beta1]
On 7/20/2010 8:46 PM, Yaakov (Cygwin/X) wrote: > On Tue, 2010-07-20 at 10:51 -0400, Charles Wilson wrote: >> Well, I guess the replacement package for the gcc(3)-mingw stuff can >> just create symlinks: >> /usr/lib/mingw -> /usr/i686-pc-mingw32/sys-root/mingw/lib >> /usr/include/mingw -> /usr/i686-pc-mingw32/sys-root/mingw/include >> (or, rather, gcc-mingw-* could simply be obsoleted -- and the >> replacement version having a postinstall script that does what the >> gcc-mingw-* preremove scripts SHOULD have done, which is to remove the >> /usr/i686-pc-mingw32/{bin,include/lib} symlinks, and a new repackaged >> version of gcc(3)-core could generate the /usr/{lib,include}/mingw >> symlinks itself. > > If gcc-mingw* go away then you don't need /usr/{include,lib}/mingw, > because -mno-cygwin will no longer work, right? > > Either way, changing /usr/{include,lib}/mingw from directories into > symlinks is not so simple, as the current mingw-* packages all install > into there. > >> However, depending on how the system includes are ordered, gcc3 >> -mno-cygwin will then either use the "cygwin" w32api in >> /usr/include/w32api, or the cross versions mixed in to >> /usr/i686-pc-mingw32/sys-root/mingw/include. > > How would gcc3 know about the mingw sysroot? So, here's how I got a working 'gcc-3 -mno-cygwin' and 'g++-3 -mno-cygwin' with the new sysroot-style mingw-runtime and mingw-w32api. ::: technically, steps #1--#3 below have to be done before you try ::: to install, build, or use the new i686-mingw cross toolchain, but ::: I list them here for completeness. 1) Upgraded the old mingw-runtime-3.18-1 (in /usr/{lib,include}/mingw to the new cross-compiled mingw-runtime-3.18-2 (in /usr/i686-pc-mingw32/sys-root/mingw/{lib,include}. 2) Uninstalled gcc-mingw-core-20050522-1 gcc-mingw-g++-20050522-1 gcc-mingw-g77-20050522-1 gcc-mingw-java-20050522-1 and basically any other gcc-mingw-* you might have installed. setup.exe will whine about this, because these packages are listed as requirements of the corresponding gcc-X-3.4.4-999 package(s). However, as far as I can tell, the regular cygwin personality of those compilers works just fine, even without the gcc-mingw-* packages, so long as you don't try to use -mno-cygwin. 3) Fixup some stuff that the gcc-mingw-* preremove scripts should have done: rm -f /usr/i686-pc-mingw32/{lib,include,bin} 4) Upgraded ALL of the following: /usr/*/mingw $sysroot$prefix mingw-zlib 1.2.3-10 -> 1.2.5-2 mingw-zlib-devel1.2.3-10 -> 1.2.5-2 mingw-zlib0 1.2.3-10 -> 1.2.5-2 mingw-bzip2 1.0.5-10 -> 1.0.5-12 mingw-libbz2-devel 1.0.5-10 -> 1.0.5-12 mingw-libbz2_1 1.0.5-10 -> 1.0.5-12 mingw-liblzma-devel 4.999.9beta-11 -> 4.999.9beta-13 mingw-liblzma1 4.999.9beta-11 -> 4.999.9beta-13 mingw-xz4.999.9beta-11 -> 4.999.9beta-13 mingw-libgpg-error 1.7-1-> 1.8-2 mingw-libgpg-error-devel1.7-1-> 1.8-2 mingw-libgpg-error0 1.7-1-> 1.8-2 mingw-libgcrypt 1.4.5-1 -> 1.4.5-3 mingw-libgcrypt-devel 1.4.5-1 -> 1.4.5-3 mingw-libgcrypt11 1.4.5-1 -> 1.4.5-3 At that point, /usr/include/mingw and /usr/lib/mingw were both empty. 5) rm -f /usr/lib/mingw /usr/include/mingw 6) Now, manually download gcc-mingw-core and gcc-mingw-g++ (Yes, I know, we just uninstalled them, but run with me here). I didn't test any of the other languages, and they may or may not work; I dunno. But I was able to get C and C++ working, so I figure that's sufficient for now. wget http://mirrors.kernel.org/sourceware/cygwin/release/gcc-mingw/gcc-mingw-core/gcc-mingw-core-20050522-1.tar.bz2 wget http://mirrors.kernel.org/sourceware/cygwin/release/gcc-mingw/gcc-mingw-g++/gcc-mingw-g++-20050522-1.tar.bz2 These packages are constructed in a very odd way; I've never really understood why. But, the contents of each tarball look like this: etc/postinstall/gcc-mingw-core-3.4.4-20050522-1.tgz etc/postinstall/gcc-mingw-core.sh etc/preremove/gcc-mingw-core.sh etc/postinstall/gcc-mingw-g++-3.4.4-20050522-1.tgz etc/postinstall/gcc-mingw-g++.sh etc/preremove/gcc-mingw-g++.sh We're interested in the inner tarballs, so unpack the outer ones someplace safe, like /tmp. 7) The inner tarballs have contents like this: gcc-mingw-core-3.4.4-20050522-1.tgz lib/gcc/i686-pc-mingw32/3.4.4/cc1.exe lib/gcc/i686-pc-mingw32/3.4.4/collect2.exe lib/gcc/i686-pc-mingw32/3.4.4/crtbegin.o lib/gcc/i686-pc-mingw32/3.4.4/crtend.o lib/gcc/i686-pc-mingw32/3.4.4/include/README lib/gcc/i686-pc-mingw32/3.4.4/include/emmintrin.h lib/g
Re: cygport cross-compiling beta1
On Tue, 2010-07-20 at 00:53 -0400, Charles Wilson wrote: > But at SOME point, SOME part of what you've built on $host is supposed > to be used, eventually, by somebody, on $target, right? > > Where should THAT live? > > If I'm on linux and have a devel environment, I can always compile with > --prefix=/home/me/my-builds even if I don't have root. But...with this > cross.cygchain, I can't do that from my cross-host; I'm trapped in the > thou-shalt-have-root prison and to USE anything I build, I *must* build > with --prefix=/usr -- and then I can't use it on the $target without > root privilege. > > rpm-build doesn't restrict you that way; sure, it *defaults* to /usr, > but you CAN override (yes, there are issues on the 'rpm --install' side > wrt. root, but there are also workarounds for that). I'm not sure about > ebuild, but I *think* you can override it, too. > > Why shouldn't cygport allow that? Thinking about this further, I think the problem is that we're talking about two entirely different use cases: 1) cross-compiling a package to install into the sysroot, to be used solely on Cygwin for cross-compile other software, which would follow Cygwin packaging guidelines and use the system-default prefix; 2) cross-compiling software for use on another system (along the lines of canadian-cross.cygclass), which would follow the *host*'s packaging guidelines, or that of the user (at least wrt prefix). These are two completely different scenarios and in order to cover both of them w/o confusion, I think we should handle them separately, particularly if (as I suspect) we are forced to prepend $CROSS_SYSROOT to $prefix for case (1). So how about this: leave cross.cygclass to handle case (1), since that was what it was originally designed for, and make a new cygclass for case (2), sort of like a generalized canadian-cross.cygclass (which would be removed). The API would look something like: _HOST="i686-pc-linux-gnu" _PREFIX="/usr/local" # defaults to /mingw or /usr inherit [...] The only question is what to use for ? Yaakov
Re: cygport cross-compiling beta1
On Tue, 2010-07-20 at 00:53 -0400, Charles Wilson wrote: > http://www.mail-archive.com/libtool-patches-mXXj517/z...@public.gmane.org/msg05488.html > Paolo Bonzini mentioned that he had a different patch, and promised to > rebase to latest and repost. He didn't. I pinged him. Hopefully that comes soon, because without that, I don't see any choice but to add $CROSS_SYSROOT to $prefix. FWIW Fedora seems to do the same, and now I see why. :-( > I'll release a new cygwin libtool with whatever comes of that > discussion, as soon as it gets even close to a resolution. While you're at it, will this include support for -flto (without -Wc,)? With gcc 4.5 on the way (for several platforms) this will be important. Also, I mentioned -fstack-protector* on bug-libtool: http://thread.gmane.org/gmane.comp.gnu.libtool.bugs/7330/focus=7341 But as I noted in the next message in that thread, that doesn't help with tag=CXX. I'm not sure how to solve that short of not interfering with g++ and trusting it to link libstdc++ and libgcc without libtool's help (what a concept!). > Don't care. Fedora should be used as a guide, not a cage. Not only does it provide precedence, but now that we understand fully *why* they package the way they do, it should be easier to accept it. > But at SOME point, SOME part of what you've built on $host is supposed > to be used, eventually, by somebody, on $target, right? > > Where should THAT live? Certainly not in the sysroot (only libraries need to be there), and possibly not even built by cygport. For example, if you're cross-compiling a program to be used by others on Fedora, you would probably use rpm to build it, no? > If I'm on linux and have a devel environment, I can always compile with > --prefix=/home/me/my-builds even if I don't have root. But...with this > cross.cygchain, I can't do that from my cross-host; I'm trapped in the > thou-shalt-have-root prison and to USE anything I build, I *must* build > with --prefix=/usr -- and then I can't use it on the $target without > root privilege. And if it was prefix=/usr/local you would still need root privileges to install (at least I sure hope so), and then you get the joys of adjusting PATH and LD_LIBRARY_PATH for /usr/local. (Just went through that dealing with Solaris.) The only case where you avoid this is with a $HOME prefix, which is naturally going to be good for only personal use. > Wherever the lookup (english) string hasn't changed, the correct > translation will be found. Wherever it HAS changed, then...you get the > english version. > > I suppose that is no worse than all-english-all-the-time, in the case of > --disable-nls. The more divergent the two versions of gcc, the more > english and the less i18n you get. > > What I was worried about was *incorrect* translations, but it doesn't > appear that will happen. Not AFAIK. > Nope, I was using your cygports, with only the changes I posted in my > previous email; thus, the mingw64-i686-gcc-4.5.20100708-1.cygport I used > was directly from your cygport-cross-examples.tar.bz2. I got (this is > my reverse-sorted list of the files in the tarball): > > ... > usr/i686-w64-mingw32/lib/lib32/libiberty.a > ... > > I can send the logs, if you like. Well, AFAICS *that* is wrong, nor is that how it is in my test package. I'll have to try rebuilding it again. > But...somebody out there might have (cygwin) code that doesn't compile > with gcc4. They ought to fix their code, but...this is not an ideal world. Distros maintain patches for still-in-use older software to fix compatibility with new GCCs. Otherwise, yes, they'll actually have to fix their code. But let's face it, most distros don't support gcc-3.4 anymore, certainly not as a primary compiler, so why must we, particularly given our limited resources? Yaakov
Re: cygport cross-compiling beta1
On 7/20/2010 9:37 AM, Dave Korn wrote: > On 20/07/2010 06:26, Charles Wilson wrote: >> On 7/19/2010 9:55 PM, JonY wrote: With NLS you will still have at least partial translations, which is better than nothing, no? >>> How about setting up --with-localedir to somewhere version or target >>> specific? >> >> There isn't a '--with-localedir' option. But...there IS a --localedir >> one. How did I miss that? >> >> There's still one issue that Yaakov brought up; it's not clear that >> libintl will actually USE the correct version of the locale files...but >> some experimentation should answer that question. > > libintl looks in whatever localedir arg you pass to the bindtextdomain > function at startup. In GCC, that's LOCALEDIR which is correctly passed in > from $(localedir) in the Makefile. So, it ought to work just fine. Now I remember. It's that localedir is not passed down to the subconfigures. After using --localedir, here are the strings embedded in the gcc binary: $ strings i686-pc-mingw32-gcc-4.5.1.exe | grep locale /usr/src/devel/cross/mingw/mingw-gcc-4.5.20100701-1/src/gcc-4.5-20100701/configure --srcdir=/usr/src/devel/cross/mingw/mingw-gcc-4.5.20100701-1/src/gcc-4.5-20100701 --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib --datadir=/usr/share --localstatedir=/var --sysconfdir=/etc --datarootdir=/usr/share --docdir=/usr/share/doc/mingw-gcc --build=i686-pc-cygwin --host=i686-pc-cygwin --target=i686-pc-mingw32 --with-sysroot=/usr/i686-pc-mingw32/sys-root --with-build-sysroot=/usr/i686-pc-mingw32/sys-root --disable-multilib --disable-sjlj-exceptions --enable-lto --enable-languages=c,c++,fortran,objc,obj-c++ --enable-libgomp --enable-version-specific-runtime-libs --disable-win32-registry --with-dwarf2 --localedir=/usr/i686-pc-mingw32/share/locale /usr/share/locale setlocale So, you can see that the (top level) configure settings did specify localedir, and got recorded. But...the only other string with 'locale' in it is "/usr/share/locale" Oops. So that's not going to work, without upstream changes in the build, or local patches. -- Chuck
Re: cygport cross-compiling beta1
On Tue, 2010-07-20 at 08:43 -0400, Charles Wilson wrote: > On 7/19/2010 9:49 PM, Yaakov (Cygwin/X) wrote: > > AFAIK --enable-shared and --enable-libgomp are the defaults. > > Nope, apparently not. After making sure that pthread was installed in > /usr/i686-pc-mingw32/sys-root, and rebuilding gcc: I see, in toplevel configure.ac, target-libgomp is added to $noconfigdirs for all "non-POSIX" systems (including Cygwin!). So all cygwin and mingw* gcc's need an explicit --enable-libgomp. The documentation is not clear on that at all. Yaakov
Re: cygport cross-compiling beta1
On Tue, 2010-07-20 at 10:51 -0400, Charles Wilson wrote: > Well, I guess the replacement package for the gcc(3)-mingw stuff can > just create symlinks: > /usr/lib/mingw -> /usr/i686-pc-mingw32/sys-root/mingw/lib > /usr/include/mingw -> /usr/i686-pc-mingw32/sys-root/mingw/include > (or, rather, gcc-mingw-* could simply be obsoleted -- and the > replacement version having a postinstall script that does what the > gcc-mingw-* preremove scripts SHOULD have done, which is to remove the > /usr/i686-pc-mingw32/{bin,include/lib} symlinks, and a new repackaged > version of gcc(3)-core could generate the /usr/{lib,include}/mingw > symlinks itself. If gcc-mingw* go away then you don't need /usr/{include,lib}/mingw, because -mno-cygwin will no longer work, right? Either way, changing /usr/{include,lib}/mingw from directories into symlinks is not so simple, as the current mingw-* packages all install into there. > However, depending on how the system includes are ordered, gcc3 > -mno-cygwin will then either use the "cygwin" w32api in > /usr/include/w32api, or the cross versions mixed in to > /usr/i686-pc-mingw32/sys-root/mingw/include. How would gcc3 know about the mingw sysroot? Yaakov
Re: cygport cross-compiling beta1
On 7/20/2010 9:55 AM, Dave Korn wrote: On 20/07/2010 05:53, Charles Wilson wrote: But at SOME point, SOME part of what you've built on $host is supposed to be used, eventually, by somebody, on $target, right? Where should THAT live? On the target? I meant, *in what directory* on the target. As cross.cygclass is currently implemented, cross-compiled stuff may only be compiled-for, and installed in, the target's /usr tree, not /usr/local or /home/me/ or what-have-you. Re: libiberty.a, I have no idea why we even install that. We don't install the headers, and it has no fixed defined API or versioning, so it's basically useless. I wouldn't worry too much about where it ends up! IIRC you must have it available to link to libbfd.a and libopcodes.a, and we DO provide those and their headers (in the native toolchain). However, that's the $host version. The $target version...you're probably right. But...somebody out there might have (cygwin) code that doesn't compile with gcc4. They ought to fix their code, but...this is not an ideal world. Yeah, this is tricky. I don't mind if we end up telling people they can have *either* gcc-3 *or* x-mingw-gcc4 but not both at the same time. I guess it's worth testing if gcc-3 -mno-cygwin still works with up-to-date mingw cross-binutils and libs installed in place of the /usr/i686-pc-mingw32 symlinks. Well, I guess the replacement package for the gcc(3)-mingw stuff can just create symlinks: /usr/lib/mingw -> /usr/i686-pc-mingw32/sys-root/mingw/lib /usr/include/mingw -> /usr/i686-pc-mingw32/sys-root/mingw/include (or, rather, gcc-mingw-* could simply be obsoleted -- and the replacement version having a postinstall script that does what the gcc-mingw-* preremove scripts SHOULD have done, which is to remove the /usr/i686-pc-mingw32/{bin,include/lib} symlinks, and a new repackaged version of gcc(3)-core could generate the /usr/{lib,include}/mingw symlinks itself. However, depending on how the system includes are ordered, gcc3 -mno-cygwin will then either use the "cygwin" w32api in /usr/include/w32api, or the cross versions mixed in to /usr/i686-pc-mingw32/sys-root/mingw/include. Ditto with the system lib dirs, with respect to the w32api libs. I'll try it and see, later tonight. -- Chuck
Re: cygport cross-compiling beta1
On 20/07/2010 05:53, Charles Wilson wrote: > But at SOME point, SOME part of what you've built on $host is supposed > to be used, eventually, by somebody, on $target, right? > > Where should THAT live? On the target? > Then package an /etc/profile.d script that appends or prepends to > MANPATH and INFOPATH. > > Or assume some modicum of intelligence on the part of people with the > gumption to actually install a *cross compiler*, and allow them to > manually set INFOPATH/MANPATH as appropriate when they are using a given > cross tool. Either of these work for me, but each compiler should definitely have its own docs in its own prefix IMO. >>> Also, this packaging oddity: >>> mingw64-x86_64: usr/x86_64-w64-mingw32/lib64/libiberty.a >> x86_64-w64-mingw32 is potentially a multilib arch, hence the lib64. If >> the gcc were actually built multilib, you'd see a lib32 and lib64. >> >>> mingw64-i686: usr/i686-w64-mingw32/lib/lib32/libiberty.a >> You mean /usr/i686-w64-mingw32/lib/libiberty.a? The only way you would >> get lib32/libiberty.a (not lib/lib32) is with multilib, which I disabled >> in my builds. Re: libiberty.a, I have no idea why we even install that. We don't install the headers, and it has no fixed defined API or versioning, so it's basically useless. I wouldn't worry too much about where it ends up! >>> However, given the dependency structure of the gcc3 packages, this means >>> you pretty much have to also remove, using setup, >>> gcc-core >>> gcc-g++ >>> gcc-java >>> gcc-objc >>> >>> Note that this conflict means that the current gcc-mingw* packages (and, >>> thus, the gcc 3.x packages) conflict with the new mingw-gcc-* and >>> mingw-binutils-* packages. Also, this may have implications for the >>> upgrade process, when the "real" mingw32-gcc cross toolchain is >>> released. Even if we mark the gcc (3) packages obsolete, we probably >>> have to repackage them and their gcc-mingw* friends to fix this upgrade >>> problem, and perhaps relieve the conflict. Somehow? >> We can just remove the gcc-mingw* deps from gcc3*. But the bigger >> question is, do we still *need* gcc3 in the distro? Yes, we get mails every now and again where someone's relying on it to build a legacy app, I don't want to pull it. > Once we can build setup.exe with gcc4, then I don't believe we *need* > it. And we can *definitely* get rid of mingw-gcc*; at that point, gcc > -mno-cygwin breaks badly, and we get to say "it's been deprecated for > over a year, and now it is no longer supported. Use this spiffy new > cross compiler". > > But...somebody out there might have (cygwin) code that doesn't compile > with gcc4. They ought to fix their code, but...this is not an ideal world. Yeah, this is tricky. I don't mind if we end up telling people they can have *either* gcc-3 *or* x-mingw-gcc4 but not both at the same time. I guess it's worth testing if gcc-3 -mno-cygwin still works with up-to-date mingw cross-binutils and libs installed in place of the /usr/i686-pc-mingw32 symlinks. cheers, DaveK
Re: cygport cross-compiling beta1
On 20/07/2010 06:26, Charles Wilson wrote: > On 7/19/2010 9:55 PM, JonY wrote: >>> With NLS you will still have at least partial translations, which is >>> better than nothing, no? >>> >> How about setting up --with-localedir to somewhere version or target >> specific? > > There isn't a '--with-localedir' option. But...there IS a --localedir > one. How did I miss that? > > There's still one issue that Yaakov brought up; it's not clear that > libintl will actually USE the correct version of the locale files...but > some experimentation should answer that question. libintl looks in whatever localedir arg you pass to the bindtextdomain function at startup. In GCC, that's LOCALEDIR which is correctly passed in from $(localedir) in the Makefile. So, it ought to work just fine. (also, --disable-werror, and language Ada) >>> Just hadn't bothered with Ada, that's all. >>> >> Cygwin gcc needs to be bumped to around 4.5.0 to build it. Its stated >> somewhere in the GCC docs that you'll need the same version of gcc to >> build cross ADA. > > Yep, that's why it's on DaveK's plate. He'll probably keep the cygwin > native compiler and the mingw.org-based cross compiler in sync. I just couldn't get frankenAda to build any more at 4.5.0, so I'm aiming at fully cygwinising the language port. However in 4.5.0-1 Ada will be regressed, it'll take me until 4.5.0-2 to get it fixed up. > crash. (If you link with shared libgcc/libstdc++, you're ok). Hence why I don't use that option in the Cygwin C++ ABI. For us shared linking is the default. cheers, DaveK
Re: cygport cross-compiling beta1
On 7/19/2010 9:49 PM, Yaakov (Cygwin/X) wrote: > On Sat, 2010-07-17 at 16:26 -0400, Charles Wilson wrote: >> The following flags are used in the official mingw compiler, but not here: >> --enable-shared (but that's okay, as it is default) >> --enable-libgomp > > AFAIK --enable-shared and --enable-libgomp are the defaults. Nope, apparently not. After making sure that pthread was installed in /usr/i686-pc-mingw32/sys-root, and rebuilding gcc: ... The following languages will be built: c,c++,fortran,lto,objc,obj-c++ *** This configuration is not supported in the following subdirectories: target-libmudflap target-libgomp target-zlib (Any other directories should still work fine.) ... configure: Debug build flags set to -g3 -O0 checking for additional debug build... no configure: target-libgomp not built checking for parallel mode support... no ... Rebuilding again, this time with --enable-libgomp (and, as discussed in my reply to JonY, sadly without --enable-fully-dynamic-strings). Also, with --disable-win32-registry. As an experiment, I also added --localedir=/usr/${TOOLCHAIN_TARGET}/share/locale We'll see if that actually works. -- Chuck
Re: cygport cross-compiling beta1
On 7/19/2010 9:55 PM, JonY wrote: >> With NLS you will still have at least partial translations, which is >> better than nothing, no? >> > > How about setting up --with-localedir to somewhere version or target > specific? There isn't a '--with-localedir' option. But...there IS a --localedir one. How did I miss that? There's still one issue that Yaakov brought up; it's not clear that libintl will actually USE the correct version of the locale files...but some experimentation should answer that question. > iirc its used a bit in the cygnus tree to compile the cygwin1 dll and > strace. Not any more. >>> (also, --disable-werror, and language Ada) >> >> Just hadn't bothered with Ada, that's all. >> > > Cygwin gcc needs to be bumped to around 4.5.0 to build it. Its stated > somewhere in the GCC docs that you'll need the same version of gcc to > build cross ADA. Yep, that's why it's on DaveK's plate. He'll probably keep the cygwin native compiler and the mingw.org-based cross compiler in sync. >>> I believe --enable-fully-dynamic-strings is correct, but apparently the >>> official mingw.org 4.5.0 release does not use it. > > Careful, this option causes ABI change. Yes, and I had thought it had been used by all mingw.org post-4.x releases, since the PR24196 patch was dropped (I still have the PR# memorized...sheesh). But I was wrong; mingw.org have been using neither that patch, nor --enable-fully-dynamic-strings. This means that if you link with the static libstdc++, AND try to pass empty strings between DLLs, you may crash. (If you link with shared libgcc/libstdc++, you're ok). That's bad. And it's why TDM's builds DO use --enable-fully-dynamic-strings. Here's the build recipe for mingw.org's 4.2.1-sjlj, which is the oldest 4.x one I could find: ../gcc-4.2.1-2-src/configure --with-gcc --enable-libgomp --host=mingw32 --build=mingw32 --target=mingw32 --program-suffix=-sjlj --with-arch=i486 --with-tune=generic --disable-werror --prefix=/mingw --with-local-prefix=/mingw --enable-threads --disable-nls --enable-languages=c,c++,fortran,objc,obj-c++,ada --disable-win32-registry --enable-sjlj-exceptions --enable-libstdcxx-debug --enable-cxx-flags='-fno-function-sections -fno-data-sections' --enable-version-specific-runtime-libs --disable-bootstrap make BOOTCFLAGS="-O2 -D__USE_MINGW_ACCESS" CFLAGS="-O2 -D__USE_MINGW_ACCESS" CXXFLAGS="-mthreads -O2" LDFLAGS="-s -Wl,--stack=0x200" And, as you see, it isn't using fully-dynamic strings. So, I withdraw that suggestion. >>> Is it usual to build linux compilers with >>> --enable-fully-dynamic-strings? >> >> Apparently not, thanks for catching that. >> >>> Had to install libelf0 and libelf0-devel packages, in order to build. >> >> Right, as LTO uses libelf on ELF targets. >> > > just to clarify, win32 targeting GCC do not use libelf even when lto is > enabled. A special coff lto module is used instead. Apparently elf > readers don't work on coff :) Right, JonY, in this section we were discussing the linux-$target compiler, not the mingw*-$target ones. -- Chuck
Re: cygport cross-compiling beta1
On 7/19/2010 9:49 PM, Yaakov (Cygwin/X) wrote: > On Sat, 2010-07-17 at 16:26 -0400, Charles Wilson wrote: >> I'll address issues with libtool/pkgconfig at that >> time, as well (short version: our version of pkgconfig claims to support >> sysroot via $PKG_CONFIG_SYSROOT_DIR, but newer versions have supposedly >> improved that support. Also, there was one patch, and another promised, >> on libtool-patches addressing just this issue, in the last week). > > Where is that discussion? Fixing libtool is imperative to using the > sysroot the way I'm proposing; otherwise we'll be forced to actually > prepend $sysroot to $prefix. http://www.mail-archive.com/libtool-patc...@gnu.org/msg05488.html Paolo Bonzini mentioned that he had a different patch, and promised to rebase to latest and repost. He didn't. I pinged him. I'll release a new cygwin libtool with whatever comes of that discussion, as soon as it gets even close to a resolution. >> 1. As discussed previously, I disagree that all the mingw* toolchains > > The opinions I heard on #mingw-w64 was that /mingw was a necessary evil. > IWO they don't "like" it, but it's hard to avoid. OK. > FWIW the Fedora mingw64 packages also use $sysroot/mingw. Don't care. Fedora should be used as a guide, not a cage. >> 2. Furthermore, I think it is pretty likely that, for a linux $target, > > That depends on why you're cross-compiling. The primary point of making > cross-compiled packages for a distro is not to use them on their host > but to use them for cross-compiling other software in the build's > sysroot, so you want those prefixes to match (think RPATH). While you > could build that other software which you are going to use on the host > with cygport, that's a secondary benefit IMO (much like canadian-cross). But at SOME point, SOME part of what you've built on $host is supposed to be used, eventually, by somebody, on $target, right? Where should THAT live? If I'm on linux and have a devel environment, I can always compile with --prefix=/home/me/my-builds even if I don't have root. But...with this cross.cygchain, I can't do that from my cross-host; I'm trapped in the thou-shalt-have-root prison and to USE anything I build, I *must* build with --prefix=/usr -- and then I can't use it on the $target without root privilege. rpm-build doesn't restrict you that way; sure, it *defaults* to /usr, but you CAN override (yes, there are issues on the 'rpm --install' side wrt. root, but there are also workarounds for that). I'm not sure about ebuild, but I *think* you can override it, too. Why shouldn't cygport allow that? VVV=== cut ==VVV (3)-(5) are all dead horse issues; we've beaten them to death. I'll respond, but we don't really need to discuss these any further. Most of them are the domain of the maintainer of the specific toolchain, and if he/she disagrees with you OR me, they can easily include the appropriate commands in their cygport(5). >> 3. Toolchain Documentation (currently deleted during src_install): I >>think the documentation specific to each cross compiler that >>conflicts with the native compiler -- e.g. info files, man{2-7} >>pages not renamed by 'make install' -- should go under >> /usr/$target/share > > Fedora's MinGW packaging policy says clearly that duplicate > documentation should not be included. Again, don't care. Guide, not cage. And, once again, you are assuming as true the very thing I am saying will be false; when you proceed from different premises naturally the conclusions will differ. The documentation will not be "duplicate documentation" because the documentation for different versions of gcc is different. In the (unlikely!) event that the versions for i686-w64-mingw32 gcc and mips-sun5-solaris10 gcc are the same, then -- by accident -- you might have two copies of the same documentation. > Inventing /usr/$target/share for > the sole purpose of info and man pages which won't be found by info or > man anyway doesn't make any sense to me. Then package an /etc/profile.d script that appends or prepends to MANPATH and INFOPATH. Or assume some modicum of intelligence on the part of people with the gumption to actually install a *cross compiler*, and allow them to manually set INFOPATH/MANPATH as appropriate when they are using a given cross tool. Or the provider of the cross toolchain can patch the .texi files -- as I did, for all of the automake* and autoconf* packages -- to install versions under different names. SOMETHING. Just assuming that "the docu for the native compiler will be good enough for whatever bleeding edge or ancient cross compiler(s) are installed" seems not entirely realistic, to me. But again, this is the purview of the cygport(5), anyway. >> 4. NLS. I think the cross toolchains should be compiled with >>--disable-nls, > With NLS you will still have at least partial translations, which is > better than nothing, no? Wherever the l
Re: cygport cross-compiling beta1
On 7/20/2010 09:49, Yaakov (Cygwin/X) wrote: 4. NLS. I think the cross toolchains should be compiled with --disable-nls, if we can't figure out how to install and use the correct .mo files associated with each compiler. Otherwise, all such cross compilers must be at exactly the same version as the native one, because they will all share the native compiler's .mo files. I don't think this situation (all compilers using same version) is very likely. With NLS you will still have at least partial translations, which is better than nothing, no? How about setting up --with-localedir to somewhere version or target specific? However, given the dependency structure of the gcc3 packages, this means you pretty much have to also remove, using setup, gcc-core gcc-g++ gcc-java gcc-objc Note that this conflict means that the current gcc-mingw* packages (and, thus, the gcc 3.x packages) conflict with the new mingw-gcc-* and mingw-binutils-* packages. Also, this may have implications for the upgrade process, when the "real" mingw32-gcc cross toolchain is released. Even if we mark the gcc (3) packages obsolete, we probably have to repackage them and their gcc-mingw* friends to fix this upgrade problem, and perhaps relieve the conflict. Somehow? We can just remove the gcc-mingw* deps from gcc3*. But the bigger question is, do we still *need* gcc3 in the distro? iirc its used a bit in the cygnus tree to compile the cygwin1 dll and strace. The following flags are used in the official mingw compiler, but not here: --enable-libstdcxx-debug Reading the description, I'm not so sure we need (or want) this. --enable-shared (but that's okay, as it is default) --enable-libgomp AFAIK --enable-shared and --enable-libgomp are the defaults. --disable-win32-registry Yes, we definitely want this. (also, --disable-werror, and language Ada) Just hadn't bothered with Ada, that's all. Cygwin gcc needs to be bumped to around 4.5.0 to build it. Its stated somewhere in the GCC docs that you'll need the same version of gcc to build cross ADA. The following flags are used here, but not in the official mingw compiler: --disable-multilib --enable-lto AFAIK these are actually the defaults. I believe --enable-fully-dynamic-strings is correct, but apparently the official mingw.org 4.5.0 release does not use it. Because 'mingw-runtime' is an existing package, and the current version is 3.18-1, this new version should be -2. Oops, missed that one. But the mingw-runtime maintainer should take care of that when we're up to pushing this. Careful, this option causes ABI change. Is it usual to build linux compilers with --enable-fully-dynamic-strings? Apparently not, thanks for catching that. Had to install libelf0 and libelf0-devel packages, in order to build. Right, as LTO uses libelf on ELF targets. just to clarify, win32 targeting GCC do not use libelf even when lto is enabled. A special coff lto module is used instead. Apparently elf readers don't work on coff :)
Re: cygport cross-compiling beta1
On Sat, 2010-07-17 at 16:26 -0400, Charles Wilson wrote: > Overall, looks pretty good. I haven't used the new cygport(1) to build > *any* native cygwin packages yet, however; so I haven't tested how well > the new toolchain.cygclass works building the native cygwin toolchain. If TOOLCHAIN_TARGET="i686-pc-cygwin" then all toolchain.cygclass does is provide a default src_compile(). The cross-compiling magic only kicks in when $build != $host || $host != $target. > I'll post more after I use the new tools to build the various non-core > packages that exercise the cross.cygclass, such as linux-ncurses, > mingw-bzip2, etc. I'll address issues with libtool/pkgconfig at that > time, as well (short version: our version of pkgconfig claims to support > sysroot via $PKG_CONFIG_SYSROOT_DIR, but newer versions have supposedly > improved that support. Also, there was one patch, and another promised, > on libtool-patches addressing just this issue, in the last week). Where is that discussion? Fixing libtool is imperative to using the sysroot the way I'm proposing; otherwise we'll be forced to actually prepend $sysroot to $prefix. > 1. As discussed previously, I disagree that all the mingw* toolchains >should use --prefix=/mingw; I think the mingw64 ones should use >different prefixes, since the mingw64 folks recommend "do not >install our stuff into C:\MinGW". However, AFAIK they have not >made any official recommendation wrt cross environments and >sysroots. Fortunately, this is an easy change. For example: /mingw is hard-coded in GCC as the default prefix for all mingw* targets. That means ($sysroot)/mingw/include and ($sysroot)/mingw/lib are on the default search paths. Using another prefix will force everyone to use -I and -L flags where they otherwise wouldn't be necessary. The opinions I heard on #mingw-w64 was that /mingw was a necessary evil. IWO they don't "like" it, but it's hard to avoid. FWIW the Fedora mingw64 packages also use $sysroot/mingw. > 2. Furthermore, I think it is pretty likely that, for a linux $target, >users of a cross toolchain might want to compile for a $prefix >other than /usr (obviously the compiler's own core libs like glibc >should be in $sysroot/usr, to match the likely location of glibc on >the target). But, for other packages (say, I want to cross-compile >my own version of linux-firefox or something)...maybe I don't have >root on the linux box and can't install in /usr (or maybe I'm using >canadian-cross.cygclass to build, on cygwin, a linux-hosted cygwin- >target cross compiler, and I don't want to install that in /usr on >the linux machine). So, I think the ability to override $prefix is >necessary at least for the linux toolchain -- and, as I've mentioned >before, it is currently useful for the gcc-tool-* packages. That depends on why you're cross-compiling. The primary point of making cross-compiled packages for a distro is not to use them on their host but to use them for cross-compiling other software in the build's sysroot, so you want those prefixes to match (think RPATH). While you could build that other software which you are going to use on the host with cygport, that's a secondary benefit IMO (much like canadian-cross). > 3. Toolchain Documentation (currently deleted during src_install): I >think the documentation specific to each cross compiler that >conflicts with the native compiler -- e.g. info files, man{2-7} >pages not renamed by 'make install' -- should go under > /usr/$target/share Fedora's MinGW packaging policy says clearly that duplicate documentation should not be included. Inventing /usr/$target/share for the sole purpose of info and man pages which won't be found by info or man anyway doesn't make any sense to me. > 4. NLS. I think the cross toolchains should be compiled with >--disable-nls, if we can't figure out how to install and use the >correct .mo files associated with each compiler. Otherwise, all such >cross compilers must be at exactly the same version as the native >one, because they will all share the native compiler's .mo files. I >don't think this situation (all compilers using same version) is >very likely. With NLS you will still have at least partial translations, which is better than nothing, no? > 5. Packaging: I think that the languages for the compilers should be >split into separate packages, and that the language runtime DLLs >should have individual packages. Ditto with respect to other >packages which provide DLLs, like pthreads etc. No, no, and NO. > - mv ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/lib/libgcc_s.a > ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/${PV[1]}.*/ > + mv ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/lib64/libgcc_s.a > ${D}/usr/lib/gcc/${TOOLCHAIN_TARGET}/${PV[1]}.*/ Fixed. > Also, this packaging oddity: > mingw64-x86_64: usr/x86_64-w64-mingw32/lib64/libiberty.a x86_
Re: cygport cross-compiling beta1
On Thu, 2010-07-15 at 12:50 -0500, Yaakov (Cygwin/X) wrote: > The changes to cygport are extensive, and I have yet to break them out > into reasonable-sized commits; so they're not on git yet, but I'm > working on it. It is likely that I broke something along the way, so > I'm expecting to make further changes to restabilize things. All my changes have now been pushed into git master. I also added sample binutils/gcc for Solaris 10 x86; see the gcc .cygport for directions on how to get the necessary files off a Solaris system. (FWIW Su^WOracle provides a free image for VirtualBox.) Chuck, I'll try to respond in full tomorrow. Yaakov
Re: cygport cross-compiling beta1
On 7/15/2010 1:50 PM, Yaakov (Cygwin/X) wrote: > As promised, here is my response to the mingw-w64 ITP. > > cygport now supports several scenarios: > > 1) Cygwin-hosted (cross-)compilers, via toolchain.cygclass > > This is exclusively for binutils, gcc, and gdb, either Cygwin-native or > any other platform they support. The attached examples use a one-line > patch to config.rpath to link dynamically with libintl, but do not ship > the actual locale files, relying on the native versions' instead (as > theirs would collide and refuse to relocate, as previously discussed). > > All cross-compilers use the default sysroot, /usr/$target/sys-root. > MinGW libgcc and friends are relocated to the sysroot bindir, just as on > Fedora. Overall, looks pretty good. I haven't used the new cygport(1) to build *any* native cygwin packages yet, however; so I haven't tested how well the new toolchain.cygclass works building the native cygwin toolchain. These notes are related to my experience rebuilding all four of the *cross* toolchains and their core support libraries (runtime, w32api, pthreads, etc). I'll post more after I use the new tools to build the various non-core packages that exercise the cross.cygclass, such as linux-ncurses, mingw-bzip2, etc. I'll address issues with libtool/pkgconfig at that time, as well (short version: our version of pkgconfig claims to support sysroot via $PKG_CONFIG_SYSROOT_DIR, but newer versions have supposedly improved that support. Also, there was one patch, and another promised, on libtool-patches addressing just this issue, in the last week). General toolchain cygclass notes: = 1. As discussed previously, I disagree that all the mingw* toolchains should use --prefix=/mingw; I think the mingw64 ones should use different prefixes, since the mingw64 folks recommend "do not install our stuff into C:\MinGW". However, AFAIK they have not made any official recommendation wrt cross environments and sysroots. Fortunately, this is an easy change. For example: --- syntax.cygpart.old 2010-07-15 13:38:33.00100 -0400 +++ syntax.cygpart.new 2010-07-17 08:47:30.53900 -0400 @@ -139,6 +139,8 @@ # FIXME: certainly other exceptions exist? case ${CHOST} in *-*-mingw*) prefix=/mingw ;; + x86_64-w64-mingw*) prefix=/mingw64;; + i686-w64-mingw*)prefix=/mingw64-32 ;; *)prefix=/usr ;; esac So we can easily adapt if the #mingw64 folks recommend something different than /mingw. Let it go for now, but let's see what #mingw64 have to say about that -- especially JonY, since he is the person actually ITPing the mingw64* cross toolchains. 2. Furthermore, I think it is pretty likely that, for a linux $target, users of a cross toolchain might want to compile for a $prefix other than /usr (obviously the compiler's own core libs like glibc should be in $sysroot/usr, to match the likely location of glibc on the target). But, for other packages (say, I want to cross-compile my own version of linux-firefox or something)...maybe I don't have root on the linux box and can't install in /usr (or maybe I'm using canadian-cross.cygclass to build, on cygwin, a linux-hosted cygwin- target cross compiler, and I don't want to install that in /usr on the linux machine). So, I think the ability to override $prefix is necessary at least for the linux toolchain -- and, as I've mentioned before, it is currently useful for the gcc-tool-* packages. Fortunately, that is also an easy change. For example: --- syntax.cygpart.old 2010-07-15 13:38:33.00100 -0400 +++ syntax.cygpart.new 2010-07-17 08:55:36.18260 -0400 @@ -135,6 +135,10 @@ __host_prefix() { local prefix + if [ -n "$CYGPORT_OVERRIDE_PREFIX" -a x"${CYGPORT_OVERRIDE_PREFIX:0:1}" == x"/" ] + then + prefix=$CYGPORT_OVERRIDE_PREFIX + else # FIXME: certainly other exceptions exist? case ${CHOST} in @@ -142,6 +146,7 @@ *)prefix=/usr ;; esac + fi echo -n ${prefix} } Dead horse issues, restating: 3. Toolchain Documentation (currently deleted during src_install): I think the documentation specific to each cross compiler that conflicts with the native compiler -- e.g. info files, man{2-7} pages not renamed by 'make install' -- should go under /usr/$target/share Maybe even ALL such documentation (e.g. by using --datarootdir), and then *copies* of the man1 pages put in the usual /usr/share/man area. 4. NLS. I think the cross toolchains should be compiled with --disable-nls, if we can't figure out how to install and use the correct .mo files associated with each compiler. Otherwise, all such cross compilers must be at exactly the same version as the native one, because they will all share the nati
Re: cygport cross-compiling beta1
On Thu, 2010-07-15 at 12:50 -0500, Yaakov (Cygwin/X) wrote: > 2) Cygwin-to-other cross-compiling, via cross.cygclass > > Cross-compiling is supported for packages using autotools, cmake, or > hand-written makefiles. Packages are automatically "installed" into the > sysroot (under $D). The way I used the sysroot so far was to configure packages for their regular prefix (/usr or /mingw), then install with DESTDIR= ${D}${CROSS_SYSROOT}. Without the sysroot path hard-coded anywhere, the cross-compiled files should be usable either on the host system as-is or within the sysroot for further cross-compiling. Unfortunately I found two problems with this approach: libtool and *-config scripts (NOT pkg-config). libtool's ignorance of sysroots is well documented and seems to be considered a known bug. I've seen it manifested in at least two ways: 1) on ELF platforms, during relink; in linux-i686-glib2.0, libgio consistently refused to relink because a spurious -L/usr/lib caused Cygwin's libc.a to be picked up. The offending -L flag was caused by the line in libtool following the words "we'll fake it", triggered by relinking against libglib and friends from the same package, which libtool thought would literally reside in /usr/lib. 2) when building setup.exe, libtool complained that it could not find /mingw/lib/libgpg-error.la, a dependency_libs of libgcrypt. Of course, libgpg-error was there, under the sysroot. (Note that I manually fixed my mingw-libgcrypt test build to not interfere with the setup.exe/gcc-4.5 discussion.) The other problem would be packages relying on a foo-config script to discover foo's cflags/libs, which would be prone to output -I or -L flags for the native path instead of within the sysroot. Note that pkg-config can understand sysroots through an environment variable, which cross.cygclass uses. The alternative to mitigate these issues would be to configure with --prefix=$CROSS_SYSROOT$prefix et. al., causing the sysroot to be hard-coded throughout. This should fix these cases, but packages so built would be unsuitable for use on the host system, and using host-built packages under the sysroot would require adding the sysroot to their .la and .pc files. Thoughts? Yaakov