On 7/25/2010 5:01 AM, Yaakov (Cygwin/X) wrote: > On Sat, 2010-07-24 at 00:26 -0400, Charles Wilson wrote: >> What does gentoo do with cross compilers and sysroots? I know >> ebuild/emerge supports them; do they treat them strictly as support for >> cross compiles, or as installable images on the intended $host? I >> suspect the latter... > > Not AFAICS: > > http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=3#doc_chap2
That's sad. It's also limiting. It means that you *can't* -- using gentoo's crossdev framework -- build a -dev package for the $host, that someone ELSE could use for native development ON $host. I fear this would also be a limitation of any cross.cygclass | deploy.cygclass bifurcation we cooked up. > OTOH, cygport is NOT a package manager, it is a build tool which creates > package tarballs, which are easily unpacked on other systems. So I > wouldn't mind making a separate cygclass for that purpose, which would > also allow for $prefix customization. In fact, it's basically ready, > but I'm not sure what to call it[1]. > > [1] http://cygwin.com/ml/cygwin-apps/2010-07/msg00174.html But, could I use $WHATEVER.cygclass to build libfoo for the $host libbar for the $host -- when libbar depends on libfoo? I suspect the only way to make this work would be to have TWO $host copies of libfoo: The deploy-on-$host version, which the end user would need. But also the local $build.cross version, in order to link libbar-for-deploy-on-$host. But then you get into issues with relinking and embedded rpaths (at least for ELF; not so much for PE/COFF). It really seems like TRTTD is to make the two trees one, and *fix* the issues with libtool, pkgconfig, and foo-config. OTOH, we're just a tiny little FOSS project; if the Big Boys haven't solved this by now, and even gentoo, fedora, and debian have all given up on the project...would we be tilting at windmills to try to solve it with our limited resources here? >> We could go that route as well; it just strikes me as a shame to "give >> up" since win32 is /right there/ underneath cygwin all along... >> >> OTOH, even on fedora-x86, it'd be nice to be able to map $sysroot$prefix >> to some specific wine path (identity mount "inside" wine, as $prefix?), >> and "run" the binaries "in place". That would let you, in some cases, >> run package test suites and the like. > > For testing MinGW binaries, because they see only Windows paths, *either > way* you have a problem running in the sysroot IF $prefix et al are used > in the code at runtime (/mingw != C:\mingw) or if the tests pass > Cygwin-native paths to the executable. Yes, I know. But...many times testsuites use only relative paths, which will work fine (as long as you don't use symlinks in your source tree for some reason). It won't ALWAYS work, but at least we wouldn't be deliberately creating MORE breakages, through our choice of cross compile strategies. > If not, either because there is > no runtime data or modules to find, or because it is coded to be > relocatable (at least on Windows, which is not uncommon), then you > should still be able to test-run cross-built programs using sysroot > libraries by appending $sysroot$prefix/bin to path. That might very > well be why Fedora moves libgcc and friends into the sysroot. Yes, probably so. > AFAICS the only thing you lose by configuring --prefix=$sysroot$prefix > is transferability between the sysroot and the host, IOW: Which would seem to me to be a pretty big deal. I mean, what's the point of cross compiling something if you can never use it on the $host? Sure, you could compile all the prereqs for, say, GIMP, to create a suitable compilation environment/sysroot on $build, so that you CAN then finally go ahead an compile GIMP itself for $host. But I'd assume at that point you'd want to copy GIMP over TO the $host and use it. And that means you need -- what? a SECOND version of all of those same prereqs, only this time compiled in a non-sysroot manner, so that the runtime deps of GIMP can also be deployed over on $host? That seems...excessive. (And I'm assuming that all the deps of GIMP are simple; they don't actually have interacting dependencies with each other. That makes things even MORE complicated, and unfortunately my assumption is counterfactual.) > 1) sysroot-configured libraries would not run on the host if runtime > data/module paths are hard-coded; Well, at least .la files can be postprocessed for deployment. No such luck for compiled-in paths. There's a strong feeling among some win32 folks that any "properly" ported app will be $prefix-independent; that it will always deduce its own location and use relative paths exclusively. Nice idea, but the FOSS universe isn't there yet. > 2) sysroot-configured ELF libraries would have an extra RPATH pointing > to the sysroot, which IIUC should be harmless (could this be mitigated > by patching libtool to not use $hardcode_libdir_flag_spec when $libdir > is in $sys_lib_search_path_spec?); The is probably not an operational concern. It may be a security concern, though, in some situations. > 3) host native-built libraries would need $sysroot prepended to paths in > their foo-config/.la/.pc files (none of which should affect using host > libc's for bootstrapping) to be usable in the sysroot, and vice-versa > for sysroot-configured libs to be used on the host. Right. > But if you're only using sysroots for libraries, and you treat > building-for-host and building-for-sysroot as two different things, then > none of that should really matter. Well, maybe. It seems fairly silly to have to build everything twice though. And while conceptually it makes sense for items with no internal dependencies, I don't see how you build the following chain for deployment on $host: libA libB libC depends on libA and libB libD depends on libA appE depends on libC and libD It's easy to build all of those "for $sysroot". And since libA and libB have no deps, you can easily build them also "for $host". But when it comes time to build libC , libD, and appE "for $host" -- what do you do? Build them in such a way that they link against the $sysroot versions, but with a "normal" --prefix? So they wouldn't ACTUALLY be built against the libA and libB "for $host" libraries? I *guess* that would work -- if all you care about on $host is that appE will run. But what if you want to deploy, on $host, -dev packages for libA, libB, libC, and libD -- so that somebody else, on $host, could use them as part of a NATIVE development environment on $host? Would this procedure basically leave you with entities where you can 1) ONLY develop on $build using stuff built with --prefix=$sysroot 2) ONLY run cross-built items on $host, but never do native development with them >> See, my experience with "sys-roots" have been the "place where I install >> all the crap I will turn into the system image on embedded-platform-X". >> And that includes $sysroot$prefix/bin/busybox, etc. > > But if you're making a system image, then there will anyways be a lot of > stuff in the sysroot that you will not want on an embedded platform > (headers, .a and .la libs, pkg-config files, foo-config scripts, extra > docs, unused l10n, etc). Right, for embedded systems. I'm trying not to limit my vision to just typical embedded systems where you'd never (well, almost never) try to do actual development natively on the embedded $host. What if you DO want to create deployable -dev packages? > Gentoo's approach seems to be that you use a > different directory for a staging area, copying only what you need from > the sysroot (shared libs, essential runtime data) into there. Well, your link was from embedded gentoo, where -- naturally -- they are not concerned with enabling "native" development on the embedded $host. So of course you only copy over the barest minimum files needed at runtime, and it makes no sense to package up "dev" files for the $host. So, you build everything in $sysroot, cp over to $imageroot, and hope that any compiled-in paths with $sysroot in them don't bite you. I'm trying to think more generally than that, tho. >> Right, that's a better approach. However, you'd probably still need to >> patch foo's foo-config script itself, so that it also accepts a >> --sysroot option, right? > > Only if you configure foo with --prefix=$prefix, not with --prefix= > $sysroot$prefix. Well, of course. But if you configure with $sysroot$prefix, then your package is not "deployable". >> And, the configure.ac for packages that are clients of libfoo, may also >> need to be patched to support --with-foo-config=a-path if they don't >> already. But again, that would be acceptable upstream. > > If foo-config is found with AC_(CHECK|PATH)_PROG, which it should be, > then you just pass FOO_CONFIG=$CROSS_SYSROOT$prefix/bin/foo-config to > configure. A patch to do *that* would almost certainly be accepted and > would be simpler than --with-foo-config. VLC was just the first example I thought of, and had the code handy for. AC_PATH_PROG is more straightforward, certainly -- but vlc had some other issues to deal with IIUC, so couldn't use it directly. They basically ended up reimplementing most of its guts, tho. > Which makes me think that perhaps I should provide definitions for > CROSS_SYSROOT_{BIN,INCLUDE,LIB}DIR to make the $prefix handling opaque. I think you know my opinion about locking things down ever more tightly. Grand Moff Tarkin, meet Princess Leia... >> True, but foo-config + sysroot is broken. That's not ideal either. >> However: how many packages, that either provide or use foo-config >> scripts, are *WE* planning to build/support using the cygwin cross >> toolchains, immediately? > > $ ls -1 /usr/bin/*[-_]config | wc -l > 99 What, you want the cygwin distribution to provide cross built versions, for $host={mingw,linux,???), of all 99 of those packages? How much time do you HAVE on your hands, anyway? > Now, seven of those are Cygwin service installation scripts, but OTOH > there are certainly more which I don't have installed right now. Either > way, that leaves quite a number of libraries (not to mention other > software which depends on these libraries) which would be affected by > this. That's not what I asked. *WE* only need to support patched source (and try to push upstream) for the package *WE* provide as part of the cygwin distro. If we don't provide mingw-octave, we don't have to worry about patching octave-config so that it supports sysroot-built clients. Let whatever poor sap who DOES want to publish a cygwin-dev package corresponding to mingw-octave worry about that. >> Hmm, hrrmmm, hummmm....so, I wanted to test your script to see what it >> would do. The first one I picked was 'freetype-config'. Guess what? >> >> It already supports SYSROOT, by the simple expedient of prepending all >> output paths with the environment variable $SYSROOT (--libs sysroot >> support uses some additional logic): > > Well, you happened to pick the *only* one of those 90-plus which do > support sysroots. Yeah, well, I'm just lucky that way. >> What about using this, for your cross-sysrootize (note the backslash >> preceeding $CROSS_SYSROOT? (or, $SYSROOT) >> >> for f in $@; do >> sed -i -e "s|\([^t]\)$prefix|\1\$CROSS_SYSROOT$prefix|g" $f >> done > > That would require defining CROSS_SYSROOT manually if cross-compiling > w/o cygport. Well, yeah. You also have to manually set --build and --host. And you have to manually install the rest of the junk your cross compiler needs in its sysroot. >> Wrong? There is no "wrong" in this context; it's only software, and they >> each performed their function as designed. > > Semantics; then the design is poor. Now I'm to blame for windows' brain dead DLL design, and DLL hell? >> And they explicitly request that any DLL using the name 'zlib1.dll' be >> exactly compatible with that one. As it happens, a DLL built >> using zlib-1.2.x/win32/Makefile.gcc -- named 'zlib1.dll' -- isn't >> necessarily compatible with the official MSVC6.0-built "zlib1.dll", >> because that makefile doesn't use -mms-bitfields (among other reasons). > > As for upstream's "request", its not necessarily their place to > determine our (or mingw.org's) library naming and packaging schemes. It's not a "cygwin" or "mingw" naming scheme. THAT name, zlib1.dll, is the name the upstream maintainers use to support the native windows *platform*, and it certainly is their place to request that other people, us included, don't blunder in and deliberately introduce an incompatible, but identically named version. Given Window's brain dead DLL system. > Especially when their own Makefile apparently goes against their own > wishes. (Dealing with years of upstream misconceptions wrt Cygwin has > led me to take upstream "advice" with a grain of salt. Or sometimes a > shaker-full.) Well, in this case it's probably because it never occurred to them that the compiler that calls itself a port of GCC to Win32 doesn't actually create objects that follow the system ABI for C. (well, technically, I gather that bitfields are a bit of a gray area in the ABI...) >> But even at mingw.org, their mingw-ish zlib package doesn't pretend to >> be "the" zlib1.dll. Instead, Keith builds it as "libz-1.dll" which is >> neither "zlib1.dll", nor "mgwz.dll". > > Hmm, so I see. I thought I had checked that. > >> (And not >> attempt ABI compatibility with zlib.net's "zlib1.dll") > > Agreed. > >> The primary purpose, until now, of the mingw-{zlib,...gcrypt} libraries >> has been to support setup.exe and setup.exe alone. I want to make the >> MINIMUM changes in these packages, until we are sure that each >> additional change doesn't break setup.exe. > > And what does DLL naming have to do with setup.exe, which is > *statically* linked? Call it superstition. ZERO avoidable changes means exactly that. START with zero changes -- or as close as you can reasonably get -- then slowly introduce minor, desirable changes, even if you can't see how they might affect the end result. I've seen static libraries that embed the name of the dynamic library, because the developers wanted their log messages to "look the same" whether linked statically or dynamically. Sure, it's just a text string, but if one string is longer than the other because you changed the DLL name, then that could have knock-on effects to the memory layout of the code, etc etc... Now, none of THESE libraries do that, but...it's safer to go one step at a time. I don't want to break setup.exe. We're already fragile enough, as only one person is currently set up to compile setup.exe at all -- cgf;s linux-hosted cross compiler doesn't introduce TLS sections. And TLS sections are bad, because UPX can't yet compress them(*). (*) upstream UPX mercurial has a fix for this, but no official release has been published. If I had enough tuits I'd try to compile mercurial head and test it out... > Now that I nailed down the problem to how GCC 4.5 optimizes > autoload.c[2], both my packages and yours produce a working setup.exe. > The only question is mingw.org compatibility, which shouldn't be as > complicated as you're making it. > > [2] http://cygwin.com/ml/cygwin-apps/2010-07/msg00188.html Yes, I saw that. Seems to work fine in my test builds. >> I think we are stuck with the same conflict as mingw.org, with our > > OK, so I'll have to add it to CFLAGS et al, but it does mean that custom > Makefile-based packages need to make sure that $CFLAGS gets passed > along. A bit messy, but I see no alternative. Sad, but true. >> No, cygwin defaults to -shared-libgcc. mingw uses -static-libgcc by >> default for C, shared for all other languages: >> >> * Shared libgcc: If all modules are linked with -shared-libgcc, >> exceptions can be thrown across DLL boundaries. Note that this is >> the default for all languages other than C. To disable this, use >> -static-libgcc. > > Now if (upstream) libtool would only accept these flags... Yes. I don't really understand Ralf's objection to them, and I've reviewed the relevant email exchanges several times. I guess it boils down to the fact that you can't configure with one flag, and then build with another, because libtool records the value within itself. You asked, at one point, whether libtool could begin to trust the language drivers and avoid recording all that data. I think the answer might be a qualified yes: for $CC==some_variant_of_gcc. Other platform-sepcific drivers, probably too risky a change. But even limiting it to just gcc is a big behavioral change; probably libtool-2.4 material. -- Chuck
