Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 07/01/2012 01:41 PM, Thomas Sachau wrote: I guess, you are mixing cross-compile support in multilib profiles and cross-compile support with cross-toolchains, multilib-portage is for the first one, while crossdev is for the second one. My suggestion does not support e.g. compiling for ppc with an amd64 profile, on amd64 it only can support x86 and x32. Since all of these binaries can run with an amd64 kernel and you build for at least one target, you always have a binary around, no need for an extra HOST dependency. You can run an arm binary on amd64 (through binfmt+qemu-user static) I dont know, what exactly you mean with play properly with ld and cross-vs-host paths, so cannot respond to those. multilib works because the runtime linker picked is the right one for each ABI, thanks to qemu makes no difference if that ABI is native or not. cross vs host paths is an annoying problem due the slightly different behaviour between native and cross compiler toolchains, it tends to ignore environment variables and other small differences making dropping an native cross compiler in a qemu chroot, QUITE a creative activity. lu -- Luca Barbato Gentoo/linux http://dev.gentoo.org/~lu_zero
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
Matt Turner schrieb: On Sun, Jul 1, 2012 at 4:52 PM, Thomas Sachau to...@gentoo.org wrote: Matt Turner schrieb: I suppose that's just for ease of implementation? Not having to special-case packages that don't install binaries. I dont follow. Did you think about only having additional ABI flags for certain cases? Right. It seems to me that not having the ABI flags for packages like x11-proto/* would make more sense, but may be more work to implement. The problem here is the following: How do you know before the src_install phase, that a package has no ABI-specific content? And even, if you know that, how would you define the dependency part? This would probably become pretty complex, since it also has to support corner cases. Those ABI flags behave the same as other USE flags: When you change them, you have to recompile the package including all the content, that has been compiled previously. If you want the compile for each ABI seperate, then you would have to handle them more like SLOTS, but with a new syntax, with a collision handler for the common content and how to handle that in the user UI. I fear, that this would be way more complicated to implement in a sane way, so i dont plan to go that route. Ouch. I already think a mechanism for telling the package manager you don't need to rebuild this entire package to turn on /this/ USE flag is needed. For ABIs, the situation seems much more clear-cut. In fact, it seems rather important. This would require a good amount of changes. Since my work with multilib-portage started in 2009 and is still not in the PMS, also i tried to keep it simple, i would not even try to guess an ETA for both of your, in my eyes more complex, ideas. ;-) I of course wont prevent you from preparing them and getting them in, they may be nice, especially removing the need to recompile already compiled code. Just dont underestimate the work with it. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On Mon, 02 Jul 2012 19:06:43 +0200 Thomas Sachau to...@gentoo.org wrote: The problem here is the following: How do you know before the src_install phase, that a package has no ABI-specific content? You make every package that has ABI specific content explicitly say so, as metadata. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
Luca Barbato schrieb: On 06/29/2012 04:30 PM, Thomas Sachau wrote: It is interesting, still you need a way to define HOST dependencies (stuff you need that has to be built on the host since you run it, e.g. xcb python code generator), something to play properly with ld and cross-vs-host paths for compilers (that part is _really_ annoying for qemu-chroots) lu I guess, you are mixing cross-compile support in multilib profiles and cross-compile support with cross-toolchains, multilib-portage is for the first one, while crossdev is for the second one. My suggestion does not support e.g. compiling for ppc with an amd64 profile, on amd64 it only can support x86 and x32. Since all of these binaries can run with an amd64 kernel and you build for at least one target, you always have a binary around, no need for an extra HOST dependency. I dont know, what exactly you mean with play properly with ld and cross-vs-host paths, so cannot respond to those. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
Matt Turner schrieb: On Sun, Jul 1, 2012 at 7:29 AM, Thomas Sachau to...@gentoo.org wrote: Matt Turner schrieb: On Fri, Jun 29, 2012 at 10:30 AM, Thomas Sachau to...@gentoo.org wrote: I'm interested in this because I'm regularly annoyed with the emul- packages and also because multilib is pretty important for mips. If a package has dependencies, then those dependencies are required to have at least the same targets enabled as the package That seems like the obvious (but perhaps naive) choice. What about depending on packages that don't install libraries, like x11-proto/ packages or generators like dev-util/indent? Maybe I just don't understand. Would these packages even have ABI flags? All packages do get the ABI flags (with the needed EAPI or via enabled portage feature, which is currently in the multilib branch). If a package does not install anything ABI-specific (no headers, no libs and no binaries), then there is no overhead, since it will just get compiled/installed for one ABI, even if multiple ABI flags are enabled. I suppose that's just for ease of implementation? Not having to special-case packages that don't install binaries. I dont follow. Did you think about only having additional ABI flags for certain cases? For mips, we'd like to have gcc built as an n64 binary, which would require its run-time dependencies (mpfr, mpc, gmp, etc.) to be available as n64 as well, but the rest of the system to be n32 binaries. Does this fit with your understanding (and proposed solution) of the problem? I guess, you require additional n64 libs for gcc dependencies like mpfr, mpc and others, while your default ABI will be n32. This should work fine with my proposal, you just have the (already default enabled) n32 ABI flag enabled, which results in everything being built just for n32. For the packages, where you require additional n64 libs, you just enable the n64 ABI flag in addition. And if you need n64 binaries too, you enable the abiwrapper USE flag for them. One follow-on question. Consider having a package dev-libs/libpkg already installed for ABI=n32 -n64 and wanting to emerge another package that depends on it with ABI=-n32 n64. Will dev-libs/libpkg have to be rebuilt twice (once for the existing n32 ABI and again for the newly needed n64), or can we optimize this case by recognizing that building for multiple ABIs means entirely separate builds? Those ABI flags behave the same as other USE flags: When you change them, you have to recompile the package including all the content, that has been compiled previously. If you want the compile for each ABI seperate, then you would have to handle them more like SLOTS, but with a new syntax, with a collision handler for the common content and how to handle that in the user UI. I fear, that this would be way more complicated to implement in a sane way, so i dont plan to go that route. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 07/01/2012 04:29 AM, Thomas Sachau wrote: Matt Turner schrieb: On Fri, Jun 29, 2012 at 10:30 AM, Thomas Sachau to...@gentoo.org wrote: I'm interested in this because I'm regularly annoyed with the emul- packages and also because multilib is pretty important for mips. If a package has dependencies, then those dependencies are required to have at least the same targets enabled as the package That seems like the obvious (but perhaps naive) choice. What about depending on packages that don't install libraries, like x11-proto/ packages or generators like dev-util/indent? Maybe I just don't understand. Would these packages even have ABI flags? All packages do get the ABI flags (with the needed EAPI or via enabled portage feature, which is currently in the multilib branch). If a package does not install anything ABI-specific (no headers, no libs and no binaries), then there is no overhead, since it will just get compiled/installed for one ABI, even if multiple ABI flags are enabled. For a package like this that does not install anything ABI-specific, does the package manager still execute phases for each enabled ABI, or is there some way for the ebuild to indicate whether or not its phases need to be executed for each enabled ABI? -- Thanks, Zac
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
Zac Medico schrieb: On 07/01/2012 04:29 AM, Thomas Sachau wrote: Matt Turner schrieb: On Fri, Jun 29, 2012 at 10:30 AM, Thomas Sachau to...@gentoo.org wrote: I'm interested in this because I'm regularly annoyed with the emul- packages and also because multilib is pretty important for mips. If a package has dependencies, then those dependencies are required to have at least the same targets enabled as the package That seems like the obvious (but perhaps naive) choice. What about depending on packages that don't install libraries, like x11-proto/ packages or generators like dev-util/indent? Maybe I just don't understand. Would these packages even have ABI flags? All packages do get the ABI flags (with the needed EAPI or via enabled portage feature, which is currently in the multilib branch). If a package does not install anything ABI-specific (no headers, no libs and no binaries), then there is no overhead, since it will just get compiled/installed for one ABI, even if multiple ABI flags are enabled. For a package like this that does not install anything ABI-specific, does the package manager still execute phases for each enabled ABI, or is there some way for the ebuild to indicate whether or not its phases need to be executed for each enabled ABI? This is dynamicly checked at runtime, no need to modify the ebuilds and also no needless compilation, when there is no ABI-specific content. A more detailed answer at package manager level: After the src_install phase for the first requested ABI has been finished, the content of $DESTDIR is checked. If there is no ABI specific content, the other enabled ABIs are skipped and the following steps are done as usual. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On Sun, Jul 1, 2012 at 4:52 PM, Thomas Sachau to...@gentoo.org wrote: Matt Turner schrieb: I suppose that's just for ease of implementation? Not having to special-case packages that don't install binaries. I dont follow. Did you think about only having additional ABI flags for certain cases? Right. It seems to me that not having the ABI flags for packages like x11-proto/* would make more sense, but may be more work to implement. Those ABI flags behave the same as other USE flags: When you change them, you have to recompile the package including all the content, that has been compiled previously. If you want the compile for each ABI seperate, then you would have to handle them more like SLOTS, but with a new syntax, with a collision handler for the common content and how to handle that in the user UI. I fear, that this would be way more complicated to implement in a sane way, so i dont plan to go that route. Ouch. I already think a mechanism for telling the package manager you don't need to rebuild this entire package to turn on /this/ USE flag is needed. For ABIs, the situation seems much more clear-cut. In fact, it seems rather important.
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 07/01/2012 02:39 PM, Matt Turner wrote: On Sun, Jul 1, 2012 at 4:52 PM, Thomas Sachau to...@gentoo.org wrote: Matt Turner schrieb: I suppose that's just for ease of implementation? Not having to special-case packages that don't install binaries. I dont follow. Did you think about only having additional ABI flags for certain cases? Right. It seems to me that not having the ABI flags for packages like x11-proto/* would make more sense, but may be more work to implement. It seems like this could easily be controlled by setting a PROPERTIES or RESTRICT value in the ebuild. Also, it might be useful to give the ebuild similar control over the abiwrapper flag via PROPERTIES or RESTRICT. -- Thanks, Zac
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 07/01/2012 02:34 PM, Thomas Sachau wrote: Zac Medico schrieb: On 07/01/2012 04:29 AM, Thomas Sachau wrote: Matt Turner schrieb: On Fri, Jun 29, 2012 at 10:30 AM, Thomas Sachau to...@gentoo.org wrote: I'm interested in this because I'm regularly annoyed with the emul- packages and also because multilib is pretty important for mips. If a package has dependencies, then those dependencies are required to have at least the same targets enabled as the package That seems like the obvious (but perhaps naive) choice. What about depending on packages that don't install libraries, like x11-proto/ packages or generators like dev-util/indent? Maybe I just don't understand. Would these packages even have ABI flags? All packages do get the ABI flags (with the needed EAPI or via enabled portage feature, which is currently in the multilib branch). If a package does not install anything ABI-specific (no headers, no libs and no binaries), then there is no overhead, since it will just get compiled/installed for one ABI, even if multiple ABI flags are enabled. For a package like this that does not install anything ABI-specific, does the package manager still execute phases for each enabled ABI, or is there some way for the ebuild to indicate whether or not its phases need to be executed for each enabled ABI? This is dynamicly checked at runtime, no need to modify the ebuilds and also no needless compilation, when there is no ABI-specific content. A more detailed answer at package manager level: After the src_install phase for the first requested ABI has been finished, the content of $DESTDIR is checked. If there is no ABI specific content, the other enabled ABIs are skipped and the following steps are done as usual. I think it would be helpful to include a short explanation of these kinds of details in the GLEP, in order to help people wrap their heads around the whole thing (possibly avoiding the Huh? kind of reaction that you got from Brian). -- Thanks, Zac
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 07/01/2012 02:52 PM, Zac Medico wrote: On 07/01/2012 02:39 PM, Matt Turner wrote: On Sun, Jul 1, 2012 at 4:52 PM, Thomas Sachau to...@gentoo.org wrote: Matt Turner schrieb: I suppose that's just for ease of implementation? Not having to special-case packages that don't install binaries. I dont follow. Did you think about only having additional ABI flags for certain cases? Right. It seems to me that not having the ABI flags for packages like x11-proto/* would make more sense, but may be more work to implement. It seems like this could easily be controlled by setting a PROPERTIES or RESTRICT value in the ebuild. Also, it might be useful to give the ebuild similar control over the abiwrapper flag via PROPERTIES or RESTRICT. On irc, I asked Tommy about the abiwrapper thing, and he suggested to set IUSE=+abiwrapper for packages containing programs like qmake which are known to produce ABI specific output. -- Thanks, Zac
Re: [gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 07/01/2012 02:34 PM, Thomas Sachau wrote: Zac Medico schrieb: On 07/01/2012 04:29 AM, Thomas Sachau wrote: Matt Turner schrieb: On Fri, Jun 29, 2012 at 10:30 AM, Thomas Sachau to...@gentoo.org wrote: I'm interested in this because I'm regularly annoyed with the emul- packages and also because multilib is pretty important for mips. If a package has dependencies, then those dependencies are required to have at least the same targets enabled as the package That seems like the obvious (but perhaps naive) choice. What about depending on packages that don't install libraries, like x11-proto/ packages or generators like dev-util/indent? Maybe I just don't understand. Would these packages even have ABI flags? All packages do get the ABI flags (with the needed EAPI or via enabled portage feature, which is currently in the multilib branch). If a package does not install anything ABI-specific (no headers, no libs and no binaries), then there is no overhead, since it will just get compiled/installed for one ABI, even if multiple ABI flags are enabled. For a package like this that does not install anything ABI-specific, does the package manager still execute phases for each enabled ABI, or is there some way for the ebuild to indicate whether or not its phases need to be executed for each enabled ABI? This is dynamicly checked at runtime, no need to modify the ebuilds and also no needless compilation, when there is no ABI-specific content. A more detailed answer at package manager level: After the src_install phase for the first requested ABI has been finished, the content of $DESTDIR is checked. If there is no ABI specific content, the other enabled ABIs are skipped and the following steps are done as usual. In case anyone want some more detail, here's a follow-up question from irc: zmedico Tommy[D]: does any ELF executable qualify as abi specific and trigger builds for all ABIs? Tommy[D] zmedico: if it goes into any of /bin /usr/bin /sbin /usr/sbin and you enable the abiwrapper USE flag, yes, otherwise you just request a binary for the default ABI, so no need to rebuild everything for no need -- Thanks, Zac
[gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 07/01/2012 05:39 PM, Matt Turner wrote: On Sun, Jul 1, 2012 at 4:52 PM, Thomas Sachau to...@gentoo.org wrote: Matt Turner schrieb: I suppose that's just for ease of implementation? Not having to special-case packages that don't install binaries. I dont follow. Did you think about only having additional ABI flags for certain cases? Right. It seems to me that not having the ABI flags for packages like x11-proto/* would make more sense, but may be more work to implement. For x11-proto/* you currently do need to build the package for each individual ABI because they install a file in /usr/$(get_libdir)/pkgconfig. This should probably be changed upstream to install to /usr/share/pkgconfig, as there is nothing ABI-specific in there. - -- Jonathan Callen -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCgAGBQJP8O8VAAoJELHSF2kinlg4uToP/1swJ2Dm0W1R5z15L41kCO8F jKtc1Q+3xN/gfDuHzAR1rJpzNLGhOzX0tE+9TKePp4TLcoDKSIeOi9wR+MwncULE aOqIDPQE5n06z06FEuKqkyWUSJZjJ3sObWiH0+hhWssIorks+LEVYZ8EeLNCw0rb 4zmobbckGeMcbZvcvtISw/N7K3QuUoEDM5rj+059ViqShLxzZegEnO8WU/+h1qi7 E6G5PwKK2MhMGHdWT/Gt+yMT6bSQbNpHu78YHElQFuYSagCcBe75JMfjTKup5iu+ 15hM7ZBbiU6yyx45CV6SSXzdqdpkTBkWY7lfrMzXqa3ZtIqgDhmtdxZiNyJ0sHDe dOtYhJC2FGIGkgv3ZarCEqlybOHrgFxYfBSo0dyFzbr9bWebJgb+mg8CWVbNb4j4 3pe3jVOy5r0CNQ6/ZKRN2ZhzMBhjB0sIsoVmnAOFyqKMNCXHTBHNngQ3jMdxkrJD fhBVMkEnRr2Gmzmsgi1DZZMvRI9W/6SJo03zwYrik0VBBqyRubRRvEp7mNJO1kV/ eW+8tfStDyOHXuQherGJfCdZiHdP6puj4ZmySURhFHAyeRTEQ1Gxq+vPDIGed1Jv 23odxq+l4MW9lktPgUIrTeku4zUG//FWva74JPBUld1VyicINSCU7plFkLwF5z0o XEzrHuDZRO+k9Vdi5y1P =RrNC -END PGP SIGNATURE-
[gentoo-dev] Re: GLEP draf for cross-compile support in multilib profiles
On 06/29/2012 04:30 PM, Thomas Sachau wrote: It is interesting, still you need a way to define HOST dependencies (stuff you need that has to be built on the host since you run it, e.g. xcb python code generator), something to play properly with ld and cross-vs-host paths for compilers (that part is _really_ annoying for qemu-chroots) lu -- Luca Barbato Gentoo/linux http://dev.gentoo.org/~lu_zero
