Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Sat, Nov 22, 2014 at 08:37:56PM +, Dimitri John Ledkov wrote: gcc: (wheezy) http://sources.debian.net/src/gcc-4.7/4.7.2-5/debian/README.cross/ (jessie) http://sources.debian.net/src/gcc-4.8/4.8.3-13/debian/README.cross/ Fundamentally the standard way to build cross-compilers from debian sources remained the same and is not broken in any way. The documentation has existed, and still exists, and is still supported... Thanks for pointing this out. When I started cross building, I briefly looked at these and immediately skipped them, because they looked way out of date. I'm sorry, if this decision was flawed. I'm not sure why you refer to gcc-4.8 as the jessie compiler when most of the archive is built with gcc-4.9. I assume that this is a mistake and will correct the url to http://sources.debian.net/src/gcc-4.9/4.9.2-2/debian/README.cross/. So let me reexamine: * emdebian refers to squeeze toolchains. * http://zigzag.lvk.cs.msu.su/~nikita/debian/; The domain name no longer exists. * Then it mysteriously talks about a toolchain-source approach I've never heard off and refers to gcc versions between 3.3 and 4.3. I guess I can skip this part as it feels more like a history lesson. * In section 1.3 it tells me that I need to download a libc for my target. So at least bootstraps appear to be unsupported by the default method (according to the documentation). * Section 2 explains to build cross compilers by setting GCC_TARGET. As of dpkg version 1.17.17 this variable is no longer honoured. What is being built here is a native compiler. My conclusion is that the reference to this documentation is a straw man. The process described therein does not cover relevant cases and does not work either. On the other hand, the unsupported method has been working for me. I'd be happy to help fixing this, but I lack an understanding of how the default method is supposed to work. So I think I've done what I can do by pointing out what is wrong. Should I open a bug against gcc-4.9? Dimitri, it seems to me that your perception of reality is skewed. Your assuming that the default method was working for everyone does not match my reading of this bug log. I have the impression that you are trying to deny the usefulness of the alternate method. It feels roughly equivalent to (hypothetical) systemd proponents denying the usefulness of sysvinit by arguing that sysvinit is broken while at the same time it works for many. It is my understanding that we don't intentionally break stuff unless there is reason to do so. Please, can we rather work on making the default work for everyone and keep the non-default available until a significant amount of users is able to switch? Helmut -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141124083533.ga10...@alf.mars
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Sat, Nov 22, 2014 at 08:51:41PM +, Dimitri John Ledkov wrote: On 22 November 2014 at 16:21, Ron r...@debian.org wrote: Dimitri wrote: Thus multiarch cross tooling is not so relevant for fresh bootstraps, and/or targeting non-debian architectures, or otherwise incomplete systems (e.g. those that do not have compatible set of pre-compiled binaries that use multiarch-paths I'll leave it to Helmut to talk about his existing work with bootstraps that's been stalled by reverting this patch. I can categorically say though that we are currently using a toolchain built this way on Jessie before it was broken by this change, both for embedded systems that do run Debian, and for microcontrollers that couldn't possibly run it (memory measured in kB, no MMU). It works just fine for all of those cases. The *only* problem we have at present is that we can no longer update the Jessie systems this was being done on, because our ability to do this was removed and there appears to be no actually working replacement for it. Also since it is a source package change, rebuild outside the archive, one is free to patch it, thankfully the source packaging is open source which one can patch when rebuilding toolchains in the partially new to jessie ways. So ... you're saying that ... if someone manually restores this for themselves, they can have back the only behaviour that has been tested and is known to work (well or at all) for Jessie with m-a, but if it is restored in advance, so our users don't have to do that manually ... then the universe somehow comes to a cataclysmic end? Can you point us to the actual explanation of what *breaks*? The people who have actually been working on this _in Debian_ are well aware of what is not perfect about it and what extra work remains for post-Jessie to make it more so, and they are actually working on those things. They even have packages based on this uploaded to sid, so that the other work on fixing those things can continue, e.g.: https://packages.debian.org/sid/gcc-4.9-arm-linux-gnueabi What nobody has explained to us is what problem is *fixed* by gratuitously breaking this for existing users of Jessie. Can we all settle and move these developments to experimental targeting for stretch, instead? Nobody is suggesting that other options can't be, or shouldn't be, explored for post-Jessie. Restoring the functionality that existed before this was removed will not in any way prevent or hinder that, it just means we won't repeat the sad state of Wheezy where this became no longer possible at all. Nothing you said here explains why we can't have the best of both worlds with this. If having this working for Jessie is not so relevant to you, that's fine - but it's very relevant to quite a few other people and was working for them until a few weeks ago when support for it was simply removed. We have people prepared to do the work to keep it working. What we don't have is an explanation of what it actually broke, if anything, to warrant removing it, without comment or warning, at this late stage of the Jessie release. This bug report annoys me a lot, given the amount of inaccuracies it has in portraying the current state of the affairs - that is exaggerating the regression, when simply a desired feature by some, failed pear-review was found feature incomplete and was not fully included into jessie. You haven't actually pointed out anything inaccurate that anyone has said here, and having this working is actually a pre-requisite for some of the other work that you're saying isn't yet done, to be done. Yes, it's still a bit more awkward to build a toolchain than we all would like, but this is still infinitely easier and cleaner than anything we've ever had before. What do we gain by denying people the ability to easily use and experiment with that, in real life use cases, while we work on improving the other things too? Can you point us to this pear-review that you say occurred? The only thing I can see is that this functionality was quietly removed on the eve of the freeze, without any discussion with the people working on this _in Debian_, and that trying to discuss that is what led to this going pear-shaped and needing to be escalated to the -ctte for mediation. Given a zero sum game rules, Fortunately, this isn't a zero sum game. We can have the Good now without ruining anyone's chances of making that Perfect later. Why is it in the interest of Debian or the users of Jessie to not (continue to) have that? Ron -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141123112352.gv29...@hex.shelbyville.oz
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On 23 November 2014 at 11:23, Ron r...@debian.org wrote: On Sat, Nov 22, 2014 at 08:51:41PM +, Dimitri John Ledkov wrote: On 22 November 2014 at 16:21, Ron r...@debian.org wrote: Dimitri wrote: Thus multiarch cross tooling is not so relevant for fresh bootstraps, and/or targeting non-debian architectures, or otherwise incomplete systems (e.g. those that do not have compatible set of pre-compiled binaries that use multiarch-paths I'll leave it to Helmut to talk about his existing work with bootstraps that's been stalled by reverting this patch. I can categorically say though that we are currently using a toolchain built this way on Jessie before it was broken by this change, both for embedded systems that do run Debian, and for microcontrollers that couldn't possibly run it (memory measured in kB, no MMU). It works just fine for all of those cases. The *only* problem we have at present is that we can no longer update the Jessie systems this was being done on, because our ability to do this was removed and there appears to be no actually working replacement for it. Also since it is a source package change, rebuild outside the archive, one is free to patch it, thankfully the source packaging is open source which one can patch when rebuilding toolchains in the partially new to jessie ways. So ... you're saying that ... if someone manually restores this for themselves, they can have back the only behaviour that has been tested and is known to work (well or at all) for Jessie with m-a, but if it is restored in advance, so our users don't have to do that manually ... then the universe somehow comes to a cataclysmic end? no, you are mangling my words, and not being constructive with me. I don't know what's your involvement in this is, but I don't like working with you. Please stop. The only behaviour that has been tested and is known to work (well or at all) for Jessie with m-a has been available for a long time, and is unchanged from stable and in testing. Which bit do you not understand from this? And/or any parts of documentation I've sent in the other email? Can you point us to the actual explanation of what *breaks*? The newly introdued mualtiarch cross building in jessie to a few packages: * cannot be build on standard debian buildds * cannot build multilib toolchains * and thus resulting toolchains cannot rebuild non-trivial core debian packages These reasons have been pointed out to the people raising this bug report before. If anyone missed that for any reason, it is pointed out now. I'm not sure if you are deliberately missing portions I've emails that I sent. The people who have actually been working on this _in Debian_ are well aware of what is not perfect about it and what extra work remains for post-Jessie to make it more so, and they are actually working on those things. They even have packages based on this uploaded to sid, so that the other work on fixing those things can continue, e.g.: https://packages.debian.org/sid/gcc-4.9-arm-linux-gnueabi This package is not build on Debian and cannot be ever rebuild on Debian. And RC bug reports are filed. This concrete example is very good way to show that its upload is pre-mature. The reason is quite simple, that we do not have foreign architectures enabled on the builders, nor any easy way to enable them (being or has been fixed in sbuild), however on-demand enabling foreign architectures will not be in place until only after one stable release where it is possible to do so and buildds getting upgraded to such release. What nobody has explained to us is what problem is *fixed* by gratuitously breaking this for existing users of Jessie. The problem is introducing incomplete broken things into archive. E.g. the above uploaded package into sid FTBFS on any release: sid, testing or stable. This is obsurd to call it as actually working, and no changes to gcc-4.X packages will make cross-gcc-4.9-armel finally build from source on debian infrastructure. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766625 Can we all settle and move these developments to experimental targeting for stretch, instead? Nobody is suggesting that other options can't be, or shouldn't be, explored for post-Jessie. Restoring the functionality that existed before this was removed will not in any way prevent or hinder that, it just means we won't repeat the sad state of Wheezy where this became no longer possible at all. Nothing you said here explains why we can't have the best of both worlds with this. If having this working for Jessie is not so relevant to you, that's fine - but it's very relevant to quite a few other people and was working for them until a few weeks ago when support for it was simply removed. We have people prepared to do the work to keep it working. What we don't have is an explanation of what it actually broke, if anything, to
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Dimitri John Ledkov [2014-11-23 12:27 +]: On 23 November 2014 at 11:23, Ron r...@debian.org wrote: On Sat, Nov 22, 2014 at 08:51:41PM +, Dimitri John Ledkov wrote: On 22 November 2014 at 16:21, Ron r...@debian.org wrote: Dimitri wrote: The newly introdued mualtiarch cross building in jessie to a few packages: * cannot be build on standard debian buildds Not yet, but all the code to do this exists. The necessary sbuild is in Jessie. The necessary wanna-build patches are here: http://anonscm.debian.org/cgit/users/dkogan-guest/wanna-build.git/ (awaiting review and the stable release before potentially problematic wanna-build changes are actually applied) * cannot build multilib toolchains Correct (although this could possibly be fixed). * and thus resulting toolchains cannot rebuild non-trivial core debian packages There are _lots_ of debian packages that currently don't cross-build, for lots of reasons. A tiny number of packages _need_ multilib to build (SFAIK). So whilst this is an issue to consider, it is a fairly minor one on the scale of things. The cross-toolchains remain a) the only ones available in the archive and b) useful for building _most_ debian packages (and other stuff). These reasons have been pointed out to the people raising this bug report before. If anyone missed that for any reason, it is pointed out now. As as has been said numerous times in this thread, no-one is suggesting that the current cross-toolchains are immutable and can't be improved/replaced, but until we can actually build the alternatives (Have you fixed cross-toolchain-base for debian yet?) there is no good reason to break what is currently working (and in fact having that working _still_ isn't a good reason for breaking the 'mulitarch-multiarch' builds). I'm not sure if you are deliberately missing portions I've emails that I sent. The people who have actually been working on this _in Debian_ are well aware of what is not perfect about it and what extra work remains for post-Jessie to make it more so, and they are actually working on those things. They even have packages based on this uploaded to sid, so that the other work on fixing those things can continue, e.g.: https://packages.debian.org/sid/gcc-4.9-arm-linux-gnueabi This package is not build on Debian Yes it is and cannot be ever rebuild on Debian. Nonsense. Of course it can. And RC bug reports are filed. And they will cease to be RC as soon as Jessie is released and wanna-build updated. The work has been done. This concrete example is very good way to show that its upload is pre-mature. The reason is quite simple, that we do not have foreign architectures enabled on the builders, nor any easy way to enable them (being or has been fixed in sbuild), Yes we do. Building these packages in the Sbuild in Jessie 'just works' already. Try it: sbuild -A -s -d unstable -c sid-amd64-sbuild cross-gcc-4.9-armhf (this will fail immediately after a new gcc upload until the cross-build-deps are built, because its build-deps are not available, just as any other package would)) however on-demand enabling foreign architectures will not be in place until only after one stable release where it is possible to do so and buildds getting upgraded to such release. It will be (is!) in place in Jessie. It's had limited testing so there may prove to be problems in practice, but our testing so far indicates that it works fine. All the packages uploaded to unstable (and for Jessie in my p.d.o repo) are built in standard jessie/unstable sbuild. What nobody has explained to us is what problem is *fixed* by gratuitously breaking this for existing users of Jessie. The problem is introducing incomplete broken things into archive. 'incomplete broken' is not fair. They are quite new and we are awaiting infrastructure changes to be applied by the buildd maintainers, but that's not going to happen until after the stable release now. But the packages themselves are now in quite good shape. Unstable now contains cross-compilers for all the arches in Jessie (for amd64), built using standard sbuild. Including cross-gcc-defaults to add the wanted symlinks for all arches except mips (because mips was lagging badly at the time of the original upload so I missed that one out - this has just been corrected in cross-gcc-defaults 0.4, currently in NEW). They work to build all (most?) of the packages in Debian which are cross-buildable at all and whose cross-build-deps are installable: (e.g. test on 100 packages here: http://people.linaro.org/~wookey/buildd/testing/sbuild/latest/status.html ) Yes there is plenty of stuff that doesn't cross-build but that's not because these toolchains are particularly 'incomplete', or 'broken'. You'd get the exact same failure set with a standalone cross-toochain too, SFAIK. Convenience 'crossbuild-essential' packages could be in unstable too, but the maintainer (Doko) has only
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Dimitri wrote: Thus multiarch cross tooling is not so relevant for fresh bootstraps, and/or targeting non-debian architectures, or otherwise incomplete systems (e.g. those that do not have compatible set of pre-compiled binaries that use multiarch-paths I'll leave it to Helmut to talk about his existing work with bootstraps that's been stalled by reverting this patch. I can categorically say though that we are currently using a toolchain built this way on Jessie before it was broken by this change, both for embedded systems that do run Debian, and for microcontrollers that couldn't possibly run it (memory measured in kB, no MMU). It works just fine for all of those cases. The *only* problem we have at present is that we can no longer update the Jessie systems this was being done on, because our ability to do this was removed and there appears to be no actually working replacement for it. Can we all settle and move these developments to experimental targeting for stretch, instead? Nobody is suggesting that other options can't be, or shouldn't be, explored for post-Jessie. Restoring the functionality that existed before this was removed will not in any way prevent or hinder that, it just means we won't repeat the sad state of Wheezy where this became no longer possible at all. Nothing you said here explains why we can't have the best of both worlds with this. If having this working for Jessie is not so relevant to you, that's fine - but it's very relevant to quite a few other people and was working for them until a few weeks ago when support for it was simply removed. We have people prepared to do the work to keep it working. What we don't have is an explanation of what it actually broke, if anything, to warrant removing it, without comment or warning, at this late stage of the Jessie release. Ron -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141122162145.gu29...@hex.shelbyville.oz
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On 21 November 2014 at 19:21, Sam Hartman hartm...@debian.org wrote: Dimitri == Dimitri John Ledkov x...@debian.org writes: Dimitri Comparing squeeze and jessies - have things regressed? if Dimitri yes, how? As far as I expect, the way one uses debian Dimitri source packaging to produce cross toolchains has not Dimitri changed, nor has been affected by changes in jessie, in Dimitri comparison to squeeze. Can you give a pointer to documentation of this--the mechanism that works in squeeze and jessie? From the IRC conversation today I think the answer *may* be no that's not written up anywhere, but I'm not sure. Generally rebuilding cross variants of a given toolchain package are documented inside the said toolchain package as additional interfaces a said source package supports. Binutils: (wheezy) http://sources.debian.net/src/binutils/2.22-8/debian/README.cross/ (jessie) http://sources.debian.net/src/binutils/2.24.51.20140411-2.1/debian/README.cross/ Linux (can use any): make deb-pkg, targets upstreamed can be used to create cross header packages Other toolchain libraries: most of them support cross-building in a standard way, that is by setting DEB_HOST_* variables dpkg-architecture -aarmhf -c dpkg-buildpackage -b - to cross-compile most packages for a different target OS, armhf in this example gcc: (wheezy) http://sources.debian.net/src/gcc-4.7/4.7.2-5/debian/README.cross/ (jessie) http://sources.debian.net/src/gcc-4.8/4.8.3-13/debian/README.cross/ Fundamentally the standard way to build cross-compilers from debian sources remained the same and is not broken in any way. The documentation has existed, and still exists, and is still supported... -- Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CANBHLUhR9A9Snns11+2WzoE20pLry5Xrt7rCT=siwwomwgp...@mail.gmail.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On 22 November 2014 at 16:21, Ron r...@debian.org wrote: Dimitri wrote: Thus multiarch cross tooling is not so relevant for fresh bootstraps, and/or targeting non-debian architectures, or otherwise incomplete systems (e.g. those that do not have compatible set of pre-compiled binaries that use multiarch-paths I'll leave it to Helmut to talk about his existing work with bootstraps that's been stalled by reverting this patch. I can categorically say though that we are currently using a toolchain built this way on Jessie before it was broken by this change, both for embedded systems that do run Debian, and for microcontrollers that couldn't possibly run it (memory measured in kB, no MMU). It works just fine for all of those cases. The *only* problem we have at present is that we can no longer update the Jessie systems this was being done on, because our ability to do this was removed and there appears to be no actually working replacement for it. The standard way to build cross toolchains, the same way as in current stable release has worked and still works. This is only about newly added, incomplete features, which unfortunately are incomplete for jessie and have been sealed in the packging to not expose them. Since this is a source package, which is rebuild using out of the archive infrastructure with out of the archive procedures there is nothing in the archive that is broken thus imho this is out of scope for a tech committee to rule. Also since it is a source package change, rebuild outside the archive, one is free to patch it, thankfully the source packaging is open source which one can patch when rebuilding toolchains in the partially new to jessie ways. Can we all settle and move these developments to experimental targeting for stretch, instead? Nobody is suggesting that other options can't be, or shouldn't be, explored for post-Jessie. Restoring the functionality that existed before this was removed will not in any way prevent or hinder that, it just means we won't repeat the sad state of Wheezy where this became no longer possible at all. Nothing you said here explains why we can't have the best of both worlds with this. If having this working for Jessie is not so relevant to you, that's fine - but it's very relevant to quite a few other people and was working for them until a few weeks ago when support for it was simply removed. We have people prepared to do the work to keep it working. What we don't have is an explanation of what it actually broke, if anything, to warrant removing it, without comment or warning, at this late stage of the Jessie release. The newly introdued mualtiarch cross building in jessie to a few packages: * cannot be build on standard debian buildds * cannot build multilib toolchains * and thus resulting toolchains cannot rebuild non-trivial core debian packages These reasons have been pointed out to the people raising this bug report before. If anyone missed that for any reason, it is pointed out now. These packages cannot be build on standard debian buildds, as it requires for multiarch to be enabled and access available to foreign arch packages. This is not currently available by default and does lead to unpredictable builds at times (especially in sid - when uninstallability on one arch will cause build dependencies / dependencies to be resolved from the wrong arch where things are installable). The proposed multiarch-multiarch toolchain patches do not accomodate to build multilib tool-chains, on which our distribution currently relies to build many core libraries, bootloaders and etc. A solution for cross-toolchains which excludes multilib support and no commitments to implement that is not sufficient for Debian needs. This bug report annoys me a lot, given the amount of inaccuracies it has in portraying the current state of the affairs - that is exaggerating the regression, when simply a desired feature by some, failed pear-review was found feature incomplete and was not fully included into jessie. This bug and assertions made, are also taking away my free time that I have to work on Debian. Thus instead of making progress on packaging toolchain-base, I'm spent arguing here. Given a zero sum game rules, is a net negative for all parties involved. -- Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CANBHLUhVWPyYVJKw=tyb3zdb8v4ch20xn3m41qn7f+yd1-k...@mail.gmail.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Reading this bug report title history, it is very misleading. Building cross-toolchains, and cross-toolchains that are multiarch compatible has been possible to do before (stable) and is possible in current planned release (testing). I have provided the documentation links to that in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766708#187 New feature of gcc packaging was developed in jessie - building multiarch compatible toolchains bootstrapped with aid from pre-existing foreign-arch multiarch packages. This new feature has been enabled and exposed, however subsequently failed peer review found to be incomplete for jessie and thus public interface to those codepaths have been made private. This has been communicated to all parties involved. This bug has then been opened, and reading the history of it there are quite a few inaccuracies in it. To resolve deficiencies of the proposed patches further work is needed that is being undertaken. Specifically landing improvements to our buildd network sbuild to enable multiarch repository access during building, packaging/preparing toolchain-base packages (that is building host-arch libc linux-headers without requiring enabling multiple architectures neither during their build, nor buildds, or the resultant toolchain time nor requiring access to multiarch repositories). Not only that is stand-alone, it also is more resilient against architecture skew. For more details see wookey and/or hist talk at Mini-Debocnf 2013 in Cambridge. In that sense, developers involved are working on a consensus way to improve cross-building in Debian. There is no need for a ruling or overrides - since there is no completed solution available yet. The original request here seems to originate from dissatisfaction that proposed patches failed peer review and ended up applied, but disabled. Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/canbhlujrxabadbd92bmbjkk-w-6sfp7krhv6kdm+dnmc+cc...@mail.gmail.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Helmut wrote: On Wed, Nov 19, 2014 at 04:41:49PM -0800, Don Armstrong wrote: Are people who are doing cross-building like this actually using the code which will be in jessie? I (perhaps naïvely) would expect them to be primarily using the code in unstable, and maybe at a late stage of bring-up rebuilding all of stable. Thanks for asking this. Let me give two answers to this question: 1) No. The continuous integration happened in sid and people bootstrapping new architectures tend to use sid plus patches. However the method of bootstrapping that was known working two weeks before the freeze no longer works. Whatever method is going to be used now, it requires changing packages. Since these changes tend to not fall under the freeze policy, they are practically not mergeable. So in this answer, jessie is to be understood as a time frame: Keep things working that worked before until we are allowed to fix things. 2) Yes. People repeatedly ask for cross toolchains on stable systems. This is the very reason why Wookey tried to package them for jessie. Ultimately, they ended being late, so people will try to build them on their own and for the popular targets (mostly armhf, armel) this actually worked. We've been using this for development of embedded systems where building on native hardware (or in an emulator) is ridiculously slow (or even impossible for some devices) compared to cross building from Debian on more powerful hardware. That worked great for all releases up to and including Squeeze. For Wheezy, the late introduction of multi-arch basically broke that and doing this on that release was effectively impossible. The change which was reverted here had made doing that on Jessie possible once again, and made it a relatively trivial matter to build your own cross-toolchain using the packaged gcc source (in the absence of those being able to actually be uploaded as pre-built binaries yet). I'd be kind of sad if that stopped being possible again for the final released version of Jessie, and we had to skip yet another release before being able to do this on Debian again. It may not be the best and final answer, but it has some advantages over the proposed alternative, like being able to work with existing m-a packages without needing to rebuild custom versions of everything, and actually working on Debian today. It's not clear to me what advantage was gained by removing it before the alternative to it is actually viable to use, or what problem it had that made doing that compelling. Unless someone can show me that, I'd definitely prefer to have this functionality restored again for Jessie. It's definitely desirable to have this on a stable system, since the lock-step requirement of m-a makes it rather more painful to keep this all working on an unstable system where packages are changing rapidly (and because stable deps are kind of an important thing too :) Cheers, Ron -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141121144342.go29...@hex.shelbyville.oz
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Ron == Ron r...@debian.org writes: Ron I'd be kind of sad if that stopped being possible again for the Ron final released version of Jessie, and we had to skip yet Ron another release before being able to do this on Debian again. Ron It may not be the best and final answer, but it has some Ron advantages over the proposed alternative, like being able to Ron work with existing m-a packages without needing to rebuild Ron custom versions of everything, and actually working on Debian Ron today. I personally think that this use case--building on non-native hardware not for bootstrap but for things where that really just is the wrong answer is an important use case for our toolchains. I'm not saying it is an important use case for the debian archive. However for debian developers using debian and for our downstreams, this seems like it can be very valuable. In particular, I want to take the Debian archive or one of our downstreams, build a cross tool chain, and build additional packages against that archive that would work with the packages in that archive. As an example, I'd like to build some of the prerelease moonshot packages (http://www.project-moonshot.org/) for some arm boards and I don't want to build on those boards. I want the resulting packages to be usable by anyone without needing to install some magic cross toolchain libraries. I don't care which mechanism ]is used to produce this. I understand how to doo this with the method being removed in Jessie. I can't even understand if this is possible with the default method. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/tslr3wwvawb@mit.edu
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Comparing squeeze and jessies - have things regressed? if yes, how? As far as I expect, the way one uses debian source packaging to produce cross toolchains has not changed, nor has been affected by changes in jessie, in comparison to squeeze. Multiarch cross-building - a brand new set of features in a set of debian source packages, is only interesting for targetting existing debian architectures as it relies on the existing presence of packages from said architectures to exist (e.g. to use the newly, partially complete multiarch cross tooling to make a cross from A to B, libc6:B must already exist). Thus multiarch cross tooling is not so relevant for fresh bootstraps, and/or targeting non-debian architectures, or otherwise incomplete systems (e.g. those that do not have compatible set of pre-compiled binaries that use multiarch-paths, and as far as I can tell only debian and debian derivates have adopted support for multiarch toolchain paths). Granted, the multiarch cross toolchain tooling almost made it into jessie, however it is incomplete and from maintainer's point of view not supportable in a stable release. Can we all settle and move these developments to experimental targeting for stretch, instead? Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/canbhluhqdmzkz+dus4h0rqdgp63+aeyjsnrf6drcscnwj6p...@mail.gmail.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Dimitri == Dimitri John Ledkov x...@debian.org writes: Dimitri Comparing squeeze and jessies - have things regressed? if Dimitri yes, how? As far as I expect, the way one uses debian Dimitri source packaging to produce cross toolchains has not Dimitri changed, nor has been affected by changes in jessie, in Dimitri comparison to squeeze. Can you give a pointer to documentation of this--the mechanism that works in squeeze and jessie? From the IRC conversation today I think the answer *may* be no that's not written up anywhere, but I'm not sure. --Sam -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/tslegswtl5m@mit.edu
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Ben Longbons [2014-11-19 22:18 -0800]: If the cross tools miss jessie and go in jessie-backports, that will require a significant amount of knowledge and discipline on the part of all the package maintainers involved, to make sure that they always have matching versions despite being in different repos or they will not be useful for package developers. If they are treated as one package and go in one repo, everybody is happy. The cross-tools (apart from binutils) have missed Jessie. The release team are not going to change their minds about that. The issue of this bug was not the only problem there: missing work on britney and wanna-build means they wouldn't have migrated in time independently of this issue and I was not able to persuade the release team to make a special exception on 'release goal' grounds. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141120233309.gd27...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Don Armstrong [2014-11-19 16:41 -0800]: On Tue, 28 Oct 2014, Helmut Grohne wrote: I have to admit that the code is not exactly lightweight. I do understand the desire to get rid it and asked that a ctte ruling does not apply beyond jessie for that reason. Are people who are doing cross-building like this actually using the code which will be in jessie? I (perhaps naïvely) would expect them to be primarily using the code in unstable, and maybe at a late stage of bring-up rebuilding all of stable. I think you are right, at least for a while. The situation is as follows: Jessie: The only cross-toolchains currently available for jessie are built using with_deps_... and are available (outside the archive) to be installed. They will be updated if the gcc version in jessie is. I'm not sure to what degree anyone else will be doing much rebuilding of these packages. People might try (e.g. I just tried rebuilding them in Utopic as someone asked if that would work). Unstable: I am building cross-toolchains against each new gcc-4.9 upload to unstable, using the code in unstable, and expect to keep doing this. Those builds also use the with_deps.. method, and thus currently need the with_deps.. disabling patched out to work. The build method may change during the life of unstable, but how that will play out is not clear yet. So for me I expect to be using this functionality in unstable much more than in jessie which should be more or less stable now. Again I'm not sure how much other people will be rebuilding these packages or otherwise building cross-toolchains from gcc. There will certainly be some, especially if rebuilds in unstable are not kept up-to-date (currently a manual process of fresh cross-gcc-* source uploads, but we plan to automate this). Boostrapping tests will do it regularly. New porters will do it. Currently everyone will be using with_deps because that's the choice (after patching gcc). They may choose to build the standalone way once it's actually working. At least 3 of us are prepared to maintain the with-deps packaging rules. IMHO it makes a lot more sense to maintain it in gcc packagig where it already is rather than do it outside as a big quilt stack, but that won't work if the maintainer doesn't apply patches. I just filed 770413, for example. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141121043802.gf27...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Tue, 28 Oct 2014, Helmut Grohne wrote: I have to admit that the code is not exactly lightweight. I do understand the desire to get rid it and asked that a ctte ruling does not apply beyond jessie for that reason. Are people who are doing cross-building like this actually using the code which will be in jessie? I (perhaps naïvely) would expect them to be primarily using the code in unstable, and maybe at a late stage of bring-up rebuilding all of stable. -- Don Armstrong http://www.donarmstrong.com There's nothing remarkable about it. All one has to do is hit the right keys at the right time and the instrument plays itself. -- Bach -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141120004149.gg4...@teltox.donarmstrong.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Wed, Nov 19, 2014 at 4:41 PM, Don Armstrong d...@debian.org wrote: Are people who are doing cross-building like this actually using the code which will be in jessie? I (perhaps naïvely) would expect them to be primarily using the code in unstable, and maybe at a late stage of bring-up rebuilding all of stable. Important use case that people seem to be ignored: cross-building source code but *not* using Debian infrastructure. This still matters to Debian because it affects how much support both upstream and the package maintainers can give without having access to actual hardware. If cross-compilers are available in the exact same version (which, depending on where the bug is found, may be in the stable release or not), it is *much* easier for the people with more intimate knowledge of the package to directly support it. The code I work on isn't packaged for Debian yet, but without having cross-compilers to play with, I will *never* be able to support anything other than x86-*. I was using secretsauce builds for a while on my dev machine (when I think to test them), but that's not suitable for running my buildbot. I got excited that it was landing in time for Jessie, but because of this maintainer turf war it looks like I'll be another 3 years (i.e. until 'Stretch' is stable) without CI for non-x86 arches. I expect that my code will be ready to be packaged for Stretch, but who is going to take responsibility for finding and fixing the non-x86 bugs, given that the gcc maintainer is going out of his way to make things difficult? Honestly, disabling cross-compilers sounds like saying x86-* are the only arches that matter and negating the entire selling point of multiarch. I can't seriously believe the argument against cross-compilers is you need to enable multiarch repos - where else are you going to get all your library dependencies - bring back an entire set of ia32-libs packages, but for *every* arch? This is ludicrous, but it sounds like what the gcc maintainer is recommending. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/ca+xnfzmwhiqtbdsl9e9qovnnzdk2rerbq0xshu2v9elawy-...@mail.gmail.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Wed, 19 Nov 2014, Ben Longbons wrote: The code I work on isn't packaged for Debian yet, but without having cross-compilers to play with, I will *never* be able to support anything other than x86-*. Which suite are you currently using? I'm asking, because I want to know whether actually just doing this for jessie is enough, or whether doing this for unstable is enough, or whether we have to do it for jessie and unstable and then get a complete, tested, multiarch procedure which works in Debian which is acceptable to both porters and the gcc maintainers. This is ludicrous, but it sounds like what the gcc maintainer is recommending. Please try not to use language which escalates; calling something ludicrous isn't helpful. -- Don Armstrong http://www.donarmstrong.com The game of science is, in principle, without end. He who decides one day that scientific statements do not call for any further test, and that they can be regarded as finally verified, retires from the game. -- Sir Karl Popper _The Logic of Scientific Discovery_ §11 -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141120020535.gh4...@teltox.donarmstrong.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Wed, Nov 19, 2014 at 6:05 PM, Don Armstrong d...@debian.org wrote: On Wed, 19 Nov 2014, Ben Longbons wrote: The code I work on isn't packaged for Debian yet, but without having cross-compilers to play with, I will *never* be able to support anything other than x86-*. Which suite are you currently using? I'm asking, because I want to know whether actually just doing this for jessie is enough, or whether doing this for unstable is enough, or whether we have to do it for jessie and unstable and then get a complete, tested, multiarch procedure which works in Debian which is acceptable to both porters and the gcc maintainers. Our production server runs stable (i.e. Wheezy now, but we plan to upgrade not long after Jessie is released), and by sanity the CI server runs the same as production (and currently does x86-* builds only since there are no cross tools in Wheezy). My dev machine runs testing (Jessie), but it is *very* common for CI to catch errors that I can't, due to tiny differences between versions (gcc-4.7 4.7.2 in Wheezy has a frustrating bug with arrays of string literals but gcc-4.7 4.7.3 in Jessie was fine; the Debian-specific packaging bugs in libstdc++-dbg that have regressed in *every* new gcc release; several subtle between gdb versions (e.g. printing null pointers as '0' instead of '0x0' and breaking my script that checks gdb output) and of course the catastrophic bug #748711 (at least there's a migration plan for Stretch now (gdb-python2 package in experimental))). So, if you want upstream and Debian maintainers to support a package in $suite, you need the full set of cross tools and multiarch libs in $suite, not just in unstable. I've proven with the cross tools in unstable that my program *should* work on non-x86-* platforms, but real-world reports indicate that it does *not* on stable (Wheezy), on at least one architecture. If the cross tools miss jessie and go in jessie-backports, that will require a significant amount of knowledge and discipline on the part of all the package maintainers involved, to make sure that they always have matching versions despite being in different repos or they will not be useful for package developers. If they are treated as one package and go in one repo, everybody is happy. -Ben -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/ca+xnfzoyze6itmau2hgzrrst6pgl+ge+6vfc6yikfznkasp...@mail.gmail.com
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Hi Don, On Wed, Nov 19, 2014 at 04:41:49PM -0800, Don Armstrong wrote: Are people who are doing cross-building like this actually using the code which will be in jessie? I (perhaps naïvely) would expect them to be primarily using the code in unstable, and maybe at a late stage of bring-up rebuilding all of stable. Thanks for asking this. Let me give two answers to this question: 1) No. The continuous integration happened in sid and people bootstrapping new architectures tend to use sid plus patches. However the method of bootstrapping that was known working two weeks before the freeze no longer works. Whatever method is going to be used now, it requires changing packages. Since these changes tend to not fall under the freeze policy, they are practically not mergeable. So in this answer, jessie is to be understood as a time frame: Keep things working that worked before until we are allowed to fix things. 2) Yes. People repeatedly ask for cross toolchains on stable systems. This is the very reason why Wookey tried to package them for jessie. Ultimately, they ended being late, so people will try to build them on their own and for the popular targets (mostly armhf, armel) this actually worked. Helmut -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141120075120.ga4...@alf.mars
Re: Bug#766708: breaks multiarch cross building
Prior to these changes it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. I may be missing something here. It's true that this change sets defaults for these make variables and overrides values set in the environment. However, since the override directive is not used, can't you still override this by using command-line options to make? (This message should not be read as encouragement to add the override directive; the make documentation explicitly says that it was not invented for escalation in the war between makefiles and command arguments.) The 4.9.2-1 gcc 4.9 upload adds the override directive. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141107034222.gp28...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Helmut Grohne [2014-11-01 10:38 +0100]: On Sat, Nov 01, 2014 at 01:46:48AM +, Wookey wrote: To me that sounds like this method is actually the current de-facto default in Debian - it is certainly at least on a par. I don't think that a feature being de-facto default is a good argument to force maintaining it forever. Well, things being used is a good reason for maintaining them. Nothing is 'forever'. with_deps_on_target_arch_pkgs is currently just as well-used as the alternative. I was just trying to get across that's it's not just some obscure feature no-one cares about. There clearly is need to simplify gcc packaging and one way to do that is to remove one of the cross toolchain build methods. Is there 'clearly a need'? The gcc packaging is certainly complicated, and we'd all welcome simplification, but I'm not sure there is actualyl much scope for that int he real world. A great deal of the complexity comes from bi-arch and tri-arch multilib stuff, for example. I'd argue that more use of multiarch would mean we could drop much of that, at which point the with_deps_on_target_arch_pkgs stuff is a net (large) simplification. But so far as I know this is not something the gcc maintainer is at all interested in. There is a set of source packages (uploaded as one per target arch, for manageability reasons, but actually all coming from the same git tree) that builds cross-compilers from gcc-source. These builds currently use the with_deps_on_target_arch_pkgs because a) it works and b) it's cleaner and simpler. Undeniably, the resulting version lock has been brought forward as an argument earlier. The cross toolchains resulting from with_deps_on_target_arch_pkgs are more fragile than those resulting from the default method in the sense that they are subject to the Multi-Arch version lock problem. See #766966 as an example. Yep. That is clearly the most significant disadvantage of this packaging. It is an issue in unstable, with frequent gcc uploads but not in stable and it shouldn't be in testing either. Note that in practice as soon as you do any multiarch cross-building you are subject to the same constraint (via libgcc1) so the practial difference is relatively small. So someone doing kernel cross-building in unstable would really notice a difference. Anyone multiarch-building packages or using testing or stable would not. (I believe - subject to confirmation by testing). In the worst case, this can prevent the main gcc-X.Y packages from migrating to testing, so clearly gcc maintenance is affected. How? The cross-toolchains depends on the native toolchain libraries. I don't see what circumstance would make the cross-toolchains prevent the main gcc-X.Y packages from migrating I am. It's simple and reliable and (at least IMHO) more correct. There are tradeoffs between the two methods which I'm happy to continue to discuss and work on, but I want it kept around and working (and will help do that) until either consensus is reached amongst the gcc/cross-gcc/rebootstrap maintainers, or we decide that that's simply not possible. I think that it is obvious now that simple, reliable or correct are not universally agreed upon. For this reason, I have avoided them and resorted to arguments that assess whether a method is working (now) and whether its source is available in the archive. I have also been clear about stating objective or subjective things. If you look at the packaging of cross-gcc and cross-toolchain-base it is obvious that the former is _much_ simpler (5K rules, 12 targets vs 25K rules, 29 targets). Now to be clear, the difference in the gcc build alone between with_deps_on_target_arch_pkgs (multiarch-build-deps) and -cross standalone build-deps is nothing. The difference is in the building of linux-libc-dev (-cross), libc6-dev(-cross), libgcc1(-cross), libstdc++(-cross) etc. and the glibc/gcc bootstrap dance. That part is negligible (already done) if the multiarch-build-deps build is used and complicated if it isn't. Similarly, after a lot of messing with this it is clear to me that the simple build is 'reliable' because there is (much) less to go wrong. I've just spent another couple of hours with the standalone powerpc-cross-toolchain-base_1.2d Mattias sugested that I use as a base and have not got it to build on unstable yet. So maybe 'reliable' is just another way of stating 'simple', but it is an objective measure. And I did qualify 'more correct' as 'IMHO', as that clearly is subjective, I agree. If with_deps_on_target_arch_pkgs is going to be used for a long time, the code that makes it work likely needs to stay somewhere other than src:gcc-X.Y (because it is Matthias' right to not maintain it). Once jessie is released, moving this functionality elsewhere is feasible, so I stress that I would not like to see a ruling that forces with_deps_on_target_arch_pkgs onto src:gcc-X.Y beyond jessie. As you say, he is
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Sat, Nov 01, 2014 at 01:46:48AM +, Wookey wrote: To me that sounds like this method is actually the current de-facto default in Debian - it is certainly at least on a par. I don't think that a feature being de-facto default is a good argument to force maintaining it forever. There clearly is need to simplify gcc packaging and one way to do that is to remove one of the cross toolchain build methods. Note that the simpler with_deps_on_target_arch_pkgs build is only useful where the foreign-arch packages are already built, which makes it seem like the 'toolchain-base' method must be used for bootstrapping. However Helmut's rebootstrap has demonstrated that the with_deps_on_target_arch_pkgs method is useful there too. I must admit that I have not fully grokked how this works as I had thought that this was the one case where it didn't work. One can combine with_deps_on_target_arch_pkgs and staged builds. So while rebootstrap does the same gcc/glibc dance as the toolchain-base packages, it also uses with_deps_on_target_arch_pkgs to avoid repacking (because the repacking code is out of archive). There is a set of source packages (uploaded as one per target arch, for manageability reasons, but actually all coming from the same git tree) that builds cross-compilers from gcc-source. These builds currently use the with_deps_on_target_arch_pkgs because a) it works and b) it's cleaner and simpler. Undeniably, the resulting version lock has been brought forward as an argument earlier. The cross toolchains resulting from with_deps_on_target_arch_pkgs are more fragile than those resulting from the default method in the sense that they are subject to the Multi-Arch version lock problem. See #766966 as an example. In the worst case, this can prevent the main gcc-X.Y packages from migrating to testing, so clearly gcc maintenance is affected. This argument only applies to uploading those packages. Thus I believe it to be irrelevant in the context of this bug. Mattias has always said he doesn't want to be responsible for maintaining the cross-toolchains, which is fine, I am prepared to do that, but that also means he shouldn't get a veto on _how_ they are maintained. Obviously if he was maintaining them himself he could upload what he likes. Arguably, the version lock should get him a veto, although he doesn't seem to have exercised this reasoning in the cross-gcc rc bugs yet (or I missed it). I am not opposed to disabled with_deps_on_target_arch_pkgs in general, I am. It's simple and reliable and (at least IMHO) more correct. There are tradeoffs between the two methods which I'm happy to continue to discuss and work on, but I want it kept around and working (and will help do that) until either consensus is reached amongst the gcc/cross-gcc/rebootstrap maintainers, or we decide that that's simply not possible. I think that it is obvious now that simple, reliable or correct are not universally agreed upon. For this reason, I have avoided them and resorted to arguments that assess whether a method is working (now) and whether its source is available in the archive. If with_deps_on_target_arch_pkgs is going to be used for a long time, the code that makes it work likely needs to stay somewhere other than src:gcc-X.Y (because it is Matthias' right to not maintain it). Once jessie is released, moving this functionality elsewhere is feasible, so I stress that I would not like to see a ruling that forces with_deps_on_target_arch_pkgs onto src:gcc-X.Y beyond jessie. I would not have forwarded this issue to the tc if Matthias had not combined the bad timing with an absence of advance notice. Helmut -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141101093855.ga17...@alf.mars
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Helmut Grohne [2014-10-28 07:13 +0100]: On Mon, Oct 27, 2014 at 09:41:59PM +, Ian Jackson wrote: The most obvious bug is the one mentioned in the patch: #760770 It is about a bug in the implementation of with_deps_on_target_arch (the contended feature). I think I may not understand what's going on here. In your mail to the TC, you say: it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. You mean setting these as environment variables ? If so then it would seem that this feature has no direct effect on anyone who is not trying to use it. Is that correct ? It is correct, that builds that do not set these variables are not affected by it beyond also carrying it as dead code in the gcc packaging. https://wiki.debian.org/MultiarchCrossToolchainBuild which talks abouit the with_deps_on_target_arch_pkgs feature. It doesn't appear that #760770 has taken a great deal of Matthias's time, although it did necessitate some bug triage. One of the issues here is that the submitter wasn't explicit about using the non-default build here. Whilst it is 'default' in the sense that it's what you get if you don't set any variables, I contend that (in Debian, but not Ubuntu) it is not the default build. In fact the 'default' build has not worked (in debian) for at least a year, probably two. I have tried and failed to make it work this year (ran out of time - clearly it is possible). However the 'non-default' 'with_deps_on_target_arch_pkgs' build has been working for at least a year, and is in fact the build that everyone using cross-toolchains in Debian testing or unstable has been using for some time. It is also the one that is better documented. And now it is the one that is used in the packages recently uploaded to the archive. To me that sounds like this method is actually the current de-facto default in Debian - it is certainly at least on a par. The with_deps_on_target_arch_pkgs was developed as a 2012 GSOC project. It was done because cross-compiler builds _do_ depend on foreign-arch libraries, and setting the build up to just use the ones already natively built in the archive (where they exist) is simple and (IMHO) correct. This simplicity is why it has been very easy to use and keep working. Those libraries come from the linux, libc and gcc packages. The alternative, which is used in the 'default' build, involves either taking those existing native-built library binaries and repackaging them (using dpkg-cross) in 'classic' (pre-multiarch) cross-compiler locations, or rebuilding them (as a cross-build) as part of the build and putting them in those locations. The net result is a second copy of the libraries, shipped with the cross-compiler. And, especially in the case of the full 'toolchain-base' build, the process is complicated and fragile. The build builds linux to get linux-libc-dev-cross, libc to get libc6-dev, and then gcc. But in fact to do that it actually builds linux, binutils, gcc (stage1), libc (stage1), gcc (stage2), libc (stage2), gcc (stage3). This process does work, as is demonstrated by the use of these packages in Ubuntu for some time, but it is undeniably much more complicated than just building against already-built libraries. I am not sure whether recent changes to libc packaging have made this process simpler - I need to check current status there. Note that the simpler with_deps_on_target_arch_pkgs build is only useful where the foreign-arch packages are already built, which makes it seem like the 'toolchain-base' method must be used for bootstrapping. However Helmut's rebootstrap has demonstrated that the with_deps_on_target_arch_pkgs method is useful there too. I must admit that I have not fully grokked how this works as I had thought that this was the one case where it didn't work. Are the maintainers of the disputed features subscribed to the appropriate packages in the PTS ? I am subscribed to the binutils and gcc packages in the PTS, yes, and have been for a while, mostly to track uploads in order to keep the cross-binutils and cross-gcc packages in sync. these bugs ? It seems to me that it would be easy to come up with a workflow that allowed Matthias to usertag these kind of bugs and hand them over to the cross teams. Sounds reasonable to me. Asking Wookey whether he would like to share that work. Certainly. As I am currently using it for my cross-toolchain packages I am keen to see it kept working, at least until an alternative works and is demonstrably better/as good. What are the cross-gcc-4.9-armhf packages that are referred to ? It is a source package that uses the gcc-4.9-source binary package from the gcc-4.9 source package to build a cross compiler targeting armhf. In GNU terminology that is build=host=amd64, target=armhf. The packaging is thin compared
Re: Bug#766708: breaks multiarch cross building
On Sun, Oct 26, 2014 at 03:51:00PM +0100, Helmut Grohne wrote: diff -u gcc-4.9-4.9.1/debian/rules.defs gcc-4.9-4.9.1/debian/rules.defs --- gcc-4.9-4.9.1/debian/rules.defs +++ gcc-4.9-4.9.1/debian/rules.defs @@ -125,6 +125,9 @@ $(error Invalid architecure.) endif +# Force this, people get confused about the default. See #760770. +with_deps_on_target_arch_pkgs := + # including unversiond symlinks for binaries #with_unversioned = yes @@ -1449,9 +1447,7 @@ #ifeq ($(trunk_build),yes) # no_biarch_libs := yes #endif -ifdef DEB_CROSS_NO_BIARCH - no_biarch_libs := yes -endif +no_biarch_libs := ifeq ($(no_biarch_libs),yes) with_lib64gcc:= no Rationale: Prior to these changes it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. I may be missing something here. It's true that this change sets defaults for these make variables and overrides values set in the environment. However, since the override directive is not used, can't you still override this by using command-line options to make? (This message should not be read as encouragement to add the override directive; the make documentation explicitly says that it was not invented for escalation in the war between makefiles and command arguments.) -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141030141527.ga3...@riva.ucam.org
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
On Mon, Oct 27, 2014 at 09:41:59PM +, Ian Jackson wrote: The most obvious bug is the one mentioned in the patch: #760770 It is about a bug in the implementation of with_deps_on_target_arch (the contended feature). I think I may not understand what's going on here. In your mail to the TC, you say: it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. You mean setting these as environment variables ? If so then it would seem that this feature has no direct effect on anyone who is not trying to use it. Is that correct ? It is correct, that builds that do not set these variables are not affected by it beyond also carrying it as dead code in the gcc packaging. Of course it does have a maintenance burden on the package maintainer, which is what Don is asking about. I have to admit that the code is not exactly lightweight. I do understand the desire to get rid it and asked that a ctte ruling does not apply beyond jessie for that reason. #760770 shows an element of that but it is immediately obvious from the initial report that something odd is going on and it contains a link to #720363 which mentions Oh, my previous bug research has missed gcc-4.8 bugs. https://wiki.debian.org/MultiarchCrossToolchainBuild which talks abouit the with_deps_on_target_arch_pkgs feature. It doesn't appear that #760770 has taken a great deal of Matthias's time, although it did necessitate some bug triage. One of the issues here is that the submitter wasn't explicit about using the non-default build here. It only surfaced in message 19 and can be spotted from looking at the patch. When being asked to do a self-contained cross build (and the self-contained kinda implies not using with_deps_on_target_arch_pkgs), a log with the alternative build method is sent back. Are the maintainers of the disputed features subscribed to the appropriate packages in the PTS ? Does Matthias welcome help triaging I am not subscribed yet. The major reason is that I did not perceive the maintenance of the feature as a problem until Matthias stated it in this bug. It is certainly fixable. these bugs ? It seems to me that it would be easy to come up with a workflow that allowed Matthias to usertag these kind of bugs and hand them over to the cross teams. Sounds reasonable to me. Asking Wookey whether he would like to share that work. What are the cross-gcc-4.9-armhf packages that are referred to ? It is a source package that uses the gcc-4.9-source binary package from the gcc-4.9 source package to build a cross compiler targeting armhf. In GNU terminology that is build=host=amd64, target=armhf. The packaging is thin compared to the gcc-4.9 packaging and its goal is to enable people to just apt-get install cross toolchains rather than building them each time they need them. (I am not a maintainer of cross-gcc-4.9-*.) Judging from the replies, I would like to repeat the timing argument here: The mechanism being discussed was disabled in gcc-4.9 without any advance notice or discussion[1]. The code for supporting the default method in glibc has not yet arrived in the Debian glibc package or the BTS, but Matthias indicated that he would be working on that and he seems to make progress outside Debian. I am not opposed to using the default build method for bootstrapping new Debian architectures in principle, but in my experience it takes a long time to merge patches into the glibc packaging and the freeze is certainly not accelerating that process. I am not opposed to disabled with_deps_on_target_arch_pkgs in general, just now is the wrong time, because it is impossible to get the corresponding functionality to gcc's default cross build into glibc. Most of the changes necessary to make the alternative method work with glibc have been merged however: #743676 #754350 #756095 #742640 #745380 #752480 #755580 #756473 (but most of these changes are also necessary for the default method) Helmut [1] It is worth noting here that the upload of cross-gcc-4.9-* similarly lacked discussion. An advance notice to the gcc list or targeting experimental would have been better here. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141028061340.ga22...@alf.mars
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Hi Don, Thanks for taking this up. On Sun, Oct 26, 2014 at 02:59:13PM -0700, Don Armstrong wrote: Matthias: is the primary concern of including this patch one of maintenance burden, not primarily technical? If you want Matthias to answer your question, I think that you may want to Cc him, but since you know that very well, I am not doing so now. Could you provide some background for the CTTE on how much maintenance burden we're talking about, and what recent bugs have resulted from this particular patchset? Since it is my interest to not cause more work on Matthias, I will try to summarize his POV. The most obvious bug is the one mentioned in the patch: #760770 It is about a bug in the implementation of with_deps_on_target_arch (the contended feature). Other bugs that may be relevant here are the ones that Matthias filed against cross-gcc-4.9-* (packages that make use of with_deps_on_target_arch). These are: arm64: #766614+#766617 (appear to be duplicates) #766624 armel: #766616 #766625 armhf: #766619 #766626 mipsel: #766613 #766623 powerpc: #766618 #766621 ppc64el: #766615 #766622 The left bugs are titled ... is functional incomplete. The right bugs are titled the package fails to build from source on buildds and has unmet dependencies. Discussion appears to take place on the armhf bugs. Of the issues mentioned in those bugs the following appear to be not specific to the cross-gcc-* packaging, but specific to the with_deps_on_target_arch and DEB_CROSS_NO_BIARCH mechanisms: 1) The use of cross-architecture dependencies. Even though this is supported by dpkg and apt, it means that packages with cross architecture dependencies can only be installed after activating the relevant architecture with dpkg --add-architecture. The packages are thus uninstallable in a standard system. This limitation is the essence of with_deps_on_target_arch. (Right bugs) 2) The compilers are not functionally equivalent to the native compilers. This has two nuances. The cross-gcc-* packages do not build gobjc and golang even though the with_deps_on_target_arch mechanism supports this operation. But even disregarding this, the resulting cross compilers do something different from the native ones. Matthias has been asked[1] to elaborate on this difference and I think he is in a better position to make his case here. (Left bugs) A third category of bugs that comes to my mind is the ones I filed or was involved with: #742358 #742539 #743718 #743764 #744265 #744782 #745267 #747526 #751001 #751919 #758408 #743342 A significant portion of these was specific to either with_deps_on_target_arch or DEB_CROSS_NO_BIARCH and I was not the only one filing patches: #716795 #742606 All of these were processed by Matthias in a timely manner. Matthias was involved in all of the bugs mentioned here and most of the bugs are in direct connection to the with_deps_on_target_arch and DEB_CROSS_NO_BIARCH mechanisms. This is certainly not a negligible workload. Of course, this is not exhaustive, but it may save Matthias time digging up bug numbers at least. I sincerely hope that this message furthers the cause. Helmut [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766619#10 -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141027204133.ga16...@alf.mars
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
Helmut Grohne writes (Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building): Since it is my interest to not cause more work on Matthias, I will try to summarize his POV. Thanks. The most obvious bug is the one mentioned in the patch: #760770 It is about a bug in the implementation of with_deps_on_target_arch (the contended feature). I think I may not understand what's going on here. In your mail to the TC, you say: it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. You mean setting these as environment variables ? If so then it would seem that this feature has no direct effect on anyone who is not trying to use it. Is that correct ? Of course it does have a maintenance burden on the package maintainer, which is what Don is asking about. #760770 shows an element of that but it is immediately obvious from the initial report that something odd is going on and it contains a link to #720363 which mentions https://wiki.debian.org/MultiarchCrossToolchainBuild which talks abouit the with_deps_on_target_arch_pkgs feature. It doesn't appear that #760770 has taken a great deal of Matthias's time, although it did necessitate some bug triage. Are the maintainers of the disputed features subscribed to the appropriate packages in the PTS ? Does Matthias welcome help triaging these bugs ? It seems to me that it would be easy to come up with a workflow that allowed Matthias to usertag these kind of bugs and hand them over to the cross teams. Other bugs that may be relevant here are the ones that Matthias filed against cross-gcc-4.9-* (packages that make use of with_deps_on_target_arch). These are: ... armhf: #766619 #766626 What are the cross-gcc-4.9-armhf packages that are referred to ? Ian. -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/21582.48167.807511.154...@chiark.greenend.org.uk
Re: Bug#766708: breaks multiarch cross building
Control: reassign -1 tech-ctte Dear technical committee, I ask you to overrule the gcc maintainer and rule that the following hunks to the gcc-4.9 package must be reverted: diff -u gcc-4.9-4.9.1/debian/rules.defs gcc-4.9-4.9.1/debian/rules.defs --- gcc-4.9-4.9.1/debian/rules.defs +++ gcc-4.9-4.9.1/debian/rules.defs @@ -125,6 +125,9 @@ $(error Invalid architecure.) endif +# Force this, people get confused about the default. See #760770. +with_deps_on_target_arch_pkgs := + # including unversiond symlinks for binaries #with_unversioned = yes @@ -1449,9 +1447,7 @@ #ifeq ($(trunk_build),yes) # no_biarch_libs := yes #endif -ifdef DEB_CROSS_NO_BIARCH - no_biarch_libs := yes -endif +no_biarch_libs := ifeq ($(no_biarch_libs),yes) with_lib64gcc:= no Rationale: Prior to these changes it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. This functionality is extensively tested[1] and in particular provides the only known working way to bootstrap new architectures. I am maintaining[2] this functionality by submitting patches to the gcc package. I acknowledge that the gcc maintainer claims that this method is broken. His reasons seem to resonate mostly with in-archive cross toolchains and not with throw-away cross toolchains used for bootstrapping. I acknowledge that supporting this alternative method of building cross compilers puts additional work on the gcc maintainer, but this work is mostly limited to merging patches. Why is with_deps_on_target_arch_pkgs needed? Without this flag, gcc emits dependencies on libc*-$arch-cross. In order to satisfy these dependencies binary packages from glibc have to be repacked and renamed. When using the resulting cross toolchain to build further Debian packages there are two choices, both of which are bad: * Newly cross built packages also depend on libc6-$arch-cross. This will make them different from natively built packages and in particular makes debootstrap fail. * Newly cross built packages keep their dependency on libc6. This will make them uninstallable in the build chroot, because there is no foreign arch libc package that can be co-installed with the aforementioned libc*-$arch-cross. Either option makes the bootstrap of a new architecture fail. Why is DEB_CROSS_NO_BIARCH needed? There currently is a bug in rebootstrap that makes building multilib enabled cross-toolchains fail. Thus far I failed to figure out the reasons for this failure (beyond knowing that gcc looks for crti.o in the wrong directory). So in the spirit of having something working rather than nothing, this fiddle is needed to bootstrap multilib-enabled architectures now. I intend to eliminate the need DEB_CROSS_NO_BIARCH. Timing I believe that it is very bad timing to disable a well tested functionality given the imminent freeze. This is also a reason for me to believe that reverting the aforementioned hunks poses little additional work on the gcc maintainer: There won't be much changes to put into jessie. The subject of this bug shall be limited to jessie, because there is much time to resolve the dispute for jessie+1 in a reasonable manner and there is reason to believe that the gcc maintainer will supports finding a resolution. Helmut [1] https://wiki.debian.org/HelmutGrohne/rebootstrap [2] #716795 #742358 #742539 #743718 #743764 #744265 #744782 #745267 #747526 #751001 #751919 #758408 #743342 -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141026145056.ga17...@alf.mars
Processed: Re: Bug#766708: breaks multiarch cross building
Processing control commands: reassign -1 tech-ctte Bug #766708 [gcc-4.9] breaks multiarch cross building Bug reassigned from package 'gcc-4.9' to 'tech-ctte'. No longer marked as found in versions gcc-4.9/4.9.1-19. Ignoring request to alter fixed versions of bug #766708 to the same values previously set -- 766708: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766708 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/handler.s.b766708.141433508122287.transcr...@bugs.debian.org
