+++ Matthias Klose [2014-11-26 15:29 +0100]: I'm not sure how best to respond to all this. Obviously I disagree with some of it, and it's remarkable how much our views of the history of debian cross-compilers differ :-)
I don't think anyone wants a lot of tit-for-tat debate as that's not very constructive. I'll respond to some points inline that are particularly egregious, and include my version of the relevant history. I think the more useful and substantive technical discussion about what cross-compilers should look like in general should go in the other bug (771070), so I'll try not to get into that here. > Contrary to some claims in this thread there never was any commitment > by myself to support the ability to rely on dependencies on foreign > architectures within the Debian archive. No, it's been clear that you are not enthusiastic, but it is in the packages, and it is used, and it is working today, and there has been a long expectation by plenty of other people that it was the way to go, for example, Marcin (author of the linaro/ubuntu cross-toolchain-base packages) wrote (on https://wiki.debian.org/ToolChain/Cross in 2011/2012): ------- There is ongoing work on having multiarch dpkg working for both Debian and Ubuntu distributions. When it will get to final state both ways of building cross compiler will have to be changed because there will be no need to manually fetch target arch packages because we could just build-depend on them. But what we will have to do when we will have final multiarch support? I think that there will be will be able to abandon armel-cross-toolchain-base package in favour of binutils-cross one as there will be no need to cross build eglibc or linux headers (we will just build-depend on target packages). -------- Here's my version of the history of this: ---------- Debian cross-toolchains have existed in various forms since around 2000. First (circa 2000) was the 'toolchain-source' package which was a copy of the gcc sources with the rules modified to build cross-compilers. This suffered from divergence from the normal gcc packages, with different versions, patches and bugs. The cross-support rules in this was merged into the main gcc package, and gcc output a gcc-source package so that cross-toolchains could be built using that. I got properly involved somewhen round here in trying to push the idea to (particularly) embedded devs, but in fact anyone crossbulding, that they should use a distro cross-toolchain, not build their own damn one for every project. That was a radical idea back then. It seems a lot more normal now. For many years (since 2004) the emdebian project used the gcc cross- functionality to build cross-toolchain binaries for Debian. These builds were done by using the libc/linux-libc packages from the target arch, converted to libc-<target>-cross/linux-libc-dev-<target>-cross with dpkg-cross, then building the package against those. The 'buildcross' tool was developed to mechanise this process, and build for multiple host architectures. The problem with this method was that it could not easily be made into a standard package that would build in the archive, because there was no way to express the dependency on a foreign-arch libc/linux-libc-dev, prior to multiarch, and also because downloading as part of a package build is not permitted. Thus the packages lived outside the archive (at emdebian) for a decade or so, and became well used. Note that these toolchains were hosted at emdebian but they were _for_ Debian, not emdebian particularly. Whilst multiarch was being developed around 2009/10 it became clear that it could solve this problem of specifying foreign dependencies for cross-toolchains, and explicit-arch dependencies were included in the spec partly for that reason. Meanwhile linaro wanted cross-toolchains in Ubuntu before all this was ready so packages were created (by Marcin) which ran the whole toolchain bootstrap procedure, build-depping on linux, binutils, libc, and gcc sources, and building linux-libc-dev-<target>-cross, binutils-<triplet>, libc-<target>-cross, gcc-<triplet>, via gcc stage1, libc stage1, gcc stage2, libc stage2, gcc stage3. This was the only way to build a cross-toolchain inside a standard package at the time. Those toolchains went into Ubuntu 10.10 and at the emdebian sprint at ARM in early 2011 it was planned that despite their over-complicated build they were a lot better than nothing and would be fixed up (by Marcin) to build on Debian and uploaded there too, until multiarch-built cross-toolchains were available/practical. A GSOC 2012 project to make the necessary changes to gcc for multiarch builds, was suggested (by me), and was done (by Thibaut Girka) (mentored by hector and Marcin), and merged (thank you Doko) in late 2012. So now it was possible to build a cross-compiler very simply by just depending on the foreign-arch libraries needed, and building it like any other package. Neither Marcin nor anyone else ever got round to uploading the full-bootstrap packages to Debian (I don't know why - busy? focussed on Ubuntu?) so Debian still had no in-archive cross-toolchains for wheezy, and the emdebian toolchains were not really maintained any more as we expected a move to the new multiarch ones quickly. That took much longer than expected in the way of things. This left wheezy users with a lack of handy cross-toolchains (it was possible to install the emdebain 'unstable' toolchains with a bit of kicking but it was way less than ideal). A release goal of cross-toolchains in jessie was proposed, but official goals were abandonned so it had no actual force. Multiarch-built cross-toolchains were working (and documented) from early 2013, but still could not be usefully uploaded until the infrastructure learned about foreign-arch dependencies. Sbuild, wanna-build and britney needed changes. Sbuild was fixed in time for Jessie - it now automatically enables a foreign architecture if a package build-deps on one so that the dependency can be installed during the build. Wanna-build was also patched (to ask dose the right question) and tested in time for Jessie, but by then the admins didn't want to change such important stuff, which was fair enough. Similarly Britney needed to be taught to consider foreign arches when migrating packages. So from my POV there has been a steady progress from the toolchain-source packages towards cross-compiler builds just being another simple package build, albeit with the (almost) unique feature of having foreign-arch build-deps. The Linaro/Ubuntu cross-toolchain-base packages were an expedient diversion that was working around the limitations of the time, not something to be used any longer than necessary. --------- As you can see, my and Doko's views of the same history are significantly divergent. These are nominally just facts after all. I'm not sure I can say much useful about that. I include this because I think it illustrates that we are both operating in good faith, but disagree about what the best solution looks like. > [GSOC] > The mentor was no > help, neither understanding the existing binutils and GCC packaging, > nor willing to understand it, and still claiming that he is able to > "simplify" it. I'm not sure who you are referring to here, but just to clarify: the mentors for that project were Hector Oron and Marcin Juszkiewicz, not me. > Earlier this year, and last time at the bootstrap sprint in Paris, > Wookey committed to work on the cross-toolchain-base package (this > package isolates the bootstrap process and builds binary cross glibc > packages). I hope other participants of the bootstrap sprint can > confirm that this commitment was made. The report from that meeting https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results says: ---------- 3.13. Multiarch cross-toolchains vs single-arch cross-toolchains This contentious issue was discussed, and is partly covered under other headings. Wookey prefers the multiarch builds, Doko prefers the single-arch bootstrap builds. We agreed that either provides useful cross-toolchains. It's not clear whether it's easier to fix the Ubuntu cross-toolchain-base packages to do a bootstrap build in Debian, or to fix the blockers for multiarch builds in the archive. Whichever is working first should get uploaded. Some work on both was done during the sprint, with current multiarch builds uploaded to the [CROSS-TOOLS] repo for testing, and various fixups of the cross-toolchain-base-armhf package: * remove obsolete versioned build-deps (nearly all of them) * update versions for unstable * remove the binutils part of the build as that now comes from cross-binutils * start looking at why build fails. -------- > Then nothing happened. I That is not true! A great deal happened: * Helmut's gcc-cross-support package was updated, tested, improved and shown to work. * I spent week or two trying (and failing) to get cross-toolchains-base to build on debian, before giving up again. * Marcins lost git-repo for cross-toolchain-base was recovered. * Various cross-toolchain-base versions were merged and patches updated. * Cross-binutils packages were prepared and uploaded. * The recently-uploaded cross-gcc-* packages were prepared and uploaded. * Sbuild was fixed to add foreign-arch dependency support. * A whole sbuild release was co-ordinated with bdrung as the maintainer was poorly (RSI). * Wanna-build was fixed to add foreign-arch dependency support. * Cross-gcc-defaults packages were created. * All these cross packages were uploaded to the Alioth cross-toolchain team site. * Helmut's multiarch pkg-config changes were tested. Discussion with the maintainer took place. * A Cross-build test setup was created for the base 100 packages. * The xbuilder package was updated and uploaded to the archive. That's a great big pile of 'nothing'. I worked bloody hard. Ask my wife. > can't remember that he said anything about a change of mind during > DebConf. The big surprise then came when Wookey was uploading > initially six cross toolchain packages to unstable without saying > anything, without having worked on the cross-toolchain-base packages > at any time. As stated above that is not true. I worked on them at Connect and afterwards. I had a range of build failures due to differences between ubuntu and debian kernel packaging and ubuntu and debian libc patches. I asked Adam for help and he agreed that things needed fixing so I left it with him after Connect. He never actually got back to me with a 'it should build now'. I worked on it again at the UK minidebconf with xnox and Ben Hutchings. We fixed up the kernel-headers breakage, although only by apt-getting the kernel source, which I'm not sure is really a solution. Dmitry said he'd go away and finish getting that going for debian. I'm not sure if he's finished that yet (I've prodded a couple of times on IRC, and I think again back in this thread somewhere). I just tried building the powerpc-cross-toolchain-base-1.2d that you pointed me at a little while ago just now and it fails with CC kernel/bounds.s gcc: error: unrecognized argument in option '-mabi=aapcs-linux' gcc: note: valid arguments to '-mabi=' are: ms sysv gcc: error: unrecognized command line option '-mlittle-endian' gcc: error: unrecognized command line option '-mno-thumb-interwork' gcc: error: unrecognized command line option '-mfpu=vfp' Kbuild:35: recipe for target 'kernel/bounds.s' failed This stuff did work on Debian sometime in 2012. I'm not sure it has ever worked since. Clearly it could be made to work, but I have found it more productive to work on stuff that was already working _and_ to me at least seems like a cleaner build method. Now we should discuss in 771070 the details of this, and hopefully reach some sort of agreement about the way forward. I just put this stuff here to counter your accusations of 'no work', 'not operating in good faith', and 'just use the supported method it works fine'. > You may call this politics, or tactics, however I call > this sneaking in these packages, and his communication behaviour as > plain XXing (somebody mentioned I should just call this "not telling > the truth"). And I call it 'getting some actual cross compilers into Debian for the first time ever'. Something which no-one else has done anytime in the last 4 years since it was practical/possible. You, or anyone else, could have uploaded packages using your favoured scheme at any time. But no-one did. > At this point I decided to remove the unsupported cross > build support, and then was accused on irc by Wookey "But ultimately I > don't think a cross-gcc maintiner can operate if the gcc maintainer is > actively working against him", and Helmut complaining "I would not > have forwarded this issue to the tc if Matthias had not combined the > bad timing with an absence of advance notice". But sorry I must have > missed the absence of advance notice for the cross build uploads and > the notice of dropping the work on the cross-toolchain-base packages. > This is a reaction, not an action. You were at the bootstrap sprint with both of us. The quoted section above seems pretty clear to me. I'm sorry if the upload came as a surprise to you, but it really shouldn't have. And obviously I am very disapointed by the way that you have reacted to some actual progress on this topic by removing functionality, so the cross- packages have to patch gcc back to how it worked for the previous 2 years. > It is odd the somebody first tells people that he is working on an > agreed solution, then silently stops working on that solution and then > claims that others can't work with him. The conversation continued > together with Adam Conrad and Helmut Grohne how to provide the > cross-toolchain-base package in Debian, with Wookey ending "OK. I'll > take a look. I'm away this weekend though [...], so nothing much will > happen till sunday evg at earliest". This was three weeks ago, and > nothing happened, except another new cross toolchain or two uploaded > to unstable. So again, usual behaviour of this guy, same procedure as > every year. Right, and I have said repeatedly, including in this thread, that I am entirely prepared to improve these packages, and yes I haven't got round to doing what you asked yet, but of course that hasn't been helped by all the extra work you created by removing bits of cross-gcc functionality so I have to patch the toolchains to keep them working: see 766924, 771383 And of course once one uploads something one finds bugs, and gets users. So I spent some time making things work better, and adding i386 builds (ecause someone asked for them), and even a couple of bug fixes: 770413 I've also added the gccgo and objc frontends you asked for. And there was a miniconf in there (where as stated above we made some small progress on this), and the old emdebian server got hacked and I had to set up a new one. And I do have other work than just cross-toolchains to do. So please, less of the accusations of bad faith. Yes we have a disagreement about how this should best be done, but we both want working cross-toolchains in Debian, and I'm sorry that the first iteration of that uses my preferred build scheme, but that was what I managed to get working. I don't think having something that I am prepared to maintain, and you are prepared not to block, is actually an insoluble problem by the time Stretch is done. I agree that I need to communicate with you and others better, there has been too much IRC and not enough email. I just sent mail to the debian-cross list tonight. But you need to stop being obstructive too, otherwise this simply isn't going to work. > In the meantime there is some progress from Adam Conrad, Dimitri > Ledkov and my side on the cross-toolchain-base package. Both Wookey > and Helmut Grohne were CCed on these efforts but they choose to stay > quiet. Does cross-toolchain-base now build on Debian? Is that still the 7-stage full-bootstrap package, or just one to produce the necessary -cross libraries? I don't think the former is sensible, or maintainable, even if it does now build. > At this point I don't think it will make sense to work together > with Wookey if he keeps behaving like this. Yep. Everything is my fault. I suck big-time. I'm surprised people can bear to be in the same room. :-) > I don't mind getting help maintaining GCC in Debian, however this > shouldn't be limited leaving an odour signature in the packaging, but > involves interaction with upstream, test rebuilds, bug submissions, > bug triage and many more. The involvement of Helmut Grohne and Wookey > with these tasks is very close to zero. I submitted a bug earlier: 771383, which you instantly closed with a snarky message. That's not the way to encourage people. And now I see you've just upped the ante further with 4.9.2-4 with "Remove unsupported with_deps_on_target_arch_pkgs configurations.", closing the associated bugs: #760770, #766924, #770413. In contrast to your rejection of my contribution, I'm happy to work with you on this, even though it can be hard work sometimes. Obviously you do have a fairly effective veto as gcc maintainer, so you can get me to give up and go away if you try hard enough. Is anyone else volunteering to maintain the cross-toolchains? Or are you going to do it yourself? Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141129043026.gb27...@stoneboat.aleph1.co.uk
