Bug#766708: breaks multiarch cross building
+++ Matthias Klose [2014-10-26 15:15 +0100]: Control: severity -1 wishlist Am 26.10.2014 um 14:15 schrieb Helmut Grohne: Control: severity -1 serious It breaks working functionality without any benefit. having to maintain two ways to build a cross compiler costs a lot of my time, so yes, removing the need to support two configurations, one of them not even supporting multilibs has a big benefit for me. Unfortunately you have removed the one most people in Debian are using. Yes, there is additional complexity in having two build options, but most of that was in getting it working in the first place. It is currently working well, has been for a year or so, and is tested and maintained. Keeping it in this state is not a large overhead, and is used by other teams. If you must drop one of the build methods, there would be a lot less resistance to dropping the other one, although I don't actually think that's a good idea either. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141026214235.gc28...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Helmut Grohne [2014-10-28 07:13 +0100]: On Mon, Oct 27, 2014 at 09:41:59PM +, Ian Jackson wrote: The most obvious bug is the one mentioned in the patch: #760770 It is about a bug in the implementation of with_deps_on_target_arch (the contended feature). I think I may not understand what's going on here. In your mail to the TC, you say: it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. You mean setting these as environment variables ? If so then it would seem that this feature has no direct effect on anyone who is not trying to use it. Is that correct ? It is correct, that builds that do not set these variables are not affected by it beyond also carrying it as dead code in the gcc packaging. https://wiki.debian.org/MultiarchCrossToolchainBuild which talks abouit the with_deps_on_target_arch_pkgs feature. It doesn't appear that #760770 has taken a great deal of Matthias's time, although it did necessitate some bug triage. One of the issues here is that the submitter wasn't explicit about using the non-default build here. Whilst it is 'default' in the sense that it's what you get if you don't set any variables, I contend that (in Debian, but not Ubuntu) it is not the default build. In fact the 'default' build has not worked (in debian) for at least a year, probably two. I have tried and failed to make it work this year (ran out of time - clearly it is possible). However the 'non-default' 'with_deps_on_target_arch_pkgs' build has been working for at least a year, and is in fact the build that everyone using cross-toolchains in Debian testing or unstable has been using for some time. It is also the one that is better documented. And now it is the one that is used in the packages recently uploaded to the archive. To me that sounds like this method is actually the current de-facto default in Debian - it is certainly at least on a par. The with_deps_on_target_arch_pkgs was developed as a 2012 GSOC project. It was done because cross-compiler builds _do_ depend on foreign-arch libraries, and setting the build up to just use the ones already natively built in the archive (where they exist) is simple and (IMHO) correct. This simplicity is why it has been very easy to use and keep working. Those libraries come from the linux, libc and gcc packages. The alternative, which is used in the 'default' build, involves either taking those existing native-built library binaries and repackaging them (using dpkg-cross) in 'classic' (pre-multiarch) cross-compiler locations, or rebuilding them (as a cross-build) as part of the build and putting them in those locations. The net result is a second copy of the libraries, shipped with the cross-compiler. And, especially in the case of the full 'toolchain-base' build, the process is complicated and fragile. The build builds linux to get linux-libc-dev-cross, libc to get libc6-dev, and then gcc. But in fact to do that it actually builds linux, binutils, gcc (stage1), libc (stage1), gcc (stage2), libc (stage2), gcc (stage3). This process does work, as is demonstrated by the use of these packages in Ubuntu for some time, but it is undeniably much more complicated than just building against already-built libraries. I am not sure whether recent changes to libc packaging have made this process simpler - I need to check current status there. Note that the simpler with_deps_on_target_arch_pkgs build is only useful where the foreign-arch packages are already built, which makes it seem like the 'toolchain-base' method must be used for bootstrapping. However Helmut's rebootstrap has demonstrated that the with_deps_on_target_arch_pkgs method is useful there too. I must admit that I have not fully grokked how this works as I had thought that this was the one case where it didn't work. Are the maintainers of the disputed features subscribed to the appropriate packages in the PTS ? I am subscribed to the binutils and gcc packages in the PTS, yes, and have been for a while, mostly to track uploads in order to keep the cross-binutils and cross-gcc packages in sync. these bugs ? It seems to me that it would be easy to come up with a workflow that allowed Matthias to usertag these kind of bugs and hand them over to the cross teams. Sounds reasonable to me. Asking Wookey whether he would like to share that work. Certainly. As I am currently using it for my cross-toolchain packages I am keen to see it kept working, at least until an alternative works and is demonstrably better/as good. What are the cross-gcc-4.9-armhf packages that are referred to ? It is a source package that uses the gcc-4.9-source binary package from the gcc-4.9 source package to build a cross compiler targeting armhf. In GNU terminology that is build=host=amd64, target=armhf. The packaging is thin compared
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Helmut Grohne [2014-11-01 10:38 +0100]: On Sat, Nov 01, 2014 at 01:46:48AM +, Wookey wrote: To me that sounds like this method is actually the current de-facto default in Debian - it is certainly at least on a par. I don't think that a feature being de-facto default is a good argument to force maintaining it forever. Well, things being used is a good reason for maintaining them. Nothing is 'forever'. with_deps_on_target_arch_pkgs is currently just as well-used as the alternative. I was just trying to get across that's it's not just some obscure feature no-one cares about. There clearly is need to simplify gcc packaging and one way to do that is to remove one of the cross toolchain build methods. Is there 'clearly a need'? The gcc packaging is certainly complicated, and we'd all welcome simplification, but I'm not sure there is actualyl much scope for that int he real world. A great deal of the complexity comes from bi-arch and tri-arch multilib stuff, for example. I'd argue that more use of multiarch would mean we could drop much of that, at which point the with_deps_on_target_arch_pkgs stuff is a net (large) simplification. But so far as I know this is not something the gcc maintainer is at all interested in. There is a set of source packages (uploaded as one per target arch, for manageability reasons, but actually all coming from the same git tree) that builds cross-compilers from gcc-source. These builds currently use the with_deps_on_target_arch_pkgs because a) it works and b) it's cleaner and simpler. Undeniably, the resulting version lock has been brought forward as an argument earlier. The cross toolchains resulting from with_deps_on_target_arch_pkgs are more fragile than those resulting from the default method in the sense that they are subject to the Multi-Arch version lock problem. See #766966 as an example. Yep. That is clearly the most significant disadvantage of this packaging. It is an issue in unstable, with frequent gcc uploads but not in stable and it shouldn't be in testing either. Note that in practice as soon as you do any multiarch cross-building you are subject to the same constraint (via libgcc1) so the practial difference is relatively small. So someone doing kernel cross-building in unstable would really notice a difference. Anyone multiarch-building packages or using testing or stable would not. (I believe - subject to confirmation by testing). In the worst case, this can prevent the main gcc-X.Y packages from migrating to testing, so clearly gcc maintenance is affected. How? The cross-toolchains depends on the native toolchain libraries. I don't see what circumstance would make the cross-toolchains prevent the main gcc-X.Y packages from migrating I am. It's simple and reliable and (at least IMHO) more correct. There are tradeoffs between the two methods which I'm happy to continue to discuss and work on, but I want it kept around and working (and will help do that) until either consensus is reached amongst the gcc/cross-gcc/rebootstrap maintainers, or we decide that that's simply not possible. I think that it is obvious now that simple, reliable or correct are not universally agreed upon. For this reason, I have avoided them and resorted to arguments that assess whether a method is working (now) and whether its source is available in the archive. I have also been clear about stating objective or subjective things. If you look at the packaging of cross-gcc and cross-toolchain-base it is obvious that the former is _much_ simpler (5K rules, 12 targets vs 25K rules, 29 targets). Now to be clear, the difference in the gcc build alone between with_deps_on_target_arch_pkgs (multiarch-build-deps) and -cross standalone build-deps is nothing. The difference is in the building of linux-libc-dev (-cross), libc6-dev(-cross), libgcc1(-cross), libstdc++(-cross) etc. and the glibc/gcc bootstrap dance. That part is negligible (already done) if the multiarch-build-deps build is used and complicated if it isn't. Similarly, after a lot of messing with this it is clear to me that the simple build is 'reliable' because there is (much) less to go wrong. I've just spent another couple of hours with the standalone powerpc-cross-toolchain-base_1.2d Mattias sugested that I use as a base and have not got it to build on unstable yet. So maybe 'reliable' is just another way of stating 'simple', but it is an objective measure. And I did qualify 'more correct' as 'IMHO', as that clearly is subjective, I agree. If with_deps_on_target_arch_pkgs is going to be used for a long time, the code that makes it work likely needs to stay somewhere other than src:gcc-X.Y (because it is Matthias' right to not maintain it). Once jessie is released, moving this functionality elsewhere is feasible, so I stress that I would not like to see a ruling that forces with_deps_on_target_arch_pkgs onto src:gcc-X.Y beyond jessie. As you say, he
Re: Bug#766708: breaks multiarch cross building
Prior to these changes it was possible to build a gcc cross compiler with different properties from the default build by setting with_deps_on_target_arch_pkgs=yes and DEB_CROSS_NO_BIARCH=yes. I may be missing something here. It's true that this change sets defaults for these make variables and overrides values set in the environment. However, since the override directive is not used, can't you still override this by using command-line options to make? (This message should not be read as encouragement to add the override directive; the make documentation explicitly says that it was not invented for escalation in the war between makefiles and command arguments.) The 4.9.2-1 gcc 4.9 upload adds the override directive. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141107034222.gp28...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Ben Longbons [2014-11-19 22:18 -0800]: If the cross tools miss jessie and go in jessie-backports, that will require a significant amount of knowledge and discipline on the part of all the package maintainers involved, to make sure that they always have matching versions despite being in different repos or they will not be useful for package developers. If they are treated as one package and go in one repo, everybody is happy. The cross-tools (apart from binutils) have missed Jessie. The release team are not going to change their minds about that. The issue of this bug was not the only problem there: missing work on britney and wanna-build means they wouldn't have migrated in time independently of this issue and I was not able to persuade the release team to make a special exception on 'release goal' grounds. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141120233309.gd27...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Don Armstrong [2014-11-19 16:41 -0800]: On Tue, 28 Oct 2014, Helmut Grohne wrote: I have to admit that the code is not exactly lightweight. I do understand the desire to get rid it and asked that a ctte ruling does not apply beyond jessie for that reason. Are people who are doing cross-building like this actually using the code which will be in jessie? I (perhaps naïvely) would expect them to be primarily using the code in unstable, and maybe at a late stage of bring-up rebuilding all of stable. I think you are right, at least for a while. The situation is as follows: Jessie: The only cross-toolchains currently available for jessie are built using with_deps_... and are available (outside the archive) to be installed. They will be updated if the gcc version in jessie is. I'm not sure to what degree anyone else will be doing much rebuilding of these packages. People might try (e.g. I just tried rebuilding them in Utopic as someone asked if that would work). Unstable: I am building cross-toolchains against each new gcc-4.9 upload to unstable, using the code in unstable, and expect to keep doing this. Those builds also use the with_deps.. method, and thus currently need the with_deps.. disabling patched out to work. The build method may change during the life of unstable, but how that will play out is not clear yet. So for me I expect to be using this functionality in unstable much more than in jessie which should be more or less stable now. Again I'm not sure how much other people will be rebuilding these packages or otherwise building cross-toolchains from gcc. There will certainly be some, especially if rebuilds in unstable are not kept up-to-date (currently a manual process of fresh cross-gcc-* source uploads, but we plan to automate this). Boostrapping tests will do it regularly. New porters will do it. Currently everyone will be using with_deps because that's the choice (after patching gcc). They may choose to build the standalone way once it's actually working. At least 3 of us are prepared to maintain the with-deps packaging rules. IMHO it makes a lot more sense to maintain it in gcc packagig where it already is rather than do it outside as a big quilt stack, but that won't work if the maintainer doesn't apply patches. I just filed 770413, for example. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141121043802.gf27...@stoneboat.aleph1.co.uk
Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building
+++ Dimitri John Ledkov [2014-11-23 12:27 +]: On 23 November 2014 at 11:23, Ron r...@debian.org wrote: On Sat, Nov 22, 2014 at 08:51:41PM +, Dimitri John Ledkov wrote: On 22 November 2014 at 16:21, Ron r...@debian.org wrote: Dimitri wrote: The newly introdued mualtiarch cross building in jessie to a few packages: * cannot be build on standard debian buildds Not yet, but all the code to do this exists. The necessary sbuild is in Jessie. The necessary wanna-build patches are here: http://anonscm.debian.org/cgit/users/dkogan-guest/wanna-build.git/ (awaiting review and the stable release before potentially problematic wanna-build changes are actually applied) * cannot build multilib toolchains Correct (although this could possibly be fixed). * and thus resulting toolchains cannot rebuild non-trivial core debian packages There are _lots_ of debian packages that currently don't cross-build, for lots of reasons. A tiny number of packages _need_ multilib to build (SFAIK). So whilst this is an issue to consider, it is a fairly minor one on the scale of things. The cross-toolchains remain a) the only ones available in the archive and b) useful for building _most_ debian packages (and other stuff). These reasons have been pointed out to the people raising this bug report before. If anyone missed that for any reason, it is pointed out now. As as has been said numerous times in this thread, no-one is suggesting that the current cross-toolchains are immutable and can't be improved/replaced, but until we can actually build the alternatives (Have you fixed cross-toolchain-base for debian yet?) there is no good reason to break what is currently working (and in fact having that working _still_ isn't a good reason for breaking the 'mulitarch-multiarch' builds). I'm not sure if you are deliberately missing portions I've emails that I sent. The people who have actually been working on this _in Debian_ are well aware of what is not perfect about it and what extra work remains for post-Jessie to make it more so, and they are actually working on those things. They even have packages based on this uploaded to sid, so that the other work on fixing those things can continue, e.g.: https://packages.debian.org/sid/gcc-4.9-arm-linux-gnueabi This package is not build on Debian Yes it is and cannot be ever rebuild on Debian. Nonsense. Of course it can. And RC bug reports are filed. And they will cease to be RC as soon as Jessie is released and wanna-build updated. The work has been done. This concrete example is very good way to show that its upload is pre-mature. The reason is quite simple, that we do not have foreign architectures enabled on the builders, nor any easy way to enable them (being or has been fixed in sbuild), Yes we do. Building these packages in the Sbuild in Jessie 'just works' already. Try it: sbuild -A -s -d unstable -c sid-amd64-sbuild cross-gcc-4.9-armhf (this will fail immediately after a new gcc upload until the cross-build-deps are built, because its build-deps are not available, just as any other package would)) however on-demand enabling foreign architectures will not be in place until only after one stable release where it is possible to do so and buildds getting upgraded to such release. It will be (is!) in place in Jessie. It's had limited testing so there may prove to be problems in practice, but our testing so far indicates that it works fine. All the packages uploaded to unstable (and for Jessie in my p.d.o repo) are built in standard jessie/unstable sbuild. What nobody has explained to us is what problem is *fixed* by gratuitously breaking this for existing users of Jessie. The problem is introducing incomplete broken things into archive. 'incomplete broken' is not fair. They are quite new and we are awaiting infrastructure changes to be applied by the buildd maintainers, but that's not going to happen until after the stable release now. But the packages themselves are now in quite good shape. Unstable now contains cross-compilers for all the arches in Jessie (for amd64), built using standard sbuild. Including cross-gcc-defaults to add the wanted symlinks for all arches except mips (because mips was lagging badly at the time of the original upload so I missed that one out - this has just been corrected in cross-gcc-defaults 0.4, currently in NEW). They work to build all (most?) of the packages in Debian which are cross-buildable at all and whose cross-build-deps are installable: (e.g. test on 100 packages here: http://people.linaro.org/~wookey/buildd/testing/sbuild/latest/status.html ) Yes there is plenty of stuff that doesn't cross-build but that's not because these toolchains are particularly 'incomplete', or 'broken'. You'd get the exact same failure set with a standalone cross-toochain too, SFAIK. Convenience 'crossbuild-essential' packages could be in unstable too, but the maintainer (Doko) has only
Bug#766708: supported GCC based cross compilers in Debian
got round to uploading the full-bootstrap packages to Debian (I don't know why - busy? focussed on Ubuntu?) so Debian still had no in-archive cross-toolchains for wheezy, and the emdebian toolchains were not really maintained any more as we expected a move to the new multiarch ones quickly. That took much longer than expected in the way of things. This left wheezy users with a lack of handy cross-toolchains (it was possible to install the emdebain 'unstable' toolchains with a bit of kicking but it was way less than ideal). A release goal of cross-toolchains in jessie was proposed, but official goals were abandonned so it had no actual force. Multiarch-built cross-toolchains were working (and documented) from early 2013, but still could not be usefully uploaded until the infrastructure learned about foreign-arch dependencies. Sbuild, wanna-build and britney needed changes. Sbuild was fixed in time for Jessie - it now automatically enables a foreign architecture if a package build-deps on one so that the dependency can be installed during the build. Wanna-build was also patched (to ask dose the right question) and tested in time for Jessie, but by then the admins didn't want to change such important stuff, which was fair enough. Similarly Britney needed to be taught to consider foreign arches when migrating packages. So from my POV there has been a steady progress from the toolchain-source packages towards cross-compiler builds just being another simple package build, albeit with the (almost) unique feature of having foreign-arch build-deps. The Linaro/Ubuntu cross-toolchain-base packages were an expedient diversion that was working around the limitations of the time, not something to be used any longer than necessary. - As you can see, my and Doko's views of the same history are significantly divergent. These are nominally just facts after all. I'm not sure I can say much useful about that. I include this because I think it illustrates that we are both operating in good faith, but disagree about what the best solution looks like. [GSOC] The mentor was no help, neither understanding the existing binutils and GCC packaging, nor willing to understand it, and still claiming that he is able to simplify it. I'm not sure who you are referring to here, but just to clarify: the mentors for that project were Hector Oron and Marcin Juszkiewicz, not me. Earlier this year, and last time at the bootstrap sprint in Paris, Wookey committed to work on the cross-toolchain-base package (this package isolates the bootstrap process and builds binary cross glibc packages). I hope other participants of the bootstrap sprint can confirm that this commitment was made. The report from that meeting https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results says: -- 3.13. Multiarch cross-toolchains vs single-arch cross-toolchains This contentious issue was discussed, and is partly covered under other headings. Wookey prefers the multiarch builds, Doko prefers the single-arch bootstrap builds. We agreed that either provides useful cross-toolchains. It's not clear whether it's easier to fix the Ubuntu cross-toolchain-base packages to do a bootstrap build in Debian, or to fix the blockers for multiarch builds in the archive. Whichever is working first should get uploaded. Some work on both was done during the sprint, with current multiarch builds uploaded to the [CROSS-TOOLS] repo for testing, and various fixups of the cross-toolchain-base-armhf package: * remove obsolete versioned build-deps (nearly all of them) * update versions for unstable * remove the binutils part of the build as that now comes from cross-binutils * start looking at why build fails. Then nothing happened. I That is not true! A great deal happened: * Helmut's gcc-cross-support package was updated, tested, improved and shown to work. * I spent week or two trying (and failing) to get cross-toolchains-base to build on debian, before giving up again. * Marcins lost git-repo for cross-toolchain-base was recovered. * Various cross-toolchain-base versions were merged and patches updated. * Cross-binutils packages were prepared and uploaded. * The recently-uploaded cross-gcc-* packages were prepared and uploaded. * Sbuild was fixed to add foreign-arch dependency support. * A whole sbuild release was co-ordinated with bdrung as the maintainer was poorly (RSI). * Wanna-build was fixed to add foreign-arch dependency support. * Cross-gcc-defaults packages were created. * All these cross packages were uploaded to the Alioth cross-toolchain team site. * Helmut's multiarch pkg-config changes were tested. Discussion with the maintainer took place. * A Cross-build test setup was created for the base 100 packages. * The xbuilder package was updated and uploaded to the archive. That's a great big pile of 'nothing'. I worked bloody hard. Ask my wife. can't remember that he said anything about a change of mind
Bug#766708: counterfeiting the summary of the bootstrap sprint
+++ Matthias Klose [2014-12-04 20:41 +0100]: So in the last email Wookey enumerates a lot of things what he did during the last months. Maybe he should have mentioned his ballerina lessons used for his performances during the DebConf talks too. However ever all of these have in common, that this has nothing to do at all with the work he committed to do. All that cross-toolchain packaging work had nothing to do with the release goal of cross-toolchains in the archive? That's what I think I committed to. Only some of it was on cross-toolchain-base (but definitely not none), as opposed to multiarch builds, for reasons already discussed. Further he cites a paragraph from the debian-bootstrap sprint summary, which reads: The report from that meeting https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results says: -- 3.13. Multiarch cross-toolchains vs single-arch cross-toolchains This contentious issue was discussed, and is partly covered under other headings. Wookey prefers the multiarch builds, Doko prefers the single-arch bootstrap builds. We agreed that either provides useful cross-toolchains. It's not clear whether it's easier to fix the Ubuntu cross-toolchain-base packages to do a bootstrap build in Debian, or to fix the blockers for multiarch builds in the archive. Whichever is working first should get uploaded. According to https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results?action=diffrev1=30rev2=31 the last sentence was added on Aug 29 during DebConf, long after the sprint happened, with a commment must be almost the final review, without mentioning anything. I call this counterfeiting the summary of the sprint. I assume this helped to convince other people to sneak in these packages into Debian. What is this if not bad faith? It is one of a very long series of edits over the 10 days after the sprint (https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results?action=info) where all the sprint attendees tried to get our notes and recollections into a readable, coherent, document. You did not appear to take part - I don't know why, but assumed you were busy. I don't suppose you will believe me, but it was an honest attempt to document the event. No-one objected to any of it apart from you, now. There has been a lot of discussion about assuming good faith recently. A bit less of the 'he is a liar and a counterfeighter' would be nice. Again, the rest of the email talking about willing to work together doesn't match the his actions at all. So, I think everyone has got the general idea by now that we don't agree on this matter, and haven't for some time. I suggest that further dissection of history isn't going to help or entertain anyone, and we should retire to bug 771070 and discuss the substantive issues and what to do next. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141204202237.gl27...@stoneboat.aleph1.co.uk
Bug#771070: requirements for cross toolchain packages in the distribution
- The gist of this bug is for the gcc maintainer to require that only complicated 'does everything' cross-toolchains should be permitted in the archive. Why is it bad to have simpler, easy-to-maintain, arch-consistent cross-toolchains in the archive, especially if the latter is what someone is volunteering to maintain, and no-one is volunteering to maintain the former (yet?). The simple ones can evolve and, in the way of things, are likely to become more capable and complicated over time. But what is the point of vetoing them, especially when we _know_ that such simpler cross-toochains (all that has been available for the last decade or so), satisfy most users already. There is more to discuss, but I've gone on long enough already :-) Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ -- To UNSUBSCRIBE, email to debian-ctte-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141218163600.gs27...@stoneboat.aleph1.co.uk
Bug#771070: requirements for cross toolchain packages in the distribution
+++ Ben Longbons [2014-12-18 12:23 -0800]:
On Thu, Dec 18, 2014 at 8:36 AM, Wookey woo...@wookware.org wrote:
MA-built vs in-arch
---
I guess an interesting question is 'what does the cross-compiler
actually _use_ the foreign arch libc for'? Does it need its own
independent copy? What happens when the compiler libc-$arch-cross and
the system libc:$arch get out of sync. Does it matter?
The thought of this gives me nightmares. Glibc is very good at
backwards-compatibility as far as packages go, but does not even
attempt forwards-compatibility.
Indeed. This seems like a bad thing, but in fairness, emdebian and
ubuntu have been shipping cross-compilers where the versions do not
necessarily match for quite some time, and it hasn't caused obvious
practical problems.
So maybe this doesn't matter, but I'd really like to understand
exactly what's going on so we could see what might break, and thus
allocate an importance.
In practice I presume that all these libraries are dynamically linked
so in fact what matters is what versions are available at runtime, and
how tight the dependencies are.
If anything (I'm still confused if this means only libstdc++, or every
user program - but for those of us writing C++ I suppose it makes no
difference) gets built against libc-$arch-cross (= 2.999) but at
runtime there is only libc:$arch (= 2.998), then programs will almost
certainly fail to load because of missing symbol versions, and
possibly even fail to link.
In general, even if the compiler is built against libc-$arch-cross,
(and thus it is installed along with the toolchain) programs built
with that compiler should link against libc:$arch. I have seen cases
where they don't, and end up linking against the libc-$arch-cross copy
instead. I don't have a good handle on how common this is.
But clearly, if there isn't a libc-$arch-cross 'internal' copy present
then this problem can't arise. On the other hand, having the two
'flavours' of libc lets you install the toolchain 'within-arch', and
avoids uninstallability-due-to-multiarch-skew in unstable.
multilib vs mulitarch
-
Native compilers are not yet co-installable so have to use multilib to
target more than one ABI.
Can we please fix this? I'm tired of having to special-case my
buildbot scripts for arches only available through multilib (i586 and
gnux32 on an amd64 host; this also prevents ever running my buildbot
on anything other than amd64). For all other arches (native or
multiarch cross) the script is trivial.
This is the relevant page:
https://wiki.debian.org/CoinstallableToolchains
which has links to the bug at the bottom:666743. Helmut has the best
handle on the changes needed for this, and so far as I know Doko did
not object to the moving around of symlinks which makes this possible
(it was discussed at the bootstrap sprint
https://wiki.debian.org/Sprints/2014/BootstrapSprint/Results section 3.11).
Up to date patches would be good, to get this moving.
It does not work to just make my own ~/bin/i586-linux-gnu-gcc etc
scripts that call x86_64-linux-gnu-gcc -m32, because I also have to
worry about the libs being in the wrong place ({/usr,}/lib32 instead
of {/usr,}/lib/i586-linux-gnu, except that it's really
{/usr,}/lib/i386-linux-gnu ... why can't the numbers be consistent?
It's even i486-linux-gnu in places too ... meh, unimportant to this
discussion, except I suppose I'll need to special-case the library
locations anyway, or else lie any call my binaries packages i386 even
though they're really i586 ... it's not like everyone else doesn't
anyway).
Yeah that's all a pain. Blame the x86 people for using 3 different
triplets for the same ABI.
Multilibs do make much better sense for arches that are not in the
archive (x32/mipsn32) and are possibly the easier way to support
those, but even there, all the difficulty is in getting the right
libc:$arch or libc-$arch-cross packages. Once you have those you can
build per-arch toolchains or multilib toolchains.
The most correct solution to me seems to be a libs only archive even
for unsupported arches. This would be a huge win even for supported
arches, because 'apt-get update' with N architectures enabled is
really slow already.
Right. This is the logical outcome of using multiarch for library
dependencies. If you don't do this you have treat supported and
unsupported arches differently.
Conceptually it would be split into 3 areas that I can think of:
1. anything that ships architecture-specific binaries (not usable in
cross situations in general, but there are still useful packages like
wine32)
2. anything that ships architecture-specific libraries but not
binaries (useful for cross)
3. anything that ships no architecture-specific binaries or libraries
2 is Multi-Arch: same and 3 is Architecture: all, so we already have
an easy way to identify these sets of packages. That said, I can't
think of any
Re: Bug#797533: New CTTE members
+++ Don Armstrong [2015-09-10 09:57 -0500]: > On Wed, 09 Sep 2015, Wookey wrote: > > Well, maybe. Maybe there were discussions to that effect I didn't see. > > In that case fair enough. The impression given was of a somewhat slow > > process and members not having time to review the situation, so > > preferring to punt, and not distinguishing between the immediate issue > > for jessie and the general issue for stretch onwards. > > Have the notes from the discussion at debconf been published yet? Not quite, but I'm working on them right now (I only got back a few days ago). Should be out imminently (after giving a chance to comment - I don't want to be accused of falsification again). > I'm afraid that this issue is not going to be resolved in time for > stretch either unless things start moving. Things have already moved quite lot, and the agreement at debconf was to do it doko's way for stretch so for practical purposes it is resolved for now. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/
Re: Bug#797533: New CTTE members
+++ Didier 'OdyX' Raboud [2015-09-02 14:53 +0200]: > > One problem we have, I think, is that we allowed issues to get stalled > for quite long periods of time [0]. > What I really would hope new TC members could bring is more an ability > to react in bursts rather than a commitment to spend a fixed amount of > hours per week/month. A little perspective from an outsider who had cause to see the committee's working this year: Before interacting I had assumed that the CTTE was a powerful and effective body within debian, to which you could go if things had gone really badly wrong, and that if something was brought to the committee they would take a look quite quickly and (in the case of something up against the release deadlines) make a quick decision on whether X was out of order or not. It turns out the process is nothing like that, and indeed almost completely useless for such situations. In fact nothing happens until a monthly IRC meeting, where a backlog is considered. No-one has looked at the new issue in enough detail to form an opinion, and ways are looked for to see if the committee can avoid ruling by asking for other forms of mediation. This makes perfect sense from the committee POV, because making a decision involves properly understanding the issue, which requires time (possibly quite a lot of it), and quite often, asking the parties to go and sort this out themselves is a sensible choice. In the case of #766708 this procedure was not effective. The maintainer's use of veto power days before the freeze effectively went entirely unchallenged, and the complainants had to do a pile of work to work around it and get something workable in jessie. Ian put it quite well in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766708#305 The choice in the related bug #771070 to make us go and argue it out amongst ourselves was reasonable, and in the absence of the investment of a lot of time to properly understand the issue, probably the only realistic choice. So the process is working OK there, although there has still been disappointingly little substantive discussion of the _technical_ arguments, as was observed at debconf. So what I learned from this is that, as currently operating, the committee is incapable of making quick 'overrule unreasonableness' decisions. My overriding impression was that those involved simply did not have the time available that would be be needed to enable that. Maybe no-one has enough time, and even if one or two members did, the voting system means that quick decisions would require more than half the members to be able to invest the necessary time for understanding, which is even less likely. So it would appear to be the case than a quick response is simply not possible within our current structures? That is something to communicate to complainants because I'm not sure it's widely understood. Don't know if that's useful info or not - you probably knew all that, but hopefully it gives a little external perspective. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/
Re: Bug#797533: New CTTE members
+++ Steve Langasek [2015-09-09 12:17 -0700]: > On Wed, Sep 09, 2015 at 05:30:03PM +0100, Wookey wrote: > > > So what I learned from this is that, as currently operating, the > > committee is incapable of making quick 'overrule unreasonableness' > > decisions. My overriding impression was that those involved simply did > > not have the time available that would be be needed to enable that. > > No, what you see here is that the TC did not agree with you that the > maintainer's action was unambiguously unreasonable under the circumstances. Well, maybe. Maybe there were discussions to that effect I didn't see. In that case fair enough. The impression given was of a somewhat slow process and members not having time to review the situation, so preferring to punt, and not distinguishing between the immediate issue for jessie and the general issue for stretch onwards. I don't mean to have a go at the CTTE, or go over it all again. I was just trying to explain that the process was nothing like I had imagined it would be, and that this suprised me. Having seen how the meetings operate I do now understand why it is like it is. > If you conclude from this that raising the issue to the TC was not an > effective way to see your grievance addressed under those circumstances, > I won't disagree with you. (not forgetting that I didn't raise it to the TC (Helmut did)). Mind you, I don't know what else he could have done under the circumstances except suck it up. > But you are asserting that the reason for this > is that the TC is unable to act quickly to overrule. This is not the case; > there is historical precedent for quick overrules by the TC where there is > actual agreement that this is appropriate. OK. Fair enough. I do only have one data point :-) > That said, if developers have expectations of the TC that don't match > reality, that seems worth addressing. Well, I've had my expectations addressed. Not sure about everyone else :-) Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/
Re: Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-10-25 07:33 +0200, Tollef Fog Heen wrote: > ]] Ian Jackson > > > * Specifically, failed to give clear and constructive directions to > >those willing to do the work; > > I disagree with those characterisations. He's asked for clarifications > on what is broken without anything resembling an adequate reply. NOW, yes, when after 5 years people gave up filing bugs and brought it to the technical committee. _Now_ Ron has ample time to write very long responses. But the problem is that he didn't respond usefully in the bugs, for years. Various people said 'the debian version doesn't work, but the current upstream version does, can we upgrade please?'. And were mostly ignored or given short shrift. (Vincent just posted URLS). Normal people, who are not of the Ian and Ron (and me) 'old-school Debian, we quite like an argument' personality type, find this _extremely_ offputting. I am only involved because I work with Punit and he came to me asking for advice on what to do. He's not posting here because the last thing he wants is an argument (you know, the sort of quiet person who _really_ doesn't like arguments). He thinks it's silly that we can't have a newer version of global, but if that's what decided, then fine. He won't argue - he'll just use upstream, debian will be stuck with the ancient version, and we'll have lost a contributor. As he points out - there are many, many other things he can do with his spare time that don't involve an argument or dealing with a difficult maintainer. So if this thing is only assessed in terms of who is the most enthusiastic debater then Ron wins (he's very good at it). I realise this makes things harder for the TC to assess, because they see a rather one-sided view. I would just ask them to bear personalities and communication preferences in mind. (As I said to Punit, 'everything used to be like this, but we improved a lot in the last few years, and this is much rarer than it used to be'). Debian used to have a (largely deserved) reputation as an unpleasant project to work in. We've done a lot in the last few years to improve that situation. I invite the TC to reflect on how this would have played out if global had had a different maintainer. This is (or should be) about attitude, responsiveness, and helpfulness, at least as much as the technical (htags) debate. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-10-19 14:26 +0100, Ian Jackson wrote:
> Vincent Bernat writes ("Bug#841294: Overrule maintainer of "global" to
> package a new upstream version"):
> > Ron Lee is the current maintainer and disagrees on some issues with
> > upstream and therefore don't want to update to a more recent
> > version. See bug #574947:
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=574947
>
> Looking at the bug logs, Ron has failed in the one non-delegatable,
> non-negotiable task for the Debian maintainer of a package: namely, to
> make appropriate decisions with respect to the package and then to
> communicate those decisions.
>
> Even if the decision to keep the new upstream version out of Debian is
> correct, Ron has failed to provide a coherent explanation. He has
> failed to engage with those who were willing and ready to do the work.
>
> I think the package would be best served by having a new maintainer.
>
> (Sadly I don't think the TC is likely to agree with me. Historically
> the TC has very very slow to act against maintainers who block other
> people's work and who do not communicate properly.)
Indeed. The current situation is that the existing version is so old
that it doesn't work properly with modern code any more, but the
maintainer has refused to do any of:
1) upload a new package,
2) allow an upload which removes the offending CGI bit (which users don't
really care
about anyway)
3) write something to change the local behaviour to be satisfactory.
Upstream has been clear that they are not going to change how it
works, but they don't care if debian omits that bit or changes it
locally, so it seems to me that a maintainer has to do one of the
above three things. 5 years of this is more than long enough to
conclude that they are not doing their job adequately, and because of
our strong maintainership culture, are preventing other people from
doing that job.
It could just be NMUed, or a global6 uploaded so at least people would
have something to work with, but asking the TC to rule seems like a
more correct way to try and unbung this situation.
Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/
signature.asc
Description: Digital signature
Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-10-25 07:29 +0200, Tollef Fog Heen wrote: > ]] Wei Liu > > > Gtags in Debian doesn't work with modern code base. Last time I tried > > (several > > years ago), it segfault'ed while trying to index Linux kernel. > > FWIW, it worked fine in a test run I just did (on linux-4.9 rc 1), and > last time I used it, it also worked fine with the emacs integration, so > I don't recognise the crying from the rooftops about it being broken in > Debian. It seems that it depends on the kernel source tree in use. I just tried on a couple and found these reuslts: ~/debian/linux-4.0$ gtags ~/debian/linux-4.0$ global -s enable_dbg arch/arm64/include/asm/assembler.h 4 files generated: GPATH GTAGS GSYMS GTAGS ~/debian/linux-3.16.7-ckt9$ gtags gtags: directory stack over flow. ~/debian/linux-3.16.7-ckt9$ global -s enable_dbg global: GSYMS not found. only 2 files generated: GPATH GTAGS global 6.5.4 (upstream release) works on both. ( I also found 844330 in the process, which is just a packaging update issue ) Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-11-23 18:19 +0100, Didier 'OdyX' Raboud wrote: > The TC had an IRC meeting yesterday [0], and I (was) volunteered to wrap up > the different outcome possibilities, which would help forming our opinions. > Not all these options are exclusive, or would need an actual TC decision. > > Here we go: > > B) A fresher version of 'global' is uploaded to experimental 'soon' >(with or without interested parties' help; with or without a TC decision) > > This would imply: > - any interested party would file (and close) Debian bugs for issues and >regressions (with appropriate severities), to make the 'fitness for a >stable release' assessment easier, and earlier. OK, as Punit and I have prepared a current 6.5.5 which would be a candidate for a 'modern' release, and I think it's useful to have such a version available for people to test and file bugs against, I will do a 5-day delayed NMU to experimental. Putting it in experimental does not imply automatic transtion to stretch, nor block uploads via unstable, so seems a reasonable thing to do. This version is not perfect, but it is current with updated packaging. I'll include details of the known issues in one of the 'please can we have a new version' bugs. I think it's more useful to have the current state of play avalable than for me to keep messing with it privately. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-11-16 06:02 +1030, Ron wrote: > On Mon, Nov 14, 2016 at 04:55:06PM +0000, Wookey wrote: > > On 2016-10-25 07:29 +0200, Tollef Fog Heen wrote: > > > > > > FWIW, it worked fine in a test run I just did (on linux-4.9 rc 1), and > > > last time I used it, it also worked fine with the emacs integration, so > > > I don't recognise the crying from the rooftops about it being broken in > > > Debian. > > > > It seems that it depends on the kernel source tree in use. > > Just to clarify this here based on the details you provided when you > later reported https://bugs.debian.org/844356, it turns out that it > apparently doesn't depend on the kernel source tree or its version > as such - what you saw happens when the source tree is polluted with > infinite symlink loops, apparently due to a broken build script in > this case. Right. I was going to reply with that bug pointer here. It is an example of something that newer upstream versions cope with without failing. So is https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=756367 which I have just confirmed is still a bug with 5.7.1, but not with 6.5.5 (There was a request earlier in this thread for actual issues with 5.7.1, that are fixed in newre releases, IIRC) I added a note in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=574947 on that packaging update. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-11-16 15:23 +0100, Didier 'OdyX' Raboud wrote: [offlist] > * I'm not happy with delaying the landing of a 6.x version of global for > another release, and despite the somehwat short time before the stretch full > freeze (2017-Feb-05), we're talking about a single binary package without > reverse dependencies. I'm really afraid that a side-effect of the TC > discussion will be yet-another release without an up-to-date src:global. Thank you for some sanity on this matter. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-12-01 22:56 +0100, Philip Hands wrote: > Wookey <woo...@wookware.org> writes: > > On 2016-11-30 16:56 +, Ian Jackson wrote: > > > >> > And this last bit (integration with system web server) is the > >> > functionality that had security concerns raised by Ron [etc.] > >> > >> So, to be clear, it is this functionality which is dropped in the > >> package that you and Wookey uploaded to experimental/delayed ? > > > > Said package is available as of today: > > https://buildd.debian.org/status/package.php?p=global=experimental > > > > And that functionality was left in as it was a self-contained patch, > > pending determining whether in fact it remained useful/compatible or > > not. So you still get htmake and htconfig commands. > > > > However I have now done some checking, and no it doesn't work any more > > because it uses the btreeop command, which has been removed in current > > upstream, and the --action option to htags which is also gone. > >> - Run the htags server on a high-numbered port somehow. (Is there > >>some kind of simple scriptery provided, to do this?) > > > > There is an example in the NEWS file, which should go in the man page. It's > > trivial: > > > > cd HTML; python -m CGIHTTPServer > > or > > cd HTML; python3 -m http.server --cgi > > > > Then browse to http://localhost:8000/ > > This functionality is also provided by htags-server(1), which is > included in your package: > > https://www.mankier.com/1/htags-server > > [BTW htags-server/htags-server.1 should be added to debian/global.manpages] Well spotted. cheers for feedback. OK. Punit (and I) prepared an updated package with the above fixes, and htmake/htconfig dropped, so it's now pretty much a vanilla upstream package. Git repo is here: https://github.com/punitagrawal/global I've done a 5-day delayed NMU to experimental of 6.5.5-0.2. I'll put some more details in #816924 This is essentially a candidate for unstable/stretch, should 'putting v6 in unstable or stretch' be agreed-upon. So anyone who cares should check it out and file bugs. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-12-08 23:32 +1030, Ron wrote:
> On Mon, Dec 05, 2016 at 10:13:05PM +0100, Philip Hands wrote:
> > Ron <r...@debian.org> writes:
> >
Thanks for comprehensive reply, Ron.
I wonder if it would make more sense to have this discussion in
#816924 which is the actual bug for 'should we upgrade global to v6,
and if not, why not?' You have to be in the know to find this on the
ctte mailing list.
> > Version 6 includes a CGI script that one is expected to install in a
> > manner so hopelessly insecure that we'd not accept it in Debian.
> That one had removed the ability to run it from a secure system CGI
> location at all, and changed the way that it's generated yet again.
> It also made changes that guarantee everyone will need to fork it
> for distro use, because it now hardcodes a fun mix of paths, like
> /opt/local/bin for perl and /usr/local for global et al.
The .cgi scripts are generated from .in files which are correctly
parameterised with @PERLPATH@ and @GLOBALPATH@ etc. Upstream
unhelpfully ships pre-generated versions with the above arbitrary
local paths, but we kicked the build to force regeneration of these so
that all the scripts come out with correct debian paths. That was in
6.5.5-0.1 and is in 6.5.5-0.2 (with ctags path set correctly
too). Please file a bug if we missed any.
So this is not an outstanding issue, and no fork needed,but it would
be nice to improve upstrema's build and reduce debian patching here.
> It does comment out that line from above with a note:
> "To use this code, please uncomment in your own responsibility."
OK, so as shipped, that's not actually an active problem.
> But it also outputs a .htaccess enabling execution in the directory
> where the output is generated, whether you want to use it from there
> or not (and adds a second CGI, and a bunch of jquery doing AJAX too).
The world is absolutely full of jquery doing AJAX. That's not bad in
itself (much as some of us prefered the 'old web'. We should make it
use system jquery.
> It has a bunch of immediately obvious problems:
>
> - it still passes the unsanitised $pattern to global, which then
>passes it to popen (which again lets it be interpreted by the
>shell).
So $pattern used to be sanitised, because that's what CVE-2000-0952 is
about. Ah and in fact it still is on line 148. Hmm, but that's after
using it in a pipe, which probably isn't much use... That is pretty
shoddy.
> - it won't run with perl's taint mode enabled, because that
>simply kills it warning of unsafe operations on untrusted data.
>
> - perl's warnings aren't enabled, and it complains of other
>rookie code problems if you do.
>
> - Enabling 'use strict' makes it scream with pain and completely
>fail to compile and run.
>
>
> It would need more thorough auditing to confirm that there is(n't) an
> exploitable path through that, but given that it ignores even the most
> basic advice which perl has strongly recommended since perl4 was new
> and shiny, then if there isn't, it would be because of luck more than
> obvious care.
I don't claim any web security expertise, so will ask a
potentially-dumb question. How much of a real world problem is the
shoddiness of this script if it is only used locally, using
htags-server (which is the only functionality now provided by the
package)? It exploses the script only to localhost unless you
explicitly configure it to do more, but not 'the net'. Nothing is done
as root, but it is run as 'you' so potentially gives access to 'you's
data, rather than all of www-data's data.
The only report of an actual vulnerability I can find online ('global'
is a very unhelpful name to search for vulnerbilities!) is:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2000-0952 (from
2000 for global prior to v4.0.1
Which claimed to be fixing this very issue, but apparently it didn't stay fixed?
And this same code exists in global v5.7.1 (indeed both usages are
there, including the now-commented-out one), so how exactly was that
safer?
> It might not be utterly unfixable, but you'd have to convince upstream
> that security is important, or completely fork it for debian, which
> brings us back to exactly where we are - unless we just remove it all.
> But that would need Time, whoever does it.
I have not grokked why the shoddy code in 5.7.1 is safe but the same
shoddy code in v6 cannot be let out. Did htmake+htconfig stop people
entering $pattern in the form?
> It is what we now have enabled in the package that Wookey uploaded to
> experimental.
Indeed. Bugs (and even better patches) are welcome.
> > Also, for people that want personal access to htags there is a
> > htags-server command that brings up a dedicated server to run the CGI
> > as the invoking user, by default bound to a localhost por
Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-12-10 06:54 +1030, Ron wrote: > So I've now orphaned this package, and he has my blessing and sympathy > for being responsible for whatever happens with it from here. I haven't > filed a WNPP bug for that as we don't need to offer it to someone random. OK. Cheers. Warm potato accepted :-) > Wookey: if you want the complete git history, right back to the very first > package in 1999, you can grab it from the Vcs-Git URL in the sid package. > I'm not going to go Full Bruce and rage delete it, but eventually I should > decruft alioth and remove it from there, so if you want it you should > probably clone it somewhere that works for you. OK. I don't use git unless I have to, so I'll let Punit worry about that. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-12-09 11:58 +0900, Shigio YAMAGUCHI wrote: > Hello all, > 2016 19:05:55 +, Wookey wrote: > > The .cgi scripts are generated from .in files which are correctly > > parameterised with @PERLPATH@ and @GLOBALPATH@ etc. Upstream > > unhelpfully ships pre-generated versions with the above arbitrary > > local paths, but we kicked the build to force regeneration of these so > > that all the scripts come out with correct debian paths. That was in > > 6.5.5-0.1 and is in 6.5.5-0.2 (with ctags path set correctly > > too). Please file a bug if we missed any. > > It's my mistake. I will fix it soon. > > It is helpful if these bug reports are sent to bug-glo...@gnu.org. > Thank you in advance. Yes. Now that we have the package in some sort of reasonable shape in debian we will forward the relevant bits upstream, in order to minimise debian patches. There are quite a few things we want to discuss :-) Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-12-08 17:03 +0100, Didier 'OdyX' Raboud wrote: Sorry missed this in all yesterday's mail. > Wookey, Vincent, Punit: would any of you be willing to receive the 'global' > package maintainer hat? (which would of course come with the possibility to > share and change the maintainer status afterwards.) Punit is happy to be maintainer. I am happy to co-maintain with him (now that I'e gone the trouble of finding out a reasonable amount about the package). Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-12-08 17:33 +, Wookey wrote: > On 2016-12-08 23:32 +1030, Ron wrote: > > But it also outputs a .htaccess enabling execution in the directory > > where the output is generated, whether you want to use it from there > > or not (and adds a second CGI, and a bunch of jquery doing AJAX too). > > The world is absolutely full of jquery doing AJAX. That's not bad in > itself (much as some of us prefered the 'old web'. We should make it > use system jquery. OK. I tried making it use system jquery and some of the icons in htags seem to go missing. So the internal jquery-treeview and jquery-suggests may be incompatible. Needs a bit more research, so we'll ship the internal one in 6.5.5-0.3 for now. > > the icons and javascript and css for htags > Good point. > That's an easy fix. Included in 6.5.5-0.3 which I'm about to upload. So that fixes #587489, (which has only been pending with a patch since May 2011). (currently shipped in /usr/share/gtags rather than /use/share/global/gtags because upstream needs patching to sort that properly. One thing at a time.) And now htags browsing has nice little icons for moving about. shiny :-) We checked that 6.5.5 also fixes #847303. It does. I have a query pending with Ron for if it's OK with him to move to 0-day uploads to experimental as I don't think the conventional NMU delay is actually helping anyone in this case. It just makes things slow. I'll just replace the pending 0.2 upload with this 0.3 one for now. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: global maintainer : Draft ballot
On 2016-12-09 15:48 +0100, Didier 'OdyX' Raboud wrote: > With the precious help of Phil and Sam, here comes a draft ballot. It is > attached to this mail, and has been committed to the TC's git repository [0]. > > It contains the following ballot options: > > - Option A - Reaffirm Ron Lee as 'global' maintainer (§6.1.2) > > - Option B - Declare Wookey as 'global' maintainer (§6.1.2) > > - Option C - Decline to rule, consider case closed > > - Option FD - Further discussion The package was offered for adoption, and Punit and I have adopted it. 6.5.5 was uploaded yesterday. So somewhere between B and C happened and a vote is now moot. I guess this bug can be closed now, unless there are any loose ends to tie up? Not sure who's job that is? Thank you for everyone's attention on this matter. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: Overrule maitainer of "global" to package a new upstream version
On 2016-11-30 11:39 -0500, Sam Hartman wrote: > That is, I disagree with you that I need to address the question of > htags and figure out whether htags users are being impacted. > That might have been true for one release. > But over a longer time frame, the really strong presumption is that we > prefer a version that is being actively developed to one that isn't. > That if people won't step forward and maintain htags, it goes awy. Just to be clear, you are confusing 'htags' and 'htmake/htconfig' (I was too when this started). htags is still here, works fine, and is not going away, as Punit explained. The thing that is likely to go away is 'centralised htags browsing using system webserver' which is what Ron's htmake and htconfig provided. > But I think what you're getting here from a lot of people is a belief > that our community norm strongly favors active development and new > software. And sometimes when features are effectively not maintained in > a manner that we can package them, they go away. Right. We are a distro and by default we package what upstream provides. A maintainer can choose to go further that that and add/maintain extra functionality if they like, but it's an ongoing workload that has to be carried. It is no doubt possible to modify htmake/htconfig or write something equivalent to provide a centralised index of 'globalified' source trees, but as upstream no longer attempts to do such a thing there is a good argument that the debian packaging shouldn't either. I certainly don't think that the fact that this was done once is a good reason to stop packaging new releases. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Re: Bug#841294: Overrule maintainer of "global" to package a new upstream version
On 2016-11-30 16:56 +, Ian Jackson wrote: > On to some questions it raises for me: > > > Optionally, "htags" can generate a dynamic index (which reduces disk > > space usage) but requires an http server setup to be able to serve the > > pages. In this scenario, you will also need to be able to execute CGI > > scripts as the symbol lookup is done when serving the http request (as > > opposed to pre-processed when using static pages). > > > > And this last bit (integration with system web server) is the > > functionality that had security concerns raised by Ron [etc.] > > So, to be clear, it is this functionality which is dropped in the > package that you and Wookey uploaded to experimental/delayed ? Said package is available as of today: https://buildd.debian.org/status/package.php?p=global=experimental And that functionality was left in as it was a self-contained patch, pending determining whether in fact it remained useful/compatible or not. So you still get htmake and htconfig commands. However I have now done some checking, and no it doesn't work any more because it uses the btreeop command, which has been removed in current upstream, and the --action option to htags which is also gone. We are having a think about whether this code can be hacked about a bit to remain useful or if its approach is simply no longer relevant, given upstream changes. And I just said replying to Sam, there is a lot be said for not diverging significantly from upstream, because there is significant overhead in doing so. So this patch is currently likely to get dropped unless someone works out a way to make it work/be useful. > But AIUI this functionality remains: > > > In addition to the above mechanisms, upstream also provides "htags" > > which can be used to generate an HTML version of the index. "htags" > > uses the index created by "gtags" and the source tree as input and > > generates html files which allow you to navigate the source code in > > the browser. By default, htags generates static pages which means you > > can browse the code in a local browser without requiring a web server. > So the impact for a user of the existing functionality the regression > is not that their entire workflow is disrupted. > Rather (unless the software is improved) they have these choices: > > - Put up with pregenerated html indices. Is the only downside >of this that they use additional disk space, and presumably >cpu time to generate them ? Correct. And they can easily be symlinked from say /var/www/html/global/ to make them a) accessible from system webserver and b) conveniently listed. The static HTML is 7G for a kernel source tree. The dynamic version is 3.3G. (or 4.8G with the suggested '--suggest2' options) So there is a x2 space overhead. > - Run the htags server on a high-numbered port somehow. (Is there >some kind of simple scriptery provided, to do this?) There is an example in the NEWS file, which should go in the man page. It's trivial: cd HTML; python -m CGIHTTPServer or cd HTML; python3 -m http.server --cgi Then browse to http://localhost:8000/ > - If the machine is not really multi-user, tear down the security >boundary defending the webserver from their user account, and give >their user account access to the webserver cgi directory (or plumb >it in with symlinks). (Are there any instructions or scripts for >this?) (Also this assumes that the source code is not super >secret.) I've just started playing with this to see what can be done (bearing in mind the question of how much effort we should put into this sort of upstream divergence anyway). Simply symlinking /var/www/global/sourcetree to from gives you a convenient list of globalised htags to look through if they are static. The issue with dynamic htags (and the search functionality) is that by default your web server won't run .cgi scripts in arbitrary directories (for good reasons). I don't know if we can have a central .cgi that 'sources' the global.cgi or completion.cgi in the htags sourcetree/HTML dir and uses it, or if that's still not actually a good idea. Possibly something nifty with apache config scripts would work. Suggestions welcome. > I don't know much about this, but several of these choices seem likely > to be plausible choices for many if not most current users of htags. Right. I think the functionality upstream provides is fine. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: Digital signature
Bug#881339: marked as done (allow node-babel-preset-env to build depend on itself)
On 2018-02-22 11:59 +, Ian Jackson wrote: > Margarita Manterola <ma...@debian.org>: > > Thus, we are closing this bug now, as it's not actionable. > > > > We suggest that you work together with the FTP Masters in > > figuring out a solution to this problem. > > I'm not sure I agree with this. Me neither. (I mean maybe it's true that the TC cannot overrule the FTPmasters, but they can provide technically sensible advice). I don't see any difference between this js package and many other packages in the archive which self-bootstrap or otherwise circularly depend on themselves. So I don't see why this one is not acceptable, and thus so far as I can tell the FTP master was wrong to reject it, or at the very least we should be discussing this issue in terms of policy. It seems to me that a discussion on debian-devel would have been sensible before asking the TC to rule, but as they haven't, it's still sensible. We have plenty of expertise about circular dependencies, especially amongst those involved with cross-building and bootstrapping, where they are more obvious than in normal practice. We could clearly do a better job of formalising meta-data or mechanisms for software that has this characteristic, and we can try to persuade upstreams to do less of it where possible, but self-dependency (at both same and older versions) also occurs more frequently when cross-building or bootstrapping and is not necessarily wrong or bad (self-dependency at the same version is much more of a problem than self-dependency at older versions). Anyway, Pirate - I suggest you ask about this on debian-devel where we can have a pulic discussion about policy and whether there is anything special about this case which makes it not suitable software for the archive. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Re: Bug#914897: debootstrap, buster: Please disabled merged /usr by default
On 2018-12-03 00:37 +0100, Wouter Verhelst wrote: > This is incorrect. The current plan has some systems be merged-/usr and > others not while we are in the transition. It therefore introduces > Debian-wide complexity in that maintainers are expected to support both > merged and unmerged /usr, which causes problems (as we see here). It > also effects a change without the maintainers of the software in > question being involved, which could have unintended side effects with > some packages that we don't know yet (and that we probably won't know > about until the release of buster). > > Changing this through a policy change, as we've always done, would not > have those problems: > > - Maintainers are expected to move their own package to merged-/usr, so > they would be aware of issues that might ensue and would be able to > deal with them. > - We are not expected to be supporting merged-/usr and unmerged-/usr at > the same time; instead, we'll be gradually moving towards a > merged-/usr situation. > - We don't end up with packages' files being moved from under them. I don't want to add pile of verbiage here, but I'd just like to say that everything Wouter said makes a whole lot of sense to me. We know how to do this sort of transition, and yes it takes some time, but that's OK. Using usrmerge to try and shortcut this, producing the awkward 'dual-state' issues does not seem to me to be a good way to go. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Re: Bug#994275: Reverting breaking changes in debianutils
On 2021-09-15 01:36 +0300, Adrian Bunk wrote: > > This is a request to override the maintainer of debianutils on several > changes that were done to the package in unstable after the release of > bullseye. ... > 2. The "which" program must not print any deprecation warnings. > > The deprecation warning is misleading, and it is actually what > causes most breakage. > > For non-interactive usage, "which" printing a deprecation warning > is breaking many scripts. Just to record a relevant example which is not yet in a bug report (because tensorflow is not yet in the archive, only NEW, so no-one can file bugs against it). Use of which inside the bazel build system (in a .bzl script) (which at least tensorflow does, there may be others), causes an FTBFS because of the deprecation message. Probably this could be fixed (it's not clear exactly which layer of the system is erroring when it probably shouldn't), but a proper transition plan would mean that I would never even notice this. One which would replace another and nothing would break. That is the sort of thing I expect to happen in Debian - we are generaly good at this stuff, and it's a big reason for using the distro. I agree with Helmut's points about unnecessarily prescriptive requests to the TC, and also with his point that time matters. Right now I have to jump through hoops to build an out-of-archive which and have that local package available in the build environment in order to do any work (or go off on a tangent trying to fix bazel, or I guess I could make a nobbled local debianutils - it's all a similar level of significant hassle). If the debianutils change was just reverted I could get back to my day-job. Probably this sort of breakage was not expected by the debianutils maintainer, but it's real, and quite disruptive. I would like to add my support for a quick revert for now, followed by a more considered approach to get whatever it is done that people want. I'm sure we can work out one that doesn't break random existing which (and tempfile) usages. I appreciate Adrian bringing this to the committee and hope they can move a bit faster than usual to revert stuff so we can all have another go at this, and I can go back to not caring at all where 'which' comes from, or even which which is in use. (The only good thing about this whole thing has been the opportunity for whichy wordplay :-) Cheers Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#994275: Draft resolution for reverting changes in debianutils
On 2021-10-13 19:37 -0300, David Bremner wrote: > Sean Whitton writes: > >The which(1) program must not print any deprecation warnings. > > I remain to be convinced on this point. If I understand the issue > correctly the problem is with autopkgtests failing because they were not > expecting output on stderr. It's not just that. Builds fail too. tensorflow now FTBFS in unstable because of this change, and the way bazel deals (or fails to deal) with it. I gave details further up the thread. > I understand that people find the message annoying, and perhaps not that > useful, but I don't think that rises the level justifying overriding a > maintainer. I think causing build failures is enough reason to say this. I don't suppose that mine is the only one. Yes those builds are buggy and should not do this, and we should make efforts to find out why bazel (or possibly the build scripts it is operating on) is/are so crappy, but for now I agree that reverting this is the right thing to do. We have time to do this transition properly and quietly in the background, without causing random breakage. A message about a binary moving from one package to another does not need to be printed on every usage of that binary. Indeed it is actively unhelpful to do so. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#994275: Reverting breaking changes in debianutils
On 2021-10-24 19:08 +, Clint Adams wrote: > On Sun, Oct 17, 2021 at 02:33:44PM +0100, Wookey wrote: > > I think causing build failures is enough reason to say this. I don't > > suppose that mine is the only one. Yes those builds are buggy and > > should not do this, and we should make efforts to find out why bazel > > (or possibly the build scripts it is operating on) is/are so crappy, > > but for now I agree that reverting this is the right thing to do. > > > > We have time to do this transition properly and quietly in the > > background, without causing random breakage. A message about a binary > > moving from one package to another does not need to be printed on > > every usage of that binary. Indeed it is actively unhelpful to do so. > > Boyuan packaged GNU which and uploaded it to NEW in August. It is now > October, and neither GNU which nor *BSD which nor any other which > alternative is in unstable. Presumably one of these could have put > a band-aid on your bazel problem, though of course any version of > `which` might output things to stderr for a variety of reasons. That package is the band-aid I am currently using, but (because it has indeed not progressed through NEW yet, which is presumably down to ftpmaster bandwidth) it's a hassle where I have to make sure it is made available in the build environment each time. > Lots of things broke between buster and bullseye. Most of them broke by accident or for good reasons. You broke this semi-deliberately, by deciding not to manage a quiet background transition of the which binary to some other package, but to print a deprecation message and let other people deal with the fallout. OK, maybe you expected the change to be harmless, but you didn't revert it once it became clear that this was not a good way to do the transition. IIUC the tech committee have now told you to do it properly, as a managed transition. Good. > Is the difference that these packages aren't Essential? The difference from where I am standing is the attitude of the maintainer to doing things properly vs causing other people hassle. Nobody is making you remove/move which. If you want to move/remove which then do it in a way that doesn't break other people's stuff (as much as possible). In this case that really isn't hard to do (just wait until GNU which passes NEW, and preferably file bugs on packages that now need to depend on it). I didn't understand why you wanted to make this change in the first place, and I don't understand why you didn't just revert the message when it became clear that it was a problem, and I don't understand why you are still trying to justify your somewhat bloody-minded approach to this (should-have-been) minor issue, even after the tech comittee have agreed that it was not a good approach. Please, just remove the deprecation message, until GNU which is in unstable (like you should have done a couple of months ago, when first asked), then work out how the transition should go such that no-one using which will actually notice or care. Then you can throw away the hated binary :-) and we can all be happy. I understand that it's galling to have the TC tell you to take a different approach, especially when you've doubled-down on your approach, but I'm afraid the TC is correct here, so please, stop arguing, do what's best for the project, and soon enough this will all be over, and we'll all have what we want. And you may (or may not) choose to reflect on the the point that we _could_ have got to exactly the same point without this argument if you'd made different choices. From my personal point of view this is solved short-term the moment the deprecation message is gone in unstable, and long term as soon as GNU which enters unstable. I hope this is my last-ever word on this tiresome subject. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#1007717: Native source package format with non-native version
On 2022-03-16 15:29 +, Ian Jackson wrote: > In practice, the vast majority of packages are maintained in git on > salsa. The maintainers use those git repositories as the PFM. > but almost everyone is already treating git as primary. Is this definitely true? For example: I know I'm not doing this. I did try, and I do have some git repos on salsa, but I've mostly given up with it all and stuck with uscan and tarballs and quilt (and my trusty 'packages' directory). It's much easier for me. (The salsa repos that exist for my packages are not canonical and often stale). I'm sure Ian is right that there is a trend towards git from tarballs and dscs, but I just question whether we know it is 'the vast majority'? Are there really now very few maintainers using the 'classic tooling'? How do we know? Wookey -- Principal hats: Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#994388: dpkg currently warning about merged-usr systems
On 2022-04-08 09:36 +0200, Wouter Verhelst wrote: > > The dpkg maintainer has chosen not to engage with the TC in #994388, and > now seems to be actively subverting a validly-made TC decision. > > I do believe it reasonable to assume the dpkg maintainer has a point if > he believes that the currently-chosen way of moving forward is harmful. > However, the right way for him to make that point would have been to > engage with the TC, the body constitutionally placed to resolve > conflicts of this manner, not ignoring them and then doing whatever he > wants when the decision inevitably doesn't go his way. > I encourage the dpkg maintainer (Cc'd) to engage with the TC in this > matter. It is not yet too late; That all sounds reasonable, but there is the long-standing issue that Guillem has never accepted that the TC has authority over the project. I forget the details, but given that he does not see it as valid it's not surprising that he is not engaging with it. However he should engage with this dpkg bug. It's an important one. Guillem, I'm sure you see all this as repeating yourself, but we cannot either reach a solution, or determine that one is not possible given current maintainership and TC decisions, without discussing the details. Like Wouter, I am inclined to agree with the dpkg maintainer that the current plan is broken, but I also accept the TC's authority. SFAICT We've made a right mess of this by not applying our usual technical rigour (BICBW - I have not followed all the ins and out of this).. At this point I am more disappointed in the people who keep insisting that 'it mostly works' is good enough, than I am in the bloody-minded dpkg-maintainer. Debian is not a 'mostly works' project. We do things properly - or at least we used to, even if that takes a long time. Some people have cited the multiarch dpkg changes as an example of work on dpkg being difficult. I was quite closely involved in all that and yes it took a long time but it was done right in the end, and it was the dpkg maintainer who insited on it being done right. Yes there was an incompatible syntax change but that was due to Ubuntu releasing with an implementation that was not good enough for upstream. It was annoying at the time but the pain was fairly short and we ended up in a better place in the long term. SFAICT the dpkg maintainer is applying the same rigour here, and the only way to fix this is for people who want to get usrmerge done to engage, with patches. If they want to prove that no patches for the current approach will ever be accepted, that can only be done by engaging further. Yes it will be hard work, but if it's not done we are just stuck. Wookey -- Principal hats: Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
