Re: Bits from the Release Team (Jessie freeze info)
Am 29.10.2013 17:48, schrieb Ian Jackson: (Mind you, I have my doubts about a process which counts people promising to do work - it sets up some rather unfortunate incentives. I guess it's easier to judge and more prospective than a process which attempts to gauge whether the work has been done well enough.) As an example I remember having received several complains from e.g. the GCC maintainers in regards to the state of gcc on various ports[1]. Here I would suspect a patch would be sufficient without needing to actually NMU gcc to get the fix in. There are also stuff like the port concerns from DSA that attention. Right. that's not enough. GCC has some primary and some secondary release architectures. Toolchain support for primary architectures in Debian should be waived, for the secondary and other architectures, Debian needs somebody who is maintaining the relationship between Debian and upstream. Surprisingly this is the case for many non-release Debian architectures like kfreebsd, the Hurd, alpha, hppa, m68k, but not for Debian release architectures like s390, sparc, ia64 and mips*. So we really need somebody to care about this, where the port is considered a secondary citizen or no citizen, or we should demote a port to the ports archive. Matthias -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/527c2170.8020...@debian.org
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On Tue, Nov 05, 2013 at 08:53:05AM +0100, Niels Thykier wrote: [1] I certainly wouldn't have space for something like this: http://en.wikipedia.org/wiki/File:Z800_2066_JKU.jpeg (and much less the money. Yeah I know that is technically not an s390, but as I understand it, an s390 should be around that size) That is in fact a s390, and pretty much the smallest of the zSeries you can find. You wouldn't be able to do anything with it even if you got it, though - it doesn't have internal storage at all (no Mainframe has, except the previous-generation Multiprise 3000 series), and requires external FICON/ESCON SAN storage to boot/operate. So you'd have to take a big clunky enterprise array of disks as well - just another full rack, if you're lucky. I was offered one of these z800 at some point, and had to pass on it too. Oh, and then there's the licensing stuff... chances of getting the required IBM assistance to get it (re)installed are about on par with Justin Bieber's chances of getting elected as President. There's an emulator (hercules) which can run zSeries operating systems on top of Linux, if you can get your hands on those. Anyway, sorry for going offtopic. :-) Lennert signature.asc Description: Digital signature
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
Niels Thykier writes (Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))): On 2013-11-03 16:03, Steven Chamberlain wrote: http://udd.debian.org/bugs.cgi?release=jessie_or_sidmerged=ignfnewerval=7kfreebsd=1sortby=severitysorto=desccseverity=1ctags=1 Actually, I meant a real BTS page (e.g. like [1]) rather than just a list of tagged bugs. The list of tagged bugs definitely have it uses, but it does not give me an overview of which bugs should be fixed by the maintainer of the given package and which the porters should fix. I think this would be a good idea. If the psuedo-package had a predictable name which depended only on the architecture, even better. Ian. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/21113.13532.963985.569...@chiark.greenend.org.uk
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On Tue, 05 Nov 2013, Niels Thykier wrote: In this regard; I am guilty of filing some those bugs without tagging them. Honestly, adding the tags get a bit in the way right now. If a package FTBFS on 4 architectures, I have to dig up 3-4 different usertags (with different user) and associate it with the bug. This sounds like a case where we should turn these usertags into fully fledged tags. [Or alternatively, they should just be made usertags under the debian-po...@lists.debian.org user or similar.] I'm OK with assisting with either, but I need to know which solution porters would prefer. -- Don Armstrong http://www.donarmstrong.com For those who understand, no explanation is necessary. For those who do not, none is possible. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131105183439.gy9...@rzlab.ucr.edu
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On Tue, 05 Nov 2013, Don Armstrong wrote: On Tue, 05 Nov 2013, Niels Thykier wrote: In this regard; I am guilty of filing some those bugs without tagging them. Honestly, adding the tags get a bit in the way right now. If a package FTBFS on 4 architectures, I have to dig up 3-4 different usertags (with different user) and associate it with the bug. This sounds like a case where we should turn these usertags into fully fledged tags. [Or alternatively, they should just be made usertags under the debian-po...@lists.debian.org user or similar.] I would also be OK with creating a pseudopackage as well as Ian suggested. [Or multiple pseudopackages.] Something like i386.ports.debian.org or similar would work, with each current port getting a pseudopackage, and the maintainer of the pseudopackage being the ports list. -- Don Armstrong http://www.donarmstrong.com America was far better suited to be the World's Movie Star. The world's tequila-addled pro-league bowler. The world's acerbic bi-polar stand-up comedian. Anything but a somber and tedious nation of socially responsible centurions. -- Bruce Sterling, _Distraction_ p122 -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131105185031.gz9...@rzlab.ucr.edu
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
Hi, On 05/11/13 18:50, Don Armstrong wrote: On Tue, 05 Nov 2013, Don Armstrong wrote: This sounds like a case where we should turn these usertags into fully fledged tags. [Or alternatively, they should just be made usertags under the debian-po...@lists.debian.org user or similar.] Either of those sounds good. Fully-fledged tags would be the easiest for most reporters to remember to use, but I wonder if this pollutes the tag namespace. [Or multiple pseudopackages.] Something like i386.ports.debian.org or similar would work, with each current port getting a pseudopackage, and the maintainer of the pseudopackage being the ports list. Would that be only for generic issues with a port, not specific to a package? I doubt this would be used much. These bugs might typically be reassigned to kernel packages or eglibc anyway. Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/527949c5.8040...@pyro.eu.org
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On Tue, 05 Nov 2013, Wouter Verhelst wrote: Well, I did ask for the creation of port-specific tags back at debconf8 (if I'm not mistaken), but you told me to go for usertags instead ;-) Sounds familiar. Usertags have the advantage of not requiring me to do any work. But presumably at the time I hadn't thought of the difficulties of coordinating all of the different usertags between porters. Yes, I think that's a good idea; it would avoid issues where maintainers are waiting on porters and vice versa, since the reassigning of a bug to a port pseudopackage would make it clear who's waiting for whom. Additionally, it would allow porters to have a todo list of things that need to be done for their port but aren't specific to any one package (or of which the root cause hasn't been found yet, e.g., recently compiled binaries segfault, but we don't know why yet) If you're going down this road, I would appreciate it if ports listed on debian-ports.org would also be getting pseudopackages. Since they would all be under the same ports.debian.org (or similar) namespace, I wouldn't have a problem with it. [My main concern about pseudopackages is polluting the package namespace; since I can't imagine anyone ever wanting to create a package called someport.ports.debian.org for a sane reason, that shouldn't be a big deal.] It would also be possible (in the meantime) for bugs to be assigned to both the port-specific pseudopackage, and the original package which spawned the bug. In any event, if a few active porters wouldn't mind creating a wishlist bug against bugs.debian.org for this with a suggested course of action, I'd appreciate it. Assuming there is no significant disagreement about that course of action, I'd like to implement it within a week or so. -- Don Armstrong http://www.donarmstrong.com PowerPoint is symptomatic of a certain type of bureaucratic environment: one typified by interminable presentations with lots of fussy little bullet-points and flashy dissolves and soundtracks masked into the background, to try to convince the audience that the goon behind the computer has something significant to say. -- Charles Stross _The Jennifer Morgue_ p33 -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131105201345.ga9...@rzlab.ucr.edu
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On 03/11/13 10:54, Niels Thykier wrote: Come to think of it; maybe we should have a BTS page for each of the ports (e.g. a pseudo package in the BTS). We've had this on kfreebsd, due it to having been a release goal: http://udd.debian.org/bugs.cgi?release=jessie_or_sidmerged=ignfnewerval=7kfreebsd=1sortby=severitysorto=desccseverity=1ctags=1 It uses usertags, but someone has to set those. Porters usually set them on bugs they file; but quite often FTBFS on kfreebsd bugs are filed without being tagged or Cc'd to our list, so someone has to periodically look for and tag things. Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/527665af.2040...@pyro.eu.org
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On 2013-11-03 16:03, Steven Chamberlain wrote: On 03/11/13 10:54, Niels Thykier wrote: Come to think of it; maybe we should have a BTS page for each of the ports (e.g. a pseudo package in the BTS). We've had this on kfreebsd, due it to having been a release goal: http://udd.debian.org/bugs.cgi?release=jessie_or_sidmerged=ignfnewerval=7kfreebsd=1sortby=severitysorto=desccseverity=1ctags=1 Actually, I meant a real BTS page (e.g. like [1]) rather than just a list of tagged bugs. The list of tagged bugs definitely have it uses, but it does not give me an overview of which bugs should be fixed by the maintainer of the given package and which the porters should fix. It uses usertags, but someone has to set those. Porters usually set them on bugs they file; but quite often FTBFS on kfreebsd bugs are filed without being tagged or Cc'd to our list, so someone has to periodically look for and tag things. Regards, In this regard; I am guilty of filing some those bugs without tagging them. Honestly, adding the tags get a bit in the way right now. If a package FTBFS on 4 architectures, I have to dig up 3-4 different usertags (with different user) and associate it with the bug. Do we have a tool you can give a source package, a version plus a list of architectures and it will generate a bug with the right tags? I think that would help a lot for me. ~Niels [1] http://bugs.debian.org/release.debian.org -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52789fdb.2000...@thykier.net
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On 2013-11-03 23:04, Steve Langasek wrote: On Sun, Nov 03, 2013 at 11:54:34AM +0100, Niels Thykier wrote: [...] I suppose a sponsor-only DD could be sufficient, provided that the sponsor knows the porters well enough to be willing to sign off on e.g. access to porter boxes. I guess the sponsor would also need to dedicate time to mentor (new?) porters on workflows and on quicks like when is a FTBFS RC and when it isn't etc. Why would the sponsor need to be involved in getting the porters access to porter boxes? Porter boxes exist so that DDs *not* involved in a port have access to a machine of the architecture and can keep their packages working. I've never heard of a porter who didn't have access to their own box for porting work. I will not rule out that it was a poor choice of example on my part for ia64 (and maybe powerpc), which is(/are) the concrete port(s) we would be talking in this case. That said, it is my understanding that one does not simply own an s390(x)[1]. Nor would I be concerned to have arm porters that worked on all 3 arm ports while possessing hardware only for a (non-empty) subset of those architectures. ~Niels [1] I certainly wouldn't have space for something like this: http://en.wikipedia.org/wiki/File:Z800_2066_JKU.jpeg (and much less the money. Yeah I know that is technically not an s390, but as I understand it, an s390 should be around that size) -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5278a3e1.30...@thykier.net
Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On 2013-10-29 17:48, Ian Jackson wrote: Niels Thykier writes (Re: Bits from the Release Team (Jessie freeze info)): [...] As mentioned we are debating whether the 5 DDs requirement still makes sense. Would you say that we should abolish the requirement for DD porters completely? I.e. Even if there are no (soon to be) DDs, we should consider the porter requirements fulfilled as long as they are enough active porters behind the port[0]? I don't have a good feel for the answer to that question. It's just that if it is the case that a problem with ports is the lack of specifically DDs, rather than porter effort in general, then sponsorship is an obvious way to solve that problem. If you feel that that's not really the main problem then a criterion which counts porters of any status would be better. I suppose a sponsor-only DD could be sufficient, provided that the sponsor knows the porters well enough to be willing to sign off on e.g. access to porter boxes. I guess the sponsor would also need to dedicate time to mentor (new?) porters on workflows and on quicks like when is a FTBFS RC and when it isn't etc. (Mind you, I have my doubts about a process which counts people promising to do work - it sets up some rather unfortunate incentives. I guess it's easier to judge and more prospective than a process which attempts to gauge whether the work has been done well enough.) [...] Thanks, Ian. Ah, you are not the first to question this process. Obviously, we intend to keep people up on their promise by actively maintaining their port. Sadly, we do not have a clear definition of actively maintained ports and I doubt we will have it any time soon either. But porters can start by working on the concerns from DSA (if they haven't already done so). Ports having gcc-4.6 as default compiler might also consider improving in that area[1]. Then there are more concrete things like ruby's test suite seg. faulting on ia64 (#593141), ld seg. faulting with --as-needed on ia64 (#718047[2]), a lot of ruby packages being stuck on kfreebsd-any due to ruby2.0 FTBFS (#726095[3]). Personally, I would also expect that key-packages work on all ports (on which they are built) in general[4]. All of the (non-mild) DSA concerns are already something we will officially hold against the ports. Most of the other issues listed above are not official concerns. However, I would not be surprised if most of them became official issues eventually. Until we have a clear definition of actively maintained ports, I would recommend porters to err on the side of being verbose over being silent. As an example, lack of visible reply to mails like [5] makes it seem like nobody is home. Mind you, I am not saying porters have the responsibility to fix every problem forwarded to their port list. I am also aware that sometimes issues/mail disappear in the depths of people inbox - heck it happens to me as well. Come to think of it; maybe we should have a BTS page for each of the ports (e.g. a pseudo package in the BTS). That way it would at least be easier for all people to find outstanding issues for the port[6]. It would also give us the possiblity to trivial declare a problem RC (or not) for ports. (Plus, then I won't have to update some random file on release.d.o for every new issue :P) Anyhow, I hope to be able to give a more official statement in the near future. ~Niels [1] Nothing official yet, but gcc-4.6 (and earlier) /might/ not be acceptable as a default for Jessie. [2] Apparently it is controversial whether that bug should be RC, but it definitely looks like that kind of thing that will cause issues for ia64 later. [3] That one got a patch, but it might be worth it to put some pressure on the maintainer or even doing a NMU. [4] A rule of thumb could be something like your port should probably not be listed here in the long run: http://udd.debian.org/bugs.cgi?release=jessie_and_sidmerged=ignkeypackages=onlyfnewerval=7flastmodval=7rc=1sortby=idsorto=asc [5] https://lists.debian.org/debian-mips/2013/08/msg5.html Btw, this is not intended to single out mips. [6] I know that people have been usertags for issues that affect the port, but it is not clear to me that all those usertags bugged is something we expect porters to fix. Rather it seems more like porters tagging the FTBFS bugs they file. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52762b6a.5060...@thykier.net
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
Niels Thykier dixit: Then there are more concrete things like ruby's test suite seg. faulting on ia64 (#593141), ld seg. faulting with --as-needed on ia64 And only statically linked klibc-compiled executables work on IA64, not dynamically linked ones. I’ve looked into it, but Itanic is so massively foreign I didn’t manage to find out anything except that the problem appears to happen before main. Until we have a clear definition of actively maintained ports, I would recommend porters to err on the side of being verbose over being silent. I’ve held off on the m68k side because I think the role call was only for architectures in the main archive, right? [1] Nothing official yet, but gcc-4.6 (and earlier) /might/ not be acceptable as a default for Jessie. Didn't Doko say he’d want 4.8? We (on the m68k side) are putting effort into that one, since 4.7 appears to only be used by eglibc right now. And 4.6 for GNAT, but gnat-4.8 is new, and the ICE may be fixed as there’s active upstream on the GCC/m68k side. bye, //mirabilos -- diogenese Beware of ritual lest you forget the meaning behind it. igli yeah but it means if you really care about something, don't ritualise it, or you will lose it. don't fetishise it, don't obsess. or you'll forget why you love it in the first place. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1311031444380.31...@herc.mirbsd.org
Re: Potential issues for most ports (Was: Re: Bits from the Release Team (Jessie freeze info))
On Sun, Nov 03, 2013 at 11:54:34AM +0100, Niels Thykier wrote: On 2013-10-29 17:48, Ian Jackson wrote: Niels Thykier writes (Re: Bits from the Release Team (Jessie freeze info)): [...] As mentioned we are debating whether the 5 DDs requirement still makes sense. Would you say that we should abolish the requirement for DD porters completely? I.e. Even if there are no (soon to be) DDs, we should consider the porter requirements fulfilled as long as they are enough active porters behind the port[0]? I don't have a good feel for the answer to that question. It's just that if it is the case that a problem with ports is the lack of specifically DDs, rather than porter effort in general, then sponsorship is an obvious way to solve that problem. If you feel that that's not really the main problem then a criterion which counts porters of any status would be better. I suppose a sponsor-only DD could be sufficient, provided that the sponsor knows the porters well enough to be willing to sign off on e.g. access to porter boxes. I guess the sponsor would also need to dedicate time to mentor (new?) porters on workflows and on quicks like when is a FTBFS RC and when it isn't etc. Why would the sponsor need to be involved in getting the porters access to porter boxes? Porter boxes exist so that DDs *not* involved in a port have access to a machine of the architecture and can keep their packages working. I've never heard of a porter who didn't have access to their own box for porting work. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: Bits from the Release Team (Jessie freeze info)
Niels Thykier writes (Bits from the Release Team (Jessie freeze info)): Results of porter roll-call === ... Summary table: Arch || DDs || NMs/DMs || Other || Total - ---++-++-++---++-- armel || 5 || 0 || 2 ||7 armhf || 6 || 1 || 2 ||9 hurd-i386 || 5 || 0 || 3 ||8 ia64 || *0* || 0 || 3 ||3 kfreebsd-amd64 || 5 || 0 || 2 ||6 kfreebsd-i386 || 5 || 0 || 2 ||6 mips || 2 || 0 || 1 ||3 mipsel || 2 || 0 || 1 ||3 powerpc[1] || (1) || 0 || 2 || 2.5? s390x || 1 || 0 || 1 ||2 sparc || 1 || 0 || 0 ||1 ... Based on the number of porters, we are considering changing the current requirements of 5 DDs to better reflect the reality of the situation. We will follow up in a future bits on the changes. Thanks. I think it is disappointing to find that we may be dropping architectures where a significant amount of effort is available, simply because the volunteers don't have enough status - specifically, because of a lack of DDs. I'm keen that Debian should continue to support a wide range of architectures. Would it help if I, as a DD, volunteered to sponsor porter uploads for any architecture ? That is I guess I'm volunteering to become a new kind of person - a non-port-specific porter sponsor. Obviously I will review the debdiff etc. I'm an experienced C programmer with some background in C language lawyering and portability stuff, so I should usually be able to do a decent review of a patch even on an unfamiliar architecture. In fact, regardless of what the release team decide for the policy, I would be happy to sponsor porter uploads. Please just email me. Ian. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/21103.52917.876297.985...@chiark.greenend.org.uk
Re: Bits from the Release Team (Jessie freeze info)
On 2013-10-29 16:05, Ian Jackson wrote: Niels Thykier writes (Bits from the Release Team (Jessie freeze info)): Results of porter roll-call === ... Summary table: Arch || DDs || NMs/DMs || Other || Total - ---++-++-++---++-- armel || 5 || 0 || 2 ||7 armhf || 6 || 1 || 2 ||9 hurd-i386 || 5 || 0 || 3 ||8 ia64 || *0* || 0 || 3 ||3 kfreebsd-amd64 || 5 || 0 || 2 ||6 kfreebsd-i386 || 5 || 0 || 2 ||6 mips || 2 || 0 || 1 ||3 mipsel || 2 || 0 || 1 ||3 powerpc[1] || (1) || 0 || 2 || 2.5? s390x || 1 || 0 || 1 ||2 sparc || 1 || 0 || 0 ||1 ... Based on the number of porters, we are considering changing the current requirements of 5 DDs to better reflect the reality of the situation. We will follow up in a future bits on the changes. Thanks. You are welcome. :) I think it is disappointing to find that we may be dropping architectures where a significant amount of effort is available, simply because the volunteers don't have enough status - specifically, because of a lack of DDs. As mentioned we are debating whether the 5 DDs requirement still makes sense. Would you say that we should abolish the requirement for DD porters completely? I.e. Even if there are no (soon to be) DDs, we should consider the porter requirements fulfilled as long as they are enough active porters behind the port[0]? I'm keen that Debian should continue to support a wide range of architectures. Would it help if I, as a DD, volunteered to sponsor porter uploads for any architecture ? That is I guess I'm volunteering to become a new kind of person - a non-port-specific porter sponsor. I suppose that could help ports without a DD if we allowed such to be in testing. However, it is my understanding that our main issue with ports often is that they are not actively maintained (or appears to lack active maintenance). As an example I remember having received several complains from e.g. the GCC maintainers in regards to the state of gcc on various ports[1]. Here I would suspect a patch would be sufficient without needing to actually NMU gcc to get the fix in. There are also stuff like the port concerns from DSA that attention. Obviously I will review the debdiff etc. I'm an experienced C programmer with some background in C language lawyering and portability stuff, so I should usually be able to do a decent review of a patch even on an unfamiliar architecture. In fact, regardless of what the release team decide for the policy, I would be happy to sponsor porter uploads. Please just email me. Ian. :) ~Niels [0] Leaving the definition of active porter vaguely defined for now. [1] Obviously, I haven't been keeping track of them so I had to ask for a reminder. https://lists.debian.org/debian-release/2013/10/msg00710.html -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/526fdfe3.7060...@thykier.net
Re: Bits from the Release Team (Jessie freeze info)
Niels Thykier writes (Re: Bits from the Release Team (Jessie freeze info)): On 2013-10-29 16:05, Ian Jackson wrote: I'm keen that Debian should continue to support a wide range of architectures. Would it help if I, as a DD, volunteered to sponsor porter uploads for any architecture ? That is I guess I'm volunteering to become a new kind of person - a non-port-specific porter sponsor. ... I suppose that could help ports without a DD if we allowed such to be in testing. Indeed. However, it is my understanding that our main issue with ports often is that they are not actively maintained (or appears to lack active maintenance). Right. As mentioned we are debating whether the 5 DDs requirement still makes sense. Would you say that we should abolish the requirement for DD porters completely? I.e. Even if there are no (soon to be) DDs, we should consider the porter requirements fulfilled as long as they are enough active porters behind the port[0]? I don't have a good feel for the answer to that question. It's just that if it is the case that a problem with ports is the lack of specifically DDs, rather than porter effort in general, then sponsorship is an obvious way to solve that problem. If you feel that that's not really the main problem then a criterion which counts porters of any status would be better. (Mind you, I have my doubts about a process which counts people promising to do work - it sets up some rather unfortunate incentives. I guess it's easier to judge and more prospective than a process which attempts to gauge whether the work has been done well enough.) As an example I remember having received several complains from e.g. the GCC maintainers in regards to the state of gcc on various ports[1]. Here I would suspect a patch would be sufficient without needing to actually NMU gcc to get the fix in. There are also stuff like the port concerns from DSA that attention. Right. Thanks, Ian. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/21103.59120.410686.914...@chiark.greenend.org.uk
on bootstrapping ports (was: Re: Bits from the Release Team (Jessie freeze info))
Hi Peter, Quoting peter green (2013-10-27 01:11:24) Johannes Schauer wrote: Until these two issues are fixed we will not be able to get an algorithmic answer to the question of what constitutes the minimum required set of packages. There is also the complication of what I will call non-key self building compilers. fpc is an example Yes, this is also an issue and the two issues I mentioned are only a necessity but not a sufficiency for the whole bootstrap problem. In practice, it is of course not only needed to know that one must be able to build src:fpc without all the fp-* binaries (that's what my algorithms do right now) but also that this is really possible in practice. Since I only work with the algorithmic side of things I often forget to mention that one naturally needs more than correct meta data (dependency relationships) to make everything work :) You will find your example (fpc) in the section of Type 1 Self-Cycles on http://mister-muffin.de/bootstrap/stats/ together with other compilers like for haskell, sml or lisp. We call Type 1 Self-Cycles those where the source package directly build depends on a binary package it builds. Those are the most obvious self cycles and they are mostly made up of the non-key self building compilers as you call them. These are not needed to bootstrap the core of debian but if one wants to bootstrap all of debian they will need to be built. Indeed, none of the Type 1 Self-Cycles are needed to bootstrap the core of Debian. Unfortunately though, most of the Type 2 Self-Cycles are. You will find many surprising (at least to me) examples in the section of Type 2 Self-Cycles under the above link. We call Type 2 Self-Cycles those where the source package indirectly and strongly [1] build depends on a binary package it builds. They are hard to find because the only tool which is able to compute strong dependencies (afaik) is dose3 which is used by botch to do the required computations. [1] www.mancoosi.org/papers/esem-2009.pdf Surely every maintainer of source packages involved in a Type 1 Self-Cycle knows about this issue. Because Type 2 Self-Cycles are non-obvious we could in the future (once build profiles are available) embed this information in the pts for the relevant packages. On the other hand, there only exist a small number (26 for amd64) source packages involved in Type 2 Self-Cycles so it might be enough to just post priority wishlist bug reports for each of them. Since the only way to build them is with themselves they cannot be bootstrapped natively even with the help of build profiles. So the only way to bootstrap them is to either cross-build them or start with a binary from upstream. And even compilers which allow some way of bootstrapping them, do not have this process automated (ghc [2], mlton [3]). This is fine under the assumption that initial porting is rarely done and once it's done does not have to be repeated. But it does not allow continuous testing of bootstrappability of the whole archive. [2] http://ghc.haskell.org/trac/ghc/wiki/Building/Porting [3] http://mlton.org/PortingMLton cheers, josch -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131027083354.16796.89806@hoothoot
Re: Bits from the Release Team (Jessie freeze info)
Hi, On Mittwoch, 23. Oktober 2013, Stewart Smith wrote: Jenkins can have slaves on remote hosts, via SSH. It runs a small java app there, so as long as the arch has a JVM then you're pretty right. that JVM is not even needed, just schedule jobs via ssh and be done. cheers, Holger signature.asc Description: This is a digitally signed message part.
Re: Bits from the Release Team (Jessie freeze info)
Hi, (I was not able to find the debian-ports list on lists.debian.org (so I subscribed via email) did I just miss it?) Quoting Steven Chamberlain (2013-10-23 22:04:59) I had a play with the 'botch' tool (see description[1]) for determining build order when bootstrapping an architecture. botch author here. Just stumbled upon this already a few day old email in my inbox :) To start off with it determines a minimum required set of packages - you'd normally cross-compile those from another system. This set (see attached example list for kfreebsd-amd64 wheezy) looks to me like what constitutes the 'toolchain'. This minimum set of packages which has to be cross compiled (because no binary package of the target architecture exists at this point) is what we call the minimal native build system (the name is misleading as disjunct dependencies make different choices of this set possible). Currently it is not possible to present a correct selection of source packages which have to be cross compiled to produce the minimal build system. What we currently do is to just do: grep-dctrl -X \( -FPackage build-essential --or -FPackage debhelper --or -FEssential yes) and assume that the resulting list of packages (the one you attached to your last email) is cross compilable from nothing. This is of course not the case in practice but a formal analysis is not possible yet. This is because multiarch annotations are missing in some packages and because of the problem of how to handle source packages directly depending on gcc, g++, binutils etc in the cross compilation case. While the first one is easy to fix there doesnt exist a solution for the second one yet. Build profiles would be able to solve the second problem. Until these two issues are fixed we will not be able to get an algorithmic answer to the question of what constitutes the minimum required set of packages. On the other hand wookey did lots of ubuntu crossing and identified that with just (I think it was) a dozen modified packages (reducing their build dependencies to break cross build dependency cycles) he was able to cross build all of these packages: http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html So while an automated analysis is not possible right now, it also does not seem to be necessary to have this automated. Having the to-be-crossed selection of packages retrieved automatically becomes more interesting as more source packages are known to be cross-compilable including all their required recursive build dependencies. The list will be different for each port, and change over time. This example included freebsd-libs, freebsd-utils and kfreebsd-kernel-headers but of course others won't. Thanks for trying out botch for kfreebsd :) AIUI those packages should be able to rebuild each of themselves without any other dependencies. Should but unfortunately they are not :( In fact, to nativel rebuild the minimal build system for amd64 (just essential:yes + build-essential + debhelper) one needs to be able to compile 383 source packages (excluding the source packages in the minimal build system itself). This is as of debconf13 when I last ran those scripts. You can look at the numbers here as well: http://mister-muffin.de/bootstrap/stats/ These 383 source packages include ugly ones like iceweasel, nautilus, webkit, network-manager, mysql, kde4libs which as you can imagine draw in half of what makes a modern desktop system and thus might naively have been dismissed as non-essential for the bootstrapping purpose at all. But of course this list will be different between arches. I think doing that regularly would be a good health check for a port's toolchain. Probably these packages would be the focus of the reproducible-builds project, at least from a security point-of-view. Any differences in output of subsequent builds are of interest, and would potentially reveal when significant changes or bugs were introduced too. Being able to use botch to automatically bootstrap all arches from scratch has always been one of botch's goals. Unfortunately the build profile discussion is holding up the implementation of this in practice but guillem promised to look into this for dpkg 1.17.2. cheers, josch -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131026213931.16796.6@hoothoot
Re: Bits from the Release Team (Jessie freeze info)
Johannes Schauer j.scha...@email.de (2013-10-26): (I was not able to find the debian-ports list on lists.debian.org (so I subscribed via email) did I just miss it?) Dead list: http://lists.debian.org/debian-ports/ AFAICT it's now an alias for all debian-$port lists. Mraw, KiBi. signature.asc Description: Digital signature
Re: Bits from the Release Team (Jessie freeze info)
Johannes Schauer wrote: Until these two issues are fixed we will not be able to get an algorithmic answer to the question of what constitutes the minimum required set of packages. There is also the complication of what I will call non-key self building compilers. fpc is an example These are not needed to bootstrap the core of debian but if one wants to bootstrap all of debian they will need to be built. Since the only way to build them is with themselves they cannot be bootstrapped natively even with the help of build profiles. So the only way to bootstrap them is to either cross-build them or start with a binary from upstream. -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/526c4c1c.3060...@p10link.net
Re: Bits from the Release Team (Jessie freeze info)
On Wed, Oct 23, 2013 at 12:36 AM, Stewart Smith
stew...@flamingspork.com wrote:
Jenkins can have slaves on remote hosts, via SSH. It runs a small java
app there, so as long as the arch has a JVM then you're pretty right.
For whatever definition of small. I've seen it consuming 1 GiB of memory...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/camuhmdvz3jwmdujds762z-cnhv4z5c9wuuf5rkanarqbsdx...@mail.gmail.com
Re: Bits from the Release Team (Jessie freeze info)
I run Jenkins at my job. Small is around 256mb. Plus the Jenkins server can
sit on a high-memory machine and the agent just sit on a 68k box doing
builds. Small is like 64M ram. You Amiga/Atari guys seem to have oodles of
ram to work with Lol.
On Oct 23, 2013 2:45 AM, Geert Uytterhoeven ge...@linux-m68k.org wrote:
On Wed, Oct 23, 2013 at 12:36 AM, Stewart Smith
stew...@flamingspork.com wrote:
Jenkins can have slaves on remote hosts, via SSH. It runs a small java
app there, so as long as the arch has a JVM then you're pretty right.
For whatever definition of small. I've seen it consuming 1 GiB of memory...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker.
But
when I'm talking to journalists I just say programmer or something like
that.
-- Linus Torvalds
--
To UNSUBSCRIBE, email to debian-68k-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact
listmas...@lists.debian.org
Archive:
http://lists.debian.org/camuhmdvz3jwmdujds762z-cnhv4z5c9wuuf5rkanarqbsdx...@mail.gmail.com
Re: Bits from the Release Team (Jessie freeze info)
Small is 64m ram not 256m. I just woke up and was catching up on things. My
apologies.
On Oct 23, 2013 7:20 AM, Britt Dodd brittman...@gmail.com wrote:
I run Jenkins at my job. Small is around 256mb. Plus the Jenkins server
can sit on a high-memory machine and the agent just sit on a 68k box doing
builds. Small is like 64M ram. You Amiga/Atari guys seem to have oodles of
ram to work with Lol.
On Oct 23, 2013 2:45 AM, Geert Uytterhoeven ge...@linux-m68k.org
wrote:
On Wed, Oct 23, 2013 at 12:36 AM, Stewart Smith
stew...@flamingspork.com wrote:
Jenkins can have slaves on remote hosts, via SSH. It runs a small java
app there, so as long as the arch has a JVM then you're pretty right.
For whatever definition of small. I've seen it consuming 1 GiB of
memory...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker.
But
when I'm talking to journalists I just say programmer or something like
that.
-- Linus Torvalds
--
To UNSUBSCRIBE, email to debian-68k-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact
listmas...@lists.debian.org
Archive:
http://lists.debian.org/camuhmdvz3jwmdujds762z-cnhv4z5c9wuuf5rkanarqbsdx...@mail.gmail.com
Re: Bits from the Release Team (Jessie freeze info)
On 22/10/13 23:36, Stewart Smith wrote: Jenkins can have slaves on remote hosts, via SSH. It runs a small java app there, so as long as the arch has a JVM then you're pretty right. That may be useful to set up on some arches, for things where Jenkins needs direct control over CPU-intensive tasks. Building and testing d-i, for example. But for this, I would imagine only the test suite needs to run over SSH, and the master Jenkins instance just has to process the output? For the proposed test suite to be as accessible as possible to a new/upcoming port, the barrier to using it ought to be very low. A working JVM is quite a lot to ask, the current openjdk-7 is not even built for mipsel in more. mipsel buildds and porterboxes had only 1GB RAM maximum until now, and that is heavily used already for their current tasks. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: OpenPGP digital signature
Re: Bits from the Release Team (Jessie freeze info)
Geert Uytterhoeven ge...@linux-m68k.org writes: On Wed, Oct 23, 2013 at 12:36 AM, Stewart Smith stew...@flamingspork.com wrote: Jenkins can have slaves on remote hosts, via SSH. It runs a small java app there, so as long as the arch has a JVM then you're pretty right. For whatever definition of small. I've seen it consuming 1 GiB of memory... with 'm68k' in your email address your definition of small is likely much different than my many years in large scale databases small :) That being said... I haven't recently seen a slave jenkins java process more than one or two hundred mb. This is (of course) absolutely insane, as is the 4-6GB jenkins master process. However, dollars per GB of memory is suitably low that it's not worth me fixing it, instead it just sits there annoying me as it could undoubtedly be better -- Stewart Smith pgpoK_F3wP8Yb.pgp Description: PGP signature
Re: Bits from the Release Team (Jessie freeze info)
On 23/10/13 12:55, Stewart Smith wrote: Geert Uytterhoeven ge...@linux-m68k.org writes: On Wed, Oct 23, 2013 at 12:36 AM, Stewart Smith stew...@flamingspork.com wrote: Jenkins can have slaves on remote hosts, via SSH. It runs a small java app there, so as long as the arch has a JVM then you're pretty right. For whatever definition of small. I've seen it consuming 1 GiB of memory... with 'm68k' in your email address your definition of small is likely much different than my many years in large scale databases small :) Come to think of it, it must take a day or more for m68k to rebuild eglibc. This is a more serious problem than resources needed by Jenkins. We can't ask them to rebuild their entire toolchain each night! For the goal of software freedom, it shouldn't be too difficult for anyone to do that, though. We should be trying to make it easier. Maybe it would be permissible for the toolchain test suite to run on a faster platform, and cross-compile, or use any sort of emulation available in Debian free packages. If it were technically feasible for each Debian port to rebuild its toolchain and some essential packages, at least once per week, I think that would be an accomplishment. And the smaller the initial set of packages required to boostrap the process, the better. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: OpenPGP digital signature
Re: Bits from the Release Team (Jessie freeze info)
Steven Chamberlain dixit: Come to think of it, it must take a day or more for m68k to rebuild eglibc. This is a more serious problem than resources needed by Kernel takes a day now (on the fastest VMs), eglibc 3 days, gcc 5 days (since gcj got folded into it; add another day or so once gnat will also be folded). Jenkins. We can't ask them to rebuild their entire toolchain each night! No OpenJDK either (can probably be fixed, but zero is sloow). Additionally, with only, say, 256 or 768 MiB physmem, running additional software on the buildds is something you do not want, considering how much RAM building some stuff takes (I had to use about 5 GiB of swap to link Webkit, and imagine just how much paging that involves, also in terms of time). Building GCC isn’t exactly resource-saving. (Even running apt/dpkg isn’t due to the sheer size of the archive, though Guillem kindly reduced memory usage in the upcoming dpkg upload.) I think with my “better SCC proposal” we could have a sliding scale for this, but I’d oppose using something OpenJDK-based for that (think of mipsel, too). Especially as simple mksh scripts would take care of the job too (including CGI for web export ;). bye, //mirabilos -- Solange man keine schmutzigen Tricks macht, und ich meine *wirklich* schmutzige Tricks, wie bei einer doppelt verketteten Liste beide Pointer XORen und in nur einem Word speichern, funktioniert Boehm ganz hervorragend. -- Andreas Bogk über boehm-gc in d.a.s.r -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pine.bsm.4.64l.1310231242320.29...@herc.mirbsd.org
Re: Bits from the Release Team (Jessie freeze info)
Hi Niels, This was quite interesting as it seems to tie in with some other projects that are already being pursued... On 21/10/13 16:42, Niels Thykier wrote: I would love for us to have an automated system to give us a weather-report on the toolchain for each architecture. It would be nice both for us to see how ports are doing and for porters to spot and fix problems early. That sounds a lot like the purpose of Jenkins[0], but I'm not sure if it's exactly suitable. It seems a little heavy, that someone could more easily be able to script some cron jobs for a task than learn how to use it. And Jenkins isn't available yet on all arches; some ports may not have hardware powerful enough to run it. Maybe that doesn't matter - a single Jenkins instance might be able to launch jobs via remote shells to other boxes, running the actual test suite there, or maybe just to fetch, analyse and report on the resulting log files. Ideally I'd like to see a set of command-line scripts runnable either from cron, or maybe someday by Jenkins jobs if someone wants to set that up. And packaged up for people to use at home! [0]: http://jenkins.debian.net/ Which implies a set of packages being the current version of the overwhelming part of the archive plus all of d-i. However, that is not something you just build, so having a smaller set as a basic test would probably be way more useful. I am not aware of such a basic test set, so feel free to propose one. Some people have been trying to identify small sets of essential packages already, in the context of bootstrapping an architecture[1]. I wonder if that's likely to overlap with this? It encompasses toolchain and essential arch-specific packages. I imagine a healthy port should be able to bootstrap itself with only current package versions. If this was being tested regularly it could let porters know if circular dependencies are introduced, for example. [1]: https://wiki.debian.org/DebianBootstrap#Toolchain I would maybe take that a little further and say that a system is only stable if it can bootstrap itself, install and boot into the resulting system, and repeat the whole process again... I like the toolchain nightly thing as well. I don't think it is required, but it sounds like the kind of thing that would help people spot issues sooner rather than later! And this also ties in with the reproducible-builds project[2] (not sure if you were hinting at that before). The 'toolchain' is of particular concern because the security of the whole system depends on it. Differences in the output of builds needs to be avoided, or otherwise explained. It would help greatly if there were frequent builds happening so we could see unexpected changes occurring. [2]: https://wiki.debian.org/ReproducibleBuilds So if something can make something that fulfills all the above goals it would certainly be beneficial :) Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5266df9d.9040...@pyro.eu.org
Re: Bits from the Release Team (Jessie freeze info)
Steven Chamberlain ste...@pyro.eu.org writes: On 21/10/13 16:42, Niels Thykier wrote: I would love for us to have an automated system to give us a weather-report on the toolchain for each architecture. It would be nice both for us to see how ports are doing and for porters to spot and fix problems early. That sounds a lot like the purpose of Jenkins[0], but I'm not sure if it's exactly suitable. It seems a little heavy, that someone could more easily be able to script some cron jobs for a task than learn how to use it. It's actually a pretty low barrier to entry, if you know what commands you need to run, it's pretty easy to get started with jenkins (create job, have it execute shell commands, write shell in box, hit build). I'd say that it's about 10 times more likely you'll get it right in Jenkins before you get it right in cron. And Jenkins isn't available yet on all arches; some ports may not have hardware powerful enough to run it. Maybe that doesn't matter - a single Jenkins instance might be able to launch jobs via remote shells to other boxes, running the actual test suite there, or maybe just to fetch, analyse and report on the resulting log files. Jenkins can have slaves on remote hosts, via SSH. It runs a small java app there, so as long as the arch has a JVM then you're pretty right. -- Stewart Smith pgpNXu03LMv_e.pgp Description: PGP signature
Re: Bits from the Release Team (Jessie freeze info)
On 2013-10-19 16:38, Jeremiah C. Foster wrote: Hello, On Sun, Oct 13, 2013 at 05:01:31PM +0200, Niels Thykier wrote: [snip freeze policy] Hi, I s/-arm/-ports/'ed the CC, since I figured the rest of the porters would find the answer equally interesting. Results of porter roll-call === [...] That said, we would like to encourage porters behind all ports to ensure that the toolchain is up to date and working. We are aware of at least gcc on mips having its test suite disabled[GCC]. Other ports may suffer from similar issues and we hope to have those resolved sooner rather than later. We are currently waiting for the gcc maintainers to compile a list of such issues. So I can extrapolate from this that ensuring that the toolchain is up to date and working is a key activity of a porter. Yes; build-essential being broken is obviously a problem. But also having the same default compiler on all architectures is also desired. If my assumption is correct, is there a complete definition of the toolchain as we see it in Debian that a porter might reasonably be expected to use to do thier porting? I do not have an complete list of packages, although it will definitely include build-essential. My intuition is that toolchain should include any compiler used by packages on that architecture[1] (e.g. if the arch has built haskell packages, it should have a working haskell compiler as well). But as said, that is my personally view and not an official statement. In addition, I wonder if there is a way to report the status of the toolchain and what sort of expectations are there around up to date? I would love for us to have an automated system to give us a weather-report on the toolchain for each architecture. It would be nice both for us to see how ports are doing and for porters to spot and fix problems early. As for up-to-date, I don't have a complete answer here. I seem to remember the GCC maintainers being frustrated at having to maintain gcc-4.6 (it is apparently still default for some architectures) despite gcc-4.8 being the latest stable release. Is it expected to build Debian toolchain nightly and run a specific test suite? Is the expectation that one uses pbuilder and builds a set of packages? What we got in the policy so far[2]: Installer: The architecture must have a working,tested installer. [...] Archive coverage: The architecture needs to have successfully compiled the current version of the overwhelming part of the archive [...] Which implies a set of packages being the current version of the overwhelming part of the archive plus all of d-i. However, that is not something you just build, so having a smaller set as a basic test would probably be way more useful. I am not aware of such a basic test set, so feel free to propose one. I like the toolchain nightly thing as well. I don't think it is required, but it sounds like the kind of thing that would help people spot issues sooner rather than later! Perhaps this is outlined on the wiki somewhere and if not perhaps it ought to be? Regards, Jeremiah Having documentation on it would definitely be a good thing. For actual requirements, we should add them to the policy[2], but having a wiki-page of recommended porter practises/tests would probably be a nice addition too. ~Niels [1] My rationale for this is that we would like to be able to rebuild/reproduce builds, which would require a working compiler. [2] http://release.debian.org/jessie/arch_policy.html -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52654b6d.9020...@thykier.net
