Re: Re-evaluating architecture inclusion in unstable/experimental
On Fri, Oct 12, 2018 at 11:13:30PM +, Thorsten Glaser wrote: As a (lapsed) porter for x32 here in Debian, I don't think it's worth keeping. Despite all the touted benefits, no one uses the arch. There's a That’s wrong. I’ve crossgraded my desktop at work to x32, with M-A to i386 for a continuously smaller-becoming number of things. Strictly it's wrong, but he likely meant "nearly no-one". Thus: I propose to drop x32, and reallocate your tuits to other archs. That would inconvenience me quite a bit, *and* tell me that all the effort I invested, all the time spent hacking, debugging, learning and teaching, is worth nothing. The same is true for any dropped port, but beware of the sunk cost fallacy. -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland ⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net ⠈⠳⣄ Please do not CC me, I am subscribed to the list.
Re: Re-evaluating architecture inclusion in unstable/experimental
Eek! I only saw this by accident, as I’m also not subscribed to d-devel. >On Sat, Sep 29, 2018 at 07:48:09AM +0100, Ben Hutchings wrote: >> - x32 has 64-bit time_t and that never worked quite right. This may >> have finally been fixed this year by Arnd Bergmann's work, but I'm >> not sure. They have. Also, most of the ioctls. His work is important for other architectures, both future ones and current-ones-which- will-need-a-new-ABI-in-2038 ones, and so it’s good that this is showcased on x32. >> - There have been several regressions specific to x32, some of which >> stuck around for a year or so before someone and fixed them. I’m trying to fix things as I go along. I’m not always skilled enough to fix everything, though, nor are maintainers applying patches, or upstreams. I know Adrian also does, and occasionally others. >As a (lapsed) porter for x32 here in Debian, I don't think it's worth >keeping. Despite all the touted benefits, no one uses the arch. There's a That’s wrong. I’ve crossgraded my desktop at work to x32, with M-A to i386 for a continuously smaller-becoming number of things. >Thus: I propose to drop x32, and reallocate your tuits to other archs. That would inconvenience me quite a bit, *and* tell me that all the effort I invested, all the time spent hacking, debugging, learning and teaching, is worth nothing. It’s also surprising how this has not found its way to me on the ports mailing list, which I’m fairly sure I’m subscribed to. [ m68k ] > But the available hardware is either too slow to be useful, or only > available through crowdfunding (maybe, eventually). In quite a *lot* of scenarios, slowness of hardware isn’t the most important criterion. People are willing to take it, as the gains outweigh it, for a number of applications. Also, having a lot of ports helps. Just this week, I found a stack overflow in dietlibc because the testsuite failed on alpha. It was an integer array overwriting half the return address, found only by sheer luck of not only the actual stack layout of the function working for us but also its architectural constraints, such as general memory and stack layout. (I could tell of many more such cases, where porting found bugs in code, but most of them are quite some time ago.) bye, //mirabilos -- When he found out that the m68k port was in a pretty bad shape, he did not, like many before him, shrug and move on; instead, he took it upon himself to start compiling things, just so he could compile his shell. How's that for dedication. -- Wouter, about my Debian/m68k revival
Re: Re-evaluating architecture inclusion in unstable/experimental
On 03.10.2018 18:01, John Paul Adrian Glaubitz wrote: >> For s390x I can say that the port was driven without any commercial >> interest on both Aurelien's and my side > The question is though: Is there quantifiable amount of users that is > running Debian on such big iron instead of one of the Linux enterprise > distributions on the market? If the argument is about maintenance burden, > then does it justify to support Debian on s390x when the number of users > is small? And, if yes, why does that not apply to ppc64, for example? > (I would mention sparc64 here as well, but there is actually a valid > blocker which is the lack of supply of new hardware for DSA). I cannot speak to ppc64. ppc64el is useful as I'm told POWER can be competitive to Intel/AMD-based services. But I don't know how many users would run Debian. For s390x, IBM does not publicly admit that there are people running Debian, but there are a few. Almost all of them turn popcon off - most of the VMs can't talk to the internet. Of course I don't know if the availability of Ubuntu significantly changed that. They were able to invest much more time into polishing the port and most people just want some kind of Debian derivative. Historically the base system has been very well maintained by IBM, though. So the effort to keep it running has been relatively small. This recently changed somewhat, given that the primary focus is on enterprise distributions, in that stuff like Javascript interpreters don't work well. Essentially it boils down to server workloads that companies need to run, so as Docker and Go became popular, IBM implemented support for it. The same happened for v8 as used by Node. OpenJDK 9 finally comes with a JIT, so you don't have to use IBM Java anymore. And to IBM's credit, they even contributed some bits back to d-i. Although some of those still await testing and merging. The Ubuntu changes did not flow back / were not mergable as-is into Debian. It's always a tradeoff between how much work is necessary to keep the port alive and how many people use it. As long as the port keeps itself working, that's sort of fine in my experience. Once you need to sheperd a lot of things that all break (like the MIPSens historically had to, even working around broken CPUs) or need to deal with 2 GB virtual address space or don't have modern languages like Go or Rust around, it quickly approaches the point where it's not worth it anymore. Kind regards Philipp Kern
Re: Re-evaluating architecture inclusion in unstable/experimental
Hi Philipp! On 10/3/18 4:29 PM, Philipp Kern wrote: > Please excuse my ignorance, but which architecture do we still have with > 2 GiB address space? The main point of removing s390 was that this was > unsustainable. The 32-bit MIPS architectures have this limitation which causes various build issues up to the point that maintainers have to reduce optimization levels for the C/C++ compiler to be able to build a package. Another issue is that the Rust compiler, despite being fully supported on 32-bit MIPS, cannot be built natively there because the build process runs out of memory at some point. >> I have seen IBM people on multiple occasions in various upstream >> projects trying to remove code for older POWER targets because >> they insisted anything below POWER8 is not supported anymore. In >> some cases like Golang with success [1]. > > Yeah, IBM behavior has been incredibly frustrating here on the System z > side, too. Essentially they end up actively removing support for > anything they don't support anymore. Somewhat relieves me to hear that I'm not the only one running into this problem. I find it frustrating though since it leaves a bad impression on what attitude IBM has towards open source and the community. > To some degree I understand this behavior: It's a real relieve to not > need to support something old and crufty when you're the engineer on the > other side having to do that. Even when such support is contributed, > someone needs to keep it working and they won't keep old hardware around > for that. Sure, I understand that. But in the case of Golang, the necessary extra code paths are really manageable. In fact, have my own tree of Golang where I reverted the changes which dropped POWER5 support and I keep rebasing this tree from time to time and it still works fine. It does work without problems on the x86 side with the kernel and the toolchain still supporting rather old hardware. POWER5 isn't that old. > But it has very awkward implications on the people that still have that > hardware for one reason or another and don't actually rely on a support > contract. My main argument is that if there are still a reasonable amount of users and there are still enough people to help maintain a port, it should not removed just because it's old. In the end, software is written to be useful to users and not for the sake of the people writing it. > For s390x I can say that the port was driven without any commercial > interest on both Aurelien's and my side The question is though: Is there quantifiable amount of users that is running Debian on such big iron instead of one of the Linux enterprise distributions on the market? If the argument is about maintenance burden, then does it justify to support Debian on s390x when the number of users is small? And, if yes, why does that not apply to ppc64, for example? (I would mention sparc64 here as well, but there is actually a valid blocker which is the lack of supply of new hardware for DSA). Thanks for the insight! Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Re-evaluating architecture inclusion in unstable/experimental
ср, 3 окт. 2018 г. в 17:48, Jonathan Dowland : > > On Sat, Sep 29, 2018 at 05:05:17PM +0200, John Paul Adrian Glaubitz > wrote: > >Well, I have had people from IBM fix 32-bit PowerPC code. There is > >naturally more involvement behind the 64-bit stuff because that's where > >the commercial interests are. > > The kernel itself dropped 32bit powerpc support years ago, IIRC. Hmm, no. ls -l /arch/powerpc/platforms/ shows all of them. -- With best wishes Dmitry
Re: Re-evaluating architecture inclusion in unstable/experimental
On Sat, Sep 29, 2018 at 05:05:17PM +0200, John Paul Adrian Glaubitz wrote: Well, I have had people from IBM fix 32-bit PowerPC code. There is naturally more involvement behind the 64-bit stuff because that's where the commercial interests are. The kernel itself dropped 32bit powerpc support years ago, IIRC. -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland ⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net ⠈⠳⣄ Please do not CC me, I am subscribed to the list.
Re: Re-evaluating architecture inclusion in unstable/experimental
On 29.09.2018 00:30, John Paul Adrian Glaubitz wrote: > On 9/28/18 11:26 PM, Adam D. Barratt wrote: >> On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: >>> So, it's not always a purely technical decision whether a port >>> remains a release architecture. It's also often highly political and >>> somehow also influenced by commercial entities. >> Please don't make implications like that unless you can back them up. > Well, I cannot prove it. But when I see that we have ports as release > architectures with hardware where atomics in hardware don't even work > correctly and the virtual address space is limited to 2 GiB per process > while on the other hand perfectly healthy and maintained ports like > powerpc and ppc64 which have actually a measurable userbase and interest > in the community are axed or barred from being a release architecture, > then I have my doubts that those decisions aren't also driven by > commercial interests or politics. Please excuse my ignorance, but which architecture do we still have with 2 GiB address space? The main point of removing s390 was that this was unsustainable. > I have seen IBM people on multiple occasions in various upstream > projects trying to remove code for older POWER targets because > they insisted anything below POWER8 is not supported anymore. In > some cases like Golang with success [1]. Yeah, IBM behavior has been incredibly frustrating here on the System z side, too. Essentially they end up actively removing support for anything they don't support anymore. To some degree I understand this behavior: It's a real relieve to not need to support something old and crufty when you're the engineer on the other side having to do that. Even when such support is contributed, someone needs to keep it working and they won't keep old hardware around for that. But it has very awkward implications on the people that still have that hardware for one reason or another and don't actually rely on a support contract. For s390x I can say that the port was driven without any commercial interest on both Aurelien's and my side. Kind regards Philipp Kern
Re: Re-evaluating architecture inclusion in unstable/experimental
On Sat, 2018-09-29 at 17:05 +0200, John Paul Adrian Glaubitz wrote: > On 9/29/18 8:48 AM, Ben Hutchings wrote: > > On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: > > [...] > > > Furthermore, several of the ports are in very healthy condition and > > > even surpass some release architectures. The powerpc and ppc64 ports, > > > for example, build more packages than any of the mips* ports. > > > > I would be very happy to never have to wait for MIPS builds again. > > I don't have a problem with waiting for slow machines. I just think this > shows that the decisions what becomes a release architecture and what > doesn't isn't purely meritocratic. It's been a long time since Debian could run infrastructure on donated hardware. We need hardware that is reliable and that can be quickly replaced when (not if) it fails. So commercial availability is, and should continue to be, a release qualification. It is also unacceptable that the release of an urgent security update should have to wait a long time for builds. So speed matters. [...] > I think the problem that I have is that the release qualifications are > not practical in the sense that they don't necessarily meet our users > expectations. As I said, I think a tier-based system would be more > practical as it would allow ports to be degraded more gracefully instead > of just kicking them out like it was done with 32-Bit PowerPC. [...] Could you be more specific? Ben. -- Ben Hutchings For every action, there is an equal and opposite criticism. - Harrison signature.asc Description: This is a digitally signed message part
Re: Re-evaluating architecture inclusion in unstable/experimental
On Sat, Sep 29, 2018 at 05:11:31PM +0200, John Paul Adrian Glaubitz wrote:
> On 9/29/18 12:02 PM, Adam Borowski wrote:
> > With primary use cases being hosting of multiple containers, and/or running
> > a large number-crunching cluster. Sacrificing lots of human hassle for a 7%
> > speed boost makes you a "Gentoo ricer"[1] on a desktop/laptop, but could be
> > a nice thing on a $1M cluster.
>
> It can be up to 50% which is why Intel's own C/C++ compiler actually provides
> an option to automatically generate x86_64 code with 32-bit pointers:
>
> https://software.intel.com/en-us/node/523141
Yeah, but that 7% figure I noted is an already very optimistic number for a
typical task. A good part of programs (like shell scripts) show 0%, and
there are some that are actually slower (impossible in theory, but there are
missed optimizations which work on mainstream ABIs).
So an user would have to evaluate every single use case to see whether it's
worth the effort. In the only case I know where such an evaluation was
meant for a real cluster, the guy who asked me decided that "no".
> > Thus: I propose to drop x32, and reallocate your tuits to other archs.
>
> Thanks, but x32 doesn't really use any resources and as long as the stuff
> works, I don't see a reason to keep people from using it. If Linux upstream
> decides to ax it, we will ax it here as well. Until then, it allows doing
> some extra CI for package builds and edge cases.
Right -- as I haven't done any x32 fixes recently, it's not up to me to
decide. I'm just voicing me having been an active porter in the past but
then getting disheartened by the lack of use.
You might also want to take the domain debian-x32.org from me. Not sure how
useful jessie era packages (including d-i which never got upstreamed) will
be after buster...
On the other hand, x32 is not a shut case, hardware that can run it being
ubiquitous. Archs like sh4 whose entire reason to exist ("a fully free
architecture") got eaten by riscv and that have no meaningful presence in
the wild, on the other hand...
Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.
Re: Re-evaluating architecture inclusion in unstable/experimental
On 9/29/18 12:02 PM, Adam Borowski wrote: > With primary use cases being hosting of multiple containers, and/or running > a large number-crunching cluster. Sacrificing lots of human hassle for a 7% > speed boost makes you a "Gentoo ricer"[1] on a desktop/laptop, but could be > a nice thing on a $1M cluster. It can be up to 50% which is why Intel's own C/C++ compiler actually provides an option to automatically generate x86_64 code with 32-bit pointers: > https://software.intel.com/en-us/node/523141 > But, recently someone approached me with exactly that use case. He also > required jessie, which happens to be the only release I have a consistent > set of packages for. The evaluation result was: nope. > > A short while ago Linus pondered dropping x32, as it complicates kernel > syscall code. The only proof he was told that "it's still in use" was my > https://debian-x32.org -- yet that site hasn't been updated since jessie. I was actually on this discussion and I provided Linus with an up-to-date x32 chroot to test the changes which apparently worked. > Thus: I propose to drop x32, and reallocate your tuits to other archs. Thanks, but x32 doesn't really use any resources and as long as the stuff works, I don't see a reason to keep people from using it. If Linux upstream decides to ax it, we will ax it here as well. Until then, it allows doing some extra CI for package builds and edge cases. And as long people invest resources in ports like Hurd and kFreeBSD, x32 is the lesser burden to be worried about. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Re-evaluating architecture inclusion in unstable/experimental
On 9/29/18 8:48 AM, Ben Hutchings wrote: > On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: > [...] >> Furthermore, several of the ports are in very healthy condition and >> even surpass some release architectures. The powerpc and ppc64 ports, >> for example, build more packages than any of the mips* ports. > > I would be very happy to never have to wait for MIPS builds again. I don't have a problem with waiting for slow machines. I just think this shows that the decisions what becomes a release architecture and what doesn't isn't purely meritocratic. >> As for the kernel maintenance, except for Alpha, all the ports have >> at least one kernel maintainer: >> >> * hppa: actively maintained by two people who also maintain the toolchain > > But the hardware is EOL since 2008. Well, the hardware is usually build like tanks and if people keep using it and others keep maintaining it, why throw something out when it's still working? SAP is still maintaining and supporting JDK on PA-RISC and IA64 for paying customers, FWIW. >> * ia64: maintained by one paid developer at Intel > > To the extent of 2 whole commits this year! I'm pretty sure this is > not actually his job any more. Jason Duerstock talked to him and he said, he's still doing this as a job at Intel because basically no one told him to stop doing it. >> * m68k: maintained by multiple independent kernel maintainers, at least >> five people involved upstream; port is also getting new drivers > > But the available hardware is either too slow to be useful, or only > available through crowdfunding (maybe, eventually). Yes, this is being worked on. There are FPGA-based implementations which allow to boost Amiga and Atari hardware to considerably higher speeds. It's not done on a large commercial scale, so things take longer to develop. But people are working on it and bugs get fixed, code maintained and drivers added. >> * powerpc/ppc64: maintained mostly by IBM people > > IBM looks after 64-bit configurations, so ppc64 is covered but not > powerpc. Well, I have had people from IBM fix 32-bit PowerPC code. There is naturally more involvement behind the 64-bit stuff because that's where the commercial interests are. >> * sh4: maintained by two kernel maintainers > > That may be, but the Debian port isn't stable enough to *build* a > kernel: https://buildd.debian.org/status/logs.php?pkg=linux=sh4 That's a compiler bug, I know. >> * sparc64: used to be maintained by a team at Oracle but Oracle recently >> changed their mind about Linux on SPARC; now it's back to >> David Miller and some additional volunteers > > Oracle laid off their SPARC team. Fujitsu has another SPARC processor > in development, but only for supercomputers (and they seem to be > switching to Aarch64 later). So we can expect this architecture to > slowly fade away now. What Oracle is really doing and planning isn't really clear. I have talked to multiple people at Oracle directly and the consensus is that there isn't a consensus what is going to happen next. But again, this just supports my point that the decisions which ports become release architectures is mostly driven by commercial interests and not whether there are users in the community. Or is there actually a large community of POWER8 and z-Series Debian users? >> * x32: maintained by the people who maintain the x86 port of the kernel > > H. Peter Anvin did the original port in 2012, but so far as I can see > none of the x86 maintainers have done any development on it since then. > And yes, development work has been needed: > > - x32 has 64-bit time_t and that never worked quite right. This may > have finally been fixed this year by Arnd Bergmann's work, but I'm > not sure. > - syscall restart was broken until 2015 when someone actually tested > it. > - There have been several regressions specific to x32, some of which > stuck around for a year or so before someone and fixed them. > > [...] > > So, by all means have fun working on these ports, but aside from ppc64 > I don't see any prospect of them meeting release qualifications. I think the problem that I have is that the release qualifications are not practical in the sense that they don't necessarily meet our users expectations. As I said, I think a tier-based system would be more practical as it would allow ports to be degraded more gracefully instead of just kicking them out like it was done with 32-Bit PowerPC. I don't have a proper concept yet, but I think the OBS approach is more versatile, for example. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Re-evaluating architecture inclusion in unstable/experimental
[trimmed cross-post] On Sat, Sep 29, 2018 at 07:48:09AM +0100, Ben Hutchings wrote: > On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: > > * x32: maintained by the people who maintain the x86 port of the kernel > > H. Peter Anvin did the original port in 2012, but so far as I can see > none of the x86 maintainers have done any development on it since then. > And yes, development work has been needed: > > - x32 has 64-bit time_t and that never worked quite right. This may > have finally been fixed this year by Arnd Bergmann's work, but I'm > not sure. > - syscall restart was broken until 2015 when someone actually tested > it. > - There have been several regressions specific to x32, some of which > stuck around for a year or so before someone and fixed them. As a (lapsed) porter for x32 here in Debian, I don't think it's worth keeping. Despite all the touted benefits, no one uses the arch. There's a big difference between popcon 8 for an arch requiring obscure hardware or a FPGA, vs popcon that low for hardware everyone of us owns. Unlike kfreebsd-[x86] you can even conveniently run x32 in a chroot or lxc, yet no one does. The advertised benefits of the port are: * memory savings * a few percent speed gain over amd64 With primary use cases being hosting of multiple containers, and/or running a large number-crunching cluster. Sacrificing lots of human hassle for a 7% speed boost makes you a "Gentoo ricer"[1] on a desktop/laptop, but could be a nice thing on a $1M cluster. But, recently someone approached me with exactly that use case. He also required jessie, which happens to be the only release I have a consistent set of packages for. The evaluation result was: nope. A short while ago Linus pondered dropping x32, as it complicates kernel syscall code. The only proof he was told that "it's still in use" was my https://debian-x32.org -- yet that site hasn't been updated since jessie. Thus: I propose to drop x32, and reallocate your tuits to other archs. Meow! [1]. No offense for real Gentoo users, I mean the stereotype. -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary, ⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex, ⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.
Re: Re-evaluating architecture inclusion in unstable/experimental
On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: [...] > Furthermore, several of the ports are in very healthy condition and > even surpass some release architectures. The powerpc and ppc64 ports, > for example, build more packages than any of the mips* ports. I would be very happy to never have to wait for MIPS builds again. > As for the kernel maintenance, except for Alpha, all the ports have > at least one kernel maintainer: > > * hppa: actively maintained by two people who also maintain the toolchain But the hardware is EOL since 2008. > * ia64: maintained by one paid developer at Intel To the extent of 2 whole commits this year! I'm pretty sure this is not actually his job any more. > * m68k: maintained by multiple independent kernel maintainers, at least > five people involved upstream; port is also getting new drivers But the available hardware is either too slow to be useful, or only available through crowdfunding (maybe, eventually). > * powerpc/ppc64: maintained mostly by IBM people IBM looks after 64-bit configurations, so ppc64 is covered but not powerpc. > * sh4: maintained by two kernel maintainers That may be, but the Debian port isn't stable enough to *build* a kernel: https://buildd.debian.org/status/logs.php?pkg=linux=sh4 > * sparc64: used to be maintained by a team at Oracle but Oracle recently > changed their mind about Linux on SPARC; now it's back to > David Miller and some additional volunteers Oracle laid off their SPARC team. Fujitsu has another SPARC processor in development, but only for supercomputers (and they seem to be switching to Aarch64 later). So we can expect this architecture to slowly fade away now. > * x32: maintained by the people who maintain the x86 port of the kernel H. Peter Anvin did the original port in 2012, but so far as I can see none of the x86 maintainers have done any development on it since then. And yes, development work has been needed: - x32 has 64-bit time_t and that never worked quite right. This may have finally been fixed this year by Arnd Bergmann's work, but I'm not sure. - syscall restart was broken until 2015 when someone actually tested it. - There have been several regressions specific to x32, some of which stuck around for a year or so before someone and fixed them. [...] So, by all means have fun working on these ports, but aside from ppc64 I don't see any prospect of them meeting release qualifications. Ben. -- Ben Hutchings For every action, there is an equal and opposite criticism. - Harrison signature.asc Description: This is a digitally signed message part
Re: Re-evaluating architecture inclusion in unstable/experimental
On 9/28/18 11:26 PM, Adam D. Barratt wrote: > On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: >> So, it's not always a purely technical decision whether a port >> remains a release architecture. It's also often highly political and >> somehow also influenced by commercial entities. > > Please don't make implications like that unless you can back them up. Well, I cannot prove it. But when I see that we have ports as release architectures with hardware where atomics in hardware don't even work correctly and the virtual address space is limited to 2 GiB per process while on the other hand perfectly healthy and maintained ports like powerpc and ppc64 which have actually a measurable userbase and interest in the community are axed or barred from being a release architecture, then I have my doubts that those decisions aren't also driven by commercial interests or politics. I have seen IBM people on multiple occasions in various upstream projects trying to remove code for older POWER targets because they insisted anything below POWER8 is not supported anymore. In some cases like Golang with success [1]. > Adam > (involved to greater or lesser extent in every decision as to whether > an architecture should be added to or removed from testing over the > past almost decade, with no commercial interest involved in any way) I understand that. But if stuff that gets removed despite people using it and despite others maintaining it, I am having my doubts. And it has happened more than once where I had to explain users why there aren't any stable releases anymore they can install on their PowerPC-based machines. I really do not mean to accuse anyone of malevolence, I am merely speaking from my personal observations that sometimes such decisions do not necessarily meet the expectations of our users and that there might be a path for improvement, e.g in the form of support tiers like they exist in NetBSD. Thanks, Adrian > [1] https://github.com/golang/go/issues/19074 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Re-evaluating architecture inclusion in unstable/experimental
On Fri, 2018-09-28 at 14:16 +0200, John Paul Adrian Glaubitz wrote: > So, it's not always a purely technical decision whether a port > remains a release architecture. It's also often highly political and > somehow also influenced by commercial entities. Please don't make implications like that unless you can back them up. Adam (involved to greater or lesser extent in every decision as to whether an architecture should be added to or removed from testing over the past almost decade, with no commercial interest involved in any way)
Re: Re-evaluating architecture inclusion in unstable/experimental
Hello! I just saw this mail this morning by accident while browsing the archives, I am not subscribed to debian-devel. > The ftpmaster team would like to clarify which Debian ports should and/or would like to continue to be part of Debian unstable and experimental. I'm not sure what context you are referring to? Are you talking about architectures that are present in the main archive or do you also include Debian Ports which has its own FTP server and buildds maintained outside DSA? > As outlined on the Debian Archive Criteria page[0], the key points to consider are whether the architecture has been part of a stable release, whether it is *likely* to be part of a stable release, as well as whether it currently has a sensible number of active maintainers. All ports in Debian Ports have at least one maintainer otherwise it wouldn't be possible to keep the pace with unstable. Furthermore, several of the ports are in very healthy condition and even surpass some release architectures. The powerpc and ppc64 ports, for example, build more packages than any of the mips* ports. As for the kernel maintenance, except for Alpha, all the ports have at least one kernel maintainer: * hppa: actively maintained by two people who also maintain the toolchain * ia64: maintained by one paid developer at Intel * m68k: maintained by multiple independent kernel maintainers, at least five people involved upstream; port is also getting new drivers * powerpc/ppc64: maintained mostly by IBM people * sh4: maintained by two kernel maintainers * sparc64: used to be maintained by a team at Oracle but Oracle recently changed their mind about Linux on SPARC; now it's back to David Miller and some additional volunteers * x32: maintained by the people who maintain the x86 port of the kernel > Whilst you may be happy to continue the work of maintaining the port regardless, don't forget that excess or otherwise unnecessary architectures involve a shared maintenance burden as well as incurring non-trivial requirements on mirror/Debian resources. Debian Ports has its own mirrors. Besides, I think one of the main selling points of Debian is its portability. If we take away this feature, we become less attractive. If the maintenance burden is too high, it might be an idea to switch to a more flexible and powerful infrastructure like SUSE's OBS which makes handling of new ports much easier. It's very easy to set up your own OBS, so people within SUSE do that to maintain their own ports, for example. > The statistics and graphs available on the debian-ports page[1] may provide some objective statistics or reflection on the actual suitability of your architecture's continued inclusion. Jepp. And as you can see, several ports that are thought to be dead long time ago are healthier than some of the official Debian release architectures simply because there are huge communities behind it. The m68k port has a very strong and enthusiastic community which is why the port has already survived multiple other architectures in the kernel like TileGX and similar. There are people building new m68k hardware, writing new drivers and there is even an ongoing effort to create an m68k backend for LLVM so that we can eventually even start building Rust packages (I have already added partial target support for m68k in a local Rust branch). So, it's not always a purely technical decision whether a port remains a release architecture. It's also often highly political and somehow also influenced by commercial entities. Although I think Debian, being a community distribution, should be more oriented towards its community and not towards commercial interests. > So, in the first instance, would you like to continue being part of unstable/experimental? All Debian Ports architectures [1] should remain part of unstable and experimental and are very actively maintained by the Debian Ports team. You can find us in #debian-ports on OFTC. FWIW, we have our own instance of the transition tracker: > https://ben.jrtc27.com/ Thanks, Adrian > [1] https://buildd.debian.org/stats/graph-ports-big.png -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Re-evaluating architecture inclusion in unstable/experimental
Hi, With my debian-ports hat on, let me answer the parts related to that. First of all as a reminder, debian-ports was originally just there to help bootstrapping an architecture and prove it meets the ftpmasters criteria to become an official architecture. It has been used that way for example for armel, armhf, arm64, kfreebsd-amd64, kfreebsd-i386 and s390x. On 2018-09-02 15:21, Samuel Thibault wrote: > Hello, > > Luke W Faraone, le lun. 27 août 2018 00:33:58 -0700, a ecrit: > > So, in the first instance, would you like to continue being part of > > unstable/experimental? > > Well, I can simply point at what we said last time (IIRC) the question > was raised, here are the importants point we see in being on debian > instead of debian-ports: > > https://lists.debian.org/debian-devel/2015/05/msg00070.html > > > Samuel Thibault wrote for the debian-hurd team: > > * Appearing on packages' and maintainers' PTS > > pages like http://buildd.debian.org/bash and > > https://buildd.debian.org/sthiba...@debian.org > > This has been changed since then: debian-ports architectures show up > there. Yes both instances have been merged during Debconf 15. > > * Getting binNMUs from d-release transitions > > I believe this is also now done for debian-ports architectures? This > really saves a lot of duplicate work for porters. For what I understand the release team schedule binNMUs on all architectures that have the necessary packages built. As long as the architecture is not lagging that should therefore be the case. > > * Appearing on http://release.debian.org/transitions/ and > > https://qa.debian.org/dose/debcheck/unstable_main/index.html > > We're fine with d-release not looking at the hurd column. But *we* > > however do look at it, and would be sad to completely lose that. It > > could be in a completely separate page or column, that would not pose > > problem. > > I don't know if we have this for debian-ports? debian-ports is only about hosting the archive, so we do not have that. I guess that would need to be added the same way as for the buildd.d.o merge. [...] > > Whilst you may be happy to continue the work of maintaining the port > > regardless, don't forget that excess or otherwise unnecessary > > architectures involve a shared maintenance burden as well as incurring > > non-trivial requirements on mirror/Debian resources. > > Concerning mirroring, it is indeed useless to mirror hurd-i386 > worldwide. Considering maintenance burden, I'm a bit afraid of here > simply moving the load from the ftpmaster team to the debian-ports > ftpmaster team. I don't know the details, so can't say, I'm just Cc-ing > both teams. I agree for the mirroring part, that's why there are very few debian-ports mirrors (the original plan was even to have none of them) and that it is served through the deb.debian.org CDN. Besides that mirroring aspect, I agree moving an architecture from the official to the debian-ports archive is just moving the load to a different set of Debian person and different set of Debian hardware. Regards, Aurelien -- Aurelien Jarno GPG: 4096R/1DDD8C9B aurel...@aurel32.net http://www.aurel32.net
Re: Re-evaluating architecture inclusion in unstable/experimental
On Mon, 2018-09-03 at 01:07 +0200, Samuel Thibault wrote: > > > So has the debian patch been submitted in #900240 upstream by you or Petter > > Reinholdtsen yet? I don't believe so! > > I don't think so either, it'd be marked forwarded. That doesn't mean you > can't help with it. Regardless who is submitting patches upstream, two problem remains: 1) The delay until upstream makes a new release. 2) The delay until the Debian Maintainer creates an updated package from that release (if ever). Example: emacs26 And how many maintainers do cherry-pick patches accepted upstream, very few :(
Re: Re-evaluating architecture inclusion in unstable/experimental
Svante Signell, le lun. 03 sept. 2018 01:18:20 +0200, a ecrit: > On Mon, 2018-09-03 at 01:07 +0200, Samuel Thibault wrote: > > > > > So has the debian patch been submitted in #900240 upstream by you or > > > Petter > > > Reinholdtsen yet? I don't believe so! > > > > I don't think so either, it'd be marked forwarded. That doesn't mean you > > can't help with it. > > Regardless who is submitting patches upstream, two problem remains: > 1) The delay until upstream makes a new release. Nope, the maintainer said he was happy to cherry-pick. > 2) The delay until the Debian Maintainer creates an updated package from that > release (if ever). Example: emacs26 Again, cherry-pick don't need to wait for the introduction of the new release to Debian. > And how many maintainers do cherry-pick patches accepted upstream, very few > :( Here the maintainer explicitly said we would happily do it. Samuel
Re: Re-evaluating architecture inclusion in unstable/experimental
On Sun, Sep 02, 2018 at 11:39:08PM +0200, Svante Signell wrote: > On Sun, 2018-09-02 at 19:46 +0200, Samuel Thibault wrote: > > If it's not tricky, a NMU should be able to fix it easily. > > I'm sorry Samuel, I asked both you and James Clarke, Cc:ed, for help on this > issue and you both said it was not possible to NMU cmake, even if you are both > DD's. See bugs > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905140 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900240 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905138 > > I think that the power of a package maintainer is far too big, making it > possible to reject/ignore important and other bugs, especially with patches, > for > non-released architectures (and effectively block NMUs). The dev ref is quite clear about the right of porters to NMU: # 5.10.2.2. When to do a source NMU if you are a porter [...] # If you are a porter doing an NMU for unstable, the above # guidelines for porting should be followed, with two variations. # Firstly, the acceptable waiting period — the time between when # the bug is submitted to the BTS and when it is OK to do an NMU — # is seven days for porters working on the unstable distribution. # This period can be shortened if the problem is critical and # imposes hardship on the porting effort, at the discretion of the # porter group. (Remember, none of this is Policy, just mutually # agreed upon guidelines.) For uploads to stable or testing, please # coordinate with the appropriate release team first. Broken cmake pretty much stops the whole port (as it has direct and indirect rbdeps all around), this certainly counts as a "hardship". > I think the next step would be to bring the responsibilities and commitments > of > a Package Maintainer to the TC, in addition to the full control of everything > related to that package. Maybe the recent salvaging of packages could be > helpful > in the future regarding this problem. Stopping a whole port for a reason as flimsy as "I'm not interested in maintaining patches for things that clearly belong upstream." sounds like something that warrants escalation, yeah. Meow! -- ⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition: ⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17] ⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37] ⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]
Re: Re-evaluating architecture inclusion in unstable/experimental
Svante Signell, le lun. 03 sept. 2018 01:06:11 +0200, a ecrit: > On Mon, 2018-09-03 at 00:19 +0200, Samuel Thibault wrote: > > Felix Geyer wrote: > > > I suggest that instead you respond to questions on bugs you opened. > > > I'm not interested in maintaining patches for things that clearly > > > belong upstream. Once upstream has reviewed the changes I'm happy to > > > cherry-pick them. > > This is a comment on the Hurd bug, #900200, not the kfreebsd one, #905138. That doesn't mean the comment doesn't hold generally. > And I did not open that bug, you did! That doesn't mean you can't help with it. > > And I agree with it (and also one of the reasons why I didn't just > > NMU-ed cmake with the hurd patch). > > So has the debian patch been submitted in #900240 upstream by you or Petter > Reinholdtsen yet? I don't believe so! I don't think so either, it'd be marked forwarded. That doesn't mean you can't help with it. Samuel
Re: Re-evaluating architecture inclusion in unstable/experimental
On Mon, 2018-09-03 at 00:19 +0200, Samuel Thibault wrote: > > > I'm sorry Samuel, I asked both you and James Clarke, Cc:ed, for help on this > > issue and you both said it was not possible to NMU cmake, even if you are > > both > > DD's. > > For my part, I was not talking about that patch, but about the > hurd-related patches. > > For others, I can simply quote what was said: > > Felix Geyer wrote: > > I suggest that instead you respond to questions on bugs you opened. > > I'm not interested in maintaining patches for things that clearly > > belong upstream. Once upstream has reviewed the changes I'm happy to > > cherry-pick them. This is a comment on the Hurd bug, #900200, not the kfreebsd one, #905138. And I did not open that bug, you did! > And I agree with it (and also one of the reasons why I didn't just > NMU-ed cmake with the hurd patch). So has the debian patch been submitted in #900240 upstream by you or Petter Reinholdtsen yet? I don't believe so!
Re: Re-evaluating architecture inclusion in unstable/experimental
Svante Signell, le dim. 02 sept. 2018 23:39:08 +0200, a ecrit: > On Sun, 2018-09-02 at 19:46 +0200, Samuel Thibault wrote: > > Svante Signell, le dim. 02 sept. 2018 19:45:19 +0200, a ecrit: > > > On Sun, 2018-09-02 at 15:21 +0200, Samuel Thibault wrote: > > > > > > > > > > > > The statistics and graphs available on the debian-ports page[1] may > > > > > provide some objective statistics or reflection on the actual > > > > > suitability of your architecture's continued inclusion. > > > > > [1]: https://buildd.debian.org/stats/ > > > > > > > > Such statistics are really difficult to get any real conclusion from. > > > > Sometimes 10% packages are missing just for one tricky nonLinux-specific > > > > issue in one package. > > > > > > Correct: One example is cmake for both hurd-i386 and kfreebsd-any. > > > It does not even have to be tricky. > > > > If it's not tricky, a NMU should be able to fix it easily. > > I'm sorry Samuel, I asked both you and James Clarke, Cc:ed, for help on this > issue and you both said it was not possible to NMU cmake, even if you are both > DD's. For my part, I was not talking about that patch, but about the hurd-related patches. For others, I can simply quote what was said: Felix Geyer wrote: > I suggest that instead you respond to questions on bugs you opened. > I'm not interested in maintaining patches for things that clearly > belong upstream. Once upstream has reviewed the changes I'm happy to > cherry-pick them. And I agree with it (and also one of the reasons why I didn't just NMU-ed cmake with the hurd patch). > I think that the power of a package maintainer is far too big, making it > possible to reject/ignore important and other bugs, especially with patches, > for > non-released architectures (and effectively block NMUs). He is not rejecting things, he is saying that belongs to upstream, which is true. Help with that and things will flow. > I think the next step would be to bring the responsibilities and commitments > of > a Package Maintainer to the TC, Nope. > Maybe the recent salvaging of packages could be helpful in the future > regarding this problem. Nope. Samuel
Re: Re-evaluating architecture inclusion in unstable/experimental
On Sun, 2018-09-02 at 19:46 +0200, Samuel Thibault wrote: > Svante Signell, le dim. 02 sept. 2018 19:45:19 +0200, a ecrit: > > On Sun, 2018-09-02 at 15:21 +0200, Samuel Thibault wrote: > > > > > > > > > The statistics and graphs available on the debian-ports page[1] may > > > > provide some objective statistics or reflection on the actual > > > > suitability of your architecture's continued inclusion. > > > > [1]: https://buildd.debian.org/stats/ > > > > > > Such statistics are really difficult to get any real conclusion from. > > > Sometimes 10% packages are missing just for one tricky nonLinux-specific > > > issue in one package. > > > > Correct: One example is cmake for both hurd-i386 and kfreebsd-any. > > It does not even have to be tricky. > > If it's not tricky, a NMU should be able to fix it easily. I'm sorry Samuel, I asked both you and James Clarke, Cc:ed, for help on this issue and you both said it was not possible to NMU cmake, even if you are both DD's. See bugs https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905140 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900240 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905138 I think that the power of a package maintainer is far too big, making it possible to reject/ignore important and other bugs, especially with patches, for non-released architectures (and effectively block NMUs). I think the next step would be to bring the responsibilities and commitments of a Package Maintainer to the TC, in addition to the full control of everything related to that package. Maybe the recent salvaging of packages could be helpful in the future regarding this problem.
Re: Re-evaluating architecture inclusion in unstable/experimental
Svante Signell, le dim. 02 sept. 2018 19:45:19 +0200, a ecrit: > On Sun, 2018-09-02 at 15:21 +0200, Samuel Thibault wrote: > > > > > > The statistics and graphs available on the debian-ports page[1] may > > > provide some objective statistics or reflection on the actual > > > suitability of your architecture's continued inclusion. > > > [1]: https://buildd.debian.org/stats/ > > > > Such statistics are really difficult to get any real conclusion from. > > Sometimes 10% packages are missing just for one tricky nonLinux-specific > > issue in one package. > > Correct: One example is cmake for both hurd-i386 and kfreebsd-any. > It does not even have to be tricky. If it's not tricky, a NMU should be able to fix it easily. Samuel
Re: Re-evaluating architecture inclusion in unstable/experimental
On Sun, 2018-09-02 at 15:21 +0200, Samuel Thibault wrote:
>
> > The statistics and graphs available on the debian-ports page[1] may
> > provide some objective statistics or reflection on the actual
> > suitability of your architecture's continued inclusion.
> > [1]: https://buildd.debian.org/stats/
>
> Such statistics are really difficult to get any real conclusion from.
> Sometimes 10% packages are missing just for one tricky nonLinux-specific
> issue in one package.
Correct: One example is cmake for both hurd-i386 and kfreebsd-any.
It does not even have to be tricky. For kfreebsd the patch(es) is attached
below!
Index: cmake-3.11.2/bootstrap
===
--- cmake-3.11.2.orig/bootstrap
+++ cmake-3.11.2/bootstrap
@@ -1352,7 +1352,7 @@ else
libs="${libs} -ldl -lrt"
;;
*BSD*)
- libs="${libs} -lkvm"
+ libs="${libs} -lkvm -lfreebsd-glue"
;;
*SunOS*)
# Normally libuv uses '-D_XOPEN_SOURCE=500 -std=c90' on Solaris 5.10,
--- a/debian/control 2018-05-19 10:51:17.0 +0200
+++ b/debian_control 2018-07-29 17:38:11.272777000 +0200
@@ -15,6 +15,7 @@
librhash-dev,
libuv1-dev (>= 1.10),
procps [!hurd-any],
+ freebsd-glue [kfreebsd-any],
python3-sphinx,
qtbase5-dev ,
zlib1g-dev
Re: Re-evaluating architecture inclusion in unstable/experimental
Hello, Luke W Faraone, le lun. 27 août 2018 00:33:58 -0700, a ecrit: > So, in the first instance, would you like to continue being part of > unstable/experimental? Well, I can simply point at what we said last time (IIRC) the question was raised, here are the importants point we see in being on debian instead of debian-ports: https://lists.debian.org/debian-devel/2015/05/msg00070.html Samuel Thibault wrote for the debian-hurd team: > * Appearing on packages' and maintainers' PTS > pages like http://buildd.debian.org/bash and > https://buildd.debian.org/sthiba...@debian.org This has been changed since then: debian-ports architectures show up there. > * Getting binNMUs from d-release transitions I believe this is also now done for debian-ports architectures? This really saves a lot of duplicate work for porters. > * Appearing on http://release.debian.org/transitions/ and > https://qa.debian.org/dose/debcheck/unstable_main/index.html > We're fine with d-release not looking at the hurd column. But *we* > however do look at it, and would be sad to completely lose that. It > could be in a completely separate page or column, that would not pose > problem. I don't know if we have this for debian-ports? > * Being considered as "second-class citizen" As said at the time, this is rather already the case. Luke W Faraone, le lun. 27 août 2018 00:33:58 -0700, a ecrit: > As outlined on the Debian Archive Criteria page[0], the key points to > consider are whether the architecture has been part of a stable release, > whether it is *likely* to be part of a stable release, as well as > whether it currently has a sensible number of active maintainers. Considering how even quite a few Linux architectures ports are not making it, I don't think we could say it likely that hurd-i386 be part of a stable release. > Whilst you may be happy to continue the work of maintaining the port > regardless, don't forget that excess or otherwise unnecessary > architectures involve a shared maintenance burden as well as incurring > non-trivial requirements on mirror/Debian resources. Concerning mirroring, it is indeed useless to mirror hurd-i386 worldwide. Considering maintenance burden, I'm a bit afraid of here simply moving the load from the ftpmaster team to the debian-ports ftpmaster team. I don't know the details, so can't say, I'm just Cc-ing both teams. > The statistics and graphs available on the debian-ports page[1] may > provide some objective statistics or reflection on the actual > suitability of your architecture's continued inclusion. > [1]: https://buildd.debian.org/stats/ Such statistics are really difficult to get any real conclusion from. Sometimes 10% packages are missing just for one tricky nonLinux-specific issue in one package. Samuel
Re-evaluating architecture inclusion in unstable/experimental
(resending with corrected address for debian-bsd) Dear ports maintainer, The ftpmaster team would like to clarify which Debian ports should and/or would like to continue to be part of Debian unstable and experimental. As outlined on the Debian Archive Criteria page[0], the key points to consider are whether the architecture has been part of a stable release, whether it is *likely* to be part of a stable release, as well as whether it currently has a sensible number of active maintainers. Whilst you may be happy to continue the work of maintaining the port regardless, don't forget that excess or otherwise unnecessary architectures involve a shared maintenance burden as well as incurring non-trivial requirements on mirror/Debian resources. The statistics and graphs available on the debian-ports page[1] may provide some objective statistics or reflection on the actual suitability of your architecture's continued inclusion. So, in the first instance, would you like to continue being part of unstable/experimental? [0]: https://ftp-master.debian.org/archive-criteria.html [1]: https://buildd.debian.org/stats/ Cheers, Luke Faraone signature.asc Description: OpenPGP digital signature
Re-evaluating architecture inclusion in unstable/experimental
Dear ports maintainer, The ftpmaster team would like to clarify which Debian ports should and/or would like to continue to be part of Debian unstable and experimental. As outlined on the Debian Archive Criteria page[0], the key points to consider are whether the architecture has been part of a stable release, whether it is *likely* to be part of a stable release, as well as whether it currently has a sensible number of active maintainers. Whilst you may be happy to continue the work of maintaining the port regardless, don't forget that excess or otherwise unnecessary architectures involve a shared maintenance burden as well as incurring non-trivial requirements on mirror/Debian resources. The statistics and graphs available on the debian-ports page[1] may provide some objective statistics or reflection on the actual suitability of your architecture's continued inclusion. So, in the first instance, would you like to continue being part of unstable/experimental? [0]: https://ftp-master.debian.org/archive-criteria.html [1]: https://buildd.debian.org/stats/ Cheers, Luke Faraone signature.asc Description: OpenPGP digital signature
