Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
spoke again to TL and asked if pine64 would be willing to look at sponsorship witn rockpro64 boards (the ones that take 4x PCIe): if someone from debian were to contact him direct he would happily consider it. i then asked him if i could cc him into this discussion and he said he was way *way* too busy to enter into another mailing list discussion, so if someone from debian emails me privately, off-list, i will then cc him and/or put them in touch with him on irc. i can also be reached on freenode and oftc as "lkcl", if that is easier. l.
Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
On Fri, Jun 29, 2018 at 8:13 PM, Luke Kenneth Casson Leighton wrote: > On Fri, Jun 29, 2018 at 6:59 PM, Jonathan Wiltshire wrote: > >>> also worth noting, they're working on a 2U rackmount server which >>> will have i think something insane like 48 Rock64Pro boards in one >>> full-length case. > >> None of this addresses the basic DSA requirement of remote management. >> Troubling local hands to change a disk once in a while is reasonable; being >> blocked waiting for a power cycle on a regular basis is not (and I can't >> imagine hosting sponsors are wild about their employees' time being used >> for that either). > > i know exactly what you mean, i've had to deal with data centres. > i'll make sure that TL Lim is aware of this, and will ask him if > there's a way to include remote power-management / power-cycling of > boards in the planned product or if it's already been thought of. TL informs me that all the power and reset signals for all 48 of the RockPro64s tucked into the full-length 2U case are brought out to the back panel. an MCU (or MCUs) or SBC (or SBCs) may therefore be connected directly to those in order to provide *individual* remote power / reset management of all 48 RockPro64s. DIY remote power management, but it is actual remote power management. l.
Bug#880675: (no subject)
Fixed in glibc, but not in CLDR yet: https://sourceware.org/bugzilla/show_bug.cgi?id=22996
Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 On Fri, Jun 29, 2018 at 8:31 PM, Florian Weimer wrote: > * Luke Kenneth Casson Leighton: > >> that is not a surprise to hear: the massive thrashing caused by the >> linker phase not being possible to be RAM-resident will be absolutely >> hammering the drives beyond reasonable wear-and-tear limits. which is >> why i'm recommending people try "-Wl,--no-keep-memory". > > Note that ld will sometimes stuff everything into a single RWX segment > as a result, which is not desirable. florian, thank you for responding: i've put a copy of the insights that you give into the bugreport at https://sourceware.org/bugzilla/show_bug.cgi?id=22831#c16 > Unfortunately, without significant investment into historic linker > technologies (with external sorting and that kind of stuff), yes, ah ha! funnily enough the algorithm that i was asked to create back in 1988 was an external matrix-multiply, i take it you are talking about the same thing, where linking is done using explicit load-process-save cycles rather than relying on swap. > I don't > think it is viable to build 32-bit software natively in the near > future. i noted an alternative strategy in the bugreport: if binutils *at the very least* were to look at the available resident RAM and only malloc'd and used up to that amount, and kept only a few (or even just one) object file in memory at a time and did all the linking for that, it would be infinitely better than the current situation which is *only going to get worse*. > Maybe next year only a few packages will need exceptions, but > the number will grow with each month. well... that ignores the fact that at some point in the next few years there will be a package that needs 16 GB of resident RAM to link. and a few years after that it will be 32 GB. and that's on 64-bit systems. the package's name will probably be "firefox", given the current growth rate. does debian *really* want to have to upgrade all 64-bit systems in the build farm first to 16 GB RAM and then to 32 GB and then to 64 GB?? can the powerpc64 systems and all other 64-bit architectures even *be* upgraded to 16 GB then 32 GB then 64 GB of RAM?? basically the problems faced by 32-bit systems are a warning shot across the bows about ld not really being kept up-to-date with the increases in software complexity that's being thrown at it. it's *NOT* just about 32-bit. this problem can basically be faced REactively... or it can be faced PROactively: the historic linker strategies that you mention are i feel going to be needed in some years' time *even for 64-bit*. l.
Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
* Luke Kenneth Casson Leighton: > that is not a surprise to hear: the massive thrashing caused by the > linker phase not being possible to be RAM-resident will be absolutely > hammering the drives beyond reasonable wear-and-tear limits. which is > why i'm recommending people try "-Wl,--no-keep-memory". Note that ld will sometimes stuff everything into a single RWX segment as a result, which is not desirable. Unfortunately, without significant investment into historic linker technologies (with external sorting and that kind of stuff), I don't think it is viable to build 32-bit software natively in the near future. Maybe next year only a few packages will need exceptions, but the number will grow with each month. Building on 64-bit kernels will delay the inevitable because more address space is available to user space, but that's probably 12 to 18 month extended life-time for native building.
Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
On Fri, Jun 29, 2018 at 6:59 PM, Jonathan Wiltshire wrote: >> also worth noting, they're working on a 2U rackmount server which >> will have i think something insane like 48 Rock64Pro boards in one >> full-length case. > None of this addresses the basic DSA requirement of remote management. > Troubling local hands to change a disk once in a while is reasonable; being > blocked waiting for a power cycle on a regular basis is not (and I can't > imagine hosting sponsors are wild about their employees' time being used > for that either). i know exactly what you mean, i've had to deal with data centres. i'll make sure that TL Lim is aware of this, and will ask him if there's a way to include remote power-management / power-cycling of boards in the planned product or if it's already been thought of. l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Fri, Jun 29, 2018 at 06:29:48PM +0200, Geert Uytterhoeven wrote:
> Are you sure you're not interchanging A8 and A9, cfr. Linux kernel commit
> e388b80288aade31 ("ARM: spectre-v2: add Cortex A8 and A15 validation of the
> IBE bit")?
Yes. That is the main reason the A9 is faster than the A8 at the same
clock speed by quite a bit.
For example http://www.360doc.com/content/12/0806/14/350555_228637047.shtml
says:
Cortex-A9 has many advanced features for a RISC CPU, such as speculative
data accesses, branch prediction, multi-issuing of instructions,
hardware cache coherency, out-of-order execution and register renaming.
Cortex-A8 does not have these, except for dual-issuing instructions and
branch prediction.
So the A8 still has branch prediction so it can keep the pipeline fed,
even with in order execution. Spectre isn't really about out-of-order
excution as much as it is about speculative memory accesses which some
in-order execution chips have too.
--
Len Sorensen
Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
On Fri, Jun 29, 2018 at 06:05:55PM +0100, Luke Kenneth Casson Leighton wrote: > apologies for repeating it again: this is why i'm recommending people > try "-Wl,--no-keep-memory" on the linker phase as if it works as > intended it will almost certainly drastically reduce memory usage to > the point where it will stay, for the majority of packages, well > within the 2GB limit i.e. within resident RAM. [...] > most of them won't have native SATA: very few 32-bit ARM systems do. > GbE is not that common either (so decent-speed network drives are > challenging, as well). so they'll almost certainly be USB-based > (USB-to-SATA, which is known-unreliable), and putting such vast > amounts of drive-hammering through USB-to-SATA due to thrashing isn't > going to help :) [...] > > the allwinner A20 and R40 are the two low-cost ARM systems that i'm > aware of that have native SATA. > > > there is however a new devboard that is reasonably cheap and should > be available really soon: the Rock64Pro (not to be confused with the > Rock64, which does NOT have PCie), from pine64: > https://www.pine64.org/?page_id=61454 > > it's one of the first *low-cost* ARM dev-boards that i've seen which > has 4GB of RAM and has a 4x PCIe slot. the team have tested it out > with an NVMe SSD and also 4x SATA PCIe cards: they easily managed to > hit 20 Gigabits per second on the NVMe drive (2500 mbytes/sec). > > also worth noting, they're working on a 2U rackmount server which > will have i think something insane like 48 Rock64Pro boards in one > full-length case. > > the Rock64Pro uses the RK3399 which is a 4-core CortexA53 plus 2-core > CortexA72 for a total 6-core SMP system, all 64-bit. > > if anyone would like me to have a word with TL Lim (the CEO of > pine64) i can see if he is willing and able to donate some Rock64Pro > boards to the debian farm, let me know. None of this addresses the basic DSA requirement of remote management. Troubling local hands to change a disk once in a while is reasonable; being blocked waiting for a power cycle on a regular basis is not (and I can't imagine hosting sponsors are wild about their employees' time being used for that either). Development boards just don't cut it any longer. -- Jonathan Wiltshire j...@debian.org Debian Developer http://people.debian.org/~jmw 4096R: 0xD3524C51 / 0A55 B7C5 1223 3942 86EC 74C3 5394 479D D352 4C51
Re: Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
On Fri, Jun 29, 2018 at 5:21 PM, Steve McIntyre wrote: >>2G is also way too little memory these days for a new buildd. > > Nod - lots of packages are just too big for that now. apologies for repeating it again: this is why i'm recommending people try "-Wl,--no-keep-memory" on the linker phase as if it works as intended it will almost certainly drastically reduce memory usage to the point where it will stay, for the majority of packages, well within the 2GB limit i.e. within resident RAM. i'm really not sure why the discussions continue not to take this into account, repeating the status-quo and accepting "packages are too big" as if there is absolutely no possible way that this problem may be solved or even attempted to be solved... ever. i am very confused by this. perhaps it is down to latency in discussions as new people contribute (but have signficant delay on reading), i don't know. > Future options > == > > I understand DSA's reluctance to continue supporting dev boards as > build platforms - I've been the one working on some of these machines > in the machine room at Arm, and it's painful when you can't reliably > reboot or get onto the console of crashed machines. We've also had a > spate of disk failures recently which has caused extended downtime. that is not a surprise to hear: the massive thrashing caused by the linker phase not being possible to be RAM-resident will be absolutely hammering the drives beyond reasonable wear-and-tear limits. which is why i'm recommending people try "-Wl,--no-keep-memory". ... oh, i have an idea which people might like to consider trying. it's to use "-Wl,--no-keep-memory" on the linker phase of 32-bit builds. did i mention that already? :) it might save some build hardware from being destroyed if people try using "-Wl,--no-keep-memory"! > I'm just in the middle of switching the arm64 machines here to using > SW RAID to mitigate that in future, and that's just not an option on > the dev boards. We want to move away from dev boards for these > reasons, at the very least. most of them won't have native SATA: very few 32-bit ARM systems do. GbE is not that common either (so decent-speed network drives are challenging, as well). so they'll almost certainly be USB-based (USB-to-SATA, which is known-unreliable), and putting such vast amounts of drive-hammering through USB-to-SATA due to thrashing isn't going to help :) the allwinner A20 and R40 are the two low-cost ARM systems that i'm aware of that have native SATA. there is however a new devboard that is reasonably cheap and should be available really soon: the Rock64Pro (not to be confused with the Rock64, which does NOT have PCie), from pine64: https://www.pine64.org/?page_id=61454 it's one of the first *low-cost* ARM dev-boards that i've seen which has 4GB of RAM and has a 4x PCIe slot. the team have tested it out with an NVMe SSD and also 4x SATA PCIe cards: they easily managed to hit 20 Gigabits per second on the NVMe drive (2500 mbytes/sec). also worth noting, they're working on a 2U rackmount server which will have i think something insane like 48 Rock64Pro boards in one full-length case. the Rock64Pro uses the RK3399 which is a 4-core CortexA53 plus 2-core CortexA72 for a total 6-core SMP system, all 64-bit. if anyone would like me to have a word with TL Lim (the CEO of pine64) i can see if he is willing and able to donate some Rock64Pro boards to the debian farm, let me know. l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
Hi Lennart,
debian-ports -> debian-arm
On Fri, Jun 29, 2018 at 5:00 PM Lennart Sorensen
wrote:
> On Fri, Jun 29, 2018 at 10:20:50AM +0100, Luke Kenneth Casson Leighton wrote:
> > in addition, arm64 is usually speculative OoO (Cavium ThunderX V1
> > being a notable exception) which means it's vulnerable to spectre and
> > meltdown attacks, whereas 32-bit ARM is exclusively in-order. if you
> > want to GUARANTEE that you've got spectre-immune hardware you need
> > either any 32-bit system (where even Cortex A7 has virtualisattion) or
> > if 64-bit is absolutely required use Cortex A53.
>
> The Cortex A8, A7 and A5 are in order. The A9, A15, A17 etc are out of
> order execution. So any 32 bit arm worth using is pretty much always
> out of order execution.
Are you sure you're not interchanging A8 and A9, cfr. Linux kernel commit
e388b80288aade31 ("ARM: spectre-v2: add Cortex A8 and A15 validation of the
IBE bit")?
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
On Fri, Jun 29, 2018 at 11:23:25AM +0200, Julien Cristau wrote: >On 06/29/2018 09:16 AM, Uwe Kleine-König wrote: >>> >>> [DSA Sprint report]: >>> https://lists.debian.org/debian-project/2018/02/msg4.html >> >> In this report Julien Cristau wrote: >> >>> In short, the hardware (development boards) we're currently using to >>> build armel and armhf packages aren't up to our standards, and we >>> really, really want them to go away when stretch goes EOL (expected in >>> 2020). We urge arm porters to find a way to build armhf packages in >>> VMs or chroots on server-class arm64 hardware. >> >> If the concerns are mostly about the hardware not being rackable, there >> is a rackable NAS by Netgear: >> >> >> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs >> >> with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 >> GiB) are good enough. The machine can run mainline Linux[1]. I think >> U-Boot doesn't support this machine in mainline though. >> >Rackable, while good, is only part of it. The main part is remote >management. I'm not seeing any mention of ipmi or anything like that in >the datasheet? > >2G is also way too little memory these days for a new buildd. Nod - lots of packages are just too big for that now. So, here's a summary of the current build/porter machines we have for the arm ports. armel/armhf *only* build machines = We're mostly using dev boards to build 32-bit arm stuff yet. * 7 Marvell Armada XP machines: dev boards supplied in a box, 4GiB RAM - see http://photos.einval.com/gallery/2014_marvell_buildd These are nice powerful little machines, but they're a PITA to manage for power (won't cold boot without a button push) and serial (USB serial exposed only). We have 6 of these as build boxes and 1 porter box, and I have a spare ready to go into service if desired. They *don't* have NEON, so we also still have: * 1 Freescale imx53 dev board as a porter box - old, slow Cortex A8 machine with only 1GiB of RAM. This works better for serial, but remote power issues again - needs a button push to cold boot. Will happily retire this once we have NEON available by default. Each of these dev boards only has support for 1 disk, so disk failures are painful. arm64 build machines These are all more normal computers, with better support for remote power and serial, DIMM slots, multiple disks, etc. * APM Mustang (X-Gene 1): officially EOL, but working fine for now. Normal server-class machine (although supplied in a small desktop case!) with some onboard server management stuff. We currently have one of these. We used to have more loaned/hosted by Linaro, and I've had an offer of more of these if we're interested. They'll build and run A32 (32-bit instruction set) as well as A64. * Gigabyte MP30-AR0 (X-Gene 1): server systems based on the Mustang core - see https://b2b.gigabyte.com/Server-Motherboard/MP30-AR0-rev-11#ov Capable of building/running A32 and A64. * AMD Seattle (Opteron A1100): officially EOL too, but working fine. Same as the Softiron 3000, 2U rackmount case. Capable of building/running A32 and A64. One of these has just been configured to build armhf only. Future options == I understand DSA's reluctance to continue supporting dev boards as build platforms - I've been the one working on some of these machines in the machine room at Arm, and it's painful when you can't reliably reboot or get onto the console of crashed machines. We've also had a spate of disk failures recently which has caused extended downtime. I'm just in the middle of switching the arm64 machines here to using SW RAID to mitigate that in future, and that's just not an option on the dev boards. We want to move away from dev boards for these reasons, at the very least. So, at the moment as far as I can see we're happy with our current arm64 build machines. They are ageing, so obviously I'm continuing to look out for new options there as well. *But* my priority is new options for 32-bit building too. Following standard Debian practice, we want to build *natively* (i.e. not using cross-building or using hardware emulation). Building 32-bit code on a 64-bit platform should not be an issue so long as the platform can also execute that 32-bit code directly. I am not aware of any 32-bit Arm *server* platforms shipping today. Some have existed in the past (e.g. Calxeda), but almost universally people have moved on to 64-bit now. The awkward thing that is now becoming more common in the arm64 server world is that quite a few of the vendors are not seeing any value in A32 support so they're leaving it out of their CPUs. We'll need to be careful about that. Options I can see today for new publically available machines are here. I'm sure I'll have missed something obvious - please feel free to improve on this list! *
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
> On Jun 29, 2018, at 9:50 AM, Jonathan Wiltshire wrote: > >> On Fri, Jun 29, 2018 at 11:04:26PM +0900, Roger Shimizu wrote: >> I see armel is already not a candidate for buster [0]. >> So it seems we can discuss armhf, but no armel at all. >> I don't agree with this idea. >> And I think we should treat armel and armhf equally. > > Please review the mail which originated this thread [1] where you will see > that both armel and armhf are affected by DSA's concern. If I understand > correctly, virtualisation of architectures in general is a possible > solution but there are problems in the case of these two. I have just talked to a colleague at SUSE about this and apparently Alex Graf from SUSE’s QEMU/KVM team has fixed many bugs regarding ARM32 on ARM64 virtualization. If I understand correctly, SUSE builds ARMv7 on ARM64 without problems. I have reached out to Alex Graf and asked him for clarification on what possibilities we currently have. Adrian
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 11:04:26PM +0900, Roger Shimizu wrote: > I see armel is already not a candidate for buster [0]. > So it seems we can discuss armhf, but no armel at all. > I don't agree with this idea. > And I think we should treat armel and armhf equally. Please review the mail which originated this thread [1] where you will see that both armel and armhf are affected by DSA's concern. If I understand correctly, virtualisation of architectures in general is a possible solution but there are problems in the case of these two. At the end of the day, if Debian can't reliably run builders for an architecture we do not do users a service by pretending to be able to support it in a formal release. A freeze may be for Christmas, but stable is for at least five+ years. -- Jonathan Wiltshire j...@debian.org Debian Developer http://people.debian.org/~jmw 4096R: 0xD3524C51 / 0A55 B7C5 1223 3942 86EC 74C3 5394 479D D352 4C51
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Fri, Jun 29, 2018 at 10:20:50AM +0100, Luke Kenneth Casson Leighton wrote: > in addition, arm64 is usually speculative OoO (Cavium ThunderX V1 > being a notable exception) which means it's vulnerable to spectre and > meltdown attacks, whereas 32-bit ARM is exclusively in-order. if you > want to GUARANTEE that you've got spectre-immune hardware you need > either any 32-bit system (where even Cortex A7 has virtualisattion) or > if 64-bit is absolutely required use Cortex A53. The Cortex A8, A7 and A5 are in order. The A9, A15, A17 etc are out of order execution. So any 32 bit arm worth using is pretty much always out of order execution. For 64 bit, I think the A35 and A53 are in order while the A57, A72 etc are out of order. Of course non Cortex designs vary (I think Marvel's JP4 core was out of order execution for example). After all, in general in order execution equals awful performance. -- Len Sorensen
Re: armel/armhf arch qualification for buster: call for DSA, Security, toolchain concernsj
Open Network Linux developed for the Open Compute Project has a vested interest in maintaining support for armel at least. I would be interested in sponsoring or donating rack mountable switches using armel processors. On Fri, Jun 29, 2018 at 3:04 AM, W. Martin Borgert wrote: > Quoting Uwe Kleine-König : > >> If the concerns are mostly about the hardware not being rackable, there >> is a rackable NAS by Netgear: >> >> https://www.netgear.com/business/products/storage/readynas/ >> RN2120.aspx#tab-techspecs >> > > This seems to be out of stock and discontinued, unfortunately. > > Anyway, I'm relatively sure, that I can convince my boss to sponsor/donate > both armel and armhf hardware for Debian, if that is of any help. Or arm64 > used in "32 bits mode". > >
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 10:04 PM, Uwe Kleine-König wrote: > Hello Julien, > > On 06/29/2018 11:23 AM, Julien Cristau wrote: >>> If the concerns are mostly about the hardware not being rackable, there >>> is a rackable NAS by Netgear: >>> >>> >>> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs >>> >>> with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 >>> GiB) are good enough. The machine can run mainline Linux[1]. I think >>> U-Boot doesn't support this machine in mainline though. >>> >> Rackable, while good, is only part of it. The main part is remote >> management. I'm not seeing any mention of ipmi or anything like that in >> the datasheet? > > you can access the serial console, but I don't think there is built-in > support for something IPMI-like. > >> 2G is also way too little memory these days for a new buildd. > > Then the machine is out, the amount of RAM isn't upgradable. I don't think 2GB is not enough for 32-bit machine. I see armel is already not a candidate for buster [0]. So it seems we can discuss armhf, but no armel at all. I don't agree with this idea. And I think we should treat armel and armhf equally. Both armel and armhf are working fine are millions of boards and embedded devices, and have stable quality [1]. They deserve the support from a community driven distro. [0] https://release.debian.org/buster/arch_qualify.html [1] https://lists.debian.org/debian-arm/2017/11/msg00061.html Cheers, -- Roger Shimizu, GMT +9 Tokyo PGP/GPG: 4096R/6C6ACD6417B3ACB1
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
Hello Julien, On 06/29/2018 11:23 AM, Julien Cristau wrote: >> If the concerns are mostly about the hardware not being rackable, there >> is a rackable NAS by Netgear: >> >> >> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs >> >> with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 >> GiB) are good enough. The machine can run mainline Linux[1]. I think >> U-Boot doesn't support this machine in mainline though. >> > Rackable, while good, is only part of it. The main part is remote > management. I'm not seeing any mention of ipmi or anything like that in > the datasheet? you can access the serial console, but I don't think there is built-in support for something IPMI-like. > 2G is also way too little memory these days for a new buildd. Then the machine is out, the amount of RAM isn't upgradable. Best regards Uwe signature.asc Description: OpenPGP digital signature
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Fri, Jun 29, 2018 at 12:50 PM, Julien Cristau wrote: > Everyone, please avoid followups to debian-po...@lists.debian.org. > Unless something is relevant to *all* architectures (hint: discussion of > riscv or arm issues don't qualify), keep replies to the appropriate > port-specific mailing list. apologies, julien: as an outsider i'm not completely familiar with the guidelines. the reduction in memory-usage at the linker phase "-Wl,--no-keep-memory" however - and the associated inherent slowly-inexorably-increasing size is i feel definitely something that affects all ports. it is really *really* tricky to get any kind of traction *at all* with people on this. it's not gcc's problem to solve, it's not one package's problem to solve, it's not any one distros problem to solve, it's not any one port's problem to solve and so on, *and* it's a slow-burn problem that's taking *literally* a decade to become more and more of a problem. consequently getting reports and feedback to the binutils team is... damn hard. l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On 06/27/2018 10:03 PM, Niels Thykier wrote: > Hi, > > > As part of the interim architecture qualification for buster, we request > that DSA, the security team and the toolchain maintainers review and > update their list of known concerns for buster release architectures. > Everyone, please avoid followups to debian-po...@lists.debian.org. Unless something is relevant to *all* architectures (hint: discussion of riscv or arm issues don't qualify), keep replies to the appropriate port-specific mailing list. Thanks, Julien
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 12:06 PM, John Paul Adrian Glaubitz wrote: > On 06/29/2018 10:41 AM, Luke Kenneth Casson Leighton wrote: >> On Fri, Jun 29, 2018 at 8:16 AM, Uwe Kleine-König >> wrote: >> In short, the hardware (development boards) we're currently using to build armel and armhf packages aren't up to our standards, and we really, really want them to go away when stretch goes EOL (expected in 2020). We urge arm porters to find a way to build armhf packages in VMs or chroots on server-class arm64 hardware. >> >> from what i gather the rule is that the packages have to be built >> native. is that a correct understanding or has the policy changed? > > Native in the sense that the CPU itself is not emulated which is the case > when building arm32 packages on arm64. ok. that's clear. thanks john. > I think that building on arm64 after fixing the bug in question is the > way to move forward. I'm surprised the bug itself hasn't been fixed yet, > doesn't speak for ARM. if you mean ARM hardware (OoO), it's too late. systems are out there with OoO speculative execution bugs in the hardware (and certainly more to be found), and they're here to stay unfortunately. if you mean that buildd on 32-bit systems could be modified to pass "-Wl,--no-keep-memory" to all linker phases to see if that results in the anticipated dramatic reduction in memory usage, that's straightforward to try, nothing to do with ARM themselves. l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Fri, Jun 29, 2018 at 12:23 PM, Adam D. Barratt wrote: >> i don't know: i'm an outsider who doesn't have the information in >> short-term memory, which is why i cc'd the debian-riscv team as they >> have current facts and knowledge foremost in their minds. which is >> why i included them. > > It would have been wiser to do so *before* stating that nothing was > happening as if it were a fact. true... apologies. >> ah. so what you're saying is, you could really do with some extra >> help? > > I don't think that's ever been in dispute for basically any core team > in Debian. :) > That doesn't change the fact that the atmosphere around the change in > question has made me feel very uncomfortable and unenthused about SRM > work. (I realise that this is somewhat of a self-feeding energy > monster.) i hear ya. l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Fri, 2018-06-29 at 11:44 +0100, Luke Kenneth Casson Leighton wrote: [...] > On Fri, Jun 29, 2018 at 10:35 AM, Adam D. Barratt > wrote: > > > > what is the reason why that package is not moving forward? > > > > I assume you're referring to the dpkg upload that's in proposed- > > updates > > waiting for the point release in two weeks time? > > i don't know: i'm an outsider who doesn't have the information in > short-term memory, which is why i cc'd the debian-riscv team as they > have current facts and knowledge foremost in their minds. which is > why i included them. It would have been wiser to do so *before* stating that nothing was happening as if it were a fact. > > I'm also getting very tired of the repeated vilification of SRM > > over > > this, and if there were any doubt can assure you that it is not > > increasing at least my inclination to spend my already limited free > > time on Debian activity. > > ah. so what you're saying is, you could really do with some extra > help? I don't think that's ever been in dispute for basically any core team in Debian. That doesn't change the fact that the atmosphere around the change in question has made me feel very uncomfortable and unenthused about SRM work. (I realise that this is somewhat of a self-feeding energy monster.) Regards, Adam
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On 06/29/2018 10:41 AM, Luke Kenneth Casson Leighton wrote: > On Fri, Jun 29, 2018 at 8:16 AM, Uwe Kleine-König > wrote: > >>> In short, the hardware (development boards) we're currently using to >>> build armel and armhf packages aren't up to our standards, and we >>> really, really want them to go away when stretch goes EOL (expected in >>> 2020). We urge arm porters to find a way to build armhf packages in >>> VMs or chroots on server-class arm64 hardware. > > from what i gather the rule is that the packages have to be built > native. is that a correct understanding or has the policy changed? Native in the sense that the CPU itself is not emulated which is the case when building arm32 packages on arm64. We're also building i386 packages on amd64 and we used to build powerpc packages on ppc64 (and we will continue to do that once the move of powerpc to ports has been completed). I think that building on arm64 after fixing the bug in question is the way to move forward. I'm surprised the bug itself hasn't been fixed yet, doesn't speak for ARM. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
--- crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68 On Fri, Jun 29, 2018 at 10:35 AM, Adam D. Barratt wrote: >> what is the reason why that package is not moving forward? > > I assume you're referring to the dpkg upload that's in proposed-updates > waiting for the point release in two weeks time? i don't know: i'm an outsider who doesn't have the information in short-term memory, which is why i cc'd the debian-riscv team as they have current facts and knowledge foremost in their minds. which is why i included them. > I'm also getting very tired of the repeated vilification of SRM over > this, and if there were any doubt can assure you that it is not > increasing at least my inclination to spend my already limited free > time on Debian activity. ah. so what you're saying is, you could really do with some extra help? l.
armel/armhf arch qualification for buster: call for DSA, Security, toolchain concernsj
Quoting Uwe Kleine-König : If the concerns are mostly about the hardware not being rackable, there is a rackable NAS by Netgear: https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs This seems to be out of stock and discontinued, unfortunately. Anyway, I'm relatively sure, that I can convince my boss to sponsor/donate both armel and armhf hardware for Debian, if that is of any help. Or arm64 used in "32 bits mode".
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Fri, 2018-06-29 at 10:20 +0100, Luke Kenneth Casson Leighton wrote: [...] > debian-riscv has been repeatedly asking for a single zero-impact > line > to be included in *one* file in *one* dpkg-related package which > would > allow riscv to stop being a NMU architecture and become part of > debian/unstable (and quickly beyond), for at least six months, now. > cc'ing the debian-riscv list because they will know the details about > this. it's really quite ridiculous that a single one-line change > having absolutely no effect on any other architecture whatsover is > not > being actioned and is holding debian-riscv back because of that. > > what is the reason why that package is not moving forward? I assume you're referring to the dpkg upload that's in proposed-updates waiting for the point release in two weeks time? Please check your facts before ranting, particularly with such a wide cross-posting. Also, ttbomk, the dpkg change landing in stable is not likely to magically lead to the architecture being added to unstable - a decision which is not the release team's to make or block. Again, please ensure you've actually done your research. I'm also getting very tired of the repeated vilification of SRM over this, and if there were any doubt can assure you that it is not increasing at least my inclination to spend my already limited free time on Debian activity. Regards, Adam
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
[s/debian-ports/debian-arm/] On 06/29/2018 09:16 AM, Uwe Kleine-König wrote: > Hello, > > On Wed, Jun 27, 2018 at 08:03:00PM +, Niels Thykier wrote: >> armel/armhf: >> >> >> * Undesirable to keep the hardware running beyond 2020. armhf VM >>support uncertain. (DSA) >>- Source: [DSA Sprint report] >> >> [DSA Sprint report]: >> https://lists.debian.org/debian-project/2018/02/msg4.html > > In this report Julien Cristau wrote: > >> In short, the hardware (development boards) we're currently using to >> build armel and armhf packages aren't up to our standards, and we >> really, really want them to go away when stretch goes EOL (expected in >> 2020). We urge arm porters to find a way to build armhf packages in >> VMs or chroots on server-class arm64 hardware. > > If the concerns are mostly about the hardware not being rackable, there > is a rackable NAS by Netgear: > > > https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs > > with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 > GiB) are good enough. The machine can run mainline Linux[1]. I think > U-Boot doesn't support this machine in mainline though. > Rackable, while good, is only part of it. The main part is remote management. I'm not seeing any mention of ipmi or anything like that in the datasheet? 2G is also way too little memory these days for a new buildd. Cheers, Julien
Re: Arch qualification for buster: call for DSA, Security, toolchain concerns
On Wed, Jun 27, 2018 at 9:03 PM, Niels Thykier wrote: > armel/armhf: > > > * Undesirable to keep the hardware running beyond 2020. armhf VM >support uncertain. (DSA) >- Source: [DSA Sprint report] [other affected 32-bit architectures removed but still relevant] ... i'm not sure how to put this other than to just ask the question. has it occurred to anyone to think through the consequences of not maintaining 32-bit versions of debian for the various different architectures? there are literally millions of ARM-based tablets and embedded systems out there which will basically end up in landfill if a major distro such as debian does not take a stand and push back against the "well everything's going 64-bit so why should *we* bother?" meme. arm64 is particularly inefficient and problematic compared to aarch32: the change in the instruction set resulted in dropping some of the more efficiently-encoded instructions that extend a 64-bit program size, require a whopping FIFTY PERCENT instruction-cache size increase to compensate for, whch increased power consumption by over 15%. in addition, arm64 is usually speculative OoO (Cavium ThunderX V1 being a notable exception) which means it's vulnerable to spectre and meltdown attacks, whereas 32-bit ARM is exclusively in-order. if you want to GUARANTEE that you've got spectre-immune hardware you need either any 32-bit system (where even Cortex A7 has virtualisattion) or if 64-bit is absolutely required use Cortex A53. basically, abandoning or planning to abandon 32-bit ARM *right now* leaves security-conscious end-users in a really *really* dicey position. > We are currently unaware of any new architectures likely to be ready in > time for inclusion in buster. debian-riscv has been repeatedly asking for a single zero-impact line to be included in *one* file in *one* dpkg-related package which would allow riscv to stop being a NMU architecture and become part of debian/unstable (and quickly beyond), for at least six months, now. cc'ing the debian-riscv list because they will know the details about this. it's really quite ridiculous that a single one-line change having absolutely no effect on any other architecture whatsover is not being actioned and is holding debian-riscv back because of that. what is the reason why that package is not moving forward? l.
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
On Fri, Jun 29, 2018 at 8:16 AM, Uwe Kleine-König wrote: > Hello, > > On Wed, Jun 27, 2018 at 08:03:00PM +, Niels Thykier wrote: >> armel/armhf: >> >> >> * Undesirable to keep the hardware running beyond 2020. armhf VM >>support uncertain. (DSA) >>- Source: [DSA Sprint report] >> >> [DSA Sprint report]: >> https://lists.debian.org/debian-project/2018/02/msg4.html > > In this report Julien Cristau wrote: > >> In short, the hardware (development boards) we're currently using to >> build armel and armhf packages aren't up to our standards, and we >> really, really want them to go away when stretch goes EOL (expected in >> 2020). We urge arm porters to find a way to build armhf packages in >> VMs or chroots on server-class arm64 hardware. from what i gather the rule is that the packages have to be built native. is that a correct understanding or has the policy changed? > > If the concerns are mostly about the hardware not being rackable, there > is a rackable NAS by Netgear: > > > https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs > > with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 > GiB) are good enough. no matter how much RAM there is it's never going to be "enough", and letting systems go into swap is also not a viable option [2] i've been endeavouring to communicate the issue for many many years wrt building (linking) of very large packages, for a long, *long* time. as it's a strategic cross-distro problem that's been very very slowly creeping up on *all* distros as packages inexorably creep up in size, reaching people about the problem and possible solutions is extremely difficult. eventually i raised a bug on binutils and it took several months to communicate the extent and scope of the problem even to the developer of binutils: https://sourceware.org/bugzilla/show_bug.cgi?id=22831 the problem is that ld from binutils by default, unlike gcc which looks dynamically at how much RAM is available, loads absolutely all object files into memory and ASSUMES that swap space is going to take care of any RAM deficiencies. unfortunately due to the amount of cross-referencing that takes place in the linker phase this "strategy" causes MASSIVE thrashing, even if one single object file is sufficient to cause swapping. this is particularly pertinent for systems which compile with debug info switched on as it is far more likely that a debug compile will go into swap, due to the amount of RAM being consumed. firefox now requires 7GB of resident RAM, making it impossible to compile on 32-bit systems webkit-based packages require well over 2GB RAM (and have done for many years). i saw one scientific package a couple years back that could not be compiled for 32-bit systems either. all of this is NOT the fault of the PACKAGES [1], it's down to the fact that *binutils* - ld's default memory-allocation strategy - is far too aggressive. the main developer of ld has this to say: Please try if "-Wl,--no-keep-memory" works. now, that's *not* a good long-term "solution" - it's a drastic, drastic hack that cuts the optimisation of keeping object files in memory stone dead. it'll work... it will almost certainly result in 32-bit systems being able to successfully link applications that previously failed... but it *is* a hack. someone really *really* needs to work with the binutils developer to *properly* solve this. if any package maintainer manages to use the above hack to successfully compile 32-bit packages that previously completely ran out of RAM or otherwise took days to complete, please do put a comment to that effect in the binutiols bugreport, it will help everyone in the entire GNU/Linux community to do so. l. [1] really, it is... developers could easily split packages into dynamic-loadable modules, where each module easily compiles well below even 2GB or 1GB of RAM. they choose not to, choosing instead to link hundreds of object files into a single executable (or library). asking so many developers to change their strategy however... yyeah :) big task, i ain't taking responsibility for that one. [2] the amount of memory being required for the linker phase of large packages over time goes up, and up, and up, and up... when is it going to stop? never. so just adding more RAM is never going to "solve" the problem, is it? it just *avoids* the problem. letting even 64-bit systems go into swap is a huge waste of resources as builds that go into swap will consume far more resources and time. so *even on 64-bit systems* this needs solving.
Re: Arch qualification for buster: call for DSA, Security, toolchain concernsj
Hello, On Wed, Jun 27, 2018 at 08:03:00PM +, Niels Thykier wrote: > armel/armhf: > > > * Undesirable to keep the hardware running beyond 2020. armhf VM >support uncertain. (DSA) >- Source: [DSA Sprint report] > > [DSA Sprint report]: > https://lists.debian.org/debian-project/2018/02/msg4.html In this report Julien Cristau wrote: > In short, the hardware (development boards) we're currently using to > build armel and armhf packages aren't up to our standards, and we > really, really want them to go away when stretch goes EOL (expected in > 2020). We urge arm porters to find a way to build armhf packages in > VMs or chroots on server-class arm64 hardware. If the concerns are mostly about the hardware not being rackable, there is a rackable NAS by Netgear: https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2 GiB) are good enough. The machine can run mainline Linux[1]. I think U-Boot doesn't support this machine in mainline though. Apart from that the people in #debian-arm (e.g. Sledge) seem to be positive that at least armhf should be fine to be built on arm64 hardware. Best regards Uwe [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/armada-xp-netgear-rn2120.dts signature.asc Description: PGP signature
