Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-26 Thread Luke Kenneth Casson Leighton
On Mon, Jan 7, 2019 at 11:30 PM Mike Hommey wrote: > > it would be extremely useful to confirm that 32-bit builds can in fact > > be completed, simply by adding "-Wl no-keep-memory" to any 32-bit > > builds that are failing at the linker phase due to lack of memory. > > Note that Firefox is

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-09 Thread Luke Kenneth Casson Leighton
https://sourceware.org/bugzilla/show_bug.cgi?id=22831 sorry using phone to type, mike, comment 25 shows some important options to ld gold would it be possible to retry with those? 32 bit. Disabling mmap looks really important as clearly a 4gb+ binary is guaranteed going to fail to fit into 32bit

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-08 Thread Luke Kenneth Casson Leighton
On Tue, Jan 8, 2019 at 7:26 AM Luke Kenneth Casson Leighton wrote: > > On Tue, Jan 8, 2019 at 7:01 AM Luke Kenneth Casson Leighton > wrote: > trying this: > > $ python evil_linker_torture.py 3000 400 200 50 > > running with "make -j4" is going to take a few hours. ok so that did the

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
On Tue, Jan 8, 2019 at 7:01 AM Luke Kenneth Casson Leighton wrote: > i'm going to see if i can get above the 4GB mark by modifying the > Makefile to do 3,000 shared libraries instead of 3,000 static object > files. fail. shared libraries link extremely quickly. reverted to static, trying

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
$ python evil_linker_torture.py 3000 100 100 50 ok so that managed to get up to 1.8GB resident memory, paused for a bit, then doubled it to 3.6GB, and a few seconds later successfully outputted a binary. i'm going to see if i can get above the 4GB mark by modifying the Makefile to do 3,000

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
On Tue, Jan 8, 2019 at 6:27 AM Luke Kenneth Casson Leighton wrote: > i'm just running the above, will hit "send" now in case i can't hit > ctrl-c in time on the linker phase... goodbye world... :) $ python evil_linker_torture.py 2000 50 100 200 $ make -j8 oh, err... whoopsie... is this

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
$ python evil_linker_torture.py 2000 50 100 200 ok so it's pretty basic, and arguments of "2000 50 10 100" resulted in around a 10-15 second linker phase, which top showed to be getting up to around the 2-3GB resident memory range. "2000 50 100 200" should start to make even a system

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
On Tuesday, January 8, 2019, Mike Hommey wrote: > On Mon, Jan 07, 2019 at 11:46:41PM +, Luke Kenneth Casson Leighton > wrote: > > > At some point apps are going to become so insanely large that not even > > disabling debug info will help. > > That's less likely, I'd say. Debug info *is*

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Mike Hommey
On Mon, Jan 07, 2019 at 11:46:41PM +, Luke Kenneth Casson Leighton wrote: > On Tuesday, January 8, 2019, Mike Hommey wrote: > > > . > > > > Note that Firefox is built with --no-keep-memory > > --reduce-memory-overheads, and that was still not enough for 32-bts > > builds. GNU gold instead of

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
On Tuesday, January 8, 2019, Mike Hommey wrote: > . > > Note that Firefox is built with --no-keep-memory > --reduce-memory-overheads, and that was still not enough for 32-bts > builds. GNU gold instead of BFD ld was also given a shot. That didn't > work either. Presently, to make things link at

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Mike Hommey
On Mon, Jan 07, 2019 at 10:28:31AM +, Luke Kenneth Casson Leighton wrote: > On Sun, Jan 6, 2019 at 11:46 PM Steve McIntyre wrote: > > > > [ Please note the cross-post and respect the Reply-To... ] > > > > Hi folks, > > > > This has taken a while in coming, for which I apologise. There's a lot

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
(hi edmund, i'm reinstating debian-devel on the cc list as this is not a debian-arm problem, it's *everyone's* problem) On Mon, Jan 7, 2019 at 12:40 PM Edmund Grimley Evans wrote: > > i spoke with dr stallman a couple of weeks ago and confirmed that in > > the original version of ld that he

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Edmund Grimley Evans
> i spoke with dr stallman a couple of weeks ago and confirmed that in > the original version of ld that he wrote, he very very specifically > made sure that it ONLY allocated memory up to the maximum *physical* > resident available amount (i.e. only went into swap as an absolute > last resort),

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Steve McIntyre
On Mon, Jan 07, 2019 at 09:54:32AM +, Edmund Grimley Evans wrote: >The Haskell CP15 failures might be this: > >https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864847 > >Since it is claimed there that the CP15 instructions come from LLVM, >the Mono failures might have a very similar cause and

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Luke Kenneth Casson Leighton
On Sun, Jan 6, 2019 at 11:46 PM Steve McIntyre wrote: > > [ Please note the cross-post and respect the Reply-To... ] > > Hi folks, > > This has taken a while in coming, for which I apologise. There's a lot > of work involved in rebuilding the whole Debian archive, and many many > hours spent

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-07 Thread Edmund Grimley Evans
The Haskell CP15 failures might be this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864847 Since it is claimed there that the CP15 instructions come from LLVM, the Mono failures might have a very similar cause and solution.

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-06 Thread Steve McIntyre
On Mon, Jan 07, 2019 at 12:07:49AM +, peter green wrote: >On 06/01/19 23:45, Steve McIntyre wrote: >> In my initial testing for rebuilding armhf only, I did not enable >> either of these. I was then finding *lots* of "Illegal Instruction" >> crashes due to CP15 barrier usage in

Re: Rebuilding the entire Debian archive twice on arm64 hardware for fun and proft

2019-01-06 Thread peter green
On 06/01/19 23:45, Steve McIntyre wrote: In my initial testing for rebuilding armhf only, I did not enable either of these. I was then finding *lots* of "Illegal Instruction" crashes due to CP15 barrier usage in armhf Haskell and Mono programs. This suggests that the baseline