On Mon, Jul 30, 2012 at 09:30:18AM +0100, Nicholas Clark wrote:
> With QAST and PIRT merged, I can build Rakudo on a Raspberry Pi in about 12
> hours, albeit with a lot of swapping (to a real disk)
> This was NQP at 6bd8bc5eb0f7cd29, Rakudo at c1bfbbb233c86036.
With NQP now using QAST, it now takes about 10 hours.
(NQP at 539f35bfb326c482, Rakudo d776f087d053fb3a, Parrot 29796c6a158575e8)
> > So, I commandeered a mobile phone charger to power an external USB hard
> > drive,
> > and now with 512Mb of swap tried this:
> >
> > $ /usr/bin/time -v ./perl6 --setting=NULL --optimize=3 --target=pir
> > --stagestats --output=src/gen/CORE.setting.pir src/gen/CORE.setting; sleep
> > 30; killall vmstat
> > Stage parse: 52535.669
> > Stage syntaxcheck: 0.387
> > Stage past: 0.221
> > Stage optimize: 9936.757
>
> So those were the times a month ago. Following Jonathan's excellent work on
> QAST and PIRT, the numbers now are:
>
> The following step can take a long time, please be patient.
> ./perl6 --setting=NULL --optimize=3 --target=pir --stagestats
> --output=src/gen/C
> ORE.setting.pir src/gen/CORE.setting
> Stage start : 0.000
> Stage parse : 5220.855
> Stage syntaxcheck: 0.048
> Stage past : 0.015
> Stage optimize : 14076.714
> Stage post : 11584.416
> Stage pir : 1077.496
Stage start : 0.000
Stage parse : 14896.354
Stage syntaxcheck: 0.079
Stage past : 0.059
Stage optimize : 1433.099
Stage post : 8498.762
Stage pir : 479.165
Notice that the relative times spent in the various stages is very
different from before, and from a machine that isn't swapping:
> The relative time breakdown is interesting. Here's the output on an 8Gb
> FreeBSD machine, which (probably) isn't swapping:
>
> Stage start : 0.000
> Stage parse : 258.672
> Stage syntaxcheck: 0.000
> Stage past : 0.000
> Stage optimize : 10.648
> Stage post : 88.219
> Stage pir : 20.760
Stage start : 0.000
Stage parse : 242.209
Stage syntaxcheck: 0.000
Stage past : 0.000
Stage optimize : 12.577
Stage post : 59.713
Stage pir : 4.514
(whereas the parent of yesterday's QAST merge, b8c0683f8d91f3d2, shows this:
Stage start : 0.000
Stage parse : 256.579
Stage syntaxcheck: 0.000
Stage past : 0.000
Stage optimize : 9.599
Stage post : 70.484
Stage pir : 5.732
)
The FreeBSD machine seems to be using about 8% more RAM but taking 12% less
time to compile the setting now than immediately prior to the merge.
Output from /usr/bin/time --verbose on the Raspberry Pi isn't fantastically
enlightening, but it does record over a million page faults:
Command being timed: "./perl6 --setting=NULL --optimize=3 --target=pir
--stagestats --output=src/gen/CORE.setting.pir src/gen/CORE.setting"
User time (seconds): 4928.15
System time (seconds): 2556.30
Percent of CPU this job got: 29%
Elapsed (wall clock) time (h:mm:ss or m:ss): 7:01:59
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 196356
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1097505
Minor (reclaiming a frame) page faults: 29154940
Voluntary context switches: 1117520
Involuntary context switches: 4188169
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
I think that it maxed out at about 260M of swap used. An x86 FreeBSD machine
with enough RAM not be swapping has maximum RSS size of 527892K.
Anyway, yay! Progress. Thanks Jonathan and everyone else.
Nicholas Clark
PS Perl 5 takes about 2 hours to build and 4 hours to test on the same
hardware. :-) Well, not quite the same as it doesn't *need* special
arrangements for lots of swap.
