With QAST and PIRT merged, I can build Rakudo on a Raspberry Pi in about 12
hours, albeit with a lot of swapping (to a real disk)
This was NQP at 6bd8bc5eb0f7cd29, Rakudo at c1bfbbb233c86036.
On Thu, Jun 28, 2012 at 09:49:32PM +0100, Nicholas Clark wrote:
Still need this:
> It seems that dyncall doesn't recognise arm6el as a class of ARM architecture
> which it supports. Given that it supports iOS, and has a configure option for
> ARM, it seems pretty clear that it does, so I bodged it this this:
>
> nick@raspberrypi ~/Perl/rakudo/nqp $ git diff
> diff --git a/Configure.pl b/Configure.pl
> index 5539471..f1a1c00 100644
> --- a/Configure.pl
> +++ b/Configure.pl
> @@ -125,7 +125,7 @@ MAIN: {
> system_or_die('cd 3rdparty\dyncall && Configure.bat' .
> $configure_a
> $config{'dyncall_build'} = "cd 3rdparty/dyncall && $make
> BUILD_DIR=
> } else {
> - system_or_die('cd 3rdparty/dyncall && sh configure');
> + system_or_die('cd 3rdparty/dyncall && sh configure
> --target-arm-arm
> if ($^O eq 'netbsd') {
> $config{'dyncall_build'} = "cd 3rdparty/dyncall &&
> BUILD_DIR=.
> } else {
Still see this, and I really don't have time to dig into it:
> NQP fails 1 test:
>
> Test Summary Report
> -------------------
> t/nqp/60-bigint.t (Wstat: 0 Tests: 34 Failed: 3)
> Failed tests: 28-29, 34
> Files=64, Tests=584, 773 wallclock secs ( 5.03 usr 0.61 sys + 751.26 cusr
> 11.36 csys = 768.26 CPU)
Still need this:
> So, I commandeered a mobile phone charger to power an external USB hard drive,
> and now with 512Mb of swap tried this:
>
> $ /usr/bin/time -v ./perl6 --setting=NULL --optimize=3 --target=pir
> --stagestats --output=src/gen/CORE.setting.pir src/gen/CORE.setting; sleep
> 30; killall vmstat
> Stage parse: 52535.669
> Stage syntaxcheck: 0.387
> Stage past: 0.221
> Stage optimize: 9936.757
So those were the times a month ago. Following Jonathan's excellent work on
QAST and PIRT, the numbers now are:
The following step can take a long time, please be patient.
./perl6 --setting=NULL --optimize=3 --target=pir --stagestats --output=src/gen/C
ORE.setting.pir src/gen/CORE.setting
Stage start : 0.000
Stage parse : 5220.855
Stage syntaxcheck: 0.048
Stage past : 0.015
Stage optimize : 14076.714
Stage post : 11584.416
Stage pir : 1077.496
The relative time breakdown is interesting. Here's the output on an 8Gb
FreeBSD machine, which (probably) isn't swapping:
Stage start : 0.000
Stage parse : 258.672
Stage syntaxcheck: 0.000
Stage past : 0.000
Stage optimize : 10.648
Stage post : 88.219
Stage pir : 20.760
Note how "parse" is roughly twice the time of everything else. Whereas on
the Raspberry Pi, "otimize" is almost 3 times as bad as "parse", "post"
twice as bad, and even "pir" is 20% of it. I observed that vmstat was
showing only about 80% of time consumed by swapping at the end of "parse",
whereas as soon as "optimize" started it was 99% swap hell. Jonathan
commented on IRC that this will be because the optimiser is running all over
the parse tree in memory.
I didn't log enough to get the stat reliably, but maximum swap usage I could
spot in screen's scrollback of vmstat was 314792K, which seems plausible.
[Oh, "bug" report - the formatting assumes that no stage takes more than 999
seconds. That's two orders of magnitude optimistic :-)]
make test looks like this:
/usr/bin/perl t/harness t/00-parrot t/01-sanity
t/00-parrot/02-op-math.t ........... ok
t/00-parrot/03-op-logic.t .......... ok
t/00-parrot/01-literals.t .......... ok
t/00-parrot/04-op-cmp.t ............ ok
t/00-parrot/05-var.t ............... ok
t/00-parrot/06-op-inplace.t ........ No subtests run
t/00-parrot/08-var-array.t ......... ok
t/00-parrot/07-op-string.t ......... ok
t/00-parrot/09-pir.t ............... ok
t/01-sanity/01-tap.t ............... ok
t/01-sanity/02-counter.t ........... ok
t/00-parrot/10-regex.t ............. ok
t/01-sanity/03-equal.t ............. ok
t/01-sanity/05-sub.t ............... ok
t/01-sanity/04-if.t ................ ok
t/01-sanity/06-eqv.t ............... ok
t/01-sanity/08-simple-multisubs.t .. ok
t/01-sanity/07-isa.t ............... ok
t/01-sanity/09-end-blocks.t ........ ok
t/01-sanity/11-defined.t ........... ok
t/01-sanity/10-say.t ............... ok
t/01-sanity/12-try.t ............... ok
t/01-sanity/99-test-basic.t ........ ok
Test Summary Report
-------------------
t/00-parrot/06-op-inplace.t (Wstat: 11 Tests: 0 Failed: 0)
Non-zero wait status: 11
Parse errors: No plan found in TAP output
Files=23, Tests=241, 745 wallclock secs ( 2.37 usr 0.13 sys + 728.93 cusr 10.85
csys = 742.28 CPU)
Result: FAIL
I re-ran ./perl6 t/00-parrot/06-op-inplace.t and it passed. I ran it a few
more times and it passed. Then it SEGVd. I captured coredumps. backtraces
attached. The problem is always here:
Program terminated with signal 11, Segmentation fault.
#0 0x419f1150 in Rakudo_md_ct_dispatch (interp=0x67a,
dispatcher=<optimized out>, capture=<optimized out>, result=0x2c57d88)
at multidispatch.c:1007
1007 !STABLE(param)->type_check(interp, param,
type_obj)) {
(gdb) p param
$1 = (PMC * const) 0x2c200a8
(gdb) p *param
$2 = {flags = 43879896, vtable = 0x0, data = 0x4, _metadata = 0x0}
(gdb) p param
$1 = (PMC * const) 0x23340a8
(gdb) p *param
$2 = {flags = 34524632, vtable = 0x0, data = 0x4, _metadata = 0x0}
(gdb) p param
$1 = (PMC * const) 0x1acd0a8
(gdb) p *param
$2 = {flags = 25714136, vtable = 0x0, data = 0x4, _metadata = 0x0}
(gdb) printf "%x\n", param->flags
1885dd8
but as you can see, the addresses vary. I assume that there's Address Space
Layout Randomization here, and that it's tickling a GC bug. This also might
explain why earlier some things SEGVd but then didn't when re-run.
(This stuff is hard - Perl 5 has had assertion failures during destruction
exposed due to subtle changes in "random" ordering)
/usr/bin/time --verbose on Linux seems to give some plausible numbers.
In case they are useful, here they are:
Command being timed: "perl Configure.pl --gen-parrot=RELEASE_4_6_0
--gen-nqp"
User time (seconds): 9509.43
System time (seconds): 157.23
Percent of CPU this job got: 94%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:50:37
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 123484
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 259
Minor (reclaiming a frame) page faults: 4339170
Voluntary context switches: 22500
Involuntary context switches: 250309
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Command being timed: "make all"
User time (seconds): 9876.71
System time (seconds): 3285.74
Percent of CPU this job got: 36%
Elapsed (wall clock) time (h:mm:ss or m:ss): 10:04:34
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 200408
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1415979
Minor (reclaiming a frame) page faults: 34761380
Voluntary context switches: 1443684
Involuntary context switches: 5413424
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Command being timed: "make test"
User time (seconds): 736.84
System time (seconds): 11.34
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 12:30.80
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 58036
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 48
Minor (reclaiming a frame) page faults: 390696
Voluntary context switches: 1392
Involuntary context switches: 78429
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Oooh, bug? make test exits with zero even when tests fail.
So, be careful what you wish for:
> It would be wonderful if Rakudo's memory usage would drop. I think that that's
> going to be easier than finding a faster swap device.
Thanks Jonathan and everyone else. QAST and PIRT make a massive difference.
And I think that this demonstrates that it's just about possible to build
Rakudo on pretty much anything, if you're got some swap and you lack the
virtue of impatience. :-)
Nicholas Clark
(gdb) p param
$1 = (PMC * const) 0x2c200a8
(gdb) p *param
$2 = {flags = 43879896, vtable = 0x0, data = 0x4, _metadata = 0x0}
(gdb) where
#0 0x419f1150 in Rakudo_md_ct_dispatch (interp=0x67a,
dispatcher=<optimized out>, capture=<optimized out>, result=0x2c57d88)
at multidispatch.c:1007
#1 0x419e49e0 in Parrot_perl6_multi_dispatch_ct_p_p_p_p (
cur_opcode=0x217ea64, interp=0x18ea028) at perl6_ops.c:5915
#2 0x40323f64 in runops_fast_core (interp=0x18ea028,
runcore_unused=<optimized out>, pc=<optimized out>)
at src/runcore/cores.c:499
#3 0x40323bf8 in runops_int (interp=0x18ea028, offset=<optimized out>)
at src/runcore/main.c:220
#4 0x4030af30 in runops (interp=0x18ea028, offs=1) at src/call/ops.c:126
#5 0x4030680c in Parrot_pcc_invoke_from_sig_object (interp=0x18ea028,
sub_obj=<optimized out>, call_object=<optimized out>) at src/call/pcc.c:338
#6 0x402fa3ac in Parrot_ext_call (interp=0x18ea028, sub_pmc=0x1cf374c,
signature=0x404061a4 "P->") at src/extend.c:158
#7 0x403d7e50 in Parrot_Task_invoke (interp=0x18ea028, _self=0x18ea028,
next=0x404239f4) at src/pmc/task.c:166
#8 0x403067cc in Parrot_pcc_invoke_from_sig_object (interp=0x18ea028,
sub_obj=0x1cf3940, call_object=0x1cf3738) at src/call/pcc.c:330
#9 0x402fa3ac in Parrot_ext_call (interp=0x18ea028, sub_pmc=0x1cf3940,
signature=0x4040bcf0 "->") at src/extend.c:158
#10 0x40327e54 in Parrot_cx_next_task (interp=0x18ea028,
scheduler=<optimized out>) at src/scheduler.c:218
#11 0x40328568 in Parrot_cx_outer_runloop (interp=0x18ea028)
at src/scheduler.c:147
#12 0x403285e8 in Parrot_cx_begin_execution (interp=0x18ea028, main=0x198cf30,
argv=0x197ecfc) at src/scheduler.c:111
#13 0x40330264 in Parrot_pf_execute_bytecode_program (interp=0x18ea028,
pbc=<optimized out>, args=0x197ecfc) at src/packfile/api.c:2713
#14 0x402f6754 in Parrot_api_run_bytecode (interp_pmc=0x1978120,
pbc=0x404239f4, args=0x1978120) at src/embed/bytecode.c:161
#15 0x00008fa0 in main (argc=2, argv=0xbef22214) at perl6.c:614
(gdb) p param
$1 = (PMC * const) 0x23340a8
(gdb) p *param
$2 = {flags = 34524632, vtable = 0x0, data = 0x4, _metadata = 0x0}
(gdb) where
#0 0x419d4150 in Rakudo_md_ct_dispatch (interp=0x67a,
dispatcher=<optimized out>, capture=<optimized out>, result=0x236bd88)
at multidispatch.c:1007
#1 0x419c79e0 in Parrot_perl6_multi_dispatch_ct_p_p_p_p (
cur_opcode=0x1892a64, interp=0xffe028) at perl6_ops.c:5915
#2 0x401fef64 in runops_fast_core (interp=0xffe028,
runcore_unused=<optimized out>, pc=<optimized out>)
at src/runcore/cores.c:499
#3 0x401febf8 in runops_int (interp=0xffe028, offset=<optimized out>)
at src/runcore/main.c:220
#4 0x401e5f30 in runops (interp=0xffe028, offs=1) at src/call/ops.c:126
#5 0x401e180c in Parrot_pcc_invoke_from_sig_object (interp=0xffe028,
sub_obj=<optimized out>, call_object=<optimized out>) at src/call/pcc.c:338
#6 0x401d53ac in Parrot_ext_call (interp=0xffe028, sub_pmc=0x140774c,
signature=0x402e11a4 "P->") at src/extend.c:158
#7 0x402b2e50 in Parrot_Task_invoke (interp=0xffe028, _self=0xffe028,
next=0x402fe9f4) at src/pmc/task.c:166
#8 0x401e17cc in Parrot_pcc_invoke_from_sig_object (interp=0xffe028,
sub_obj=0x1407940, call_object=0x1407738) at src/call/pcc.c:330
#9 0x401d53ac in Parrot_ext_call (interp=0xffe028, sub_pmc=0x1407940,
signature=0x402e6cf0 "->") at src/extend.c:158
#10 0x40202e54 in Parrot_cx_next_task (interp=0xffe028,
scheduler=<optimized out>) at src/scheduler.c:218
#11 0x40203568 in Parrot_cx_outer_runloop (interp=0xffe028)
at src/scheduler.c:147
#12 0x402035e8 in Parrot_cx_begin_execution (interp=0xffe028, main=0x10a0f30,
argv=0x1092cfc) at src/scheduler.c:111
#13 0x4020b264 in Parrot_pf_execute_bytecode_program (interp=0xffe028,
pbc=<optimized out>, args=0x1092cfc) at src/packfile/api.c:2713
#14 0x401d1754 in Parrot_api_run_bytecode (interp_pmc=0x108c120,
pbc=0x402fe9f4, args=0x108c120) at src/embed/bytecode.c:161
#15 0x00008fa0 in main (argc=2, argv=0xbef86214) at perl6.c:614
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 2 259976 11852 44 2540 1671 31 2048 31 9114 1854 3 18 0 79
1 0 260128 13052 44 1512 998 102 1339 102 8795 1242 35 13 0 53
1 1 260744 11372 44 868 457 139 1989 139 9016 916 60 12 0 28
1 1 262184 11792 44 876 435 546 4989 546 10440 1799 44 17 0 39
0 3 262184 11612 44 948 408 477 4412 477 10380 1623 42 18 0 41
0 1 262176 17792 44 1472 435 472 3669 472 9940 1591 40 18 0 41
1 0 262068 13112 44 2576 540 0 638 0 8461 641 65 8 0 27
1 0 262184 12032 44 1088 970 97 1396 97 8802 1228 41 13 0 46
1 0 262180 14852 44 2724 1015 340 1906 340 9221 1825 10 16 0 75
0 1 262092 12152 44 1288 670 0 762 0 8488 793 57 9 0 33
1 1 262180 11612 44 868 458 173 2557 173 9221 1015 50 11 0 39
1 0 262172 30572 44 1476 502 608 2108 608 9348 1727 45 17 0 37
1 1 262104 24872 44 2652 747 0 856 0 8537 799 58 7 0 35
1 0 261896 20672 44 2652 693 0 693 0 8452 790 56 10 0 33
1 1 261564 15572 44 2652 906 0 906 0 8535 997 47 9 0 44
1 0 261092 14312 44 2652 926 0 926 0 8558 1066 36 13 0 51
0 1 261044 10472 44 2652 601 0 601 0 8397 669 66 6 0 28
0 1 262180 28352 44 1712 398 362 958 362 8792 1182 43 13 0 44
0 1 262160 23072 44 2528 741 0 810 0 8508 844 55 8 0 37
1 0 262064 19172 44 2528 566 0 566 0 8389 656 63 9 0 28
1 0 262184 11296 44 1024 1988 148 2536 148 9200 2211 20 16 0 64
2 2 262184 9228 44 1832 674 718 1756 718 9463 1838 21 24 0 55
2 1 262176 8332 44 3012 462 1577 584 1577 9906 3083 4 31 0 65
0 2 262220 8152 44 1744 1277 898 1780 898 10238 2321 3 28 0 70
3 1 262184 9180 44 1848 882 746 1282 746 9292 2110 2 24 0 74
1 2 263272 9268 44 1532 929 1437 1304 1437 10086 2494 1 28 0 72
0 2 266124 9636 44 1472 1524 1845 2070 1845 10411 3020 1 33 0 65
3 0 267232 8256 44 1496 1380 604 1750 604 9309 1874 0 24 0 76
0 1 269428 13836 44 1408 946 566 1299 566 9086 1914 3 20 0 77
0 1 282516 11948 44 684 1210 3069 2023 3069 10138 2561 6 34 0 60
1 1 314792 23380 60 13092 818 6723 2264 9002 14870 3280 5 42 0 54
1 0 9164 186716 152 19796 979 0 2260 0 9014 1117 31 18 0 50
1 0 9164 186732 152 19796 0 0 0 0 8132 31 100 0 0 0
1 0 9164 185660 152 19796 0 0 0 0 8132 31 100 0 0 0
1 0 9164 177080 152 20060 0 0 53 0 8147 34 99 1 0 0
2 0 9136 170240 152 20156 9 0 28 0 8143 54 99 1 0 0
1 0 9136 165140 152 20156 0 0 0 0 8132 31 99 1 0 0
1 0 9136 160760 152 20156 0 0 0 6 8076 33 99 1 0 0
1 0 9136 156800 152 20156 0 0 0 0 8099 32 99 1 0 0
1 0 9136 152780 152 20156 0 0 0 0 8132 30 100 0 0 0
1 0 9136 149060 152 20156 0 0 0 0 8132 31 99 1 0 0
1 0 9136 145760 152 20156 0 0 0 0 8132 30 100 0 0 0
1 0 9136 142640 152 20156 0 0 0 0 8132 30 99 1 0 0
1 0 9136 139640 152 20156 0 0 0 0 8131 31 99 1 0 0
1 0 9136 137120 152 20156 0 0 0 0 8132 30 100 0 0 0
1 0 9136 134120 152 20156 0 0 0 0 8132 31 100 0 0 0
1 0 9136 131540 152 20156 0 0 0 0 8132 31 99 1 0 0
1 0 9136 127460 152 20156 0 0 0 0 8132 30 99 1 0 0
1 0 9136 122480 152 20156 0 0 0 0 8132 30 99 1 0 0
1 0 9136 122120 152 20156 0 0 0 0 8131 30 100 0 0 0
1 0 9136 121820 152 20156 0 0 0 0 8132 30 100 0 0 0
1 0 9136 121580 152 20156 0 0 0 0 8132 31 99 1 0 0
1 0 9136 121400 152 20156 0 0 0 0 8132 31 100 0 0 0
1 0 9136 121220 152 20156 0 0 0 0 8132 30 100 0 0 0
1 0 9104 117920 292 22980 30 0 622 0 8254 143 98 2 0 0
1 0 9104 117860 300 22980 0 0 0 3 8093 35 100 0 0 0
1 0 9104 117740 300 22980 0 0 0 0 8132 31 100 0 0 0
1 0 9104 117620 300 22980 0 0 0 0 8132 30 100 0 0 0
1 0 9104 117500 300 22980 0 0 0 0 8132 31 100 0 0 0
