Bill Ricker wrote: > Tuesday December 10, 2013 E51-376 > Tim King - Benchmarking Perl6 vs Perl5 > > * Perl 6's suitability for "production." (Your definition may vary.) > * Experiences of the state and stability of Rakudo. > * Benchmarks of my favorite P6 features. > * Lines of P6 code from everyday life. > * Some comparisons of P6 with P5.
If you missed the talk, we're working on getting a version posted online. (Tim confirmed that the audio capture was successful, and the slides have been rendered to video. Need to edit the two together.) A PDF of the slides will also be posted. Here's an edited version of what I "live blogged" to our #boston.pm IRC room (*spoiler alert*): perl6-bench was used as the bench testing framework. It was created by Geoffrey Broadwell, and covered in his "The Need for Speed" talk at YAPC::NA 2013 http://bit.ly/P6BenchTalk What remains to make p6 production ready? According to Carl Mäsak: Features, concurrency, cpan, speed (a big one). "Production ready" is hard to define, much like quality. From a coder's perspective, will p6 code be elegant? Are the sorts of modules I need available? Will new p6 version break the code I write today? Will it be fast enough? Production ready is relative to what you want to use p6 for. The answer may be different for a LAN backup script vs. a web application. perl6-bench automates creating the sandboxes for each interpreter (p5, and several p6 versions). And automates running the tests and logging and graphing the times. The comparisons are intentionally excluding startup time. Still learning p6 and don't yet know how to optimize p6 code. NQP = Not Quite Perl. Used as the base language in some of the VMs. They compile P6 to NQP. First bench test is a trivial while loop. The graph shows p5 hitting near 33 million iterations per second. nqp-parrot reaches ~3M itr/s. While nqp-jvp, due to optimizations, eventually after enough iterations starts to run a bit faster than p5. nqp-jvm: just over 33 million iterations per second p5: just below 33 million iterations per second nqp-parrot: 3M itr/s rakudo-jvm: 300K itr/s rakudo-parrot: 65K itr/s The JVM lead holds true only under some circumstances. Incorporating a string concatenation operation causes nqp-jvm performance to plunge (peaking at only 65K itr/s, and dropping off with more cycles). p5: 8 million iterations per second nqp-parrot: 2M itr/s nqp-jvm: 65K itr/s (peak; not consistent as iteration count goes up) rakudo-parrot: 32K itr/s rakudo-jvm: 32K itr/s (peak; not consistent as iteration count goes up) Next benchmark is rc-forest-fire (cell automaton algorithm). p5: 512 frames/s (all numbers rounded to nearest line on chart) nqp-parrot: 64 fps niecza: 4 fps rakudo-parrot: 0.5 fps The test was repeated on a faster machine, adding more interpreters, but dropping niecza. nqp-jvm: 2048 fps (over a limited range of cycles) p5: 1024 fps nqp-parrot: 128 fps rakudo-jvm: 1 fps rakudo-parrot: 0.5 fps Next benchmark is fractal-tree, which generates a scalable vector graphic of a tree using a fractal algorithm. It is used to test object construction and method calls. P5: 65k branches per second rakudo-jvm: peaked at 128 branches/s Moose version of fractal tree hit near 8k branches/s. Strong typed version of fractal-tree in p6 is actually slightly slower than dynamic types. First version of fractal-tree in p6 iterated over arrays multiple times, while p5 code iterated once in a bigger loop. The big loop p5 approach was then ported to p6. Now p6 version peaks at 256 branches/s. Now for some more real-world benchmarks using common CPAN modules that have been ported to p6. Next benchmark is P5 vs. P6 versions of JSON::Tiny. P6 version uses Perl 6's custom grammar feature. P5: 64 iterations/s nqp-jvm: 8 itr/s rakudo-jvm: 1 itr/s nqp-parrot: 0.5 itr/s rakudo-parrot: 0.1 itr/s JSON::Tiny was next ported to Marpa::R2 in p5 to see how p5 performs while providing a parser syntax similar to a p6 grammar. JSON::Marpa: 8 itr/s Marpa matched nqp-jvm, though Marpa had more linear performance at low iteration counts (before the JVM has had a chance to optimize). Next benchmark is HTML::Template. p5: 256 pages per second rakudo-jvm: 4 pps rakudo-parrot: 0.5 pps Next is File::Compare. p5: 4096 MB/s rakudo-jvm: 128 MB/s rakudo-parrot: 16 MB/s So is p6 ready? Because Rakudo's correctness is questionable, when things don't work, it leaves you scratching your head wondering if it is your bug or Rakudo's. Stability is a really important metric of readiness. The perl6-bench couldn't work with the version of rakudo-parrot from just a few months ago. Only with the latest version. -Tom -- Tom Metro The Perl Shop, Newton, MA, USA "Predictable On-demand Perl Consulting." http://www.theperlshop.com/ _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm
