> "Production ready" is hard to define, much like quality. Yeah, but "1000 times slower" is objective and pretty definitive.
> Bill Ricker wrote: >> Tuesday December 10, 2013 E51-376 >> Tim King - Benchmarking Perl6 vs Perl5 >> >> * Perl 6's suitability for "production." (Your definition may vary.) >> * Experiences of the state and stability of Rakudo. >> * Benchmarks of my favorite P6 features. >> * Lines of P6 code from everyday life. >> * Some comparisons of P6 with P5. > > If you missed the talk, we're working on getting a version posted > online. (Tim confirmed that the audio capture was successful, and the > slides have been rendered to video. Need to edit the two together.) A > PDF of the slides will also be posted. > > > Here's an edited version of what I "live blogged" to our #boston.pm IRC > room (*spoiler alert*): > > perl6-bench was used as the bench testing framework. It was created by > Geoffrey Broadwell, and covered in his "The Need for Speed" talk at > YAPC::NA 2013 > http://bit.ly/P6BenchTalk > > What remains to make p6 production ready? > According to Carl Mäsak: Features, concurrency, cpan, speed (a big one). > > "Production ready" is hard to define, much like quality. > > From a coder's perspective, will p6 code be elegant? Are the sorts of > modules I need available? Will new p6 version break the code I write > today? Will it be fast enough? > > Production ready is relative to what you want to use p6 for. The answer > may be different for a LAN backup script vs. a web application. > > perl6-bench automates creating the sandboxes for each interpreter (p5, > and several p6 versions). And automates running the tests and logging > and graphing the times. > > The comparisons are intentionally excluding startup time. Still learning > p6 and don't yet know how to optimize p6 code. > > NQP = Not Quite Perl. Used as the base language in some of the VMs. They > compile P6 to NQP. > > First bench test is a trivial while loop. The graph shows p5 hitting > near 33 million iterations per second. nqp-parrot reaches ~3M itr/s. > While nqp-jvp, due to optimizations, eventually after enough iterations > starts to run a bit faster than p5. > nqp-jvm: just over 33 million iterations per second > p5: just below 33 million iterations per second > nqp-parrot: 3M itr/s > rakudo-jvm: 300K itr/s > rakudo-parrot: 65K itr/s > > The JVM lead holds true only under some circumstances. Incorporating a > string concatenation operation causes nqp-jvm performance to plunge > (peaking at only 65K itr/s, and dropping off with more cycles). > p5: 8 million iterations per second > nqp-parrot: 2M itr/s > nqp-jvm: 65K itr/s (peak; not consistent as iteration count goes up) > rakudo-parrot: 32K itr/s > rakudo-jvm: 32K itr/s (peak; not consistent as iteration count goes up) > > Next benchmark is rc-forest-fire (cell automaton algorithm). > p5: 512 frames/s (all numbers rounded to nearest line on chart) > nqp-parrot: 64 fps > niecza: 4 fps > rakudo-parrot: 0.5 fps > > The test was repeated on a faster machine, adding more interpreters, but > dropping niecza. > nqp-jvm: 2048 fps (over a limited range of cycles) > p5: 1024 fps > nqp-parrot: 128 fps > rakudo-jvm: 1 fps > rakudo-parrot: 0.5 fps > > Next benchmark is fractal-tree, which generates a scalable vector > graphic of a tree using a fractal algorithm. It is used to test object > construction and method calls. > P5: 65k branches per second > rakudo-jvm: peaked at 128 branches/s > > Moose version of fractal tree hit near 8k branches/s. > > Strong typed version of fractal-tree in p6 is actually slightly slower > than dynamic types. > > First version of fractal-tree in p6 iterated over arrays multiple times, > while p5 code iterated once in a bigger loop. The big loop p5 approach > was then ported to p6. Now p6 version peaks at 256 branches/s. > > Now for some more real-world benchmarks using common CPAN modules that > have been ported to p6. > > Next benchmark is P5 vs. P6 versions of JSON::Tiny. P6 version uses Perl > 6's custom grammar feature. > P5: 64 iterations/s > nqp-jvm: 8 itr/s > rakudo-jvm: 1 itr/s > nqp-parrot: 0.5 itr/s > rakudo-parrot: 0.1 itr/s > > JSON::Tiny was next ported to Marpa::R2 in p5 to see how p5 performs > while providing a parser syntax similar to a p6 grammar. > JSON::Marpa: 8 itr/s > > Marpa matched nqp-jvm, though Marpa had more linear performance at low > iteration counts (before the JVM has had a chance to optimize). > > Next benchmark is HTML::Template. > p5: 256 pages per second > rakudo-jvm: 4 pps > rakudo-parrot: 0.5 pps > > Next is File::Compare. > p5: 4096 MB/s > rakudo-jvm: 128 MB/s > rakudo-parrot: 16 MB/s > > So is p6 ready? > Because Rakudo's correctness is questionable, when things don't work, it > leaves you scratching your head wondering if it is your bug or Rakudo's. > > Stability is a really important metric of readiness. The perl6-bench > couldn't work with the version of rakudo-parrot from just a few months > ago. Only with the latest version. > > > -Tom > > -- > Tom Metro > The Perl Shop, Newton, MA, USA > "Predictable On-demand Perl Consulting." > http://www.theperlshop.com/ > > _______________________________________________ > Boston-pm mailing list > [email protected] > http://mail.pm.org/mailman/listinfo/boston-pm > -- _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm
