Re: A practical benchmark shows speed challenges for Perl 6
On Wed, Mar 30, 2016 at 3:20 PM, Elizabeth Mattijsenwrote: > Thanks for your thoughts! > > I’ve implemented $*DEFAULT-READ-ELEMS in > https://github.com/rakudo/rakudo/commit/5bd1e . > > Of course, all of this is provisional, and open for debate and bikeshedding. Thanks! And that was fast! Allowing DEFAULT-READ-ELEMS to be set from the environment's a good idea that I hadn't thought of- since it is a machine-dependent performance tweak, letting it be set outside the code is a good idea. I had originally envisioned this as an "option" to "sub open" for fine-grained control as to which IO::Handles got what DEFAULT-READ-ELEMS, but I'm not sure it belongs there. After all it is a performance-related tweak and I'm liking the idea of it being primarily set from the environment; setting it in the code means you're writing something for a particular host, don't need to change the spec to support that. Is there anything similar on the "write" side- output buffering- that could use this treatment? -y
Re: A practical benchmark shows speed challenges for Perl 6
> On 30 Mar 2016, at 16:06, yarywrote: > > Cross-posting to the compiler group- > > On Wed, Mar 30, 2016 at 8:10 AM, Elizabeth Mattijsen wrote: >> If you know the line endings of the file, using >> IO::Handle.split($line-ending) (note the actual character, rather than a >> regular expression) might help. That will read in the file in chunks of 64K >> and then lazily serve lines from that chunk. > > This reminds me of a pet peeve I had with p5: Inability to easily > change the default buffer size for reading & writing. > > I'm the lone Perl expert at $work and at one point was trying to keep > a file processing step in perl. These files were about 100x the size > of the server's RAM, consisted of variable-length newline-terminated > text, the processing was very light, there would be a few running in > parallel. The candidate language, C#, has a text-file-reading object > that lets you set its read-ahead buffer on creation/opening the file- > can't remember the details. That size had a large impact on the > performance of this task. With perl... I could not use the > not-so-well-documented IO::Handle->setvbuf because my OS didn't > support it. I did hack together something with sysread, but C# won in > the end due partly to that. > > It seems this "hiding-of-buffer" sub-optimal situation is being > repeated in Perl6: neither https://doc.perl6.org/routine/open nor > http://doc.perl6.org/type/IO::Handle mention a buffer, yet IO::Handle > reads ahead and buffers. Experience shows that being able to adjust > this buffer can help in certain situations. Also consider that perl5 > has defaulted to 4k and 8k, whereas perl6 is apparently using 64k, as > evidence that this buffer needs to change as system builds evolve. > > Please make this easily readable & settable, anywhere it's implemented! Thanks for your thoughts! I’ve implemented $*DEFAULT-READ-ELEMS in https://github.com/rakudo/rakudo/commit/5bd1e . Of course, all of this is provisional, and open for debate and bikeshedding. Liz
Re: A practical benchmark shows speed challenges for Perl 6
Cross-posting to the compiler group- On Wed, Mar 30, 2016 at 8:10 AM, Elizabeth Mattijsenwrote: > If you know the line endings of the file, using > IO::Handle.split($line-ending) (note the actual character, rather than a > regular expression) might help. That will read in the file in chunks of 64K > and then lazily serve lines from that chunk. This reminds me of a pet peeve I had with p5: Inability to easily change the default buffer size for reading & writing. I'm the lone Perl expert at $work and at one point was trying to keep a file processing step in perl. These files were about 100x the size of the server's RAM, consisted of variable-length newline-terminated text, the processing was very light, there would be a few running in parallel. The candidate language, C#, has a text-file-reading object that lets you set its read-ahead buffer on creation/opening the file- can't remember the details. That size had a large impact on the performance of this task. With perl... I could not use the not-so-well-documented IO::Handle->setvbuf because my OS didn't support it. I did hack together something with sysread, but C# won in the end due partly to that. It seems this "hiding-of-buffer" sub-optimal situation is being repeated in Perl6: neither https://doc.perl6.org/routine/open nor http://doc.perl6.org/type/IO::Handle mention a buffer, yet IO::Handle reads ahead and buffers. Experience shows that being able to adjust this buffer can help in certain situations. Also consider that perl5 has defaulted to 4k and 8k, whereas perl6 is apparently using 64k, as evidence that this buffer needs to change as system builds evolve. Please make this easily readable & settable, anywhere it's implemented! -y