Re: A practical benchmark shows speed challenges for Perl 6

Perhaps the entity in the best position to figure out the optimum buffer size is the program itself. The information on file size, available memory, is there, so a calculation when the file is opened ought to yield the answer for best performance. On 4/2/16, yary wrote: > I

Re: A practical benchmark shows speed challenges for Perl 6

I think I understand the objections to the proposed environment variable now. I'm branching Rakudo, and attempting my first ever patch, in order to make handle-specific buffer size possible. Leaving in the env var, as that still has possible use... On Apr 2, 2016 8:47 AM, "Tom Browder"

Re: A practical benchmark shows speed challenges for Perl 6

On Thursday, March 31, 2016, Elizabeth Mattijsen wrote: > > On 31 Mar 2016, at 14:12, Tom Browder wrote: ... > > but I'll put the code on my github account later today. I'm sure it can be > tweaked, > but the gross differences between Perl 6 and Perl 5

Re: A practical benchmark shows speed challenges for Perl 6

Not sure I understand the disagreement. "the correct buffer size is often per file, not per program/invocation, so a one-size-fits-all envar is the wrong approach"- if you're saying "it would be great to have the buffer size be an option to 'open'," then I agree. It would be nice to have that

Re: A practical benchmark shows speed challenges for Perl 6

On Fri, Apr 1, 2016 at 11:09 AM, yary wrote: > Setting the buffer size is better done by the user, not the > programmer. Often the user and the programmer are one and the same, in > which case, the programmer knows the environment and can set the > environment variables- or

Re: A practical benchmark shows speed challenges for Perl 6

Actually I would characterize it as Before: The programmer had no control over the buffer size, and the user of the code had no way of adjusting the buffer to a particular system. Currently: The programmer has control over the buffer size, and the user of the code can adjust the buffer to a

Re: A practical benchmark shows speed challenges for Perl 6

On Fri, Apr 1, 2016 at 2:00 PM, Elizabeth Mattijsen wrote: > Sorry if I wasn’t clear: If there is no dynamic var, it will make one: > either from the environment, or set it to 64K (like it was before). So no > programmer action is ever needed if they’re not interested in that

Re: A practical benchmark shows speed challenges for Perl 6

> On 01 Apr 2016, at 13:50, Jan Ingvoldstad wrote: > > On Thu, Mar 31, 2016 at 10:36 AM, Elizabeth Mattijsen wrote: > > The reasoning behind _not_ setting things via environment variables, is > > that this means the programmer now needs to worry what e.g.

Re: A practical benchmark shows speed challenges for Perl 6

On Thu, Mar 31, 2016 at 10:36 AM, Elizabeth Mattijsen wrote: > > The reasoning behind _not_ setting things via environment variables, is > that this means the programmer now needs to worry what e.g. the webserver > running the Perl program does, and there are unknown stability

Re: A practical benchmark shows speed challenges for Perl 6

On Thursday, March 31, 2016, Elizabeth Mattijsen wrote: ... > Perhaps you could do a —profile on a case that runs about 5 seconds or so, > to get an idea of the bottleneck? > > Or could you gist the code that does the actual processing?? Okay, Liz, I'll work on the

Re: A practical benchmark shows speed challenges for Perl 6

On Mar 31, 2016, at 8:56 AM, Tom Browder > wrote: On Thursday, March 31, 2016, Elizabeth Mattijsen > wrote: > On 31 Mar 2016, at 14:12, Tom Browder > > wrote: >

Re: A practical benchmark shows speed challenges for Perl 6

On Thursday, March 31, 2016, Elizabeth Mattijsen wrote: > > On 31 Mar 2016, at 14:12, Tom Browder > wrote: > > Liz, it's a simple reader of a text file. The only line processing is a > print of the number of characters of each line. I

Re: A practical benchmark shows speed challenges for Perl 6

> On 31 Mar 2016, at 14:12, Tom Browder wrote: > Liz, it's a simple reader of a text file. The only line processing is a > print of the number of characters of each line. I guess I should eliminate > that but I assumed that was neglible since all the reader scripts do

A practical benchmark shows speed challenges for Perl 6

Liz, it's a simple reader of a text file. The only line processing is a print of the number of characters of each line. I guess I should eliminate that but I assumed that was neglible since all the reader scripts do the same. I put the code on pastebin earlier, but I'll put the code on my

Re: A practical benchmark shows speed challenges for Perl 6

> On 31 Mar 2016, at 09:50, Jan Ingvoldstad wrote: > > On Wed, Mar 30, 2016 at 9:20 PM, Elizabeth Mattijsen wrote: > > Thanks for your thoughts! > > I’ve implemented $*DEFAULT-READ-ELEMS in > https://github.com/rakudo/rakudo/commit/5bd1e . > > Of course,

Re: A practical benchmark shows speed challenges for Perl 6

On Wed, Mar 30, 2016 at 9:20 PM, Elizabeth Mattijsen wrote: > > > Thanks for your thoughts! > > I’ve implemented $*DEFAULT-READ-ELEMS in > https://github.com/rakudo/rakudo/commit/5bd1e . > > Of course, all of this is provisional, and open for debate and > bikeshedding. > >

Re: A practical benchmark shows speed challenges for Perl 6

> On 31 Mar 2016, at 00:22, Tom Browder wrote: > On Tue, Mar 29, 2016 at 10:29 PM, Timo Paulssen wrote: >> On 03/30/2016 03:45 AM, Timo Paulssen wrote: >> >> Could you try using $filename.IO.slurp.lines instead of $filename.IO.lines >> and see if that

Re: A practical benchmark shows speed challenges for Perl 6

On Tue, Mar 29, 2016 at 10:29 PM, Timo Paulssen wrote: > On 03/30/2016 03:45 AM, Timo Paulssen wrote: > > Could you try using $filename.IO.slurp.lines instead of $filename.IO.lines > and see if that makes things any faster? > - Timo > > > Actually, the method on an IO::Handle

Re: A practical benchmark shows speed challenges for Perl 6

On 03/30/2016 04:11 PM, yary wrote: On Wed, Mar 30, 2016 at 3:20 PM, Elizabeth Mattijsen wrote: Thanks for your thoughts! I’ve implemented $*DEFAULT-READ-ELEMS in https://github.com/rakudo/rakudo/commit/5bd1e . Of course, all of this is provisional, and open for debate and

Re: A practical benchmark shows speed challenges for Perl 6

On Wed, Mar 30, 2016 at 3:20 PM, Elizabeth Mattijsen wrote: > Thanks for your thoughts! > > I’ve implemented $*DEFAULT-READ-ELEMS in > https://github.com/rakudo/rakudo/commit/5bd1e . > > Of course, all of this is provisional, and open for debate and bikeshedding. Thanks! And

Re: A practical benchmark shows speed challenges for Perl 6

> On 30 Mar 2016, at 16:06, yary wrote: > > Cross-posting to the compiler group- > > On Wed, Mar 30, 2016 at 8:10 AM, Elizabeth Mattijsen wrote: >> If you know the line endings of the file, using >> IO::Handle.split($line-ending) (note the actual character,

Re: A practical benchmark shows speed challenges for Perl 6

On 30/03/16 13:40, Tom Browder wrote: On Tue, Mar 29, 2016 at 10:29 PM, Timo Paulssen wrote: On 03/30/2016 03:45 AM, Timo Paulssen wrote: Could you try using $filename.IO.slurp.lines instead of $filename.IO.lines and see if that makes things any faster? ... Actually, the

Re: A practical benchmark shows speed challenges for Perl 6

Cross-posting to the compiler group- On Wed, Mar 30, 2016 at 8:10 AM, Elizabeth Mattijsen wrote: > If you know the line endings of the file, using > IO::Handle.split($line-ending) (note the actual character, rather than a > regular expression) might help. That will read in

Re: A practical benchmark shows speed challenges for Perl 6

> On 30 Mar 2016, at 13:40, Tom Browder wrote: > On Tue, Mar 29, 2016 at 10:29 PM, Timo Paulssen wrote: >> On 03/30/2016 03:45 AM, Timo Paulssen wrote: >> >> Could you try using $filename.IO.slurp.lines instead of $filename.IO.lines >> and see if that

Re: A practical benchmark shows speed challenges for Perl 6

On Tue, Mar 29, 2016 at 10:29 PM, Timo Paulssen wrote: > On 03/30/2016 03:45 AM, Timo Paulssen wrote: > > Could you try using $filename.IO.slurp.lines instead of $filename.IO.lines > and see if that makes things any faster? ... > Actually, the method on an IO::Handle is called

Re: A practical benchmark shows speed challenges for Perl 6

On 03/30/2016 03:45 AM, Timo Paulssen wrote: > Could you try using $filename.IO.slurp.lines instead of > $filename.IO.lines and see if that makes things any faster? > - Timo Actually, the method on an IO::Handle is called "slurp-rest"; slurp would only work with a filename instead. - Timo

Re: A practical benchmark shows speed challenges for Perl 6

On 03/29/2016 10:47 PM, Tom Browder wrote: > On Wednesday, February 3, 2016, Tom Browder > wrote: > > I use Perl for heavy duty text processing. A question on Perl Monks > about Perl 5's handling of a large input file got me wondering

Re: A practical benchmark shows speed challenges for Perl 6

On 02/03/2016 02:59 PM, Tom Browder wrote: > I tried the suggestion from Bart Wiegmans to compile the program: > > $ perl6 --target=mbc --output=read-file-test.moarvm read-file-test.p6 > $ time perl6 read-file-test.moarvm large-1-gb-file.txt > Error while reading from file: Malformed UTF-8 > > So

Re: A practical benchmark shows speed challenges for Perl 6

On Wednesday, February 3, 2016, Tom Browder wrote: > I use Perl for heavy duty text processing. A question on Perl Monks > about Perl 5's handling of a large input file got me wondering how the > two Perls compare at the moment. I see no significant improvement using my

Re: A practical benchmark shows speed challenges for Perl 6

On Wed, Feb 3, 2016 at 9:40 AM, Elizabeth Mattijsen wrote: > Is the code available somewhere? Would love to try some optimizations on it. Liz, code is in public pastes in Pastebin. Links are here: 1. The master Perl 6 script to run the tests: http://pastebin.com/MDmumWq0

A practical benchmark shows speed challenges for Perl 6

I use Perl for heavy duty text processing. A question on Perl Monks about Perl 5's handling of a large input file got me wondering how the two Perls compare at the moment. I wrote a couple of simple programs, in both languages, to write and read a 10 Gb text file filled with identical

Re: A practical benchmark shows speed challenges for Perl 6

> On 03 Feb 2016, at 14:59, Tom Browder wrote: > I use Perl for heavy duty text processing. A question on Perl Monks > about Perl 5's handling of a large input file got me wondering how the > two Perls compare at the moment. > > I wrote a couple of simple programs, in both

Re: A practical benchmark shows speed challenges for Perl 6

On Wed, Feb 3, 2016 at 9:40 AM, Elizabeth Mattijsen wrote: >> On 03 Feb 2016, at 14:59, Tom Browder wrote: >> I use Perl for heavy duty text processing. A question on Perl Monks >> about Perl 5's handling of a large input file got me wondering how the >>