> On 31 Mar 2016, at 00:22, Tom Browder <tom.brow...@gmail.com> wrote: > On Tue, Mar 29, 2016 at 10:29 PM, Timo Paulssen <t...@wakelift.de> wrote: >> On 03/30/2016 03:45 AM, Timo Paulssen wrote: >> >> Could you try using $filename.IO.slurp.lines instead of $filename.IO.lines >> and see if that makes things any faster? >> - Timo >> >> >> Actually, the method on an IO::Handle is called "slurp-rest"; slurp would >> only work with a filename instead. >> - Timo > > Okay, I've done a comparison of the three methods on a 1 Gb file: > > IO.lines > real 2m11.827s > user 2m10.036s > sys 0m1.468s > > IO.split > real 1m51.504s > user 1m51.136s > sys 0m0.352s > > IO.slurp-rest > real 2m9.821s > user 2m6.268s > sys 0m3.532s > > and Perl 5: > > real 0m4.614s > user 0m4.328s > sys 0m0.280s
Just a sanity check: is this including the processing that you need to do, or just an empty loop reading lines? If the former, perhaps the overhead is not in the reading per se, but in the processing. Judging from the difference in your benchmark and mine, I would guess that only 40 seconds of the IO.lines case is actually involved in reading, and the other 90 are actually spent in processing. Perhaps you could do a —profile on a case that runs about 5 seconds or so, to get an idea of the bottleneck? Or could you gist the code that does the actual processing?? Liz
