On 5/16/06, Bryan Harris <[EMAIL PROTECTED]> wrote:
If I'm reading in many-megabyte files, is it considered to be more efficient to read it into an array, then loop over the array? Or is reading a line at a time okay?
Processing a file one line at a time is always okay.
I realize the second will use more memory, but what's a few megabytes in today's computers?
A few megabytes here, some more megabytes there, pretty soon you're talking about a lot of memory. Remember, the file will take up even more space in memory than it does on disk. If your system runs low, some memory will be swapped out to disk—and oops now you're using the disk anyway.
I'm more worried about the OS having to go back to the disk a couple hundred-thousand times -- seems like it'd be hard on the disk.
Don't worry about being hard on the disk. Put it completely out of your mind. Perl and the OS take care of low-level crud like that, so you don't have to. (If I weren't telling you to forget about this, I'd mention that Perl and the OS read the file one "block" at a time into a memory buffer, and thus the disk doesn't do any extra work to allow your program to see the data one line at a time.) Hope this helps! --Tom Phoenix Stonehenge Perl Training -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] <http://learn.perl.org/> <http://learn.perl.org/first-response>
