David --- Senior Programmer Analyst --- WGO
[mailto:[EMAIL PROTECTED]
Sent: Tuesday, June 01, 2004 11:21 AM
To: Steven Manross; [EMAIL PROTECTED]
Subject: RE: parsing large file for character usage
[EMAIL PROTECTED] wrote:
> Can anyone seem to think of a less memory intensive way of doing this
&g
:0;
}
Tobias
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On
> Behalf Of Steven Manross
> Sent: Tuesday, June 01, 2004 12:08 PM
> To: [EMAIL PROTECTED]
> Subject: parsing large file for character usage
>
>
> Can anyone seem to t
[EMAIL PROTECTED] wrote:
> Can anyone seem to think of a less memory intensive way of doing this
> (besides chopping the file up into smaller chunks -- which is my next
> step unless I receive a better option?
>
> The txt file is 335MB.
>
> The column and row delimiter can change, but needs to be
The files I am dealing with can be upwards of 42,000 MB and larger. I
tried fancier ways of processing, with arrays and such, and blew out my
memory every time.
I finally had to re-write my entire script to process line by line,
holding almost nothing in memory. It is not pretty and not fancy,
Can anyone seem to think of a less memory intensive way of doing this
(besides chopping the file up into smaller chunks -- which is my next
step unless I receive a better option?
The txt file is 335MB.
The column and row delimiter can change, but needs to be an "odd
mutli-char delimiter" due to t
