You should be able to read in an entire file of even several hundred megabytes, assuming the rest of the data you have in memory is not too excessively large. Most 32-bit operating systems leave a virtual address space of just under 2GB for each program. In that 2GB you fit your application program, some shared library code, the heap and stack space, etc. -- but there should still be more than 1GB for your data.

Even if your computer only has 128MB of RAM, with modern hard drives and virtual memory architecture... I'd figure on being able to handle files of around 200MB on smaller semi-modern systems, maybe as big as 1GB or more on high-end systems, while loading the whole thing into RAM.

Note that this may vary somewhat depending on the operating system you target.

As far as using RegEx, you could only use it on the portion of the file loaded into memory. You might consider doing that, and realize that if there is no match, then you only have a part of the block in memory, so read some more and try again...

On Dec 14, 2004, at 9:46 AM, Randy Padawer wrote:

Thanks for the comments. Well, the problem with reading the whole file at once is that some of these files can indeed be huge -- several megabytes. Here's what I'm doing... The text file is made up of many individual emails concatenated end-to-end. Since the emails may have originated from different ISP accounts, they may use different delimeters, i.e., ">From " or "X-Message-Info: " or "Return-Path: <" or "From [EMAIL PROTECTED]". I have written a simple program that will separate the huge file back into individual text files, one for each email. It works well if the email is from a single account (with its unique delimeter). However if, say, Apple's mail.app is pulling mail from multiple accounts, I can't "read from" any of those delimeters. Can I do the regex routine (something I don't quite yet understand... but will soon!) on a file I've not wholly read? Thanks again for your great help. -- Randy
_______________________________________________
use-revolution mailing list
[EMAIL PROTECTED]
http://lists.runrev.com/mailman/listinfo/use-revolution



-----------------------------------------------------------
Frank D. Engel, Jr.  <[EMAIL PROTECTED]>

$ ln -s /usr/share/kjvbible /usr/manual
$ true | cat /usr/manual | grep "John 3:16"
John 3:16 For God so loved the world, that he gave his only begotten Son, that whosoever believeth in him should not perish, but have everlasting life.
$




___________________________________________________________
$0 Web Hosting with up to 120MB web space, 1000 MB Transfer
10 Personalized POP and Web E-mail Accounts, and much more.
Signup at www.doteasy.com

_______________________________________________
use-revolution mailing list
[EMAIL PROTECTED]
http://lists.runrev.com/mailman/listinfo/use-revolution

Reply via email to