[phpug] Large text file handling

2009-01-06 Thread Michael
Now that I have the other problem fixed I want to revisit the issue of large file handling - This is the code (placed within an array loop) - -- // Determine MIME type: $mt=MIME_Type::autoDetect($filename); // If suitable MIME type open, read and process: if

[phpug] Re: Large text file handling

2009-01-06 Thread Matthew White
Whats your memory_limit set to in php.ini - looks like 768MB from your error message. You might want to set it to -1 which is no memory limit and see what happens.. Try phpinfo as well. Matt On Wed, Jan 7, 2009 at 1:10 PM, Michael mich...@networkstuff.co.nz wrote: Now that I have the other

[phpug] Re: Large text file handling

2009-01-06 Thread Matthew White
which is line 124 On Wed, Jan 7, 2009 at 1:26 PM, Michael mich...@networkstuff.co.nz wrote: On Wed, 07 Jan 2009 13:21:51 Matthew White wrote: Whats your memory_limit set to in php.ini - looks like 768MB from your error message. You might want to set it to -1 which is no memory limit and

[phpug] Re: Large text file handling

2009-01-06 Thread Rory Casey
You need to parse large text files in chunks. I think someone on your previous thread outlined a promising strategy for this. --~--~-~--~~~---~--~~ NZ PHP Users Group: http://groups.google.com/group/nzphpug To post, send email to nzphpug@googlegroups.com To

[phpug] Re: Large text file handling

2009-01-06 Thread Steve Boyd
$fh = fopen('data.txt'); while ($line = fgets($fh)) { preg_match(...,$line); ... } fclose($fh); far as i know this chunks the text file on a per line basis. if not fopen('data.txt',4096); chunks its On Jan 7, 1:10 pm, Michael mich...@networkstuff.co.nz wrote: Now that I have the other

[phpug] Re: PHP script execution time

2009-01-06 Thread mak1e
Try bash scripting. On Jan 2, 11:25 am, Michael mich...@networkstuff.co.nz wrote: I have a PHP CLI script here that reads files, usually thousands or 10's of thousands in a batch, runs them through a preg_match and records results to a MySQL database. 5,000 - 10,000 files works OK, but go

[phpug] Re: Large text file handling

2009-01-06 Thread Michael
On Wed, 07 Jan 2009 15:41:44 Steve Boyd wrote: $fh = fopen('data.txt'); while ($line = fgets($fh)) { preg_match(...,$line); ... } fclose($fh); The issue is with this code, is while it reads the whole file, the array out put of preg_match_all is empty. (note that preg_match also returns