----- Original Message ----- From: "flint" <fmill...@gmail.com>
To: "דניאל דנון" <danondan...@gmail.com>
Sent: Sunday, May 31, 2009 9:21 AM
Subject: Re: [PHP] mysql_query takes long time...

have you actually tried running the script against your data?

i'm running MySQL/PHP on an old P3/933Mhz box with only 384M memory... I had a situation just like yours. I had a text file of 600,000 lines, the script reads a line, does regex matches/replacements on it, and inserts to SQL. it took the script about 15 mins to process the entire file and put all the data into MySQL.

and remember that's on a P3/933 so a faster system will obviously speed that up significantly

don't worry about the time MySQL reports for an individual query. it is dependent on many factors. but once you open your SQL connection and start streaming data in it will actually go much faster than that. just run the script and give it a try... even add some debugging output, somethign like this maybe:

$counter = 0;
while (!feof($fhandle)) {
 $line = fgets($fhandle);
 if ($counter % 100 = 0) { echo $counter . " records done\n"; }

that will give you a printout every 100 records processed... a nice way to track progress.


----- Original Message ----- From: "דניאל דנון" <danondan...@gmail.com>
To: "PHP General List" <php-general@lists.php.net>
Sent: Sunday, May 31, 2009 7:18 AM
Subject: [PHP] mysql_query takes long time...

I've a file of about 500,000 lines, each line contains a string in variety of lengths, but no less then 3 characters and usually no more then.... 120.
average of about 80, and maximum of about 250.

I made a PHP script to fetch the data (using fgets), process it and insert
it to a MySQL database.

The problem is inserting to MySQL takes about 0.02 seconds, which looks like
nothing - but when you have 500,000 lines to insert...
The while goes like that:

fgets from file
x1 = some function about the string
x2 = some other function about the string
x3 = the string
insert into table (field1, field2, field3) VALUES (x1, x2, x3)


I was wondering, is there any faster way to perform it, assuming I have to
do it with PHP?
also, if it matters - the MySQL table got id in auto increment.

Yours, Daniel.

Use ROT26 for best security

PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Reply via email to