On Thu, 7 Oct 2004 09:05:55 +0200, Evan Morris
> I have a text file containing strings. The text file is pretty massive,
> about 895 MB. I need to load the words in the text file into the database in
> such a way that there is a single occurrence of each word in the table. 

I'm glad you found that the indexes worked for you.  I was curious if
your file already had one word per line?  If so, you could do a cat |
sort | uniq to pre-process the file and rid yourself of any duplicates
to eliminate the need for the check altogether.

I think that getting the data into a ready-to-load format before
involving the database would be quicker overall.   Not to disparage
PHP, but I would use Perl for such an endeavor.  PHP seems to have a
lot of memory overhead associated with text parsing.

PHP Database Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Reply via email to