On Thu, 7 Oct 2004 09:05:55 +0200, Evan Morris
<[EMAIL PROTECTED]> wrote:
> I have a text file containing strings. The text file is pretty massive,
> about 895 MB. I need to load the words in the text file into the database in
> such a way that there is a single occurrence of each word in the table.
I'm glad you found that the indexes worked for you. I was curious if
your file already had one word per line? If so, you could do a cat |
sort | uniq to pre-process the file and rid yourself of any duplicates
to eliminate the need for the check altogether.
I think that getting the data into a ready-to-load format before
involving the database would be quicker overall. Not to disparage
PHP, but I would use Perl for such an endeavor. PHP seems to have a
lot of memory overhead associated with text parsing.
PHP Database Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php