I Clancy,
Have you tried to convert your content to an .ini file? I heard that the
php function parse_ini_file is very fast for reading this kind of
files, although for writing you have to use some sort of php class, that
can be slower than your method.
Depends what you trying to do I guess.
Regards
Clancy wrote:
I have been pondering whether it would be feasible to work with a 100,000 entry
index
file, and had put yesterday aside to do some timing tests. I first generated
some sample
index files of various lengths. Each entry consisted of a single line with the
form
ASDF;rhubarb, rhubarb,
where the ASDF is a randomly generated four character index, and the rest of
the line is
filling, which varies slightly in length and contents from line to line, just
in case
something tried to get smart and cache the line. The average lenth of the line
is about
80 bytes.
Then I wrote another program which read the file into an array, using the four
character
index as the key, and the filling as the contents, sorted the array, and then
rewrote it
to another file, reporting the elapsed time after each step.
My first version used fgets() to read the source file a line at a time, and
fwrite() to
write the new file. This version performed quite consistently, and took
approximately 1.3
seconds to read in a 100,000 entry 7.86Mb file, and another 5 seconds to write
it out
again.
I then read the discussion following fschnittke's post File write operation
slows to a
crawl ... and wondered if the suggestions made there would help.
First I used file() to read the entire file into memory, then processed each
line into the
form required to set up my matrix. This gave a useful improvement for small
files, halving
the time required to read and process a 10,000 entry 815 kB file, but for a
30,000 entry
file it had dropped to about 15%, and it made little difference for a 300,000
entry file.
Then I tried writing my whole array into a single horrendous string, and using
file_put_contents() to write out the whole string in one bang. I started
testing on a
short file, and thought I was onto a good thing, as it halved the time to write
out a
10,000 entry 800 K file. But as I increased the file size it began to fail
dismally. With
a 30,000 entry file it was 20% slower, and at 100,000 entries it was three
times slower.
On Shawn McKenzie's suggestion, I also tried replacing fgets() with
stream_get_line(). As
I had anticipated any difference was well within below the timing noise level.
In conclusion, for short (1MB!) files, using file() to read the whole file into
memory is
substantially better than using fgets() to read the file a line at a time, but
the
advantage rapidly diminishes for longer files. Similarly using
file_put_contents() in
place of fwrite() to write it out again is better for short files (up to
perhaps 1 MB) but
the performance deteriorates rapidly above this.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
