Re: read() vs fread()
On Friday 20 February 2009 21:07:57 Junsuk Shin wrote:
Hi BSD guys,
While I was doing simple file read test, I found that there is a huge
difference in file read performance between read() and fread(). I'm
wondering if I'm doing something wrong or if someone has experienced
similar things.
Here is what I did,
For the specific application, I need to bypass cache (I read only
once, and that's all)
The test file is 700Mbytes dummy file.
Test app just reads the whole file.
Test is done on FreeBSD 7.1 amd 64, Celeron E1200, WD Caviar SE16 SATA 7200
RPM
For test 1,
fd = open(name, O_RDONLY | O_DIRECT);
while(...) {
cnt = read();
}
for test 2,
fd = open(name, O_RDONLY | O_DIRECT);
file = fdopen(fd,r);
while(...) {
cnt = fread();
}
test 1 takes about 11.64 seconds (63 MBytes/s), and test 2 takes about
51.53 seconds (14 MBytes/s)
If I use the pair of fopen() and fread(), it will have cache effect,
so the result doesn't say much of hdd performance.
Personally, I don't think the overhead of fread() (wrapper in libc) is
that huge. What would be the reason for this?
The reason is that by default a FILE has a really small internal buffer. Take
a look at gstat(8) while running the test: you can clearly see an insane
amount of I/O requests being done (almost 5000 reads per second on my HDD).
To solve this call setvbuf(3):
setvbuf(file, buf, _IOFBF, bufsize);
A bufsize of 16k or bigger should help a lot. After this modification, I see
about 900 reads per second (using bufsize = 64k) and the read speed is equal
to the read(2) case.
Regards,
Pieter de Goeje
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: read() vs fread()
setvbuf(file, buf, _IOFBF, bufsize) solved the problem perfectly.
Thanks a lot.
On Fri, Feb 20, 2009 at 4:09 PM, Pieter de Goeje pie...@degoeje.nl wrote:
On Friday 20 February 2009 21:07:57 Junsuk Shin wrote:
Hi BSD guys,
While I was doing simple file read test, I found that there is a huge
difference in file read performance between read() and fread(). I'm
wondering if I'm doing something wrong or if someone has experienced
similar things.
Here is what I did,
For the specific application, I need to bypass cache (I read only
once, and that's all)
The test file is 700Mbytes dummy file.
Test app just reads the whole file.
Test is done on FreeBSD 7.1 amd 64, Celeron E1200, WD Caviar SE16 SATA 7200
RPM
For test 1,
fd = open(name, O_RDONLY | O_DIRECT);
while(...) {
cnt = read();
}
for test 2,
fd = open(name, O_RDONLY | O_DIRECT);
file = fdopen(fd,r);
while(...) {
cnt = fread();
}
test 1 takes about 11.64 seconds (63 MBytes/s), and test 2 takes about
51.53 seconds (14 MBytes/s)
If I use the pair of fopen() and fread(), it will have cache effect,
so the result doesn't say much of hdd performance.
Personally, I don't think the overhead of fread() (wrapper in libc) is
that huge. What would be the reason for this?
The reason is that by default a FILE has a really small internal buffer. Take
a look at gstat(8) while running the test: you can clearly see an insane
amount of I/O requests being done (almost 5000 reads per second on my HDD).
To solve this call setvbuf(3):
setvbuf(file, buf, _IOFBF, bufsize);
A bufsize of 16k or bigger should help a lot. After this modification, I see
about 900 reads per second (using bufsize = 64k) and the read speed is equal
to the read(2) case.
Regards,
Pieter de Goeje
--
Junsuk
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
