brain wrote:
* Phil Carns <[EMAIL PROTECTED]> [2008 10 17, 10:46]:
I have two configuration suggestions that you can try.
Hello Phil, hello everybody,
I followed your suggestions and I verified that adding
TroveMethod alt-aio
*and* increasing the ServerJobFlowTimeoutSecs to 300 I don't get any
error, even under heavy load. Now I am wondering if the performances I
get are acceptable or I am just 'hiding' the underlying problem.
So, I first tried to write to the ext3 slice directly, from every
single host, without using pvfs2, with the following command:
dd if=/dev/zero of="bigfile" bs=4K count=2000000
It gave me an average of 24 Mb/s with 6 hosts writing at the same time;
with a single host writing the value is 59 Mb/s.
The same test using pvfs2 resulted in an average 5.7 Mb/s with 5 hosts
writing (on a total of 6 hosts: 1 metadata, 5 I/O), which means 6 times
slower than direct (but parallel) I/O. Is that the "price to pay" :)
for the distributed filesystem?
If you think that the test I did wasn't meaningful, or need more data
to evaluate it, please just ask, and I'll post all the details.
I think that you need to try a larger block size in dd. The 4K access
size is not going to perform well on PVFS. Do you get better results if
you switch to bs=16M and adjust the count accordingly?
There is a little bit more information on this issue here:
http://www.pvfs.org/cvs/pvfs-2-7-branch.build/doc/pvfs2-faq/pvfs2-faq.php#SECTION00072000000000000000
It may also be interesting to try adding conv=fsync both cases to see
how the performance changes if you add in the cost of flushing to disk
at the end.
thanks,
-Phil
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
