Hi guys,
I am a PVFS2 newbie and made some performance tests using IOZone, but the
results puzzle me. I have 16 machines. One is meta data server, and other 15
machines are both PVFS2 IO servers and clients. Each client machine runs one
IOZone process, so the aggregate performance is measured. Those machines are
configured as follows: one Intel i7-860 processor, 16GB DDR3 memory and 1TB
SATA hard disk. They are connected through a gigabit Ethernet switch. The OS is
Debian Lenny (2.6.26 kernel). The PVFS2 is 2.8.2 with default configuration.
The IOZone command used is: ./iozone -i 0 -i 1 -i 2 -r 4m -s 32g -t 15 -+m
pvfs_client_list. Since the memory capacity for each machine is 16GB, so I set
the test file size to 32GB to exercise the PVFS2 heavily. The result is listed
below:
Record Size 4096 KB
File size set to 33554432 KB
Network distribution mode enabled.
Command line used: ./iozone -i 0 -i 1 -i 2 -r 4m -s 32g -t 15 -+m
pvfs_client_list
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 15 processes
Each process writes a 33554432 Kbyte file in 4096 Kbyte records
Test running:
Children see throughput for 15 initial writers = 785775.56 KB/sec
Min throughput per process = 50273.01 KB/sec
Max throughput per process = 53785.79 KB/sec
Avg throughput per process = 52385.04 KB/sec
Min xfer = 31375360.00 KB
Test running:
Children see throughput for 15 rewriters = 612876.38 KB/sec
Min throughput per process = 39466.78 KB/sec
Max throughput per process = 41843.63 KB/sec
Avg throughput per process = 40858.43 KB/sec
Min xfer = 31649792.00 KB
Test running:
Children see throughput for 15 readers = 366397.27 KB/sec
Min throughput per process = 9371.45 KB/sec
Max throughput per process = 29229.74 KB/sec
Avg throughput per process = 24426.48 KB/sec
Min xfer = 10760192.00 KB
Test running:
Children see throughput for 15 re-readers = 370985.14 KB/sec
Min throughput per process = 9850.98 KB/sec
Max throughput per process = 29660.86 KB/sec
Avg throughput per process = 24732.34 KB/sec
Min xfer = 11145216.00 KB
Test running:
Children see throughput for 15 random readers = 257970.32 KB/sec
Min throughput per process = 8147.65 KB/sec
Max throughput per process = 20084.32 KB/sec
Avg throughput per process = 17198.02 KB/sec
Min xfer = 13615104.00 KB
Test running:
Children see throughput for 15 random writers = 376059.73 KB/sec
Min throughput per process = 24060.38 KB/sec
Max throughput per process = 26446.96 KB/sec
Avg throughput per process = 25070.65 KB/sec
Min xfer = 30527488.00 KB
I have three questions:
1. Why does write outperforms rewrite significantly? According to IOZone's
document, rewrite is supposed to perform better, since it writes to a file
which already exists, and the metadata is already there.
2. Why is write/random-write faster than read/random-read so much? This
result is really unexpected. I feel that read is supposed to be faster. Is
there anything wrong in my result numbers?
3. Observing the max and min throughput per process in each test item, you
can find that in write/re-write/random-write, the difference between max and
min is acceptable; while in read/re-read/random-read, the max throughput is
about two or three times of the min number. How can I explain this result? Is
it normal?
These results are out of my expectation. Is it possible that they are caused by
faulty hardware (network or disk) or configuration?
Any advice is appreciated.
Sincerely,
Wantao_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
