You may not be seeing a bottleneck right now, but it will cause a bottleneck in a high-I/O environment....just for future reference.
Let me take a closer looks at the numbers and compare them to some numbers that I have for OrangeFS-2.8.4. Becky -- Becky Ligon HPC Admin Staff PVFS/OrangeFS Developer Clemson University 864-650-4065 > Hi Becky, > > Thanks for your reply. I am using PVFS2.8.2. I agree that multiple > metadata servers will boost the performance, but I feel that those > questions are not rose from the metaserver bottleneck. Even when only one > IOZone process is started, my second question is still there. But in this > case, the re-write is slightly faster than write. > > BTW, I just measured the read/write performance of each disk using dd > command, it is able to write data at about 90MB/s and read data at > 120MB/s. > > Wantao > > > ------------------ Original ------------------ > From: "Becky Ligon"<[email protected]>; > Date: Fri, May 13, 2011 10:04 PM > To: "Wantao"<[email protected]>; > Cc: "pvfs2-users"<[email protected]>; > Subject: Re: [Pvfs2-users] Questions for IOZone performance test results > > > Which version of PVFS are you using? > > Your setup will work better if each of your 16 servers are both meta and > I/O servers. Your current configuration causes a bottleneck at the > metadata server. > > BEcky > -- > Becky Ligon > HPC Admin Staff > PVFS/OrangeFS Developer > Clemson University/Omnibond.com > 864-650-4065 > >> Hi guys, >> >> I am a PVFS2 newbie and made some performance tests using IOZone, but >> the >> results puzzle me. I have 16 machines. One is meta data server, and >> other >> 15 machines are both PVFS2 IO servers and clients. Each client machine >> runs one IOZone process, so the aggregate performance is measured. >> Those >> machines are configured as follows: one Intel i7-860 processor, 16GB >> DDR3 >> memory and 1TB SATA hard disk. They are connected through a gigabit >> Ethernet switch. The OS is Debian Lenny (2.6.26 kernel). The PVFS2 is >> 2.8.2 with default configuration. >> >> The IOZone command used is: ./iozone -i 0 -i 1 -i 2 -r 4m -s 32g -t 15 >> -+m >> pvfs_client_list. Since the memory capacity for each machine is 16GB, so >> I >> set the test file size to 32GB to exercise the PVFS2 heavily. The >> result >> is listed below: >> >> Record Size 4096 KB >> File size set to 33554432 KB >> Network distribution mode enabled. >> Command line used: ./iozone -i 0 -i 1 -i 2 -r 4m -s 32g -t 15 -+m >> pvfs_client_list >> Output is in Kbytes/sec >> Time Resolution = 0.000001 seconds. >> Processor cache size set to 1024 Kbytes. >> Processor cache line size set to 32 bytes. >> File stride size set to 17 * record size. >> Throughput test with 15 processes >> Each process writes a 33554432 Kbyte file in 4096 Kbyte records >> >> Test running: >> Children see throughput for 15 initial writers = 785775.56 >> KB/sec >> Min throughput per process = 50273.01 KB/sec >> Max throughput per process = 53785.79 KB/sec >> Avg throughput per process = 52385.04 KB/sec >> Min xfer = 31375360.00 KB >> >> Test running: >> Children see throughput for 15 rewriters = 612876.38 KB/sec >> Min throughput per process = 39466.78 KB/sec >> Max throughput per process = 41843.63 KB/sec >> Avg throughput per process = 40858.43 KB/sec >> Min xfer = 31649792.00 KB >> >> Test running: >> Children see throughput for 15 readers = 366397.27 KB/sec >> Min throughput per process = 9371.45 KB/sec >> Max throughput per process = 29229.74 KB/sec >> Avg throughput per process = 24426.48 KB/sec >> Min xfer = 10760192.00 KB >> >> Test running: >> Children see throughput for 15 re-readers = 370985.14 KB/sec >> Min throughput per process = 9850.98 KB/sec >> Max throughput per process = 29660.86 KB/sec >> Avg throughput per process = 24732.34 KB/sec >> Min xfer = 11145216.00 KB >> >> Test running: >> Children see throughput for 15 random readers = 257970.32 >> KB/sec >> Min throughput per process = 8147.65 KB/sec >> Max throughput per process = 20084.32 KB/sec >> Avg throughput per process = 17198.02 KB/sec >> Min xfer = 13615104.00 KB >> >> Test running: >> Children see throughput for 15 random writers = 376059.73 >> KB/sec >> Min throughput per process = 24060.38 KB/sec >> Max throughput per process = 26446.96 KB/sec >> Avg throughput per process = 25070.65 KB/sec >> Min xfer = 30527488.00 KB >> >> I have three questions: >> 1. Why does write outperforms rewrite significantly? According to >> IOZone's document, rewrite is supposed to perform better, since it >> writes to a file which already exists, and the metadata is already >> there. >> 2. Why is write/random-write faster than read/random-read so much? >> This >> result is really unexpected. I feel that read is supposed to be faster. >> Is there anything wrong in my result numbers? >> 3. Observing the max and min throughput per process in each test item, >> you can find that in write/re-write/random-write, the difference >> between >> max and min is acceptable; while in read/re-read/random-read, the max >> throughput is about two or three times of the min number. How can I >> explain this result? Is it normal? >> >> These results are out of my expectation. Is it possible that they are >> caused by faulty hardware (network or disk) or configuration? >> >> Any advice is appreciated. >> >> Sincerely, >> Wantao_______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
