Thanks Kyle! I disabled the "TroveSyncMeta" to "no", but it doesn't make a difference. The poster you mentioned said the problem occurs with serial programs, and parallel program wont' see such a hit. what's serial/parallel here mean? I let 8 iozone processes run on 8 nodes and aggregate the bandwidth. Isn't it a parallel program?
http://www.beowulf-underground.org/pipermail/pvfs2-users/2009-April/002770.html -Neutron On Fri, Apr 30, 2010 at 8:17 PM, Kyle Schochenmaier <[email protected]> wrote: > Hi Neutron - > > Check out a very similar thread from about a year ago : > http://www.beowulf-underground.org/pipermail/pvfs2-users/2009-April/002770.html > > Im not sure if we found out what was actually causing the performance > differences.. but its probably a start to look into. > > Best, > Kyle Schochenmaier > > > > On Fri, Apr 30, 2010 at 7:08 PM, neutron <[email protected]> wrote: >> Hi all, >> >> I have a pvfs2-2.8.1 with 4 DS. "BMIModules" is set to "bmi_ib". I >> use simple-tripe mode with stipe-size =1MB. >> >> 8 client nodes mount pvfs2, and I use iozone to measure the write >> throughput, with each client node runs one iozone process. >> Each process writes 1GB file with block size = 4MB. >> >> I got the following results: >> >> Test running: >> >> Children see throughput for 8 initial writers = 98513.53 KB/sec >> Min throughput per process = 11406.64 KB/sec >> Max throughput per process = 12642.13 KB/sec >> Avg throughput per process = 12314.19 KB/sec >> Min xfer = 946176.00 KB >> CPU Utilization: Wall time 86.492 CPU time 6.355 CPU >> utilization 7.35 % >> >> >> Test running: >> Children see throughput for 8 rewriters = 2499467.09 KB/sec >> Min throughput per process = 289479.62 KB/sec >> Max throughput per process = 321507.38 KB/sec >> Avg throughput per process = 312433.39 KB/sec >> Min xfer = 946176.00 KB >> CPU utilization: Wall time 3.272 CPU time 5.990 CPU >> utilization 183.08 % >> >> >> We can see the "initial write" gets very low throughput, while the >> "rewrite" gives very high bandwidth. >> My questions are: >> 1. Why 8 client nodes as a whole get low throughput? >> 2. Why at "rewrite" phase, the write bandwidth boosts ? >> >> >> >> -Neutron >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
