Dear Phil,
Phil Carns wrote:
Tony Kew wrote:
[...]
The test dir was set up with 16K strips on each node for a 1024K stripe.
iozone set up to write in 1024K blocks. The job took about an hour and
five minutes to run...
Initial write 668,321.05 KB/sec
Rewrite 677,999.64 KB/sec
Read 300,937.30 KB/sec
Re-read 313,756.56 KB/sec
You realize that you can create this same setup with the default
simple stripe distribution too if you use a 16K strip size, right?
[...]
Yes, I just wanted a test I could then compare with the results of similar
distributions using the simple_stripe and twod_stripe distributions.
In my tests so far varstrip_dist set up to be the same as a simple_stripe
is slower, markedly slower when the filesystem is on the same nodes
as iozone is running on (i.e. an "on the fly" filesystem built across
the nodes in a PBS job) - I don't know if this is what you might reasonably
expect
from
http://www.ccr.buffalo.edu/display/WEB/PVFS+version+2+scratch+filesystem+on+U2+-+Test+Results
16 client nodes to 16 (different) server nodes using simple_strip
distribution:
Initial write
321,058.83 KB/sec
Rewrite
212,626.53 KB/sec
Read
161,611.95 KB/sec
Re-read
154,799.58 KB/sec
16 client nodes to 16 (different) server nodes using varstrip_dist
distribution
configured to create the same stripe as above:
Initial write
298,962.00 KB/sec
Rewrite
197,318.79 KB/sec
Read
203,210.17 KB/sec
Re-read
186,866.10 KB/sec
For an "on the fly" filesystem i.e. 16 clients accessing a PVFSv2
filesystem on
the same nodes using simple_strip distribution:
Initial write
219,306.19 KB/sec
Rewrite
130,799.13 KB/sec
Read
183,249.66 KB/sec
Re-read
191,565.02 KB/sec
For an "on the fly" filesystem i.e. 16 clients accessing a PVFSv2
filesystem on
the same nodes using varstrip_dist distribution configured to create the
same
stripe as above:
Initial write
69,381.52 KB/sec
Rewrite
69,611.84 KB/sec
Read
64,910.00 KB/sec
Re-read
65,248.22 KB/sec
It may be the case (given the 64 node 1024K stripe results) that on-the-fly
filesystems need larger strips (a larger stripe) to get similar
performance from
a varstrip_dist distribution
Thanks for your help,
Tony
Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203
CoE Office: (716) 881-8930 Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
Cell: (716) 560-0910 Home: (716) 874-2126
"I love deadlines, I love the whooshing noise they make as they go by."
Douglas Adams
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
