On Sat, 18 Apr 2009, SV wrote:
I just bought 2540s and your posts have introduced me to iozone,
(tho I know I had it bookmarked but never used it). I am in the
process of mostly reconstructing your tests to see if I can get
similar numbers.
In all this 'firmware' business you haven't mentioned your firmware
revs. I got 2 batches of 2540s, some with ver 6.x and the latest
with ver 7.35.
I was using a 6.x version but am now using the latest 7.x version.
I most recently benchmarked using the iozone command:
iozone -a -i 0 -i 1 -y 64 -q 512 -n 8G -g 64G
Also what are your server head(s)? I'm benchmarking with a T5440 and
zfs, but in the end state the 2540s are attached to SF6800 and vxfs.
This is actually a Sun Workstation (yes, Sun used to make
workstations). It is the Ultra-40M2 which is a 4 core Opteron system
with 20GB of RAM. I am using Solaris 10 U5 but updated with all
recent patches. You can read about my setup via the whitepaper at
"http://www.simplesystems.org/users/bfriesen/zfs-discuss/2540-zfs-performance.pdf";.
I was unable to re-create the strange performance results I reported
here earlier. The strange results seem to be some sort of temporary
glitch. Now what I get are very nice results which show a huge
performance improvement over previous array firmware (and Solaris 10
OS) versions:
KB reclen write rewrite read reread
8388608 64 376010 485644 1998561 1929227
8388608 128 403383 506406 2021847 1969137
8388608 256 667399 398293 2001608 2015879
8388608 512 550215 714685 1504432 1504288
16777216 64 399558 40658 827006 860225
16777216 128 420510 408866 827691 860041
16777216 256 372008 415116 820045 860286
16777216 512 414277 408672 752200 793789
33554432 64 369841 41078 549795 549268
33554432 128 361873 361049 539266 545397
33554432 256 376535 364493 544556 548643
33554432 512 375182 379954 535026 552884
67108864 64 358299 41317 554315 554937
67108864 128 359965 354854 550869 554271
67108864 256 356492 352997 550460 559263
67108864 512 355282 359267 545358 547017
Notice that at the time I summarized my results in February 2008,
Sun's own benchmark results only showed something like 130MB/second
write for large files. The stock 2540 was only producing write
performance of about 150MB/second with my configuration. After making
a number of adjustments and applying magic "tweaks" suggested by Sun
engineers, the performance went up to a peak of 273MB/second.
The new CAM 6.2 and new firmware no longer seems to offer the
adjustments which were used before but the baseline performance is now
increased to a peak of 359MB/second. So the same hardware which was
previously rated at 130MB/second is now managing 359MB/second. The
write performance is close to physical limits. This is what we
commonly call "Sun Engineering".
I have also done multi-threaded synchronous read/write tests using the
command:
iozone -m -t 8 -T -O -r 8k -o -s 2G
in a ZFS filesystem set to use 8k blocks and am seeing considerably
improved random write IOPS as well (3708.89 ops/sec vs 3176.50
ops/sec).
Bob
--
Bob Friesenhahn
[email protected], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
