Ross,
On Wed, Jul 1, 2009 at 10:33 AM, Jim Dunham<[email protected]>
wrote:
Ross,
I don't know if it's my network setup or what at this point, but
during my
benchmarks I am unable to get more then 20-23MB/s doing 4k
sequential reads
with 4 outstanding I/Os. This is constant whether going to a ZVOL
or a soft
partition.
My setup:
Server: Dell PE 2950, 2 quad xeons, 4GB memory, 2 on board bnx
(management
network only), intel quad igb running LAG to a layer 3 switch,
flow control
and jumbo frames enabled, Solaris 10 update 7 with all the latest
patches
installed. Storage is 14 SAS 15k in a MD1000 enclosure hooked up
to a Dell
PERC 6e w/ 512MB of BBU write-back cache, disks setup as
individual RAID0
disks, then put in a zpool as 2 raidz2 sets of 7 drives each. I
have the
write-back enabled on each disk for ZIL performance since I don't
have an
SSD disk for ZIL logging.
Client ESX windows 2003r2 guest running MS ini 2.08.
I believe the switch and the ESX are setup properly tests against
a Linux
host running IET and HW RAID6 show it's capable of around 30-40MB/s.
Any feedback on this would be appreciated as I am getting a little
burnt
out trying to diagnose this.
Getting 20-23MB/s doing 4k sequential reads is between 5K to 6K
IOPS. These
being constant going between ZVOL or soft partition, I would have
to surmise
that CPU performance is likely one of your limiting factors. What
is the CPU
cost of the iSCSI Target Daemon (iscsitgtd) while driving this I/O
load?
The IOPS estimate is dead on. I didn't run top during the benchmark,
so I can't say definitively what it was, the load avg was about 1.5,
but there was NFS operations occuring on a separate zpool at the time,
so that isn't reliable.
1.5 what ?
I'll re-run the test and look at the usage.
If you are measuring 20-23MB/s performance on the iSCSI Client
side, what is
the iSCSI Target's I/O performance in accessing the ZVOLs or soft
partitions? Specific to comparing iSCSI client and target
performance, does
the work load performance change, as the I/O size changes?
On the client side I was seeing an 18-20% cpu usage, on a quad cpu esx
guest, the load was single cpu bound, but changed cpus during the run.
The benchmark runs workloads of my choosing, the random workload gives
me pretty much the disks' capabilities, it's the sequential that is
causing a problem. Sequential is sensitive to network latency so that
makes me think it's network related, but I'll shoot at anything at
this point.
Is iscsitgt multi threaded in the backing store routines?
Absolutely.
Can it efficiently handle multiple outstanding I/Os?
Multiple outstanding I/Os, yes. Efficiently, not that great. As the
number of threads per iSCSI Initiator, or the number of iSCSI
Initiators increase, there is thread contention across certain daemon
locks.
Your description above states 4 outstanding I/Os, likely from a single
iSCSI Initiator, so lock contention is not an issue.
Because if the backing store I/O is single threaded that would also
show the same performance degradation between it and the Linux target
that uses 8 I/O threads per target for backing store operations.
-Ross
- Jim
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
