While testing iscsi targets exported from thumpers via 10GbE and imported 10GbE
on T2000s I am not seeing the throughput I expect, and more importantly there
is a tremendous amount of read IO happending on a purely sequential write
workload. (Note all systems have Sun 10GbE cards and are running Nevada b65.)
Simple write workload (from T2000):
# time dd if=/dev/zero of=/dev/rdsk/c6t010000144F210ECC00002A004675E957d0 \
bs=64k count=1000000
Performance of iscsi target pool on new blocks:
bash-3.00# zpool iostat thumper1-vdev0 1
thumper1-vdev0 17.4G 2.70T 0 526 0 63.6M
thumper1-vdev0 17.5G 2.70T 0 564 0 60.5M
thumper1-vdev0 17.5G 2.70T 0 0 0 0
thumper1-vdev0 17.5G 2.70T 0 0 0 0
thumper1-vdev0 17.5G 2.70T 0 0 0 0
Configuration of zpool/iscsi target:
# zpool status thumper1-vdev0
pool: thumper1-vdev0
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
thumper1-vdev0 ONLINE 0 0 0
c0t7d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
c5t7d0 ONLINE 0 0 0
c6t7d0 ONLINE 0 0 0
c7t7d0 ONLINE 0 0 0
c8t7d0 ONLINE 0 0 0
errors: No known data errors
The first thing is that for this pool I was expecting 200-300MB/s throughput,
since it is a simple stripe across 6, 500G disks. In fact, a direct local
workload (directly on thumper1) of the same type confirms what I expected:
bash-3.00# dd if=/dev/zero of=/dev/zvol/rdsk/thumper1-vdev0/iscsi bs=64k
count=1000000 &
bash-3.00# zpool iostat thumper1-vdev0 1
thumper1-vdev0 20.4G 2.70T 0 2.71K 0 335M
thumper1-vdev0 20.4G 2.70T 0 2.92K 0 374M
thumper1-vdev0 20.4G 2.70T 0 2.88K 0 368M
thumper1-vdev0 20.4G 2.70T 0 2.84K 0 363M
thumper1-vdev0 20.4G 2.70T 0 2.57K 0 327M
The second thing, is that when overwriting already written blocks via the iscsi
target (from the T2000) I see a lot of read bandwidth for blocks that are being
completely overwritten. This does not seem to slow down the write performance,
but 1) it is not seem in the direct case; and 2) it consumes channel bandwidth
unnecessarily.
bash-3.00# zpool iostat thumper1-vdev0 1
thumper1-vdev0 8.90G 2.71T 279 783 31.7M 95.9M
thumper1-vdev0 8.90G 2.71T 281 318 31.7M 29.1M
thumper1-vdev0 8.90G 2.71T 139 0 15.8M 0
thumper1-vdev0 8.90G 2.71T 279 0 31.7M 0
thumper1-vdev0 8.90G 2.71T 139 0 15.8M 0
Can anyone help to explain what I am seeing, or give me some guidance on
diagnosing the cause of the following:
- The bottleneck in accessing the iscsi target from the T2000
- The cause of the extra read bandwidth when overwriting blocks on the iscsi
target from the T2000.
Any help is much appreciated,
paul
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
