I am able to reproduce this performance problem on a machine with an Apricorn
Sil3124 four port card with four SATA disks attached.
I created a ZFS like this:
zpool create -f zfsatan mirror c7t0d0 c7t1d0 mirror c7t2d0 c7t3d0
zfs create zfsatan/fs01
The performance problem can be reproduced with dd:
time dd if=/dev/zero bs=1024x1024x1024 count=8 of=/zfsatan/storfil
The first time this command is run after the machine is rebooted, it goes
quickly:
real 1m7.179s
user 0m0.001s
sys 0m12.558s
But after running it several times, you see this slowdown:
real 2m40.493s
user 0m0.001s
sys 0m12.744s
The posting describes seeing I/O stop for a few seconds using "iostat -cxn 1".
I am able to reproduce this as well. I/O stops on one of the disks and it shows
%w at 100 and wait at 10.0. Then I/O stops on the other disks as well until the
original disk resumes and then they all resume. Here is an excerpt from "iostat
-cxn 1" during a dd:
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 661.0 0.0 83342.6 9.0 1.0 13.6 1.5 100 99 c7t0d0
0.0 530.0 0.0 66457.2 6.1 0.7 11.6 1.3 69 68 c7t1d0
0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0
0.0 724.0 0.0 91290.6 7.7 0.9 10.7 1.2 87 86 c7t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
0 0 0 100
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 4.0 0.0 141.5 0.0 0.0 0.5 0.2 0 0 c7t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0
0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
0 0 0 100
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0
0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
4 4 0 92
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
1.0 0.0 64.0 0.0 0.0 0.0 0.0 21.6 0 2 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0
0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
7 14 0 79
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 2.0 0.0 4.0 0.0 0.0 0.0 0.1 0 0 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0
0.0 779.0 0.0 98317.7 9.0 1.0 11.6 1.2 100 95 c7t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0
2.0 0.0 8.3 0.0 0.0 0.0 0.0 1.4 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
0 2 0 98
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 36.0 0.0 70.5 0.0 0.0 0.0 0.0 0 0 c7t0d0
0.0 32.0 0.0 70.5 0.1 0.1 2.2 1.6 5 5 c7t1d0
0.0 764.0 0.0 93251.4 7.8 0.9 10.2 1.2 89 88 c7t2d0
0.0 24.0 0.0 38.0 0.1 0.0 2.6 2.0 4 5 c7t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
0 23 0 77
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
1.0 0.0 34.5 0.0 0.0 0.0 0.0 8.5 0 1 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0
0.0 529.0 0.0 66568.0 6.8 0.8 12.8 1.5 79 79 c7t0d0
0.0 620.0 0.0 77960.7 6.8 0.8 11.0 1.3 79 78 c7t1d0
0.0 638.0 0.0 80391.0 6.7 0.8 10.5 1.2 78 78 c7t2d0
0.0 588.0 0.0 73991.5 7.1 0.8 12.0 1.4 80 80 c7t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
hs-uego02-03.west:/export/home1/01/ml40262
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
mf-uego02-01.west:/usr/dist
cpu
us sy wt id
0 5 0 95
I will be able to work on this once I am done with my current RFE.
--
This message posted from opensolaris.org
_______________________________________________
storage-discuss mailing list
storage-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
