> > I am able to reproduce this performance problem on a machine with an > > Apricorn Sil3124 four port card with four SATA disks attached. > > > > I created a ZFS like this: > > zpool create -f zfsatan mirror c7t0d0 c7t1d0 mirror c7t2d0 c7t3d0 > > zfs create zfsatan/fs01 > > > > The performance problem can be reproduced with dd: > > time dd if=/dev/zero bs=1024x1024x1024 count=8 of=/zfsatan/storfil > > > > The first time this command is run after the machine is rebooted, it > > goes quickly: > > real 1m7.179s > > user 0m0.001s > > sys 0m12.558s > > > > But after running it several times, you see this slowdown: > > real 2m40.493s > > user 0m0.001s > > sys 0m12.744s > > > > The posting describes seeing I/O stop for a few seconds using > > "iostat -cxn 1". I am able to reproduce this as well. I/O stops on > > one of the > > disks and it shows %w at 100 and wait at 10.0. Then I/O stops on the > > other disks as well until the original disk resumes and then they > > all > > resume. Here is an excerpt from "iostat -cxn 1" during a dd: > > > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 661.0 0.0 83342.6 9.0 1.0 13.6 1.5 100 99 c7t0d0 > > 0.0 530.0 0.0 66457.2 6.1 0.7 11.6 1.3 69 68 c7t1d0 > > 0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0 > > 0.0 724.0 0.0 91290.6 7.7 0.9 10.7 1.2 87 86 c7t3d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 0 0 0 100 > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 4.0 0.0 141.5 0.0 0.0 0.5 0.2 0 0 c7t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0 > > 0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 0 0 0 100 > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0 > > 0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 4 4 0 92 > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 1.0 0.0 64.0 0.0 0.0 0.0 0.0 21.6 0 2 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0 > > 0.0 0.0 0.0 0.0 10.0 0.0 0.0 0.0 100 0 c7t2d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 7 14 0 79 > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 0.0 2.0 0.0 4.0 0.0 0.0 0.0 0.1 0 0 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t1d0 > > 0.0 779.0 0.0 98317.7 9.0 1.0 11.6 1.2 100 95 c7t2d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c7t3d0 > > 2.0 0.0 8.3 0.0 0.0 0.0 0.0 1.4 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 0 2 0 98 > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 36.0 0.0 70.5 0.0 0.0 0.0 0.0 0 0 c7t0d0 > > 0.0 32.0 0.0 70.5 0.1 0.1 2.2 1.6 5 5 c7t1d0 > > 0.0 764.0 0.0 93251.4 7.8 0.9 10.2 1.2 89 88 c7t2d0 > > 0.0 24.0 0.0 38.0 0.1 0.0 2.6 2.0 4 5 c7t3d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 0 23 0 77 > > extended device statistics > > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > > 1.0 0.0 34.5 0.0 0.0 0.0 0.0 8.5 0 1 c6t0d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c5t0d0 > > 0.0 529.0 0.0 66568.0 6.8 0.8 12.8 1.5 79 79 c7t0d0 > > 0.0 620.0 0.0 77960.7 6.8 0.8 11.0 1.3 79 78 c7t1d0 > > 0.0 638.0 0.0 80391.0 6.7 0.8 10.5 1.2 78 78 c7t2d0 > > 0.0 588.0 0.0 73991.5 7.1 0.8 12.0 1.4 80 80 c7t3d0 > > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > > hs-uego02-03.west:/export/home1/01/ml40262 0.0 0.0 0.0 0.0 0.0 0.0 > > 0.0 0.0 0 0 mf-uego02-01.west:/usr/dist > > cpu us sy wt id > > 0 5 0 95 > > > > I will be able to work on this once I am done with my current RFE.
Thanks for the reply Mark, if there are any documents that you can point me at in the mean time please feel free to do so ;) iMx _______________________________________________ storage-discuss mailing list storage-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/storage-discuss
