I doubt ZIL has sequential write pattern, but asking [EMAIL PROTECTED] could yield more authoritative answer. My understanding is you are hitting poor mtron random write performance.
Regards, Andrey On Fri, Oct 17, 2008 at 6:48 PM, Eric Sproul <[EMAIL PROTECTED]> wrote: > Eric Sproul wrote: >> I'm reinstalling with AHCI on, so I'll follow up once we've been able to >> re-run >> our tests. > > The oddity continues-- now I've got the box built the way I intended, so > here's > the full spec, and then I'll explain why I'm puzzled. > > Supermicro X7DB8 board > 2x 2.83GHz quad-core Xeon > 32GB RAM > 2x 80GB SATA disks mirrored in rpool > 6x 1TB 7200rpm SAS disks in raidz pool "data" > with log on mirror of 2x Mtron 3035 64GB SSDs > LSI 1068E SAS controller > Intel onboard ESB2 SATA for OS and log disks > > pool: data > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > data ONLINE 0 0 0 > raidz1 ONLINE 0 0 0 > c0t2d0 ONLINE 0 0 0 > c0t3d0 ONLINE 0 0 0 > c0t4d0 ONLINE 0 0 0 > c0t5d0 ONLINE 0 0 0 > c0t6d0 ONLINE 0 0 0 > c0t7d0 ONLINE 0 0 0 > logs ONLINE 0 0 0 > mirror ONLINE 0 0 0 > c2t2d0 ONLINE 0 0 0 > c2t3d0 ONLINE 0 0 0 > > We started up pgbench and created a 30GB database, which took 5.5 hours to > finish. That seemed excessively long, and the following snapshot of iostat > output is representative of what we saw consistently through the run, which > was > heavy on sequential inserts. > > extended device statistics > r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t0d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0 > 0.0 30.0 0.0 3584.7 4.1 0.9 135.7 29.1 81 87 c2t2d0 > 0.0 30.0 0.0 3584.7 4.1 0.9 135.7 29.1 81 87 c2t3d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t1d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t4d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t5d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t7d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t2d0 > 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t3d0 > > The %w and wsvc_t > 0 are new since changing over to AHCI mode. When running > as > IDE, we barely saw those metrics break above 0. We also saw the w/s drop from > around 100 to around 30. We only occasionally saw bursts of writes to the c0 > spindles. > > I understand about the expected poor random write ops for the SSDs, but I > thought that since the ZIL was a log, the writes would be almost entirely > sequential. Either that's not the case or something else is going on. Could > the log being a mirror have anything to do with it? I tend to think not, > since > those log writes would be in parallel, and we'd expect the write performance > of > a single drive. > > Interestingly, creating this database was not any slower than when we were > running with log on IDE disks and only 4 raidz spindles. Perhaps the extra 2 > spindles helped make up for the (apparently) reduced log performance. > > I'm very puzzled at our results and curious whether it can be blamed entirely > on > not having the right kind of SSD, or whether some other tunable could unlock > the > performance we'd expect from this setup. > > Thanks, > Eric > > p.s.- thanks Frederic for the Bitmicro link-- when those drives come out in Q1 > '09 we'll be taking a close look at them-- the preliminary specs look > promising. > _______________________________________________ > storage-discuss mailing list > [email protected] > http://mail.opensolaris.org/mailman/listinfo/storage-discuss > _______________________________________________ storage-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/storage-discuss
