Eric Sproul wrote:
> I'm reinstalling with AHCI on, so I'll follow up once we've been able to
> re-run
> our tests.
The oddity continues-- now I've got the box built the way I intended, so here's
the full spec, and then I'll explain why I'm puzzled.
Supermicro X7DB8 board
2x 2.83GHz quad-core Xeon
32GB RAM
2x 80GB SATA disks mirrored in rpool
6x 1TB 7200rpm SAS disks in raidz pool "data"
with log on mirror of 2x Mtron 3035 64GB SSDs
LSI 1068E SAS controller
Intel onboard ESB2 SATA for OS and log disks
pool: data
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c0t2d0 ONLINE 0 0 0
c0t3d0 ONLINE 0 0 0
c0t4d0 ONLINE 0 0 0
c0t5d0 ONLINE 0 0 0
c0t6d0 ONLINE 0 0 0
c0t7d0 ONLINE 0 0 0
logs ONLINE 0 0 0
mirror ONLINE 0 0 0
c2t2d0 ONLINE 0 0 0
c2t3d0 ONLINE 0 0 0
We started up pgbench and created a 30GB database, which took 5.5 hours to
finish. That seemed excessively long, and the following snapshot of iostat
output is representative of what we saw consistently through the run, which was
heavy on sequential inserts.
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0
0.0 30.0 0.0 3584.7 4.1 0.9 135.7 29.1 81 87 c2t2d0
0.0 30.0 0.0 3584.7 4.1 0.9 135.7 29.1 81 87 c2t3d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t1d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t4d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t5d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t7d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t3d0
The %w and wsvc_t > 0 are new since changing over to AHCI mode. When running as
IDE, we barely saw those metrics break above 0. We also saw the w/s drop from
around 100 to around 30. We only occasionally saw bursts of writes to the c0
spindles.
I understand about the expected poor random write ops for the SSDs, but I
thought that since the ZIL was a log, the writes would be almost entirely
sequential. Either that's not the case or something else is going on. Could
the log being a mirror have anything to do with it? I tend to think not, since
those log writes would be in parallel, and we'd expect the write performance of
a single drive.
Interestingly, creating this database was not any slower than when we were
running with log on IDE disks and only 4 raidz spindles. Perhaps the extra 2
spindles helped make up for the (apparently) reduced log performance.
I'm very puzzled at our results and curious whether it can be blamed entirely on
not having the right kind of SSD, or whether some other tunable could unlock the
performance we'd expect from this setup.
Thanks,
Eric
p.s.- thanks Frederic for the Bitmicro link-- when those drives come out in Q1
'09 we'll be taking a close look at them-- the preliminary specs look promising.
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
