> If it's just performance you're after for small > writes, I wonder if you've considered putting the ZIL > on an NVRAM card? It looks like this can give > something like a 20x performance increase in some > situations: > > http://blogs.sun.com/perrin/entry/slog_blog_or_bloggin > g_on
That's certainly interesting reading, but it may be just a tad optimistic. For example, it lists a throughput of 211 MB/sec with only *one* disk in the main pool - which unless that's also a solid-state disk is clearly unsustainable (i.e., you're just seeing the performance while the solid-state log is filling up, rather than what the performance will eventually stabilize at: my guess is that the solid-state log may be larger than the file being updated, in which case updates just keep accumulating there without *ever* being forced to disk, which is unlikely to occur in most normal environments). The numbers are a bit strange in other areas as well. In the case of a single pool disk and no slog, 11 MB/sec represents about 1400 synchronous 8 KB updates per second on a disk with only about 1/10th that IOPS capacity even with queuing enabled (and when you take into account the need to propagate each such synchronous update all the way back to the superblock it begins to look somewhat questionable even from the bandwidth point of view). One might suspect that what's happening is that once the first synchronous write has been submitted a whole bunch of additional ones accumulate while waiting for the disk to finish the first, and that ZFS is smart enough not to queue them up to the disk (which would require full-path updates for every one of them) but instead to gather them in its own cache and write them all back at once in one fell swoop (including a single update for the ancestor path) when the disk is free again. This would explain not only the otherwise suspicious performance but also why adding the slog provides so little improvement; it's also a tribute to the care that the ZFS developers put into this aspect of their implementation. On the other hand, when an slog is introduced performance actually *declines* in systems with more than one pool disk, suggesting that the developers paid somewhat less attention to this aspect of the implementation (where if the updates are held and batched similarly to my conjecture above they ought to be able to reach something close to the disk's streaming-sequential bandwidth, unless there's some pathological interaction with the pool-disk updates that should have been avoidable). Unless I'm missing something the bottom line appears to be that in the absence of an NVRAM-based slog you might be just as well (and sometimes better) off not using one at all. - bill This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
