To answer your questions more directly, zilstat is what I used to check what the ZIL was doing:
http://www.richardelling.com/Home/scripts-and-programs-1/zilstat While I have added a mirrored log device, I haven't tried adding multiple sets of mirror log devices, but I think it should work. I believe that a failed unmirrored log device is only a problem if the pool is ungracefully closed before ZFS notices that the log device failed (ie, simultaneous power failure and log device failure), so mirroring them may not be required. Tim On Wed, Oct 3, 2012 at 2:54 PM, Timothy Coalson <tsc...@mst.edu> wrote: > I found something similar happening when writing over NFS (at > significantly lower throughput than available on the system directly), > specifically that effectively all data, even asynchronous writes, were > being written to the ZIL, which I eventually traced (with help from Richard > Elling and others on this list) at least partially to the linux NFS client > issuing commit requests before ZFS wanted to write the asynchronous data to > a txg. I tried fiddling with zfs_write_limit_override to get more data > onto normal vdevs faster, but this reduced performance (perhaps setting a > tunable to make ZFS not throttle writes while hitting the write limit could > fix that), and didn't cause it to go significantly easier on the ZIL > devices. I decided to live with the default behavior, since my main > bottleneck is ethernet anyway, and the projected lifespan of the ZIL > devices was fairly large due to our workload. > > I did find that setting logbias=throughput on a zfs filesystem caused it > to act as though the ZIL devices weren't there, which actually reduced > commit times under continuous streaming writes (mostly due to having more > throughput for the same amount of data to commit, in large chunks, but the > zilstat script also reported less writing to the ZIL blocks (which are > allocated from normal vdevs without a ZIL device, or with > logbias=throughput) under this condition, so perhaps there is more to the > story), so if you have different workloads for different datasets, this > could help (since it isn't a poolwide setting). Obviously, small > synchronous writes to that zfs filesystem will take a large hit from this > setting. > > It would be nice if there was a feature in ZFS that could direct small > commits to ZIL blocks on log devices, but behave like logbias=throughput > for large commits. It would probably need manual tuning, but it would > treat SSD log devices more gently, and increase performance for large > contiguous writes. > > If you can't configure ZFS to write less data to the ZIL, I think a RAM > based ZIL device would be a good way to get throughput up higher (and less > worries about flash endurance, etc). > > Tim > > On Wed, Oct 3, 2012 at 1:28 PM, Schweiss, Chip <c...@innovates.com> wrote: > >> I'm in the planing stages of a rather larger ZFS system to house >> approximately 1 PB of data. >> >> I have only one system with SSDs for L2ARC and ZIL, The ZIL seems to be >> the bottle neck for large bursts of data being written. I can't confirm >> this for sure, but the when throwing enough data at my storage pool and the >> write latency starts rising, the ZIL write speed hangs close the max >> sustained throughput I've measured on the SSD (~200 MB/s). >> >> The pool when empty w/o L2ARC or ZIL it was tested with Bonnie++ and >> showed ~1300MB/s serial read and ~800MB/s serial write speed. >> >> How can I determine for sure that my ZIL is my bottleneck? If it is the >> bottleneck, is it possible to keep adding mirrored pairs of SSDs to the ZIL >> to make it faster? Or should I be looking for a DDR drive, ZeusRAM, etc. >> >> Thanks for any input, >> -Chip >> >> >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss@opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >> >
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
