Re: [zfs-discuss] Making ZIL faster

2012-10-05 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: Neil Perrin [mailto:neil.per...@oracle.com] In general - yes, but it really depends. Multiple synchronous writes of any size across multiple file systems will fan out across the log devices. That is because there is a separate independent log chain for each file system. Also large

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Schweiss, Chip
Thanks for all the input. It seems information on the performance of the ZIL is sparse and scattered. I've spent significant time researching this the past day. I'll summarize what I've found. Please correct me if I'm wrong. - The ZIL can have any number of SSDs attached either mirror or

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: Andrew Gabriel [mailto:andrew.gabr...@cucumber.demon.co.uk] Temporarily set sync=disabled Or, depending on your application, leave it that way permanently. I know, for the work I do, most systems I support at most locations have sync=disabled. It all depends on the workload.

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Schweiss, Chip . The ZIL can have any number of SSDs attached either mirror or individually.   ZFS will stripe across these in a raid0 or raid10 fashion depending on how you configure. I'm

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Andrew Gabriel
Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Schweiss, Chip How can I determine for sure that my ZIL is my bottleneck? If it is the bottleneck, is it possible to keep adding

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Neil Perrin
On 10/04/12 05:30, Schweiss, Chip wrote: Thanks for all the input. It seems information on the performance of the ZIL is sparse and scattered. I've spent significant time researching this the past day. I'll summarize what I've found. Please correct me if I'm wrong.

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Richard Elling
Thanks Neil, we always appreciate your comments on ZIL implementation. One additional comment below... On Oct 4, 2012, at 8:31 AM, Neil Perrin neil.per...@oracle.com wrote: On 10/04/12 05:30, Schweiss, Chip wrote: Thanks for all the input. It seems information on the performance of the

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Schweiss, Chip
Again thanks for the input and clarifications. I would like to clarify the numbers I was talking about with ZiL performance specs I was seeing talked about on other forums. Right now I'm getting streaming performance of sync writes at about 1 Gbit/S. My target is closer to 10Gbit/S. If I

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Schweiss, Chip If I get to build it this system, it will house a decent size VMware NFS storage w/ 200+ VMs, which will be dual connected via 10Gbe.   This is all medical imaging research. 

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Neil Perrin The ZIL code chains blocks together and these are allocated round robin among slogs or if they don't exist then the main pool devices. So, if somebody is doing sync writes as

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Richard Elling
On Oct 4, 2012, at 1:33 PM, Schweiss, Chip c...@innovates.com wrote: Again thanks for the input and clarifications. I would like to clarify the numbers I was talking about with ZiL performance specs I was seeing talked about on other forums. Right now I'm getting streaming performance

Re: [zfs-discuss] Making ZIL faster

2012-10-04 Thread Neil Perrin
On 10/04/12 15:59, Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Neil Perrin The ZIL code chains blocks together and these are allocated round robin among slogs or if they

[zfs-discuss] Making ZIL faster

2012-10-03 Thread Schweiss, Chip
I'm in the planing stages of a rather larger ZFS system to house approximately 1 PB of data. I have only one system with SSDs for L2ARC and ZIL, The ZIL seems to be the bottle neck for large bursts of data being written.I can't confirm this for sure, but the when throwing enough data at my

Re: [zfs-discuss] Making ZIL faster

2012-10-03 Thread Timothy Coalson
I found something similar happening when writing over NFS (at significantly lower throughput than available on the system directly), specifically that effectively all data, even asynchronous writes, were being written to the ZIL, which I eventually traced (with help from Richard Elling and others

Re: [zfs-discuss] Making ZIL faster

2012-10-03 Thread Timothy Coalson
To answer your questions more directly, zilstat is what I used to check what the ZIL was doing: http://www.richardelling.com/Home/scripts-and-programs-1/zilstat While I have added a mirrored log device, I haven't tried adding multiple sets of mirror log devices, but I think it should work. I

Re: [zfs-discuss] Making ZIL faster

2012-10-03 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Schweiss, Chip How can I determine for sure that my ZIL is my bottleneck?  If it is the bottleneck, is it possible to keep adding mirrored pairs of SSDs to the ZIL to make it faster?  Or