Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
Richard Elling wrote: On Oct 15, 2009, at 1:19 PM, Javier Conde wrote: Hello, I've seen in the what's new of Solaris 10 update 8 just released that ZFS now includes the primarycache and secondarycache properties. Is this the equivalent of the UFS directio? No. UFS directio does 3 things: 1. unbuffered I/O 2. allow concurrent writers (no single-writer lock) 3. provide an improved async I/O code path for the record - iirc UFS will also disable read-aheads with directio. -- Robert Milkowski http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
Hello, I've seen in the what's new of Solaris 10 update 8 just released that ZFS now includes the primarycache and secondarycache properties. Is this the equivalent of the UFS directio? Does it have a similar behavior? I'm thinking about having a database on ZFS with this option, and Oracle recommends to have directio when working on top of a file system. Thanks in advance and best regards, Javi ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
Suggest you start with the man page http://docs.sun.com/app/docs/doc/819-2240/zfs-1m On 10/15/2009 4:19 PM, Javier Conde wrote: Hello, I've seen in the what's new of Solaris 10 update 8 just released that ZFS now includes the primarycache and secondarycache properties. Is this the equivalent of the UFS directio? Does it have a similar behavior? I'm thinking about having a database on ZFS with this option, and Oracle recommends to have directio when working on top of a file system. Thanks in advance and best regards, Javi ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
Other than how to turn these features on and off, only so much performance related info can be shoe horned into a man page. You might check out these blogs: http://blogs.sun.com/roch/entry/people_ask_where_are_we See the direct I/O section http://blogs.sun.com/bobn/entry/what_s_new_in_solaris These are just brief reviews though. I would like to see more information about how to use these parameters. Cindy On 10/15/09 14:28, Torrey McMahon wrote: Suggest you start with the man page http://docs.sun.com/app/docs/doc/819-2240/zfs-1m On 10/15/2009 4:19 PM, Javier Conde wrote: Hello, I've seen in the what's new of Solaris 10 update 8 just released that ZFS now includes the primarycache and secondarycache properties. Is this the equivalent of the UFS directio? Does it have a similar behavior? I'm thinking about having a database on ZFS with this option, and Oracle recommends to have directio when working on top of a file system. Thanks in advance and best regards, Javi ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
On Oct 15, 2009, at 1:19 PM, Javier Conde wrote: Hello, I've seen in the what's new of Solaris 10 update 8 just released that ZFS now includes the primarycache and secondarycache properties. Is this the equivalent of the UFS directio? No. UFS directio does 3 things: 1. unbuffered I/O 2. allow concurrent writers (no single-writer lock) 3. provide an improved async I/O code path For #1, the ZFS equivalent is primarycache=none, but you might find that primarycache=metadata is more appropriate, depending on your workload. For #2 #3, ZFS did not have these limitations from the start. Does it have a similar behavior? As you can see, this question doesn't really make sense because the definition of directio is not consistent across the many implementations calling themselves *directio* :-) I'm thinking about having a database on ZFS with this option, and Oracle recommends to have directio when working on top of a file system. As always, YMMV. But in general, I think primarycache=metadata is a reasonable default for Oracle workloads on ZFS file systems or volumes when there is no L2ARC (cache) device. Tests clearly show that the L2ARC can benefit Oracle database workloads and represent a new way to look at the price/performance analysis for databases. http://blogs.sun.com/fishworks/resource/oow09_ss7000_oracle.pdf -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
On Thu, 15 Oct 2009, Richard Elling wrote: No. UFS directio does 3 things: 1. unbuffered I/O 2. allow concurrent writers (no single-writer lock) 3. provide an improved async I/O code path For #1, the ZFS equivalent is primarycache=none, but you might find that primarycache=metadata is more appropriate, depending on your workload. I don't think that #1 gets you all the way there for unbuffered I/O. It would have to bypass most of the I/O scheduling logic for that to work, and bypassing the I/O scheduling logic would likely invalidate zfs's ordering assurances, screw up the data layout, and lead to poor write performance. The primarycache=none option seems to simply disable the data cache, which means that written data also does not remain in the ARC. That does not mean that written data is not buffered before it is written. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
IIRC the trigger for this thread was the suggestion that primarycache=none be set on datasets used for swap. Presumably swap only gets used when memory is low or exhausted, so it would it be correct to say that it wouldn't make any sense for swap to be in /any/ cache? If this isn't what primarycache=none means, shouldn't there be a disable-cache-entirely flag for datasets used for swap? I guess reads from swap must be buffered somewhere, so it would be an optimization to have such reads buffered in a read cache. But wouldn't the read cache be real small at this point? It's enough to make your head spin :-) -- Frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
On Thu, 15 Oct 2009, Frank Middleton wrote: IIRC the trigger for this thread was the suggestion that primarycache=none be set on datasets used for swap. Presumably swap only gets used when memory is low or exhausted, so it would it be correct to say that it wouldn't make any sense for swap to be in /any/ cache? If this isn't The MMU page cache (memory) is the common interface to swap so swap is often cached. It does not make sense to cache it twice though. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] primarycache and secondarycache properties on Solaris 10 u8
On Oct 15, 2009, at 3:13 PM, Bob Friesenhahn wrote: On Thu, 15 Oct 2009, Richard Elling wrote: No. UFS directio does 3 things: 1. unbuffered I/O 2. allow concurrent writers (no single-writer lock) 3. provide an improved async I/O code path For #1, the ZFS equivalent is primarycache=none, but you might find that primarycache=metadata is more appropriate, depending on your workload. I don't think that #1 gets you all the way there for unbuffered I/O. It would have to bypass most of the I/O scheduling logic for that to work, and bypassing the I/O scheduling logic would likely invalidate zfs's ordering assurances, screw up the data layout, and lead to poor write performance. Another overloaded term? Oracle uses O_DSYNC, but with directio for UFS you basically get that, too. In other words, UFS directio is not a solution for general purpose workloads. I think primarycache=none will garner similar disfavor as it effectively makes all writes sync and disables prefetching. Historically, buffered I/O is a bigger problem when the amount of memory for buffering is limited. Consider that UFS directio arrived back in the day when the average server held perhaps hundreds of MB, every byte mattered. Today RAM is not so constrained, though you do need to be wary of large page exhaustion for Oracle. -- richard The primarycache=none option seems to simply disable the data cache, which means that written data also does not remain in the ARC. That does not mean that written data is not buffered before it is written. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
