Re: [zfs-discuss] Zpool LUN Sizes

2012-10-28 Thread Fajar A. Nugraha
On Sat, Oct 27, 2012 at 9:16 PM, Edward Ned Harvey
(opensolarisisdeadlongliveopensolaris)
opensolarisisdeadlongliveopensola...@nedharvey.com wrote:
 From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
 boun...@opensolaris.org] On Behalf Of Fajar A. Nugraha

 So my
 suggestion is actually just present one huge 25TB LUN to zfs and let
 the SAN handle redundancy.

 create a bunch of 1-disk volumes and let ZFS handle them as if they're JBOD.

Last time I use IBM's enterprise storage (which was, admittedly, a
long time ago) you can't even do that. And looking at Morris' mail
address, it should be revelant :)

... or probably it's just me who haven't found how to do that. Which
why I suggested just use whatever the SAN can present :)

-- 
Fajar
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zpool LUN Sizes

2012-10-28 Thread Gary Mills
On Sun, Oct 28, 2012 at 04:43:34PM +0700, Fajar A. Nugraha wrote:
 On Sat, Oct 27, 2012 at 9:16 PM, Edward Ned Harvey
 (opensolarisisdeadlongliveopensolaris)
 opensolarisisdeadlongliveopensola...@nedharvey.com wrote:
  From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
  boun...@opensolaris.org] On Behalf Of Fajar A. Nugraha
 
  So my
  suggestion is actually just present one huge 25TB LUN to zfs and let
  the SAN handle redundancy.
 
  create a bunch of 1-disk volumes and let ZFS handle them as if they're JBOD.
 
 Last time I use IBM's enterprise storage (which was, admittedly, a
 long time ago) you can't even do that. And looking at Morris' mail
 address, it should be revelant :)
 
 ... or probably it's just me who haven't found how to do that. Which
 why I suggested just use whatever the SAN can present :)

You are entering the uncharted waters of ``multi-level disk
management'' here.  Both ZFS and the SAN use redundancy and error-
checking to ensure data integrity.  Both of them also do automatic
replacement of failing disks.  A good SAN will present LUNs that
behave as perfectly reliable virtual disks, guaranteed to be error
free.  Almost all of the time, ZFS will find no errors.  If ZFS does
find an error, there's no nice way to recover.  Most commonly, this
happens when the SAN is powered down or rebooted while the ZFS host
is still running.

-- 
-Gary Mills--refurb--Winnipeg, Manitoba, Canada-
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zpool LUN Sizes

2012-10-27 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
 From: Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
 
 Performance is much better if you use mirrors instead of raid.  (Sequential
 performance is just as good either way, but sequential IO is unusual for most
 use cases. Random IO is much better with mirrors, and that includes scrubs 
 resilvers.)

Even if you think you use sequential IO...  If you use snapshots...  Thanks to 
the nature of snapshot creation  deletion  the nature of COW, you probably 
don't have much sequential IO in your system, after a couple months of actual 
usage.  Some people use raidzN, but I always use mirrors.

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zpool LUN Sizes

2012-10-27 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
 From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
 boun...@opensolaris.org] On Behalf Of Fajar A. Nugraha
 
 So my
 suggestion is actually just present one huge 25TB LUN to zfs and let
 the SAN handle redundancy.

Oh - No

Definitely let zfs handle the redundancy.  Because ZFS is doing the 
checksumming, if it finds a cksum error, it needs access to the redundant copy 
in order to correct it.  If you let the SAN handle the redundancy, then zfs 
finds a cksum error, and your data is unrecoverable.  (Just the file in 
question, not the whole pool or anything like that.)

The answer to Morris's question, about size of LUNs and so forth...  It really 
doesn't matter what size the LUNs are.  Just choose based on your redundancy 
and performance requirements.  Best would be to go JBOD, or if that's not 
possible, create a bunch of 1-disk volumes and let ZFS handle them as if 
they're JBOD.

Performance is much better if you use mirrors instead of raid.  (Sequential 
performance is just as good either way, but sequential IO is unusual for most 
use cases. Random IO is much better with mirrors, and that includes scrubs  
resilvers.)

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zpool LUN Sizes

2012-10-27 Thread Timothy Coalson
On Sat, Oct 27, 2012 at 9:21 AM, Edward Ned Harvey
(opensolarisisdeadlongliveopensolaris) 
opensolarisisdeadlongliveopensola...@nedharvey.com wrote:

  From: Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
 
  Performance is much better if you use mirrors instead of raid.
  (Sequential
  performance is just as good either way, but sequential IO is unusual for
 most
  use cases. Random IO is much better with mirrors, and that includes
 scrubs 
  resilvers.)

 Even if you think you use sequential IO...  If you use snapshots...
  Thanks to the nature of snapshot creation  deletion  the nature of COW,
 you probably don't have much sequential IO in your system, after a couple
 months of actual usage.  Some people use raidzN, but I always use mirrors.


This may be the case if you often rewrite portions of files, so especially
database usage, but if you generally write entire new files rather than
modifying old ones, I wouldn't expect fragmentation to be that bad.  The
particular workload I have is like this, if a file is changed, it is
overwritten entirely, so I went with raidz2 vdevs for more capacity.
However, I'm not exactly pushing the limits of the pool performance, as my
bottleneck is network.

Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zpool LUN Sizes

2012-10-26 Thread Fajar A. Nugraha
On Sat, Oct 27, 2012 at 4:08 AM, Morris Hooten mhoo...@us.ibm.com wrote:
 I'm creating a zpool that is 25TB in size.

 What are the recommendations in regards to LUN sizes?

 For example:

 Should I have 4 x 6.25 TB LUNS to add to the zpool or 20 x 1.25TB LUNs to
 add to the pool?

 Or does it depend on the size of the san disks themselves?

More like you shouldn't let the SAN mess with the disks and let zfs
see the disks as JBOD.

... but then again, your SAN might not let you do that. So my
suggestion is actually just present one huge 25TB LUN to zfs and let
the SAN handle redundancy.

Or if your SAN can't do it either, just let it give
whatever-biggest-size LUN that it can, and simply use stripe on zfs
side.



 Or should I divide the zpool up and make several smaller zpools?

If you're going to use them for a single purpose anyway (e.g. storing
database files from a single db, or whatever), I don't see the benefit
in doing that.

-- 
Fajar
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zpool LUN Sizes

2012-10-26 Thread Timothy Coalson
Disclaimer: I haven't used LUNs with ZFS, so take this with a grain of salt.

On Fri, Oct 26, 2012 at 4:08 PM, Morris Hooten mhoo...@us.ibm.com wrote:

 I'm creating a zpool that is 25TB in size.

 What are the recommendations in regards to LUN sizes?


The first standard advice I can give is that ZFS wants to be in charge of
the redundancy, otherwise it can only tell you when there is a data error,
but it can't fix it (other than possibly metadata copies, or if you set
copies for data).  So, if you can have the LUNs be real disks, or otherwise
set up so that they reside on independent storage, something like a pool of
raidz2 vdevs, or of mirror pairs, is the general idea to provide good data
integrity.  When ZFS detects data corruption, and it can't fix it (no
redundancy or problems in too many of the constituent sectors to
successfully reconstruct), it will return an I/O error, and not a best
effort at what it thinks the data was, so beware insufficient redundancy.


 For example:

 Should I have 4 x 6.25 TB LUNS to add to the zpool or 20 x 1.25TB LUNs to
 add to the pool?

 Or does it depend on the size of the san disks themselves?


Without redundancy controlled by ZFS, I am unsure whether having multiple
separate LUNs will change performance significantly, it probably depends
most upon whether the LUNs actually perform independently (if saturating
one doesn't make any other LUN significantly slower).  When in doubt,
benchmark.


 Or should I divide the zpool up and make several smaller zpools?


Multiple storage zpools are not usually needed, one main point of ZFS is
that you can have as many filesystems as you want on each pool.  As for
performance, zpool performance should, in theory, scale with the
performance of the underlying devices, so two small zpools shouldn't be
faster in aggregate than one large zpool.

The recommended configuration will depend on how you intend to use it, and
what your constraints are, which aren't clear.

Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss