Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-03 Thread Richard Elling
On Aug 2, 2012, at 5:40 PM, Nigel W wrote: On Thu, Aug 2, 2012 at 3:39 PM, Richard Elling richard.ell...@gmail.com wrote: On Aug 1, 2012, at 8:30 AM, Nigel W wrote: Yes. +1 The L2ARC as is it currently implemented is not terribly useful for storing the DDT in anyway because each DDT

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-02 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov 2012-08-01 23:40, opensolarisisdeadlongliveopensolaris пишет: Agreed, ARC/L2ARC help in finding the DDT, but whenever you've got a snapshot destroy (happens every 15 minutes)

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-02 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov In some of my cases I was lucky enough to get a corrupted /sbin/init or something like that once, and the box had no other BE's yet, so the OS could not do anything reasonable

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-02 Thread Richard Elling
On Aug 1, 2012, at 2:41 PM, Peter Jeremy wrote: On 2012-Aug-01 21:00:46 +0530, Nigel W nige...@nosun.ca wrote: I think a fantastic idea for dealing with the DDT (and all other metadata for that matter) would be an option to put (a copy of) metadata exclusively on a SSD. This is on my

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-02 Thread Richard Elling
On Aug 1, 2012, at 8:30 AM, Nigel W wrote: On Wed, Aug 1, 2012 at 8:33 AM, Sašo Kiselkov skiselkov...@gmail.com wrote: On 08/01/2012 04:14 PM, Jim Klimov wrote: chances are that some blocks of userdata might be more popular than a DDT block and would push it out of L2ARC as well... Which

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-02 Thread Peter Jeremy
On 2012-Aug-02 18:30:01 +0530, opensolarisisdeadlongliveopensolaris opensolarisisdeadlongliveopensola...@nedharvey.com wrote: Ok, so the point is, in some cases, somebody might want redundancy on a device that has no redundancy. They're willing to pay for it by halving their performance. This

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-02 Thread Nigel W
On Thu, Aug 2, 2012 at 3:39 PM, Richard Elling richard.ell...@gmail.com wrote: On Aug 1, 2012, at 8:30 AM, Nigel W wrote: Yes. +1 The L2ARC as is it currently implemented is not terribly useful for storing the DDT in anyway because each DDT entry is 376 bytes but the L2ARC reference is 176

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-07-31 17:55, opensolarisisdeadlongliveopensolaris пишет: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Nico Williams The copies thing is a really only for laptops, where the likelihood of redundancy is very low ZFS also stores

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Sašo Kiselkov
On 08/01/2012 12:04 PM, Jim Klimov wrote: Probably DDT is also stored with 2 or 3 copies of each block, since it is metadata. It was not in the last ZFS on-disk spec from 2006 that I found, for some apparent reason ;) That's probably because it's extremely big (dozens, hundreds or even

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-08-01 16:22, Sašo Kiselkov пишет: On 08/01/2012 12:04 PM, Jim Klimov wrote: Probably DDT is also stored with 2 or 3 copies of each block, since it is metadata. It was not in the last ZFS on-disk spec from 2006 that I found, for some apparent reason ;) The idea of the pun was that the

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Availability of the DDT is IMHO crucial to a deduped pool, so I won't be surprised to see it forced to triple copies. Agreed, although, the DDT is also paramount to performance.

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Sašo Kiselkov
On 08/01/2012 03:35 PM, opensolarisisdeadlongliveopensolaris wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Availability of the DDT is IMHO crucial to a deduped pool, so I won't be surprised to see it forced to triple

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-08-01 17:35, opensolarisisdeadlongliveopensolaris пишет: Personally, I've never been supportive of the whole copies idea. If you need more than one redundant copy of some data, that's why you have pool redundancy. You're just hurting performance by using copies. And protecting against

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-08-01 17:55, Sašo Kiselkov пишет: On 08/01/2012 03:35 PM, opensolarisisdeadlongliveopensolaris wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Availability of the DDT is IMHO crucial to a deduped pool, so I won't be

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Sašo Kiselkov
On 08/01/2012 04:14 PM, Jim Klimov wrote: 2012-08-01 17:55, Sašo Kiselkov пишет: On 08/01/2012 03:35 PM, opensolarisisdeadlongliveopensolaris wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Availability of the DDT is IMHO

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Nigel W
On Wed, Aug 1, 2012 at 8:33 AM, Sašo Kiselkov skiselkov...@gmail.com wrote: On 08/01/2012 04:14 PM, Jim Klimov wrote: chances are that some blocks of userdata might be more popular than a DDT block and would push it out of L2ARC as well... Which is why I plan on investigating implementing

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread opensolarisisdeadlongliveopensolaris
From: Sašo Kiselkov [mailto:skiselkov...@gmail.com] Sent: Wednesday, August 01, 2012 9:56 AM On 08/01/2012 03:35 PM, opensolarisisdeadlongliveopensolaris wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Availability

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread opensolarisisdeadlongliveopensolaris
From: opensolarisisdeadlongliveopensolaris Sent: Wednesday, August 01, 2012 2:08 PM L2ARC is a read cache. Hence the R and C in L2ARC. This means two major things: #1 Writes don't benefit, and #2 There's no way to load the whole DDT into the cache anyway. So you're guaranteed to

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-08-01 22:07, opensolarisisdeadlongliveopensolaris пишет: L2ARC is a read cache. Hence the R and C in L2ARC. R is replacement, but what the hell ;) This means two major things: #1 Writes don't benefit, and #2 There's no way to load the whole DDT into the cache anyway. So you're

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Tomas Forsman
On 01 August, 2012 - opensolarisisdeadlongliveopensolaris sent me these 1,8K bytes: From: Sa??o Kiselkov [mailto:skiselkov...@gmail.com] Sent: Wednesday, August 01, 2012 9:56 AM On 08/01/2012 03:35 PM, opensolarisisdeadlongliveopensolaris wrote: From:

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Well, there is at least a couple of failure scenarios where copies1 are good: 1) A single-disk pool, as in a laptop. Noise on the bus, media degradation, or any other

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov 2012-08-01 22:07, opensolarisisdeadlongliveopensolaris пишет: L2ARC is a read cache. Hence the R and C in L2ARC. R is replacement, but what the hell ;) This means two

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-08-01 23:40, opensolarisisdeadlongliveopensolaris пишет: Agreed, ARC/L2ARC help in finding the DDT, but whenever you've got a snapshot destroy (happens every 15 minutes) you've got a lot of entries you need to write. Those are all scattered about the pool... Even if you can find them

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Jim Klimov
2012-08-01 23:34, opensolarisisdeadlongliveopensolaris пишет: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Well, there is at least a couple of failure scenarios where copies1 are good: 1) A single-disk pool, as in a laptop.

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-08-01 Thread Peter Jeremy
On 2012-Aug-01 21:00:46 +0530, Nigel W nige...@nosun.ca wrote: I think a fantastic idea for dealing with the DDT (and all other metadata for that matter) would be an option to put (a copy of) metadata exclusively on a SSD. This is on my wishlist as well. I believe ZEVO supports it so possibly

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-31 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Nico Williams The copies thing is a really only for laptops, where the likelihood of redundancy is very low ZFS also stores multiple copies of things that it considers extra important.

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-30 Thread GREGG WONDERLY
On Jul 29, 2012, at 3:12 PM, opensolarisisdeadlongliveopensolaris opensolarisisdeadlongliveopensola...@nedharvey.com wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov I wondered if the copies attribute can be considered

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-30 Thread John Martin
On 07/29/12 14:52, Bob Friesenhahn wrote: My opinion is that complete hard drive failure and block-level media failure are two totally different things. That would depend on the recovery behavior of the drive for block-level media failure. A drive whose firmware does excessive (reports of up

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-30 Thread Brandon High
On Mon, Jul 30, 2012 at 7:11 AM, GREGG WONDERLY gregg...@gmail.com wrote: I thought I understood that copies would not be on the same disk, I guess I need to go read up on this again. ZFS attempts to put copies on separate devices, but there's no guarantee. -B -- Brandon High :

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-30 Thread Nico Williams
The copies thing is a really only for laptops, where the likelihood of redundancy is very low (there are some high-end laptops with multiple drives, but those are relatively rare) and where this idea is better than nothing. It's also nice that copies can be set on a per-dataset manner (whereas

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-29 Thread Roy Sigurd Karlsbakk
copies won't help much if the pool is unavailable. It may, however, help if, say, you have a RAIDz2, and two drives die, and htere are errors on a third drive, but not sufficiently bad for zfs to reject the pool roy - Opprinnelig melding - Hello all, Over the past few years there

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-29 Thread Bob Friesenhahn
On Sun, 29 Jul 2012, Jim Klimov wrote: Would extra copies on larger disks actually provide the extra reliability, or only add overheads and complicate/degrade the situation? My opinion is that complete hard drive failure and block-level media failure are two totally different things.

Re: [zfs-discuss] Can the ZFS copies attribute substitute HW disk redundancy?

2012-07-29 Thread opensolarisisdeadlongliveopensolaris
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov I wondered if the copies attribute can be considered sort of equivalent to the number of physical disks - limited to seek times though. Namely, for the same amount of storage