Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-24 Thread David Brodbeck
On Mon, Oct 15, 2012 at 5:02 PM, Richard Elling richard.ell...@richardelling.com wrote: There is some interesting research that shows how scrubs for RAID-5 systems can contaminate otherwise good data. The reason is that if a RAID-5 parity mismatch occurs, how do you know where the data

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-15 Thread Heinrich van Riel
On Mon, Oct 15, 2012 at 6:21 PM, Jason Matthews ja...@broken.net wrote: From: heinrich.vanr...@gmail.com [mailto:heinrich.vanr...@gmail.com] My point is most high end storage units has some form of data verification process that is active all the time. As does ZFS. The blocks are

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-15 Thread Richard Elling
much rather bank on the controller. my few cents on scrubs. Thanks From: Jim Klimov Sent: ‎October‎ ‎13‎, ‎2012 ‎9‎:‎02 To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] Zfs stability Scrubs 2012-10-13 7:26, Michael Stapleton wrote: The VAST

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-15 Thread Jim Klimov
2012-10-16 3:57, Heinrich van Riel wrote: Understood, if full backups are executed weekly/monthly no scrub is required. I'd argue that this is not a completely true statement. It might hold for raidzN backing storage with single-copy blocks, but if mirrors and/or two or three copies are

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Roel_D
Thank you all for the good answers! So if i put it all together : 1. ZFS is, in mirror and RAID configs, the best currently available option for reliable data 2. Without scrubs data is checked on every read for integrity 3. Unread data will not be checked for integrity 4. Scrubs will solve point

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Roel_D
10. If SUN had listen to the engineers instead of financials it now would have been marketleader in the server market ;-( Op 13 okt. 2012 om 09:56 heeft Roel_D openindi...@out-side.nl het volgende geschreven: Thank you all for the good answers! So if i put it all together : 1. ZFS is,

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Jim Klimov
2012-10-13 2:06, Jan Owoc wrote: All scrubbing does is put stress on drives and verify that data can still be read from them. If a hard drive ever fails on you and you need to replace it (how often does that happen?), then you know hey, just last week all the other hard drives were able to read

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Jim Klimov
A few more comments: 2012-10-13 11:56, Roel_D wrote: Thank you all for the good answers! So if i put it all together : 1. ZFS is, in mirror and RAID configs, the best currently available option for reliable data Yes, though even it is not replacement for backups, because data loss can be

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Jim Klimov
2012-10-13 7:26, Michael Stapleton wrote: The VAST majority of data centers are not storing data in storage that does checksums to verify data, that is just the reality. Regular backups and site replication rule. And this actually concerns me... we help maintain some deployments built by

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Michael Stapleton
Nice list. You could add: 10. Dedup comes with a price. Mike On Sat, 2012-10-13 at 09:56 +0200, Roel_D wrote: Thank you all for the good answers! So if i put it all together : 1. ZFS is, in mirror and RAID configs, the best currently available option for reliable data 2. Without

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-13 Thread Michael Stapleton
Some basic thoughts: The one advantage of using a storage array instead of a JBOD is the write cache when doing random writes. But the cost is that you loose the data integrity features if the ZFS pool is not configured with redundancy. ZFS works best when it has multiple direct paths to

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Michael Stapleton
It is easy to understand that zfs srubs can be useful, But, How often do we scrub or the equivalent of any other file system? UFS? VXFS? NTFS? ... ZFS has scrubs as a feature, but is it a need? I do not think so. Other file systems accept the risk, mostly because they can not really do anything if

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Roel_D
Maybe people associate doing scrubs with something like doing NTFS defrags? Well if read all the posts and because i installed napp-it on my homeserver which has a scrub scheduler i was almost at the point of assuming such. I recently bought a secondhand x4140 just because it performs so well.

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Reginald Beardsley
--- On Fri, 10/12/12, Michael Stapleton michael.staple...@techsologic.com wrote: I'm only writing this because I get the feeling some people think scrubs are a need. Maybe people associate doing scrubs with something like doing NTFS defrags? I normally do scrubs when I think about

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Michael Stapleton
The problem is when people are overly paranoid because the feature exists and end up causing problems by doing scrubs when they should not because they feel they need to. Skilled admins also understand SLAs. Mike On Fri, 2012-10-12 at 14:38 -0700, Reginald Beardsley wrote: --- On Fri,

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Jan Owoc
On Fri, Oct 12, 2012 at 3:07 PM, Michael Stapleton michael.staple...@techsologic.com wrote: It is easy to understand that zfs srubs can be useful, But, How often do we scrub or the equivalent of any other file system? UFS? VXFS? NTFS? ... If your data has checksums, it is standard practice to

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Jerry Kemp
But that the deal with mailing list everywhere. Be they OI or what ever else. Be it some problem someone is having, or some way to enhance a product, or to get it to do something it was never intended to do. Support mailing list and forums wouldn't exist if people didn't have problems that the

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Doug Hughes
So?}?\, a lot of people have already answered this in various ways. I'm going to provide a little bit of direct answer and focus to some of those other answers (and emphasis) On 10/12/2012 5:07 PM, Michael Stapleton wrote: It is easy to understand that zfs srubs can be useful, But, How often

Re: [OpenIndiana-discuss] Zfs stability Scrubs

2012-10-12 Thread Michael Stapleton
I'm not a mathematician, but can anyone calculate the chance of the Same 8K datablock on Both submirrors Going bad on terabyte drives, before the data is ever read and fixed automatically during normal read operations? And if you are not doing mirroring, you have already accepted a much larger