Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Marc Bevand
Marc Bevand m.bevand at gmail.com writes: Well let's look at a concrete example: - cheapest 15k SAS drive (73GB): $180 [1] - cheapest 7.2k SATA drive (160GB): $40 [2] (not counting a 80GB at $37) The SAS drive most likely offers 2x-3x the IOPS/$. Certainly not 180/40=4.5x Doh! I said the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Bob Friesenhahn
On Thu, 2 Oct 2008, Joerg Schilling wrote: I am not going to accept blanket numbers... If you claim that a 15k drive offers more than 2x the IOPS/s of a 7.2k drive you would need to show your computation. SAS and SATA use the same cable and in case you buy server grade SATA disks, you also

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Bob Friesenhahn
On Thu, 2 Oct 2008, Marc Bevand wrote: Doh! I said the opposite of what I meant. Let me rephrase: The SAS drive offers at most 2x-3x the IOPS (optimistic), but at 180/40=4.5x the price. Therefore the SATA drive has better IOPS/$. I doubt that anyone will successfully argue that SAS drives

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Ahmed Kamal
Thanks for the info. I am not really after big performance, I am already on SATA and it's good enough for me. What I really really can't afford is data loss. The CAD designs our engineers are working on can sometimes be really worth a lot. But still we're a small company and would rather save and

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Joerg Schilling
Bob Friesenhahn [EMAIL PROTECTED] wrote: On Thu, 2 Oct 2008, Joerg Schilling wrote: I am not going to accept blanket numbers... If you claim that a 15k drive offers more than 2x the IOPS/s of a 7.2k drive you would need to show your computation. SAS and SATA use the same cable and in

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Bob Friesenhahn
On Thu, 2 Oct 2008, Ahmed Kamal wrote: What is the real/practical possibility that I will face data loss during the next 5 years for example ? As storage experts please help me interpret whatever numbers you're going to throw, so is it a really really small chance, or would you be worried

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Tim
On Thu, Oct 2, 2008 at 11:43 AM, Joerg Schilling [EMAIL PROTECTED] wrote: You seem to missunderstand drive physics. With modern drives, seek times are not a dominating factor. It is the latency time that is rather important and this is indeed 1/rotanilnal-speed. On the other side you

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Tim
On Thu, Oct 2, 2008 at 10:56 AM, Bob Friesenhahn [EMAIL PROTECTED] wrote: I doubt that anyone will successfully argue that SAS drives offer the best IOPS/$ value as long as space, power, and reliability factors may be ignored. However, these sort of enterprise devices exist in order to

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Al Hopper
On Thu, Oct 2, 2008 at 3:09 AM, Marc Bevand [EMAIL PROTECTED] wrote: Erik Trimble Erik.Trimble at Sun.COM writes: Marc Bevand wrote: 7500rpm (SATA) drives clearly provide the best TB/$, throughput/$, IOPS/$. You can't argue against that. To paraphrase what was said earlier in this thread,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Al Hopper
On Thu, Oct 2, 2008 at 12:46 PM, Bob Friesenhahn [EMAIL PROTECTED] wrote: On Thu, 2 Oct 2008, Ahmed Kamal wrote: What is the real/practical possibility that I will face data loss during the next 5 years for example ? As storage experts please help me interpret whatever numbers you're going to

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Nicolas Williams
On Thu, Oct 02, 2008 at 12:46:59PM -0500, Bob Friesenhahn wrote: On Thu, 2 Oct 2008, Ahmed Kamal wrote: What is the real/practical possibility that I will face data loss during the next 5 years for example ? As storage experts please help me interpret whatever numbers you're going to throw,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Erik Trimble
OK, we cut off this thread now. Bottom line here is that when it comes to making statements about SATA vs SAS, there are ONLY two statements which are currently absolute: (1) a SATA drive has better GB/$ than a SAS drive (2) a SAS drive has better throughput and IOPs than a SATA drive This

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Marc Bevand
Erik Trimble Erik.Trimble at Sun.COM writes: Bottom line here is that when it comes to making statements about SATA vs SAS, there are ONLY two statements which are currently absolute: (1) a SATA drive has better GB/$ than a SAS drive (2) a SAS drive has better throughput and IOPs than a

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-02 Thread Richard Elling
I hate to drag this thread on, but... Erik Trimble wrote: OK, we cut off this thread now. Bottom line here is that when it comes to making statements about SATA vs SAS, there are ONLY two statements which are currently absolute: (1) a SATA drive has better GB/$ than a SAS drive In

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Casper . Dik
On Tue, 30 Sep 2008, Robert Thurlow wrote: Modern NFS runs over a TCP connection, which includes its own data validation. This surely helps. Less than we'd sometimes like :-) The TCP checksum isn't very strong, and we've seen corruption tied to a broken router, where the Ethernet

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Darren J Moffat
[EMAIL PROTECTED] wrote: On Tue, 30 Sep 2008, Robert Thurlow wrote: Modern NFS runs over a TCP connection, which includes its own data validation. This surely helps. Less than we'd sometimes like :-) The TCP checksum isn't very strong, and we've seen corruption tied to a broken router,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Joerg Schilling
Tim [EMAIL PROTECTED] wrote: Hmm ... well, there is a considerable price difference, so unless someone says I'm horribly mistaken, I now want to go back to Barracuda ES 1TB 7200 drives. By the way, how many of those would saturate a single (non trunked) Gig ethernet link ? Workload NFS

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Brian Hechinger
On Wed, Oct 01, 2008 at 01:03:28AM +0200, Ahmed Kamal wrote: Hmm ... well, there is a considerable price difference, so unless someone says I'm horribly mistaken, I now want to go back to Barracuda ES 1TB 7200 drives. By the way, how many of those would saturate a single (non trunked) Gig

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Joerg Schilling
David Magda [EMAIL PROTECTED] wrote: On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives suffer badly on random workloads. Well, if you can probably afford more SATA drives for the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Moore, Joe
Toby Thain Wrote: ZFS allows the architectural option of separate storage without losing end to end protection, so the distinction is still important. Of course this means ZFS itself runs on the application server, but so what? The OP in question is not running his network clients on

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Moore, Joe
Ian Collins wrote: I think you'd be surprised how large an organisation can migrate most, if not all of their application servers to zones one or two Thumpers. Isn't that the reason for buying in server appliances? Assuming that the application servers can coexist in the only 16GB

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Al Hopper
On Wed, Oct 1, 2008 at 8:52 AM, Brian Hechinger [EMAIL PROTECTED] wrote: On Wed, Oct 01, 2008 at 01:03:28AM +0200, Ahmed Kamal wrote: Hmm ... well, there is a considerable price difference, so unless someone says I'm horribly mistaken, I now want to go back to Barracuda ES 1TB 7200 drives. By

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Darren J Moffat
Moore, Joe wrote: Toby Thain Wrote: ZFS allows the architectural option of separate storage without losing end to end protection, so the distinction is still important. Of course this means ZFS itself runs on the application server, but so what? The OP in question is not running his

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Al Hopper
On Wed, Oct 1, 2008 at 9:34 AM, Moore, Joe [EMAIL PROTECTED] wrote: Ian Collins wrote: I think you'd be surprised how large an organisation can migrate most, if not all of their application servers to zones one or two Thumpers. Isn't that the reason for buying in server appliances?

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Moore, Joe
Darren J Moffat wrote: Moore, Joe wrote: Given the fact that NFS, as implemented in his client systems, provides no end-to-end reliability, the only data protection that ZFS has any control over is after the write() is issued by the NFS server process. NFS can provided on the wire

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Matthew Sweeney
On 10/01/08 10:46, Al Hopper wrote: On Wed, Oct 1, 2008 at 9:34 AM, Moore, Joe [EMAIL PROTECTED] wrote: Ian Collins wrote: I think you'd be surprised how large an organisation can migrate most, if not all of their application servers to zones one or two Thumpers. Isn't that the reason

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Tim wrote: I think you'd be surprised how large an organisation can migrate most, if not all of their application servers to zones one or two Thumpers. Isn't that the reason for buying in server appliances? I think you'd be surprised how quickly they'd be fired for

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Tim
On Wed, Oct 1, 2008 at 9:18 AM, Joerg Schilling [EMAIL PROTECTED] wrote: David Magda [EMAIL PROTECTED] wrote: On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives suffer badly on

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Tim
On Wed, Oct 1, 2008 at 10:28 AM, Bob Friesenhahn [EMAIL PROTECTED] wrote: On Wed, 1 Oct 2008, Tim wrote: I think you'd be surprised how large an organisation can migrate most, if not all of their application servers to zones one or two Thumpers. Isn't that the reason for buying in server

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Casper . Dik
Ummm, no. SATA and SAS seek times are not even in the same universe.= They most definitely do not use the same mechanics inside. Whoever told y= ou that rubbish is an outright liar. Which particular disks are you guys talking about? I;m thinking you guys are talking about the same 3.5 w/

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Tim
On Wed, Oct 1, 2008 at 11:20 AM, [EMAIL PROTECTED] wrote: Ummm, no. SATA and SAS seek times are not even in the same universe.= They most definitely do not use the same mechanics inside. Whoever told y= ou that rubbish is an outright liar. Which particular disks are you guys talking

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread dmagda
On Wed, October 1, 2008 10:18, Joerg Schilling wrote: SATA and SAS disks usually base on the same drive mechanism. The seek times are most likely identical. Some SATA disks support tagged command queueing and others do not. I would asume that there is no speed difference between SATA with

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Joerg Schilling wrote: SATA and SAS disks usually base on the same drive mechanism. The seek times are most likely identical. This must be some sort of urban legend. While the media composition and drive chassis is similar, the rest of the product clearly differs. The

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Miles Nordin
t == Tim [EMAIL PROTECTED] writes: t So what would be that the application has to run on Solaris. t And requires a LUN to function. ITYM requires two LUN's, or else when your filesystem becomes corrupt after a crash the sysadmin will get blamed for it. Maybe you can deduplicate the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Tim
On Wed, Oct 1, 2008 at 11:53 AM, Ahmed Kamal [EMAIL PROTECTED] wrote: Thanks for all the opinions everyone, my current impression is: - I do need as much RAM as I can afford (16GB look good enough for me) Depends on both the workload, and the amount of storage behind it. From your

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Nicolas Williams
On Tue, Sep 30, 2008 at 09:54:04PM -0400, Miles Nordin wrote: ok, I get that S3 went down due to corruption, and that the network checksums I mentioned failed to prevent the corruption. The missing piece is: belief that the corruption occurred on the network rather than somewhere else.

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, [EMAIL PROTECTED] wrote: To get the same storage between capacity with SAS drives and SATA drives, you'd probably have to put the SAS drives in a RAID-5/6/Z configuration to be more space efficient. However by doing this you'd be losing spindles, and therefore IOPS. With

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Joerg Schilling
Tim [EMAIL PROTECTED] wrote: Ummm, no. SATA and SAS seek times are not even in the same universe. They most definitely do not use the same mechanics inside. Whoever told you that rubbish is an outright liar. It is extremely unlikely that two drives from the same manufacturer and with the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Miles Nordin
cd == Casper Dik [EMAIL PROTECTED] writes: cd The whole packet lives in the memory of the switch/router and cd if that memory is broken the packet will be send damaged. that's true, but by algorithmically modifying the checksum to match your ttl decrementing and MAC address

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Joerg Schilling wrote: Ummm, no. SATA and SAS seek times are not even in the same universe. They most definitely do not use the same mechanics inside. Whoever told you that rubbish is an outright liar. It is extremely unlikely that two drives from the same manufacturer

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Richard Elling
Ahmed Kamal wrote: Thanks for all the opinions everyone, my current impression is: - I do need as much RAM as I can afford (16GB look good enough for me) - SAS disks offers better iops better MTBF than SATA. But Sata offers enough performance for me (to saturate a gig link), and its MTBF

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Joerg Schilling
Bob Friesenhahn [EMAIL PROTECTED] wrote: On Wed, 1 Oct 2008, Joerg Schilling wrote: SATA and SAS disks usually base on the same drive mechanism. The seek times are most likely identical. This must be some sort of urban legend. While the media composition and drive chassis is similar,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Robert Thurlow
Miles Nordin wrote: sounds like they are not good enough though, because unless this broken router that Robert and Darren saw was doing NAT, yeah, it should not have touch the TCP/UDP checksum. I believe we proved that the problem bit flips were such that the TCP checksum was the same, so

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Nicolas Williams
On Wed, Oct 01, 2008 at 12:22:56PM -0500, Tim wrote: - This will mainly be used for NFS sharing. Everyone is saying it will have bad performance. My question is, how bad is bad ? Is it worse than a plain Linux server sharing NFS over 4 sata disks, using a crappy 3ware raid card with

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Tim
On Wed, Oct 1, 2008 at 12:51 PM, Joerg Schilling [EMAIL PROTECTED] wrote: Did you recently look at spec files from drive manufacturers? If you look at drives in the same category, the difference between a SATA and a SAS disk is only the firmware and the way the drive mechanism has been

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Nicolas Williams
On Wed, Oct 01, 2008 at 11:54:55AM -0600, Robert Thurlow wrote: Miles Nordin wrote: sounds like they are not good enough though, because unless this broken router that Robert and Darren saw was doing NAT, yeah, it should not have touch the TCP/UDP checksum. I believe we proved that

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Joerg Schilling wrote: Did you recently look at spec files from drive manufacturers? Yes. If you look at drives in the same category, the difference between a SATA and a The problem is that these drives (SAS / SATA) are generally not in the same category so your

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Bill Sommerfeld
On Wed, 2008-10-01 at 11:54 -0600, Robert Thurlow wrote: like they are not good enough though, because unless this broken router that Robert and Darren saw was doing NAT, yeah, it should not have touch the TCP/UDP checksum. NAT was not involved. I believe we proved that the problem bit

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Marc Bevand
Tim tim at tcsac.net writes: That's because the faster SATA drives cost just as much money as their SAS counterparts for less performance and none of the advantages SAS brings such as dual ports. SAS drives are far from always being the best choice, because absolute IOPS or throughput

Re: [zfs-discuss] Quantifying ZFS reliability

2008-10-01 Thread Erik Trimble
Marc Bevand wrote: Tim tim at tcsac.net writes: That's because the faster SATA drives cost just as much money as their SAS counterparts for less performance and none of the advantages SAS brings such as dual ports. SAS drives are far from always being the best choice, because

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread MC
The good news is that even though the answer to your question is no, it doesn't matter because it sounds like what you are doing is a piece of cake :) Given how cheap hardware is, and how modest your requirements sound, I expect you could build multiple custom systems for the cost of an EMC

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? - Any other limitations of the big two NAS vendors as compared to zfs ? - I still don't have my original question answered, I want to somehow assess the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread David Magda
On Sep 30, 2008, at 06:58, Ahmed Kamal wrote: - I still don't have my original question answered, I want to somehow assess the reliability of that zfs storage stack. If there's no hard data on that, then if any storage expert who works with lots of systems can give his impression of

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? If they are not at the end, they can't do end-to-end data validation. Ideally, application writers would do this, but it is a lot

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Ahmed Kamal wrote: - I still don't have my original question answered, I want to somehow assess the reliability of that zfs storage stack. If there's no hard data on that, then if any storage expert who works with lots of systems can give his impression of the reliability

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
I guess I am mostly interested in MTDL for a zfs system on whitebox hardware (like pogo), vs dataonTap on netapp hardware. Any numbers ? On Tue, Sep 30, 2008 at 4:36 PM, Bob Friesenhahn [EMAIL PROTECTED] wrote: On Tue, 30 Sep 2008, Ahmed Kamal wrote: - I still don't have my original

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Ahmed Kamal wrote: I guess I am mostly interested in MTDL for a zfs system on whitebox hardware (like pogo), vs dataonTap on netapp hardware. Any numbers ? Barring kernel bugs or memory errors, Richard Elling's blog entry seems to be the best place use as a guide:

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: I guess I am mostly interested in MTDL for a zfs system on whitebox hardware (like pogo), vs dataonTap on netapp hardware. Any numbers ? It depends to a large degree on the disks chosen. NetApp uses enterprise class disks and you can expect better reliability from such

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Miles Nordin
ak == Ahmed Kamal [EMAIL PROTECTED] writes: ak I need to answer and weigh against the cost. I suggest translating the reliability problems into a cost for mitigating them: price the ZFS alternative as two systems, and keep the second system offline except for nightly backup. Since you care

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Thanks guys, it seems the problem is even more difficult than I thought, and it seems there is no real measure for the software quality of the zfs stack vs others, neutralizing the hardware used under both. I will be using ECC RAM, since you mentioned it, and I will shift to using enterprise disks

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 2:10 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: * Now that I'm using ECC RAM, and enterprisey disks, Does this put this solution in par with low end netapp 2020 for example ? *sort of*. What are you going to be using it for? Half the beauty of NetApp are all the

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread MC
Just to confuse you more, I mean, give you another point of view: - CPU: 1 Xeon Quad Core E5410 2.33GHz 12MB Cache 1333MHz The reason the Xeon line is good is because it allows you to squeeze maximum performance out of a given processor technology from Intel, possibly getting the highest

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 6:58 AM, Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? Blunty - no remote storage can have it by definition. The checksum needs to be computed as close as

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Erik Trimble
Toby Thain wrote: On 30-Sep-08, at 6:58 AM, Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? Blunty - no remote storage can have it by definition. The checksum

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 4:26 PM, Toby Thain [EMAIL PROTECTED]wrote: On 30-Sep-08, at 6:58 AM, Ahmed Kamal wrote: Thanks for all the answers .. Please find more questions below :) - Good to know EMC filers do not have end2end checksums! What about netapp ? Blunty - no remote storage

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Erik Trimble
Will Murnane wrote: On Tue, Sep 30, 2008 at 21:48, Tim [EMAIL PROTECTED] wrote: why ZFS can do this and hardware solutions can't (being several unreliable subsystems away from the data). So how is a Server running Solaris with a QLogic HBA connected to an FC JBOD any different

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread A Darren Dunham
On Tue, Sep 30, 2008 at 03:19:40PM -0700, Erik Trimble wrote: To make Will's argument more succinct (wink), with a NetApp, undetectable (by the NetApp) errors can be introduced at the HBA and transport layer (FC Switch, slightly damage cable) levels. ZFS will detect such errors, and fix

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 5:19 PM, Erik Trimble [EMAIL PROTECTED] wrote: To make Will's argument more succinct (wink), with a NetApp, undetectable (by the NetApp) errors can be introduced at the HBA and transport layer (FC Switch, slightly damage cable) levels. ZFS will detect such errors,

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Intel mainstream (and indeed many tech companies') stuff is purposely stratified from the enterprise stuff by cutting out features like ECC and higher memory capacity and using different interface form factors. Well I guess I am getting a Xeon anyway There is nothing magical about SAS

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 6:03 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: Hmm ... well, there is a considerable price difference, so unless someone says I'm horribly mistaken, I now want to go back to Barracuda ES 1TB 7200 drives. By the way, how many of those would saturate a single (non

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Robert Thurlow
Ahmed Kamal wrote: BTW, for everyone saying zfs is more reliable because it's closer to the application than a netapp, well at least in my case it isn't. The solaris box will be NFS sharing and the apps will be running on remote Linux boxes. So, I guess this makes them equal. How about a

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Miles Nordin wrote: I think you need two zpools, or zpool + LVM2/XFS, some kind of two-filesystem setup, because of the ZFS corruption and panic/freeze-on-import problems. Having two zpools helps with other If ZFS provides such a terrible experience for you can I be

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Ahmed Kamal wrote: So, I guess this makes them equal. How about a new reliable NFS protocol, that computes the hashes on the client side, sends it over the wire to be written remotely on the zfs storage node ?! Modern NFS runs over a TCP connection, which includes its own

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Nicolas Williams
On Tue, Sep 30, 2008 at 06:09:30PM -0500, Tim wrote: On Tue, Sep 30, 2008 at 6:03 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: BTW, for everyone saying zfs is more reliable because it's closer to the application than a netapp, well at least in my case it isn't. The solaris box will be NFS

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Miles Nordin
rt == Robert Thurlow [EMAIL PROTECTED] writes: rt introduces a separate protection domain for the NFS link. There are checksums in the ethernet FCS, checksums in IP headers, checksums in UDP headers (which are sometimes ignored), and checksums in TCP (which are not ignored). There might be

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread David Magda
On Sep 30, 2008, at 19:44, Miles Nordin wrote: There are checksums in the ethernet FCS, checksums in IP headers, checksums in UDP headers (which are sometimes ignored), and checksums in TCP (which are not ignored). There might be an RPC layer checksum, too, not sure. Not of which helped

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread David Magda
On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives suffer badly on random workloads. Well, if you can probably afford more SATA drives for the purchase price, you can put them in a

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 7:15 PM, David Magda [EMAIL PROTECTED] wrote: On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives suffer badly on random workloads. Well, if you can probably afford

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Well, if you can probably afford more SATA drives for the purchase price, you can put them in a striped-mirror set up, and that may help things. If your disks are cheap you can afford to buy more of them (space, heat, and power not withstanding). Hmm, that's actually cool ! If I configure

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Robert Thurlow
Bob Friesenhahn wrote: On Wed, 1 Oct 2008, Ahmed Kamal wrote: So, I guess this makes them equal. How about a new reliable NFS protocol, that computes the hashes on the client side, sends it over the wire to be written remotely on the zfs storage node ?! Modern NFS runs over a TCP

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Robert Thurlow
Miles Nordin wrote: There are checksums in the ethernet FCS, checksums in IP headers, checksums in UDP headers (which are sometimes ignored), and checksums in TCP (which are not ignored). There might be an RPC layer checksum, too, not sure. Different arguments can be made against each, I

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Tim wrote: On Tue, Sep 30, 2008 at 7:15 PM, David Magda [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On Sep 30, 2008, at 19:09, Tim wrote: SAS has far greater performance, and if your workload is extremely random, will have a longer MTBF. SATA drives

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
I observe that there are no disk vendors supplying SATA disks with speed 7,200 rpm. It is no wonder that a 10k rpm disk outperforms a 7,200 rpm disk for random workloads. I'll attribute this to intentional market segmentation by the industry rather than a deficiency in the transfer

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: I observe that there are no disk vendors supplying SATA disks with speed 7,200 rpm. It is no wonder that a 10k rpm disk outperforms a 7,200 rpm disk for random workloads. I'll attribute this to intentional market segmentation by the industry rather than

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
So, performance aside, does SAS have other benefits ? Data integrity ? How would a 8 raid1 sata compare vs another 8 smaller SAS disks in raidz(2) ? Like apples and pomegranates. Both should be able to saturate a GbE link. You're the expert, but isn't the 100M/s for streaming not random

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Wed, 1 Oct 2008, Ahmed Kamal wrote: Always assuming 2 spare disks, and Using the sata disks, I would configure them in raid1 mirror (raid6 for the 400G), Besides being cheaper, I would get more useable space (4TB vs 2.4TB), Better performance of raid1 (right?), and better data reliability

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 8:13 PM, Ahmed Kamal [EMAIL PROTECTED] wrote: So, performance aside, does SAS have other benefits ? Data integrity ? How would a 8 raid1 sata compare vs another 8 smaller SAS disks in raidz(2) ? Like apples and pomegranates. Both should be able to saturate a GbE

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Bob Friesenhahn
On Tue, 30 Sep 2008, Robert Thurlow wrote: Modern NFS runs over a TCP connection, which includes its own data validation. This surely helps. Less than we'd sometimes like :-) The TCP checksum isn't very strong, and we've seen corruption tied to a broken router, where the Ethernet

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ahmed Kamal
Hm, richard's excellent Graphs here http://blogs.sun.com/relling/tags/mttdl as well as his words say he prefers mirroring over raidz/raidz2 almost always. It's better for performance and MTTDL. Since 8 sata raid1 is cheaper and probably more reliable than 8 raidz2 sas (and I dont need extra sas

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 6:31 PM, Tim wrote: On Tue, Sep 30, 2008 at 5:19 PM, Erik Trimble [EMAIL PROTECTED] wrote: To make Will's argument more succinct (wink), with a NetApp, undetectable (by the NetApp) errors can be introduced at the HBA and transport layer (FC Switch, slightly damage

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Miles Nordin
rt == Robert Thurlow [EMAIL PROTECTED] writes: dm == David Magda [EMAIL PROTECTED] writes: dm Not of which helped Amazon when their S3 service went down due dm to a flipped bit: ok, I get that S3 went down due to corruption, and that the network checksums I mentioned failed to prevent

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 8:50 PM, Toby Thain [EMAIL PROTECTED]wrote: * NetApp's block-appended checksum approach appears similar but is in fact much stronger. Like many arrays, NetApp formats its drives with 520-byte sectors. It then groups them into 8-sector blocks: 4K of data (the WAFL

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Richard Elling
Ahmed Kamal wrote: So, performance aside, does SAS have other benefits ? Data integrity ? How would a 8 raid1 sata compare vs another 8 smaller SAS disks in raidz(2) ? Like apples and pomegranates. Both should be able to saturate a GbE link. You're the expert, but

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Toby Thain
On 30-Sep-08, at 9:54 PM, Tim wrote: On Tue, Sep 30, 2008 at 8:50 PM, Toby Thain [EMAIL PROTECTED] wrote: NetApp's block-appended checksum approach appears similar but is in fact much stronger. Like many arrays, NetApp formats its drives with 520-byte sectors. It then groups them

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Nicolas Williams
On Tue, Sep 30, 2008 at 08:54:50PM -0500, Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that everyone is using ZFS on the server they're running the application. That does a

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Ian Collins
Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that everyone is using ZFS on the server they're running the application. That does a GREAT job of creating disparate storage

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 10:44 PM, Toby Thain [EMAIL PROTECTED]wrote: ZFS allows the architectural option of separate storage without losing end to end protection, so the distinction is still important. Of course this means ZFS itself runs on the application server, but so what? --Toby So

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Wed, Oct 1, 2008 at 12:24 AM, Ian Collins [EMAIL PROTECTED] wrote: Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that everyone is using ZFS on the server they're

Re: [zfs-discuss] Quantifying ZFS reliability

2008-09-30 Thread Tim
On Tue, Sep 30, 2008 at 11:58 PM, Nicolas Williams [EMAIL PROTECTED] wrote: On Tue, Sep 30, 2008 at 08:54:50PM -0500, Tim wrote: As it does in ANY fileserver scenario, INCLUDING zfs. He is building a FILESERVER. This is not an APPLICATION server. You seem to be stuck on this idea that

[zfs-discuss] Quantifying ZFS reliability

2008-09-29 Thread Ahmed Kamal
Hi everyone, We're a small Linux shop (20 users). I am currently using a Linux server to host our 2TBs of data. I am considering better options for our data storage needs. I mostly need instant snapshots and better data protection. I have been considering EMC NS20 filers and Zfs based solutions.

  1   2   >