Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread Roy Sigurd Karlsbakk
- Daniel Carosone d...@geek.com.au skrev:

 On Mon, Apr 26, 2010 at 10:02:42AM -0700, Chris Du wrote:
  SAS: full duplex
  SATA: half duplex
  
  SAS: dual port
  SATA: single port (some enterprise SATA has dual port)
  
  SAS: 2 active channel - 2 concurrent write, or 2 read, or 1 write
 and 1 read
  SATA: 1 active channel - 1 read or 1 write
  
  SAS: Full error detection and recovery on both read and write
  SATA: error detection and recovery on write, only error detection on
 read
 
 SAS:  Full SCSI TCQ
 SATA: Lame ATA NCQ

What's so lame about NCQ?

roy
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread David Dyer-Bennet

On Mon, April 26, 2010 17:21, Edward Ned Harvey wrote:

 Also, if you've got all those disks in an array, and they're MTBF is ...
 let's say 25,000 hours ... then 3 yrs later when they begin to fail, they
 have a tendency to all fail around the same time, which increases the
 probability of exceeding your designed level of redundancy.

It's useful to consider this when doing mid-life upgrades.  Unfortunately
there's not too much useful to be done right now with RAID setups.

With mirrors, when adding some disks mid-life (seems like a common though
by no means universal scenario to not fully populate the chassis at first,
and add more 1/3 to 1/2 way through the projected life), with some extra
trouble one can attach a new disk as a n+1st disk in an existing mirror,
wait for the resilver, and detach an old disk.  That mirror is now one new
disk and one old disk, rather than two disks of the same age.  Then build
a new mirror out of the freed disk plus another new disk.  Now you've got
both mirrors consisting of disks of different ages, less prone to failing
at the same time.  (Of course this doesn't work when you're using bigger
drives for the mid-life kicker, and most of the time it would make sense
to do so.)

Even buying different (mixed) brands initially doesn't help against aging;
only against batch or design problems.

Hey, you know what might be helpful?  Being able to add redundancy to a
raid vdev.  Being able to go from RAIDZ2 to RAIDZ3 by adding another drive
of suitable size.  Also being able to go the other way.  This lets you do
the trick of temporarily adding redundancy to a vdev while swapping out
devices one at a time to eventually upgrade the size (since you're
deliberately creating a fault situation, increasing redundancy before you
do it makes loads of sense!).

 I recently bought 2x 1Tb disks for my sun server, for $650 each.  This was
 enough to make me do the analysis, why am I buying sun branded overpriced
 disks?  Here is the abridged version:

No argument that, in the existing market, with various levels of need,
this is often the right choice.

I find it deeply frustrating and annoying that this dilemma exists
entirely due to bad behavior by the disk companies, though.  First they
sell deliberately-defective drives (lie about cache flush, for example)
and then they (in conspiracy with an accomplice company) charge us many
times the cost of the physical hardware for fixed versions.  This MUST be
stopped.  This is EXACTLY what standards exist for -- so we can buy
known-quantity products in a competitive market.

-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread Bob Friesenhahn

On Tue, 27 Apr 2010, David Dyer-Bennet wrote:


Hey, you know what might be helpful?  Being able to add redundancy to a
raid vdev.  Being able to go from RAIDZ2 to RAIDZ3 by adding another drive
of suitable size.  Also being able to go the other way.  This lets you do
the trick of temporarily adding redundancy to a vdev while swapping out
devices one at a time to eventually upgrade the size (since you're
deliberately creating a fault situation, increasing redundancy before you
do it makes loads of sense!).


You can already replace one drive with another (zpool replace) so as 
long as there is space for the new drive, it is not necessary to 
degrade the array and lose redundancy while replacing a device.  As 
long as you can physically add a drive to the system (even 
temporarily) it is not necessary to deliberately create a fault 
situation.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread David Dyer-Bennet

On Tue, April 27, 2010 10:38, Bob Friesenhahn wrote:
 On Tue, 27 Apr 2010, David Dyer-Bennet wrote:

 Hey, you know what might be helpful?  Being able to add redundancy to a
 raid vdev.  Being able to go from RAIDZ2 to RAIDZ3 by adding another
 drive
 of suitable size.  Also being able to go the other way.  This lets you
 do
 the trick of temporarily adding redundancy to a vdev while swapping out
 devices one at a time to eventually upgrade the size (since you're
 deliberately creating a fault situation, increasing redundancy before
 you
 do it makes loads of sense!).

 You can already replace one drive with another (zpool replace) so as
 long as there is space for the new drive, it is not necessary to
 degrade the array and lose redundancy while replacing a device.  As
 long as you can physically add a drive to the system (even
 temporarily) it is not necessary to deliberately create a fault
 situation.

I don't think I understand your scenario here.  The docs online at
http://docs.sun.com/app/docs/doc/819-5461/gazgd?a=view describe uses of
zpool replace that DO run the array degraded for a while, and don't seem
to mention any other.

Could you be more detailed?
-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread Bob Friesenhahn

On Tue, 27 Apr 2010, David Dyer-Bennet wrote:


I don't think I understand your scenario here.  The docs online at
http://docs.sun.com/app/docs/doc/819-5461/gazgd?a=view describe uses of
zpool replace that DO run the array degraded for a while, and don't seem
to mention any other.

Could you be more detailed?


If a disk has failed, then it makes sense to physically remove the old 
disk, insert a new one, and do 'zpool replace tank c1t1d0'.  However 
if the disk has not failed, then you can install a new disk in another 
location and use the two argument form of replace like 'zpool replace 
tank c1t1d0 c1t1d7'.  If I understand things correctly, this allows 
you to replace one good disk with another without risking the data in 
your pool.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread David Dyer-Bennet

On Tue, April 27, 2010 11:17, Bob Friesenhahn wrote:
 On Tue, 27 Apr 2010, David Dyer-Bennet wrote:

 I don't think I understand your scenario here.  The docs online at
 http://docs.sun.com/app/docs/doc/819-5461/gazgd?a=view describe uses
 of
 zpool replace that DO run the array degraded for a while, and don't seem
 to mention any other.

 Could you be more detailed?

 If a disk has failed, then it makes sense to physically remove the old
 disk, insert a new one, and do 'zpool replace tank c1t1d0'.  However
 if the disk has not failed, then you can install a new disk in another
 location and use the two argument form of replace like 'zpool replace
 tank c1t1d0 c1t1d7'.  If I understand things correctly, this allows
 you to replace one good disk with another without risking the data in
 your pool.

I don't see any reason to think the old device remains in use until the
new device is resilvered, and if it doesn't, then you're down one level of
redundancy the instant the old device goes out of service.

I don't have a RAIDZ group, but trying this while there's significant load
on the group, it should be easy to see if there's traffic on the old drive
after the resilver starts.  If there is, that would seem to be evidence
that it's continuing to use the old drive while resilvering to the new
one, which would be good.

-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread Bob Friesenhahn

On Tue, 27 Apr 2010, David Dyer-Bennet wrote:


I don't have a RAIDZ group, but trying this while there's significant load
on the group, it should be easy to see if there's traffic on the old drive
after the resilver starts.  If there is, that would seem to be evidence
that it's continuing to use the old drive while resilvering to the new
one, which would be good.


If you have a pool on just a single drive and you use 'zpool replace 
foo bar' to move the pool data from drive 'foo' to drive 'bar', does 
it stop reading drive 'foo' immediately when the transfer starts? 
Please do me a favor and check this for me.


Thanks,

Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread Daniel Carosone
On Tue, Apr 27, 2010 at 10:36:37AM +0200, Roy Sigurd Karlsbakk wrote:
 - Daniel Carosone d...@geek.com.au skrev:
  SAS:  Full SCSI TCQ
  SATA: Lame ATA NCQ
 
 What's so lame about NCQ?

Primarily, the meager number of outstanding requests; write cache is
needed to pretend the writes are done straight away and free up the
slots for reads.  

If you want throughput, you want to hand the disk controller as many
requests as possible, so it can optimise seek order.  If you have
especially latency-sensitive requests, you need to manage the queue
carefully with either system.

--
Dan.

pgpf0r3L8VyeA.pgp
Description: PGP signature
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-27 Thread Brandon High
On Tue, Apr 27, 2010 at 2:47 PM, Daniel Carosone d...@geek.com.au wrote:
 What's so lame about NCQ?

 Primarily, the meager number of outstanding requests; write cache is
 needed to pretend the writes are done straight away and free up the
 slots for reads.

NCQ handles 32 outstanding operations. Considering that ZFS limits the
outstanding requests to 10 (as of snv_125 I think?), that's not an
issue. TCQ supports between 16 and 64 bits for the tags, depending on
the implementation and underlying protocol.

TCQ allows a command to be added to the as head of the queue, ordered,
or simple. I don't believe that NCQ allows multiple queuing methods,
and I can't be bothered to check.

-B

-- 
Brandon High : bh...@freaks.com
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Roy Sigurd Karlsbakk
- Dave Pooser dave@alfordmedia.com skrev:

 I'm building another 24-bay rackmount storage server, and I'm
 considering
 what drives to put in the bays. My chassis is a Supermicro SC846A, so
 the
 backplane supports SAS or SATA; my controllers are LSI3081E, again
 supporting SAS or SATA.
 
 Looking at drives, Seagate offers an enterprise (Constellation) 2TB
 7200RPM
 drive in both SAS and SATA configurations; the SAS model offers one
 quarter
 the buffer (16MB vs 64MB on the SATA model), the same rotational
 speed, and
 costs 10% more than its enterprise SATA twin. (They also offer a
 Barracuda
 XT SATA drive; it's roughly 20% less expensive than the Constellation
 drive,
 but rated at 60% the MTBF of the others and a predicted rate of
 nonrecoverable errors an order of magnitude higher.)
 
 Assuming I'm going to be using three 8-drive RAIDz2 configurations,
 and
 further assuming this server will be used for backing up home
 directories
 (lots of small writes/reads), how much benefit will I see from the
 SAS
 interface?

We haver a similar system, SuperMicro 24-bay server with 22x2TB (and two SSDs 
for the root) configured as three RAIDz2 sets with seven drives each and a 
spare. We chose 'desktop' drives, since they offer (more or less) the same 
speed and with that redundancy, the chance for pool failure is so low, I guess 
'enterprise' drives wouldn't help a lot more.

About SAS vs SATA, I'd guess you won't be able to see any change at all. The 
bottleneck is the drives, not the interface to them.

roy
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Edward Ned Harvey
 From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
 boun...@opensolaris.org] On Behalf Of Dave Pooser
 
 (lots of small writes/reads), how much benefit will I see from the SAS
 interface?

In some cases, SAS outperforms SATA.  I don't know what circumstances those
are.

I think the main reason anyone buys SAS disks is for reliability reasons.  I
maintain data centers for two companies, one of which uses all SAS, and the
other uses mostly SATA.  I have replaced many SATA disks in the last 3
years, and I have never replaced a single SAS disk.

I don't know if my experience would be reflected in the published MTBF of
the disks in question.  Sometimes those numbers are sort of fudged, so I
don't trust 'em or bother to look at them.

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Edward Ned Harvey
 From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
 boun...@opensolaris.org] On Behalf Of Roy Sigurd Karlsbakk
 
 About SAS vs SATA, I'd guess you won't be able to see any change at
 all. The bottleneck is the drives, not the interface to them.

That doesn't agree with my understanding.  My understanding, for a single
disk, you're right.  No disk can come near the bus speed for either SATA or
SAS.  But SCSI vs ATA, the SCSI is supposed to have a more efficient bus
utilization when many disks are all doing thing simultaneously, such as they
might in a big old RAID, 48 disks, etc, like you have.

'Course none of that matters if you're serving it all over a 1Gb ether.  ;-)

I don't know under what circumstances SAS performance would be SATA.  Nor do
I know by how much.

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Richard Elling
On Apr 25, 2010, at 10:02 PM, Dave Pooser wrote:

 I'm building another 24-bay rackmount storage server, and I'm considering
 what drives to put in the bays. My chassis is a Supermicro SC846A, so the
 backplane supports SAS or SATA; my controllers are LSI3081E, again
 supporting SAS or SATA.
 
 Looking at drives, Seagate offers an enterprise (Constellation) 2TB 7200RPM
 drive in both SAS and SATA configurations; the SAS model offers one quarter
 the buffer (16MB vs 64MB on the SATA model), the same rotational speed, and
 costs 10% more than its enterprise SATA twin. (They also offer a Barracuda
 XT SATA drive; it's roughly 20% less expensive than the Constellation drive,
 but rated at 60% the MTBF of the others and a predicted rate of
 nonrecoverable errors an order of magnitude higher.)
 
 Assuming I'm going to be using three 8-drive RAIDz2 configurations, and
 further assuming this server will be used for backing up home directories
 (lots of small writes/reads), how much benefit will I see from the SAS
 interface?

For a single connection from a host to a disk, they are basically equivalent.
SAS shines with multiple connections to one or more hosts.  Hence, SAS 
is quite popular when implementing HA clusters.

Note: drive differentiation is market driven, not technology driven.
 -- richard

ZFS storage and performance consulting at http://www.RichardElling.com
ZFS training on deduplication, NexentaStor, and NAS performance
Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com 





___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Chris Du
SAS: full duplex
SATA: half duplex

SAS: dual port
SATA: single port (some enterprise SATA has dual port)

SAS: 2 active channel - 2 concurrent write, or 2 read, or 1 write and 1 read
SATA: 1 active channel - 1 read or 1 write

SAS: Full error detection and recovery on both read and write
SATA: error detection and recovery on write, only error detection on read

If you connect only one disk per port, not a big deal. If you connect multiple 
disks to raid card, or through backplane, expander, SAS makes big difference on 
reliability. 

If I had the money, I always go with SAS.
-- 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Brandon High
On Sun, Apr 25, 2010 at 10:02 PM, Dave Pooser dave@alfordmedia.com wrote:
 Assuming I'm going to be using three 8-drive RAIDz2 configurations, and
 further assuming this server will be used for backing up home directories
 (lots of small writes/reads), how much benefit will I see from the SAS
 interface?

SAS drives are generally intended to be used in a multi-drive / RAID
environment, and are delivered with TLER / CCTL / ERC enabled to
prevent them from falling out of arrays when they hit a read error.

SAS drives will generally have a longer warranty than desktop drives.

The SMART command set in ATA-7 and the ATA-8 spec should eliminate the
distinction, but until it's fully supported by manufacturers desktop
drives may not degrade as gracefully in an array when hitting an
error. From what I've read, both WD and Seagate desktop drives ignore
the ERC command. Samsung drives are reported to work, and I'm not sure
about Hitachi.

So far as backplanes are concerned - You can connect the backplane
with SAS and still use SATA drives.

-B

-- 
Brandon High : bh...@freaks.com
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Roy Sigurd Karlsbakk
- Brandon High bh...@freaks.com skrev:

 SAS drives are generally intended to be used in a multi-drive / RAID
 environment, and are delivered with TLER / CCTL / ERC enabled to
 prevent them from falling out of arrays when they hit a read error.
 
 SAS drives will generally have a longer warranty than desktop drives.

With 2TB drives priced at €150 or lower, I somehow think paying for drive 
lifetime is far more expensive than getting a few more drives and add redundancy

Just my 2c

roy
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Dave Pooser
On 4/26/10 10:10 AM, Richard Elling richard.ell...@gmail.com wrote:

 SAS shines with multiple connections to one or more hosts.  Hence, SAS
 is quite popular when implementing HA clusters.

So that would be how one builds something like the active/active controller
failover in standalone RAID boxes. Is there a good resource on doing
something like that with an OpenSolaris storage server? I could see that as
a project I might want to attempt.
-- 
Dave Pooser, ACSA
Manager of Information Services
Alford Media  http://www.alfordmedia.com


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Bob Friesenhahn

On Mon, 26 Apr 2010, Roy Sigurd Karlsbakk wrote:


SAS drives will generally have a longer warranty than desktop drives.


With 2TB drives priced at €150 or lower, I somehow think paying for 
drive lifetime is far more expensive than getting a few more drives 
and add redundancy


This really depends on if you are willing to pay in advance, or pay 
after the failure.  Even with redundancy, the cost of a failure may be 
high due to loss of array performance and system administration time. 
Array performance may go into the toilet during resilvers, depending 
on the redundancy configuration and the type of drives used.


All types of drives fail but typical SATA drives fail more often.

Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Roy Sigurd Karlsbakk
 This really depends on if you are willing to pay in advance, or pay 
 after the failure.  Even with redundancy, the cost of a failure may be
 
 high due to loss of array performance and system administration time.
 
 Array performance may go into the toilet during resilvers, depending 
 on the redundancy configuration and the type of drives used.
 
 All types of drives fail but typical SATA drives fail more often.

Failure ratio does not depend on interface. Enterprise grade SATA drives have 
the same build quality as with their SAS brothers and sisters. With RAIDz2 or 
-3, you're quite sure things will work fine even after a disk failure, and the 
performance penalty isn't that bad. Choosing SAS over SATA for a single setup 
must be more of a religious approach

roy
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Gary Mills
On Mon, Apr 26, 2010 at 01:32:33PM -0500, Dave Pooser wrote:
 On 4/26/10 10:10 AM, Richard Elling richard.ell...@gmail.com wrote:
 
  SAS shines with multiple connections to one or more hosts.  Hence, SAS
  is quite popular when implementing HA clusters.
 
 So that would be how one builds something like the active/active controller
 failover in standalone RAID boxes. Is there a good resource on doing
 something like that with an OpenSolaris storage server? I could see that as
 a project I might want to attempt.

This is interesting.  I have a two-node SPARC cluster that uses a
multi-initiator SCSI array for shared storage.  As an application
server, it need only two disks in the array.  They are a ZFS mirror.
This all works quite nicely under Sun Cluster.

I'd like to duplicate this configuration with two small x86 servers
and a small SAS array, also with only two disks.  It should be easy to
find a pair of 1U servers, but what's the smallest SAS array that's
available?  Does it need an array controller?  What's needed on the
servers to connect to it?

-- 
-Gary Mills--Unix Group--Computer and Network Services-
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Edward Ned Harvey
 From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
 boun...@opensolaris.org] On Behalf Of Roy Sigurd Karlsbakk
 
 With 2TB drives priced at €150 or lower, I somehow think paying for
 drive lifetime is far more expensive than getting a few more drives and
 add redundancy

If you have a 48-disk enclosure, and you've configured 6x 8disk raid-6 or 
raidz2 volumes, how do you add more disks to increase redundancy?

Point is:  Adding disks often means adding slots, and since adding slots ain't 
free, it would generally translate not so much as adding slots, but decreasing 
the number of usable drive capacities...  And keeping an inventory of offline 
spares for the sake of immediate replacement upon failure.

Also, you'll only find the cheapest generic disks available at the stated 
price.  If you have one of those disks fail 6 months from now, you will not be 
able to purchase that model drive again.  (God forbid you should have to 
replace one 3 yrs from now, when the current implementation of SAS or SATA 
isn't even for sale anymore, and you can't even get a suitable equivalent 
replacement.)

I hate it whenever people over-simplify and say disk is cheap.

Also, if you've got all those disks in an array, and they're MTBF is ... let's 
say 25,000 hours ... then 3 yrs later when they begin to fail, they have a 
tendency to all fail around the same time, which increases the probability of 
exceeding your designed level of redundancy.

I recently bought 2x 1Tb disks for my sun server, for $650 each.  This was 
enough to make me do the analysis, why am I buying sun branded overpriced 
disks?  Here is the abridged version:

We recently had an Apple XRAID system lose a disk.  It's 3 yrs old.  It uses 
500G ATA-133 disks, which are not available from anywhere at any price...  
Except Apple was willing to sell us one for $1018.  Naturally, we declined to 
make that purchase.  We did find some disks available from various sources, 
which should be equivalent, but not Apple branded or certified; functional 
equivalents but not identical.  Prices around $200 to $300.

I asked around, apple admins who had used generic disks in their Xraid systems. 
 About 50% said they used generic disks with no problem.  The other 50% were 
mixed between we used generic disks, seemed to work, but had strange problems 
like horrible performance or disk suddenly going offline and coming back online 
again spontaneously and we tried to use generic disks, but the system refused 
to even acknowledge the disk present in the system.

Also, take a look in the present mailing list, many people complaining of 
drives with firmwares that incorrectly acknowledge cache flushes before they're 
actually flushed.  Even then, we're talking about high end Intel SSD's.  And 
the consequence of incorrect firmware is data loss.  Maybe even pool loss.

The reason why we pay for overpriced disks is to get the manufacturer's seal of 
approval, the Apple or Sun or Dell branded firmware.  The availability of mfgr 
warranties, the long-term supportability.  It costs about 4x-5x more per disk 
to buy up front, but since you have to buy 2x as many generic disks (for the 
sake of spare inventory availability) you're only paying 2x overall, and you 
can rest much more assured in the stability.

Even at the higher hardware price, the value of the data is presumed to be much 
greater than the cost of the hardware.  So then it's easy to justify higher 
cost hardware, with the belief it'll be somehow lower data risk.

Sometimes people will opt for cheaper.  Sometimes people will opt for lower 
risk.

I just hate it when people oversimplify and say disk is cheap.  That is so 
over simplified, it doesn't benefit anyone.

end rant  begin breathe ...

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread James C. McPherson

On 26/04/10 03:02 PM, Dave Pooser wrote:

I'm building another 24-bay rackmount storage server, and I'm considering
what drives to put in the bays. My chassis is a Supermicro SC846A, so the
backplane supports SAS or SATA; my controllers are LSI3081E, again
supporting SAS or SATA.

Looking at drives, Seagate offers an enterprise (Constellation) 2TB 7200RPM
drive in both SAS and SATA configurations; the SAS model offers one quarter
the buffer (16MB vs 64MB on the SATA model), the same rotational speed, and
costs 10% more than its enterprise SATA twin. (They also offer a Barracuda
XT SATA drive; it's roughly 20% less expensive than the Constellation drive,
but rated at 60% the MTBF of the others and a predicted rate of
nonrecoverable errors an order of magnitude higher.)

Assuming I'm going to be using three 8-drive RAIDz2 configurations, and
further assuming this server will be used for backing up home directories
(lots of small writes/reads), how much benefit will I see from the SAS
interface?


I would expect to see the SAS drives have built-in support
for multipathing, with no extra hardware required.

Also, hear yourself chanting but SAS is more ENTERPRISEY
over and over again :-)

I don't know of any other specific difference between Enterprise
SATA and SAS drives.


James C. McPherson
--
Senior Software Engineer, Solaris
Oracle
http://www.jmcp.homeunix.com/blog
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] SAS vs SATA: Same size, same speed, why SAS?

2010-04-26 Thread Daniel Carosone
On Mon, Apr 26, 2010 at 10:02:42AM -0700, Chris Du wrote:
 SAS: full duplex
 SATA: half duplex
 
 SAS: dual port
 SATA: single port (some enterprise SATA has dual port)
 
 SAS: 2 active channel - 2 concurrent write, or 2 read, or 1 write and 1 read
 SATA: 1 active channel - 1 read or 1 write
 
 SAS: Full error detection and recovery on both read and write
 SATA: error detection and recovery on write, only error detection on read

SAS:  Full SCSI TCQ
SATA: Lame ATA NCQ

--
Dan.



pgpfPAxGyNIbj.pgp
Description: PGP signature
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss