Re: [zfs-discuss] SATA hardware advice
Thanks for the sugestions, especially all the HP info and build pictures. Two things crossed my mind on the hardware front. The first is regarding the SSDs you have pictured, mounted in sleds. Any Proliant that I've read about connects the hotswap drives via a SAS backplane. So how did you avoid that (physically) to make the direct SATA connections? The second is regarding a conversation I had with HP pre-sales. A rep actually told me, in no uncertain terms, that using non-HP HBAs, RAM, or drives would completely void my warranty. I assume this is BS but I wonder if anyone has ever gotten resistance due to 3rd party hardware. In the States, at least, there is the Magnuson–Moss act. I'm just not sure if it applies to servers. Back to SATA though. I can appreciate fully about not wanting to take unnecessary risks, but there are a few things that don't sit well with me. A little background: this is to be a backup server for a small/medium business. The data, of course, needs to be safe, but we don't need extreme HA. I'm aware of two specific issues with SATA drives: the TLER/CCTL setting, and the issue with SAS expanders. I have to wonder if these account for most of the bad rap that SATA drives get. Expanders are built into nearly all of the JBODs and storage servers I've found (including the one in the serverfault post), so they must be in common use. So I'll ask again: are there any issues when connecting SATA drives directly to a HBA? People are, after all, talking left and right about using SATA SSDs... as long as they are connected directly to the MB controller. We might just do SAS at this point for peace of mind. It just bugs me that you can't use inexpensive disks in a R.A.I.D. I would think that RAIDZ and AHCI could handle just about any failure mode by now. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
AFAIK, most ZFS based storage appliance are move to SAS with 7200 rpm or 15k rpm most SSD are SATA and are connecting to on bd SATA with IO chips On 12/19/2011 9:59 AM, tono wrote: Thanks for the sugestions, especially all the HP info and build pictures. Two things crossed my mind on the hardware front. The first is regarding the SSDs you have pictured, mounted in sleds. Any Proliant that I've read about connects the hotswap drives via a SAS backplane. So how did you avoid that (physically) to make the direct SATA connections? The second is regarding a conversation I had with HP pre-sales. A rep actually told me, in no uncertain terms, that using non-HP HBAs, RAM, or drives would completely void my warranty. I assume this is BS but I wonder if anyone has ever gotten resistance due to 3rd party hardware. In the States, at least, there is the Magnuson–Moss act. I'm just not sure if it applies to servers. Back to SATA though. I can appreciate fully about not wanting to take unnecessary risks, but there are a few things that don't sit well with me. A little background: this is to be a backup server for a small/medium business. The data, of course, needs to be safe, but we don't need extreme HA. I'm aware of two specific issues with SATA drives: the TLER/CCTL setting, and the issue with SAS expanders. I have to wonder if these account for most of the bad rap that SATA drives get. Expanders are built into nearly all of the JBODs and storage servers I've found (including the one in the serverfault post), so they must be in common use. So I'll ask again: are there any issues when connecting SATA drives directly to a HBA? People are, after all, talking left and right about using SATA SSDs... as long as they are connected directly to the MB controller. We might just do SAS at this point for peace of mind. It just bugs me that you can't use inexpensive disks in a R.A.I.D. I would think that RAIDZ and AHCI could handle just about any failure mode by now. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Hung-Sheng Tsao Ph D. Founder Principal HopBit GridComputing LLC cell: 9734950840 http://laotsao.wordpress.com/ http://laotsao.blogspot.com/ attachment: laotsao.vcf___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
On Dec 19, 2011, at 7:52 AM, Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D. wrote: AFAIK, most ZFS based storage appliance are move to SAS with 7200 rpm or 15k rpm most SSD are SATA and are connecting to on bd SATA with IO chips Most *cheap* SSDs are SATA. But if you want to use them in a cluster configuration, you need to use a SAS device that supports multiple initiators, such as those from STEC. - Garrett On 12/19/2011 9:59 AM, tono wrote: Thanks for the sugestions, especially all the HP info and build pictures. Two things crossed my mind on the hardware front. The first is regarding the SSDs you have pictured, mounted in sleds. Any Proliant that I've read about connects the hotswap drives via a SAS backplane. So how did you avoid that (physically) to make the direct SATA connections? The second is regarding a conversation I had with HP pre-sales. A rep actually told me, in no uncertain terms, that using non-HP HBAs, RAM, or drives would completely void my warranty. I assume this is BS but I wonder if anyone has ever gotten resistance due to 3rd party hardware. In the States, at least, there is the Magnuson–Moss act. I'm just not sure if it applies to servers. Back to SATA though. I can appreciate fully about not wanting to take unnecessary risks, but there are a few things that don't sit well with me. A little background: this is to be a backup server for a small/medium business. The data, of course, needs to be safe, but we don't need extreme HA. I'm aware of two specific issues with SATA drives: the TLER/CCTL setting, and the issue with SAS expanders. I have to wonder if these account for most of the bad rap that SATA drives get. Expanders are built into nearly all of the JBODs and storage servers I've found (including the one in the serverfault post), so they must be in common use. So I'll ask again: are there any issues when connecting SATA drives directly to a HBA? People are, after all, talking left and right about using SATA SSDs... as long as they are connected directly to the MB controller. We might just do SAS at this point for peace of mind. It just bugs me that you can't use inexpensive disks in a R.A.I.D. I would think that RAIDZ and AHCI could handle just about any failure mode by now. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Hung-Sheng Tsao Ph D. Founder Principal HopBit GridComputing LLC cell: 9734950840 http://laotsao.wordpress.com/ http://laotsao.blogspot.com/ laotsao.vcf___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
On 16 Dec 2011, at 23:48, Edmund White wrote: If you're building from scratch, please choose nearline/midline SAS disks instead of SATA if you're looking for capacity. For detailed reasoning, see: http://serverfault.com/a/331504/13325 For the server, I've had great success with HP ProLiant systems, focusing on the DL380 G6/G7 models. If you can budget 4U of rackspace, the DL370 G6 is a good option that can accommodate 14LFF or 24 SFF disks (or a combination). I've built onto DL180 G6 systems as well. If you do the DL180 G6, you'll need a 12-bay LFF model. I'd recommend a Lights-Out 100 license key to gain remote console. The backplane has a built-in SAS expander, so you'll only have a single 4-lane SAS cable to the controller. I typically use LSI controllers. In the DL180, I would spec a LSI 9211-4i SAS HBA. You have room to mount a ZIL or L2Arc internally and leverage the motherboard SATA ports. Otherwise, consider a LSI 9211-8i HBA and use the second 4-land SAS connector for those. See: http://www.flickr.com/photos/ewwhite/sets/72157625918734321/ for an example of the DL380 G7 build. Can you explain how you got the SSDs into the HP sleds? Did you buy blank sleds from somewhere, or cannibalise some cheap HP drives? I assumed some part of the HP hardware would freak out if it ever saw a drive with non-HP firmware - is that a problem? We've got an HP D2700 JBOD attached to an LSI SAS 9208 controller in a DL360G7, and I'm keen on getting a ZIL into the mix somewhere - either into the JBOD or the spare bays in the DL360. Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
On 12/17/11 8:27 PM, Chris Ridd chrisr...@mac.com wrote: Can you explain how you got the SSDs into the HP sleds? Did you buy blank sleds from somewhere, or cannibalise some cheap HP drives? I assumed some part of the HP hardware would freak out if it ever saw a drive with non-HP firmware - is that a problem? We've got an HP D2700 JBOD attached to an LSI SAS 9208 controller in a DL360G7, and I'm keen on getting a ZIL into the mix somewhere - either into the JBOD or the spare bays in the DL360. Chris Chris, It's possible to obtain the HP drive carriers in bulk on eBay. I haven't had many issues with HP backplanes or RAID controllers complaining about non-HP disks. There was one instance of a particular Intel SSD that didn't provide proper temperature data to the HP drive backplane, but that's the worst issue I've ever encountered. Later revisions of the same SSD worked. I also have DL380 G7 with D2700 JBOD setups running. In one, I'm using a Pliant/Sandisk SSD for ZIL. The other has a DDRdrive installed in the storage head. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
On 17 Dec 2011, at 19:35, Edmund White wrote: On 12/17/11 8:27 PM, Chris Ridd chrisr...@mac.com wrote: Can you explain how you got the SSDs into the HP sleds? Did you buy blank sleds from somewhere, or cannibalise some cheap HP drives? I assumed some part of the HP hardware would freak out if it ever saw a drive with non-HP firmware - is that a problem? We've got an HP D2700 JBOD attached to an LSI SAS 9208 controller in a DL360G7, and I'm keen on getting a ZIL into the mix somewhere - either into the JBOD or the spare bays in the DL360. Chris Chris, It's possible to obtain the HP drive carriers in bulk on eBay. I haven't So they are - googling thinks they are hp 378343-002 and seems to find a good number for sale. Good tip, thanks! had many issues with HP backplanes or RAID controllers complaining about non-HP disks. There was one instance of a particular Intel SSD that didn't provide proper temperature data to the HP drive backplane, but that's the worst issue I've ever encountered. Later revisions of the same SSD worked. I also have DL380 G7 with D2700 JBOD setups running. In one, I'm using a Pliant/Sandisk SSD for ZIL. The other has a DDRdrive installed in the storage head. A DDRdrive is beyond our budget :-( My plan B was to put an OCZ revodrive in the spare PCIe slot. But an HP drive carrier + cheap small SSD would be perfect. Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] SATA hardware advice
I could use some help with choosing hardware for a storage server. For budgetary and density reasons, we had settled on LFF SATA drives in the storage server. I had closed in on models from HP (DL180 G6) and IBM (x3630 M3), before discovering warnings against connecting SATA drives with SAS expanders. So I'd like to ask what's the safest way to manage SATA drives. We're looking for a 12 (ideally 14) LFF server, 2-3U, similar to the above models. The HP and IBM models both come with SAS expanders built into their backplanes. My questions are: 1. Kludginess aside, can we build a dependable SMB server using integrated HP or IBM expanders plus the workaround (allow-bus-device-reset=0) presented here: http://gdamore.blogspot.com/2010/12/update-on-sata-expanders.html ? 2. Would it be better to find a SATA card with lots of ports, and make 1:1 connections? I found some cards (arc-128, Adaptec 2820SA) w/Solaris support, for example, but I don't know how reliable they are or whether they support a clean JBOD mode. 3. Assuming native SATA is the way to go, where should we look for hardware? I'd like the IBM HP options because of the LOM warranty, but I wouldn't think the hot-swap backplane offers any way to bypass the SAS expanders (correct me if I'm wrong here!). I found this JBOD: http://www.pc-pitstop.com/sata_enclosures/sat122urd.asp I also know about SuperMicro. Are there any other vendors or models worth considering? Thanks! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
imho, if possible pick sas 7200 hdd no hw-raid for ZFS mirror and with ZIL and good size memory Sent from my iPad On Dec 16, 2011, at 17:36, t...@ownmail.net wrote: I could use some help with choosing hardware for a storage server. For budgetary and density reasons, we had settled on LFF SATA drives in the storage server. I had closed in on models from HP (DL180 G6) and IBM (x3630 M3), before discovering warnings against connecting SATA drives with SAS expanders. So I'd like to ask what's the safest way to manage SATA drives. We're looking for a 12 (ideally 14) LFF server, 2-3U, similar to the above models. The HP and IBM models both come with SAS expanders built into their backplanes. My questions are: 1. Kludginess aside, can we build a dependable SMB server using integrated HP or IBM expanders plus the workaround (allow-bus-device-reset=0) presented here: http://gdamore.blogspot.com/2010/12/update-on-sata-expanders.html ? 2. Would it be better to find a SATA card with lots of ports, and make 1:1 connections? I found some cards (arc-128, Adaptec 2820SA) w/Solaris support, for example, but I don't know how reliable they are or whether they support a clean JBOD mode. 3. Assuming native SATA is the way to go, where should we look for hardware? I'd like the IBM HP options because of the LOM warranty, but I wouldn't think the hot-swap backplane offers any way to bypass the SAS expanders (correct me if I'm wrong here!). I found this JBOD: http://www.pc-pitstop.com/sata_enclosures/sat122urd.asp I also know about SuperMicro. Are there any other vendors or models worth considering? Thanks! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
If you're building from scratch, please choose nearline/midline SAS disks instead of SATA if you're looking for capacity. For detailed reasoning, see: http://serverfault.com/a/331504/13325 For the server, I've had great success with HP ProLiant systems, focusing on the DL380 G6/G7 models. If you can budget 4U of rackspace, the DL370 G6 is a good option that can accommodate 14LFF or 24 SFF disks (or a combination). I've built onto DL180 G6 systems as well. If you do the DL180 G6, you'll need a 12-bay LFF model. I'd recommend a Lights-Out 100 license key to gain remote console. The backplane has a built-in SAS expander, so you'll only have a single 4-lane SAS cable to the controller. I typically use LSI controllers. In the DL180, I would spec a LSI 9211-4i SAS HBA. You have room to mount a ZIL or L2Arc internally and leverage the motherboard SATA ports. Otherwise, consider a LSI 9211-8i HBA and use the second 4-land SAS connector for those. See: http://www.flickr.com/photos/ewwhite/sets/72157625918734321/ for an example of the DL380 G7 build. -- Edmund White ewwh...@mac.com On 12/17/11 12:24 AM, Hung-Sheng Tsao (laoTsao) laot...@gmail.com wrote: imho, if possible pick sas 7200 hdd no hw-raid for ZFS mirror and with ZIL and good size memory Sent from my iPad On Dec 16, 2011, at 17:36, t...@ownmail.net wrote: I could use some help with choosing hardware for a storage server. For budgetary and density reasons, we had settled on LFF SATA drives in the storage server. I had closed in on models from HP (DL180 G6) and IBM (x3630 M3), before discovering warnings against connecting SATA drives with SAS expanders. So I'd like to ask what's the safest way to manage SATA drives. We're looking for a 12 (ideally 14) LFF server, 2-3U, similar to the above models. The HP and IBM models both come with SAS expanders built into their backplanes. My questions are: 1. Kludginess aside, can we build a dependable SMB server using integrated HP or IBM expanders plus the workaround (allow-bus-device-reset=0) presented here: http://gdamore.blogspot.com/2010/12/update-on-sata-expanders.html ? 2. Would it be better to find a SATA card with lots of ports, and make 1:1 connections? I found some cards (arc-128, Adaptec 2820SA) w/Solaris support, for example, but I don't know how reliable they are or whether they support a clean JBOD mode. 3. Assuming native SATA is the way to go, where should we look for hardware? I'd like the IBM HP options because of the LOM warranty, but I wouldn't think the hot-swap backplane offers any way to bypass the SAS expanders (correct me if I'm wrong here!). I found this JBOD: http://www.pc-pitstop.com/sata_enclosures/sat122urd.asp I also know about SuperMicro. Are there any other vendors or models worth considering? Thanks! ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] SATA hardware advice
On Sat, Dec 17, 2011 at 6:48 AM, Edmund White ewwh...@mac.com wrote: If you can budget 4U of rackspace, the DL370 G6 is a good option that can accommodate 14LFF or 24 SFF disks (or a combination). I've built onto DL180 G6 systems as well. If you do the DL180 G6, you'll need a 12-bay LFF model. I'd recommend a Lights-Out 100 license key to gain remote console. The backplane has a built-in SAS expander, so you'll only have a single 4-lane SAS cable to the controller. I typically use LSI controllers. In the DL180, I would spec a LSI 9211-4i SAS HBA. You have room to mount a ZIL or L2Arc internally and leverage the motherboard SATA ports. Otherwise, consider a LSI 9211-8i HBA and use the second 4-land SAS connector for those. See: http://www.flickr.com/photos/ewwhite/sets/72157625918734321/ for an example of the DL380 G7 build. I assume you bought the controller separately, not from HP, right? Are there any other parts you need to buy separately? (e.g. cables) How about the disks? are they from HP? -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
