Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On 10/25/2012 05:59 AM, Jerry Kemp wrote: I have just acquired a new JBOD box that will be used as a media center/storage for home use only on my x86/x64 box running OpenIndiana b151a7 currently. Its strictly a JBOD, no hw raid options, with an eSATA port to each drive. I am looking for suggestions for an HBA card with at least (2), but (4) external eSATA ports would be nice. I know enough to stay away from the port expander things. I do not need the HBA to support any internal drives. In reviewing the archives/past post, it seems that LSI is the way to go. I would like to spend USD $200 - $300, but would spend more if necessary for a good, trouble free HBA. I made this comment as I went to look at some of the LSI cards previously mentioned, and found they were priced $500 - $600 and up. TIA for any pointers, Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Cheers, -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Scrub and checksum permutations
Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror halves? That is, IIRC, the scrub should try to read all allocated blocks and if they are read in OK - fine; if not - fix in-place with redundant data or copies, if available. On the other hand, if the first tested mirror half/block copy/raidzN permutation has yielded no errors, are other variants still checked? The descriptions I've seen so far put an emphasis on verifying all copies of the blocks and probably all the mirror halves - perhaps because these examples make it easier to describe the concept. I don't think I saw a statement that for raidzN blocks all of the combinations are verified to work (plain userdata sectors, and parity permutations with these sectors). Can someone in the know say yes? ;) Thanks, //Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Scrub and checksum permutations
I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to verify all redundant data. The point of a scrub is to ensure all data is correct. On 2012-10-25 10:25, Jim Klimov wrote: Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror halves? That is, IIRC, the scrub should try to read all allocated blocks and if they are read in OK - fine; if not - fix in-place with redundant data or copies, if available. On the other hand, if the first tested mirror half/block copy/raidzN permutation has yielded no errors, are other variants still checked? The descriptions I've seen so far put an emphasis on verifying all copies of the blocks and probably all the mirror halves - perhaps because these examples make it easier to describe the concept. I don't think I saw a statement that for raidzN blocks all of the combinations are verified to work (plain userdata sectors, and parity permutations with these sectors). Can someone in the know say yes? ;) Thanks, //Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Scrub and checksum permutations
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Karl Wagner I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to verify all redundant data. The point of a scrub is to ensure all data is correct. Same for me. Think about it: When you write some block, it computes parity bits, and writes them to the redundant parity disks. When you later scrub the same data, it wouldn't make sense to do anything other than repeating this process, to verify all the disks including parity. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Scrub and checksum permutations
2012-10-25 15:30, Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) пишет: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Karl Wagner I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to verify all redundant data. The point of a scrub is to ensure all data is correct. Same for me. Think about it: When you write some block, it computes parity bits, and writes them to the redundant parity disks. When you later scrub the same data, it wouldn't make sense to do anything other than repeating this process, to verify all the disks including parity. Logically, yes - I agree this is what we expect to be done. However, at least with the normal ZFS reading pipeline, reads of redundant copies and parities only kick in if the first read variant of the block had errors (HW IO errors, checksum mismatch). In case of raidzN, normal reads should first try to use the unencumbered plain userdata sectors (with IO speeds like striping), and if there are errors - retry with permutations based on parity sectors and different combinations of userdata sectors, until it makes a block whose checksum matches the expected value - or fails by running out of combinations. If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in order to ensure that parity sectors are intact and can be used for data recovery if a plain sector fails. Likely, raidzN scrubs should show as compute-intensive tasks compared to similar mirror scrubs. I thought about it, wondered and posted the question, and went on to my other work. I did not (yet) research the code to find first-hand, partly because gurus might know the answer and reply faster than I dig into it ;) Thanks, //Jim Klimov ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. I purchased an eSATA card (from SIIG, http://www.siig.com/) with the intention to try it with Solaris 10 to see if it would work but have not tried plugging it in yet. It seems likely that a numer of cheap eSATA cards may work. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On 10/25/2012 04:09 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. eSATA is just SATA with a different connector - all you need is a cheap conversion cable or appropriate eSATA-SATA bracket, e.g. http://www.satacables.com/html/sata-pci-brackets.html Cheers, -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On 10/25/2012 04:11 PM, Sašo Kiselkov wrote: On 10/25/2012 04:09 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. eSATA is just SATA with a different connector - all you need is a cheap conversion cable or appropriate eSATA-SATA bracket, e.g. http://www.satacables.com/html/sata-pci-brackets.html P.S. if you're going to use a port conversion bracket, I suggest you get the internal version of the SAS card (i.e. the Dell PERC H200) - that ships with two SFF-8087 connectors. Get a SFF-8087 breakout cable to split out the individual SATA connections (e.g. http://www.ipcdirect.net/catalog/c-sff8087-4sb.jpg ) and just hook those up to the SATA-eSATA bracket. From there on you can take it using standard eSATA cabling directly to your JBOD. Cheers, -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Scrub and checksum permutations
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Logically, yes - I agree this is what we expect to be done. However, at least with the normal ZFS reading pipeline, reads of redundant copies and parities only kick in if the first read variant of the block had errors (HW IO errors, checksum mismatch). I haven't read or written the code myself personally, so I'm not authoritative. But I certainly know I've heard it said on this list before, that when you read a mirror, it only reads one side (as you said) unless there's an error; this allows a mirror to read 2x faster than a single disk (which I confirm by benchmarking.) However, a scrub reads both sides, all redundant copies of the data. I'm personally comfortably confident assuming this is true also for reading the redundant copies of raidzN data. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On Thu, Oct 25, 2012 at 10:13 AM, Sašo Kiselkov skiselkov...@gmail.comwrote: On 10/25/2012 04:11 PM, Sašo Kiselkov wrote: On 10/25/2012 04:09 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. eSATA is just SATA with a different connector - all you need is a cheap conversion cable or appropriate eSATA-SATA bracket, e.g. http://www.satacables.com/html/sata-pci-brackets.html P.S. if you're going to use a port conversion bracket, I suggest you get the internal version of the SAS card (i.e. the Dell PERC H200) - that ships with two SFF-8087 connectors. Get a SFF-8087 breakout cable to split out the individual SATA connections (e.g. http://www.ipcdirect.net/catalog/c-sff8087-4sb.jpg ) and just hook those up to the SATA-eSATA bracket. From there on you can take it using standard eSATA cabling directly to your JBOD. You could also do something like converting an 8087 to an 8088 and get an 8088 - 4x eSATA breakout cable. That would save a lot of space in your case if you're going to have more than 8 drives in this rig. Random googling: http://www.pc-pitstop.com/sas_cables_adapters/AD8788-2.asp and http://www.amazon.com/HighPoint-External-Mini-SAS-SFF8088-Ext-MS-1MES/dp/B000JQ51CM or something. Cheers, -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Patrick Hahn ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On 10/25/2012 04:28 PM, Patrick Hahn wrote: On Thu, Oct 25, 2012 at 10:13 AM, Sašo Kiselkov skiselkov...@gmail.comwrote: On 10/25/2012 04:11 PM, Sašo Kiselkov wrote: On 10/25/2012 04:09 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. eSATA is just SATA with a different connector - all you need is a cheap conversion cable or appropriate eSATA-SATA bracket, e.g. http://www.satacables.com/html/sata-pci-brackets.html P.S. if you're going to use a port conversion bracket, I suggest you get the internal version of the SAS card (i.e. the Dell PERC H200) - that ships with two SFF-8087 connectors. Get a SFF-8087 breakout cable to split out the individual SATA connections (e.g. http://www.ipcdirect.net/catalog/c-sff8087-4sb.jpg ) and just hook those up to the SATA-eSATA bracket. From there on you can take it using standard eSATA cabling directly to your JBOD. You could also do something like converting an 8087 to an 8088 and get an 8088 - 4x eSATA breakout cable. That would save a lot of space in your case if you're going to have more than 8 drives in this rig. Random googling: http://www.pc-pitstop.com/sas_cables_adapters/AD8788-2.asp and http://www.amazon.com/HighPoint-External-Mini-SAS-SFF8088-Ext-MS-1MES/dp/B000JQ51CM or something. Nice, good to know that mini-SAS to eSATA cables exist. Thanks for the info, you learn something new every day. Cheers, -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On Thu, 25 Oct 2012, Sašo Kiselkov wrote: On 10/25/2012 04:09 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. eSATA is just SATA with a different connector - all you need is a cheap conversion cable or appropriate eSATA-SATA bracket, e.g. http://www.satacables.com/html/sata-pci-brackets.html While this can certainly work, according to Wikipedia (http://en.wikipedia.org/wiki/Esata#eSATA), eSATA is more than just SATA with a different connector. eSATA specifies a higher voltage range (minimum voltage) than SATA. It may be that a HBA already uses this range, or maybe not. Text I read says that maximum cable length is significantly reduced if an adaptor is used. Also, I am curious to know how well hot-swap works with an enterprise-class SAS HBA and these cheap eSATA adaptors. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On 10/25/2012 05:40 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: On 10/25/2012 04:09 PM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. eSATA is just SATA with a different connector - all you need is a cheap conversion cable or appropriate eSATA-SATA bracket, e.g. http://www.satacables.com/html/sata-pci-brackets.html While this can certainly work, according to Wikipedia (http://en.wikipedia.org/wiki/Esata#eSATA), eSATA is more than just SATA with a different connector. eSATA specifies a higher voltage range (minimum voltage) than SATA. It may be that a HBA already uses this range, or maybe not. Text I read says that maximum cable length is significantly reduced if an adaptor is used. Also, I am curious to know how well hot-swap works with an enterprise-class SAS HBA and these cheap eSATA adaptors. It may be that you'll get reduced cabling range (only up to SATA lengths, obviously), but it works. The voltage differences are very small and should only come into play when you're pushing the envelope of the cable length. Cheers, -- Saso ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
On 10/25/2012 11:44 AM, Sašo Kiselkov wrote: It may be that you'll get reduced cabling range (only up to SATA lengths, obviously), but it works. The voltage differences are very small and should only come into play when you're pushing the envelope of the cable length. I have a two-drive esata hot swap box I use for monthly zfs backups. Target disk is a WD black 1TB. The enclosure takes an esata connector, so I had to get a sata/esata gender adapter to allow me to plug one of the sata connectors on my 8087 forward breakout cable into it. It's only 18 inches long or so, and I've never had any issues at all. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Scrub and checksum permutations
On Thu, Oct 25, 2012 at 7:35 AM, Jim Klimov jimkli...@cos.ru wrote: If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in order to ensure that parity sectors are intact and can be used for data recovery if a plain sector fails. Likely, raidzN scrubs should show as compute-intensive tasks compared to similar mirror scrubs. It should only be as compute intensive as writes - it can read the userdata and parity sectors, ensure the userdata checksum matches (reconstruct and do a fresh write in the rare cases it is not), and then recalculate the parity sectors from the verified user sectors, and compare them to the parity sectors it actually read. The only reason it would need to use the combinatorial approach is if it has a checksum mismatch and needs to rebuild the data in the presence of bit rot. I thought about it, wondered and posted the question, and went on to my other work. I did not (yet) research the code to find first-hand, partly because gurus might know the answer and reply faster than I dig into it ;) I recently wondered this also and am glad you asked, I hope someone can answer definitively. Tim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Scrub and checksum permutations
2012-10-25 21:17, Timothy Coalson wrote: On Thu, Oct 25, 2012 at 7:35 AM, Jim Klimov jimkli...@cos.ru mailto:jimkli...@cos.ru wrote: If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in order to ensure that parity sectors are intact and can be used for data recovery if a plain sector fails. Likely, raidzN scrubs should show as compute-intensive tasks compared to similar mirror scrubs. It should only be as compute intensive as writes - it can read the userdata and parity sectors, ensure the userdata checksum matches (reconstruct and do a fresh write in the rare cases it is not), and then recalculate the parity sectors from the verified user sectors, and compare them to the parity sectors it actually read. Hmmm, that's another way to skin the cat, and it makes more sense ;) //Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] suggestions for e-SATA HBA card on x86/x64
Hello Bob, Thanks for the SIIG pointer, most of the stuff I had archived from this list pointed to LSI products. I poked around on the site and reviewed SIIG's SATA and SAS HBA. I also hit up their search engine. I'm not implying I did an all inclusive search, but nothing I came across on their site indicated any type of Solaris or *Solaris distro support. Did I miss something on the site? Or maybe one of their sales people let you know this stuff worked with Solaris? Or should it just work as long as it meets SAS or SATA standards? Thanks again for your reply, Jerry Kemp On 10/25/12 09:09 AM, Bob Friesenhahn wrote: On Thu, 25 Oct 2012, Sašo Kiselkov wrote: Look for Dell's 6Gbps SAS HBA cards. They can be had new for $100 and are essentially rebranded LSI 9200-8e cards. Always try to look for OEM cards with LSI, because buying directly from them is incredibly expensive. Do these support eSATA? It seems unlikely. I purchased an eSATA card (from SIIG, http://www.siig.com/) with the intention to try it with Solaris 10 to see if it would work but have not tried plugging it in yet. It seems likely that a numer of cheap eSATA cards may work. Bob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
