Re: FreeBSD 6.0 as storage server with raid5?
Sorry for the late reply ... Torfinn Ingolfsen [EMAIL PROTECTED] wrote: Highpoint RocketRAID 1640(4 ports) Promise FastTrak S150 SX4(4 ports) Promise FastTrak S150 SX4-M(4 ports) Highpoint RocketRAID 1810A(4 ports) Highpoint RocketRAID 1820A(8 ports) Intel RAID Controller SRCS16(6 ports) [...] 1) can I install several 4-port RAID controllers in one machine? Yes. There's no reason why you shouldn't be able to do that. I can get two RocketRAID 1640's for the price of one RocketRAID 1820A. Note that the HPT 18x0A have an onboard processor which does the XOR (parity) calculations for RAID-5. For the non-A versions, the host CPU has to do those calculations (in the driver), which can be quite a noticable burden. 3) disks - I understand the importance of using disks from different production runs. Some people have suggested to use disks of different brands as well, but there were no conclusions. Should I stick to the same brand and model, or can I use disks of different brands as long as they are of the same size? Well, both strategies have advantages and disadvantages. But given the fact that the big commercial appliance companies (such as NetApp) ship their filers with disks of the same brand, I guess it's not terrible mistake to do so. 4) PSU considerations. How big a PSU do I need if I want to run two controllers and eight disks in one machine? are there any rules of thumb for sizing this? There are usually three numbers of power-usage provided for hard disks: power usage when idle, max power usage when busy during normal operation, and max poer usage during spin-up. The latter is the largest number, of course. If your disks all spin up at the same time, then you PSU have to be able to provide enough power for the sum of all of your disks (plus the rest of the system), which can be quite a lot. Some controllers have a setting to enable staged starting of the drives, i.e. one after the other. This can reduce the maximum PSU requirement considerably if you have a lot of disks. Best regards Oliver PS: I assume you already know this, but I'd like to note this for all readers: RAID (of any kind) does not replace reliable backups. Always make proper backups, independent of your RAID (if any). -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. (On the statement print 42 monkeys + 1 snake:) By the way, both perl and Python get this wrong. Perl gives 43 and Python gives 42 monkeys1 snake, when the answer is clearly 41 monkeys and 1 fat snake.-- Jim Fulton ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
On Tue, 13 Dec 2005 17:51:02 +0100 (CET) Oliver Fromme [EMAIL PROTECTED] wrote: There's no reason why you shouldn't be able to do that. Noted. Good to know. :-) Note that the HPT 18x0A have an onboard processor which does the XOR (parity) calculations for RAID-5. For the non-A versions, the host CPU has to do those calculations (in the driver), which can be quite a noticable burden. Noted, this is valuable information, thanks! I found a test: http://www6.tomshardware.com/storage/20040625/sata-raid-04.html and this test confirms that the RocketRAID 1640 is software raid only. Some controllers have a setting to enable staged starting of the drives, i.e. one after the other. This can reduce the maximum PSU requirement considerably if you have a lot of disks. Noted, I'll look out for this. PS: I assume you already know this, but I'd like to note this for all readers: RAID (of any kind) does not replace reliable backups. Always make proper backups, independent of your RAID (if any). I agree 100%, and this cannot be told to often. In my case, the backup server will probably be another FreeBSD box with external firewire drives, to keep costs down. -- Regards, Torfinn Ingolfsen, Norway ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, Dec 08, 2005 at 08:49:15PM +0100, Christian Brueffer wrote: On Thu, Dec 08, 2005 at 02:31:00PM +0100, Stijn Hoop wrote: On Thu, Dec 08, 2005 at 12:18:57AM +0100, Torfinn Ingolfsen wrote: I was thinking about gvinum for the storage server, but given the current documentation and the discussions about it now, I don't want to risk it. IMHO it's pretty stable in 6.0. I've been running gvinum RAID-5 for a while now; other than one strange panic (something to do with out of memory situations, see kern/89660) I haven't had a hitch yet. That said, I haven't needed to replace a disk yet either (I've demoed this but it was not yet needed in production use). In short, don't write gvinum off just yet. Documentation is around the corner (as a result of a SoC project). Actually gvinum(8) has been committed to CURRENT and RELENG_6 a couple of days ago. Ah, I missed that, great :-) --Stijn -- Help Wanted: Telepath. You know where to apply. pgpjxs5Of2tKy.pgp Description: PGP signature
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, Dec 08, 2005 at 05:04:57PM -0700, secmgr wrote: Christian Brueffer wrote: On Thu, Dec 08, 2005 at 12:18:57AM +0100, Torfinn Ingolfsen wrote: In short, don't write gvinum off just yet. Documentation is around the corner (as a result of a SoC project). Actually gvinum(8) has been committed to CURRENT and RELENG_6 a couple of days ago. - Christian Whatever you do, don't complain about it on this list, or you'll just be told that if you really wanted raid, you should be running SCSI disks and a raid adapter. They may allow that 3ware does ok, but no ATA drive should ever be relied on and even s/w raid on scsi is only for ignorant lusers who are too cheap to do the right thing. Those who think I run to hyperbole need only visit the archives. One can only hope that gvinum actually works in 6 vs the buggy and incomplete alpha code that shipped in 5.x. Having a man page is nice, but I'd rather have a raid 5 set that didn't panic the system and corrupt the set when it lost a drive (and this with modern scsi drives and adapter). I'd strongly suggest anyone using GEOM raid to do some fault insertion testing of their setup prior to actually relying on it. Hmm, wasn't that a bug in the 5.3-RELEASE version that was fixed shortly after the release? - Christian -- Christian Brueffer [EMAIL PROTECTED] [EMAIL PROTECTED] GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D pgp8ZAncFjEvQ.pgp Description: PGP signature
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, Dec 08, 2005 at 05:04:57PM -0700, secmgr wrote: Whatever you do, don't complain about it on this list, or you'll just be told that if you really wanted raid, you should be running SCSI disks and a raid adapter. They may allow that 3ware does ok, but no ATA drive should ever be relied on and even s/w raid on scsi is only for ignorant lusers who are too cheap to do the right thing. Those who think I run to hyperbole need only visit the archives. I know what you mean but I think this write-up is a bit too harsh. As long as the goal is to get _as reliable as possible_ without spending too much money, I think software RAID has its niche. Besides, I've seen a few hardware RAID controllers having issues themselves (and they weren't the cheapest ones available either). One can only hope that gvinum actually works in 6 vs the buggy and incomplete alpha code that shipped in 5.x. Having a man page is nice, but I'd rather have a raid 5 set that didn't panic the system and corrupt the set when it lost a drive (and this with modern scsi drives and adapter). I haven't seen this (luckily!). I do know that sometime in 5.x there was a RAID-5 bug in gvinum but then again the whole vinum/gvinum transition was pretty dodgy back then. This is why I waited until 6.x to migrate. I'd strongly suggest anyone using GEOM raid to do some fault insertion testing of their setup prior to actually relying on it. I did. It worked. --Stijn -- There are of course many problems connected with life, of which some of the most popular are 'Why are people born?', 'Why do they die?', and `Why do they spend so much of the intervening time wearing digital watches?' -- Douglas Adams, The Hitchhikers Guide To The Galaxy pgpjc3SVsnCEy.pgp Description: PGP signature
Re: FreeBSD 6.0 as storage server with raid5?
Stijn Hoop wrote this message on Fri, Dec 09, 2005 at 10:30 +0100: Besides, I've seen a few hardware RAID controllers having issues themselves (and they weren't the cheapest ones available either). Yep, and because of failure to get proper vender support, software raid is looking more attractive to the company I work at... Delaying upgrades of the OS due to hardware raid issues, and then finding out that the ICP aka Adaptec no longer supports iir beyond 5.2-R (no 5.4-R or any 6.x and beyond) makes it harder for us to justify spending money on hardware raid. (who knows that the next brand of hardware raid isn't going to have the another issue, and decide to not provide support)... At least they finally released a new patch against 5.4-R, but we can barely get them to support us on 4.7-R (which has the same driver as 4.11-R), let alone when we had issues on 5.4-R... So, even though software raid may not be as reliable among other things, at least you have the source and can fix it, and don't have to wait many months just to wait longer for the vendor to fix the issue... -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
Torfinn Ingolfsen wrote: I was thinking about gvinum for the storage server, but given the current documentation and the discussions about it now, I don't want to risk it. So, I'm looking at hardware raid 5 controllers. From this list, You could use graid3(8) - it has data+parity components like raid5. I've been using it for more than a year now and didn't have problems with it. Didn't have to try recovery from a dead disk also, but should work ok. It's like regular RAID3 but uses sector-sized data chunks. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, Dec 08, 2005 at 12:18:57AM +0100, Torfinn Ingolfsen wrote: I was thinking about gvinum for the storage server, but given the current documentation and the discussions about it now, I don't want to risk it. IMHO it's pretty stable in 6.0. I've been running gvinum RAID-5 for a while now; other than one strange panic (something to do with out of memory situations, see kern/89660) I haven't had a hitch yet. That said, I haven't needed to replace a disk yet either (I've demoed this but it was not yet needed in production use). In short, don't write gvinum off just yet. Documentation is around the corner (as a result of a SoC project). --Stijn -- Well, Brahma said, even after ten thousand explanations, a fool is no wiser, but an intelligent man requires only two thousand five hundred. -- The Mahabharata. pgpW4eaFpgDLj.pgp Description: PGP signature
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, 08 Dec 2005 14:15:48 +0100 Ivan Voras [EMAIL PROTECTED] wrote: You could use graid3(8) - it has data+parity components like raid5. What about write performance? Based on RAID3 documentation I have read (on the 'net, so I cannot vouch for the correctness of it), you will get worse write performance than with RAID5. Does anybody have some real numbers here? I've been using it for more than a year now and didn't have problems with it. Didn't have to try recovery from a dead disk also, but should work ok. I would prefer a firsthand crash report that tells what happened, and a step-by-step here's how I fixed it guide, but ok. :-) -- Regards, Torfinn Ingolfsen, Norway ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, Dec 08, 2005 at 02:31:00PM +0100, Stijn Hoop wrote: On Thu, Dec 08, 2005 at 12:18:57AM +0100, Torfinn Ingolfsen wrote: I was thinking about gvinum for the storage server, but given the current documentation and the discussions about it now, I don't want to risk it. IMHO it's pretty stable in 6.0. I've been running gvinum RAID-5 for a while now; other than one strange panic (something to do with out of memory situations, see kern/89660) I haven't had a hitch yet. That said, I haven't needed to replace a disk yet either (I've demoed this but it was not yet needed in production use). In short, don't write gvinum off just yet. Documentation is around the corner (as a result of a SoC project). Actually gvinum(8) has been committed to CURRENT and RELENG_6 a couple of days ago. - Christian -- Christian Brueffer [EMAIL PROTECTED] [EMAIL PROTECTED] GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D pgp4XVVYLawMp.pgp Description: PGP signature
Re: FreeBSD 6.0 as storage server with raid5?
On Thu, Dec 08, 2005 at 08:30:33PM +0100, Torfinn Ingolfsen wrote: On Thu, 08 Dec 2005 14:15:48 +0100 Ivan Voras [EMAIL PROTECTED] wrote: You could use graid3(8) - it has data+parity components like raid5. What about write performance? Based on RAID3 documentation I have read (on the 'net, so I cannot vouch for the correctness of it), you will get worse write performance than with RAID5. Does anybody have some real numbers here? I've been using it for more than a year now and didn't have problems with it. Didn't have to try recovery from a dead disk also, but should work ok. I would prefer a firsthand crash report that tells what happened, and a step-by-step here's how I fixed it guide, but ok. :-) It is REALLY FAST! Make sure you use 6.0-RELEASE or later so that you have the fix for lock-ups in certain usage patterns and also configure it for performance with graid3 label -r. -- Brian Fundakowski Feldman \'[ FreeBSD ]''\ [EMAIL PROTECTED] \ The Power to Serve! \ Opinions expressed are my own. \,,\ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
Christian Brueffer wrote: On Thu, Dec 08, 2005 at 12:18:57AM +0100, Torfinn Ingolfsen wrote: In short, don't write gvinum off just yet. Documentation is around the corner (as a result of a SoC project). Actually gvinum(8) has been committed to CURRENT and RELENG_6 a couple of days ago. - Christian Whatever you do, don't complain about it on this list, or you'll just be told that if you really wanted raid, you should be running SCSI disks and a raid adapter. They may allow that 3ware does ok, but no ATA drive should ever be relied on and even s/w raid on scsi is only for ignorant lusers who are too cheap to do the right thing. Those who think I run to hyperbole need only visit the archives. One can only hope that gvinum actually works in 6 vs the buggy and incomplete alpha code that shipped in 5.x. Having a man page is nice, but I'd rather have a raid 5 set that didn't panic the system and corrupt the set when it lost a drive (and this with modern scsi drives and adapter). I'd strongly suggest anyone using GEOM raid to do some fault insertion testing of their setup prior to actually relying on it. jim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 6.0 as storage server with raid5?
Whatever you do, don't complain about it on this list, or you'll just be told that if you really wanted raid, you should be running SCSI disks Ah, no please complain so that if s/w raid gives you trouble, there will be something to point to when and if people doubt there are still problems (if indeed there are) Though I think jim is being entirely too harsh: The scary, poorly tested part of software raid is recovery. Thousands might roll out a s/w raid but if the h/w raid wasn't cost justified its unlikely that the HDs in the raid are actually going to be pressed into failing in any reasonable period of time that would reveal trouble in the recovery/degraded operating modes. Second, if you use s/w raid, pay close attention to the way your partitions line up. Third, SATA drives are actually quite good. You're primarily looking at a degraded MTBF versus a server grade SCSI disk. This could well mean just about nothing if your transaction volume is actually pretty low. imo, it would be nice to see MTBF quoted in a few parts: MTBF while seeking regularly (i.e., at some duty cycle) and MTBF in the bearings and other rotational components alone (MTBF while the disk is spun-up), and number of spin-up/spin-down cycles. Fourth, the major limitations on SATA drives right now is that FreeBSD does not support NCQ and therefore has no access to reliable write-completion information wrt SATA drives. and adapter). I'd strongly suggest anyone using GEOM raid to do some fault insertion testing of their setup prior to actually relying on it. This is very good advice. -Jon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
