RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Sandy Rutherford Sent: Wednesday, June 22, 2005 3:09 AM To: Ted Mittelstaedt Cc: freebsd-questions@freebsd.org; Alex Zbyslaw Subject: RE: Yet another RAID Question (YARQ) On Wed, 22 Jun 2005 01:00:09 -0700, Ted Mittelstaedt [EMAIL PROTECTED] said: With a RAID-1 card, mirroring, there are 2 ways to setup reads. The first way makes the assumption that you are mirroring purely for fault tolerance. In that case you would NOT see a ANY read from the second disk. The reason is that every time you read you move the heads, and the more head movement the quicker the disk wears out. OK. I wasn't aware that some RAID cards allow you to tune reads in this way. Mine, which is a Mylex DAC1100, does not. I was speaking from more of a designers theoretical standpoint rather than a users. From a practical standpoint I would think that the marketing department of any RAID card manufacturer would throw up their hands in horror if a engineer suggested doing it this way - the marketing people would say that the buyers of the card would think it was broken if they didn't see blinky lights on all the disk drives all the time. :-) You see many otherwise good designs fucked up this way by marketing people. :-( Placing exactly the same amount of head movement on both disks means that if you setup a mirror with new disks of the same model, which is pretty much how most people do it, the MTBF on both disks is the same, and if you put equal activity on both disks your making a very good chance that they will fail at the same time, or very close to the same time. This assumes a small standard deviation --- much smaller than I would think is reasonable. I don't think that I have ever seen standard deviation data quoted by a manufacturer, which of course makes any MTBF data that they provide worthless. Ah, but you see your working with the definition of MTBF that I used, and that the general public uses, NOT the definition of MTBF that Seagate uses. (or the other disk manufacturers) Seagate wrote a paper on this titled: Seagate Technology Paper 338.1 Estimating Drive Reliability in Desktop Computers and Consumer Electronic Systems that explains how they define MTBF. Basically, they define MTBF as what percentage of disks will fail in the FIRST year. What they are saying is if you purchase 160 Cheetahs and run them at 100% duty cycle for 1 year then there is 100% chance that 1 out of the 160 will fail. Thus, if you only purchase 80 disks and run them at 100% duty cycle for 1 year, then you only have a 50% chance that 1 will fail. And so on. Ain't statistics grand? You can make them say anything! For an encore Seagate went on to prove that their CEO would live 3 centuries by statistical grouping. :-) So, in getting back to the gist of what I was saying, the issue is as you mentioned standard deviation. I think we all understand that in a disk drive assembly line that it's all robotic, and that there is an extremely high chance that disk drives that are within a few serial numbers of each other are going to have virtually identical characteristics. In fact I would say using the Seagate MTBF definition, that 1 in every 160 drives manufactured in a particular run is going to have a significant enough deviation to fail at a significantly different period of time, given identical workload. In short you have better than 99% chance that if you install 2 brand new Cheetahs that are from the same production run, they will have virtually identical characteristics. And, failure due to wear is going to be very similar - there's only so many times the disk head can seek before it's bearings are worn out - and your proposing to give them the exact same usage. The interesting thing about this is that as quoted MTBF goes up, the closer and closer to identical all your disk drives have to be. So the funny thing is that in a RAID-1 array, your better off with cheapo Barracutas which have much greater deviation between each drive, than the more expensive Cheetahs that have less deviation between each drive. I agree with all of this. However, I do indeed see alternate flickering and the RAID array is sitting right in front of me. I expect this has to do with how the intensity of the activity lights is tied to seek vs read. If it matters, the drives are Cheetahs and they are in a Sun Multipack hot swap box. I think the reason your seeing alternation is that the disks are so damn fast that they complete their reads well before their internal buffers have finished emptying themselves over the SCSI bus to the array card. In other words, you wasted your money on your fast disks, if you had used slower disks you would see identical read performance but you would see less alternative flickering and more simultaneous and continuous activity. If you got a faster array card you wouldn't see the alternative
RE: Yet another RAID Question (YARQ)
On Wed, 22 Jun 2005 23:37:20 -0700, Ted Mittelstaedt [EMAIL PROTECTED] said: Seagate wrote a paper on this titled: Seagate Technology Paper 338.1 Estimating Drive Reliability in Desktop Computers and Consumer Electronic Systems that explains how they define MTBF. Basically, they define MTBF as what percentage of disks will fail in the FIRST year. Is this in the public domain? I wouldn't mind having a look at it. What they are saying is if you purchase 160 Cheetahs and run them at 100% duty cycle for 1 year then there is 100% chance that 1 out of the 160 will fail. Thus, if you only purchase 80 disks and run them at 100% duty cycle for 1 year, then you only have a 50% chance that 1 will fail. And so on. Ain't statistics grand? You can make them say anything! For an encore Seagate went on to prove that their CEO would live 3 centuries by statistical grouping. :-) Now don't knock statistics. The problem does not lie with statistics, but with its misuse by people who do not understand what they are doing. No, I am not a statistician; however, I am a mathematician. So, in getting back to the gist of what I was saying, the issue is as you mentioned standard deviation. I think we all understand that in a disk drive assembly line that it's all robotic, and that there is an extremely high chance that disk drives that are within a few serial numbers of each other are going to have virtually identical characteristics. In fact I would say using the Seagate MTBF definition, that 1 in every 160 drives manufactured in a particular run is going to have a significant enough deviation to fail at a significantly different period of time, given identical workload. I am not so sure. If we were talking about can openers, I would agree. However, a disk drive is basically a mechanical object which performs huge numbers of mechanical actions over the course of a number of years. Even extremely minute variations in the physical characteristics of the materials could lead to substantive variations over time. However, the operative word here is could. Real data is required. I tried to google for a relevant study, but came up empty. This surprised me as it seems like the sort of thing that masses of data should have been collected for. In short you have better than 99% chance that if you install 2 brand new Cheetahs that are from the same production run, they will have virtually identical characteristics. And, failure due to wear is going to be very similar - there's only so many times the disk head can seek before it's bearings are worn out - and your proposing to give them the exact same usage. I think the reason your seeing alternation is that the disks are so damn fast that they complete their reads well before their internal buffers have finished emptying themselves over the SCSI bus to the array card. In other words, you wasted your money on your fast disks, Not much money. After having been burned by failures of lower end drives, I bought high-end stuff on EBay. Made me nervous at the beginning, because who knows how many flights of stairs the drive bounced down before it was popped into the mail, and for that matter, who knows how many flights of stairs it bounced down while it was in the mail. However, so far it has worked out quite well. if you had used slower disks you would see identical read performance but you would see less alternative flickering and more simultaneous and continuous activity. If you got a faster array card you wouldn't see the alternative flickering. Or, it could be the PCI bus not being fast enough for the array card. It's almost certainly the PCI bus. The DAC1100, although not state-of-the-art, is still reasonably fast. It has 3 U2W channels and it could certainly max out my PCI bus. Ah well, a computer just wouldn't be a computer without blinking lights on it!!! ;-) Gotta agree there;-) Once upon a time I had the dip switch settings required to boot a PDP-11 from the front panel memorized, because I had to do it so often. Our data runs extended far beyond the typical uptime, so we did checkpoints by dumping the relevant bits of core to a teletype and I used to have to re-type in the data from the teletype when we brought it back up after a crash. Even on an old PDP-11, this took a while. We needed 3 months+ of uptime and we did well if we could keep that thing up for longer than a week. I became well-acquainted with those dip switches. Sandy ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Sandy Rutherford Sent: Thursday, June 23, 2005 1:15 AM To: Ted Mittelstaedt Cc: freebsd-questions@freebsd.org Subject: RE: Yet another RAID Question (YARQ) On Wed, 22 Jun 2005 23:37:20 -0700, Ted Mittelstaedt [EMAIL PROTECTED] said: Seagate wrote a paper on this titled: Seagate Technology Paper 338.1 Estimating Drive Reliability in Desktop Computers and Consumer Electronic Systems that explains how they define MTBF. Basically, they define MTBF as what percentage of disks will fail in the FIRST year. Is this in the public domain? I wouldn't mind having a look at it. I don't think it is but you can find ANYTHING on the Internet no matter how embarassing or private: http://www.digit-life.com/articles/storagereliability/ Ain't statistics grand? You can make them say anything! For an encore Seagate went on to prove that their CEO would live 3 centuries by statistical grouping. :-) Now don't knock statistics. The problem does not lie with statistics, but with its misuse by people who do not understand what they are doing. No, I am not a statistician; however, I am a mathematician. Then I am expecting you to read Seagates paper and after laughing your ass off, post a review of it here. :-) So, in getting back to the gist of what I was saying, the issue is as you mentioned standard deviation. I think we all understand that in a disk drive assembly line that it's all robotic, and that there is an extremely high chance that disk drives that are within a few serial numbers of each other are going to have virtually identical characteristics. In fact I would say using the Seagate MTBF definition, that 1 in every 160 drives manufactured in a particular run is going to have a significant enough deviation to fail at a significantly different period of time, given identical workload. I am not so sure. If we were talking about can openers, I would agree. However, a disk drive is basically a mechanical object which performs huge numbers of mechanical actions over the course of a number of years. Even extremely minute variations in the physical characteristics of the materials could lead to substantive variations over time. However, the operative word here is could. Real data is required. I tried to google for a relevant study, but came up empty. This surprised me as it seems like the sort of thing that masses of data should have been collected for. I'm sure they are but it's all going to be useful to the competitors so I doubt the companies that collected the data will let it out. What your asking for are nothing less than the recipie for setting costs levels to make a disk drive assembly line profitable - and that is an assembly line that even at the best of it, operates with a razor thin margin. Getting back to the physical characteristics, yes I had thought of that too and it is a consideration on reliability. However, the speed and tolerances of these things is so tight that any significant manufacturing deviation from the design is going to have the effect of seriously shortening lifetime. Consider also the typical automobile engine - by comparison to drive manufacturing the allowable variations are huge - yet for most cars, the engines all fail around the 200,000 mile mark. I think manufacturing deviations effects are staggered - during the first year they matter the most, then in successive years they don't matter much. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Sandy Rutherford Sent: Tuesday, June 21, 2005 6:16 PM To: Alex Zbyslaw Cc: freebsd-questions@freebsd.org; Ted Mittelstaedt Subject: Re: Yet another RAID Question (YARQ) On Mon, 20 Jun 2005 11:36:32 +0100, Alex Zbyslaw [EMAIL PROTECTED] said: Ted Mittelstaedt wrote: Sandy Rutherford Sent: Sunday, June 19, 2005 10:52 PM In order to boost read performance, a RAID card should interleave reading from a RAID-1 volume by reading alternately from one drive and then the other. You can see this in alternate blinking of the activity lights of the drives. If you are not seeing this when copying a large file, then this would suggest that a RAID-1 volume is not working as it should. Incorrect. What you are describing is RAID-0. RAID-1 is mirroring. Here's I don't think you read the message correctly. It said that *reads* were interleaved not that the *data* was interleaved. That's exactly what I said. Thanks. Ted, I am aware that RAID 1 is mirroring. However, any proper implementation of RAID 1 should also boost read performance and if during a read you are not seeing activity on both drives in the RAID 1 volume, then I would say this is a good indication that something is wrong. OK, I didn't bother replying earlier but now your both chiming in so I'll kill 2 birds with one stone I guess. First of all you didn't say seeing activity on both drives in the RAID 1 volume It's a cute attempt at a save on your part, but it is not what I said was wrong with your statement. You actually said: a RAID card should interleave reading from a RAID-1 volume by reading alternately from one drive and then the other. You can see this in alternate blinking of the activity lights of the drives. I said that is incorrect with RAID-1 because it is. And I will explain why. With a RAID-1 card, mirroring, there are 2 ways to setup reads. The first way makes the assumption that you are mirroring purely for fault tolerance. In that case you would NOT see a ANY read from the second disk. The reason is that every time you read you move the heads, and the more head movement the quicker the disk wears out. Placing exactly the same amount of head movement on both disks means that if you setup a mirror with new disks of the same model, which is pretty much how most people do it, the MTBF on both disks is the same, and if you put equal activity on both disks your making a very good chance that they will fail at the same time, or very close to the same time. Thus, for an optimal fault tolerance you would favor the first disk. You cannot do that with writes into a mirror, of course, since both drives must be updated. But you can do it with reads - you just read only from the first disk. Thus the first disk will most likely fail first, and the second disk will most likely not fail very close to the time that the first one fails. Thus the admin has maximum time to get a replacement disk in there. So much for the first way. The second way on a mirror is to try to setup reads to enhance speed in addition to fault tolerance. With this setup you interleave reads. You read a few blocks from the first disk, then a few blocks from the second, then a few blocks from the first, etc. etc. However, the kicker is that you do this AT THE SAME TIME. The disk heads are both continuiously reading, because the read speed of the heads are so much slower than the time it takes to move the data out of the drive and into main memory, that each disk is 'running dry' so fast that by alternating the read, your giving the drive a chance to catch up. So there is never a time the head isn't either reading or seeking for the next read, thus the disk drive lights are going to be both on solid at the same time. They will not be alternate blinking Indeed, if they really are alternating back and forth, then your read throughput will be no higher than a continuious read from a single disk. The ONLY time your going to see alternate blinking on a read is in a stripe set, RAID-0. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of P.U.Kruppa Sent: Monday, June 20, 2005 6:28 AM To: Ted Mittelstaedt Cc: P.U.Kruppa; freebsd-questions@freebsd.org Subject: RE: Yet another RAID Question (YARQ) As for knowing if a disk has failed, I think the only way to know is to watch the little lights on the disk front. After reading Alex' story about running a RAID 1 with a defect disc for three years, I believe it will suffice, when I check things with every system upgrade. The little lights change color and blink an angry red when a disk dies. You don't want to run it 3 years. For one thing, the warranty might expire. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
On Wed, 22 Jun 2005 01:00:09 -0700, Ted Mittelstaedt [EMAIL PROTECTED] said: With a RAID-1 card, mirroring, there are 2 ways to setup reads. The first way makes the assumption that you are mirroring purely for fault tolerance. In that case you would NOT see a ANY read from the second disk. The reason is that every time you read you move the heads, and the more head movement the quicker the disk wears out. OK. I wasn't aware that some RAID cards allow you to tune reads in this way. Mine, which is a Mylex DAC1100, does not. Placing exactly the same amount of head movement on both disks means that if you setup a mirror with new disks of the same model, which is pretty much how most people do it, the MTBF on both disks is the same, and if you put equal activity on both disks your making a very good chance that they will fail at the same time, or very close to the same time. This assumes a small standard deviation --- much smaller than I would think is reasonable. I don't think that I have ever seen standard deviation data quoted by a manufacturer, which of course makes any MTBF data that they provide worthless. Seagate quotes a MTBF of 1.4 million hours for their 10K Cheetah. That's 160 years! Assuming you actually believe that, there is no way the std dev on that number is less than a month. I would imagine that ~10yrs would be more reasonable. Unless you have better numbers, I would say that setting up RAID 1 as you describe above is just plain silly. BTW, since Seagate offers a 5 year warranty, I don't think that even they believe their own MTBF numbers. Or perhaps they do know the std dev and it's 155 years? The second way on a mirror is to try to setup reads to enhance speed in addition to fault tolerance. With this setup you interleave reads. You read a few blocks from the first disk, then a few blocks from the second, then a few blocks from the first, etc. etc. However, the kicker is that you do this AT THE SAME TIME. The disk heads are both continuiously reading, because the read speed of the heads are so much slower than the time it takes to move the data out of the drive and into main memory, that each disk is 'running dry' so fast that by alternating the read, your giving the drive a chance to catch up. So there is never a time the head isn't either reading or seeking for the next read, thus the disk drive lights are going to be both on solid at the same time. They will not be alternate blinking Indeed, if they really are alternating back and forth, then your read throughput will be no higher than a continuious read from a single disk. I agree with all of this. However, I do indeed see alternate flickering and the RAID array is sitting right in front of me. I expect this has to do with how the intensity of the activity lights is tied to seek vs read. If it matters, the drives are Cheetahs and they are in a Sun Multipack hot swap box. Anyway, this is all minutia... I think that it is fair to say that the main point of this thread is that if the behaviour of the drives' activity lights is not consistent with your RAID setup, then you should investigate --- regardless of what your RAID admin tool is saying. Would you agree with this? Sandy ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of P.U.Kruppa Sent: Monday, June 20, 2005 9:28 AM To: Ted Mittelstaedt Cc: P.U.Kruppa; freebsd-questions@freebsd.org Subject: RE: Yet another RAID Question (YARQ) On Mon, 20 Jun 2005, Ted Mittelstaedt wrote: On Sun, 19 Jun 2005, Ted Mittelstaedt wrote: What model of Proliant? ML 350 G4 Oh good, we have a customer that has been looking at one of these for FreeBSD and I'm glad to hear that you didn't have problems with it. Absolutely smooth - and I am really no kind of computer expert. With these all you get is hot-swap support although you might have to do a camcontrol rescan after swapping the disk. Yes, I have read that in some recent thread. Actually, the Windows management tools for this raid controller on a server are observational as well. There is no rebuild tool or anything like that. When we set these systems up for customers (All the recent Proliants use the same RAID controller) we usually configure them RAID-5 with 4 physical disks, the setup will set 3 of the disks in the array, and one a hot-spare. And in the event of a disk failure, which you can tell by looking at the disk drive lights, or going into the management interface, you simply pull out the bad disk and put in the replacement and the RAID card takes care of the rest of it. The City of Wuppertal couldn't buy me a third disc, because that would have superceded the limit of 2.5 kEURO, which would have required some special administrative act ... :-) . As for knowing if a disk has failed, I think the only way to know is to watch the little lights on the disk front. After reading Alex' story about running a RAID 1 with a defect disc for three years, I believe it will suffice, when I check things with every system upgrade. I know this technique isn't feasable in all situations, but I try to have duplicate hardware. Especially with my IDE RAID1 servers, I'll from time to time during a maintenance window pop one of the RAID disks out, throw it in another box and ensure BOTH machines boot up with individual disks. This is a sure test to ensure RAID is working. Mind you, I also back up using rsync for critical stuff to another box, and to tape as well. Steve Uli. * * Peter Ulrich Kruppa - Wuppertal - Germany * * ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Yet another RAID Question (YARQ)
Steve Bertrand wrote: I know this technique isn't feasable in all situations, but I try to have duplicate hardware. Especially with my IDE RAID1 servers, I'll from time to time during a maintenance window pop one of the RAID disks out, throw it in another box and ensure BOTH machines boot up with individual disks. This is a sure test to ensure RAID is working. Mind you, I also back up using rsync for critical stuff to another box, and to tape as well. Luckily we did the rsync and tape stuff (though it hasn't been needed yet). I guess you need not just the spare hardware (which is possible if you have more than one server to start with and two can come out at the same time) but the maintenance window to a) pop the disk and then b) rebuild the RAID afterwards. At least, I'm assuming that the RAID-1 is just going to treat the disk you pop back in as of unknown status and re-mirror it. Good thoughts, thanks, --Alex ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Yet another RAID Question (YARQ)
On Mon, 20 Jun 2005 11:36:32 +0100, Alex Zbyslaw [EMAIL PROTECTED] said: Ted Mittelstaedt wrote: Sandy Rutherford Sent: Sunday, June 19, 2005 10:52 PM In order to boost read performance, a RAID card should interleave reading from a RAID-1 volume by reading alternately from one drive and then the other. You can see this in alternate blinking of the activity lights of the drives. If you are not seeing this when copying a large file, then this would suggest that a RAID-1 volume is not working as it should. Incorrect. What you are describing is RAID-0. RAID-1 is mirroring. Here's I don't think you read the message correctly. It said that *reads* were interleaved not that the *data* was interleaved. That's exactly what I said. Thanks. Ted, I am aware that RAID 1 is mirroring. However, any proper implementation of RAID 1 should also boost read performance and if during a read you are not seeing activity on both drives in the RAID 1 volume, then I would say this is a good indication that something is wrong. Sandy ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Sandy Rutherford Sent: Sunday, June 19, 2005 10:52 PM In order to boost read performance, a RAID card should interleave reading from a RAID-1 volume by reading alternately from one drive and then the other. You can see this in alternate blinking of the activity lights of the drives. If you are not seeing this when copying a large file, then this would suggest that a RAID-1 volume is not working as it should. Incorrect. What you are describing is RAID-0. RAID-1 is mirroring. Here's a link: http://www.pcguide.com/ref/hdd/perf/raid/index.htm Here's the different RAID levels: RAID-0 interleaving between 2 or more disks. Primary purpose is to combine multiple disks into a larger volume. Gives maximum amount of space with no fault tolerance. RAID-1 mirroring Requires pairs of disks. Primary purpose is to give fault tolerance. Most commonly used with cheaper IDE disks and IDE RAID cards. Uses fewest number of disk drives for fault tolerance. Very easy to design so that if 1 disk dies the array of disks continues without interruption RAID-2 Bit level striping. Not used in modern systems (the scheme was overengineered, basically) RAID-3 Byte level striping. Rarely seen in modern RAID controllers. RAID-4 Block level striping. Rarely seen in modern RAID controllers. RAID-5 Block level striping - with distributed parity. Requires a minimum of 3 disks. The primary purpose is to give the volume-combining features of RAID-0 with the redundancy of RAID-1. This is the most popular RAID. But it is more difficult to design for so the cheaper controllers sometimes will halt the system if a disk is lost. Also requires drivers in the OS to allow online rebuilding of a replaced disk drive. Requires significant CPU processing on the RAID card for parity calculations. RAID-6 Same as RAID-5 except parity is dual distributed, not single distributed. Not common althogh some manufacturers call their proprietary extensions to RAID-5, raid 6 RAID-7 Patented RAID solution of Storage Computer Corporation that first showed up in their OmniRAID stuff, now seen in their CyberBorgVSA. (influence of Star Trek in the product name, there) Ted Mittelstaedt Author, The FreeBSD Corporate Networker's Guide http://www.freebsd-corp-net-guide.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of P.U.Kruppa Sent: Sunday, June 19, 2005 6:37 PM To: Ted Mittelstaedt Cc: P.U.Kruppa; freebsd-questions@freebsd.org Subject: RE: Yet another RAID Question (YARQ) On Sun, 19 Jun 2005, Ted Mittelstaedt wrote: What model of Proliant? ML 350 G4 Oh good, we have a customer that has been looking at one of these for FreeBSD and I'm glad to hear that you didn't have problems with it. With these all you get is hot-swap support although you might have to do a camcontrol rescan after swapping the disk. However, the RAID card intelligence is supposed to operate independently of the disk driver to do the remirroring or parity rebuilding. In theory you should be able to simply yank out a failed disk and slap in a replacement and the operating system shouldn't even notice anything. No matter what the OS in use. Actually, the Windows management tools for this raid controller on a server are observational as well. There is no rebuild tool or anything like that. When we set these systems up for customers (All the recent Proliants use the same RAID controller) we usually configure them RAID-5 with 4 physical disks, the setup will set 3 of the disks in the array, and one a hot-spare. And in the event of a disk failure, which you can tell by looking at the disk drive lights, or going into the management interface, you simply pull out the bad disk and put in the replacement and the RAID card takes care of the rest of it. As for knowing if a disk has failed, I think the only way to know is to watch the little lights on the disk front. And that is true of the Windows tools also - unless you install a complete HP Systems Insight Manager console (generally on a separate machine) which talks to all your little HP servers that run the various HP-SIM agents that talk to the raid card, etc. If I were you I would test all this by pulling a disk and seeing what happens. HP just released a binary driver for this series of RAID cards for Linux in May 2005. It supports RedHat and Suse. I do not know if they ship software notification tools with this binary driver, or if it also talks to a HP-SIM console. Ted ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Yet another RAID Question (YARQ)
Ted Mittelstaedt wrote: Sandy Rutherford Sent: Sunday, June 19, 2005 10:52 PM In order to boost read performance, a RAID card should interleave reading from a RAID-1 volume by reading alternately from one drive and then the other. You can see this in alternate blinking of the activity lights of the drives. If you are not seeing this when copying a large file, then this would suggest that a RAID-1 volume is not working as it should. Incorrect. What you are describing is RAID-0. RAID-1 is mirroring. Here's I don't think you read the message correctly. It said that *reads* were interleaved not that the *data* was interleaved. --Alex ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
On Mon, 20 Jun 2005, Ted Mittelstaedt wrote: On Sun, 19 Jun 2005, Ted Mittelstaedt wrote: What model of Proliant? ML 350 G4 Oh good, we have a customer that has been looking at one of these for FreeBSD and I'm glad to hear that you didn't have problems with it. Absolutely smooth - and I am really no kind of computer expert. With these all you get is hot-swap support although you might have to do a camcontrol rescan after swapping the disk. Yes, I have read that in some recent thread. Actually, the Windows management tools for this raid controller on a server are observational as well. There is no rebuild tool or anything like that. When we set these systems up for customers (All the recent Proliants use the same RAID controller) we usually configure them RAID-5 with 4 physical disks, the setup will set 3 of the disks in the array, and one a hot-spare. And in the event of a disk failure, which you can tell by looking at the disk drive lights, or going into the management interface, you simply pull out the bad disk and put in the replacement and the RAID card takes care of the rest of it. The City of Wuppertal couldn't buy me a third disc, because that would have superceded the limit of 2.5 kEURO, which would have required some special administrative act ... :-) . As for knowing if a disk has failed, I think the only way to know is to watch the little lights on the disk front. After reading Alex' story about running a RAID 1 with a defect disc for three years, I believe it will suffice, when I check things with every system upgrade. Uli. * * Peter Ulrich Kruppa - Wuppertal - Germany * * ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Yet another RAID Question (YARQ)
P.U.Kruppa wrote: our school has just received a new HP ProLiant and I set up FreeBSD 5.4 -RELEASE with a RAID 1 system on it (using ciss driver). Is there any software tool which can show me the state of the two SCSI discs (if one is failing or if they are mirrored sorrectly) or is it sufficient to watch the little LEDs on the box? That's a good question, and I only have a partial answer for you. You can look at sysutils/smartmontools port which will show you the SMART status for the disks that it can see. If you can see both disks (which I *think* would have devices like /dev/sd0, /dev/sd1) then you ought to know if the disks are failing. SCSI disks, from my limited experience, don't show as much info as ATA disks, but so far both Quantum and Fujitsu do seem to have supported SMART at a basic level. You should be able to tell what FreeBSD can see in the way of disks by examining /var/run/dmesg.boot. A word of caution, though. A Linux system (which I help administer) had two SCSI disks mounted as RAID-1 through some kind of Adaptec controller. Recently the machine crashed and it transpires that one of the disks hadn't been written to since 2002! I am told that the bootup screen showed the RAID-1 as working, and Linux could *only* see one virtual disk -- the supposed RAID mirror. So, I think your question is a very good one! We had (apparently) no way of knowing what was going on. The machine crashed with no messages whatsoever, after losing all access to its disks, and there was no indication that RAID-1 was not functioning. I *think* that the RAID controller should spot when a disk is failing and notify you (through its driver) through console messages and /var/log/messages. I too would love an answer to this question for any decent SCSI controller under FreeBSD (e.g. Dell PowerEdge 2850 with PERC 4e/Di RAID controller). Can you, in general, see through the RAID controller to monitor individual disks? --Alex ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Yet another RAID Question (YARQ)
On Sun, 19 Jun 2005, Alex Zbyslaw wrote: P.U.Kruppa wrote: our school has just received a new HP ProLiant and I set up FreeBSD 5.4 -RELEASE with a RAID 1 system on it (using ciss driver). Is there any software tool which can show me the state of the two SCSI discs (if one is failing or if they are mirrored sorrectly) or is it sufficient to watch the little LEDs on the box? That's a good question, and I only have a partial answer for you. You can look at sysutils/smartmontools port which will show you the SMART status for the disks that it can see. If you can see both disks (which I *think* would have devices like /dev/sd0, /dev/sd1) then you ought to know if the disks are failing. SCSI disks, from my limited experience, don't show as much info as ATA disks, but so far both Quantum and Fujitsu do seem to have supported SMART at a basic level. You should be able to tell what FreeBSD can see in the way of disks by examining /var/run/dmesg.boot. A word of caution, though. A Linux system (which I help administer) had two SCSI disks mounted as RAID-1 through some kind of Adaptec controller. Recently the machine crashed and it transpires that one of the disks hadn't been written to since 2002! I am told that the bootup screen showed the RAID-1 as working, and Linux could *only* see one virtual disk -- the supposed RAID mirror. So, I think your question is a very good one! We had (apparently) no way of knowing what was going on. The machine crashed with no messages whatsoever, after losing all access to its disks, and there was no indication that RAID-1 was not functioning. I *think* that the RAID controller should spot when a disk is failing and notify you (through its driver) through console messages and /var/log/messages. I too would love an answer to this question for any decent SCSI controller under FreeBSD (e.g. Dell PowerEdge 2850 with PERC 4e/Di RAID controller). Can you, in general, see through the RAID controller to monitor individual disks? No, the HP manual says one can check the disks via some sort of LED blinking code. But I have no experience with that, since it is a new machine. Uli. --Alex * * Peter Ulrich Kruppa - Wuppertal - Germany * * ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
What model of Proliant? Ted -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of P.U.Kruppa Sent: Saturday, June 18, 2005 10:56 PM To: freebsd-questions@freebsd.org Subject: Yet another RAID Question (YARQ) Hi everbody, our school has just received a new HP ProLiant and I set up FreeBSD 5.4 -RELEASE with a RAID 1 system on it (using ciss driver). Is there any software tool which can show me the state of the two SCSI discs (if one is failing or if they are mirrored sorrectly) or is it sufficient to watch the little LEDs on the box? Regards, Uli. * * Peter Ulrich Kruppa - Wuppertal - Germany * * ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Yet another RAID Question (YARQ)
On Sun, 19 Jun 2005, Ted Mittelstaedt wrote: What model of Proliant? ML 350 G4 Uli. Ted -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of P.U.Kruppa Sent: Saturday, June 18, 2005 10:56 PM To: freebsd-questions@freebsd.org Subject: Yet another RAID Question (YARQ) Hi everbody, our school has just received a new HP ProLiant and I set up FreeBSD 5.4 -RELEASE with a RAID 1 system on it (using ciss driver). Is there any software tool which can show me the state of the two SCSI discs (if one is failing or if they are mirrored sorrectly) or is it sufficient to watch the little LEDs on the box? Regards, Uli. * * Peter Ulrich Kruppa - Wuppertal - Germany * * ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] * * Peter Ulrich Kruppa - Wuppertal - Germany * * ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Yet another RAID Question (YARQ)
On Sun, 19 Jun 2005 12:28:14 +0100, Alex Zbyslaw [EMAIL PROTECTED] said: A word of caution, though. A Linux system (which I help administer) had two SCSI disks mounted as RAID-1 through some kind of Adaptec controller. Recently the machine crashed and it transpires that one of the disks hadn't been written to since 2002! I am told that the bootup screen showed the RAID-1 as working, and Linux could *only* see one virtual disk -- the supposed RAID mirror. So, I think your question is a very good one! We had (apparently) no way of knowing what was going on. The machine crashed with no messages whatsoever, after losing all access to its disks, and there was no indication that RAID-1 was not functioning. In order to boost read performance, a RAID card should interleave reading from a RAID-1 volume by reading alternately from one drive and then the other. You can see this in alternate blinking of the activity lights of the drives. If you are not seeing this when copying a large file, then this would suggest that a RAID-1 volume is not working as it should. Sandy ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
