Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
Another feature to look for is spin down of the dedicated hot spare. Go Vikings :) Patrick > On Feb 21, 2016, at 7:23 AM, Marcus MERIGHI wrote: > > ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 21:05 (CET): >> So glad to understand better what's in the box. >> >> Also please note that I'm not trying to suggest to implement lots of >> crap, am perfectly clear that high security is correlated with low >> complexity. >> >> On 2016-02-21 00:29, Marcus MERIGHI wrote: >>> ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 16:43 (CET): >> .. >>> You appear to mean bioctl(8). Thats the only place I could find the word >>> 'patrol'. bioctl(8) can control more than softraid(4) devices. >>> >>> bio(4): >>>The following device drivers register with bio for volume >>> management: >>> >>> ami(4) American Megatrends Inc. MegaRAID >>> PATA/SATA/SCSI RAID controller >>> arc(4) Areca Technology Corporation SAS/SATA RAID >>> controller >>> cac(4) Compaq Smart Array 2/3/4 SCSI RAID controller >>> ciss(4)Compaq Smart Array SAS/SATA/SCSI RAID >>> controller >>> ips(4) IBM SATA/SCSI ServeRAID controller >>> mfi(4) LSI Logic & Dell MegaRAID SAS RAID controller >>> mpi(4) LSI Logic Fusion-MPT Message Passing Interface >>> mpii(4)LSI Logic Fusion-MPT Message Passing Interface >>> II >>> softraid(4)Software RAID >>> >>> It is talking about controlling a HW raid controller, in that 'patrol' >>> paragraph, isn't it? >> >> So by this you mean that patrolling is really implemented for >> softraid?? > > No, I said the opposite. > > I'm sure my english language capabilities are not perfect. But what you > make of it is really surprising! (And even funny in the cabaret way.) > > I'll keep trying. But sooner or later we'll have to take this off list. > Or to newbies. There you get help from the same people but without > having your misinterpretations in the 'official' archives for other poor > souls to find ;-) > > http://mailman.theapt.org/listinfo/openbsd-newbies > >> (Karel and Constantine don't agree??) >> >> So I just do.. "bioctl -t start sdX" wher sdX is the name of my softraid >> device, and it'll do the "scrub" as in reading through all underlying > > bioctl(8) is clear, I think: > -t patrol-function > Control the RAID card's patrol functionality, if > supported. patrol-function may be one of: > > Why do you think it will work for softraid(4) when it says it does for > hardware-RAID? > > I have a theory: you have some experience with other Operating Systems > and their built in help system that have led you to not fully read but > just search/skim for keywords. Do yourself (and me) a favour and read > them fully. Top to bottom. Take every word as put there thoughtfully, > not in a hurry. You can find manpage content discussions all over the > archives. manpages are taken seriously. > > Please repeat: bio(4)/bioctl(8) controls RAID devices. These can be in > hardware or software. Some functions (-a, -b, -H, -t, -u) are only > useable/usefull when controlling a hardware RAID. The manpage even gives > direct clues on whether hardware- or software RAID is the topic. First > synopsis, second synopsis. 'The options for RAID controllers are as > follows:' (=hardware) 'In addition to the relevant options listed above, > the options for softraid(4) devices are as follows:' (=software). > Did you note the 'relevant' part? That word is there on purpose, I > suppose. It is there to tell you that not all, but the relevant parts of > the hardware RAID parameters also apply to software RAID (that comes > below). I would consider '-v' relevant, '-a' ('Control the RAID card's > alarm functionality, if supported') not. > > (Example: what '-a' does for hardware RAID can be done with sensorsd(8) > for software RAID (=softraid(4)). Once a softraid volume is configured, > you get 'hw.sensors.softraid0.drive0=online (sd1), OK'. > Try 'sysctl hw.sensors.softraid0'.) > >> physical media to check its internal integrity so for RAID1C that will be >> data readability and that checksums are correct, and "doas bioctl > softraid0" >> will show me the % status, and if I don't get any errors before it goes > back >> to normal it means the patrol was successful right? > > No idea, never had a hardware RAID controller. > >> (And as usual patrol is implemented to have the lowest priority, so it >> should not interfere extreemely much with ordinary SSD softraid operation.) > > I think the patrolling is done by the hardware RAID controller. > bioctl(8) just commands it to do so. > * Rebuild - I think I saw some console dump of the status of a rebuild process on the net, so MAYBE or NO..? >>> >>> That's what it looks like: >>> >>> $ doas bioctl softraid0 >>
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 21:05 (CET): > So glad to understand better what's in the box. > > Also please note that I'm not trying to suggest to implement lots of > crap, am perfectly clear that high security is correlated with low > complexity. > > On 2016-02-21 00:29, Marcus MERIGHI wrote: > >ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 16:43 (CET): > .. > >You appear to mean bioctl(8). Thats the only place I could find the word > >'patrol'. bioctl(8) can control more than softraid(4) devices. > > > >bio(4): > > The following device drivers register with bio for volume > >management: > > > > ami(4) American Megatrends Inc. MegaRAID > > PATA/SATA/SCSI RAID controller > > arc(4) Areca Technology Corporation SAS/SATA RAID > > controller > > cac(4) Compaq Smart Array 2/3/4 SCSI RAID controller > > ciss(4)Compaq Smart Array SAS/SATA/SCSI RAID > >controller > > ips(4) IBM SATA/SCSI ServeRAID controller > > mfi(4) LSI Logic & Dell MegaRAID SAS RAID controller > > mpi(4) LSI Logic Fusion-MPT Message Passing Interface > > mpii(4)LSI Logic Fusion-MPT Message Passing Interface > >II > > softraid(4)Software RAID > > > >It is talking about controlling a HW raid controller, in that 'patrol' > >paragraph, isn't it? > > So by this you mean that patrolling is really implemented for > softraid?? No, I said the opposite. I'm sure my english language capabilities are not perfect. But what you make of it is really surprising! (And even funny in the cabaret way.) I'll keep trying. But sooner or later we'll have to take this off list. Or to newbies. There you get help from the same people but without having your misinterpretations in the 'official' archives for other poor souls to find ;-) http://mailman.theapt.org/listinfo/openbsd-newbies > (Karel and Constantine don't agree??) > > So I just do.. "bioctl -t start sdX" wher sdX is the name of my softraid > device, and it'll do the "scrub" as in reading through all underlying bioctl(8) is clear, I think: -t patrol-function Control the RAID card's patrol functionality, if supported. patrol-function may be one of: Why do you think it will work for softraid(4) when it says it does for hardware-RAID? I have a theory: you have some experience with other Operating Systems and their built in help system that have led you to not fully read but just search/skim for keywords. Do yourself (and me) a favour and read them fully. Top to bottom. Take every word as put there thoughtfully, not in a hurry. You can find manpage content discussions all over the archives. manpages are taken seriously. Please repeat: bio(4)/bioctl(8) controls RAID devices. These can be in hardware or software. Some functions (-a, -b, -H, -t, -u) are only useable/usefull when controlling a hardware RAID. The manpage even gives direct clues on whether hardware- or software RAID is the topic. First synopsis, second synopsis. 'The options for RAID controllers are as follows:' (=hardware) 'In addition to the relevant options listed above, the options for softraid(4) devices are as follows:' (=software). Did you note the 'relevant' part? That word is there on purpose, I suppose. It is there to tell you that not all, but the relevant parts of the hardware RAID parameters also apply to software RAID (that comes below). I would consider '-v' relevant, '-a' ('Control the RAID card's alarm functionality, if supported') not. (Example: what '-a' does for hardware RAID can be done with sensorsd(8) for software RAID (=softraid(4)). Once a softraid volume is configured, you get 'hw.sensors.softraid0.drive0=online (sd1), OK'. Try 'sysctl hw.sensors.softraid0'.) > physical media to check its internal integrity so for RAID1C that will be > data readability and that checksums are correct, and "doas bioctl softraid0" > will show me the % status, and if I don't get any errors before it goes back > to normal it means the patrol was successful right? No idea, never had a hardware RAID controller. > (And as usual patrol is implemented to have the lowest priority, so it > should not interfere extreemely much with ordinary SSD softraid operation.) I think the patrolling is done by the hardware RAID controller. bioctl(8) just commands it to do so. > >> * Rebuild - I think I saw some console dump of the status of a rebuild > >>process on the net, so MAYBE or NO..? > > > >That's what it looks like: > > > >$ doas bioctl softraid0 > >Volume Status Size Device > >softraid0 0 Rebuild12002360033280 sd6 RAID5 35% done > > 0 Rebuild 4000786726912 0:0.0 noencl > > 1 Online 4000786726912 0:1.0 noencl > > 2 Online 400078672
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On 20 February 2016 at 14:29, Tinker wrote: [..] > On 2016-02-21 04:39, Constantine A. Murenin wrote: [..] >> When you do http://mdoc.su/o/newfs.8, it does not write to every >> sector of the underlying partition; thus you cannot expect all sectors >> to be the same. > > > Ah right, so at least to prepare for a RAID1C rebuild to work, at raid setup > time before disklabel/newfs, one should initialize by doing "dd if=/dev/zero > of=thelogicalraiddevice". This would make the logical view from within softraid_raid1 appear the same (e.g., an `sd3` regardless of which chunk it is being read from), but the underlying `sd{0,1,2}a` chunks that would be backing it up would still be different, because metadata (http://bxr.su/o/sys/dev/softraidvar.h#sr_metadata). C.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On 2016-02-21 05:05, Karel Gardas wrote: The RAID 1 discipline does not initialize the mirror upon creation. This is by design because all sectors that are read are written first. There is no point in wasting a lot of time syncing random data. I'm afraid the claim "all sectors that are read are written first" is not generally right. E.g. disklabel On 2016-02-21 04:39, Constantine A. Murenin wrote: .. Wait, I don't see the philosophical problem that you seem to be highlighting here ("sectors are .. written first" and "on top of the filesystem" and "forcing reads to be done from a certain chunk only"), what am I missing/what is it that I don't understand? When you do http://mdoc.su/o/newfs.8, it does not write to every sector of the underlying partition; thus you cannot expect all sectors to be the same. Ah right, so at least to prepare for a RAID1C rebuild to work, at raid setup time before disklabel/newfs, one should initialize by doing "dd if=/dev/zero of=thelogicalraiddevice". Also, so you mean that "patrol" even if it's in the manual is not supported for RAID1 nor for RAID1C nor for RAID5 or any other raid discipline. Looking forward to see what Marcus says & test myself & read code. Heh, I didn't even know about this "patrol"; learn something new every day! A BXR.SU for "patrol" has a few MFI(4) hits across the BSDs, and ultimately reveals the `bioc_patrol` symbol (well, a struct, really, http://bxr.su/o/sys/dev/biovar.h#bioc_patrol), a search of which reveals that http://bxr.su/o/sys/dev/ic/mfi.c#mfi_ioctl_patrol is the only driver reference for this symbol. So, I don't think you'll be getting any patrol from softraid(4); it was added very recently by uebayasi@ on 2015-05-29, based on mfiutil from FreeBSD (http://mdoc.su/f/mfiutil.8), and only for mfi(4). Aha, so to sum up the findings: * Scrub aka patrol is not supported (on the level of softraid) - perfectly fine. * Hot spares are supported (plug in with "bioctl -H" and plug out with "bioctl -O") * Rebuild is supported, and I should just figure out how (what do the two "bioctl -R" variants do respectively, and can it be done live, and does that mean it works like a hot-plug-in). * Hotswap is supported if hot-plug-in is supported via "bioctl -R", so should clarify that. So just some minor piece of clarity left now. And also the separate SATA controller post. All in all I think it looks neat.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On Sat, Feb 20, 2016 at 8:44 PM, Constantine A. Murenin wrote: > > Scrub cannot possibly be supported due to the design of the softraid: > > http://mdoc.su/o/softraid.4 > > The RAID 1 discipline does not initialize the mirror upon creation. This > is by design because all sectors that are read are written first. There > is no point in wasting a lot of time syncing random data. I'm afraid the claim "all sectors that are read are written first" is not generally right. E.g. disklabel
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On 20 February 2016 at 12:23, Tinker wrote: > > On 2016-02-21 01:29, Karel Gardas wrote: >> >> scrub is IIRC not supported by any softraid yet. > > > But there's "patrol"! > > "bioctl -t start mysoftraid" [...] > On 2016-02-21 02:44, Constantine A. Murenin wrote: >> >> On 20 February 2016 at 10:29, Karel Gardas wrote: >>> >>> scrub is IIRC not supported by any softraid yet. Rebuild by all which >>> support redundancy. Marcus recommendation to read man pages can just >>> be highlighted here. Otherwise just read the code for ultimate >>> reference of what is or is not done. >> >> >> Scrub cannot possibly be supported due to the design of the softraid: >> >> http://mdoc.su/o/softraid.4 >> >> The RAID 1 discipline does not initialize the mirror upon creation. >> This is by design because all sectors that are read are written first. >> There >> is no point in wasting a lot of time syncing random data. >> >> >> IIRC, other raid disciplines are not that much different, either. >> >> E.g., a scrub implementation would have to be implemented on top of >> the filesystem, and would have to be able to temporarily force the >> reads to be done from a certain chunk only. >> >> Long-term, it'll probably be easier to re-do the logic to actually >> zero-out all the unused sectors, if scrub support is deemed important. >> Which is why things like ZFS are superior due to having the awareness >> of the underlying storage blocks. > > > Wait, I don't see the philosophical problem that you seem to be highlighting > here ("sectors are .. written first" and "on top of the filesystem" and > "forcing reads to be done from a certain chunk only"), what am I > missing/what is it that I don't understand? When you do http://mdoc.su/o/newfs.8, it does not write to every sector of the underlying partition; thus you cannot expect all sectors to be the same. > > > Also, so you mean that "patrol" even if it's in the manual is not supported > for RAID1 nor for RAID1C nor for RAID5 or any other raid discipline. Looking > forward to see what Marcus says & test myself & read code. Heh, I didn't even know about this "patrol"; learn something new every day! A BXR.SU for "patrol" has a few MFI(4) hits across the BSDs, and ultimately reveals the `bioc_patrol` symbol (well, a struct, really, http://bxr.su/o/sys/dev/biovar.h#bioc_patrol), a search of which reveals that http://bxr.su/o/sys/dev/ic/mfi.c#mfi_ioctl_patrol is the only driver reference for this symbol. So, I don't think you'll be getting any patrol from softraid(4); it was added very recently by uebayasi@ on 2015-05-29, based on mfiutil from FreeBSD (http://mdoc.su/f/mfiutil.8), and only for mfi(4). C.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On Sat, Feb 20, 2016 at 9:23 PM, Tinker wrote: > > On 2016-02-21 01:29, Karel Gardas wrote: >> >> scrub is IIRC not supported by any softraid yet. > > > But there's "patrol"! > > "bioctl -t start mysoftraid" bioctl also supports hardware raid cards besides softraid, so that's what you are looking on now IMHO.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On 2016-02-21 01:29, Karel Gardas wrote: scrub is IIRC not supported by any softraid yet. But there's "patrol"! "bioctl -t start mysoftraid" Rebuild by all which support redundancy. Yey! Clarified by Marcus & looking forward to his clarification Marcus recommendation to read man pages can just be highlighted here. Otherwise just read the code for ultimate reference of what is or is not done. Yey! On 2016-02-21 02:44, Constantine A. Murenin wrote: On 20 February 2016 at 10:29, Karel Gardas wrote: scrub is IIRC not supported by any softraid yet. Rebuild by all which support redundancy. Marcus recommendation to read man pages can just be highlighted here. Otherwise just read the code for ultimate reference of what is or is not done. Scrub cannot possibly be supported due to the design of the softraid: http://mdoc.su/o/softraid.4 The RAID 1 discipline does not initialize the mirror upon creation. This is by design because all sectors that are read are written first. There is no point in wasting a lot of time syncing random data. IIRC, other raid disciplines are not that much different, either. E.g., a scrub implementation would have to be implemented on top of the filesystem, and would have to be able to temporarily force the reads to be done from a certain chunk only. Long-term, it'll probably be easier to re-do the logic to actually zero-out all the unused sectors, if scrub support is deemed important. Which is why things like ZFS are superior due to having the awareness of the underlying storage blocks. Wait, I don't see the philosophical problem that you seem to be highlighting here ("sectors are .. written first" and "on top of the filesystem" and "forcing reads to be done from a certain chunk only"), what am I missing/what is it that I don't understand? Also, so you mean that "patrol" even if it's in the manual is not supported for RAID1 nor for RAID1C nor for RAID5 or any other raid discipline. Looking forward to see what Marcus says & test myself & read code.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
Marcus, Holy moly, that is beautiful. So glad to understand better what's in the box. Also please note that I'm not trying to suggest to implement lots of crap, am perfectly clear that high security is correlated with low complexity. On 2016-02-21 00:29, Marcus MERIGHI wrote: ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 16:43 (CET): .. You appear to mean bioctl(8). Thats the only place I could find the word 'patrol'. bioctl(8) can control more than softraid(4) devices. bio(4): The following device drivers register with bio for volume management: ami(4) American Megatrends Inc. MegaRAID PATA/SATA/SCSI RAID controller arc(4) Areca Technology Corporation SAS/SATA RAID controller cac(4) Compaq Smart Array 2/3/4 SCSI RAID controller ciss(4)Compaq Smart Array SAS/SATA/SCSI RAID controller ips(4) IBM SATA/SCSI ServeRAID controller mfi(4) LSI Logic & Dell MegaRAID SAS RAID controller mpi(4) LSI Logic Fusion-MPT Message Passing Interface mpii(4)LSI Logic Fusion-MPT Message Passing Interface II softraid(4)Software RAID It is talking about controlling a HW raid controller, in that 'patrol' paragraph, isn't it? So by this you mean that patrolling is really implemented for softraid?? (Karel and Constantine don't agree??) So I just do.. "bioctl -t start sdX" wher sdX is the name of my softraid device, and it'll do the "scrub" as in reading through all underlying physical media to check its internal integrity so for RAID1C that will be data readability and that checksums are correct, and "doas bioctl softraid0" will show me the % status, and if I don't get any errors before it goes back to normal it means the patrol was successful right? (And as usual patrol is implemented to have the lowest priority, so it should not interfere extreemely much with ordinary SSD softraid operation.) * Rebuild - I think I saw some console dump of the status of a rebuild process on the net, so MAYBE or NO..? That's what it looks like: $ doas bioctl softraid0 Volume Status Size Device softraid0 0 Rebuild12002360033280 sd6 RAID5 35% done 0 Rebuild 4000786726912 0:0.0 noencl 1 Online 4000786726912 0:1.0 noencl 2 Online 4000786726912 0:2.0 noencl 3 Online 4000786726912 0:3.0 noencl Yey!! Wait, can you explain to me what I would write instead of "device" and "channel:target[.lun]" in "bioctl -R device" and "bioctl -R channel:target[.lun]", AND what effect those would have? Say that my sd0 and sd1 SSD:s run a RAID1C already, can I then make softraid extend my RAID1C with my sd2 SSD by "rebuilding" it, as a way to live-copy in all my data to sd2, so this would work as a kind of live attach even if expensive? Does it work for a softraid that's live already? * Hotspare - MAYBE, "man softraid" says "Currently there is no automated mechanism to recover from failed disks.", but that is not so specific wording, and I think I read a hint somewhere that there is hotspare functionality. bioctl(8) -H channel:target[.lun] If the device at channel:target[.lun] is currently marked ``Unused'', promote it to being a ``Hot Spare''. That's the only mention of 'hot spare'. And again talking about controlling a hardware RAID controller, isn't it? What is 'not so specific' about 'no' (as in "Currently there is *no* automated mechanism to recover from failed disks")? Awesome. I guess "bioctl softraid0" will list which hotspares there are currently, and that "-d" will drop a hotspare. The fact that there is hotspare functionality, means that there are cases when softraid will take a disk out of use. That will be when that disk reports itself as COMPLETELY out of use ALL BY ITSELF, such as self-detaching itself on the level of the SATA controller or reporting failure via some SMART command? A disk just half-breaking with broken sectors and 99% IO slowdown will not cause it to go offline though so I guess I should buy enterprise drives with IO access time guarantees then. * Hotswap - MAYBE, this would depend on if there's rebuild. Only disconnect ("bioctl -O" I think; "bioctl -d" is to.. unmount or self-destruct a softraid?) bioctl -O should fail the chunk specified, simulating hardware failure. After this command you have an 'Offline' chunk in the 'bioctl' output. bioctl -d 'detach', not 'destroy'; just as sdX appears when you assamble a softraid volume, this makes it go away. better unmount before... So "-d" is to take down a whole softraid. "-O" could work to take out a single physical disk but it's unclean. So then, there is a very unrefined hotswappin
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On 20 February 2016 at 10:29, Karel Gardas wrote: > scrub is IIRC not supported by any softraid yet. Rebuild by all which > support redundancy. Marcus recommendation to read man pages can just > be highlighted here. Otherwise just read the code for ultimate > reference of what is or is not done. Scrub cannot possibly be supported due to the design of the softraid: http://mdoc.su/o/softraid.4 The RAID 1 discipline does not initialize the mirror upon creation. This is by design because all sectors that are read are written first. There is no point in wasting a lot of time syncing random data. IIRC, other raid disciplines are not that much different, either. E.g., a scrub implementation would have to be implemented on top of the filesystem, and would have to be able to temporarily force the reads to be done from a certain chunk only. Long-term, it'll probably be easier to re-do the logic to actually zero-out all the unused sectors, if scrub support is deemed important. Which is why things like ZFS are superior due to having the awareness of the underlying storage blocks. C.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
scrub is IIRC not supported by any softraid yet. Rebuild by all which support redundancy. Marcus recommendation to read man pages can just be highlighted here. Otherwise just read the code for ultimate reference of what is or is not done.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 16:43 (CET): > On 2016-02-20 22:23, Marcus MERIGHI wrote: > >ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 15:29 (CET): > >>This email is an attempt to get some knowledge on how softraid works. > > > >So many of your questions are answered if you start with bioctl(8)[1], > >and continue with softraid(4)[2]. Maybe bio(4)[3] helps, too. > > > >What's there is usually documented. What's not documented is usually not > >there. Or was it the other way around? ;-) > > > >[1]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/bioctl.8 > >[2]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/softraid.4 > >[3]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/bio.4 > > > >Happy reading, Marcus > > Marcus, I read the docs carefully, and based on them, my best > understanding is still unclear on all 4 points - > > * Scrub - MAYBE??? There's some words about "patrol" in the man page that > could mean it's there. You appear to mean bioctl(8). Thats the only place I could find the word 'patrol'. bioctl(8) can control more than softraid(4) devices. bio(4): The following device drivers register with bio for volume management: ami(4) American Megatrends Inc. MegaRAID PATA/SATA/SCSI RAID controller arc(4) Areca Technology Corporation SAS/SATA RAID controller cac(4) Compaq Smart Array 2/3/4 SCSI RAID controller ciss(4)Compaq Smart Array SAS/SATA/SCSI RAID controller ips(4) IBM SATA/SCSI ServeRAID controller mfi(4) LSI Logic & Dell MegaRAID SAS RAID controller mpi(4) LSI Logic Fusion-MPT Message Passing Interface mpii(4)LSI Logic Fusion-MPT Message Passing Interface II softraid(4)Software RAID It is talking about controlling a HW raid controller, in that 'patrol' paragraph, isn't it? > * Rebuild - I think I saw some console dump of the status of a rebuild > process on the net, so MAYBE or NO..? That's what it looks like: $ doas bioctl softraid0 Volume Status Size Device softraid0 0 Rebuild12002360033280 sd6 RAID5 35% done 0 Rebuild 4000786726912 0:0.0 noencl 1 Online 4000786726912 0:1.0 noencl 2 Online 4000786726912 0:2.0 noencl 3 Online 4000786726912 0:3.0 noencl > * Hotspare - MAYBE, "man softraid" says "Currently there is no automated > mechanism to recover from failed disks.", but that is not so specific > wording, and I think I read a hint somewhere that there is hotspare > functionality. bioctl(8) -H channel:target[.lun] If the device at channel:target[.lun] is currently marked ``Unused'', promote it to being a ``Hot Spare''. That's the only mention of 'hot spare'. And again talking about controlling a hardware RAID controller, isn't it? What is 'not so specific' about 'no' (as in "Currently there is *no* automated mechanism to recover from failed disks")? > * Hotswap - MAYBE, this would depend on if there's rebuild. Only disconnect > ("bioctl -O" I think; "bioctl -d" is to.. unmount or self-destruct a > softraid?) bioctl -O should fail the chunk specified, simulating hardware failure. After this command you have an 'Offline' chunk in the 'bioctl' output. bioctl -d 'detach', not 'destroy'; just as sdX appears when you assamble a softraid volume, this makes it go away. better unmount before... > The man pages are sometimes over-minimalistic with respect to an individual > user who's trying to learn, this is why I'm asking for your clarification. I am quite sure the man pages are kept as condensed as they are on purpose. You can always read mplayer(1) if you want something lengthy ;-) > So your clarifications would still be much appreciated. Nothing authoritative from me! I am just trying to flatten your learning curve. Bye, Marcus
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
On 2016-02-20 22:23, Marcus MERIGHI wrote: ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 15:29 (CET): This email is an attempt to get some knowledge on how softraid works. So many of your questions are answered if you start with bioctl(8)[1], and continue with softraid(4)[2]. Maybe bio(4)[3] helps, too. What's there is usually documented. What's not documented is usually not there. Or was it the other way around? ;-) [1]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/bioctl.8 [2]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/softraid.4 [3]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/bio.4 Happy reading, Marcus Marcus, I read the docs carefully, and based on them, my best understanding is still unclear on all 4 points - * Scrub - MAYBE??? There's some words about "patrol" in the man page that could mean it's there. * Rebuild - I think I saw some console dump of the status of a rebuild process on the net, so MAYBE or NO..? * Hotspare - MAYBE, "man softraid" says "Currently there is no automated mechanism to recover from failed disks.", but that is not so specific wording, and I think I read a hint somewhere that there is hotspare functionality. * Hotswap - MAYBE, this would depend on if there's rebuild. Only disconnect ("bioctl -O" I think; "bioctl -d" is to.. unmount or self-destruct a softraid?) The man pages are sometimes over-minimalistic with respect to an individual user who's trying to learn, this is why I'm asking for your clarification. So your clarifications would still be much appreciated.
Re: OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
ti...@openmailbox.org (Tinker), 2016.02.20 (Sat) 15:29 (CET): > This email is an attempt to get some knowledge on how softraid works. So many of your questions are answered if you start with bioctl(8)[1], and continue with softraid(4)[2]. Maybe bio(4)[3] helps, too. What's there is usually documented. What's not documented is usually not there. Or was it the other way around? ;-) [1]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/bioctl.8 [2]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/softraid.4 [3]http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/bio.4 Happy reading, Marcus > There's basically zero docs on these topics out here (all docs are > about how to set it up first & subsequent times in non-failure cases). > > If you would be able to respond in "HOWTO form" would be awesome, then > at least this will be the Ultimate Softraid Reference :D > > Q1: > Can softraid do > > a) Scrub (as in read all the underlying media and check they're in check, > e.g. "raid fsck"), > > b) Hotspare (as in have one or more pre-designated physical disks that the > softraid would switch into use if one of the physical disks in use already > breaks), > > c) Hotswap (as in I could unplug one of the physical disks in a raid live, > and presuming my HBA allows it, the OpenBSD kernel + softraid will be happy > to plug in and plug out disks live - and I guess perhaps that needs to play > together with the rebuild then)? > > > Q2: > And, how do you use the rebuild + these three things? Specifically: > > a) How do I check the status of a rebuild from the console? (E.g. disk > blablaX rebuilding 12.3% done, get report if the rebuild failed, etc.) > b) How do I plug in a physical disk live into my RAID1 softraid? > > c) How do I plug out a physical disk live from my RAID1 softraid? > > d) How do I initiate a rebuild on a RAID1 softraid, such as plugging in the > brand new disk sdX into my RAID1 softraid or sdX was used previously or > suffered a power outage or whatever - how do I rebuild it? > > e) How do I get a listing of all my softraids and of all the physical disks > plugged into them, from the console? > > f) How do I tell softraid to "scrub" i.e. intregrity-check all the physical > disks running in the softraid, including every sector so actually all of > their surfaces are read (presuming that the RAID takes up all the disks)? > > g) How do I tell my RAID1 softraid that if sdX dies, then sdY should be > plugged in (i.e. hot spare)? > > Also, how do I get a printout of that configuration from the console, > and how do I remove a hotspare? > > h) After I plug out a physical disk from my softraid ( c) above), what > should I generally need to do before unplugging it also physically - just > wait 60 seconds or do some magic "SATA power off" command or the like? > (Given an ultra nice SATA or HW RAID controller such as LSI HBA or LSI > Megaraid.) > > i) When I plug in a physical disk on my SAS/SATA HBA (be it an LSI HBA or > LSI HW RAID), will OpenBSD autodetect it and assign a device name for it > just like it does for USB devices so it's complete hotswap/plug and play? > > j) When I plug in a softraid that worked before, what kind of trouble could > happen - if it won't get going at all, should I just reset from backup, or > is there some kind of "rescue rebuild" I could attempt? > > k) Do I need to know anything about DUID vs. device name use within the > softraid? > > > Looking forward to your explanation, thanks!! :DD > Tinker > > > !DSPAM:56c879bd171961725512869!
OpenBSD softraid can do scrub, hotspare, hotswap? How do rebuild + those 3 really done? (Absence of docs and howtos - ultimate Q!)
Hi, This email is an attempt to get some knowledge on how softraid works. There's basically zero docs on these topics out here (all docs are about how to set it up first & subsequent times in non-failure cases). If you would be able to respond in "HOWTO form" would be awesome, then at least this will be the Ultimate Softraid Reference :D Q1: Can softraid do a) Scrub (as in read all the underlying media and check they're in check, e.g. "raid fsck"), b) Hotspare (as in have one or more pre-designated physical disks that the softraid would switch into use if one of the physical disks in use already breaks), c) Hotswap (as in I could unplug one of the physical disks in a raid live, and presuming my HBA allows it, the OpenBSD kernel + softraid will be happy to plug in and plug out disks live - and I guess perhaps that needs to play together with the rebuild then)? Q2: And, how do you use the rebuild + these three things? Specifically: a) How do I check the status of a rebuild from the console? (E.g. disk blablaX rebuilding 12.3% done, get report if the rebuild failed, etc.) b) How do I plug in a physical disk live into my RAID1 softraid? c) How do I plug out a physical disk live from my RAID1 softraid? d) How do I initiate a rebuild on a RAID1 softraid, such as plugging in the brand new disk sdX into my RAID1 softraid or sdX was used previously or suffered a power outage or whatever - how do I rebuild it? e) How do I get a listing of all my softraids and of all the physical disks plugged into them, from the console? f) How do I tell softraid to "scrub" i.e. intregrity-check all the physical disks running in the softraid, including every sector so actually all of their surfaces are read (presuming that the RAID takes up all the disks)? g) How do I tell my RAID1 softraid that if sdX dies, then sdY should be plugged in (i.e. hot spare)? Also, how do I get a printout of that configuration from the console, and how do I remove a hotspare? h) After I plug out a physical disk from my softraid ( c) above), what should I generally need to do before unplugging it also physically - just wait 60 seconds or do some magic "SATA power off" command or the like? (Given an ultra nice SATA or HW RAID controller such as LSI HBA or LSI Megaraid.) i) When I plug in a physical disk on my SAS/SATA HBA (be it an LSI HBA or LSI HW RAID), will OpenBSD autodetect it and assign a device name for it just like it does for USB devices so it's complete hotswap/plug and play? j) When I plug in a softraid that worked before, what kind of trouble could happen - if it won't get going at all, should I just reset from backup, or is there some kind of "rescue rebuild" I could attempt? k) Do I need to know anything about DUID vs. device name use within the softraid? Looking forward to your explanation, thanks!! :DD Tinker