Dump the data to a backup; recreate softraid disk; restore data from
dump.
On Mon, Feb 09, 2009 at 11:38:23AM -0600, Aaron Poffenberger wrote:
> On Feb 9, 2009, at 10:38, Marco Peereboom wrote:
>
>> Unfortunately manual rebuild does not work yet on softraid. I'll add
>> that to the man page.
>
> Thanks.
>
> So is the current rebuild process for a failed drive in softraid to
> build a new array and copy the data from the degraded array to the new
> array?
>
> Cheers,
>
> Aaron
>
>
>>
>> On Mon, Feb 09, 2009 at 10:14:24AM -0600, Aaron Poffenberger wrote:
>>> I noticed that bioctl(8) now includes the -R flag to kick off a
>>> manual rebuild. I upgraded to the current snapshot and setup a test
>>> mirror to try out the new feature but I must not understand how the
>>> syntax works. Assuming bioctl -R works with softraid(4), can someone
>>> tell me how to use it correctly? Here's the initial setup of the
>>> array and the steps I took:
>>>
>>> # bioctl -i sd4
>>> Volume Status Size Device softraid0 0 Online
>>> 500099595776 sd4 RAID1
>>> 0 Online 500105176064 0:0.0 noencl <wd5a>
>>> 1 Online 500105176064 0:1.0 noencl <wd6a>
>>> 2 Online 500099595776 0:2.0 noencl <wd5d>
>>>
>>> I built the array following the instructions in softraid(4).
>>> Everything works as expected. I copied a few 100 MBs from /usr/src
>>> to the new array. I then stopped the array with:
>>>
>>> # bioctl -d sd4
>>>
>>> And then attempted to re-create the array without the last element:
>>>
>>> # bioctl -c 1 -l /dev/wd5a,/dev/wd6a softraid0
>>>
>>> which failed with message:
>>>
>>> bioctl: BIOCCREATERAID: Invalid argument
>>>
>>> so I forced it with:
>>>
>>> # bioctl -C force -c 1 -l /dev/wd5a,/dev/wd6a softraid0
>>>
>>> which resulted in this message:
>>>
>>> softraid0: not all chunks were provided
>>> softraid0: can't attach metadata type 0
>>> softraid0: not all chunks were provided
>>> softraid0: can't attach metadata type 0
>>> scsibus2 at softraid0: 1 targets, initiator 1
>>> sd4 at scsibus2 targ 0 lun 0: <OPENBSD, SR RAID 1, 003> SCSI2
>>> 0/direct fixed
>>> sd4: 476937MB, 512 bytes/sec, 976767923 sec total
>>>
>>> I then mounted the array and copied more data to it after which I
>>> again deleted it with:
>>>
>>> # bioctl -d sd4
>>>
>>> And then re-created the array with:
>>>
>>> # bioctl -c 1 -l /dev/wd5a,/dev/wd6a,/dev/wd5d softraid0
>>>
>>> which gave this message:
>>>
>>> scsibus2 at softraid0: 1 targets, initiator 1
>>> sd4 at scsibus2 targ 0 lun 0: <OPENBSD, SR RAID 1, 003> SCSI2
>>> 0/direct fixed
>>> sd4: 476937MB, 512 bytes/sec, 976767923 sec total
>>>
>>> bioctl -i sd4 showed:
>>>
>>> Volume Status Size Device
>>> softraid0 0 Online 500105176064 sd4 RAID1
>>> 0 Online 500105176064 0:0.0 noencl <wd5a>
>>> 1 Online 500105176064 0:1.0 noencl <wd6a>
>>>
>>> I had to delete and recreate with '-C force' to see all three drives.
>>>
>>> # bioctl -i sd4
>>> Volume Status Size Device softraid0 0 Online
>>> 500099595776 sd4 RAID1
>>> 0 Online 500105176064 0:0.0 noencl <wd5a>
>>> 1 Online 500105176064 0:1.0 noencl <wd6a>
>>> 2 Online 500099595776 0:2.0 noencl <wd5d>
>>>
>>> The array mounts but surely must be "dirty" or inconsistent since I
>>> copied data to the degraded array. So I tried the rebuild command in
>>> various ways and never could find parameters that would work:
>>>
>>> # bioctl -R /dev/wd5d sd4
>>> # bioctl -R 0:2.0 sd4
>>>
>>> both gave the error:
>>>
>>> bioctl: BIOCSETSTATE: Invalid argument
>>>
>>> How do I initiate a manual rebuild?
>>>
>>> A couple of other questions:
>>> 1) How do I manually fail one element in a softraid mirror? I deleted
>>> the array and forcibly recreated it minus one element.
>>> a) Is that correct? Do I need to run any consistency check after
>>> doing so?
>>> 2) Is there a way to add new drive to a running softraid so I can
>>> then rebuild using it?
>>> 3) In the event of an unclean shutdown, is there any process like
>>> `radictl -P all` that should be run?
>>>
>>> Cheers,
>>>
>>> --Aaron
>>>
>>> The kernel is custom in that RAIDFrame is enabled. I'm trying to move
>>> a mirrored RAIDFrame device to softraid. Other than that no changes
>>> were made.
>>>
>>> OpenBSD 4.5-beta (GENERIC) #1: Mon Feb 9 00:05:23 CST 2009
>>> r...@home.poffenberger.org:/usr/src/sys/arch/i386/compile/GENERIC
>>> cpu0: AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ ("AuthenticAMD"
>>> 686-class, 512KB L2 cache) 2.01 GHz
>>> cpu0:
>>> FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,CX16
>>> real mem = 1072193536 (1022MB)
>>> avail mem = 1028075520 (980MB)
>>> mainbus0 at root
>>> bios0 at mainbus0: AT/286+ BIOS, date 11/29/06, BIOS32 rev. 0 @
>>> 0xf1fc0, SMBIOS rev. 2.4 @ 0xf0000 (67 entries)
>>> bios0: vendor Phoenix Technologies, LTD version "ASUS M2NPV-VM ACPI
>>> BIOS Revision 0603" date 11/29/2006
>>> bios0: ASUSTek Computer INC. M2NPV-VM
>>> acpi0 at bios0: rev 2
>>> acpi0: tables DSDT FACP HPET MCFG APIC
>>> acpi0: wakeup devices HUB0(S5) XVRA(S5) XVRB(S5) XVRC(S5) USB0(S4)
>>> USB2(S4) AZAD(S5) MMAC(S5) MMCI(S5) UAR1(S5) UAR2(S5) PS2M(S4)
>>> PS2K(S4)
>>> acpitimer0 at acpi0: 3579545 Hz, 24 bits
>>> acpihpet0 at acpi0: 25000000 Hz
>>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
>>> cpu0 at mainbus0: apid 0 (boot processor)
>>> cpu0: apic clock running at 200MHz
>>> cpu at mainbus0: not configured
>>> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins
>>> ioapic0: misconfigured as apic 0, remapped to apid 2
>>> acpiprt0 at acpi0: bus 0 (PCI0)
>>> acpiprt1 at acpi0: bus 4 (HUB0)
>>> acpicpu0 at acpi0
>>> acpitz0 at acpi0: critical temperature 75 degC
>>> acpibtn0 at acpi0: PWRB
>>> bios0: ROM list: 0xc0000/0x10000 0xd0000/0x4000! 0xd4000/0x4800
>>> pci0 at mainbus0 bus 0: configuration mode 1 (bios)
>>> "NVIDIA C51 Host" rev 0xa2 at pci0 dev 0 function 0 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 1 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 2 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 3 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 4 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 5 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 6 not configured
>>> "NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 7 not configured
>>> ppb0 at pci0 dev 2 function 0 "NVIDIA C51 PCIE" rev 0xa1
>>> pci1 at ppb0 bus 1
>>> ppb1 at pci0 dev 3 function 0 "NVIDIA C51 PCIE" rev 0xa1
>>> pci2 at ppb1 bus 2
>>> ppb2 at pci0 dev 4 function 0 "NVIDIA C51 PCIE" rev 0xa1
>>> pci3 at ppb2 bus 3
>>> vga1 at pci3 dev 0 function 0 "ATI Radeon X300" rev 0x00
>>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
>>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
>>> radeondrm0 at vga1: apic 2 int 16 (irq 11)
>>> drm0 at radeondrm0
>>> "ATI Radeon X300 Sec" rev 0x00 at pci3 dev 0 function 1 not configured
>>> "NVIDIA MCP51 Host" rev 0xa2 at pci0 dev 9 function 0 not configured
>>> pcib0 at pci0 dev 10 function 0 "NVIDIA MCP51 ISA" rev 0xa3
>>> nviic0 at pci0 dev 10 function 1 "NVIDIA MCP51 SMBus" rev 0xa3
>>> iic0 at nviic0
>>> spdmem0 at iic0 addr 0x50: 512MB DDR2 SDRAM non-parity PC2-6400CL3
>>> spdmem1 at iic0 addr 0x51: 512MB DDR2 SDRAM non-parity PC2-6400CL3
>>> iic1 at nviic0
>>> "NVIDIA MCP51 Memory" rev 0xa3 at pci0 dev 10 function 2 not configured
>>> ohci0 at pci0 dev 11 function 0 "NVIDIA MCP51 USB" rev 0xa3: apic 2
>>> int 20 (irq 5), version 1.0, legacy support
>>> ehci0 at pci0 dev 11 function 1 "NVIDIA MCP51 USB" rev 0xa3: apic 2
>>> int 20 (irq 10)
>>> usb0 at ehci0: USB revision 2.0
>>> uhub0 at usb0 "NVIDIA EHCI root hub" rev 2.00/1.00 addr 1
>>> pciide0 at pci0 dev 13 function 0 "NVIDIA MCP51 IDE" rev 0xa1: DMA,
>>> channel 0 configured to compatibility, channel 1 configured to
>>> compatibility
>>> wd0 at pciide0 channel 0 drive 0: <Maxtor 6L250R0>
>>> wd0: 16-sector PIO, LBA48, 239372MB, 490234752 sectors
>>> atapiscsi0 at pciide0 channel 0 drive 1
>>> scsibus0 at atapiscsi0: 2 targets, initiator 7
>>> cd0 at scsibus0 targ 0 lun 0: <SONY, DVD RW DRU-510A, 1.0a> ATAPI
>>> 5/cdrom removable
>>> wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 6
>>> cd0(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 2
>>> pciide0: channel 1 disabled (no drives)
>>> pciide1 at pci0 dev 14 function 0 "NVIDIA MCP51 SATA" rev 0xa1: DMA
>>> pciide1: using apic 2 int 20 (irq 5) for native-PCI interrupt
>>> wd1 at pciide1 channel 0 drive 0: <ST3500641AS>
>>> wd1: 16-sector PIO, LBA48, 476938MB, 976771055 sectors
>>> wd1(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5
>>> wd2 at pciide1 channel 1 drive 0: <ST3500641AS>
>>> wd2: 16-sector PIO, LBA48, 476940MB, 976773168 sectors
>>> wd2(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5
>>> pciide2 at pci0 dev 15 function 0 "NVIDIA MCP51 SATA" rev 0xa1: DMA
>>> pciide2: using apic 2 int 20 (irq 5) for native-PCI interrupt
>>> wd3 at pciide2 channel 0 drive 0: <ST3500641AS>
>>> wd3: 16-sector PIO, LBA48, 476940MB, 976773168 sectors
>>> wd3(pciide2:0:0): using PIO mode 4, Ultra-DMA mode 5
>>> wd4 at pciide2 channel 1 drive 0: <ST3500641AS>
>>> wd4: 16-sector PIO, LBA48, 476940MB, 976773168 sectors
>>> wd4(pciide2:1:0): using PIO mode 4, Ultra-DMA mode 5
>>> ppb3 at pci0 dev 16 function 0 "NVIDIA MCP51 PCI-PCI" rev 0xa2
>>> pci4 at ppb3 bus 4
>>> skc0 at pci4 dev 8 function 0 "D-Link Systems DGE-530T B1" rev 0x11,
>>> Yukon Lite (0x9): apic 2 int 16 (irq 7)
>>> sk0 at skc0 port A: address 00:17:9a:84:62:9c
>>> eephy0 at sk0 phy 0: 88E1011 Gigabit PHY, rev. 5
>>> pciide3 at pci4 dev 9 function 0 "CMD Technology SiI3512 SATA" rev
>>> 0x01: DMA
>>> pciide3: using apic 2 int 17 (irq 5) for native-PCI interrupt
>>> pciide3: port 0: device present, speed: 1.5Gb/s
>>> wd5 at pciide3 channel 0 drive 0: <SAMSUNG HD103UJ>
>>> wd5: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
>>> wd5(pciide3:0:0): using BIOS timings, Ultra-DMA mode 7
>>> pciide3: port 1: device present, speed: 1.5Gb/s
>>> wd6 at pciide3 channel 1 drive 0: <WDC WD10EACS-00ZJB0>
>>> wd6: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
>>> wd6(pciide3:1:0): using BIOS timings, Ultra-DMA mode 6
>>> pchb0 at pci0 dev 24 function 0 "AMD AMD64 0Fh HyperTransport" rev 0x00
>>> pchb1 at pci0 dev 24 function 1 "AMD AMD64 0Fh Address Map" rev 0x00
>>> pchb2 at pci0 dev 24 function 2 "AMD AMD64 0Fh DRAM Cfg" rev 0x00
>>> kate0 at pci0 dev 24 function 3 "AMD AMD64 0Fh Misc Cfg" rev 0x00:
>>> core rev BH-F2
>>> isa0 at pcib0
>>> isadma0 at isa0
>>> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
>>> com0: console
>>> com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
>>> pckbc0 at isa0 port 0x60/5
>>> pckbd0 at pckbc0 (kbd slot)
>>> pckbc0: using irq 1 for kbd slot
>>> wskbd0 at pckbd0: console keyboard, using wsdisplay0
>>> pcppi0 at isa0 port 0x61
>>> midi0 at pcppi0: <PC speaker>
>>> spkr0 at pcppi0
>>> it0 at isa0 port 0x2e/2: IT8716F rev 0, EC port 0x290
>>> npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
>>> usb1 at ohci0: USB revision 1.0
>>> uhub1 at usb1 "NVIDIA OHCI root hub" rev 1.00/1.00 addr 1
>>> mtrr: Pentium Pro MTRR support
>>> Kernelized RAIDframe activated
>>> umass0 at uhub0 port 7 configuration 1 interface 0 "Atech Flash
>>> PRO-Gear XM-4U" rev 2.00/1.00 addr 2
>>> umass0: using SCSI over Bulk-Only
>>> scsibus1 at umass0: 2 targets, initiator 0
>>> sd0 at scsibus1 targ 1 lun 0: <USB2.0, CardReader CF, 0100> SCSI0
>>> 0/direct removable
>>> sd0: drive offline
>>> sd1 at scsibus1 targ 1 lun 1: <USB2.0, CardReader SM XD, 0100> SCSI0
>>> 0/direct removable
>>> sd1: drive offline
>>> sd2 at scsibus1 targ 1 lun 2: <USB2.0, CardReader MS, 0100> SCSI0
>>> 0/direct removable
>>> sd2: drive offline
>>> sd3 at scsibus1 targ 1 lun 3: <USB2.0, CardReader SD, 0100> SCSI0
>>> 0/direct removable
>>> sd3: drive offline
>>> softraid0 at root
>>> softraid0: wd1a can not read metadata version 8, expected 3
>>> softraid0: wd2a can not read metadata version 0, expected 3
>>> softraid0: wd3a can not read metadata version -1, expected 3
>>> softraid0: wd4a can not read metadata version 0, expected 3
>>> softraid0: wd6d can not read metadata version 1970570146, expected 3
>>> scsibus2 at softraid0: 1 targets, initiator 1
>>> sd4 at scsibus2 targ 0 lun 0: <OPENBSD, SR RAID 1, 003> SCSI2
>>> 0/direct fixed
>>> sd4: 476937MB, 512 bytes/sec, 976767923 sec total
>>> root on wd0a swap on wd0b dump on wd0b
