Re: Softraid 3TB Problems
Joel, Would the fact that my two 3TB drives (ST3000DM001) use 4k sectors be the reason I wasn't getting it working then? I will try to zero the drives and start from scratch again, and see if that works too. I had a feeling that deleting the disklabels with 'z' option wasn't actually getting things back to the way they were when I first started out. Thank you for your input! -Brandon On Sat, Mar 2, 2013 at 11:38 PM, Joel Sing j...@sing.id.au wrote: On Sun, 3 Mar 2013, Brandon Tanner wrote: By the way, does softraid on amd64 support 4096 bytes per sector? No. There is a large amount of work required to fix this since everything in softraid was originally designed around 512-byte blocks. It is somewhere on my TODO list, however I do not currently even have the hardware for development/testing. -- Reason is not automatic. Those who deny it cannot be conquered by it. Do not count on them. Leave them alone. -- Ayn Rand
Re: Softraid 3TB Problems
On Sun, Mar 03, 2013 at 11:15:54AM -0600, Brandon Tanner wrote: Joel, Would the fact that my two 3TB drives (ST3000DM001) use 4k sectors be the reason I wasn't getting it working then? Both of the drives in the dmesg you posted report 512-byte sectors. Ken I will try to zero the drives and start from scratch again, and see if that works too. I had a feeling that deleting the disklabels with 'z' option wasn't actually getting things back to the way they were when I first started out. Thank you for your input! -Brandon On Sat, Mar 2, 2013 at 11:38 PM, Joel Sing j...@sing.id.au wrote: On Sun, 3 Mar 2013, Brandon Tanner wrote: By the way, does softraid on amd64 support 4096 bytes per sector? No. There is a large amount of work required to fix this since everything in softraid was originally designed around 512-byte blocks. It is somewhere on my TODO list, however I do not currently even have the hardware for development/testing. -- Reason is not automatic. Those who deny it cannot be conquered by it. Do not count on them. Leave them alone. -- Ayn Rand
Re: Softraid 3TB Problems
On Sun, 3 Mar 2013 12:35:29 -0500 Kenneth R Westerback kwesterb...@rogers.com wrote: On Sun, Mar 03, 2013 at 11:15:54AM -0600, Brandon Tanner wrote: Joel, Would the fact that my two 3TB drives (ST3000DM001) use 4k sectors be the reason I wasn't getting it working then? Both of the drives in the dmesg you posted report 512-byte sectors. Ken The Seagate datasheet [1] claims that this drive (ST3000DM001) uses 4k sectors. Otherwise can I confirm that a Western Digital WD30EFRX 3TB drive with 512b sectors works fine with crypto softraid. kind regards, Robert [1] http://www.seagate.com/files/staticfiles/docs/pdf/datasheet/disc/barracuda-desktop-hdd-ds-1770-1-1212us.pdf
Re: Softraid 3TB Problems
On Sun, Mar 03, 2013 at 07:54:07PM +0100, Robert wrote: On Sun, 3 Mar 2013 12:35:29 -0500 Kenneth R Westerback kwesterb...@rogers.com wrote: On Sun, Mar 03, 2013 at 11:15:54AM -0600, Brandon Tanner wrote: Joel, Would the fact that my two 3TB drives (ST3000DM001) use 4k sectors be the reason I wasn't getting it working then? Both of the drives in the dmesg you posted report 512-byte sectors. Ken The Seagate datasheet [1] claims that this drive (ST3000DM001) uses 4k sectors. Seagate confusion in this http://www.seagate.com/staticfiles/support/docs/manual/desktop/Barracuda%207200.14/100686584d.pdf It says: Bytes per sector (4K physical emulated at 512-byte sectors) 4096 Other numbers are reported using 512 byte sectors. -Otto Otherwise can I confirm that a Western Digital WD30EFRX 3TB drive with 512b sectors works fine with crypto softraid. kind regards, Robert [1] http://www.seagate.com/files/staticfiles/docs/pdf/datasheet/disc/barracuda-desktop-hdd-ds-1770-1-1212us.pdf
Re: Softraid 3TB Problems
On 03/03/13 00:34, Joel Sing wrote: On Sun, 3 Mar 2013, Brandon Tanner wrote: ... OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused # bioctl -c 1 -l /dev/sd1a,/dev/sd2a softraid0 softraid0: SR RAID 1 volume attached as sd3 This will assemble the volume from existing metadata if it exists. Any chance you created a 2TB 'a' partition to start with and created a softraid volume with it, then resized/recreated the disklabels? I'd certainly suggest zeroing the drives (via dd or similar), or using -C force (dd is more certain). The size is read directly from the disklabel, but only when the metadata is first created (after the metadata exists, we read the size from the metadata). All of the variables involved appear to be 64-bit types so I do not think that 32-bit truncation is occurring, although there are some signed/unsigned issues that should be addressed at some point. If zeroing and recreating the metadata fails to solve the issue, I can provide a diff that adds some debug info. I just took my pair of test 3TB disks (thanks to the donor!) and verified that, if zeroed first, there is no issue setting up a (most of) 3TB RAID1 via softraid. Nick.
Re: Softraid 3TB Problems
On 03/03/13 20:17, Nick Holland wrote: On 03/03/13 00:34, Joel Sing wrote: On Sun, 3 Mar 2013, Brandon Tanner wrote: ... OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused # bioctl -c 1 -l /dev/sd1a,/dev/sd2a softraid0 softraid0: SR RAID 1 volume attached as sd3 This will assemble the volume from existing metadata if it exists. Any chance you created a 2TB 'a' partition to start with and created a softraid volume with it, then resized/recreated the disklabels? I'd certainly suggest zeroing the drives (via dd or similar), or using -C force (dd is more certain). The size is read directly from the disklabel, but only when the metadata is first created (after the metadata exists, we read the size from the metadata). All of the variables involved appear to be 64-bit types so I do not think that 32-bit truncation is occurring, although there are some signed/unsigned issues that should be addressed at some point. If zeroing and recreating the metadata fails to solve the issue, I can provide a diff that adds some debug info. I just took my pair of test 3TB disks (thanks to the donor!) and verified that, if zeroed first, there is no issue setting up a (most of) 3TB RAID1 via softraid. Nick. Nick. Sigh. You know very well that neither you nor miod are supposed to possess that kind of recent hardware. ;-)
Re: Softraid 3TB Problems
On Sun, Mar 03, 2013 at 07:54:07PM +0100, Robert wrote: On Sun, 3 Mar 2013 12:35:29 -0500 Kenneth R Westerback kwesterb...@rogers.com wrote: On Sun, Mar 03, 2013 at 11:15:54AM -0600, Brandon Tanner wrote: Joel, Would the fact that my two 3TB drives (ST3000DM001) use 4k sectors be the reason I wasn't getting it working then? Both of the drives in the dmesg you posted report 512-byte sectors. Ken The Seagate datasheet [1] claims that this drive (ST3000DM001) uses 4k sectors. Otherwise can I confirm that a Western Digital WD30EFRX 3TB drive with 512b sectors works fine with crypto softraid. kind regards, Robert [1] http://www.seagate.com/files/staticfiles/docs/pdf/datasheet/disc/barracuda-desktop-hdd-ds-1770-1-1212us.pdf Many drives use 4K internally but still report to the outside world that they use 512 byte sectors. Handling the translation/packing/unpacking themselves. OpenBSD just reports (and trusts) what the drive reports in response to READ_CAPACITY/READ_CAPACITY_16. Ken
Re: Softraid 3TB Problems
I got it working!!!
What Joel said was key. I had probably done it wrong the first time (didn't
use the b option in disklabel), and everytime I re-tried it after that, I
wasn't starting from scratch (zero'ing the two raid drives). So I booted
the SeaTools for DOS CD, and zero'ed the two 3TB drives this time, and then
re-installed with the March 1, 2013 snapshot of 5.3 (don't think this is
what fixed it though), and did the same procedure in my first post, and now
it works! When Joel said that, the light just went on in my head, cause I
had been wondering how to reverse the procedure and start over, but had
only been using the 'z' option in disklabel, and had only dd if=/dev/zero
the resulting softraid volume, not the 2 real volumes each time I started
over to try again.
A warm Thanks to everyone for their input.
Also, about the 4k sectors, I think Ken is right, I think Seagate's
smartalign technology takes care of it all behind the scenes so to speak,
and the OS still sees 512 byte sectors.
Might be worth a mention in the FAQ or something that if you don't setup
raid1 correctly the first time, to start over, you need to detach the raid
using bioctl, clear the disklabels, zero the raid volumes, and reinit them
in fdisk?
On Sun, Mar 3, 2013 at 3:53 PM, Kenneth R Westerback kwesterb...@rogers.com
wrote:
On Sun, Mar 03, 2013 at 07:54:07PM +0100, Robert wrote:
On Sun, 3 Mar 2013 12:35:29 -0500
Kenneth R Westerback kwesterb...@rogers.com wrote:
On Sun, Mar 03, 2013 at 11:15:54AM -0600, Brandon Tanner wrote:
Joel,
Would the fact that my two 3TB drives (ST3000DM001) use 4k sectors
be the
reason I wasn't getting it working then?
Both of the drives in the dmesg you posted report 512-byte sectors.
Ken
The Seagate datasheet [1] claims that this drive (ST3000DM001) uses 4k
sectors.
Otherwise can I confirm that a Western Digital WD30EFRX 3TB drive with
512b sectors works fine with crypto softraid.
kind regards,
Robert
[1]
http://www.seagate.com/files/staticfiles/docs/pdf/datasheet/disc/barracuda-desktop-hdd-ds-1770-1-1212us.pdf
Many drives use 4K internally but still report to the outside world that
they use 512 byte sectors. Handling the translation/packing/unpacking
themselves. OpenBSD just reports (and trusts) what the drive reports
in response to READ_CAPACITY/READ_CAPACITY_16.
Ken
Re: Softraid 3TB Problems
On Sat, Mar 02, 2013 at 01:18:09PM -0600, Brandon Tanner wrote: Anyone else having trouble getting bioctl to see more than 2TB when creating softraid0? I've got 2 x 3TB drives, BIOS sees them fine. dmesg on bootup: sd1 at scsibus0 targ 3 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e0bcda5 sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors sd2 at scsibus0 targ 4 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e15ec80 sd2: 2861588MB, 512 bytes/sector, 5860533168 sectors notice the correct # of sectors above. disklabel -E sd1 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused disklabel -E sd2 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused # bioctl -c 1 -l /dev/sd1a,/dev/sd2a softraid0 softraid0: SR RAID 1 volume attached as sd3 sd3 at scsibus2 targ 1 lun 0: OPENBSD, SR RAID 1, 005 SCSI2 0/direct fixed sd3: 2097148MB, 512 bytes/sector, 4294961093 sectors Yet, bioctl only sees it for 4294961093 sectors :( Softraid apparently truncates at 32 bits somewhere along the line. You might try instrumenting places where ssd_size is set, and where READ_CAPACITY/READ_CAPACITY_16 are processed. A quick scan did not cause a 32 bit truncation to leap out at me. Ken # bioctl -h sd3 Volume Status Size Device softraid0 0 Online 2.0T sd3 RAID1 0 Online 2.0T 0:0.0 noencl sd1a 1 Online 2.0T 0:1.0 noencl sd2a I have been troubleshooting this issue with the help of Scott McEachern extensively, and we are both stumped. He suggested I put this on misc to see if jsing might know anything. I have an Intel Desktop Motherboard model DP43TF. I can newfs the drives individually and mount them as 2.7TB, it's only when trying to make a softraid volume that it doesn't see more than 2TB. I am using the Feb 21, 2013 snapshot of 5.3, and my dmesg is at: http://pastebin.com/jqnpSsnC Anyone else experiencing this? http://search.gmane.org/?author=Scott+McEachernsort=date
Re: Softraid 3TB Problems
By the way, does softraid on amd64 support 4096 bytes per sector? On Sat, Mar 2, 2013 at 5:05 PM, Kenneth R Westerback kwesterb...@rogers.com wrote: On Sat, Mar 02, 2013 at 01:18:09PM -0600, Brandon Tanner wrote: Anyone else having trouble getting bioctl to see more than 2TB when creating softraid0? I've got 2 x 3TB drives, BIOS sees them fine. dmesg on bootup: sd1 at scsibus0 targ 3 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e0bcda5 sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors sd2 at scsibus0 targ 4 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e15ec80 sd2: 2861588MB, 512 bytes/sector, 5860533168 sectors notice the correct # of sectors above. disklabel -E sd1 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused disklabel -E sd2 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused # bioctl -c 1 -l /dev/sd1a,/dev/sd2a softraid0 softraid0: SR RAID 1 volume attached as sd3 sd3 at scsibus2 targ 1 lun 0: OPENBSD, SR RAID 1, 005 SCSI2 0/direct fixed sd3: 2097148MB, 512 bytes/sector, 4294961093 sectors Yet, bioctl only sees it for 4294961093 sectors :( Softraid apparently truncates at 32 bits somewhere along the line. You might try instrumenting places where ssd_size is set, and where READ_CAPACITY/READ_CAPACITY_16 are processed. A quick scan did not cause a 32 bit truncation to leap out at me. Ken # bioctl -h sd3 Volume Status Size Device softraid0 0 Online 2.0T sd3 RAID1 0 Online 2.0T 0:0.0 noencl sd1a 1 Online 2.0T 0:1.0 noencl sd2a I have been troubleshooting this issue with the help of Scott McEachern extensively, and we are both stumped. He suggested I put this on misc to see if jsing might know anything. I have an Intel Desktop Motherboard model DP43TF. I can newfs the drives individually and mount them as 2.7TB, it's only when trying to make a softraid volume that it doesn't see more than 2TB. I am using the Feb 21, 2013 snapshot of 5.3, and my dmesg is at: http://pastebin.com/jqnpSsnC Anyone else experiencing this? http://search.gmane.org/?author=Scott+McEachernsort=date
Re: Softraid 3TB Problems
On Sat, Mar 02, 2013 at 07:25:18PM -0600, Brandon Tanner wrote: By the way, does softraid on amd64 support 4096 bytes per sector? No matter what the architecture is softraid to date does not support devices with anything other than 512 bytes/sector. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: Softraid 3TB Problems
On Sat, Mar 02, 2013 at 07:25:18PM -0600, Brandon Tanner wrote: By the way, does softraid on amd64 support 4096 bytes per sector? I don't think so, but I haven't refreshed my memory of that recently. Ken On Sat, Mar 2, 2013 at 5:05 PM, Kenneth R Westerback kwesterb...@rogers.com wrote: On Sat, Mar 02, 2013 at 01:18:09PM -0600, Brandon Tanner wrote: Anyone else having trouble getting bioctl to see more than 2TB when creating softraid0? I've got 2 x 3TB drives, BIOS sees them fine. dmesg on bootup: sd1 at scsibus0 targ 3 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e0bcda5 sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors sd2 at scsibus0 targ 4 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e15ec80 sd2: 2861588MB, 512 bytes/sector, 5860533168 sectors notice the correct # of sectors above. disklabel -E sd1 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused disklabel -E sd2 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused # bioctl -c 1 -l /dev/sd1a,/dev/sd2a softraid0 softraid0: SR RAID 1 volume attached as sd3 sd3 at scsibus2 targ 1 lun 0: OPENBSD, SR RAID 1, 005 SCSI2 0/direct fixed sd3: 2097148MB, 512 bytes/sector, 4294961093 sectors Yet, bioctl only sees it for 4294961093 sectors :( Softraid apparently truncates at 32 bits somewhere along the line. You might try instrumenting places where ssd_size is set, and where READ_CAPACITY/READ_CAPACITY_16 are processed. A quick scan did not cause a 32 bit truncation to leap out at me. Ken # bioctl -h sd3 Volume Status Size Device softraid0 0 Online 2.0T sd3 RAID1 0 Online 2.0T 0:0.0 noencl sd1a 1 Online 2.0T 0:1.0 noencl sd2a I have been troubleshooting this issue with the help of Scott McEachern extensively, and we are both stumped. He suggested I put this on misc to see if jsing might know anything. I have an Intel Desktop Motherboard model DP43TF. I can newfs the drives individually and mount them as 2.7TB, it's only when trying to make a softraid volume that it doesn't see more than 2TB. I am using the Feb 21, 2013 snapshot of 5.3, and my dmesg is at: http://pastebin.com/jqnpSsnC Anyone else experiencing this? http://search.gmane.org/?author=Scott+McEachernsort=date
Re: Softraid 3TB Problems
On Sun, 3 Mar 2013, Brandon Tanner wrote: Anyone else having trouble getting bioctl to see more than 2TB when creating softraid0? I've got 2 x 3TB drives, BIOS sees them fine. dmesg on bootup: sd1 at scsibus0 targ 3 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e0bcda5 sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors sd2 at scsibus0 targ 4 lun 0: ATA, ST3000DM001-1CH1, CC24 SCSI3 0/direct fixed naa.5000c5005e15ec80 sd2: 2861588MB, 512 bytes/sector, 5860533168 sectors notice the correct # of sectors above. disklabel -E sd1 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused disklabel -E sd2 p T OpenBSD area: 64-5860533168; size: 2.7T; free: 0.0T #size offset fstype [fsize bsize cpg] a: 2.7T 64RAID c: 2.7T0 unused # bioctl -c 1 -l /dev/sd1a,/dev/sd2a softraid0 softraid0: SR RAID 1 volume attached as sd3 This will assemble the volume from existing metadata if it exists. Any chance you created a 2TB 'a' partition to start with and created a softraid volume with it, then resized/recreated the disklabels? I'd certainly suggest zeroing the drives (via dd or similar), or using -C force (dd is more certain). The size is read directly from the disklabel, but only when the metadata is first created (after the metadata exists, we read the size from the metadata). All of the variables involved appear to be 64-bit types so I do not think that 32-bit truncation is occurring, although there are some signed/unsigned issues that should be addressed at some point. If zeroing and recreating the metadata fails to solve the issue, I can provide a diff that adds some debug info. sd3 at scsibus2 targ 1 lun 0: OPENBSD, SR RAID 1, 005 SCSI2 0/direct fixed sd3: 2097148MB, 512 bytes/sector, 4294961093 sectors Yet, bioctl only sees it for 4294961093 sectors :( # bioctl -h sd3 Volume Status Size Device softraid0 0 Online 2.0T sd3 RAID1 0 Online 2.0T 0:0.0 noencl sd1a 1 Online 2.0T 0:1.0 noencl sd2a I have been troubleshooting this issue with the help of Scott McEachern extensively, and we are both stumped. He suggested I put this on misc to see if jsing might know anything. I have an Intel Desktop Motherboard model DP43TF. I can newfs the drives individually and mount them as 2.7TB, it's only when trying to make a softraid volume that it doesn't see more than 2TB. I am using the Feb 21, 2013 snapshot of 5.3, and my dmesg is at: http://pastebin.com/jqnpSsnC Anyone else experiencing this? http://search.gmane.org/?author=Scott+McEachernsort=date -- Reason is not automatic. Those who deny it cannot be conquered by it. Do not count on them. Leave them alone. -- Ayn Rand
Re: Softraid 3TB Problems
On Sun, 3 Mar 2013, Brandon Tanner wrote: By the way, does softraid on amd64 support 4096 bytes per sector? No. There is a large amount of work required to fix this since everything in softraid was originally designed around 512-byte blocks. It is somewhere on my TODO list, however I do not currently even have the hardware for development/testing. -- Reason is not automatic. Those who deny it cannot be conquered by it. Do not count on them. Leave them alone. -- Ayn Rand
