Re: Disk partition not recognized
On Mon, Dec 27, 2021 at 7:28 PM Rob Whitlock wrote: > Thanks for the work tracking down the problems. I reformatted the hard > drive to see if that would do anything and then I installed OpenBSD 7.0 > like you suggested and it started working. I used Disk Utility in MacOS > 10.15.7 Catalina, and when I reformatted it I got some errors from Disk > Utility. My guess is that Disk Utility is doing something incorrectly. > Correction: I reformatted it with diskutil, but I have since reformatted it with Disk Utility and it shows up in OpenBSD 7.0 as well.
Re: Disk partition not recognized
On Sat, Dec 25, 2021 at 8:46 AM Crystal Kolipe wrote: > OK, the issue lies with the four byte checksum at offset 0x58 in sector 1. > > Testing on OpenBSD 7.0 release and using your GPT: > > The kernel enters spoofgptlabel and reads sector 1. > > When we call gpt_chk_parts, the calculated checksum comes to 0x0BE89E52, > whereas the on-disk checksum is 0x3F7A886C, as you can see in the hexdumps. > > Note that the on-disk checksum is stored in little-endian format. > > As a result, gpt_chk_parts returns EINVAL. When control returns to > spoofgptlabel, it doesn't read the partitions contained within, and goes on > to try to read the second GPT at sector dsize-1, which in your case is > sector 9767541167. > > That's the reason why you don't see the non-OpenBSD partitions in your, > (spoofed), disklabel, the on-disk checksum of the partition entries does > not match the calculated checksum, so the kernel considers the GPT to be > invalid. > > If you want to test removing the call to gpt_chk_parts, thereby forcing > the kernel to parse whatever it finds and ignoring any checksum errors, the > attached diffs should allow you to do that. As you said that you were > still running OpenBSD 6.9, I've produced a diff against that too, including > the change in line 609 that I mentioned earlier, but it's untested. There > were other changes to this code between 6.9 and 7.0 that I have not really > looked at. > > On OpenBSD 7.0, with the diff applied, I am able to parse the GPT that you > supplied. > > I doubt that a kernel option to disable the checksum verification would be > appropriate or welcome, but I don't know how common the problem is. > Thanks for the work tracking down the problems. I reformatted the hard drive to see if that would do anything and then I installed OpenBSD 7.0 like you suggested and it started working. I used Disk Utility in MacOS 10.15.7 Catalina, and when I reformatted it I got some errors from Disk Utility. My guess is that Disk Utility is doing something incorrectly.
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 08:11:31PM -0300, Crystal Kolipe wrote: > On Thu, Dec 23, 2021 at 07:28:19PM -0300, Crystal Kolipe wrote: > > On Thu, Dec 23, 2021 at 04:15:50PM -0500, Rob Whitlock wrote: > > > On Thu, Dec 23, 2021 at 3:24 PM Crystal Kolipe > > > > > > wrote: > > > > > > > Again, there is nothing there that would stop it working. > > > > > > > > You have an MBR partition of type EE starting on sector 1, which is > > > > what is > > > > checked for in gpt_chk_mbr, so unless I'm overlooking something it's > > > > probably chocking in gpt_chk_hdr due to something unexpected in the GPT > > > > header, > > > > (LBA block 1). > > > > > > > > > > Here is LBA block 1: > > > > OK, I now know why it's not working :-). > > > > Either: > > > > * Upgrade to OpenBSD 7.0 > > > > or > > > > * Change line 610 of /usr/src/kern/subr_disk.c as it changed between > > version 1.241 and version 1.242: > > > > https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/kern/subr_disk.c.diff?r1=1.241=1.242=h > > > > ... and recompile the kernel. > > > > Either way, the spoofed disklabel will then include your non-OpenBSD > > partitions. > > > > And in future, please test the latest version of the code before reporting > > a bug ;-). > > Hang on, that's only part of the problem, there is something else wrong too... > > I was testing with a slightly different GPT header, (block 1), when I observed > the issue I described above. The fix I gave does indeed work for the GPT > header that I was using for testing. > > However, I just tested it with your exact GPT header block, and it still fails > to see the non-OpenBSD partitions. > > But looking at the two, the only fields that are different are the four bytes > at offset 0x10, which are a CRC, the 16 bytes at offset 0x38, which are the > disk GUID, and the four bytes at offset 0x58, which are another CRC. > > I won't have time to look in to this further for the next couple of days, > which is why I'm posting this in case somebody else wants to step up and > resolve it. > > --- works.hex Thu Dec 23 20:02:08 2021 > +++ doesnt.hexThu Dec 23 20:02:02 2021 > @@ -6,11 +6,11 @@ > * > 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa > |..U.| > 0200 45 46 49 20 50 41 52 54 00 00 01 00 5c 00 00 00 |EFI > PART\...| > -0210 c2 7d c2 16 00 00 00 00 01 00 00 00 00 00 00 00 > |.}..| > +0210 34 b3 c1 18 00 00 00 00 01 00 00 00 00 00 00 00 > |4...| > 0220 ae d9 30 46 02 00 00 00 22 00 00 00 00 00 00 00 > |..0F"...| > -0230 8d d9 30 46 02 00 00 00 07 8f ed 99 ec 89 df 45 > |..0F...E| > -0240 a5 38 96 c6 05 d7 c5 e9 02 00 00 00 00 00 00 00 > |.8..| > -0250 80 00 00 00 80 00 00 00 52 9e e8 0b 00 00 00 00 > |R...| > +0230 8d d9 30 46 02 00 00 00 69 b0 0a 57 69 18 ed 44 > |..0Fi..Wi..D| > +0240 91 1b a5 68 af 12 75 ff 02 00 00 00 00 00 00 00 > |...h..u.| > +0250 80 00 00 00 80 00 00 00 6c 88 7a 3f 00 00 00 00 > |l.z?| > 0260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > || > * > 0400 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b > |(s*..K...>.;| OK, the issue lies with the four byte checksum at offset 0x58 in sector 1. Testing on OpenBSD 7.0 release and using your GPT: The kernel enters spoofgptlabel and reads sector 1. When we call gpt_chk_parts, the calculated checksum comes to 0x0BE89E52, whereas the on-disk checksum is 0x3F7A886C, as you can see in the hexdumps. Note that the on-disk checksum is stored in little-endian format. As a result, gpt_chk_parts returns EINVAL. When control returns to spoofgptlabel, it doesn't read the partitions contained within, and goes on to try to read the second GPT at sector dsize-1, which in your case is sector 9767541167. That's the reason why you don't see the non-OpenBSD partitions in your, (spoofed), disklabel, the on-disk checksum of the partition entries does not match the calculated checksum, so the kernel considers the GPT to be invalid. If you want to test removing the call to gpt_chk_parts, thereby forcing the kernel to parse whatever it finds and ignoring any checksum errors, the attached diffs should allow you to do that. As you said that you were still running OpenBSD 6.9, I've produced a diff against that too, including the change in line 609 that I mentioned earlier, but it's untested. There were other changes to this code between 6.9 and 7.0 that I have not really looked at. On OpenBSD 7.0, with the diff applied, I am able to parse the GPT that you supplied. I doubt that a kernel option to disable the checksum verification would be appropriate or welcome, but I don't know how common the problem is. --- subr_disk.c.distTue Jan 19 16:36:48 2021 +++ subr_disk.c Sat Dec 25 10:30:01 2021 @@ -606,7 +606,7 @@ if (dp2->dp_typ != DOSPTYP_EFI)
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 07:28:19PM -0300, Crystal Kolipe wrote: > On Thu, Dec 23, 2021 at 04:15:50PM -0500, Rob Whitlock wrote: > > On Thu, Dec 23, 2021 at 3:24 PM Crystal Kolipe > > wrote: > > > > > Again, there is nothing there that would stop it working. > > > > > > You have an MBR partition of type EE starting on sector 1, which is what > > > is > > > checked for in gpt_chk_mbr, so unless I'm overlooking something it's > > > probably chocking in gpt_chk_hdr due to something unexpected in the GPT > > > header, > > > (LBA block 1). > > > > > > > Here is LBA block 1: > > OK, I now know why it's not working :-). > > Either: > > * Upgrade to OpenBSD 7.0 > > or > > * Change line 610 of /usr/src/kern/subr_disk.c as it changed between > version 1.241 and version 1.242: > > https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/kern/subr_disk.c.diff?r1=1.241=1.242=h > > ... and recompile the kernel. > > Either way, the spoofed disklabel will then include your non-OpenBSD > partitions. > > And in future, please test the latest version of the code before reporting a > bug ;-). Hang on, that's only part of the problem, there is something else wrong too... I was testing with a slightly different GPT header, (block 1), when I observed the issue I described above. The fix I gave does indeed work for the GPT header that I was using for testing. However, I just tested it with your exact GPT header block, and it still fails to see the non-OpenBSD partitions. But looking at the two, the only fields that are different are the four bytes at offset 0x10, which are a CRC, the 16 bytes at offset 0x38, which are the disk GUID, and the four bytes at offset 0x58, which are another CRC. I won't have time to look in to this further for the next couple of days, which is why I'm posting this in case somebody else wants to step up and resolve it. --- works.hex Thu Dec 23 20:02:08 2021 +++ doesnt.hex Thu Dec 23 20:02:02 2021 @@ -6,11 +6,11 @@ * 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..U.| 0200 45 46 49 20 50 41 52 54 00 00 01 00 5c 00 00 00 |EFI PART\...| -0210 c2 7d c2 16 00 00 00 00 01 00 00 00 00 00 00 00 |.}..| +0210 34 b3 c1 18 00 00 00 00 01 00 00 00 00 00 00 00 |4...| 0220 ae d9 30 46 02 00 00 00 22 00 00 00 00 00 00 00 |..0F"...| -0230 8d d9 30 46 02 00 00 00 07 8f ed 99 ec 89 df 45 |..0F...E| -0240 a5 38 96 c6 05 d7 c5 e9 02 00 00 00 00 00 00 00 |.8..| -0250 80 00 00 00 80 00 00 00 52 9e e8 0b 00 00 00 00 |R...| +0230 8d d9 30 46 02 00 00 00 69 b0 0a 57 69 18 ed 44 |..0Fi..Wi..D| +0240 91 1b a5 68 af 12 75 ff 02 00 00 00 00 00 00 00 |...h..u.| +0250 80 00 00 00 80 00 00 00 6c 88 7a 3f 00 00 00 00 |l.z?| 0260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 0400 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b |(s*..K...>.;|
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 04:15:50PM -0500, Rob Whitlock wrote: > On Thu, Dec 23, 2021 at 3:24 PM Crystal Kolipe > wrote: > > > Again, there is nothing there that would stop it working. > > > > You have an MBR partition of type EE starting on sector 1, which is what is > > checked for in gpt_chk_mbr, so unless I'm overlooking something it's > > probably chocking in gpt_chk_hdr due to something unexpected in the GPT > > header, > > (LBA block 1). > > > > Here is LBA block 1: OK, I now know why it's not working :-). Either: * Upgrade to OpenBSD 7.0 or * Change line 610 of /usr/src/kern/subr_disk.c as it changed between version 1.241 and version 1.242: https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/kern/subr_disk.c.diff?r1=1.241=1.242=h ... and recompile the kernel. Either way, the spoofed disklabel will then include your non-OpenBSD partitions. And in future, please test the latest version of the code before reporting a bug ;-).
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 3:24 PM Crystal Kolipe wrote: > Again, there is nothing there that would stop it working. > > You have an MBR partition of type EE starting on sector 1, which is what is > checked for in gpt_chk_mbr, so unless I'm overlooking something it's > probably chocking in gpt_chk_hdr due to something unexpected in the GPT > header, > (LBA block 1). > Here is LBA block 1: 0200: 4546 4920 5041 5254 0100 5c00 EFI PART\... 0210: 34b3 c118 0100 4... 0220: aed9 3046 0200 2200 ..0F"... 0230: 8dd9 3046 0200 69b0 0a57 6918 ed44 ..0Fi..Wi..D 0240: 911b a568 af12 75ff 0200 ...h..u. 0250: 8000 8000 6c88 7a3f l.z? 0260: 0270: 0280: 0290: 02a0: 02b0: 02c0: 02d0: 02e0: 02f0: 0300: 0310: 0320: 0330: 0340: 0350: 0360: 0370: 0380: 0390: 03a0: 03b0: 03c0: 03d0: 03e0: 03f0:
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 02:25:32PM -0500, Rob Whitlock wrote: > On Thu, Dec 23, 2021 at 2:14 PM Crystal Kolipe > wrote: > > > On Thu, Dec 23, 2021 at 01:15:52PM -0500, Rob Whitlock wrote: > > > On Thu, Dec 23, 2021 at 12:22 PM Crystal Kolipe < > > kolip...@exoticsilicon.com> > > > wrote: > > > > > > > If the spoofed label does not include your non-OpenBSD partitions, > > then for > > > > some reason the kernel is not parsing the data from the GPT, and we > > will > > > > presumably need a hexdump of the GPT to see why. > > > > > > > > > > Here is the GPT (the third sector on the disk): > > > > There is nothing unusual about these GPT entries. Every field apart from > > the > > partition serial numbers is identical to what would be written by creating > > the > > layout you described in your first email using OpenBSD fdisk. > > > > When I create this exact layout, the spoofed disklabel includes the > > non-OpenBSD > > partitions. > > > > I suspect that your MBR is trashed. Can you send a dump of the first > > sector, > > LBA 0? > > > > Sure, here it is. Again, there is nothing there that would stop it working. You have an MBR partition of type EE starting on sector 1, which is what is checked for in gpt_chk_mbr, so unless I'm overlooking something it's probably chocking in gpt_chk_hdr due to something unexpected in the GPT header, (LBA block 1).
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 2:14 PM Crystal Kolipe wrote: > On Thu, Dec 23, 2021 at 01:15:52PM -0500, Rob Whitlock wrote: > > On Thu, Dec 23, 2021 at 12:22 PM Crystal Kolipe < > kolip...@exoticsilicon.com> > > wrote: > > > > > If the spoofed label does not include your non-OpenBSD partitions, > then for > > > some reason the kernel is not parsing the data from the GPT, and we > will > > > presumably need a hexdump of the GPT to see why. > > > > > > > Here is the GPT (the third sector on the disk): > > There is nothing unusual about these GPT entries. Every field apart from > the > partition serial numbers is identical to what would be written by creating > the > layout you described in your first email using OpenBSD fdisk. > > When I create this exact layout, the spoofed disklabel includes the > non-OpenBSD > partitions. > > I suspect that your MBR is trashed. Can you send a dump of the first > sector, > LBA 0? > Sure, here it is. : 0010: 0020: 0030: 0040: 0050: 0060: 0070: 0080: 0090: 00a0: 00b0: 00c0: 00d0: 00e0: 00f0: 0100: 0110: 0120: 0130: 0140: 0150: 0160: 0170: 0180: 0190: 01a0: 01b0: 00fe 01c0: eefe 0100 feff 01d0: 01e0: 01f0: 55aa ..U.
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 01:15:52PM -0500, Rob Whitlock wrote: > On Thu, Dec 23, 2021 at 12:22 PM Crystal Kolipe > wrote: > > > If the spoofed label does not include your non-OpenBSD partitions, then for > > some reason the kernel is not parsing the data from the GPT, and we will > > presumably need a hexdump of the GPT to see why. > > > > Here is the GPT (the third sector on the disk): There is nothing unusual about these GPT entries. Every field apart from the partition serial numbers is identical to what would be written by creating the layout you described in your first email using OpenBSD fdisk. When I create this exact layout, the spoofed disklabel includes the non-OpenBSD partitions. I suspect that your MBR is trashed. Can you send a dump of the first sector, LBA 0?
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 12:22 PM Crystal Kolipe wrote: > If the spoofed label does not include your non-OpenBSD partitions, then for > some reason the kernel is not parsing the data from the GPT, and we will > presumably need a hexdump of the GPT to see why. > Here is the GPT (the third sector on the disk): 0400: 2873 2ac1 1ff8 d211 ba4b 00a0 c93e c93b (s*..K...>.; 0410: 864c bda4 7d17 024e a9f9 afc5 1ade 8d87 .L..}..N 0420: 2800 2740 0600 (...'@.. 0430: 4500 4600 4900 2000 E.F.I. . 0440: 5300 7900 7300 7400 6500 6d00 2000 5000 S.y.s.t.e.m. .P. 0450: 6100 7200 7400 6900 7400 6900 6f00 6e00 a.r.t.i.t.i.o.n. 0460: 0470: 0480: a2a0 d0eb e5b9 3344 87c0 68b6 b726 99c7 ..3D..h..&.. 0490: e5c7 5771 8f97 434a 96af fa66 4871 0488 ..Wq..CJ...fHq.. 04a0: 0048 0600 ffd7 3046 0200 .H0F 04b0: 04c0: 04d0: 04e0: 04f0: 0500: 0510: 0520: 0530: 0540: 0550: 0560: 0570: 0580: 0590: 05a0: 05b0: 05c0: 05d0: 05e0: 05f0:
Re: Disk partition not recognized
Rob Whitlock wrote: > On Thu, Dec 23, 2021 at 1:15 AM Theo de Raadt wrote: > > > > Crystal Kolipe wrote: > > > > > On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > > > > A problem seems to be that there is no disklabel entry for the ExFAT > > > > partition. > > > > > > You probably wrote a BSD disklabel to the disk before creating the > ExFAT partition. > > > > > > If there is no on-disk disklabel, the kernel will create one in memory > based on information from other partitioning schemes, (MBR, GPT). So in > this case, as you change those MBR or GPT partitions, those changes will be > reflected in the disklabel that the kernel sees. > > > > > > Once you actually write a disklabel to the disk, that on-disk disklabel > is then used in place of calculating one each time the disk is attached, > and the automatic parsing of MBR and GPT partition information stops. > > > > > > To solve your problem, you need to add the details of the ExFAT > partition to the BSD disklabel. You can either do that manually with the > disklabel command, or since you do not have any OpenBSD partitions on the > disk, you could overwrite the on-disk disklabel, allow the kernel to > generate one automatically with the correct information, then optionally > force it to be written to the disk by running disklabel and entering 'w' at > the interactive prompt. > > > > This can be investigated with > > > > disklabel -d > > > > (BTW, when the disklabel is constructed from other information on the > disk, > > we call it a "spoofed label") > > I would like to avoid modifying the data on the disk. Is there a way to use > disklabel to update the in-core copy of the disklabel with a spoofed label, > without also writing it to disk? I see in the disklabel(5) manual page that > the DIOCSDINFO ioctl updates the in-core copy, so it seems it should be > technically possible, but I don't see how to do it with the disklabel(8) > program. My understanding of disklabel -d is that it gives you a default > disklabel to start with, but does not affect how or where the disklabel is > written. disklabel -d is a read operation. You can look at what it reads, and seperately add partitions and write back the label.
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 12:08:49PM -0500, Rob Whitlock wrote: > On Thu, Dec 23, 2021 at 1:15 AM Theo de Raadt wrote: > > > > Crystal Kolipe wrote: > > > > > On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > > > > A problem seems to be that there is no disklabel entry for the ExFAT > > > > partition. > > > > > > You probably wrote a BSD disklabel to the disk before creating the > ExFAT partition. > > > > > > If there is no on-disk disklabel, the kernel will create one in memory > based on information from other partitioning schemes, (MBR, GPT). So in > this case, as you change those MBR or GPT partitions, those changes will be > reflected in the disklabel that the kernel sees. > > > > > > Once you actually write a disklabel to the disk, that on-disk disklabel > is then used in place of calculating one each time the disk is attached, > and the automatic parsing of MBR and GPT partition information stops. > > > > > > To solve your problem, you need to add the details of the ExFAT > partition to the BSD disklabel. You can either do that manually with the > disklabel command, or since you do not have any OpenBSD partitions on the > disk, you could overwrite the on-disk disklabel, allow the kernel to > generate one automatically with the correct information, then optionally > force it to be written to the disk by running disklabel and entering 'w' at > the interactive prompt. > > > > This can be investigated with > > > > disklabel -d > > > > (BTW, when the disklabel is constructed from other information on the > disk, > > we call it a "spoofed label") > > I would like to avoid modifying the data on the disk. Is there a way to use > disklabel to update the in-core copy of the disklabel with a spoofed label, > without also writing it to disk? I see in the disklabel(5) manual page that > the DIOCSDINFO ioctl updates the in-core copy, so it seems it should be > technically possible, but I don't see how to do it with the disklabel(8) > program. My understanding of disklabel -d is that it gives you a default > disklabel to start with, but does not affect how or where the disklabel is > written. The output from 'disklabel -d' will simply show us the spoofed label regardless of whether there is a real disklabel already written to the disk or not, (which I now suspect that there is not, having noticed that your duid is ). If the spoofed label includes your non-OpenBSD partitions, then presumably the disk already has a real label written to it which does not include them. If the spoofed label does not include your non-OpenBSD partitions, then for some reason the kernel is not parsing the data from the GPT, and we will presumably need a hexdump of the GPT to see why.
Re: Disk partition not recognized
On Thu, Dec 23, 2021 at 1:15 AM Theo de Raadt wrote: > > Crystal Kolipe wrote: > > > On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > > > A problem seems to be that there is no disklabel entry for the ExFAT > > > partition. > > > > You probably wrote a BSD disklabel to the disk before creating the ExFAT partition. > > > > If there is no on-disk disklabel, the kernel will create one in memory based on information from other partitioning schemes, (MBR, GPT). So in this case, as you change those MBR or GPT partitions, those changes will be reflected in the disklabel that the kernel sees. > > > > Once you actually write a disklabel to the disk, that on-disk disklabel is then used in place of calculating one each time the disk is attached, and the automatic parsing of MBR and GPT partition information stops. > > > > To solve your problem, you need to add the details of the ExFAT partition to the BSD disklabel. You can either do that manually with the disklabel command, or since you do not have any OpenBSD partitions on the disk, you could overwrite the on-disk disklabel, allow the kernel to generate one automatically with the correct information, then optionally force it to be written to the disk by running disklabel and entering 'w' at the interactive prompt. > > This can be investigated with > > disklabel -d > > (BTW, when the disklabel is constructed from other information on the disk, > we call it a "spoofed label") I would like to avoid modifying the data on the disk. Is there a way to use disklabel to update the in-core copy of the disklabel with a spoofed label, without also writing it to disk? I see in the disklabel(5) manual page that the DIOCSDINFO ioctl updates the in-core copy, so it seems it should be technically possible, but I don't see how to do it with the disklabel(8) program. My understanding of disklabel -d is that it gives you a default disklabel to start with, but does not affect how or where the disklabel is written.
Re: Disk partition not recognized
Crystal Kolipe wrote: > On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > > A problem seems to be that there is no disklabel entry for the ExFAT > > partition. > > You probably wrote a BSD disklabel to the disk before creating the ExFAT > partition. > > If there is no on-disk disklabel, the kernel will create one in memory based > on information from other partitioning schemes, (MBR, GPT). So in this case, > as you change those MBR or GPT partitions, those changes will be reflected in > the disklabel that the kernel sees. > > Once you actually write a disklabel to the disk, that on-disk disklabel is > then used in place of calculating one each time the disk is attached, and the > automatic parsing of MBR and GPT partition information stops. > > To solve your problem, you need to add the details of the ExFAT partition to > the BSD disklabel. You can either do that manually with the disklabel > command, or since you do not have any OpenBSD partitions on the disk, you > could overwrite the on-disk disklabel, allow the kernel to generate one > automatically with the correct information, then optionally force it to be > written to the disk by running disklabel and entering 'w' at the interactive > prompt. This can be investigated with disklabel -d (BTW, when the disklabel is constructed from other information on the disk, we call it a "spoofed label")
Re: Disk partition not recognized
I am not sure you can find someone who knows (almost) everything about disks and partitions even on OpenBSD area. There are so many "standards" and "methods" that the utilities got their names in time: fdisk - %#@& disk dd - destroy disk disklabel - diskbabel But maybe I am mistaken.
Re: Disk partition not recognized
On Wed, Dec 22, 2021 at 05:29:34PM +0100, Tilo Stritzky wrote:
> (With an MBR disk you could force feed a handcrafted disklabel but
> that won't work here because on a GPT disk without OpenBSD partition
> the disklabel and the primary GPT share a physical sector and that
> won't work.)
That is incorrect.
At one time, the disklabel program would try to write it to the second block,
I.E. the sector after the MBR. This would indeed fail if a GPT was already
there.
However, if you test this on OpenBSD 7.0-release, you will see that the
disklabel will happily be written elsewhere:
# dd if=/dev/zero of=/tmp/vd bs=1m count=512
# vnconfig vnd0 /tmp/vd
# fdisk -e vnd0 # Create a non-OpenBSD GPT partition
# disklabel -E vnd0 # Write the disklabel to the media
# hexdump -C /tmp/vd
ea 05 00 c0 07 8c c8 8e d0 bc fc ff 8e d8 b8 a0 ||
0010 07 8e c0 31 f6 31 ff b9 00 02 fc f3 a4 ea 22 00 |...1.1".|
0020 a0 07 1e 07 0e 1f b4 02 cd 16 a8 03 74 0d b0 07 |t...|
0030 e8 de 00 67 80 0d b4 01 00 00 01 f6 c2 80 75 08 |...g..u.|
0040 be 49 01 e8 bf 00 b2 80 be be 01 b9 04 00 8a 04 |.I..|
0050 3c 80 74 0f 83 c6 10 e2 f5 be 7d 01 e8 a6 00 fb |<.t...}.|
0060 f4 eb fc 88 d0 24 0f 04 30 a2 3a 01 b0 34 28 c8 |.$..0.:..4(.|
0070 a2 47 01 56 be 2d 01 67 f6 05 b4 01 00 00 01 75 |.G.V.-.g...u|
0080 01 46 e8 80 00 5e 26 67 c7 05 fe 01 00 00 00 00 |.F...^|
0090 67 f6 05 b4 01 00 00 01 75 34 88 14 bb aa 55 b4 |g...u4U.|
00a0 41 cd 13 8a 14 72 27 81 fb 55 aa 75 21 f6 c1 01 |Ar'..U.u!...|
00b0 74 1c b0 2e e8 5a 00 66 8b 4c 08 67 66 89 0d 25 |tZ.f.L.gf..%|
00c0 01 00 00 56 b4 42 be 1d 01 cd 13 5e 73 1a b0 3b |...V.B.^s..;|
00d0 e8 3e 00 8a 74 01 8b 4c 02 b8 01 02 31 db cd 13 |.>..t..L1...|
00e0 73 06 be 65 01 e9 74 ff be 90 01 e8 17 00 26 67 |s..e..t...|
00f0 81 3d fe 01 00 00 55 aa 75 05 ea 00 7c 00 00 be |.=U.u...|...|
0100 74 01 e9 57 ff 50 fc ac 84 c0 74 0f e8 02 00 eb |t..W.Pt.|
0110 f6 50 53 b4 0e bb 01 00 cd 10 5b 58 c3 10 00 01 |.PS...[X|
0120 00 00 00 c0 07 00 00 00 00 00 00 00 00 21 55 73 |.!Us|
0130 69 6e 67 20 64 72 69 76 65 20 58 2c 20 70 61 72 |ing drive X, par|
0140 74 69 74 69 6f 6e 20 59 00 4d 42 52 20 6f 6e 20 |tition Y.MBR on |
0150 66 6c 6f 70 70 79 20 6f 72 20 6f 6c 64 20 42 49 |floppy or old BI|
0160 4f 53 0d 0a 00 0d 0a 52 65 61 64 20 65 72 72 6f |OS.Read erro|
0170 72 0d 0a 00 4e 6f 20 4f 2f 53 0d 0a 00 4e 6f 20 |r...No O/S...No |
0180 61 63 74 69 76 65 20 70 61 72 74 69 74 69 6f 6e |active partition|
0190 0d 0a 00 90 00 00 00 00 00 00 00 00 00 00 00 00 ||
01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ||
01b0 00 00 00 00 00 00 4f 78 00 00 00 00 00 00 00 ff |..Ox|
01c0 ff ff ee ff ff ff 01 00 00 00 ff ff ff ff 00 00 ||
01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ||
*
01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..U.|
0200 45 46 49 20 50 41 52 54 00 00 01 00 5c 00 00 00 |EFI PART\...|
0210 65 43 91 cc 00 00 00 00 01 00 00 00 00 00 00 00 |eC..|
0220 ff ff 0f 00 00 00 00 00 22 00 00 00 00 00 00 00 |"...|
0230 de ff 0f 00 00 00 00 00 f2 63 79 02 b9 99 39 48 |.cy...9H|
0240 99 97 2e 77 79 98 35 f5 02 00 00 00 00 00 00 00 |...wy.5.|
0250 80 00 00 00 80 00 00 00 a7 be 55 7f 00 00 00 00 |..U.|
0260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ||
*
0400 a2 a0 d0 eb e5 b9 33 44 87 c0 68 b6 b7 26 99 c7 |..3D..h..&..|
0410 41 60 b8 b5 2b 54 10 41 ba 44 ee f8 3e 55 7b 50 |A`..+T.A.D..>U{P|
0420 22 00 00 00 00 00 00 00 de ff 0f 00 00 00 00 00 |"...|
0430 00 00 00 00 00 00 00 00 66 00 6f 00 6f 00 00 00 |f.o.o...|
0440 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ||
*
vvv BSD disklabel here vvv
4600 57 45 56 82 0c 00 00 00 76 6e 64 20 64 65 76 69 |WEV.vnd devi|
4610 63 65 00 00 00 00 00 00 66 69 63 74 69 74 69 6f |ce..fictitio|
4620 75 73 00 00 00 00 00 00 00 02 00 00 64 00 00 00 |us..d...|
4630 01 00 00 00 f5 28 00 00 64 00 00 00 00 00 10 00 |.(..d...|
4640 38 88 61 0e cf 69 90 5f 00 00 00 00 00 00 00 00 |8.a..i._|
4650 22 00 00 00 df ff 0f 00 00 00 00 00 00 00 00 00 |"...|
4660 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ||
4670 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 ||
4680 00 00 00 00 57 45 56 82 97 e8 10 00 00 20 00 00 |WEV.. ..|
4690 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
Re: Disk partition not recognized
On Wed, Dec 22, 2021 at 5:23 AM Crystal Kolipe wrote: > On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > > A problem seems to be that there is no disklabel entry for the ExFAT > > partition. > > You probably wrote a BSD disklabel to the disk before creating the ExFAT > partition. > I formatted the disk on a MacOS system, so I'm pretty sure there is no disklabel on the disk. > If there is no on-disk disklabel, the kernel will create one in memory > based on information from other partitioning schemes, (MBR, GPT). So in > this case, as you change those MBR or GPT partitions, those changes will be > reflected in the disklabel that the kernel sees. > > Once you actually write a disklabel to the disk, that on-disk disklabel is > then used in place of calculating one each time the disk is attached, and > the automatic parsing of MBR and GPT partition information stops. > > To solve your problem, you need to add the details of the ExFAT partition > to the BSD disklabel. You can either do that manually with the disklabel > command, or since you do not have any OpenBSD partitions on the disk, you > could overwrite the on-disk disklabel, allow the kernel to generate one > automatically with the correct information, then optionally force it to be > written to the disk by running disklabel and entering 'w' at the > interactive prompt. > I would like to not modify the on-disk contents. Is there a way to get OpenBSD to recognize the partition without writing things to the disk?
Re: Disk partition not recognized
On 22/12/21 13:35 Tilo Stritzky wrote: Um, well.. What I was going to say is, for some reason the default disklabel doesn't pick up your partitions (it should also show the EFI, but doesn't). As a quick-and-dirty fix you cold try and change the partition ID to something that's picked up by the default label. 3f929abc-650c-4237-af34-2027ddc585e5 should work. For a proper fix, find where the partition IDs are kept and add yours. (With an MBR disk you could force feed a handcrafted disklabel but that won't work here because on a GPT disk without OpenBSD partition the disklabel and the primary GPT share a physical sector and that won't work.) > On 21/12/21 18:04 Rob Whitlock wrote: > > I have two disks, one an MBR partitioned 1TB external SSD, and the other a > > GPT partitioned 5TB external HDD. Both have a single ExFAT partition on > > them and both have the same contents. Both show up as sd1 under "sysctl > > hw.disknames" (when plugged in one at a time, that is). I am able to mount > > the MBR partitioned SSD with the command > > > > mount.exfat-fuse /dev/sd1i /mnt > > > > however when I try the same command with the GPT partitioned HDD I get the > > error > > > > FUSE exfat 1.2.8 > > ERROR: failed to open '/dev/sd1i': Device not configured. > > > > I checked that the /dev/sd1i block device exists. I am running OpenBSD 6.9. > > Here's the output of disklabel sd1 > > > > # /dev/rsd1c: > > type: SCSI > > disk: SCSI disk > > label: Expansion Desk > > duid: > > flags: > > bytes/sector: 512 > > sectors/track: 63 > > tracks/cylinder: 255 > > sectors/cylinder: 16065 > > cylinders: 608001 > > total sectors: 9767541167 > > boundstart: 0 > > boundend: 9767541167 > > drivedata: 0 > > > > 16 partitions: > > #size offset fstype [fsize bsize cpg] > > c: 97675411670 unused > > > > > > Here's the output of fdisk -v sd1 > > Primary GPT: > > Disk: sd1 Usable LBA: 34 to 9767541133 [9767541167 Sectors] > > GUID: 570ab069-1869-44ed-911b-a568af1275ff > >#: type [ start: size ] > > guid name > > > >0: EFI Sys [ 40: 409600 ] > > a4bd4c86-177d-4e02-a9f9-afc51ade8d87 EFI System Partition > > > >1: FAT12[ 411648: 9767129088 ] > > 7157c7e5-978f-4a43-96af-fa6648710488 > > > > > > Secondary GPT: > > Disk: sd1 Usable LBA: 34 to 9767541133 [9767541167 Sectors] > > GUID: 570ab069-1869-44ed-911b-a568af1275ff > >#: type [ start: size ] > > guid name > > > >0: EFI Sys [ 40: 409600 ] > > a4bd4c86-177d-4e02-a9f9-afc51ade8d87 EFI System Partition > > > >1: FAT12[ 411648: 9767129088 ] > > 7157c7e5-978f-4a43-96af-fa6648710488 > > > > > > MBR: > > Disk: sd1 geometry: 267349/255/63 [4294961685 Sectors] > > Offset: 0 Signature: 0xAA55 > > Starting Ending LBA Info: > > #: id C H S - C H S [ start:size ] > > --- > > 0: EE 0 0 2 - 267349 89 3 [ 1: 4294967294 ] EFI GPT > > > > 1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused > > > > 2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused > > > > 3: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused > > > > > > A problem seems to be that there is no disklabel entry for the ExFAT > > partition. Additionally, xxd successfully reads the first few sectors of > > /dev/sd1c so I don't think the hardware is the issue. > > > > How can I mount the HDD ExFAT partition? > > > > Thanks! > > > > Rob >
Re: Disk partition not recognized
James Cook wrote: > I thought the disklabel lives at the start of the OpenBSD partition. That is incorrect.
Re: Disk partition not recognized
On Wed, Dec 22, 2021 at 07:21:41AM -0300, Crystal Kolipe wrote: > On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > > A problem seems to be that there is no disklabel entry for the ExFAT > > partition. > > You probably wrote a BSD disklabel to the disk before creating the ExFAT > partition. > > If there is no on-disk disklabel, the kernel will create one in memory based > on information from other partitioning schemes, (MBR, GPT). So in this case, > as you change those MBR or GPT partitions, those changes will be reflected in > the disklabel that the kernel sees. > > Once you actually write a disklabel to the disk, that on-disk disklabel is > then used in place of calculating one each time the disk is attached, and the > automatic parsing of MBR and GPT partition information stops. > > To solve your problem, you need to add the details of the ExFAT partition to > the BSD disklabel. You can either do that manually with the disklabel > command, or since you do not have any OpenBSD partitions on the disk, you > could overwrite the on-disk disklabel, allow the kernel to generate one > automatically with the correct information, then optionally force it to be > written to the disk by running disklabel and entering 'w' at the interactive > prompt. I thought the disklabel lives at the start of the OpenBSD partition. Since that disk has no OpenBSD partition, it can't have a disklabel, right? Could the in-memory disklabel be out of sync? Does the problem persist if you reboot, or detach/re-attach the disk? -- James
Re: Disk partition not recognized
On 21/12/21 18:04 Rob Whitlock wrote: > I have two disks, one an MBR partitioned 1TB external SSD, and the other a > GPT partitioned 5TB external HDD. Both have a single ExFAT partition on > them and both have the same contents. Both show up as sd1 under "sysctl > hw.disknames" (when plugged in one at a time, that is). I am able to mount > the MBR partitioned SSD with the command > > mount.exfat-fuse /dev/sd1i /mnt > > however when I try the same command with the GPT partitioned HDD I get the > error > > FUSE exfat 1.2.8 > ERROR: failed to open '/dev/sd1i': Device not configured. > > I checked that the /dev/sd1i block device exists. I am running OpenBSD 6.9. > Here's the output of disklabel sd1 > > # /dev/rsd1c: > type: SCSI > disk: SCSI disk > label: Expansion Desk > duid: > flags: > bytes/sector: 512 > sectors/track: 63 > tracks/cylinder: 255 > sectors/cylinder: 16065 > cylinders: 608001 > total sectors: 9767541167 > boundstart: 0 > boundend: 9767541167 > drivedata: 0 > > 16 partitions: > #size offset fstype [fsize bsize cpg] > c: 97675411670 unused > > > Here's the output of fdisk -v sd1 > Primary GPT: > Disk: sd1 Usable LBA: 34 to 9767541133 [9767541167 Sectors] > GUID: 570ab069-1869-44ed-911b-a568af1275ff >#: type [ start: size ] > guid name > >0: EFI Sys [ 40: 409600 ] > a4bd4c86-177d-4e02-a9f9-afc51ade8d87 EFI System Partition > >1: FAT12[ 411648: 9767129088 ] > 7157c7e5-978f-4a43-96af-fa6648710488 > > > Secondary GPT: > Disk: sd1 Usable LBA: 34 to 9767541133 [9767541167 Sectors] > GUID: 570ab069-1869-44ed-911b-a568af1275ff >#: type [ start: size ] > guid name > >0: EFI Sys [ 40: 409600 ] > a4bd4c86-177d-4e02-a9f9-afc51ade8d87 EFI System Partition > >1: FAT12[ 411648: 9767129088 ] > 7157c7e5-978f-4a43-96af-fa6648710488 > > > MBR: > Disk: sd1 geometry: 267349/255/63 [4294961685 Sectors] > Offset: 0 Signature: 0xAA55 > Starting Ending LBA Info: > #: id C H S - C H S [ start:size ] > --- > 0: EE 0 0 2 - 267349 89 3 [ 1: 4294967294 ] EFI GPT > > 1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused > > 2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused > > 3: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused > > > A problem seems to be that there is no disklabel entry for the ExFAT > partition. Additionally, xxd successfully reads the first few sectors of > /dev/sd1c so I don't think the hardware is the issue. > > How can I mount the HDD ExFAT partition? > > Thanks! > > Rob
Re: Disk partition not recognized
On Tue, Dec 21, 2021 at 06:04:28PM -0500, Rob Whitlock wrote: > A problem seems to be that there is no disklabel entry for the ExFAT > partition. You probably wrote a BSD disklabel to the disk before creating the ExFAT partition. If there is no on-disk disklabel, the kernel will create one in memory based on information from other partitioning schemes, (MBR, GPT). So in this case, as you change those MBR or GPT partitions, those changes will be reflected in the disklabel that the kernel sees. Once you actually write a disklabel to the disk, that on-disk disklabel is then used in place of calculating one each time the disk is attached, and the automatic parsing of MBR and GPT partition information stops. To solve your problem, you need to add the details of the ExFAT partition to the BSD disklabel. You can either do that manually with the disklabel command, or since you do not have any OpenBSD partitions on the disk, you could overwrite the on-disk disklabel, allow the kernel to generate one automatically with the correct information, then optionally force it to be written to the disk by running disklabel and entering 'w' at the interactive prompt.
