Re: mkimg used to create gpt image, problem booting
Andrey V. Elsukov wrote this message on Mon, Aug 25, 2014 at 19:02 +0400: > On 25.08.2014 18:40, Marcel Moolenaar wrote: > >> Also, now FreeBSD 11.0 uses different first usable LBA. By default it is > >> 4k aligned. And this creates some incompatibility with older versions. > >> You can't do `gpart restore` and get the same table, as you had on older > >> system. > > > > It sounds restore is broken then. The restore command > > cannot ever assume anything about the GPT. Including > > the tool that created the GPT. In order to restore a > > GPT, it must be properly backed-up. The backup header > > and table should suffice most of the time for that > > purpose as it's a replica, but as soon as meta-data is > > missing and the restore command has to guess, things > > will go wrong. > > `gpart restore` just uses a number of commands to geom_part(4) to create > partition table similar to what was backed up. If your partition table > on the old system had a partition that starts from LBA 34, now `gpart > create` isn't able to create such partition table. Because by default > the first usable LBA is 40. Luckily, gpart restore won't work: # gpart backup /dev/md0 GPT 4 1 freebsd-ufs 8 262144 # gpart restore md1 < /tmp/foob.gpt.back gpart: entries '4': Invalid argument So, we're somewhat safe, guess gpart restore needs to learn how to handle entries properly We should fix this, since other OS's might not use 128 for entries.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On 25.08.2014 18:40, Marcel Moolenaar wrote: >> Also, now FreeBSD 11.0 uses different first usable LBA. By default it is >> 4k aligned. And this creates some incompatibility with older versions. >> You can't do `gpart restore` and get the same table, as you had on older >> system. > > It sounds restore is broken then. The restore command > cannot ever assume anything about the GPT. Including > the tool that created the GPT. In order to restore a > GPT, it must be properly backed-up. The backup header > and table should suffice most of the time for that > purpose as it's a replica, but as soon as meta-data is > missing and the restore command has to guess, things > will go wrong. `gpart restore` just uses a number of commands to geom_part(4) to create partition table similar to what was backed up. If your partition table on the old system had a partition that starts from LBA 34, now `gpart create` isn't able to create such partition table. Because by default the first usable LBA is 40. -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Aug 25, 2014, at 12:48 AM, Andrey V. Elsukov wrote: > On 25.08.2014 03:55, Marcel Moolenaar wrote: >>> Though, w/ people dd'ing images onto disks, and using growfs to grow >>> as necessary, we might want to allocate a few more more than the >>> minimum... I do agree that we want to keep sizes to a minimum... >> >> One thing I can maybe do is simply fill the empty sectors >> that are there because of alignment. If you add -P 4K to >> mkimg, then the first partition will by 4K aligned and you >> have about 5 sectors unused between the end of the GPT >> table and the first sector of the first partition. I may >> as well extend the table to cover those unued sectors. > > IMHO, mkimg should behave like gpart and create images in > gpart-compatible way. It already does. There's s difference between behaving like something else or behaving exactly identical to that something. The same applies to compatible. It is not the same as identical. There is no compatibility issue. mkimg follows the GPT specification (modulo bugs) and gpart happily groks the partition table. > Some users may want to copy the partition layout > from the image to real hardware and they will not be able to do it. I'm inclined to say that generally speaking this is not possible. The GPT has metadata in the first few sectors *and* the last few sectors and LBAs of these sectors are part of the metadata. You cannot blindly copy an image onto a physical medium unless the image and the physical medium are of exactly the same size. Odds are they are not. To reliably transfer or convert an image (e.g. RAW->VHD) one must modify the image as part of the process. Not a hard rule, but best to assume as a rule of thumb. This seems to warrant a utility all by itself. > Also, now FreeBSD 11.0 uses different first usable LBA. By default it is > 4k aligned. And this creates some incompatibility with older versions. > You can't do `gpart restore` and get the same table, as you had on older > system. It sounds restore is broken then. The restore command cannot ever assume anything about the GPT. Including the tool that created the GPT. In order to restore a GPT, it must be properly backed-up. The backup header and table should suffice most of the time for that purpose as it's a replica, but as soon as meta-data is missing and the restore command has to guess, things will go wrong. -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: mkimg used to create gpt image, problem booting
On 25.08.2014 03:55, Marcel Moolenaar wrote: >> Though, w/ people dd'ing images onto disks, and using growfs to grow >> as necessary, we might want to allocate a few more more than the >> minimum... I do agree that we want to keep sizes to a minimum... > > One thing I can maybe do is simply fill the empty sectors > that are there because of alignment. If you add -P 4K to > mkimg, then the first partition will by 4K aligned and you > have about 5 sectors unused between the end of the GPT > table and the first sector of the first partition. I may > as well extend the table to cover those unued sectors. IMHO, mkimg should behave like gpart and create images in gpart-compatible way. Some users may want to copy the partition layout from the image to real hardware and they will not be able to do it. Also, now FreeBSD 11.0 uses different first usable LBA. By default it is 4k aligned. And this creates some incompatibility with older versions. You can't do `gpart restore` and get the same table, as you had on older system. > However, this is a pretty side-ways way to end up with a > GPT table that has some extra room. Maybe having scheme- > specific options for this kind of thing is not bad. At > least EBR and APM have the same "problem" and the BSD > disk label can also be created with more than just 8 > partitions. I thought about implementing `gpart modify` or `gpart set` -n entries to change number of entries when it is possible (i.e. disklabel(8) can do it, but gpart doesn't). Also in r228076 I changed APM code to calculate maximum number of entries depending from available free space. -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On 24.08.2014 20:59, Craig Rodrigues wrote: > Also, in the gptboot man page, it mentions that gptboot can only boot > on systems with 128 partitions or less. This seems like an artificial > restriction. > Does the gptboot code really enforce this? Not that I have a system with more > than 128 partitions. :) It's because gptboot uses static buffer to read and write GPT table. -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On 24.08.2014 19:23, Marcel Moolenaar wrote:
>
> On Aug 24, 2014, at 2:11 AM, Andrey V. Elsukov wrote:
>
>> On 24.08.2014 06:14, Craig Rodrigues wrote:
>
>>> I did some further debugging inside the loader by doing the following.
>>> -> I added "CFLAGS += -DPART_DEBUG" to sys/boot/common/Makefile.inc
>>> -> I added DEBUG() statements all over sys/boot/common/part.c
>>>
>>> I observed that in sys/boot/common/part.c in the ptbl_gptread() function,
>>> that in this section:
>>>
>>>305 ent = (struct gpt_ent *)tbl;
>>>306 size = MIN(hdr.hdr_entries * hdr.hdr_entsz,
>>>307 MAXTBLSZ * table->sectorsize);
>>>308 for (i = 0; i < size / hdr.hdr_entsz; i++, ent++) {
>>>309 if (uuid_equal(&ent->ent_type, &gpt_uuid_unused,
>>> NULL))
>>>310 continue;
>>>
>>> ent->ent_type is all 0's, which matches gpt_uuid_unused, so it bails
>>> out of the loop and never adds the gpt partitions to the list of partitions
>>> that the loader can access.
>>
>> Yes, the problem is in the ptable_gptread() function. I'll commit the fix.
>>
>
> Actually, no. There is *a* problem in that function:
> The function does not respect hdr.hdr_entsz when it
> needs the next entry. It simply uses "ent++", which
> is fixed our definition of struct gpt_ent and may
> not match the definition of the writer.
Yes, you are right. I'll fix this. Thanks.
> I don't see how the loader is responsible for *the*
> problem. All I see in qemu is that the loader, when
> it reads a sector, isn't getting the actual sector
> data that's in the image.
>
> Just do a ktrace on qemu and you'll see what I mean.
> YMMV of course,
Also there is bootparttest utility in the tools/tools/bootparttest. I
think it can be useful for debugging.
--
WBR, Andrey V. Elsukov
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Aug 24, 2014, at 4:31 PM, John-Mark Gurney wrote: >> >> With mkimg, you know exactly how many partitions you are creating >> , so you don't need to specify 128 as the number of partitions. > > Though, w/ people dd'ing images onto disks, and using growfs to grow > as necessary, we might want to allocate a few more more than the > minimum... I do agree that we want to keep sizes to a minimum... One thing I can maybe do is simply fill the empty sectors that are there because of alignment. If you add -P 4K to mkimg, then the first partition will by 4K aligned and you have about 5 sectors unused between the end of the GPT table and the first sector of the first partition. I may as well extend the table to cover those unued sectors. However, this is a pretty side-ways way to end up with a GPT table that has some extra room. Maybe having scheme- specific options for this kind of thing is not bad. At least EBR and APM have the same "problem" and the BSD disk label can also be created with more than just 8 partitions. Related: o Is there a need to create images with empty space at the end of in between partitions? o Is there a need to create partitions with specific indices (i.e. index 6 for a typical 'f' partition)? Answers to these questions could help figure this out... -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: mkimg used to create gpt image, problem booting
Craig Rodrigues wrote this message on Sun, Aug 24, 2014 at 15:37 -0700: > On Sun, Aug 24, 2014 at 12:05 PM, Marcel Moolenaar wrote: > > > > On Aug 24, 2014, at 9:59 AM, Craig Rodrigues wrote: > > > >> On Sun, Aug 24, 2014 at 2:11 AM, Andrey V. Elsukov > >> wrote: > >>> > >>> Yes, the problem is in the ptable_gptread() function. I'll commit the fix. > >> > > > >> Should mkimg be changed to create a partition table with 128 entries > >> by default, to match older versions of FreeBSD which do not have this fix? > > > > Maybe. 128 is the suggested default. It's not a hard lower > > limit. Technically speaking, it's perfectly fine to create > > just enough entries to fill a single sector. Then again, > > code makes all kinds of assumptions or has all kinds of > > bugs -- just like the logic in the loader apparently. > > > > By having mkimg create a large table, even if it's knows > > up front that it doesn't have to may prevent broken code > > from tripping over, bit it surely bloats the image for > > no reason. > > I see what you are saying. If you have a new device > and do "gpart create -s GPT", then this value in sys/geom/part/g_part_gpt.c: > > .gps_minent = 128, > > causes the logic in the g_part_ctl_create() function in sys/geom/part/g_part.c > to set the number of partitions to 128, and then it calls g_part_ctl_create() > which creates the partition table with 128 empty entries. > > "gpart create" doesn't know how many partitions you want, so it > needs to allocate some space up front for the partition table, > and then you can do "gpart add" to add the partitions later. > > With mkimg, you know exactly how many partitions you are creating > , so you don't need to specify 128 as the number of partitions. Though, w/ people dd'ing images onto disks, and using growfs to grow as necessary, we might want to allocate a few more more than the minimum... I do agree that we want to keep sizes to a minimum... > Since only gpart was available for creating GPT partitions, the side-effect > of always having 128 partitions hid the bug in the loader. > > Hopefully Andrey's fix can be backported to at least stable/9, because > the loader bug seems to have been there since at least 2012. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Sun, Aug 24, 2014 at 12:05 PM, Marcel Moolenaar wrote: > > On Aug 24, 2014, at 9:59 AM, Craig Rodrigues wrote: > >> On Sun, Aug 24, 2014 at 2:11 AM, Andrey V. Elsukov wrote: >>> >>> Yes, the problem is in the ptable_gptread() function. I'll commit the fix. >> > >> Should mkimg be changed to create a partition table with 128 entries >> by default, to match older versions of FreeBSD which do not have this fix? > > Maybe. 128 is the suggested default. It's not a hard lower > limit. Technically speaking, it's perfectly fine to create > just enough entries to fill a single sector. Then again, > code makes all kinds of assumptions or has all kinds of > bugs -- just like the logic in the loader apparently. > > By having mkimg create a large table, even if it's knows > up front that it doesn't have to may prevent broken code > from tripping over, bit it surely bloats the image for > no reason. I see what you are saying. If you have a new device and do "gpart create -s GPT", then this value in sys/geom/part/g_part_gpt.c: .gps_minent = 128, causes the logic in the g_part_ctl_create() function in sys/geom/part/g_part.c to set the number of partitions to 128, and then it calls g_part_ctl_create() which creates the partition table with 128 empty entries. "gpart create" doesn't know how many partitions you want, so it needs to allocate some space up front for the partition table, and then you can do "gpart add" to add the partitions later. With mkimg, you know exactly how many partitions you are creating , so you don't need to specify 128 as the number of partitions. Since only gpart was available for creating GPT partitions, the side-effect of always having 128 partitions hid the bug in the loader. Hopefully Andrey's fix can be backported to at least stable/9, because the loader bug seems to have been there since at least 2012. -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Aug 24, 2014, at 9:59 AM, Craig Rodrigues wrote: > On Sun, Aug 24, 2014 at 2:11 AM, Andrey V. Elsukov wrote: >> >> Yes, the problem is in the ptable_gptread() function. I'll commit the fix. > > Should mkimg be changed to create a partition table with 128 entries > by default, to match older versions of FreeBSD which do not have this fix? Maybe. 128 is the suggested default. It's not a hard lower limit. Technically speaking, it's perfectly fine to create just enough entries to fill a single sector. Then again, code makes all kinds of assumptions or has all kinds of bugs -- just like the logic in the loader apparently. By having mkimg create a large table, even if it's knows up front that it doesn't have to may prevent broken code from tripping over, bit it surely bloats the image for no reason. -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: mkimg used to create gpt image, problem booting
On Sun, Aug 24, 2014 at 8:23 AM, Marcel Moolenaar wrote:
>
> On Aug 24, 2014, at 2:11 AM, Andrey V. Elsukov wrote:
>
>> On 24.08.2014 06:14, Craig Rodrigues wrote:
>
>>> I did some further debugging inside the loader by doing the following.
>>> -> I added "CFLAGS += -DPART_DEBUG" to sys/boot/common/Makefile.inc
>>> -> I added DEBUG() statements all over sys/boot/common/part.c
>>>
>>> I observed that in sys/boot/common/part.c in the ptbl_gptread() function,
>>> that in this section:
>>>
>>>305 ent = (struct gpt_ent *)tbl;
>>>306 size = MIN(hdr.hdr_entries * hdr.hdr_entsz,
>>>307 MAXTBLSZ * table->sectorsize);
>>>308 for (i = 0; i < size / hdr.hdr_entsz; i++, ent++) {
>>>309 if (uuid_equal(&ent->ent_type, &gpt_uuid_unused,
>>> NULL))
>>>310 continue;
>>>
>>> ent->ent_type is all 0's, which matches gpt_uuid_unused, so it bails
>>> out of the loop and never adds the gpt partitions to the list of partitions
>>> that the loader can access.
>>
>> Yes, the problem is in the ptable_gptread() function. I'll commit the fix.
>>
>
> Actually, no. There is *a* problem in that function:
> The function does not respect hdr.hdr_entsz when it
> needs the next entry. It simply uses "ent++", which
> is fixed our definition of struct gpt_ent and may
> not match the definition of the writer.
Yes, you are correct. I looked at the GPT format here:
http://en.wikipedia.org/wiki/GUID_Partition_Table
Although the default size in the specification for a gpt entry is 128 bytes,
which matches the size of our "struct gpt_entry",
technically, the gpt header could specify a gpt entry size that is something
other than 128 bytes. I guess the only restriction seems to be that
you cannot have variable sized gpt entries.they have to match the
size of the gpt entry specified in the gpt header.
I guess we haven't hit this yet because there are probably very few
peopel creating gpt tables with entry sizes other than 128, but in the
future, who knows?
--
Craig
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Sun, Aug 24, 2014 at 2:11 AM, Andrey V. Elsukov wrote:
>
> Yes, the problem is in the ptable_gptread() function. I'll commit the fix.
Index: head/sys/boot/common/part.c
===
--- head/sys/boot/common/part.c (revision 270444)
+++ head/sys/boot/common/part.c (revision 270445)
@@ -254,8 +254,8 @@
table->sectorsize);
if (phdr != NULL) {
/* Read the primary GPT table. */
- size = MIN(MAXTBLSZ,
- phdr->hdr_entries * phdr->hdr_entsz / table->sectorsize);
+ size = MIN(MAXTBLSZ, (phdr->hdr_entries * phdr->hdr_entsz +
+ table->sectorsize - 1) / table->sectorsize);
if (dread(dev, tbl, size, phdr->hdr_lba_table) == 0 &&
gpt_checktbl(phdr, tbl, size * table->sectorsize,
table->sectors - 1) == 0) {
I can confirm that r270445 fixes the problem for me, where I can
now QEMU boot a GPT partitioned image created with mkimg.
I put some more debugging in the code, and found this:
(1) GPT IMAGE CREATED WITH MKIMG
===
phdr->hdr_entries = 2, phdr->hdr_entsz = 128, table->sectorsize = 512
(2) GPT IMAGE CREATED WITH BSDINSTALL
===
phdr->hdr_entries = 128, phdr->hdr_entsz = 128, table->sectorsize = 512
Does gpart create a fixed partition table with 128 entries?
That would explain a lot.
Also, in the gptboot man page, it mentions that gptboot can only boot
on systems with 128 partitions or less. This seems like an artificial
restriction.
Does the gptboot code really enforce this? Not that I have a system with more
than 128 partitions. :)
Should mkimg be changed to create a partition table with 128 entries
by default, to match older versions of FreeBSD which do not have this fix?
Thanks for the fix!
--
Craig
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Aug 24, 2014, at 2:11 AM, Andrey V. Elsukov wrote:
> On 24.08.2014 06:14, Craig Rodrigues wrote:
>> I did some further debugging inside the loader by doing the following.
>> -> I added "CFLAGS += -DPART_DEBUG" to sys/boot/common/Makefile.inc
>> -> I added DEBUG() statements all over sys/boot/common/part.c
>>
>> I observed that in sys/boot/common/part.c in the ptbl_gptread() function,
>> that in this section:
>>
>>305 ent = (struct gpt_ent *)tbl;
>>306 size = MIN(hdr.hdr_entries * hdr.hdr_entsz,
>>307 MAXTBLSZ * table->sectorsize);
>>308 for (i = 0; i < size / hdr.hdr_entsz; i++, ent++) {
>>309 if (uuid_equal(&ent->ent_type, &gpt_uuid_unused,
>> NULL))
>>310 continue;
>>
>> ent->ent_type is all 0's, which matches gpt_uuid_unused, so it bails
>> out of the loop and never adds the gpt partitions to the list of partitions
>> that the loader can access.
>
> Yes, the problem is in the ptable_gptread() function. I'll commit the fix.
>
Actually, no. There is *a* problem in that function:
The function does not respect hdr.hdr_entsz when it
needs the next entry. It simply uses "ent++", which
is fixed our definition of struct gpt_ent and may
not match the definition of the writer.
I don't see how the loader is responsible for *the*
problem. All I see in qemu is that the loader, when
it reads a sector, isn't getting the actual sector
data that's in the image.
Just do a ktrace on qemu and you'll see what I mean.
YMMV of course,
--
Marcel Moolenaar
mar...@xcllnt.net
signature.asc
Description: Message signed with OpenPGP using GPGMail
Re: mkimg used to create gpt image, problem booting
On 24.08.2014 06:14, Craig Rodrigues wrote:
> Hi,
>
> I did some more experiments, and found that after /boot/loader runs,
> if I break into the loader prompt and type "lsdev", I would get this:
>
> (1) GPT Disk image which boots under QEMU, made by bsdinstall
> ==
> View from loader
>
> OK lsdev
> cd devices:
> disk devices:
> disk0:BIOS drive A:
> disk1:BIOS drive C:
> disk1p1: FreeBSD boot
> disk1p2: FreeBSD UFS
> disk1p3: FreeBSD swap
> pxe devices:
>
>
> View from gpart, after we mdconfig the disk image
>
> => 34 10485693 md0 GPT (5.0G)
> 34 1281 freebsd-boot (64K)
>162 99592962 freebsd-ufs (4.7G)
>99594585242883 freebsd-swap (256M)
> 10483746 1981 - free - (991K)
>
>
> (2) GPT Disk image which fails to boot under QEMU, made by mkimg
> ===
> View from loader
>
> OK lsdev
> cd devices:
> disk devices:
> disk0:BIOS drive A:
> disk1:BIOS drive C:
> pxe devices:
>
> View from gpart, after we mdconfig the disk image
>
>
> => 3 1784944 md1 GPT (872M)
> 3 321 freebsd-boot (16K)
>35 17849122 freebsd-ufs (872M)
>
>
>
> This leads me to believe that there is logic in /boot/loader,
> which is not in GEOM, that fails to parse the GPT produced by mkimg.
>
>
> I did some further debugging inside the loader by doing the following.
> -> I added "CFLAGS += -DPART_DEBUG" to sys/boot/common/Makefile.inc
> -> I added DEBUG() statements all over sys/boot/common/part.c
>
> I observed that in sys/boot/common/part.c in the ptbl_gptread() function,
> that in this section:
>
> 305 ent = (struct gpt_ent *)tbl;
> 306 size = MIN(hdr.hdr_entries * hdr.hdr_entsz,
> 307 MAXTBLSZ * table->sectorsize);
> 308 for (i = 0; i < size / hdr.hdr_entsz; i++, ent++) {
> 309 if (uuid_equal(&ent->ent_type, &gpt_uuid_unused,
> NULL))
> 310 continue;
>
> ent->ent_type is all 0's, which matches gpt_uuid_unused, so it bails
> out of the loop and never adds the gpt partitions to the list of partitions
> that the loader can access.
>
> I'm not familiar with the GPT format, nor am I familiar with the
> gpt code inside the loader, and how it differs from GEOM.
>
> Do you have any further ideas of where to hunt for the root cause of
> the problem?
Yes, the problem is in the ptable_gptread() function. I'll commit the fix.
--
WBR, Andrey V. Elsukov
signature.asc
Description: OpenPGP digital signature
Re: mkimg used to create gpt image, problem booting
Hi,
I did some more experiments, and found that after /boot/loader runs,
if I break into the loader prompt and type "lsdev", I would get this:
(1) GPT Disk image which boots under QEMU, made by bsdinstall
==
View from loader
OK lsdev
cd devices:
disk devices:
disk0:BIOS drive A:
disk1:BIOS drive C:
disk1p1: FreeBSD boot
disk1p2: FreeBSD UFS
disk1p3: FreeBSD swap
pxe devices:
View from gpart, after we mdconfig the disk image
=> 34 10485693 md0 GPT (5.0G)
34 1281 freebsd-boot (64K)
162 99592962 freebsd-ufs (4.7G)
99594585242883 freebsd-swap (256M)
10483746 1981 - free - (991K)
(2) GPT Disk image which fails to boot under QEMU, made by mkimg
===
View from loader
OK lsdev
cd devices:
disk devices:
disk0:BIOS drive A:
disk1:BIOS drive C:
pxe devices:
View from gpart, after we mdconfig the disk image
=> 3 1784944 md1 GPT (872M)
3 321 freebsd-boot (16K)
35 17849122 freebsd-ufs (872M)
This leads me to believe that there is logic in /boot/loader,
which is not in GEOM, that fails to parse the GPT produced by mkimg.
I did some further debugging inside the loader by doing the following.
-> I added "CFLAGS += -DPART_DEBUG" to sys/boot/common/Makefile.inc
-> I added DEBUG() statements all over sys/boot/common/part.c
I observed that in sys/boot/common/part.c in the ptbl_gptread() function,
that in this section:
305 ent = (struct gpt_ent *)tbl;
306 size = MIN(hdr.hdr_entries * hdr.hdr_entsz,
307 MAXTBLSZ * table->sectorsize);
308 for (i = 0; i < size / hdr.hdr_entsz; i++, ent++) {
309 if (uuid_equal(&ent->ent_type, &gpt_uuid_unused, NULL))
310 continue;
ent->ent_type is all 0's, which matches gpt_uuid_unused, so it bails
out of the loop and never adds the gpt partitions to the list of partitions
that the loader can access.
I'm not familiar with the GPT format, nor am I familiar with the
gpt code inside the loader, and how it differs from GEOM.
Do you have any further ideas of where to hunt for the root cause of
the problem?
Thanks.
--
Craig
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Sat, Aug 23, 2014 at 12:11 PM, Marcel Moolenaar wrote: > > Could be. Try the -P option to mkimg. It sets the > underlying (unexposed) physical sector size while > still working with the visible 512 bytes sectors. > The net effect is that for the GPT scheme things > get aligned to the physical sector size and that > it also causes the image size to be rounded. > > You can also try emitting vmdk directly to see if > that makes a difference. vmdk also has the side- > effect of rounding the image to the grain size. I tried the following experiments: mkimg -v -f vmdk -s gpt -b test1/boot/pmbr -p freebsd-boot:=test1/boot/gptboot -p freebsd-ufs:=/tmp/file.img -o /tmp/foo1.vmdk When I tried to boot the image in QEMU, I had the same problem as before. It looks like it started writing the image on block 3, same as before. I also tried adding the -P flag, with different values like 2048 and 4096. I ran into the same problem. Hmm. -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Aug 23, 2014, at 12:00 PM, Craig Rodrigues wrote: > > I ran the following crazy experiment, just to see what would happen: > > dd if=/dev/md1s2 of=/dev/md0s2 bs=8192 > > I then tried to boot the first image with QEMU, and it booted successfully, > with my UFS file system that I had previously created with makefs. > > I'm not sure where to look for the problem. I notice that > in the non-working image, the offset starts at block 3, > while in the working image, the offset starts at block 34. > > Is that enough to make things not boot? Could be. Try the -P option to mkimg. It sets the underlying (unexposed) physical sector size while still working with the visible 512 bytes sectors. The net effect is that for the GPT scheme things get aligned to the physical sector size and that it also causes the image size to be rounded. You can also try emitting vmdk directly to see if that makes a difference. vmdk also has the side- effect of rounding the image to the grain size. -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: mkimg used to create gpt image, problem booting
On Fri, Aug 22, 2014 at 1:45 PM, Marcel Moolenaar wrote: > >> If I mdconfig the foo1.img disk image, and do a gpart show, I see: >> >> => 3 1784944 md0 GPT (872M) >>3 321 freebsd-boot (16K) >> 35 17849122 freebsd-ufs (872M) >> >> Any idea what I am doing wrong? > > To the best of my knowledge, qemu is the thing you're > doing wrong :-) Hi, I transferred foo1.img to a Mac with VirtualBox, converted it to VMDK with "VBoxManage convertfromraw --format VMDK", and tried to boot it in VirtualBox. I got the same error as in QEMU. It looks like /boot/loader runs, but I cannot do "ls" to see the root file system. I created another disk image with a GPT layout, but this time using the FreeBSD bsdinstall inside a bhyve VM. I noticed the following: WORKING IMAGE BOOTS IN QEMU, CREATED WITH BSDINSTALL = => 34 10485693 md0 GPT (5.0G) 34 1281 83bd6b9d-7f41-11dc-be0b-001560b84f0f (64K) 162 99592962 516e7cb6-6ecf-11d6-8ff8-00022d09712b (4.7G) 99594585242883 516e7cb5-6ecf-11d6-8ff8-00022d09712b (256M) 10483746 1981 - free - (991K) DOES NOT BOOT IN QEMU, CREATED WITH MKIMG === => 3 1784944 md1 GPT (872M) 3 321 83bd6b9d-7f41-11dc-be0b-001560b84f0f (16K) 35 17849122 516e7cb6-6ecf-11d6-8ff8-00022d09712b (872M) I ran the following crazy experiment, just to see what would happen: dd if=/dev/md1s2 of=/dev/md0s2 bs=8192 I then tried to boot the first image with QEMU, and it booted successfully, with my UFS file system that I had previously created with makefs. I'm not sure where to look for the problem. I notice that in the non-working image, the offset starts at block 3, while in the working image, the offset starts at block 34. Is that enough to make things not boot? -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On 22 August 2014 16:45, Marcel Moolenaar wrote: > > I have so far not been able to boot an image created > by mkimg with a FreeBSD-hosted qemu. > o VMware and VirtualBox are fine. > o A non-FreeBSD hosted qemu also works fine. For what it's worth, I have no trouble booting a mkimg-created image with QEMU on FreeBSD, and have been using it regularly for vt(4) and UEFI development. Some particulars: * Host is amd64 stable/10 with some additional MFC candidates (uname reports "FreeBSD 10.0-STABLE #1 r268946+94ba9c8(stable-10)") * My mkimg command was "mkimg -s gpt -b $ROOTDIR/boot/pmbr -p efi:=$ROOTDIR/boot/boot1.efifat -p freebsd-boot:=$ROOTDIR/boot/gptboot -p freebsd-ufs:=ufs -p freebsd-swap::1M -o img" * QEMU installed from the package, qemu-devel-2.0.0_2 * SeaBIOS version info printed at startup is the same as Craig's ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: mkimg used to create gpt image, problem booting
On Aug 22, 2014, at 9:49 AM, Craig Rodrigues wrote: > > (5) Tried to boot the image with qemu: > > qemu-system-x86_64 -m 2048 -hda /tmp/foo1.img *snip* > If I mdconfig the foo1.img disk image, and do a gpart show, I see: > > => 3 1784944 md0 GPT (872M) >3 321 freebsd-boot (16K) > 35 17849122 freebsd-ufs (872M) > > Any idea what I am doing wrong? To the best of my knowledge, qemu is the thing you're doing wrong :-) I have so far not been able to boot an image created by mkimg with a FreeBSD-hosted qemu. o VMware and VirtualBox are fine. o A non-FreeBSD hosted qemu also works fine. If your host is running -current, make sure to set MALLOC_CONF=junk:false. It improves behaviour on FreeBSD for boot0/boo1. HTH (probably not), -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
