Re: Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Ian Campbell i...@debian.org (2015-02-01): Given that I don't really understand how this additional firmware partition stuff is supposed to fit together (i.e. what impact changing its partition number or adding another VFAT partition etc would have on the code which finds/loads from it) I'm very reluctant to be the one to suggest that we should change things here for Jessie given we've had the beta and are frozen etc. Perhaps we should park this until Stretch? (Kibi, any thoughts on that?) Revisiting that during the stretch cycle doesn't seem shocking to me. Mraw, KiBi. signature.asc Description: Digital signature
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, i understand that debian-cd mailing list is in charge for discussion of amd64 mini.iso and its generating software: http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/config/x86.cfg http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/util/geniso_hybrid_plus_firmware_partition For the sake of persistence i add my assessment to bug 776317, although it somewhat exceeds the scope of a bug report. - The bug reporter's EFI firmware mistook the appended firmware partion (an empty FAT filesystem) for the EFI System Partition. The real EFI partition is located inside the ISO filesystem and not advertised by MBR. So the ISO did not boot from USB stick until the file tree from the EFI partition was copied to the firmware partition. Other EFIs might not be so tolerant and insist in MBR partition type 0xef or in a GPT partition in order to find the first piece of GRUB2 software. The very dense partition layout of debian-7*amd64-netinst.iso obviously serves the needs and quirks of the EFIs which were encountered so far. But it makes partition editors scream. The firmware partition is intended for being used on a hard-disk-like device, where the wish for partition editing is natural, because the device is larger than the ISO image. So how can mini.iso become friendly towards partition editors ? I will discuss three layout alternatives: - UEFI compliant Legacy MBR based - UEFI compliant GPT based - Rogue but well tested amd64-netinst with firmware partition My conclusion is that UEFI compliant Legacy MBR is most promising in respect to work effort and ease of use. The production of this layout is essentially sketched in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776317#45 and improved in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776317#50 The following paragraph also shows how to achieve this with xorriso to avoid the need for post-processing the ISO image. --- UEFI 2.4, 5.2.1 Legacy Master Boot Record (MBR) and 5.2.2 OS Types allow to use a pure MBR partition table and to mark the EFI system partition by partition type 0xef. This proposal lets three MBR partitions emerge which cover one ISO and two FAT filesystems. The EFI partition bears the proper type 0xef. It would ease enlarging of the firmware partition if it is number 3 and the EFI partition becomes number 2. It is desirable to have the ISO filesystem in a mountable partition. Because the El Torito boot image efi.img is inside the ISO and thus inside partition 1, this implies the need for an extra copy of the FAT filesystem with file path /efi/boot/bootx64.efi in it. (I will think about El Torito outside of ISO. It is weird but not impossible.) In this case the appended EFI FAT image should be aligned to full MiB, in order to fit into the cylinder ideology of partition editors. The wasted space is not really usable on an MBR partitioned device, because any further partition would begin at a cylinder boundary anyway. If the SYSLINUX MBR template file isohdpfx.bin is in reach then one can omit the run of isohybrid and fdisk and rather let xorriso do the work of partitioning. This works already with xorriso-1.3.2 as of Debian Jessie resp. Debian testing: xorriso -as mkisofs \ -r -J \ -isohybrid-mbr ...maybe.in.$(TEMP_SYSLINUX).../isohdpfx.bin \ -c boot.cat \ -b isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table \ -eltorito-alt-boot \ -e boot/grub/efi.img -no-emul-boot \ -append_partition 2 0xef $(TEMP_CD_TREE)/boot/grub/efi.img \ -append_partition 3 0x01 $(FIRMWARE_VOLUME_FILE) \ -partition_cyl_align all \ -o $(TEMP_MINIISO) $(TEMP_CD_TREE); \ The variable FIRMWARE_VOLUME_FILE needs to be set to the path of the FAT image, which is currently created in util/geniso_hybrid_plus_firmware_partition by mkfs.msdos . The FAT image would have to be created before the xorriso run and removed aftwerwards. Resulting partition layout: System area summary: MBR isohybrid cyl-align-all ISO image size/512 : 45056 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status TypeStart Blocks MBR partition : 1 0x80 0x17045056 MBR partition : 2 0x00 0xef45056 2048 MBR partition : 3 0x00 0x014710412288 With some more waste of space by duplicate ISO and Joliet directory trees, one can achieve an even more conventinal layout where the first partition does not include the MBR: xorriso -as mkisofs \ -partition_offset 16 \ ...all options from previous example... yields: Partition offset : 16 ... MBR partition table: N Status TypeStart Blocks MBR partition : 1 0x80 0x17 6444992 MBR partition :
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, Ian Campbell: Thank you (again?) for all the excellent info you've provided here and in your followups. I hope this does not mean that enough is said on that topic. It seems (IMHO only) that mini.iso is mostly useful for d-i developers to dev test the netboot installer images without needing to rebuild the full netinst images I am currently preparing a conclusion post to this bug, where i emphasize that mini.iso is friendly to partition editors (when on USB stick or alike). The amd64-netinst images are crammed with baits for BIOS and EFI. They violate about any specs and best practice about partitioning. So if the firmware partition on hard-disk-like media has its use cases, then it seems worthwhile to develop a layout for EFI bootability which preserves the feature of mini.iso to look like a sanely partitioned device. But I think most end users would be better off being directed to the proper netinst images instead, I agree as long as only the bootability of the ISO is of interest. An end user trying to edit the partitioning of amd64-netinst will probably get a rough ride. I dimly remember that firmware partitions were used to allow the user to combine a free Debian with proprietary drivers. Something like in https://www.debian.org/releases/stable/i386/ch06s04.html.en At some point it seems like improving mini.iso more for end users would simply be duplicating effort with the CD team. It depends on how large the potential audience for the firmware issue is, and how much the booting OS of mini.iso can make use of the FAT filesystem and its content. The partition was not added just for fun, i guess. Are there traces inside the booting Debian that it looks for such a partition and reads files from it ? Above document says: debian-installer only prompts for firmware needed by kernel modules loaded during the installation. and Any firmware loaded during the installation will be copied automatically to the installed system. - Ok, back to my work in libisofs to enable the production of standard-compliant GPT together with isohybrid and appended partitions. When it is done, i will propose three alternatives: MBR based, GPT based, crammed netinst plus firmware partition. Only the MBR based one can be achieved by released xorriso. Have a nice day :) Thomas -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2869154389342951...@scdbackup.webframe.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
On Tue, 2015-01-27 at 17:03 +0100, Thomas Schmitt wrote: Thank you (again?) for all the excellent info you've provided here and in your followups. I did not guess too bad with my repacking experiment. But this does not explain how the empty FAT partition got to the end of mini.iso I see in http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/config/x86.cfg#n347 geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) So obviously the appended partition was not intended for EFI but for additional firmware. Ah yes, that would seem to make sense. Given that I don't really understand how this additional firmware partition stuff is supposed to fit together (i.e. what impact changing its partition number or adding another VFAT partition etc would have on the code which finds/loads from it) I'm very reluctant to be the one to suggest that we should change things here for Jessie given we've had the beta and are frozen etc. Perhaps we should park this until Stretch? (Kibi, any thoughts on that?) It's not really clear to me who the intended audience of these mini.iso's are: It seems (IMHO only) that mini.iso is mostly useful for d-i developers to dev test the netboot installer images without needing to rebuild the full netinst images (so both legacy and EFI capabilities are useful there). But I think most end users would be better off being directed to the proper netinst images instead, they are more polished and better tested etc (since they share code with the larger ISO images and what have you). At some point it seems like improving mini.iso more for end users would simply be duplicating effort with the CD team. Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1422785352.15317.20.ca...@debian.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
On Tue, Jan 27, 2015 at 05:59:26PM +0100, Thomas Schmitt wrote: Hi, i just realize that the xorriso run in http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/config/x86.cfg does not produce an isohybrid image. Therefore one currently cannot simply omit the isohybrid run in geniso_hybrid_plus_firmware_partition. Option -isohybrid-mbr is a precondition for -isohybrid-gpt-basdat which marks the EFI boot image in MBR and GPT. It needs the disk path to the file isohdpfx.bin in the local SYSLINUX installation (from where file isolinux.bin stems). debian-cd for debian-7*-amd64.iso does: -isohybrid-mbr syslinux/usr/lib/syslinux/isohdpfx.bin (Don't ask me where its pwd is at that moment.) It's in a directory just above the root of the temporary CD tree - see http://anonscm.debian.org/cgit/debian-cd/debian-cd.git/tree/tools/boot/jessie/boot-x86 for context. Just before that point, we've extracted a copy of the syslinux package(s) needed to get all the files we need from them. ... Add an argument to util/geniso_hybrid_plus_firmware_partition + efi_fat=$2 and in the case [ $(GRUB_EFI) = y ] hand it down from build/config/x86.cfg : if [ $(GRUB_EFI) = y ]; then \ xorriso -as mkisofs ... ; \ +geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) \ $(TEMP_CD_TREE)/boot/grub/efi.img ; \ else \ xorriso -as mkisofs ... ; \ +geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) ; \ fi - geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) (Why those backslashes and semicolons in config/x86.cfg, btw ?) It's the syntax for mtools to work on the FAT images, that's all. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with Valor: Centurion represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150201153323.gs8...@einval.com
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
On Tue, Jan 27, 2015 at 03:57:24PM +0100, Thomas Schmitt wrote: Hi, Jack Truong wrote: I have copied the contents of efi.img to the FAT partition and it does boot on my EFI firmware now. Thank you! So your EFI indeed hopped on that partition. Possibly only because it sees no other indication where to start the boot process. I now wonder whether the MBR partition 2 is actually intended as EFI System Partition. The partition type 0x01 generally indicates a FAT12 filesystem. An EFI partition should have type 0xef. Regrettably there is no file /.disk/mkisofs in the ISO which tells the used xorriso -as mkisofs options. Yup. The *netboot* images are produced directly in the debian-installer build, which doesn't have such things. -- Steve McIntyre, Cambridge, UK.st...@einval.com This dress doesn't reverse. -- Alden Spiess -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150201152642.gr8...@einval.com
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, it might be more simple to just append efi.img as additional partition rather than unpacking it into the partition for custom firmware. -- The first step with the new argument efi_fat=$1 in util/geniso_hybrid_plus_firmware_partition and the two alternative calls of geniso_hybrid_plus_firmware_partition from build/config/x86.cfg stays as is. The other two steps get replaced by: -- If $efi_fat is not empty, append it to the image and mark it in MBR. Depending on the fact that the firmware FAT is aligned to full MiB and isohybrid padded up the ISO to full MiB, one could quite duplicate the code of util/geniso_hybrid_plus_firmware_partition from line 37 to end: # Done! echo w ) | fdisk -C $cylinders -H $heads -S $sectors $iso + fdisk_ret=$? + test $fdisk_ret = 0 || exit $fdisk_ret + + test -z $efi_fat exit 0 + + cat $efi_fat $iso + + # Until the bug is fixed: Need no new exaggerated cylinder count + + ( + + # Make new partition #3 + echo n + echo p + echo 3 + echo # use default start sector + echo # use default end sector + + # Set partition type to 0xef : FAT for EFI + echo t + echo 3 + echo 239 + + # Done! + echo w + ) | fdisk -C $cylinders -H $heads -S $sectors $iso -- The computation of cylinders seems wrong in util/geniso_hybrid_plus_firmware_partition: cylinders=$(($(stat -c %s $iso) / $heads / $sectors)) stat -c %s yields the byte count of the ISO image. Cylinder size is $heads * $sectors * 512 bytes. The 512 is missing in above computation. Just try: $ iso=mini.iso $ echo $(($(stat -c %s $iso) / 64 / 32)) 14336 That would be 14+ GiB. More realistic is: $ echo $(($(stat -c %s $iso) / 64 / 32 / 512)) 28 About letting xorriso set up isohybrid and add partitions to MBR and GPT: I am having some conceptional problems in libisofs to get appended partitions marked in GPT. The backup GPT of an EFI-enabled isohybrid is supposed to be inside the block range of the ISO filesystem. The appended partitions are not. So they should come after the backup GPT. But a GPT partition has to be located before the backup GPT. Can you tell me more about the use cases for the firmware partition ? Is it necessary that partition editors can enlarge it ? Have a nice day :) Thomas -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/106925440539388...@scdbackup.webframe.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi Jack, and thanks for your report. Jack Truong jack.tru...@uwaterloo.ca (2015-01-26): Package: cdimage.debian.org Apologies if this is the wrong package. This might be, but don't worry, we'll sort it out. I'm using the jessie rc1 amd64 mini.iso and the EFI partition doesn't seem to have anything in it. It should have efi/boot/bootarch.efi for EFI firmware to load properly. It also doesn't seem to exist in the i386 image either. mini.iso is produced during the build of the debian-installer package, and later installed on mirrors in e.g.: http://ftp.fr.debian.org/debian/dists/testing/main/installer-amd64/20141002/images/netboot/ http://ftp.fr.debian.org/debian/dists/testing/main/installer-amd64/20150107/images/netboot/ I've just grabbed mini.iso in both (respectively used for Beta 2 and RC 1), and both seem to have efi bits inside (at least according to a loop mount). I don't know much about EFI Partition etc. therefore I'm not sure how they're related to an ISO9660 image. :/ It'd be nice to start by telling us the full URL to the mini.iso you downloaded. Mraw, KiBi. signature.asc Description: Digital signature
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
On Tue, 2015-01-27 at 09:15 +0100, Cyril Brulebois wrote: I've just grabbed mini.iso in both (respectively used for Beta 2 and RC 1), and both seem to have efi bits inside (at least according to a loop mount). Not speaking of bootarch.efi but -- there was no useful grub.cfg on the x86 mini.iso until the 20150107 release (see #762618). So 20150107 is certainly necessary for a useful EFI boot from mini.iso and everything was present and correct when I was looking at #762618. It's possible I suppose that the lack of grub.cfg might look a bit like a lack of bootarch.efi under some circumstances. I don't know much about EFI Partition etc. therefore I'm not sure how they're related to an ISO9660 image. :/ (Feel free to skip this bit if you don't care about the details!) The EFI System Partition (ESP) is on the ISO as /boot/grub/efi.img, which is a file with a suitable FAT partition for EFI to read (it's also referenced by various ISO headers, el-torito/*mumble* etc). AFAICT the beta release (20141002) had everything needed apart from the grub.cfg: $ wget http://ftp.fr.debian.org/debian/dists/testing/main/installer-amd64/20141002/images/netboot/mini.iso $ isoinfo -i mini.iso -RJ -x /boot/grub/efi.img efi.img $ mdir -i efi.img ::efi/boot Volume in drive : has no label Volume Serial Number is FF65-0EDB Directory for ::/efi/boot .DIR 2014-09-27 0:08 .. DIR 2014-09-27 0:08 bootx64 efi392192 2014-09-27 0:08 3 files 392 192 bytes 10 240 bytes free $ The remainder of the grub bits are then on the ISO (outside the ESP) in /boot/grub/..., except as noted, that the grub.cfg in 20141002 was empty. $ isoinfo -i mini.iso -fRJ | grep /boot/grub /boot/grub /boot/grub/efi.img /boot/grub/font.pf2 /boot/grub/grub.cfg /boot/grub/x86_64-efi /boot/grub/x86_64-efi/acpi.mod [...lots more...] It'd be nice to start by telling us the full URL to the mini.iso you downloaded. Yes, please. Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1422348131.29309.13.ca...@debian.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, Summary: There are two EFI System Partitions in mini.iso. The one for booting from USB stick or qemu -hdb is indeed empty. One could try whether it helps to copy the files from the FAT image /boot/grub/efi.img in the ISO to MBR partition 2: - Put mini.iso onto a USB stick /dev/sdb or other hard disk device dd if=mini.iso of=/dev/sdb - Mount the ISO image mount -o loop mini.iso /mnt/iso - Mount the FAT filesystem inside /mnt/iso/boot/grub/efi.img mount -o loop /mnt/iso/boot/grub/efi.img /mnt/fat_iso - Mount the FAT filesystem in MBR partition 2 of the USB stick mount /dev/sdb2 /mnt/fat - Copy from efi.img into partition 2 of the USB stick: cp -a /mnt/fat_iso/* /mnt/fat - Check for sucessful copy: find /mnt/fat should yield /mnt/fat /mnt/fat/efi /mnt/fat/efi/boot /mnt/fat/efi/boot/bootx64.efi - Unmount what you mounted: umount /mnt/fat_iso /mnt/fat /mnt/iso - Try whether the stick boots via EFI. --- Details: Cyril Brulebois wrote: I don't know much about EFI Partition etc. therefore I'm not sure how they're related to an ISO9660 image. :/ mini.iso and debian-7*-amd64-netinst.iso differ in this aspect. In Debian amd64 netinst, the EFI System Partition is a data file inside the ISO filesystem which contains a FAT filesystem image. In this FAT image there are the files which EFI will use for booting. Debian has GRUB2 boot loader equipment in there. The block address and size of the EFI partition is published by an entry in the El Torito boot catalog, by a partition table entry in the MBR, by a partition table entry in GPT, and by a (probably useless) entry in Apple Partition Map. Inspecting mini.iso http://ftp.fr.debian.org/debian/dists/testing/main/installer-amd64/20150107/images/netboot/mini.iso by xorriso-1.3.8 yields: Volume id: 'ISOIMAGE' El Torito catalog : 59 1 El Torito cat path : /boot.cat El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x 0x00 48959 El Torito boot img : 2 UEFI y none 0x 0x008329205 El Torito img path : 1 /isolinux.bin El Torito img opts : 1 boot-info-table isohybrid-suitable El Torito img path : 2 /boot/grub/efi.img System area options: 0x0302 System area summary: MBR isohybrid cyl-align-all ISO image size/512 : 44332 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status TypeStart Blocks MBR partition : 1 0x80 0x17045056 MBR partition : 2 0x00 0x014505612288 So we have no GPT and no (probably useless) APM here. There are two EFI partitions exposed. One is the data file /boot/grub/efi.img of the ISO filesystem pointed to by the El Torito boot catalog. The other is an appended blob, marked by MBR partition 2. (The partition type 0x01 is questionable.) One can extract them by: mount -o loop mini.iso /mnt/iso cp /mnt/iso/boot/grub/efi.img eltorito_efi.img dd if=mini.iso bs=512 skip=45056 count=12288 of=mbr_efi.img Both files differ heavily in size -r--r--r-- 1 thomas thomas 425984 2015-01-27 09:59 eltorito_efi.img -rw-r--r-- 1 thomas thomas 6291456 2015-01-27 10:00 mbr_efi.img Inspecting both: $ mount -o loop eltorito_efi.img /mnt/fat $ find /mnt/fat /mnt/fat /mnt/fat/efi /mnt/fat/efi/boot /mnt/fat/efi/boot/bootx64.efi $ umount /mnt/fat $ mount -o loop mbr_efi.img /mnt/fat $ find /mnt/fat /mnt/fat $ So the El Torito advertised partition contains files, whereas the MBR advertised partition has an empty filesystem. Probably mini.iso will boot via EFI from CD/DVD/BD resp. qemu -cdrom, but not from USB stick resp. qemu -hdb. The presence of an isohybrid MBR will allow booting via BIOS emulation. --- For comparison the boot anatomy of a amd64 netinst ISO: $ xorriso-1.3.8 -indev debian-7.7.0-amd64-netinst.iso \ -report_el_torito plain -report_system_area plain ... Volume id: 'Debian 7.7.0 amd64 1' El Torito catalog : 847 1 El Torito cat path : /isolinux/boot.cat El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x 0x00 4 848 El Torito boot img : 2 UEFI y none 0x 0x00896 860 El Torito img path : 1 /isolinux/isolinux.bin El Torito img opts : 1 boot-info-table isohybrid-suitable El Torito img path : 2 /boot/grub/efi.img System area options: 0x0102 System area summary: MBR isohybrid cyl-align-on GPT APM ISO image size/512 : 454656 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status TypeStart Blocks MBR partition : 1
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi all, The ISO in question was in fact the same one mentioned (http://ftp.fr.debian.org/debian/dists/testing/main/installer-amd64/20150107/images/netboot/mini.iso). I have copied the contents of efi.img to the FAT partition and it does boot on my EFI firmware now. Thank you! On 01/27/2015 04:47 AM, Thomas Schmitt wrote: Hi, Summary: There are two EFI System Partitions in mini.iso. The one for booting from USB stick or qemu -hdb is indeed empty. One could try whether it helps to copy the files from the FAT image /boot/grub/efi.img in the ISO to MBR partition 2: - Put mini.iso onto a USB stick /dev/sdb or other hard disk device dd if=mini.iso of=/dev/sdb - Mount the ISO image mount -o loop mini.iso /mnt/iso - Mount the FAT filesystem inside /mnt/iso/boot/grub/efi.img mount -o loop /mnt/iso/boot/grub/efi.img /mnt/fat_iso - Mount the FAT filesystem in MBR partition 2 of the USB stick mount /dev/sdb2 /mnt/fat - Copy from efi.img into partition 2 of the USB stick: cp -a /mnt/fat_iso/* /mnt/fat - Check for sucessful copy: find /mnt/fat should yield /mnt/fat /mnt/fat/efi /mnt/fat/efi/boot /mnt/fat/efi/boot/bootx64.efi - Unmount what you mounted: umount /mnt/fat_iso /mnt/fat /mnt/iso - Try whether the stick boots via EFI. --- Details: Cyril Brulebois wrote: I don't know much about EFI Partition etc. therefore I'm not sure how they're related to an ISO9660 image. :/ mini.iso and debian-7*-amd64-netinst.iso differ in this aspect. In Debian amd64 netinst, the EFI System Partition is a data file inside the ISO filesystem which contains a FAT filesystem image. In this FAT image there are the files which EFI will use for booting. Debian has GRUB2 boot loader equipment in there. The block address and size of the EFI partition is published by an entry in the El Torito boot catalog, by a partition table entry in the MBR, by a partition table entry in GPT, and by a (probably useless) entry in Apple Partition Map. Inspecting mini.iso http://ftp.fr.debian.org/debian/dists/testing/main/installer-amd64/20150107/images/netboot/mini.iso by xorriso-1.3.8 yields: Volume id: 'ISOIMAGE' El Torito catalog : 59 1 El Torito cat path : /boot.cat El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x 0x00 48959 El Torito boot img : 2 UEFI y none 0x 0x008329205 El Torito img path : 1 /isolinux.bin El Torito img opts : 1 boot-info-table isohybrid-suitable El Torito img path : 2 /boot/grub/efi.img System area options: 0x0302 System area summary: MBR isohybrid cyl-align-all ISO image size/512 : 44332 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status TypeStart Blocks MBR partition : 1 0x80 0x17045056 MBR partition : 2 0x00 0x014505612288 So we have no GPT and no (probably useless) APM here. There are two EFI partitions exposed. One is the data file /boot/grub/efi.img of the ISO filesystem pointed to by the El Torito boot catalog. The other is an appended blob, marked by MBR partition 2. (The partition type 0x01 is questionable.) One can extract them by: mount -o loop mini.iso /mnt/iso cp /mnt/iso/boot/grub/efi.img eltorito_efi.img dd if=mini.iso bs=512 skip=45056 count=12288 of=mbr_efi.img Both files differ heavily in size -r--r--r-- 1 thomas thomas 425984 2015-01-27 09:59 eltorito_efi.img -rw-r--r-- 1 thomas thomas 6291456 2015-01-27 10:00 mbr_efi.img Inspecting both: $ mount -o loop eltorito_efi.img /mnt/fat $ find /mnt/fat /mnt/fat /mnt/fat/efi /mnt/fat/efi/boot /mnt/fat/efi/boot/bootx64.efi $ umount /mnt/fat $ mount -o loop mbr_efi.img /mnt/fat $ find /mnt/fat /mnt/fat $ So the El Torito advertised partition contains files, whereas the MBR advertised partition has an empty filesystem. Probably mini.iso will boot via EFI from CD/DVD/BD resp. qemu -cdrom, but not from USB stick resp. qemu -hdb. The presence of an isohybrid MBR will allow booting via BIOS emulation. --- For comparison the boot anatomy of a amd64 netinst ISO: $ xorriso-1.3.8 -indev debian-7.7.0-amd64-netinst.iso \ -report_el_torito plain -report_system_area plain ... Volume id: 'Debian 7.7.0 amd64 1' El Torito catalog : 847 1 El Torito cat path : /isolinux/boot.cat El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x 0x00 4 848 El Torito boot img : 2 UEFI y none 0x 0x00896 860 El Torito img path : 1 /isolinux/isolinux.bin El Torito img
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, Jack Truong wrote: I have copied the contents of efi.img to the FAT partition and it does boot on my EFI firmware now. Thank you! So your EFI indeed hopped on that partition. Possibly only because it sees no other indication where to start the boot process. I now wonder whether the MBR partition 2 is actually intended as EFI System Partition. The partition type 0x01 generally indicates a FAT12 filesystem. An EFI partition should have type 0xef. Regrettably there is no file /.disk/mkisofs in the ISO which tells the used xorriso -as mkisofs options. I guess that /boot/grub/efi.img got marked by an option like -e boot/grub/efi.img To mark this file in MBR and GPT, one may add option -isohybrid-gpt-basdat directly after -e and its argument. I tested by extracting the MBR and partition 2 from mini.iso, mounting it, and repacking a new mini_with_gpt.iso by: $ dd if=mini.iso bs=512 count=1 of=mini_template.mbr $ dd if=mini.iso bs=512 skip=45056 count=12288 of=mini_partition2.img $ mount -o loop mini.iso /mnt/iso $ xorriso -as mkisofs -R \ -o mini_with_gpt.iso \ -isohybrid-mbr mini_template.mbr \ -c '/boot.cat' \ -b '/isolinux.bin' \ -no-emul-boot -boot-load-size 4 -boot-info-table \ -eltorito-alt-boot \ -e '/boot/grub/efi.img' \ -isohybrid-gpt-basdat -no-emul-boot \ -append_partition 3 0x01 mini_partition2.img \ /mnt/iso This yields: Volume id: 'ISOIMAGE' El Torito catalog : 45 1 El Torito cat path : /boot.catalog El Torito images : N Pltf B Emul Ld_seg Hdpt Ldsiz LBA El Torito boot img : 1 BIOS y none 0x 0x00 4 46 El Torito boot img : 2 UEFI y none 0x 0x00832 66 El Torito img path : 1 /isolinux.bin El Torito img opts : 1 boot-info-table isohybrid-suitable El Torito img path : 2 /boot/grub/efi.img System area options: 0x0102 System area summary: MBR isohybrid cyl-align-on GPT ISO image size/512 : 45056 Partition offset : 0 MBR heads per cyl : 64 MBR secs per head : 32 MBR partition table: N Status TypeStart Blocks MBR partition : 1 0x80 0x00045056 MBR partition : 2 0x00 0xef 264 832 MBR partition : 3 0x00 0x014505612288 MBR partition path : 2 /boot/grub/efi.img GPT: N Info GPT disk GUID : 61bf3bd0595c364488002bde6f8fecfb GPT entry array: 2 248 overlapping GPT lba range : 64 44992 45055 GPT partition name : 1 490053004f00480079006200720069006400 GPT partname local : 1 ISOHybrid GPT partition GUID : 1 61bf3bd0595c364488022bde6f8fecfb GPT type GUID : 1 a2a0d0ebe5b9334487c068b6b72699c7 GPT partition flags: 1 0x1001 GPT start and size : 1 0 44992 GPT partition name : 2 490053004f004800790062007200690064003100 GPT partname local : 2 ISOHybrid1 GPT partition GUID : 2 61bf3bd0595c364488032bde6f8fecfb GPT type GUID : 2 a2a0d0ebe5b9334487c068b6b72699c7 GPT partition flags: 2 0x1001 GPT start and size : 2 264 832 GPT partition path : 2 /boot/grub/efi.img Now the empty FAT12 filesystem is in MBR partition 3. It is unclear whether it is usiable on a system that knows about GPT. (Maybe i should enhance -append_partition so that it publishes the partition in GPT too.) Of course, if the empty 6 MB partition is undesired in a 22.5 MB ISO, then one can omit option -append_partition 3 0x01 mini_partition2.img \ from above xorriso run. If one wants to eradicate GPT and only keep MBR: $ cp mini_with_gpt.iso mini_with_mbr_efi.iso $ dd if=/dev/zero bs=512 count=1 seek=1 conv=notrunc of=mini_with_mbr_efi.iso $ dd if=/dev/zero bs=512 count=1 seek=45055 conv=notrunc of=mini_with_mbr_efi.iso (The backup GPT header is the last 512 block of the ISO filesystem range. Probably the appended partition data prevent it from being recognized at all.) Have a nice day :) Thomas -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/29399543329384549...@scdbackup.webframe.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
On Tue, 2015-01-27 at 15:57 +0100, Thomas Schmitt wrote: [...] Thanks for all the excellent information, I still have fully absorbed it all... Regrettably there is no file /.disk/mkisofs in the ISO which tells the used xorriso -as mkisofs options. I guess that /boot/grub/efi.img got marked by an option like The code is at: http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/config/x86.cfg#n335 and has: xorriso -as mkisofs -r -J -b isolinux.bin -c boot.cat \ -no-emul-boot -boot-load-size 4 -boot-info-table \ -eltorito-alt-boot \ --efi-boot boot/grub/efi.img -no-emul-boot \ -o $(TEMP_MINIISO) $(TEMP_CD_TREE); \ -e boot/grub/efi.img I guess -e is short for --efi-boot? To mark this file in MBR and GPT, one may add option -isohybrid-gpt-basdat directly after -e and its argument. I don't suppose I can tempt you into sending a patch against the above git repo, can I? ;-) Ian. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1422371143.25294.40.ca...@debian.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, i just realize that the xorriso run in http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/config/x86.cfg does not produce an isohybrid image. Therefore one currently cannot simply omit the isohybrid run in geniso_hybrid_plus_firmware_partition. Option -isohybrid-mbr is a precondition for -isohybrid-gpt-basdat which marks the EFI boot image in MBR and GPT. It needs the disk path to the file isohdpfx.bin in the local SYSLINUX installation (from where file isolinux.bin stems). debian-cd for debian-7*-amd64.iso does: -isohybrid-mbr syslinux/usr/lib/syslinux/isohdpfx.bin (Don't ask me where its pwd is at that moment.) --- So the small improvement without GPT becomes more attractive: -- Add an argument to util/geniso_hybrid_plus_firmware_partition + efi_fat=$2 and in the case [ $(GRUB_EFI) = y ] hand it down from build/config/x86.cfg : if [ $(GRUB_EFI) = y ]; then \ xorriso -as mkisofs ... ; \ +geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) \ $(TEMP_CD_TREE)/boot/grub/efi.img ; \ else \ xorriso -as mkisofs ... ; \ +geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) ; \ fi - geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) (Why those backslashes and semicolons in config/x86.cfg, btw ?) -- If $efi_fat is not empty, (m)copy the directories and files from $efi_fat to $firmware_volume_name after that one was created by mkfs.msdos. (mtools expert wanted.) -- In the fdisk run at the end of util/geniso_hybrid_plus_firmware_partition - # Pedantically, set partition type to 1: FAT 16 + # Set partition type to 0xef: FAT for EFI echo t echo 2 - echo 1 + echo 239 --- Now, who wants to set up a debian-installer, make the proposed code changes, and produce a mini.iso which i can inspect ? Have a nice day :) Thomas -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/25222543353042887...@scdbackup.webframe.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Hi, xorriso -as mkisofs -r -J -b isolinux.bin -c boot.cat \ -no-emul-boot -boot-load-size 4 -boot-info-table \ -eltorito-alt-boot \ --efi-boot boot/grub/efi.img -no-emul-boot \ -o $(TEMP_MINIISO) $(TEMP_CD_TREE); \ I did not guess too bad with my repacking experiment. But this does not explain how the empty FAT partition got to the end of mini.iso I see in http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/config/x86.cfg#n347 geniso_hybrid_plus_firmware_partition $(TEMP_MINIISO) So obviously the appended partition was not intended for EFI but for additional firmware. Now if mini.iso gets GPT, then http://anonscm.debian.org/cgit/d-i/debian-installer.git/tree/build/util/geniso_hybrid_plus_firmware_partition should add its partition to GPT too. At its end, it uses fdisk to manipulate the MBR partition table. Number 2 would have to become 3 then, because slot 2 is already occupied by /boot/grub/efi.img. I guess that gdisk can do similar for GPT. Problem is that the GPT backup block needs to be moved after the end of the partition data which was appended in line 37: cat $firmware_volume_file $iso I dimly remember that gdisk has a menu item for moving the backup GPT and/or for re-creating the backup GPT. The backup GPT would actually have to be re-located whenever the ISO image gets copied onto a disk device. It might be that gdisk or other partition editors go on strike when they see the nested partitions. In this case, we might need to write MBR and GPT slots on byte level. (A re-write to geniso_hybrid_plus_firmware_partition.c would be a good idea then. Byte manipulation in shell is cumbersome.) It is a bad idea to apply the SYSLINUX program isohybrid to an ISO which was equipped with advanced isohybrid by xorriso. So one should delete line 26: isohybrid -h $heads -s $sectors $iso I guess -e is short for --efi-boot? --efi-boot XYZ does a bit more: -eltorito-alt-boot -e XYX -no-emul-boot -eltorito-alt-boot Since i propose to apply -isohybrid-gpt-basdat, one would have to switch to -e . E.g.: --eltorito-alt-boot \ ---efi-boot boot/grub/efi.img -no-emul-boot \ +-eltorito-alt-boot \ +-e boot/grub/efi.img \ +-no-emul-boot -isohybrid-gpt-basdat \ -e is quite like -b, including the inappropriate default to mark the boot image in El Torito as emulated floppy disk. Therefore option -no-emul-boot must be given. I don't suppose I can tempt you into sending a patch against the above git repo, can I? ;-) Seems we are not done with development yet. There is the problem that i cannot easily test because i have no EFI and not even a contemporary Debian. If an authorized developer can set up a suitable environment for production of mini.iso, i will help to adapt the scripts. We will have to make decisions. E.g. whether to keep a BIOS-only version of mini.iso and thus cannot hardcode the partition number in geniso_hybrid_plus_firmware_partition. Some experience with gdisk would be helpful. (I guess my Debian 6 test system can give me hints.) - Alternatively one could really copy the directories and boot program from /boot/grub/efi.img into the appended partition and give it the type 0xef. Having no GPT would avoid problems with the appended partition. But it might be that some EFI implementations will not recognize the MBR partition and insist in GPT. Only tests can tell. Have a nice day :) Thomas -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/661054846588...@scdbackup.webframe.org
Bug#776317: Jessie RC1 amd64 mini image missing efi bootloader
Package: cdimage.debian.org Apologies if this is the wrong package. I'm using the jessie rc1 amd64 mini.iso and the EFI partition doesn't seem to have anything in it. It should have efi/boot/bootarch.efi for EFI firmware to load properly. It also doesn't seem to exist in the i386 image either. I'm using a Minnowboard MAX to test the image via USB booting. -- Jack Truong IT Specialist @ Engineering Computing University of Waterloo (PHY-3019 x35147) http://jacktruong.net/ -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/54c68dd0.2000...@uwaterloo.ca
