Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Michel Bouissou wrote:
> I'm happy to report that the USB stick I made today with
> grub-mkrescue-sed.sh does boot alright on tested machines.

So GRUB itself is ok with the machine. GPT and MBR partitions alike.


> The HP tested machine has, AFAIK, no CSM "legacy" boot mode at all and I
> believe it doesn't boot anything but UEFI.

It booted the tails ISO which has no EFI boot equipment but only
an ISOLINUX El Torito boot image for BIOS and an ISOLINUX isohybrid MBR
with machine code which BIOS will execute when presented on USB stick.

The machine's firmware probably falls back to legacy mode when detecting
BIOS stuff but no EFI partition.

-

What only do our users at the distro ISO projects miss which grub-mkrescue
does right ?

Is it simply the invalid GPT which Matthew J. Garrett introduced
to get some of his test machines booting ? (Or did he introduce the MBR
partition without noticing that this invalidates GPT ?)
See
  "Anatomy of a Fedora 17 ISO image"
  https://mjg59.dreamwidth.org/11285.html
preceeded by
  "Further adventures in EFI booting"
  https://mjg59.dreamwidth.org/4957.html
The EFI images, which he and nearly all others use, are made with GRUB2.

grub-mkrescue does a similar stunt about x86 machine code without
side effects and the Block0 of an Apple Partition Map. But it does not
combine a non-protective MBR with GPT.

Michel - on his way to India - could zeroize 512-byte block 1 of the
Debian Live ISO and then try to boot it from USB stick

  dd if=/dev/zero conv=notrunc bs=512 seek=1 count=1 of=...image.or.stick...

where "...image.or.stick..." mabe either something like debian-live-*.iso
or /dev/sdc.
This would further deface the GPT and possibly de-confuse the firmware.

Michel, i will try to remember to remind you, when you are back. :))

-
The partition tables of modified and vanilla grub-mkrescue ISOs:

This is the modified one from grub-mkrescue-sed.sh :

> root@zafu:/# xorriso -indev stdio:/dev/sdb -report_el_torito plain 
> -report_system_area plain
> ...
> El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz LBA
> El Torito boot img :   1  BIOS  y   none  0x  0x00  41487
> El Torito boot img :   2  UEFI  y   none  0x  0x00   57607154
> El Torito img path :   1  /boot/grub/i386-pc/eltorito.img
> El Torito img opts :   1  boot-info-table grub2-boot-info

We see that it's with EFI equipment.
(LBA counts blocks of 2048 bytes.  Ldsiz counts blocks of 512 bytes.
 Blame IBM and Phoenix, not me.)


> MBR partition table:   N Status  TypeStart   Blocks
> MBR partition  :   1   0x80  0x00   6428552
> MBR partition  :   2   0x00  0xef28616 5760

(Here all block numbers are counted with 512 bytes per block.)

By the options manipulation of grub-mkrescue-sed.sh the EFI partition
image was appended after the mountable ISO 9660 partition. Other than
with vanilla grub-mkrescue both partitions are marked in the MBR partition
table.
Partition 1 has the Boot/Active flag set, because some firmwares ignore
media which do not have it on any partition.
Partition 2 is the EFI partition. The same bytes as the El Torito UEFI
boot image. (7154 * 2048 / 512 = 28616)

Vanilla grub-mkrescue would rather look like:

  El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz LBA
  El Torito boot img :   1  BIOS  y   none  0x  0x00  44047
  El Torito boot img :   2  UEFI  y   none  0x  0x00   5760  85
  El Torito img path :   1  /boot/grub/i386-pc/eltorito.img
  El Torito img opts :   1  boot-info-table grub2-boot-info
  El Torito img path :   2  /efi.img
  ...
  MBR partition table:   N Status  TypeStart   Blocks
  MBR partition  :   1   0x00  0xee134779
  GPT:   N  Info
  GPT disk GUID  :  04a5adf35d1adb4382bf8300bebe08a1
  GPT entry array:  20  176  separated
  GPT lba range  :  64  34734  34779
  GPT partition name :   1  4700610070003000
  GPT partname local :   1  Gap0
  ...
  GPT partition name :   2  
450046004900200062006f006f007400200070006100720074006900740069006f006e00
  GPT partname local :   2  EFI boot partition
  GPT partition GUID :   2  04a5adf35d1adb4382bd8300bebe08a1
  GPT type GUID  :   2  28732ac11ff8d211ba4b00a0c93ec93b
  GPT partition flags:   2  0x1001
  GPT start and size :   2  340  5760
  GPT partition path :   2  /efi.img
  ...
  GPT start and size :   4  34132  600

plus maybe

  APM:   N  Info
  APM block size :  2048
  ...

Other than with the mbr_only version, the EFI partition is a data file
inside the ISO 9660 filesystem: /efi.img

247 lines of explanation of the report format are available with
  xorriso -report_system_area help -report_el_torito help | less


Have a nice 

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi,

Le 31/01/2018 à 22:37, Thomas Schmitt a écrit :
> 
> In this case a run with grub-mkrescue-sed.sh will not make much sense.
> First one will have to configure GRUB to enable at least one of the EFI
> variants. I dimly remember that the machine was 64 bit, i.e. should run
> bootx64.efi.

I'm happy to report that the USB stick I made today with
grub-mkrescue-sed.sh does boot alright on tested machines.

As a side note, the machine on wich I made it is a Manjaro that boots in
UEFI 64 mode, so for sure it's grub is UEFI-capable.

The HP tested machine has, AFAIK, no CSM "legacy" boot mode at all and I
believe it doesn't boot anything but UEFI.

That said, I don't have the machine on which I made the stick at hand
and I made the ISO image in a tmpfs, so...

But directly analyzing the USB stick I made, on another (Arch) machine
now gives :


root@zafu:/# xorriso -indev stdio:/dev/sdb -report_el_torito plain
-report_system_area plain
xorriso 1.4.8 : RockRidge filesystem manipulator, libburnia project.

xorriso : NOTE : Loading ISO image tree from LBA 0
xorriso : UPDATE : 632 nodes read in 1 seconds
libisofs: WARNING : Found hidden El-Torito image. Its size could not be
figured out, so image modify or boot image patching may lead to bad results.
xorriso : NOTE : Detected El-Torito boot information which currently is
set to be discarded
Drive current: -indev 'stdio:/dev/sdb'
Media current: stdio file, overwriteable
Media status : is written , is appendable
Boot record  : El Torito , MBR grub2-mbr cyl-align-off
Media summary: 1 session, 8594 data blocks, 16.8m data, 14.8g free
Volume id: 'ISOIMAGE'
El Torito catalog  : 141  1
El Torito cat path : /boot.catalog
El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz LBA
El Torito boot img :   1  BIOS  y   none  0x  0x00  41487
El Torito boot img :   2  UEFI  y   none  0x  0x00   57607154
El Torito img path :   1  /boot/grub/i386-pc/eltorito.img
El Torito img opts :   1  boot-info-table grub2-boot-info
El Torito img blks :   2  1440
System area options: 0x4a00
System area summary: MBR grub2-mbr cyl-align-off
ISO image size/512 : 34376
Partition offset   : 16
MBR heads per cyl  : 64
MBR secs per head  : 32
MBR partition table:   N Status  TypeStart   Blocks
MBR partition  :   1   0x80  0x00   6428552
MBR partition  :   2   0x00  0xef28616 5760


Hope this helps (although I don't get much out of this myself ;-)

ॐ

-- 
Michel Bouissou  OpenPGP ID 0xEB04D09C

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH] Make grub-install check for errors from efibootmgr

[Steve McIntyre]

Code is currently ignoring errors from efibootmgr, giving users
clearly bogus output like:

Setting up grub-efi-amd64 (2.02~beta3-4) ...
Installing for x86_64-efi platform.
Could not delete variable: No space left on device
Could not prepare Boot variable: No space left on device
Installation finished. No error reported.

and then potentially unbootable systems. If efibootmgr fails,
grub-install should know that and report it!

We've been using this patch in Debian now for some time, with no ill
effects.

Signed-off-by: Steve McIntyre <93...@debian.org>
---
 grub-core/osdep/unix/platform.c | 24 +++-
 include/grub/util/install.h |  2 +-
 util/grub-install.c | 18 +-
 3 files changed, 29 insertions(+), 15 deletions(-)

diff --git a/grub-core/osdep/unix/platform.c b/grub-core/osdep/unix/platform.c
index a3fcfcaca..ca448bc11 100644
--- a/grub-core/osdep/unix/platform.c
+++ b/grub-core/osdep/unix/platform.c
@@ -78,19 +78,20 @@ get_ofpathname (const char *dev)
   dev);
 }
 
-static void
+static int
 grub_install_remove_efi_entries_by_distributor (const char *efi_distributor)
 {
   int fd;
   pid_t pid = grub_util_exec_pipe ((const char * []){ "efibootmgr", NULL }, 
);
   char *line = NULL;
   size_t len = 0;
+  int rc;
 
   if (!pid)
 {
   grub_util_warn (_("Unable to open stream from %s: %s"),
  "efibootmgr", strerror (errno));
-  return;
+  return errno;
 }
 
   FILE *fp = fdopen (fd, "r");
@@ -98,7 +99,7 @@ grub_install_remove_efi_entries_by_distributor (const char 
*efi_distributor)
 {
   grub_util_warn (_("Unable to open stream from %s: %s"),
  "efibootmgr", strerror (errno));
-  return;
+  return errno;
 }
 
   line = xmalloc (80);
@@ -119,23 +120,25 @@ grub_install_remove_efi_entries_by_distributor (const 
char *efi_distributor)
   bootnum = line + sizeof ("Boot") - 1;
   bootnum[4] = '\0';
   if (!verbosity)
-   grub_util_exec ((const char * []){ "efibootmgr", "-q",
+   rc = grub_util_exec ((const char * []){ "efibootmgr", "-q",
  "-b", bootnum,  "-B", NULL });
   else
-   grub_util_exec ((const char * []){ "efibootmgr",
+   rc = grub_util_exec ((const char * []){ "efibootmgr",
  "-b", bootnum, "-B", NULL });
 }
 
   free (line);
+  return rc;
 }
 
-void
+int
 grub_install_register_efi (grub_device_t efidir_grub_dev,
   const char *efifile_path,
   const char *efi_distributor)
 {
   const char * efidir_disk;
   int efidir_part;
+  int ret;
   efidir_disk = grub_util_biosdisk_get_osdev (efidir_grub_dev->disk);
   efidir_part = efidir_grub_dev->disk->partition ? 
efidir_grub_dev->disk->partition->number + 1 : 1;
 
@@ -151,23 +154,26 @@ grub_install_register_efi (grub_device_t efidir_grub_dev,
   grub_util_exec ((const char * []){ "modprobe", "-q", "efivars", NULL });
 #endif
   /* Delete old entries from the same distributor.  */
-  grub_install_remove_efi_entries_by_distributor (efi_distributor);
+  ret = grub_install_remove_efi_entries_by_distributor (efi_distributor);
+  if (ret)
+return ret;
 
   char *efidir_part_str = xasprintf ("%d", efidir_part);
 
   if (!verbosity)
-grub_util_exec ((const char * []){ "efibootmgr", "-q",
+ret = grub_util_exec ((const char * []){ "efibootmgr", "-q",
  "-c", "-d", efidir_disk,
  "-p", efidir_part_str, "-w",
  "-L", efi_distributor, "-l", 
  efifile_path, NULL });
   else
-grub_util_exec ((const char * []){ "efibootmgr",
+ret = grub_util_exec ((const char * []){ "efibootmgr",
  "-c", "-d", efidir_disk,
  "-p", efidir_part_str, "-w",
  "-L", efi_distributor, "-l", 
  efifile_path, NULL });
   free (efidir_part_str);
+  return ret;
 }
 
 void
diff --git a/include/grub/util/install.h b/include/grub/util/install.h
index 5910b0c09..0dba8b67f 100644
--- a/include/grub/util/install.h
+++ b/include/grub/util/install.h
@@ -210,7 +210,7 @@ grub_install_create_envblk_file (const char *name);
 const char *
 grub_install_get_default_x86_platform (void);
 
-void
+int
 grub_install_register_efi (grub_device_t efidir_grub_dev,
   const char *efifile_path,
   const char *efi_distributor);
diff --git a/util/grub-install.c b/util/grub-install.c
index 5e4cdfd2b..690f180c5 100644
--- a/util/grub-install.c
+++ b/util/grub-install.c
@@ -1848,9 +1848,13 @@ main (int argc, char *argv[])
  if (!removable && update_nvram)
{
  /* Try to make this image bootable using the EFI Boot Manager, if 
available.  */
- grub_install_register_efi (efidir_grub_dev,
-"\\System\\Library\\CoreServices",
-efi_distributor);
+ int ret;
+ ret = 

Re: [PATCH] Make grub-install check for errors from efibootmgr

[Steve McIntyre]

On Tue, Jan 30, 2018 at 06:44:05PM +0100, Daniel Kiper wrote:
>On Mon, Jan 29, 2018 at 06:54:23PM +, Steve McIntyre wrote:
>>
>> diff --git a/grub-core/osdep/unix/platform.c 
>> b/grub-core/osdep/unix/platform.c
>> index a3fcfcaca..b3a617e44 100644
>> --- a/grub-core/osdep/unix/platform.c
>> +++ b/grub-core/osdep/unix/platform.c
>> @@ -78,19 +78,20 @@ get_ofpathname (const char *dev)
>> dev);
>>  }
>>
>> -static void
>> +static int
>>  grub_install_remove_efi_entries_by_distributor (const char *efi_distributor)
>>  {
>>int fd;
>>pid_t pid = grub_util_exec_pipe ((const char * []){ "efibootmgr", NULL }, 
>> );
>>char *line = NULL;
>>size_t len = 0;
>> +  int ret;
>>
>>if (!pid)
>>  {
>>grub_util_warn (_("Unable to open stream from %s: %s"),
>>"efibootmgr", strerror (errno));
>> -  return;
>> +  return errno;
>>  }
>>
>>FILE *fp = fdopen (fd, "r");
>> @@ -98,14 +99,13 @@ grub_install_remove_efi_entries_by_distributor (const 
>> char *efi_distributor)
>>  {
>>grub_util_warn (_("Unable to open stream from %s: %s"),
>>"efibootmgr", strerror (errno));
>> -  return;
>> +  return errno;
>>  }
>>
>>line = xmalloc (80);
>>len = 80;
>>while (1)
>>  {
>> -  int ret;
>
>Oh, no, please do not do that here. If my first proposal conflicts with
>existing variables use second one. If second is bad please choose third.
>The rest of the patch LGTM.

OK, no problem. New version on its way in a sec.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
You raise the blade, you make the change... You re-arrange me 'til I'm sane...


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

it comes to me that possibly Michel's grub-mkrescue run was BIOS-only.
So Michel, take much care to come back in good shape. We have experiments
to do. :))

---

At least on Debian the boot equipment prepared by grub-mkrescue depends
on which grub-* Debian packages are installed.

"grub-pc" enables BIOS equipment. In the case of an USB stick, it's the
MBR x86 boot code which comes into effect. It knows the block address of
the El Torito boot image. So i assume it loads and executes that binary.

"grub-efi-amd64" causes grub-mkrescue to prepare an EFI System Partition
with binary /efi/boot/bootx64.efi . 

"grub-efi-ia32" causes an EFI System Partition with binary
/efi/boot/bootia32.efi .

All three can be combined.

So after a vanilla run with grub-mkrescue, it is essential to inspect
the partition tables and El Torito boot equipment by

  xorriso -indev output.iso \
  -report_el_torito plain -report_system_area plain

If no lines like
  GPT disk GUID  :  04a5adf35d1adb4382bf8300bebe08a1
  ...
  GPT start and size :   4  34132  600
are reported, then the ISO has no EFI System Partition.

In this case a run with grub-mkrescue-sed.sh will not make much sense.
First one will have to configure GRUB to enable at least one of the EFI
variants. I dimly remember that the machine was 64 bit, i.e. should run
bootx64.efi.


Have a nice day :)

Thomas


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 2/2] .mod files: Strip annobin annotations and .eh_frame, and their relocations

[Peter Jones]

This way debuginfo built from the .module will still include this
information, but the final result won't have the data we don't actually
need in the modules, either on-disk, loaded at runtime, or in prebuilt
images.

Signed-off-by: Peter Jones 
---
 grub-core/genmod.sh.in | 4 
 1 file changed, 4 insertions(+)

diff --git a/grub-core/genmod.sh.in b/grub-core/genmod.sh.in
index 3de06ee018f..1250589b3f5 100644
--- a/grub-core/genmod.sh.in
+++ b/grub-core/genmod.sh.in
@@ -58,6 +58,10 @@ if test x@TARGET_APPLE_LINKER@ != x1; then
-K grub_mod_init -K grub_mod_fini \
-K _grub_mod_init -K _grub_mod_fini \
-R .note.gnu.gold-version -R .note.GNU-stack \
+   -R .gnu.build.attributes \
+   -R .rel.gnu.build.attributes \
+   -R .rela.gnu.build.attributes \
+   -R .eh_frame -R .rela.eh_frame -R .rel.eh_frame \
-R .note -R .comment -R .ARM.exidx $tmpfile || exit 1
fi
if ! test -z "${TARGET_OBJ2ELF}"; then
-- 
2.15.0


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 1/2] mkimage: avoid copying relocations for sections that won't be copied.

[Peter Jones]

Some versions of gcc include a plugin called "annobin", and in some
build systems this is enabled by default.  This plugin creates special
ELF note sections to track which ABI-breaking features are used by a
binary, as well as a series of relocations to annotate where.

If grub is compiled with this feature, then when grub-mkimage translates
the binary to another file format which does not strongly associate
relocation data with sections (i.e. when platform is *-efi), these
relocations appear to be against the .text section rather than the
original note section.  When the binary is loaded by the PE runtime
loader, hilarity ensues.

This issue is not necessarily limited to the annobin, but could arise
any time there are relocations in sections that are not represented in
grub-mkimage's output.

This patch seeks to avoid this issue by only including relocations that
refer to sections which will be included in the final binary.

As an aside, this should also obviate the need to avoid -funwind-tables,
-fasynchronous-unwind-tables, and any sections similar to .eh_frame in
the future.  I've tested it on x86-64-efi with the following gcc command
line options (as recorded by -grecord-gcc-flags), but I still need to
test the result on some other platforms that have been problematic in
the past (especially ARM Aarch64) before I feel comfortable making
changes to the configure.ac bits:

GNU C11 7.2.1 20180116 (Red Hat 7.2.1-7) -mno-mmx -mno-sse -mno-sse2 -mno-sse3 
-mno-3dnow -msoft-float -mno-stack-arg-probe -mcmodel=large -mno-red-zone -m64 
-mtune=generic -march=x86-64 -g3 -Os -freg-struct-return -fno-stack-protector 
-ffreestanding -funwind-tables -fasynchronous-unwind-tables 
-fno-strict-aliasing -fstack-clash-protection -fno-ident -fplugin=annobin

Signed-off-by: Peter Jones 
---
 util/grub-mkimagexx.c | 81 ++-
 1 file changed, 73 insertions(+), 8 deletions(-)

diff --git a/util/grub-mkimagexx.c b/util/grub-mkimagexx.c
index a2bb05439f0..b016b061c8c 100644
--- a/util/grub-mkimagexx.c
+++ b/util/grub-mkimagexx.c
@@ -708,6 +708,13 @@ arm_get_trampoline_size (Elf_Ehdr *e,
 }
 #endif
 
+static int
+SUFFIX (is_kept_section) (Elf_Shdr *s, const struct 
grub_install_image_target_desc *image_target);
+static int
+SUFFIX (is_kept_reloc_section) (Elf_Shdr *s, const struct 
grub_install_image_target_desc *image_target,
+   Elf_Shdr *sections, Elf_Half section_entsize, 
Elf_Half num_sections,
+   const char *strtab);
+
 /* Deal with relocation information. This function relocates addresses
within the virtual address space starting from 0. So only relative
addresses can be fully resolved. Absolute addresses must be relocated
@@ -747,6 +754,14 @@ SUFFIX (relocate_addresses) (Elf_Ehdr *e, Elf_Shdr 
*sections,
Elf_Shdr *target_section;
Elf_Word j;
 
+   if (!SUFFIX (is_kept_section) (s, image_target) &&
+   !SUFFIX (is_kept_reloc_section) (s, image_target, sections, 
section_entsize, num_sections, strtab))
+ {
+   grub_util_info ("not translating relocations for omitted section 
%s",
+   strtab + grub_le_to_cpu32 (s->sh_name));
+   continue;
+ }
+
symtab_section = (Elf_Shdr *) ((char *) sections
 + (grub_target_to_host32 (s->sh_link)
* section_entsize));
@@ -1656,6 +1671,13 @@ make_reloc_section (Elf_Ehdr *e, struct 
grub_mkimage_layout *layout,
Elf_Addr section_address;
Elf_Word j;
 
+   if (!SUFFIX (is_kept_reloc_section) (s, image_target, sections, 
section_entsize, num_sections, strtab))
+ {
+   grub_util_info ("not translating the skipped relocation section %s",
+   strtab + grub_le_to_cpu32 (s->sh_name));
+   continue;
+ }
+
grub_util_info ("translating the relocation section %s",
strtab + grub_le_to_cpu32 (s->sh_name));
 
@@ -1731,6 +1753,56 @@ SUFFIX (is_bss_section) (Elf_Shdr *s, const struct 
grub_install_image_target_des
  == SHF_ALLOC) && (grub_target_to_host32 (s->sh_type) == SHT_NOBITS);
 }
 
+/* Determine if a section is going to be in the final output */
+static int
+SUFFIX (is_kept_section) (Elf_Shdr *s, const struct 
grub_install_image_target_desc *image_target)
+{
+  /* We keep .text and .data */
+  if (SUFFIX (is_text_section) (s, image_target)
+  || SUFFIX (is_data_section) (s, image_target))
+return 1;
+
+  /* And we keep .bss if we're producing PE binaries  or the target doesn't
+   * have a relocating loader.  Platforms other than EFI and U-boot shouldn't
+   * have .bss in their binaries as we build with -Wl,-Ttext.
+   */
+  if (SUFFIX (is_bss_section) (s, image_target)
+  && (image_target->id == IMAGE_EFI || !is_relocatable (image_target)))
+return 1;
+
+  /* Otherwise 

Re: A thread on grub-bug could need attention

[Daniel Kiper]

On Wed, Jan 31, 2018 at 10:38:03AM +0100, Michel Bouissou wrote:
> Hello,
>
> Le 30/01/2018 ?? 20:15, Thomas Schmitt a ??crit??:
> > It also does not work when booting the ISO images which shall install
> > the systems.
> > E.g. debian-9.3.0-amd64-netinst.iso :
> Yes. AFAIR I had tested all the latest (in december) standard
> installation media for at least :
> - Manjaro Cinnamon x64 (created either with dd or unetbootin)
> - Mint Cinnamon x64 (created either with dd or unetbootin)
> - Ubuntu 17.10
> - Debian live
> - Parted Magic (created with dd)
> (and maybe a few more, see my first mails...)
>
> - Manjaro Cinnamon after it being "normally" installed on a USB key
> instead of an HD on another machine - and tested working.
>
> None of them booted, all showed the same behaviour :
> - The machine's UEFI BIOS displays the USB Key as ?? Windows bootloader ??
> (and the true on-HD Windows bootloader bears the same label)
> - Selecting is causes the screen to go black with a cursor on the top
> left and nothing else ever happens.
>
> Now what works :
> - rEFInd USB key (made with dd)
> - Knoppix USB key (made with dd from DVD image)
> - Tails USB key (both made with dd or from the Tails installer)
> - Manjaro install key created with unetbootin THEN manually doctored to
> remove grub and install syslinux instead.
>
> - Plus the grub test keys you asked me to create and test in your
> previous mails.
>
> Interestingly, for "working" keys, the machines UEFI?? displays the USB
> key brand / model instead of ?? Windows bootloader ??

Could you check partitioning type used on each of above mentioned devices?
Is it MBR or GPT? Please remember that GPT have to be always protected by
something called "protective MBR". So, you will see MBR everywhere but in
case of GPT it should contain one entry with 0xee (GPT) type.

> Even more interesting, keys that do NOT work do not work either when
> chained to from rEFInd (inserting both keys, booting on rEFInd then
> chaining to the other key), with the same symptoms - where keys that
> work, also work when performing the same test.
>
> As I side note I'm currently packing for a trip to India and will be
> unable to perform any further tests from next week to the end of
> february (and will have very little time left for doing so before leaving)
>
> Have a nice day :)

Have a nice vacation! Drop us a line when you are back.

Daniel

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Michel Bouissou wrote:
> I would assume that I need to create a "minimal" directory with a
> "dummy" file inside like previously ?

Yes.
The experiment can well wait until you are back from travel.


> However, I don't have any "grub-mkrescue-sed.sh" file 

Get it from 
  
https://dev.lovelyhq.com/libburnia/libisoburn/raw/master/frontend/grub-mkrescue-sed.sh

It belongs to the upstream sources of xorriso, where i am developer.
The GNU xorriso tarball has it as demo in its ./frontend directory.
You can trust its good intentions as much as the ones of xorriso.
File operations only affect the grub-mkrescue temporary directory and
the /tmp directory. Files get moved, not deleted.
(The practical consequences of running it are subject to the usual
 risk of bugs, i fear.)

It works with xorriso-1.4.4 or newer. Best with current release 1.4.8.

If your installed xorriso tells an older revision when run without any
arguments, then consider to get xorriso and grub-mkrescue-sed.sh by the
tarball at
  https://www.gnu.org/software/xorriso/#download
This xorriso needs no installation but only a place where to unpack
and compile
  tar xzf xorriso-1.4.8.tar.gz
  cd xorriso-1.4.8
  ./configure && make
If you run its fellow ./frontend/grub-mkrescue-sed.sh then it will make
use of the compiled binary ./xorriso/xorriso.
Existing installed xorriso and its libraries will not be affected.


Lengthy motivation:

The purpose of grub-mkrescue-sed.sh is to modify the xorriso run underneath
grub-mkrescue to get other partition layouts.

The native layout as defined by Vladimir Serbinenko and implemented by me
obviously works fine. Its main disadvantage is that it shows at least
two not mountable GPT partitions (e.g. /dev/sdc1 and /dev/sdc4).
The ISO 9660 filesystem can only be mounted by the base device /dev/sdc.

GPT has the further disadvantage that it prescribes to have a backup
partition table at the end of the storage device. But when bootable ISOs
are made, it is not yet known how large the device will be. And even if so,
one would have to pad up the image to write that backup GPT.
So any ISO copied onto USB stick has its backup GPT at the wrong place
and any remnant backup GPT at the correct position would be in conflict
with the main GPT at image start.

The MBR partition table doesn't refer to the device end. UEFI specifies
that a MBR partition of type 0xEF shall be regarded as EFI System
Partition.

By some extra measures, the default mode of grub-mkrescue-sed.sh achieves
that all partitions are mountable and that no nested partitions emerge.
This partition table is fully acceptable to MBR partition editors like
fdisk which can then be used to add partitions and so give the rest of
the USB stick a life.
Because of the 64 block gap before partition 1, it is even possible to
install a new boot loader and use the ISO as read-only data partition
rather than as home of the operating system.

The same measures can help to make GPT with mountable ISO partition
(see description of "gpt_appended" in the script). But the problem of
misplaced backup GPT cannot be solved at the time when the ISO gets
produced.

Most modes avoid HFS+ partition and Apple Partition Map. Partition editors
and operating systems do not expect APM to be combined with MBR or GPT
by some hot x86 machine code stunt and an unusual APM block size. 


Have a nice day :)

Thomas


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi,

Le 31/01/2018 à 13:20, Thomas Schmitt a écrit :
> This run too ?
>
>   grub-mkrescue -o output.iso minimal \
> --xorriso=...path.../grub-mkrescue-sed.sh \
> -partition_offset 16  
>
I would assume that I need to create a "minimal" directory with a
"dummy" file inside like previously ?

However, I don't have any "grub-mkrescue-sed.sh" file on the system on
which I would try to build the USB key...

[root@tiru ~]# pacman -Ql grub | grep rescue
grub /usr/bin/grub-mkrescue
grub /usr/share/man/man1/grub-mkrescue.1.gz

Kind regards.

ॐ

-- 
Michel Bouissou  OpenPGP ID 0xEB04D09C


___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

i wrote:
> > It also does not work when booting the ISO images which shall install
> > the systems.

Michel Bouissou wrote:
> Yes. AFAIR I had tested all the latest (in december) standard
> installation media for at least :
> [...]
> - Debian live

debian-live-9.3.0-amd64-xfce.iso looks much like the Debian netinst ISO.

Strings from end of /mnt/fat/efi/boot/bootx64.efi :
-
search --file --set=root /.disk/info
set prefix=($root)/boot/grub
source $prefix/x86_64-efi/grub.cfg
(memdisk)/boot/grub
-

/mnt/iso/boot/grub/x86_64-efi/grub.cfg :
-
insmod part_acorn
insmod part_amiga
insmod part_apple
insmod part_bsd
insmod part_dfly
insmod part_dvh
insmod part_gpt
insmod part_msdos
insmod part_plan
insmod part_sun
insmod part_sunpc
source /boot/grub/grub.cfg
-

/mnt/iso/boot/grub/grub.cfg :
-
if loadfont $prefix/font.pf2 ; then
  set gfxmode=800x600
  insmod efi_gop
  insmod efi_uga
  insmod video_bochs
  insmod video_cirrus
  insmod gfxterm
  insmod png
  terminal_output gfxterm
fi

if background_image /isolinux/splash.png; then
  set color_normal=light-gray/black
  set color_highlight=white/black
else
  set menu_color_normal=cyan/blue
  set menu_color_highlight=white/blue
fi

insmod play
play 960 440 1 0 4 440 1
if [ ${iso_path} ] ; then
set loopback="findiso=${iso_path}"
fi

menuentry "Debian GNU/Linux Live (kernel 4.9.0-4-amd64)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
submenu "Debian Live with Localisation Support" {
menuentry "Albanian (sq)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=sq_AL.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Amharic (am)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=am_ET 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Arabic (ar)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ar_EG.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Asturian (ast)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ast_ES.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Basque (eu)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=eu_ES.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Belarusian (be)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=be_BY.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Bangla (bn)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bn_BD 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Bosnian (bs)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bs_BA.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Bulgarian (bg)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bg_BG.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Tibetan (bo)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bo_IN 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "C (C)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=C 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Catalan (ca)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ca_ES.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Chinese (Simplified) (zh_CN)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=zh_CN.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Chinese (Traditional) (zh_TW)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=zh_TW.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Croatian (hr)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=hr_HR.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Czech (cs)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=cs_CZ.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Danish (da)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=da_DK.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Dutch (nl)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=nl_NL.UTF-8 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Dzongkha (dz)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=dz_BT 
"${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "English (en)" {
  linux  

Re: [PATCH] chainloader: Fix wrong break condition (must be AND not OR)

[C. Masloch]

On at 2018-01-29 18:09 +01:00, Daniel Kiper wrote:
> On Sun, Jan 21, 2018 at 04:02:10PM +0100, C. Masloch wrote:
>> The definition of bpb's num_total_sectors_16 and num_total_sectors_32
>> is that either the 16-bit field is non-zero and is used (in which case
>> eg mkfs.fat sets the 32-bit field to zero), or it is zero and the
>> 32-bit field is used. Therefore, a BPB is invalid only if *both*
>> fields are zero; having one field as zero and the other as non-zero is
>> the case to be expected.
> 
> Could you provide reference to the spec which says that?

Here's a few descriptions pointing to that fact:


https://www.win.tue.nl/~aeb/linux/fs/fat/fat-1.html

>  The old 2-byte fields "total number of sectors" and "number of
sectors per FAT" are now zero; this information is now found in the new
4-byte fields.

(Here given in the FAT32 EBPB section but the total sectors 16/32 bit
fields semantic is true of FAT12 and FAT16 too.)


https://wiki.osdev.org/FAT#BPB_.28BIOS_Parameter_Block.29

> 19 | 2 | The total sectors in the logical volume. If this value is 0,
it means there are more than 65535 sectors in the volume, and the actual
count is stored in "Large Sectors (bytes 32-35).

> 32 | 4 | Large amount of sector on media. This field is set if there
are more than 65535 sectors in the volume.

(Doesn't specify what the "large" field is set to when unused, but as
mentioned mkfs.fat sets it to zero then.)


https://technet.microsoft.com/en-us/library/cc976796.aspx

> 0x13 | WORD | 0x |
> Small Sectors . The number of sectors on the volume represented in 16
> bits (< 65,536). For volumes larger than 65,536 sectors, this field
> has a value of zero and the Large Sectors field is used instead.

> 0x20 | DWORD | 0x01F03E00 |
> Large Sectors . If the value of the Small Sectors field is zero, this
> field contains the total number of sectors in the FAT16 volume. If the
> value of the Small Sectors field is not zero, the value of this field
> is zero.


https://staff.washington.edu/dittrich/misc/fatgen103.pdf page 10

> BPB_TotSec16 | 19 | 2 |
> This field is the old 16-bit total count of sectors on the volume.
> This count includes the count of all sectors in all four regions of the
> volume. This field can be 0; if it is 0, then BPB_TotSec32 must be
> non-zero. For FAT32 volumes, this field must be 0. For FAT12 and
> FAT16 volumes, this field contains the sector count, and
> BPB_TotSec32 is 0 if the total sector count “fits” (is less than
> 0x1).

> BPB_TotSec32 | 32 | 4 |
> This field is the new 32-bit total count of sectors on the volume.
> This count includes the count of all sectors in all four regions of the
> volume. This field can be 0; if it is 0, then BPB_TotSec16 must be
> non-zero. For FAT32 volumes, this field must be non-zero. For
> FAT12/FAT16 volumes, this field contains the sector count if
> BPB_TotSec16 is 0 (count is greater than or equal to 0x1).

(This specifies that an unused BPB_TotSec32 field is set to zero.)


>> This affects all users of grub_chainloader_patch_bpb which are in
>> chainloader.c, freedos.c, and ntldr.c
> 
> I am happy that you fix that issue but
>   https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#BPB331_OFS_15h
> shows that life is more complicated.
>
> Could you take that into account?

MS-DOS 3.20 and 3.30 BPBs aren't supported anyway. The special case for
using a partition table entry's partition length field isn't applicable
here. (If it were, the function simply shouldn't check these fields.)
And this is the first I've read of that DR-DOS extension. (And again,
the simple solution to that one is also not to check the fields for zeros.)

>> Tested with lDebug booted in qemu via grub2's
>> FreeDOS direct loading support, refer to
>> https://bitbucket.org/ecm/ldosboot + https://bitbucket.org/ecm/ldebug
> 
> Could you put your SOB here?

Like this?

Signed-off-by: C. Masloch 

(Should I submit a PATCH v2 for this?)

___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hello,

Le 30/01/2018 à 20:15, Thomas Schmitt a écrit :
> It also does not work when booting the ISO images which shall install
> the systems.
> E.g. debian-9.3.0-amd64-netinst.iso :
Yes. AFAIR I had tested all the latest (in december) standard
installation media for at least :
- Manjaro Cinnamon x64 (created either with dd or unetbootin)
- Mint Cinnamon x64 (created either with dd or unetbootin)
- Ubuntu 17.10
- Debian live
- Parted Magic (created with dd)
(and maybe a few more, see my first mails...)

- Manjaro Cinnamon after it being "normally" installed on a USB key
instead of an HD on another machine - and tested working.

None of them booted, all showed the same behaviour :
- The machine's UEFI BIOS displays the USB Key as « Windows bootloader »
(and the true on-HD Windows bootloader bears the same label)
- Selecting is causes the screen to go black with a cursor on the top
left and nothing else ever happens.

Now what works :
- rEFInd USB key (made with dd)
- Knoppix USB key (made with dd from DVD image)
- Tails USB key (both made with dd or from the Tails installer)
- Manjaro install key created with unetbootin THEN manually doctored to
remove grub and install syslinux instead.

- Plus the grub test keys you asked me to create and test in your
previous mails.

Interestingly, for "working" keys, the machines UEFI  displays the USB
key brand / model instead of « Windows bootloader »

Even more interesting, keys that do NOT work do not work either when
chained to from rEFInd (inserting both keys, booting on rEFInd then
chaining to the other key), with the same symptoms - where keys that
work, also work when performing the same test.

As I side note I'm currently packing for a trip to India and will be
unable to perform any further tests from next week to the end of
february (and will have very little time left for doing so before leaving)

Have a nice day :)

ॐ

-- 
Michel Bouissou  OpenPGP ID 0xEB04D09C



___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel