Bug#839894: [patch] fix installation to system that has many disks (>= 27) (Re: Bug#839894: installation-report: Jessie installer fails to install GRUB on a large JBOD system

[Hideki Yamane]

Hi,

 ping to debian-boot for patch review.

On Sat, 04 Feb 2017 13:15:36 +0900 Hideki Yamane  wrote:
> control: reassign -1 grub-installer
> control: tags -1 +patch
(snip)
>  More tiny fix is enough for this issue. I've confirmed it by 
>  s%/dev/[hsv]d[a-z0-9]%/dev/[hsv]d[a-z0-9]*% - adding one "*" in
>  VirtualBox (it's easy to create such environment).

 I've confirmed it works on VBox
 
https://qiita-image-store.s3.amazonaws.com/0/33063/eaebbb71-6cd7-3f86-5268-897297fd100e.png

 Could you review the patch, please?

 # It may be better to use /dev/[hsv]d[a-z0-9].* than /dev/[hsv]d[a-z0-9]*


-- 
Regards,

 Hideki Yamane henrich @ debian.or.jp/org
 http://wiki.debian.org/HidekiYamane

Bug#839894: [patch] fix installation to system that has many disks (>= 27) (Re: Bug#839894: installation-report: Jessie installer fails to install GRUB on a large JBOD system

[Hideki Yamane]

control: reassign -1 grub-installer
control: tags -1 +patch

Hi,

On Tue, 11 Oct 2016 12:13:33 +0200 "Jonathan Quick"  wrote:
> > Probably this part of grub-installer is the problem:
> >
> > 
> > /dev/[hsv]d[a-z0-9]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
> >
> >
> > /dev/[hsv]d[a-z0-9] will match /dev/sda through /dev/sdz, but NOT /dev/sdak.
> >
> > If it added /dev/sd[a-z][a-z] as well, it should work for you.
> >
> > So maybe:
> > 
> > /dev/[hsv]d[a-z0-9]|/dev/sd[a-z][a-z]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
> >
> > Not sure if other places need fixing too.
> 
> Having edited grub-installer in-place during the install process, I can
> confirm that the above change is sufficient to fix the problem and the
> installation then completes normally.

 More tiny fix is enough for this issue. I've confirmed it by 
 s%/dev/[hsv]d[a-z0-9]%/dev/[hsv]d[a-z0-9]*% - adding one "*" in
 VirtualBox (it's easy to create such environment).
--- grub-installer.orig	2017-02-03 22:51:57.598613756 +0900
+++ grub-installer	2017-02-04 00:19:37.085153540 +0900
@@ -254,7 +254,7 @@
 /dev/mapper)
 	disc_offered_devfs="$bootfs"
 	;;
-/dev/[hsv]d[a-z0-9]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
+/dev/[hsv]d[a-z0-9]*|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
 	disc_offered_devfs="$prefix"
 	;;
 *)

Bug#839894: installation-report: Jessie installer fails to install GRUB on a large JBOD system

[Jonathan Quick]

Hi Len

On Thu, October 6, 2016 8:24 pm, Lennart Sorensen wrote:
> On Thu, Oct 06, 2016 at 07:45:36AM +, Jonathan Quick wrote:
>> Whilst installing from a (Jessie) 8.6 netboot installer the GRUB
>> installation step failed, leaving the system unbootable.
>
> Sounds like the regex for matching valid disks in the installer handling
> grub install isn't expecting 2 letter names, which is certainly an unusual
> case, and if your system had put the target disk first rather than last,
> you wouldn't have had a problem either.  Of course if wheezy didn't see
> the other disks that would have solved the problem for it.
>
> Probably this part of grub-installer is the problem:
>
> 
> /dev/[hsv]d[a-z0-9]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
>
>
> /dev/[hsv]d[a-z0-9] will match /dev/sda through /dev/sdz, but NOT /dev/sdak.
>
> If it added /dev/sd[a-z][a-z] as well, it should work for you.
>
> So maybe:
> 
> /dev/[hsv]d[a-z0-9]|/dev/sd[a-z][a-z]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
>
> Not sure if other places need fixing too.

Having edited grub-installer in-place during the install process, I can
confirm that the above change is sufficient to fix the problem and the
installation then completes normally.

Thanks
  Jon

Bug#839894: installation-report: Jessie installer fails to install GRUB on a large JBOD system

[Lennart Sorensen]

On Thu, Oct 06, 2016 at 07:45:36AM +, Jonathan Quick wrote:
> Package: installation-reports
> Version: 2.58
> Severity: important
> 
> Dear Maintainer,
> 
> Whilst installing from a (Jessie) 8.6 netboot installer the GRUB installation 
> step failed, leaving the system unbootable.
> 
> -- Package-specific info:
> 
> Boot method: network
> Image version: 20150422+deb8u4+b1/images/netboot/debian-installer/amd64/linux
> Date: 6 October 2016 08:30 SAST
> 
> Machine: SuperMicro SuperServer
> Partitions: 
> Device Boot Start   End   Sectors  Size Id Type
> /dev/sdak1   2048 119883775 119881728 57.2G 83 Linux
> /dev/sdak2  119885822 125044735   5158914  2.5G  5 Extended
> /dev/sdak5  119885824 125044735   5158912  2.5G 82 Linux swap / Solaris
> 
> Base System Installation Checklist:
> [O] = OK, [E] = Error (please elaborate below), [ ] = didn't try it
> 
> Initial boot:   [O]
> Detect network card:[O]
> Configure network:  [O]
> Detect CD:  [ ]
> Load installer modules: [O]
> Clock/timezone setup:   [O]
> User/password setup:[O]
> Detect hard drives: [O]
> Partition hard drives:  [O]
> Install base system:[O]
> Install tasks:  [O]
> Install boot loader:[E]
> Overall install:[O]after recovery via rescue mode
> 
> Comments/Problems:
> 
> This was an installation of only the standard system plus a SSH server - no 
> desktop.
> Generally the installation went smoothly apart from the boot-loader step 
> which failed with no information on why.
> Trying a full desktop install gave the same problem (after a longer wait of 
> course).
> 
> Using the rescue mode (once I realised out that shells in the text mode 
> intaller didn't work and switched to the graphical
> one) I discovered that the grub2 package and its dependencies had not been 
> installed.  Once I had installed those packages,
> I was then able to use 'grub-install sdak' and 'update-grub' to get things 
> going.  Only after doing this did the option to
> reinstall GRUB appear in the rescue menu.  Now that GRUB is installed, the 
> machine boots fine into a working system.
> 
> Note this system has 36 4TB hard disk drives attached via a HBA (mpt3sas) and 
> one 64GB SSD attached to the motherboard SATA
> controller to which I was actually installing, hence the '/dev/sdak' disk 
> designation.
> 
> A test installation using a (Wheezy) 7.11 installer ran just fine without 
> showing this problem, but didn't see the HBA
> and its attached disks.

Sounds like the regex for matching valid disks in the installer handling
grub install isn't expecting 2 letter names, which is certainly an unusual
case, and if your system had put the target disk first rather than last,
you wouldn't have had a problem either.  Of course if wheezy didn't see
the other disks that would have solved the problem for it.

Probably this part of grub-installer is the problem:

case $prefix in
/dev/md)
disc_offered_devfs="$bootfs"
;;
/dev/mapper)
disc_offered_devfs="$bootfs"
;;

/dev/[hsv]d[a-z0-9]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)
disc_offered_devfs="$prefix"
;;
*)
disc_offered_devfs=$(echo "$bootfs_nodevfs" | sed 
"s:\(.*\)/.*:\1/disc:")
;;
esac

/dev/[hsv]d[a-z0-9] will match /dev/sda through /dev/sdz, but NOT /dev/sdak.

If it added /dev/sd[a-z][a-z] as well, it should work for you.

So maybe:

/dev/[hsv]d[a-z0-9]|/dev/sd[a-z][a-z]|/dev/xvd[a-z]|/dev/cciss/c[0-9]d[0-9]*|/dev/ida/c[0-9]d[0-9]*|/dev/rs/c[0-9]d[0-9]*|/dev/mmcblk[0-9]|/dev/nvme[0-9]*n[0-9]*|/dev/ad[0-9]*|/dev/da[0-9]*)

Not sure if other places need fixing too.

-- 
Len Sorensen