Public bug reported: == Comment: #0 - Heitor Ricardo Alves de Siqueira - 2016-02-19 11:37:33 == ---Problem Description--- When installing Ubuntu 16.04 to a RAID1 powerVM configuration, grub only installs to the first disk. The second disk will not have a valid PReP partition, rendering the system unbootable if the first disk fails in any way.
We are currently seeing this on a RAID1 install to two NVMe adapters. ---uname output--- Linux tul131p1 4.4.0-6-generic #21-Ubuntu SMP Tue Feb 16 20:31:37 UTC 2016 ppc64le GNU/Linux Machine Type = 8286-42A lpar ---boot type--- kexec vmlinuz/initrd ---Kernel cmdline used to launch install--- default cmdline ---Install repository type--- Internet repository ---Install repository Location--- ports.ubuntu.com ---Point of failure--- Problem during post-install (stage 2) configuration or other problem seen after reboot == Comment: #1 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 05:31:43 == Currently: 1) grub-install lists all PReP partitions in the system 2) and prefers a PReP partition on the same disk as the root/boot filesystem. 3) wipe it and install the bootloader code in it. Idea: 0) keep that as-is / backward compatible, as it works OK so far, but slightly change a particular case 1) grub-install lists all PReP partitions in the system 2) and prefers a PReP partition on the same disk as the root/boot filesystem. 3) but if the root/boot filesystem is a MD device, 4) find its component disks 5) check which ones have a PReP partition 6) wipe it and install bootloader code in it. == Comment: #4 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 08:54:04 == Hi Canonical, This bug is a follow on to LP bug 1487365 (LTC BZ bug 129040) in the area of PReP partitions and Software RAID installations. @taco-screen-team I'd suggest for the bug assignee/owner to be @mathieu-tl on this bug too. == Comment: #5 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 09:00:23 == This patch for Xenial adds support in grub-installer for handling multiple PReP partitions on component devices of MD/dmadm devices. It's backward compatible w/ the existing handling of other device types. == Comment: #6 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 09:01:01 == Debugging logs from the installation: Syslog: ~ # grep grub-install /var/log/syslog Feb 26 11:09:15 anna[7389]: DEBUG: retrieving grub-installer 1.128ubuntu2 Feb 26 11:57:48 main-menu[1776]: INFO: Menu item 'grub-installer' selected Feb 26 11:57:48 grub-installer: info: architecture: ppc64el/chrp_ibm Feb 26 11:57:52 grub-installer: info: Identified partition label for /dev/md0p2: gpt Feb 26 11:57:52 grub-installer: info: Wiping PReP partition /dev/sda1 Feb 26 11:57:52 grub-installer: info: Wiping PReP partition /dev/sdb1 Feb 26 11:57:52 grub-installer: info: Wiping PReP partition /dev/sdc1 Feb 26 11:57:53 grub-installer: info: Installing grub on '/dev/sdc1' Feb 26 11:57:53 grub-installer: info: grub-install does not support --no-floppy Feb 26 11:57:53 grub-installer: info: Running chroot /target grub-install --force "/dev/sda1" Feb 26 11:57:54 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 11:58:02 grub-installer: Installation finished. No error reported. Feb 26 11:58:02 grub-installer: info: grub-install ran successfully Feb 26 11:58:02 grub-installer: info: grub-install does not support --no-floppy Feb 26 11:58:02 grub-installer: info: Running chroot /target grub-install --force "/dev/sdb1" Feb 26 11:58:02 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 11:58:11 grub-installer: Installation finished. No error reported. Feb 26 11:58:11 grub-installer: info: grub-install ran successfully Feb 26 11:58:11 grub-installer: info: grub-install does not support --no-floppy Feb 26 11:58:11 grub-installer: info: Running chroot /target grub-install --force "/dev/sdc1" Feb 26 11:58:11 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 11:58:20 grub-installer: Installation finished. No error reported. Feb 26 11:58:20 grub-installer: info: grub-install ran successfully Shell script tracing: For the shell tracing, add the following to /usr/bin/grub-install: exec 2>/grub.debug set -x ~ # cat /grub.debug <...> + wipe_bootdevs= + /usr/lib/grub-installer/prep-bootdev + wipe_bootdev=/dev/sda1 + /usr/lib/grub-installer/prep-bootdev -l + [ /dev/sda = /dev/md ] + grep -q /dev/md[0-9]\+ + echo /dev/md0 + grep /dev/sda + grep -o /dev/[sv]d[a-z]\+ + mdadm --detail --verbose /dev/md0 + wipe_bootdevs= /dev/sda1 + [ /dev/sdb = /dev/md ] + grep -q /dev/md[0-9]\+ + echo /dev/md0 + grep /dev/sdb + grep -o /dev/[sv]d[a-z]\+ + mdadm --detail --verbose /dev/md0 + wipe_bootdevs= /dev/sda1 /dev/sdb1 + [ /dev/sdc = /dev/md ] + grep -q /dev/md[0-9]\+ + echo /dev/md0 + grep /dev/sdc + grep -o /dev/[sv]d[a-z]\+ + mdadm --detail --verbose /dev/md0 + wipe_bootdevs= /dev/sda1 /dev/sdb1 /dev/sdc1 + unset prep_p + [ -z /dev/sda1 /dev/sdb1 /dev/sdc1 ] + [ -n /dev/sda1 ] + info Wiping PReP partition /dev/sda1 + log info: Wiping PReP partition /dev/sda1 + logger -t grub-installer info: Wiping PReP partition /dev/sda1 + blockdev --getsz /dev/sda1 + log-output -t grub-installer dd if=/dev/zero of=/dev/sda1 bs=512 count=14336 + [ -n /dev/sdb1 ] + info Wiping PReP partition /dev/sdb1 + log info: Wiping PReP partition /dev/sdb1 + logger -t grub-installer info: Wiping PReP partition /dev/sdb1 + blockdev --getsz /dev/sdb1 + log-output -t grub-installer dd if=/dev/zero of=/dev/sdb1 bs=512 count=14336 + [ -n /dev/sdc1 ] + info Wiping PReP partition /dev/sdc1 + log info: Wiping PReP partition /dev/sdc1 + logger -t grub-installer info: Wiping PReP partition /dev/sdc1 + blockdev --getsz /dev/sdc1 + log-output -t grub-installer dd if=/dev/zero of=/dev/sdc1 bs=512 count=14336 <...> + bootdevs=/dev/sdc1 + bootdevs= /dev/sda1 /dev/sdb1 /dev/sdc1 <...> + info Running chroot /target grub-install --force "/dev/sda1" + log info: Running chroot /target grub-install --force "/dev/sda1" + logger -t grub-installer info: Running chroot /target grub-install --force "/dev/sda1" + log-output -t grub-installer chroot /target grub-install --force /dev/sda1 + [ 0 = 0 ] + info grub-install ran successfully + log info: grub-install ran successfully + logger -t grub-installer info: grub-install ran successfully <...> + info Running chroot /target grub-install --force "/dev/sdb1" + log info: Running chroot /target grub-install --force "/dev/sdb1" + logger -t grub-installer info: Running chroot /target grub-install --force "/dev/sdb1" + log-output -t grub-installer chroot /target grub-install --force /dev/sdb1 + [ 0 = 0 ] + info grub-install ran successfully + log info: grub-install ran successfully + logger -t grub-installer info: grub-install ran successfully <...> + info Running chroot /target grub-install --force "/dev/sdc1" + log info: Running chroot /target grub-install --force "/dev/sdc1" + logger -t grub-installer info: Running chroot /target grub-install --force "/dev/sdc1" + log-output -t grub-installer chroot /target grub-install --force /dev/sdc1 + [ 0 = 0 ] + info grub-install ran successfully + log info: grub-install ran successfully + logger -t grub-installer info: grub-install ran successfully <...> == Comment: #7 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 09:01:29 == Test scenario: 3 individual disks (2 active, 1 spare) $ for i in 1 2 3; do qemu-img create -f qcow2 disk$i.qcow2 8G; done Formatting 'disk1.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 Formatting 'disk2.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 Formatting 'disk3.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ qemu-system-ppc64 -enable-kvm \ -nographic -nodefaults \ -monitor stdio -serial pty \ -M pseries -smp 1,cores=1,threads=1 -m 8G \ \ -device spapr-vscsi \ -drive file=disk1.qcow2 \ -drive file=disk2.qcow2 \ -drive file=disk3.qcow2 \ -drive media=cdrom,file=xenial-server-ppc64el.iso \ \ -device spapr-vlan,netdev=net0,mac=4C:45:42:45:02:03 \ -netdev bridge,id=net0,br=br0 \ -S 1) On the 'Partition disks' dialog, select 'Go back', and 'Execute a shell' 2) Check the 'grub-installer' script is present ~ # ls -l /usr/bin/grub-installer -rwxr-xr-x 1 root root 44452 Jan 4 14:05 /usr/bin/grub-installer 3) Download and install the test package ~ # wget http://ausgsa.ibm.com/~mauricfo/public/bugs/bz137572/v1/grub-installer_1.128ubuntu2multiprep1_ppc64el.udeb ~ # udpkg -i *.udeb <... ignore the messages ...> 4) Verify the patch is present ~ # grep MD/mdadm /usr/bin/grub-installer # On MD/mdadm devices, each component device may have a PReP partition. # On MD/mdadm devices, each component device may have a PReP partition. 5) Return to the installation. ~ # exit 6) Proceed normally with a Software RAID based partitioning *with a PReP partition per component device.* 7) On the 'Installation finished' prompt (no errors expected), select 'Go Back', and 'Execute a shell' 8) Verify all the PReP partitions on the component devices were covered by grub-installer. (e.g., notice the 'Wiping PReP partition' and 'chroot /target grub-install' messages) ~ # grep -i grub-install /var/log/syslog Feb 26 13:19:45 anna[7383]: DEBUG: retrieving grub-installer 1.128ubuntu2 Feb 26 13:22:22 grub-installer: info: architecture: ppc64el/chrp_ibm Feb 26 13:34:36 main-menu[1769]: INFO: Menu item 'grub-installer' selected Feb 26 13:34:36 grub-installer: info: architecture: ppc64el/chrp_ibm Feb 26 13:34:38 grub-installer: info: Identified partition label for /dev/md0p2: gpt Feb 26 13:34:39 grub-installer: info: Wiping PReP partition /dev/sda1 Feb 26 13:34:39 grub-installer: info: Wiping PReP partition /dev/sdb1 Feb 26 13:34:39 grub-installer: info: Wiping PReP partition /dev/sdc1 Feb 26 13:34:55 grub-installer: info: Installing grub on '/dev/sdc1' Feb 26 13:34:55 grub-installer: info: grub-install does not support --no-floppy Feb 26 13:34:55 grub-installer: info: Running chroot /target grub-install --force "/dev/sda1" Feb 26 13:34:55 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 13:35:04 grub-installer: Installation finished. No error reported. Feb 26 13:35:04 grub-installer: info: grub-install ran successfully Feb 26 13:35:04 grub-installer: info: grub-install does not support --no-floppy Feb 26 13:35:04 grub-installer: info: Running chroot /target grub-install --force "/dev/sdb1" Feb 26 13:35:04 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 13:35:13 grub-installer: Installation finished. No error reported. Feb 26 13:35:13 grub-installer: info: grub-install ran successfully Feb 26 13:35:13 grub-installer: info: grub-install does not support --no-floppy Feb 26 13:35:13 grub-installer: info: Running chroot /target grub-install --force "/dev/sdc1" Feb 26 13:35:13 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 13:35:23 grub-installer: Installation finished. No error reported. Feb 26 13:35:23 grub-installer: info: grub-install ran successfully 9) Return to the installation, and select 'Finish the installation'. ~ # exit Boot tests: During boot, the SLOF firmware could successfully detect and load the GRUB2 code from the PReP partition of *each individual disk* (test: run guest with either disk1, disk2, or disk3.qcow2). Trying to load: from: disk ... No DOS disk-label found. Successfully loaded error: no suitable video mode found. <...> GNU GRUB version 2.02~beta2-36 <...> The only boot failure case is with only the spare disk (disk3.qcow2), since GRUB2 can't load the remaining of its code from one of the active devices. (notice it's a GRUB2 error message, so PReP was correctly detected and GRUB2 loaded from it.) Trying to load: from: disk ... No DOS disk-label found. Successfully loaded error: disk `mduuid/b3069040df508c48c7c8ddc6a19a6bc0,2' not found. Entering rescue mode... grub rescue> == Comment: #8 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 09:03:00 == Hi Heitor, Can you please check it works correctly on your PowerVM environment? I verified this works correctly on a qemu-kvm guest. Thanks == Comment: #9 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 10:41:01 == Hi Canonical, The patch is verified for 14.04 too. Please apply on 14.04 as well. Thanks == Comment: #10 - Mauricio Faria De Oliveira <[email protected]> - 2016-02-26 12:30:05 == (In reply to comment #9) > The patch is verified for 14.04 too. Please apply on 14.04 as well. Feb 26 14:17:48 anna[6955]: DEBUG: retrieving grub-installer 1.78ubuntu20.3 Feb 26 15:31:21 main-menu[1787]: INFO: Menu item 'grub-installer' selected Feb 26 15:31:21 grub-installer: info: architecture: ppc64el/chrp_ibm Feb 26 15:31:34 grub-installer: info: Identified partition label for /dev/md0p2: loop Feb 26 15:31:34 grub-installer: info: Wiping PReP partition /dev/sda1 Feb 26 15:31:34 grub-installer: info: Wiping PReP partition /dev/sdb1 Feb 26 15:31:34 grub-installer: info: Wiping PReP partition /dev/sdc1 Feb 26 15:31:50 grub-installer: info: Installing grub on '/dev/sdc1' Feb 26 15:31:50 grub-installer: info: grub-install does not support --no-floppy Feb 26 15:31:50 grub-installer: info: Wiping PReP partition /dev/sda1 Feb 26 15:31:50 grub-installer: info: Running chroot /target grub-install --force "/dev/sda1" Feb 26 15:31:50 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 15:31:59 grub-installer: Installation finished. No error reported. Feb 26 15:31:59 grub-installer: info: grub-install ran successfully Feb 26 15:31:59 grub-installer: info: grub-install does not support --no-floppy Feb 26 15:31:59 grub-installer: info: Wiping PReP partition /dev/sdb1 Feb 26 15:31:59 grub-installer: info: Running chroot /target grub-install --force "/dev/sdb1" Feb 26 15:31:59 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 15:32:08 grub-installer: Installation finished. No error reported. Feb 26 15:32:08 grub-installer: info: grub-install ran successfully Feb 26 15:32:08 grub-installer: info: grub-install does not support --no-floppy Feb 26 15:32:08 grub-installer: info: Wiping PReP partition /dev/sdc1 Feb 26 15:32:08 grub-installer: info: Running chroot /target grub-install --force "/dev/sdc1" Feb 26 15:32:08 grub-installer: Installing for powerpc-ieee1275 platform. Feb 26 15:32:16 grub-installer: Installation finished. No error reported. Feb 26 15:32:16 grub-installer: info: grub-install ran successfully ** Affects: debian-installer (Ubuntu) Importance: Undecided Assignee: Taco Screen team (taco-screen-team) Status: New ** Tags: architecture-ppc64le bugnameltc-137572 severity-high targetmilestone-inin1604 ** Tags added: architecture-ppc64le bugnameltc-137572 severity-high targetmilestone-inin1604 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1550448 Title: [PowerVM] Ubuntu 16.04 does not install bootloader on multiple PReP partitions in Software RAID1 configuration To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1550448/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
