Bug#699434: grub-installer: support for Intel Matrix Raid

2013-12-18 Thread Miquel van Smoorenburg 

On 02/12/13 10:59, Dmitrijs Ledkovs wrote:

Hello Miquel,

On 31 January 2013 12:40, Miquel van Smoorenburg miqu...@debian.org wrote:

This is by far the most intrusive patch - all the others are trivial.
Description:

- move make_device_map() upwards so that we can call it earlier
   in the script. Also, if $md_bootdev is set, add a (otherwise
   unused) (md/0) entry to device.map - otherwise grub-probe
   freaks out on upgrades.
- add is_mdraid_bootable(). It checks if the device is a Intel
   Matrix Raid device, in RAID1 mode. If not it returns false.
   Otherwise it sets $md_dev, $md_devs, $md_super and $md_level,
and returns true.



Why is it limited to only imsm  raid1 ?


Because it's the only thing I could test, and at the time ddf support 
was kind of wonky. So I thought it best to leave that out.


Also, only RAID1 is because I could not get grub to install on the MD 
device- it didn't accept that. And for example, grub-mkdevicemap doesn't 
create an entry for /dev/md... yet. So right now the code finds the 
devices underlying the RAID1 array and installs grub on all of them.


Someone needs to fix grub so that it understands MD devices directly. Or 
the current version might already do that, I haven't checked in a while.



mdadm supports imsm superblock format levels 0,1,5,10  ddf
superblock format levels 0,1,3,4,5,1E,5E,5EE,6 with extensions of
there off (stripped, mirrored, concatenated and spanned).

Would it not be best to simply check for imsm || ddf superblock
types? Given that mdadm supports most of the above RAID levels.


Sure.

Mike.


--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#699434: grub-installer: support for Intel Matrix Raid

2013-12-18 Thread Dimitri John Ledkov 
On 18 December 2013 15:14, Miquel van Smoorenburg miqu...@debian.org wrote:
 On 02/12/13 10:59, Dmitrijs Ledkovs wrote:

 Hello Miquel,

 On 31 January 2013 12:40, Miquel van Smoorenburg miqu...@debian.org
 wrote:

 This is by far the most intrusive patch - all the others are trivial.
 Description:

 - move make_device_map() upwards so that we can call it earlier
in the script. Also, if $md_bootdev is set, add a (otherwise
unused) (md/0) entry to device.map - otherwise grub-probe
freaks out on upgrades.
 - add is_mdraid_bootable(). It checks if the device is a Intel
Matrix Raid device, in RAID1 mode. If not it returns false.
Otherwise it sets $md_dev, $md_devs, $md_super and $md_level,
 and returns true.



 Why is it limited to only imsm  raid1 ?


 Because it's the only thing I could test, and at the time ddf support was
 kind of wonky. So I thought it best to leave that out.


true ddf support has improved a lot in mdadm 3.3

 Also, only RAID1 is because I could not get grub to install on the MD
 device- it didn't accept that. And for example, grub-mkdevicemap doesn't
 create an entry for /dev/md... yet. So right now the code finds the devices
 underlying the RAID1 array and installs grub on all of them.

 Someone needs to fix grub so that it understands MD devices directly. Or the
 current version might already do that, I haven't checked in a while.


yeah, trunk snapshots of grub2 properly understand isms/ddf and know
that those raid arrays are recognizable by boot-firmware (bios/uefi)
and hence allows to install direct on-to them without any other
adjustments.
I haven't tested yet, but i think some grub-installer changes will be
needed to detect that as well, but it should be done differently, i.e.
without stub entries in device.map



 mdadm supports imsm superblock format levels 0,1,5,10  ddf
 superblock format levels 0,1,3,4,5,1E,5E,5EE,6 with extensions of
 there off (stripped, mirrored, concatenated and spanned).

 Would it not be best to simply check for imsm || ddf superblock
 types? Given that mdadm supports most of the above RAID levels.


 Sure.

Ack, thanks. I'll work on uploading all the other patches, and will
work on improving grub-installer/imsmddf inter operability based on
upstream grub2 snapshot.


-- 
Regards,

Dimitri.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#699434: grub-installer: support for Intel Matrix Raid

2013-12-02 Thread Dmitrijs Ledkovs 
Hello Miquel,

On 31 January 2013 12:40, Miquel van Smoorenburg miqu...@debian.org wrote:
 This is by far the most intrusive patch - all the others are trivial.
 Description:

 - move make_device_map() upwards so that we can call it earlier
   in the script. Also, if $md_bootdev is set, add a (otherwise
   unused) (md/0) entry to device.map - otherwise grub-probe
   freaks out on upgrades.
 - add is_mdraid_bootable(). It checks if the device is a Intel
   Matrix Raid device, in RAID1 mode. If not it returns false.
   Otherwise it sets $md_dev, $md_devs, $md_super and $md_level,
and returns true.


Why is it limited to only imsm  raid1 ?

mdadm supports imsm superblock format levels 0,1,5,10  ddf
superblock format levels 0,1,3,4,5,1E,5E,5EE,6 with extensions of
there off (stripped, mirrored, concatenated and spanned).

Would it not be best to simply check for imsm || ddf superblock
types? Given that mdadm supports most of the above RAID levels.

All other changes look good.

Regards,

Dmitrijs.

 - if installing on an lvm device, it finds the underlying device,
   and checks if it's a single mdraid array. If so $disc_under_lvm
   is set.
 - if we're installing on a mdraid device and is_mdraid_bootable()
   is true, we set $disc_offered to the mdraid device, and set
   the $default_bootdev variable to all the underlying real disks.
 - when installing on an mdraid device, always create the grub device.map
 - do not try to guess the bootdevice if we already set $default_bootdev

 Note that effectively this patch does NOT change any existing
 behaviour if you are not trying to install on an Intel Matrix Raid
 RAID1 array. As that wasn't supported before anyway, there is
 no chance on regression.

 Thanks

 diff -ruN 00-ORIG/grub-installer-1.83/debian/changelog 
 grub-installer-1.83/debian/changelog
 --- 00-ORIG/grub-installer-1.83/debian/changelog2012-10-27 
 06:07:35.0 +
 +++ grub-installer-1.83/debian/changelog2013-01-29 15:03:58.462971942 
 +
 @@ -1,3 +1,12 @@
 +grub-installer (1.83+1) unstable; urgency=low
 +
 +  * Support for mdadm fakeraid (currently only IMSM formatted disks):
 +- also detect mdadm fakeraid underneath lvm, if it's just one array
 +- install grub on each disk of the underlying array
 +- add md/0 entry to device.map so grub-probe works (for upgrades etc)
 +
 + -- Miquel van Smoorenburg miqu...@debian.org  Tue, 29 Jan 2013 15:50:37 
 +0100
 +
  grub-installer (1.83) unstable; urgency=low

[ Milan Kupcevic ]
 diff -ruN 00-ORIG/grub-installer-1.83/grub-installer 
 grub-installer-1.83/grub-installer
 --- 00-ORIG/grub-installer-1.83/grub-installer  2012-10-25 01:01:57.0 
 +
 +++ grub-installer-1.83/grub-installer  2013-01-29 14:56:40.103337917 +
 @@ -217,6 +217,69 @@
 echo $ids
  }

 +make_device_map () {
 +   # If you're preseeding (hd0) et al, stop - preseed OS device names
 +   # instead.  However, for backward compatibility we ensure that a
 +   # device.map exists if you do this.
 +   [ $grub_version = grub2 ] || return 0
 +   [ ! -e $device_map ] || return 0
 +   local no_floppy
 +   case $1 in
 +   \(fd*|fd*)
 +   no_floppy=
 +   ;;
 +   *)
 +   no_floppy=--no-floppy
 +   ;;
 +   esac
 +   $chroot $ROOT grub-mkdevicemap $no_floppy
 +   if [ -n $md_bootdev ]
 +   then
 +   # This is lame- shouldn't need this, but otherwise
 +   # grub-probe freaks out.
 +   echo (md/0)$md_bootdev  $ROOT/boot/grub/device.map
 +   fi
 +}
 +
 +is_mdraid_bootable () {
 +   local device=$1
 +   local mddisk=${device#/dev/}
 +   local super
 +   local level=`cat /sys/block/$mddisk/md/level 2/dev/null ||:`
 +   local array_metadata=`cat /sys/block/$mddisk/md/metadata_version 
 2/dev/null ||:`
 +   case $array_metadata in
 +   external:/md*)
 +   local ctdisk=${array_metadata#external:/}
 +   ctdisk=${ctdisk%/*}
 +   super=`cat /sys/block/$ctdisk/md/metadata_version 
 2/dev/null ||:`
 +   super=${super#external:}
 +   ;;
 +   esac
 +   # Right now mdadm only supports imsm fakeraid, and grub doesn't know
 +   # about it at all but raid1 works regardless. So restrict to imsm + 
 raid1.
 +   if [ $level = raid1 ]  [ $super = imsm ]
 +   then
 +   # Find the disks that make up the array to that grub can
 +   # be installed onto all of them. As soon as grub supports
 +   # imsm raid we should be able to install directly to the
 +   # md device instead.
 +   local mddisks=`echo /sys/block/$mddisk/md/dev-* |
 +   sed -ne 's!/sys/block/md[0-9]\+/md/dev-\([^ 
 \*]\+\)!/dev/\1!pg'`
 +   if [ -n $mddisks ];

Bug#699434: grub-installer: support for Intel Matrix Raid

2013-12-02 Thread Dmitrijs Ledkovs 
Hello again,

On 31 January 2013 12:40, Miquel van Smoorenburg miqu...@debian.org wrote:
 Package: grub-installer
 Version: 183
 Tags: patch

 See metabug 699430, http://bugs.debian.org/699430 :
 (debian-installer: support for Intel Matrix Raid (RST, imsm)

   * Support for mdadm fakeraid (currently only IMSM formatted disks):
 - also detect mdadm fakeraid underneath lvm, if it's just one array
 - install grub on each disk of the underlying array
 - add md/0 entry to device.map so grub-probe works (for upgrades etc)


I've tried to perform a RAID0  RAID1 stripped IMSM installation,
without LVM layer and grub-installer failed both times. I will
experiment more, cause I was staring out with incomplete intel raid
with leftover partitions/bootloaders.

Regards,

Dmitrijs.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#699434: grub-installer: support for Intel Matrix Raid

2013-01-31 Thread Miquel van Smoorenburg 
Package: grub-installer
Version: 183
Tags: patch

See metabug 699430, http://bugs.debian.org/699430 :
(debian-installer: support for Intel Matrix Raid (RST, imsm)

  * Support for mdadm fakeraid (currently only IMSM formatted disks):
- also detect mdadm fakeraid underneath lvm, if it's just one array
- install grub on each disk of the underlying array
- add md/0 entry to device.map so grub-probe works (for upgrades etc)

This is by far the most intrusive patch - all the others are trivial.
Description:

- move make_device_map() upwards so that we can call it earlier
  in the script. Also, if $md_bootdev is set, add a (otherwise
  unused) (md/0) entry to device.map - otherwise grub-probe
  freaks out on upgrades.
- add is_mdraid_bootable(). It checks if the device is a Intel
  Matrix Raid device, in RAID1 mode. If not it returns false.
  Otherwise it sets $md_dev, $md_devs, $md_super and $md_level,
   and returns true.
- if installing on an lvm device, it finds the underlying device,
  and checks if it's a single mdraid array. If so $disc_under_lvm
  is set.
- if we're installing on a mdraid device and is_mdraid_bootable()
  is true, we set $disc_offered to the mdraid device, and set
  the $default_bootdev variable to all the underlying real disks.
- when installing on an mdraid device, always create the grub device.map
- do not try to guess the bootdevice if we already set $default_bootdev

Note that effectively this patch does NOT change any existing
behaviour if you are not trying to install on an Intel Matrix Raid
RAID1 array. As that wasn't supported before anyway, there is
no chance on regression.

Thanks

diff -ruN 00-ORIG/grub-installer-1.83/debian/changelog 
grub-installer-1.83/debian/changelog
--- 00-ORIG/grub-installer-1.83/debian/changelog2012-10-27 
06:07:35.0 +
+++ grub-installer-1.83/debian/changelog2013-01-29 15:03:58.462971942 
+
@@ -1,3 +1,12 @@
+grub-installer (1.83+1) unstable; urgency=low
+
+  * Support for mdadm fakeraid (currently only IMSM formatted disks):
+- also detect mdadm fakeraid underneath lvm, if it's just one array
+- install grub on each disk of the underlying array
+- add md/0 entry to device.map so grub-probe works (for upgrades etc)
+
+ -- Miquel van Smoorenburg miqu...@debian.org  Tue, 29 Jan 2013 15:50:37 
+0100
+
 grub-installer (1.83) unstable; urgency=low
 
   [ Milan Kupcevic ]
diff -ruN 00-ORIG/grub-installer-1.83/grub-installer 
grub-installer-1.83/grub-installer
--- 00-ORIG/grub-installer-1.83/grub-installer  2012-10-25 01:01:57.0 
+
+++ grub-installer-1.83/grub-installer  2013-01-29 14:56:40.103337917 +
@@ -217,6 +217,69 @@
echo $ids
 }
 
+make_device_map () {
+   # If you're preseeding (hd0) et al, stop - preseed OS device names
+   # instead.  However, for backward compatibility we ensure that a
+   # device.map exists if you do this.
+   [ $grub_version = grub2 ] || return 0
+   [ ! -e $device_map ] || return 0
+   local no_floppy
+   case $1 in
+   \(fd*|fd*)
+   no_floppy=
+   ;;
+   *)
+   no_floppy=--no-floppy
+   ;;
+   esac
+   $chroot $ROOT grub-mkdevicemap $no_floppy
+   if [ -n $md_bootdev ]
+   then
+   # This is lame- shouldn't need this, but otherwise
+   # grub-probe freaks out.
+   echo (md/0)$md_bootdev  $ROOT/boot/grub/device.map
+   fi
+}
+
+is_mdraid_bootable () {
+   local device=$1
+   local mddisk=${device#/dev/}
+   local super
+   local level=`cat /sys/block/$mddisk/md/level 2/dev/null ||:`
+   local array_metadata=`cat /sys/block/$mddisk/md/metadata_version 
2/dev/null ||:`
+   case $array_metadata in
+   external:/md*)
+   local ctdisk=${array_metadata#external:/}
+   ctdisk=${ctdisk%/*}
+   super=`cat /sys/block/$ctdisk/md/metadata_version 
2/dev/null ||:`
+   super=${super#external:}
+   ;;
+   esac
+   # Right now mdadm only supports imsm fakeraid, and grub doesn't know
+   # about it at all but raid1 works regardless. So restrict to imsm + 
raid1.
+   if [ $level = raid1 ]  [ $super = imsm ]
+   then
+   # Find the disks that make up the array to that grub can
+   # be installed onto all of them. As soon as grub supports
+   # imsm raid we should be able to install directly to the
+   # md device instead.
+   local mddisks=`echo /sys/block/$mddisk/md/dev-* |
+   sed -ne 's!/sys/block/md[0-9]\+/md/dev-\([^ 
\*]\+\)!/dev/\1!pg'`
+   if [ -n $mddisks ]; then
+   md_devs=$mddisks
+   md_super=$super
+   md_level=$level
+   for