Re: grub on ZRaid2 - SOLUTION
Hi, found probable cause why grub-installer fails with ZRAID. Bug #653929, I think the patch isn't applied yet. I wrote my own grub.cfg for the "4 drive boot-mirror root-zraid2" as update-grub failed: insmod part_msdos insmod zfs set boot=(hd0,msdos1)/boot set root=(hd0,msdos2) kfreebsd $boot/@/kfreebsd-9.0-2-amd64.gz kfreebsd_module_elf /root/@/lib/modules/9.0-2-amd64/opensolaris.ko kfreebsd_module $boot/@/zfs/zpool.cache type=/boot/zfs/zpool.cache kfreebsd_module_elf /root/@/lib/modules/9.0-2-amd64/zfs.ko set kFreeBSD.vfs.root.mountfrom=zfs:tank/root set kFreeBSD.vfs.root.mountfrom.options=rw boot Thanks to all who helped, Klaus -- k...@gmx.de -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ioenkagbbnkhenkpbeimgepbglab.k...@gmx.de
Re: grub on ZRaid2
Hi, On Mon, 21 Oct 2013 22:05:52 -0500, Jeff Epler wrote: > You might consider doing what I did: Make two zfs partitions per disk. > Create a mirrored zpool from the first, smaller partitions; place files > needed for booting there. Create a raidz(2) pool from the bigger > partitions and use it for bulk storage. In fact, I've convinced myself > that the additional redundancy in the boot pool is a feature. Nice idea! Did a 256MB mirror and a 2000GB pool on all 4 drives. grub-install did ok, but update-grub failed. Oct 22 04:56:04 grub-installer: info: Installing grub on '/dev/ada0' Oct 22 04:56:04 grub-installer: info: grub-install supports --no-floppy Oct 22 04:56:04 grub-installer: info: Running chroot /target rub-install --no-floppy --force "/dev/ada0" Oct 22 04:56:07 grub-installer: Installation finished. No error reported. Oct 22 04:56:07 grub-installer: info: grub-install ran successfully Oct 22 04:56:07 /bin/in-target: warning: /target/etc/mtab won't be updated since it is a symlink. Oct 22 04:56:07 in-target: /usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?). Oct 22 04:56:07 grub-installer: error: Running 'update-grub' failed. Oct 22 05:17:38 main-menu[4138]: (process:54620): grub-probe: error: cannot find a device for / (is /dev mounted?). Oct 22 05:17:38 main-menu[4138]: (process:54620): Error: Could not stat device /dev/disc - No such file or directory. Oct 22 05:17:38 main-menu[4138]: (process:54620): Warning: Could not get identity of device /dev/ada0 - Inappropriate ioctl for device Oct 22 05:17:38 main-menu[4138]: WARNING **: Configuring 'grub-installer' failed with error code 1 Oct 22 05:17:38 main-menu[4138]: WARNING **: Menu item 'grub-installer' failed. Regards, Klaus -- k...@gmx.de -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ioenkagbbnkhenkpbeimiendglab.k...@gmx.de
Re: grub on ZRaid2
You might consider doing what I did: Make two zfs partitions per disk. Create a mirrored zpool from the first, smaller partitions; place files needed for booting there. Create a raidz(2) pool from the bigger partitions and use it for bulk storage. In fact, I've convinced myself that the additional redundancy in the boot pool is a feature. Jeff -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131022030552.ga2...@unpythonic.net
Re: grub on ZRaid2
Hi, on Mon, 21 Oct 2013 00:18:13 +0200, Robert Millan wrote: > Did you create this pool yourself? Yes, I'm using "Automated install, FreeDSB kernel 9" with manual partioning. > Debian installer always creates zpools within MSDOS partitions, in order > to leave room for embedding. If you created a full-disk pool, GRUB > doesn't support this setup AFAIK. Besides bug #651624 using full-disk pools, I created a 4 drive full-disk pool with a logical volume mounted to /. The drives have a MSDOS partition table with 1 primary partition formatted as physical volume for ZFS. Multidisk mode for the ZFS pool: - Striped GRUB installation is fine, booting via GRUB also. - Mirror GRUB installation is fine, booting via GRUB also. - RAID-Z 1GRUB installation fails. - RAID-Z 2GRUB installation fails. BTW I never used ZFS before, so I created a 4 drive full-disk Mirror pool and removed drive 2-4 after installation. System boots from drive 1 in any slot... nice! Need to install GRUB on drive 2-4. Regards, Klaus -- k...@gmx.de -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ioenkagbbnkhenkpbeimgemnglab.k...@gmx.de
Re: grub on ZRaid2
Hi, On Mon, 21 Oct 2013 12:12:47 +0100, Steven Chamberlain wrote: > If using MSDOS, partman will automatically leave space before the first > partition for the bootloader embedding area. And to avoid Bug #651624: > you should also try to leave 1MiB or more free space at the end of the > disk - if you have 198 MB space then try to create a 197 MB ZFS > partition. Check in partman that there is "FREE SPACE" still shown. I created an empty MSDOS partition table for each of the 4 discs. Then I created a 2000GiB primary partition at the beginning of the available space and set the partition usage to "physical volume for ZFS", not bootable. This left me with about 400MB of FREE SPACE at the end of each drive. /dev/ada0 - 2.0 TB Generic IDE #1 primary2.0 TBK zfs pri/log 398.5 MB FREE SPACE /dev/ada1 - ... I created a new ZFS pool 'tank' as ZRAID2. Then a ZFS logical volume 'tank/root' with a mountpoint to '/'. ZFS pool tank, volume root - 3.7 TB Unknown #1 3.7 TBf zfs / 512.0 B unusable After the base system was installed grub-installer failed, shouldn't it looking for /target/boot/grub ? Oct 21 13:24:11 grub-installer: info: Installing grub on '/dev/ada0' Oct 21 13:24:11 grub-installer: info: grub-install supports --no-floppy Oct 21 13:24:11 grub-installer: info: Running chroot /target rub-install --no-floppy --force "/dev/ada0" Oct 21 13:24:12 grub-installer: /usr/sbin/grub-probe: error: cannot find a device for /boot/grub (is /dev mounted?). Oct 21 13:24:12 grub-installer: error: Running 'grub-install --no-floppy --force "/dev/ada0"' failed. > Or instead, you could use GPT partitions. Actually I would recommend > them for large (>=2TB) drives. If using GPT, first create a tiny 1MiB > BIOS Boot partition (for GRUB), then create the ZFS partition which > could use 100% of the remaining space. You will want to do the same on > each disk, and later tell GRUB to install bootcode onto all disks. I created GPT partitions on all 4 drives. /dev/ada0 - 2.0 TB Generic IDE 1.0 MB FREE SPACE #1 1.0 MBf biosgrub #2 2.0 TBK zfs 73.2 kB FREE SPACE /dev/ada1 - ... After the ZFS pool and the ZFS partition was created, the base system was installed. grub-installer failed again. Oct 21 15:02:05 grub-installer: info: Installing grub on '/dev/ada0' Oct 21 15:02:05 grub-installer: info: grub-install supports --no-floppy Oct 21 15:02:05 grub-installer: info: Running chroot /target rub-install --no-floppy --force "/dev/ada0" Oct 21 15:02:05 grub-installer: /usr/sbin/grub-probe: error: cannot find a device for /boot/grub (is /dev mounted?). Oct 21 15:02:05 grub-installer: error: Running 'grub-install --no-floppy --force "/dev/ada0"' failed. Regards, Klaus -- k...@gmx.de -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ioenkagbbnkhenkpbeimkeloglab.k...@gmx.de
Re: grub on ZRaid2
Hi, On 21/10/13 02:25, Klaus Melchior wrote: > On Mon, 21 Oct 2013 00:18:13 +0200, Robert Millan wrote: >> Debian installer always creates zpools within MSDOS partitions, in order >> to leave room for embedding. If you created a full-disk pool, GRUB >> doesn't support this setup AFAIK. > > What is a MSDOS partition? It's a partition, on a disk having an MSDOS partition table. For ZFS it will also need to be an MSDOS Primary partition. Primary partitions are typically numbered 1 to 4. > Is partman-base able to create MSDOS partitions for a ZRAID? Yes, you can choose the partition table type by selecting the "adaN... Generic IDE" row for each disk. If using MSDOS, partman will automatically leave space before the first partition for the bootloader embedding area. And to avoid Bug #651624: you should also try to leave 1MiB or more free space at the end of the disk - if you have 198 MB space then try to create a 197 MB ZFS partition. Check in partman that there is "FREE SPACE" still shown. Actually I would leave as much as 256MiB space at the end of each disk, in case you have problems and want to create a separate /boot partition later. Here is an example of a disk having three MSDOS Primary partitions: https://wiki.debian.org/Debian_GNU/kFreeBSD/Tips#Example Or instead, you could use GPT partitions. Actually I would recommend them for large (>=2TB) drives. If using GPT, first create a tiny 1MiB BIOS Boot partition (for GRUB), then create the ZFS partition which could use 100% of the remaining space. You will want to do the same on each disk, and later tell GRUB to install bootcode onto all disks. Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52650c2f.4080...@pyro.eu.org
Re: Re: grub on ZRaid2
In continuation to my previous post: On Mon, 21 Oct 2013 00:18:13 +0200, Robert Millan wrote: > Debian installer always creates zpools within MSDOS partitions, in order > to leave room for embedding. If you created a full-disk pool, GRUB > doesn't support this setup AFAIK. What is a MSDOS partition? A disc with an empty MSDOS partition table? Or is the partition table filled with empty primary/logical partitions? Is partman-base able to create MSDOS partitions for a ZRAID? -- k...@gmx.de -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ioenkagbbnkhenkpbeimaekcglab.k...@gmx.de
Re: Re: grub on ZRaid2
Hi, on Mon, 21 Oct 2013 00:18:13 +0200, Robert Millan wrote: > Did you create this pool yourself? No, partman-base did while expert installation. But I did a ZFS stripe installation prior, so the partitions were "physical volume for ZFS". /dev/ada0 - 2.0 TB Generic IDE #1 primary 2.0 TBK zfs /dev/ada1 - 2.0 TB Generic IDE #1 primary 2.0 TBK zfs /dev/ada2 - 2.0 TB Generic IDE #1 primary 2.0 TBK zfs /dev/ada3 - 2.0 TB Generic IDE #1 primary 2.0 TBK zfs > Debian installer always creates zpools within MSDOS partitions, in order > to leave room for embedding. If you created a full-disk pool, GRUB > doesn't support this setup AFAIK. My new try... Deleting the partition tables of the 4 discs. /dev/ada0 - 2.0 TB Generic IDE pri/log 2.0 TB FREE SPACE /dev/ada1 - 2.0 TB Generic IDE pri/log 2.0 TB FREE SPACE /dev/ada2 - 2.0 TB Generic IDE pri/log 2.0 TB FREE SPACE /dev/ada3 - 2.0 TB Generic IDE pri/log 2.0 TB FREE SPACE After selecting , , name: "tank" and selecting the 4 devices for the new ZFS pool, partman-base created a ZFS stripe: NAME USED AVAIL REFER MOUNTPOINT tank 106K 7.14T31K none There was no chance to select "Striped, Mirror, RAID-Z"! If I delete the ZFS pool and re-created it without changing the partitions, I'm able to select "RAID-Z" with parity level 2: NAME USED AVAIL REFER MOUNTPOINT tank 139K 3.55T 44.8K none But this will fail grub-installer. What is a MSDOS partition -- k...@gmx.de smime.p7s Description: S/MIME cryptographic signature
Re: grub on ZRaid2
Hi, On 20/10/2013 22:54, Klaus Melchior wrote: > Hi, > > using "Debian GNU/kFreeBSD 7.2.0 _Wheezy_ DVD-1 20131012-20:22" on 4x 2TB > drive amd64 system. I'm using the expert install with kernel 9 and > installing kernel 9.0.2 on the target. > > The Installation of GRUB on /dev/ada0 works on 4x 2TB ZFS stripe. > > But GRUB installation failes on ZRAID2 with 1 logical volume tank/root > mounted as /: >> Install GRUB to the master boot record: >> Unable to install GRUB in /dev/ada0 > Did you create this pool yourself? Debian installer always creates zpools within MSDOS partitions, in order to leave room for embedding. If you created a full-disk pool, GRUB doesn't support this setup AFAIK. -- Robert Millan -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/526456a5.3010...@debian.org
grub on ZRaid2
Hi, using "Debian GNU/kFreeBSD 7.2.0 _Wheezy_ DVD-1 20131012-20:22" on 4x 2TB drive amd64 system. I'm using the expert install with kernel 9 and installing kernel 9.0.2 on the target. The Installation of GRUB on /dev/ada0 works on 4x 2TB ZFS stripe. But GRUB installation failes on ZRAID2 with 1 logical volume tank/root mounted as /: > Install GRUB to the master boot record: > Unable to install GRUB in /dev/ada0 Regards, Klaus -- k...@gmx.de -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ioenkagbbnkhenkpbeimcejmglab.k...@gmx.de
