Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
Hi Mike, Michael Milligan wrote: Thanks for fixing! I will grab a daily image with partman 43 in it and give it a go. Be careful that you might hit http://bugs.debian.org/780115 next. If you create a pool on only one disk at first, you can easily attach another later as a mirror, after installing. Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
Yes, it was very annoying that I couldn't fix the issue by turning off all the feature@ things as once they are in use, you can't turn them off. Had to rebuild from scratch. Thanks for fixing! I will grab a daily image with partman 43 in it and give it a go. Regards, Mike On 02/27/2015 04:34 PM, Steven Chamberlain wrote: tags 775395 + confirmed user 775395 debian-...@lists.debian.org usertags 775395 + kfreebsd thanks Hi Mike, On 09/02/15 03:02, Michael Milligan wrote: This may help... 10.1 added a bunch of feature flags that grub may need to now support in the ZFS code and are probably tripping this all up. You're absolutely right, some of the new feature flags are breaking GRUB; I just confirmed it in a fresh test-install with ZFS mounted on /boot This is my fault; I was so sure that GRUB in jessie supported these, that I requested that `zpool create` no longer use -d: http://anonscm.debian.org/cgit/d-i/partman-zfs.git/commit/?id=46b42715da4ee17c95762129600958ff75f6d02d GRUB *does* support feature@lz4_compress, but otherwise, only features that are 'read-only compatible' according to zpool-features(7): http://man.freebsd.org/zpool-features Here is the list of not read-only compatible' features that are supported by GRUB - currently only lz4_compress: http://anonscm.debian.org/cgit/pkg-grub/grub.git/tree/grub-core/fs/zfs/zfs.c#n277 The most problematic ones are hole_birth and embedded_data, because they go active as soon as enabled. Some of the others wouldn't initially be a problem, until that functionality is used, but it's not safe to enable those by default. I think most of the new features are valuable and we should keep them if possible, most certainly lz4_compress (which can be used to compress metadata, even if you don't set compression=on for a dataset). So I suggest we use -d but still enable lz4 and all read-only compatible features. (`zpool create` without -d would enable *all* features by default). Thanks for reporting this! Regards,
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
tags 775395 + patch pending
thanks
Hi KiBi,
Could we have the attached fix for partman-zfs in sid please?
This would be an RC bug, except that it is kfreebsd-any.
Furthermore, could I step in as maintainer (add myself to Uploaders) for
this and just upload it myself? I believe I'd need a dak dm
authorisation for that.
Thanks,
Regards,
--
Steven Chamberlain
ste...@pyro.eu.org
From bd30e842861d39b66936716e84dd34c799a69b10 Mon Sep 17 00:00:00 2001
From: Steven Chamberlain ste...@pyro.eu.org
Date: Sun, 8 Mar 2015 13:11:58 +
Subject: [PATCH] Only enable the zpool features GRUB 2.02 supports (Closes:
#775395)
Do not enable zpool features by default, except those that are
read-only compatible, and lz4_compress which is supported by
grub-pc 2.02~ (Closes: #775395)
---
debian/changelog | 9 +
lib/zfs-base.sh | 12 +++-
2 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/debian/changelog b/debian/changelog
index 2ddf4f1..a975046 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+partman-zfs (43) unstable; urgency=medium
+
+ * Team upload.
+ * Do not enable zpool features by default, except those that are
+read-only compatible, and lz4_compress which is supported by
+grub-pc 2.02~ (Closes: #775395)
+
+ -- Steven Chamberlain ste...@pyro.eu.org Fri, 27 Feb 2015 23:50:29 +
+
partman-zfs (42) unstable; urgency=low
[ Updated translations ]
diff --git a/lib/zfs-base.sh b/lib/zfs-base.sh
index 636902e..3b7b78a 100644
--- a/lib/zfs-base.sh
+++ b/lib/zfs-base.sh
@@ -386,7 +386,17 @@ vg_create() {
vg=$1
shift
- log-output -t partman-zfs zpool create -f -m none -o altroot=/target $vg $* || return 1
+ # Feature explicitly supported by grub-pc 2.02~, see
+ # spa_feature_names[] in grub-core/fs/zfs/zfs.c
+ features=-o feature@lz4_compress=enabled
+
+ # Read-only compatible features, according to zpool-features(7)
+ for feature in async_destroy empty_bpobj spacemap_histogram
+ do
+ features=$features -o feature@${feature}=enabled
+ done
+
+ log-output -t partman-zfs zpool create -f -m none -d $features -o altroot=/target $vg $* || return 1
# Some ZFS versions don't create cachefile when -o altroot is used.
# Request it explicitly.
--
1.8.4.rc3
signature.asc
Description: Digital signature
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
Hi, Steven Chamberlain ste...@pyro.eu.org (2015-03-08): tags 775395 + patch pending thanks Hi KiBi, Could we have the attached fix for partman-zfs in sid please? This would be an RC bug, except that it is kfreebsd-any. Sure! Furthermore, could I step in as maintainer (add myself to Uploaders) for this and just upload it myself? I believe I'd need a dak dm authorisation for that. That would be ideal. I've tried to upload a .commands file to that effect, I'll seek to getting it right if my first attempt doesn't work (it's been a while since last time I needed to do so). Thanks! Mraw, KiBi. signature.asc Description: Digital signature
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
tags 775395 + confirmed user 775395 debian-...@lists.debian.org usertags 775395 + kfreebsd thanks Hi Mike, On 09/02/15 03:02, Michael Milligan wrote: This may help... 10.1 added a bunch of feature flags that grub may need to now support in the ZFS code and are probably tripping this all up. You're absolutely right, some of the new feature flags are breaking GRUB; I just confirmed it in a fresh test-install with ZFS mounted on /boot This is my fault; I was so sure that GRUB in jessie supported these, that I requested that `zpool create` no longer use -d: http://anonscm.debian.org/cgit/d-i/partman-zfs.git/commit/?id=46b42715da4ee17c95762129600958ff75f6d02d GRUB *does* support feature@lz4_compress, but otherwise, only features that are 'read-only compatible' according to zpool-features(7): http://man.freebsd.org/zpool-features Here is the list of not read-only compatible' features that are supported by GRUB - currently only lz4_compress: http://anonscm.debian.org/cgit/pkg-grub/grub.git/tree/grub-core/fs/zfs/zfs.c#n277 The most problematic ones are hole_birth and embedded_data, because they go active as soon as enabled. Some of the others wouldn't initially be a problem, until that functionality is used, but it's not safe to enable those by default. I think most of the new features are valuable and we should keep them if possible, most certainly lz4_compress (which can be used to compress metadata, even if you don't set compression=on for a dataset). So I suggest we use -d but still enable lz4 and all read-only compatible features. (`zpool create` without -d would enable *all* features by default). Thanks for reporting this! Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
This may help... 10.1 added a bunch of feature flags that grub may need to now support in the ZFS code and are probably tripping this all up. http://www.daemon-security.com/blog-2014.html Regards, Mike On 01/27/2015 02:19 PM, Steven Chamberlain wrote: Control: severity -1 important Michael Milligan wrote: Package: partman-zfs Version: 42 Severity: grave Tags: d-i Justification: renders package unusable for ZFS-based install (which is probably the reason someone is trying Debian/kFreeBSD .. to use ZFS) Since this is a kfreebsd-any package, and we're not part of the official stable release, I have to lower this to non-RC severity. We should still fix it for the GNU/kFreeBSD release though. Regards,
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
Control: severity -1 important Michael Milligan wrote: Package: partman-zfs Version: 42 Severity: grave Tags: d-i Justification: renders package unusable for ZFS-based install (which is probably the reason someone is trying Debian/kFreeBSD .. to use ZFS) Since this is a kfreebsd-any package, and we're not part of the official stable release, I have to lower this to non-RC severity. We should still fix it for the GNU/kFreeBSD release though. Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
Hi, Michael Milligan wrote: But installing Grub failed with unknown filesystem. In troubleshooting the issue, it seems grub2 (version currently is 2.02~beta2-19) does not recognize the feature@lz4_compress option (and had been previously reported) of the ZFS pools that kFreeBSD kernel 10.1 creates. Thanks for the report. It is actually expected that grub2 since 2.02 should understand these feature flags, including LZ4 compression. I thought it was working for me when I last tested. Hopefully this can be figured out without having to default to using -d (disabling new features including LZ4 metadata compression). Regards, -- Steven Chamberlain ste...@pyro.eu.org -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
On 01/27/2015 02:02 PM, Steven Chamberlain wrote: Hi, Michael Milligan wrote: But installing Grub failed with unknown filesystem. In troubleshooting the issue, it seems grub2 (version currently is 2.02~beta2-19) does not recognize the feature@lz4_compress option (and had been previously reported) of the ZFS pools that kFreeBSD kernel 10.1 creates. Thanks for the report. It is actually expected that grub2 since 2.02 should understand these feature flags, including LZ4 compression. I thought it was working for me when I last tested. Hopefully this can be figured out without having to default to using -d (disabling new features including LZ4 metadata compression). Yeah, failed for me using 10.1 kfreebsd image... I did not have time to retry it with each individual feature flag turned on/off to find the exact combination it was choking on, but at least one of them is still not recognized by grub-probe, not necessarily lz4_compress. Regards, Mike -- Michael Milligan - mi...@acmeps.com Acme Professional Services LLC -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror
Package: partman-zfs Version: 42 Severity: grave Tags: d-i Justification: renders package unusable for ZFS-based install (which is probably the reason someone is trying Debian/kFreeBSD .. to use ZFS) Dear Maintainer, Tried the 2015-01-12 nightly mini.iso Jessie image. In the partition setup via expert install, selected all disks to use with ZFS -- partitioned 3 drives into three partitions (see below for scheme) -- then proceeded to create two ZFS pools from these, a b pool from partition 1 as a mirror across three drives, then a z pool from partition 2 as raidz across three drives. It happily did all this, installed the base system, etc... -- root@bear:~# parted /dev/da0 GNU Parted 3.2 Using /dev/da0 Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) print Model: VMware Virtual disk (scsi) Disk /dev/da0: 42.9GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End SizeFile system Name Flags 3 17.4kB 1049kB 1031kB bios_grub 1 1049kB 512MB 511MB 2 512MB 42.9GB 42.4GB zfs (parted) -- But installing Grub failed with unknown filesystem. In troubleshooting the issue, it seems grub2 (version currently is 2.02~beta2-19) does not recognize the feature@lz4_compress option (and had been previously reported) of the ZFS pools that kFreeBSD kernel 10.1 creates. And there's no way to disable it once the pool is created. What I ended up doing is recreating everything again, this time interrupting and adding the -d flag to the pool creation command (done manually) to leave all the feature@* options disabled for building the system. This worked. Installation of Grub as the boot loader was successful and the system booted using /boot (and /lib/modules) as volumes on the b pool, which is a simple ZFS mirror. Please consider adding the -d flag in partman-zfs for Jessie, i.e., # zpool create -d pool dev-spec And then perhaps turning on feature@ flags that are known to work with grub for booting. Here's what my pools look like right now and all is well: -- root@bear:~# zpool status pool: b state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: none requested config: NAMESTATE READ WRITE CKSUM b ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da0p1 ONLINE 0 0 0 da1p1 ONLINE 0 0 0 da2p1 ONLINE 0 0 0 errors: No known data errors pool: z state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: none requested config: NAMESTATE READ WRITE CKSUM z ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0p2 ONLINE 0 0 0 da1p2 ONLINE 0 0 0 da2p2 ONLINE 0 0 0 errors: No known data errors root@bear:~# zpool get all b NAME PROPERTY VALUE SOURCE b size 480M- b capacity 12% - b altroot- default b health ONLINE - b guid 13665561835741433528default b version- default b bootfs - default b delegation on default b autoreplaceoff default b cachefile - default b failmode waitdefault b listsnapshots off default b autoexpand off default b dedupditto 0 default b dedupratio 1.00x - b free 420M- b allocated 60.2M - b readonly off - b comment-
