Re: ZFS feature flags
On 10/19/19 4:43 AM, Colin Watson wrote: > I don't have much to say about most of this, but noticed this bit: > > On Thu, Oct 17, 2019 at 09:48:42PM -0500, Richard Laager wrote: >> The same implementation could (and if I have my way, will) be used to >> provide a features=grub mask. This would be used for the boot pool >> (bpool) to limit it to the features supported by GRUB. This would avoid >> the dangerous message in "zpool status" which tells you to run "zpool >> upgrade" on your bpool which would then break booting from it. > > Isn't that a dubious and confusing way to spell it? I'm certainly open to a different name. > After all, like any > other ZFS implementation, GRUB's ZFS implementation has gained features > over time, and it wouldn't surprise me if it continued to do so. It > sounds like you'd need a set of versioned features for this as well as > for features=portable; but it's not clear how the decision of when to > promote a feature set to the unversioned level would be made, > particularly given GRUB's rather slow and unpredictable release cycle > and the widespread practice of backporting features by distribution > maintainers. The proposal, as I understand it, is that the unversioned features=portable is defined as the set of features common to all the latest releases of specific "tier 1" ZFS platforms: FreeBSD, illumos-gate*, Ubuntu LTS, and ZoL**. A feature is promoted into the unversioned features=portable when it is available in all of those platforms' latest releases. As new platform releases push out older releases, the set of common features expands and the definition of features=portable can be updated. * illumos-gate, which I think doesn't have releases, was going to use a time-based cutoff. ** In practice, unless Ubuntu backports something from git master or writes their own ZFS feature, Ubuntu LTS will always be a subset of the latest ZoL release. Thus, Ubuntu LTS will be the limiting factor for the common set, not the latest ZoL release. The versioned features=portable- are simply features=portable frozen in time on January 1 of . The versioned ones are not driving anything design-wise, so they are mostly irrelevant for this discussion. If people also want versioned grub-, that's trivial to do. I think there are two approaches to take for features=grub. Originally, I was thinking that features=grub would be the set of features that are supported by GRUB on the current system. This solves the `zpool status` / `zpool upgrade` issues for the boot pool, which is the main goal. However, that definition could hamper dual-booting scenarios, so features=grub should probably be similar to features=portable. It should be the set of features common to all the latest releases of specific "tier 1" ZFS platforms that use GRUB (specifically GRUB 2, not GRUB Legacy) as their bootloader. I don't think that FreeBSD uses GRUB, and neither illumos nor ZoL are whole distros, so in practice this is just Ubuntu LTS at the moment. It may be desirable to expand this to include other distros where root-on-ZFS is a thing, like Arch and Gentoo. In other words, features=grub should be the features supported by GRUB in the latest release of whichever distros people might realistically want to dual-boot off the bpool. Whether the feature arrives in Ubuntu 20.04's GRUB by GRUB release or distro backport is irrelevant. The only thing that matters is which features are supported. An additional subtlety is that GRUB opens the pool read-only, so every read-only feature is "supported" from a GRUB perspective. So what GRUB "supports" on, for example, Ubuntu 20.04 is the union of the features-for-write actually supported by GRUB in Ubuntu 20.04, plus all read-only compatible features supported by the kernel ZFS implementation on Ubuntu 20.04. (Dependencies have to kept in mind here. If feature X is read-only compatible but depends on feature Y which is not read-only compatible, and feature Y is not supported by GRUB, then feature X is not supported by GRUB.) -- Richard -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: ZFS feature flags
I don't have much to say about most of this, but noticed this bit: On Thu, Oct 17, 2019 at 09:48:42PM -0500, Richard Laager wrote: > The same implementation could (and if I have my way, will) be used to > provide a features=grub mask. This would be used for the boot pool > (bpool) to limit it to the features supported by GRUB. This would avoid > the dangerous message in "zpool status" which tells you to run "zpool > upgrade" on your bpool which would then break booting from it. Isn't that a dubious and confusing way to spell it? After all, like any other ZFS implementation, GRUB's ZFS implementation has gained features over time, and it wouldn't surprise me if it continued to do so. It sounds like you'd need a set of versioned features for this as well as for features=portable; but it's not clear how the decision of when to promote a feature set to the unversioned level would be made, particularly given GRUB's rather slow and unpredictable release cycle and the widespread practice of backporting features by distribution maintainers. -- Colin Watson [cjwat...@ubuntu.com] -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: ZFS feature flags
TL;DR: I think the installer should default to all features, so I think the current behavior is correct. I would support the installer gaining an option to default to a subset of features (e.g. using the future features=portable mechanism) to make the pool more portable. If the desktop GUI partitioning tool gains support for creating ZFS pools, I would like to see a similar option there, probably with the opposite default (i.e. defaulting to portable). I was asked to comment on this thread. I maintain the upstream root-on-ZFS HOWTO which was used as a starting point for the Ubuntu installer design. However, unless otherwise noted, my comments are my own and do not necessarily reflect a ZoL/OpenZFS consensus. An incrementing version number was simple. It worked great when ZFS development was centralized at Sun only. When they opened things up with OpenSolaris, it worked okay enough. But as ZFS development started happening in parallel at various companies and projects, it was no longer feasible to have a simple integer version number. If company A develops feature X as zpool version 29 and company B develops feature Y as zpool version 29, you have a problem. The feature flags allow for parallel development and for features to land on different platforms in any order. The "read-only compatible" flag on features is a nice detail, too. It allows using a lot more features on the boot pool than would otherwise be the case. A lot of new features originate in ZFS-on-Linux, but features also originate from Illumos and FreeBSD. (Oracle is not involved in OpenZFS development in any way. They continue to have their own, closed-source ZFS implementation on Solaris which is not interoperable with pool features at all.) A downside of parallel development is that not all platforms are guaranteed to have the same features, or land the features in the same order. So cross-platform compatibility is definitely more complicated now. If you care about cross-platform compatibility for a particular pool, you have to limit your enabled features to the common denominator among the platforms you wish to support with that particular pool. Alternatively, you can use a minimal set of old features, or even version=28, but then you miss out on useful enhancements. There has been recent (March 2019) talk upstream of adding a "features=portable" property. This would act as a feature mask, limiting the enabled features (on zpool create and zpool upgrade) to a set that is portable across "tier 1" OpenZFS platforms. The exact semantics of what counts as a "tier 1" platform was part of that discussion; the notes (remember this was March) say "FreeBSD (11.X), ZoL (0.7.X), illumos-gate (from 1 year ago)". Further discussion was that the latest Ubuntu LTS would be considered a "tier 1" platform. The idea is that the definition of features=portable would be rolling, with unchanging features=portable- (e.g. portable-2018, portable-2019) complementing it as another option. This would significantly simplify configuration of pool features, but is not yet implemented. The same implementation could (and if I have my way, will) be used to provide a features=grub mask. This would be used for the boot pool (bpool) to limit it to the features supported by GRUB. This would avoid the dangerous message in "zpool status" which tells you to run "zpool upgrade" on your bpool which would then break booting from it. If/when features=portable lands, it's probably reasonable to expose that in the installer for the root pool. This could be a checkbox, for example. It might read something like, "Disable some pool features to maximize compatibility with other OSes" or "Maximize compatibility with other OSes by disabling some pool features". The default could go either way, but based on the existing precedent and my assumption that the majority use case is a single OS install, I would have the installer _default_ to the "more features, less portable" trade-off. Those who want the "less features, more portable" trade-off could change the checkbox. I think there's a stronger argument for features=portable (or otherwise limiting features) as the default for data pools than for the root pool. Data pools are more likely to move between systems than a root pool. This would be especially true for something like an external hard drive. They're also likely to be larger, making pool rebuilds more burdensome. For a _new install_ on a root pool, the need for portable is far less; dual booting seems like the most obvious and important case, which is a good use case for the checkbox. If the desktop GUI partitioning tool gains support for creating ZFS pools, I think features=portable would be a reasonable default there (again with a checkbox), under the assumption that it is creating data pools. Whatever the default, there are obvious exceptions. For example, if/when the installer supports ZFS encryption, if the user selects encryption, the installer must enable the
