Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
Am Dienstag, dem 14.02.2023 um 16:53 -0500 schrieb Theodore Ts'o: [snip] Your arrogant and ignorant attitude is frustrating, to say the least. You don't care about the mess you create, for a feature, that probably only a handful of people will ever need (I did a quick search and didn't find anything regarding this feature - so why turn it on by default then?). But yet you have to make it the default and break running toolchains and methods. Talking to you is pointless. You cost me hours of useless work yesterday and today because you ignore the rules we have set out as a project to not frustrate each other. I have reported this to the release team now. Daniel
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
Am Dienstag, dem 14.02.2023 um 12:34 -0500 schrieb Theodore Ts'o: > [..] > In any case, a version of grub that will support the csum_seed feature > will be landing in Bookworm in just a few days. So at that point, > you'll be able to create VM images for Bookworm and Sid that will work > with the e2fsprogs in sid. The current plan of record is that it will > only be at that point that e2fsprogs will be allowed to migrate into > Bookworm. As soon as this version hits testing, you have successfully disabled the last working environment to use vmdb2 to create images of Ubuntu and Debian. As soon as this version hits Testing, one then can no longer build images for either any Ubuntu release nor Debian Bullseye or older. I can then only build images for Bookworm and Sid. Please think about that. You *seriously* break the debootstrap method of installing Debian or Ubuntu. vmdb2 ist just another tool that is broken by now, a tool that I use very often. I explained the impacts of what you are doing often enough now. By now the impact should be clear. And you are still not dealing with the aftermath of your intended action?! You haven't done a proper transition. You haven't given any affected toolchain the necessary time to adopt. You haven't documented how to disable that breaking change when creating filesystems for distributions where grub does not support this (there is not even a NEWS entry). You haven't even announced that breaking change. And yet you are going to break one of our two standard methods of installing Debian or Ubuntu. How is any of what you have been saying so far addressing any of these issues? > [..] > By the way, in the case of the csum_seed feature, it's pretty > straightforward to just run "tune2fs -O ^metadata_csum_seed > /tmp/boot.img". If the UUID has been changed since the file system > was created, you'll have to do this while the file system is > unmounted > and it will take a few seconds, but that's almost certainly not the > case with vmdb2. Well, how do you intend to distribute that information, so people who have to use the deboostrap method to install a Debian or Ubuntu release with a grub not supporting csum_seed (basically all existing releases, except for the upcoming Bookworm) know that? I do not understand why you are pushing 1.47.0 over 1.46.6, which you had uploaded just five days before the former. You haven't even presented a reason yet. Regards, Daniel
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
There is another issue with vmdb2 if you are using XFS. Starting with
xfsprogs 5.15 (which is already in testing), bigtime is enabled by
default, so that newly created XFS file systems won't be subject to
timestamp overflow in 2038. Grub didn't land support for this feature
until 8b1e5d1936ff ("fs/xfs: Add bigtime incompat feature support") in
May 2021, despite the fact that XFS has had this feature for years and
years and years.
So if you aren't using the latest security fixes, and you are using
XFS as the boot partition --- it won't work on buster and bullseye.
"Fortunately", there were were massive number security vulnerabilities
in grub2 which forced a backport of grub2 2.06 to bullseye and buster,
so if you have the security updates enabled, you'll probably be OK ---
but it was only because of massive number of security problems forced
that backport.
In any case, a version of grub that will support the csum_seed feature
will be landing in Bookworm in just a few days. So at that point,
you'll be able to create VM images for Bookworm and Sid that will work
with the e2fsprogs in sid. The current plan of record is that it will
only be at that point that e2fsprogs will be allowed to migrate into
Bookworm.
For slowly moving upstreams like grub2, distributions *have* to take
updates before grub2 finally gets around to doing a release --- to get
security fixes if nothing else! The support for csum_seed has been in
Fedora and other distributions for a while, since the patches had
landed in grub2 in June 2021. I probably should have made sure the
feature had landed in Debian's grub2 packaging earlier; that's my bad,
and my apologies for that.
Note that Debian's grub2 has well over 100 patches, nearly all of
which are backports from grub2's git repo. So the argument that
"there doesn't exist a formally released grub2 release" isn't
particularly compelling, since all the distros are backporting
patches. The only question is how *many* commits release has an
individual distribution taken.
By the way, in the case of the csum_seed feature, it's pretty
straightforward to just run "tune2fs -O ^metadata_csum_seed
/tmp/boot.img". If the UUID has been changed since the file system
was created, you'll have to do this while the file system is unmounted
and it will take a few seconds, but that's almost certainly not the
case with vmdb2.
- Ted
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
Am Dienstag, dem 14.02.2023 um 14:58 + schrieb Steve McIntyre: > On Tue, Feb 14, 2023 at 12:50:18PM +0100, Daniel Leidert wrote: > > [..] > Breakages happen like this, This breakage is *unnecessary* and it breaks at the momnent *all* debootstrap installations except for doing a bookworm/sid installation themselves. That is just stupid. Get down from your high horse and ackknowledge the problems your behavior causes. > and this has happened before in similar > circumstances. No it has not. You are doing a breakage on purpose during a freeze period while the transition period is over. Do it with a proper transition during the next release cycle. > > [..] > > Whe whole handling is completely wrong here. First, grub should have > > been fixed upstream. And the change in e2fsprogs should have been done > > only after "fixed" grub versions had settled. If we do it the other way > > around, we have to patch grub in affected distributions as well. And > > for Debian that means at least to patch Bullseye and any other release > > we want to be able to install from Bookworm. I even a lot of companies > > using Buster still. > > And I know of folks still working on Stretch and Jessie. How far back > do you want to tie things?? How about staying on topic and explaining, why this transition is necessary and has to be done the wrong way around, instead of picking the fact that older systems still exist? You break almost *all* installation right now. You also broke an Ubuntu 20.04 or 22.04 LTS installation. Are they new enough? [..] > > > > I'm critizicing the way of handling that breaking change and the > > ignorance shown reagarding the impact, not that fact that there is a > > breaking change. And it breaks a lot! This doesn't affect just a few > > minor use cases. It affects the basic way of installing a clean Ubuntu > > or Debian (or derivative) on a remote server using the debootstrap > > method. > > People using these tools need to be aware of the potential issue. What > would happen if you ran debootstrap with a filesystem that the target > distro doesn't know how to mount at all, for example? That is different from introducing a breaking change for which no grub upstream release has a fix yet. So basically the only system able to handle a filesystem created with e2fsprogs 1.47.0 is Sid right now. Can you please ignore your ego and see the problems you are causing? You push a breaking change for no reason at all. What is the gain here compared to the issues you are causing? > > And again: We are in the middle of a freeze here. And e2fsprogs > > pushes > > a breaking change that is not even handled by any existing grub > > upstream release, and is also not properly handled within Debian?! > > Grub upstream is already known to be problematic in terms of release > cycles. That is not enough and it is not a solution for the problem. There is *no* grub version out there supporting this, except for the patched version is Sid. Is this the sign for a working transition? > We now know about this particular issue (thanks Ted!) and > we've fixed it in unstable (and soon testing). Which helps exactly nobody, as it still breaks all other Debian or Ubuntu installation. I cannot belive that you intentionally break one of the standard methods to install Debian or Ubuntu for no reason at all and without a proper transition. Version 1.47.0 of e2fsprogs contains nothing necessary for Bookworm. I'll bring this to the attention of the release team. I'm sick of your ignorance. Do a proper transition and don't start it during a freeze. Daniel
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
On Tue, Feb 14, 2023 at 12:50:18PM +0100, Daniel Leidert wrote: >Am Dienstag, dem 14.02.2023 um 10:45 + schrieb Steve McIntyre: >> On Tue, Feb 14, 2023 at 11:34:13AM +0100, Daniel Leidert wrote: >> > Am Montag, dem 13.02.2023 um 21:35 -0500 schrieb Theodore Ts'o: >> >> ... >> >> > > But a generalized requirement that we be able to use debootstrap and >> > > vmdb2 to be able to bootstrap an arbitrarily old distribution doesn't >> > > seem to be reasonable. >> > >> > You are completyl wrong. This breaks a standard way of installing any >> > system supported by deboostrap with a grub without a fix to deal with >> > this version of e2fsprogs. This isn't just about vmdb2. >> > >> > What you are saying is ignorant. >> > >> > If this isn't cared about, then this version of e2fsprogs shouldn't >> > make it into Bookworm. We are in the middle of a freeze and this breaks >> > toolchains and a standard way (see [1]) of installing Debian. >> >> Sorry Daniel, but I have to (mostly!) agree with Ted here. If you're >> creating an image of an older release using newer tools then you'll >> need to be aware that sometimes the newer tools will create things >> that don't work there. If there's a bug here, I would strongly argue >> that it's in vmdb2. deboostrap (for example) includes some >> release-specific knowledge to cope with issues like this. > >debootstrap does nothing related to grub. So it is a bad example. That's why I said *like* this, not *exactly* like this. debootstrap has had knowledge of things like fs layouts etc. that older releases need (e.g. merged-/usr). >Again I refer to [1]: If the host system contains the problematic >e2fsprogs and the target system doesn't contain a grub with the fix >[2], then this breaks installations. This breaks older systems *and* >current systems. For example, I neither see the necessary grub patch >in both Ubuntu 20.04 and 22.04 either. So they also cannot be >installed using the deboostrap method and the toolchain in Sid (and >Bookworm if e2fsprogs makes it there). Breakages happen like this, and this has happened before in similar circumstances. If you're installing an older system using brand-new tools, you need to be aware of the potential for things to not work. In this particular case, all you need to do is tweak the flags on the ext4 filesystem when you create it. This isn't that hard... >[1] https://www.debian.org/releases/stable/amd64/apds03 >[2] Even "our" grub only contains a patch and not an upstream version >with support. So how can you even expect the target system to contain a >fix and be able to handle the created filesystem?! > >Whe whole handling is completely wrong here. First, grub should have >been fixed upstream. And the change in e2fsprogs should have been done >only after "fixed" grub versions had settled. If we do it the other way >around, we have to patch grub in affected distributions as well. And >for Debian that means at least to patch Bullseye and any other release >we want to be able to install from Bookworm. I even a lot of companies >using Buster still. And I know of folks still working on Stretch and Jessie. How far back do you want to tie things?? >> If we don't allow for this kind of change, that wouldn't allow us to >> *ever* make breaking changes in some packages, and that's just not >> sustainable. > >I'm critizicing the way of handling that breaking change and the >ignorance shown reagarding the impact, not that fact that there is a >breaking change. And it breaks a lot! This doesn't affect just a few >minor use cases. It affects the basic way of installing a clean Ubuntu >or Debian (or derivative) on a remote server using the debootstrap >method. People using these tools need to be aware of the potential issue. What would happen if you ran debootstrap with a filesystem that the target distro doesn't know how to mount at all, for example? >And again: We are in the middle of a freeze here. And e2fsprogs pushes >a breaking change that is not even handled by any existing grub >upstream release, and is also not properly handled within Debian?! Grub upstream is already known to be problematic in terms of release cycles. We now know about this particular issue (thanks Ted!) and we've fixed it in unstable (and soon testing). -- Steve McIntyre, Cambridge, UK.st...@einval.com Who needs computer imagery when you've got Brian Blessed?
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
Am Dienstag, dem 14.02.2023 um 10:45 + schrieb Steve McIntyre: > On Tue, Feb 14, 2023 at 11:34:13AM +0100, Daniel Leidert wrote: > > Am Montag, dem 13.02.2023 um 21:35 -0500 schrieb Theodore Ts'o: > > ... > > > > But a generalized requirement that we be able to use debootstrap and > > > vmdb2 to be able to bootstrap an arbitrarily old distribution doesn't > > > seem to be reasonable. > > > > You are completyl wrong. This breaks a standard way of installing any > > system supported by deboostrap with a grub without a fix to deal with > > this version of e2fsprogs. This isn't just about vmdb2. > > > > What you are saying is ignorant. > > > > If this isn't cared about, then this version of e2fsprogs shouldn't > > make it into Bookworm. We are in the middle of a freeze and this breaks > > toolchains and a standard way (see [1]) of installing Debian. > > Sorry Daniel, but I have to (mostly!) agree with Ted here. If you're > creating an image of an older release using newer tools then you'll > need to be aware that sometimes the newer tools will create things > that don't work there. If there's a bug here, I would strongly argue > that it's in vmdb2. deboostrap (for example) includes some > release-specific knowledge to cope with issues like this. debootstrap does nothing related to grub. So it is a bad example. Again I refer to [1]: If the host system contains the problematic e2fsprogs and the target system doesn't contain a grub with the fix [2], then this breaks installations. This breaks older systems *and* current systems. For example, I neither see the necessary grub patch in both Ubuntu 20.04 and 22.04 either. So they also cannot be installed using the deboostrap method and the toolchain in Sid (and Bookworm if e2fsprogs makes it there). [1] https://www.debian.org/releases/stable/amd64/apds03 [2] Even "our" grub only contains a patch and not an upstream version with support. So how can you even expect the target system to contain a fix and be able to handle the created filesystem?! Whe whole handling is completely wrong here. First, grub should have been fixed upstream. And the change in e2fsprogs should have been done only after "fixed" grub versions had settled. If we do it the other way around, we have to patch grub in affected distributions as well. And for Debian that means at least to patch Bullseye and any other release we want to be able to install from Bookworm. I even a lot of companies using Buster still. > If we don't allow for this kind of change, that wouldn't allow us to > *ever* make breaking changes in some packages, and that's just not > sustainable. I'm critizicing the way of handling that breaking change and the ignorance shown reagarding the impact, not that fact that there is a breaking change. And it breaks a lot! This doesn't affect just a few minor use cases. It affects the basic way of installing a clean Ubuntu or Debian (or derivative) on a remote server using the debootstrap method. And again: We are in the middle of a freeze here. And e2fsprogs pushes a breaking change that is not even handled by any existing grub upstream release, and is also not properly handled within Debian?! Daniel
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
On Tue, Feb 14, 2023 at 11:34:13AM +0100, Daniel Leidert wrote: >Am Montag, dem 13.02.2023 um 21:35 -0500 schrieb Theodore Ts'o: ... >> But a generalized requirement that we be able to use debootstrap and >> vmdb2 to be able to bootstrap an arbitrarily old distribution doesn't >> seem to be reasonable. > >You are completyl wrong. This breaks a standard way of installing any >system supported by deboostrap with a grub without a fix to deal with >this version of e2fsprogs. This isn't just about vmdb2. > >What you are saying is ignorant. > >If this isn't cared about, then this version of e2fsprogs shouldn't >make it into Bookworm. We are in the middle of a freeze and this breaks >toolchains and a standard way (see [1]) of installing Debian. Sorry Daniel, but I have to (mostly!) agree with Ted here. If you're creating an image of an older release using newer tools then you'll need to be aware that sometimes the newer tools will create things that don't work there. If there's a bug here, I would strongly argue that it's in vmdb2. deboostrap (for example) includes some release-specific knowledge to cope with issues like this. If we don't allow for this kind of change, that wouldn't allow us to *ever* make breaking changes in some packages, and that's just not sustainable. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
Am Montag, dem 13.02.2023 um 21:35 -0500 schrieb Theodore Ts'o: > On Tue, Feb 14, 2023 at 01:01:38AM +0100, Daniel Leidert wrote: > > Hi Steve, > > > > I believe that your fix to grub2 in Sid is not enough to handle > > #1030939/#1030846. > > > > This problem breaks e.g. vmdb2. I can no longer create a Bullseye > > system image with vmdb2 on Sid, because the grub-install step in the > > Bullseye chroot now fails, because the created filesystem (created with > > e2fsprogs on Sid) cannot be recognized by grub. > > I'm confused, why does using vmdb2 on *sid* involve running > grub-install on a *bulleye* chroot? My workstation is running Sid. I want to create an image for Bullseye (in this case using vmdb2, but I could do it manually as well). First, one creates the paritioning and the filesystems for the target system with the tools provided by the host system. This involves running the unfortunate version of e2fsprogs with the breaking change. Then, one installs the base system with deboostrap (also from the host system) into the created partitions/filesystem. Then one (bind-)mounts the virtual filesystems into the target systems, does a chroot into it, and then one finishes the installation inside the chroot, including running grub-install. This is standard and common behavior. I have never ever seen in my whole life someone to use grub2 from Sid to make a grub-install for a Bullseye target. I have not even seen anybody suggest that. Consider another example: Server providers use GRML or Bookworm with e2fsprogs 1.47.0 as their rescue system. Now people are no longer able to create a Bullseye system using the deboostrap method [1]. If the host system uses e2fsprogs 1.47.0 or above and the target system for [1] uses a grub without a fix, then this no longer works. [1] https://www.debian.org/releases/stable/amd64/apds03 > That seems to be extremely ill-advised. I'm sorry? I think you are completely missing the point. > If you are trying to install a bullseye system, why > can't you using vmdb2 on bullseye? Because I am using Sid and because I use features of vmdb2 which are not available in the version in Bullseye. And this feature breaks vmdb2 and similar tools. It also breaks a standard way of installing Debian systems. > And if you are trying to install a sid or bookworm system using vmdb2, > why can't you (or vmdb2) use a sid or bookworm chroot for doing the > grub-install? What are you talking about? You basically break the toolchains and then you suggest to do non-standard stuff to handle this or even avoid doing installations of affected systems?! > > I guess that the fix applied to grub2 in Sid would have to be applied > > to grub2 in Bullseye as well (and basically to any grub2 package in any > > Debian or Ubuntu or Raspbian release supported by debootstrap). > > I can understand why we need to keep things synchronized so that a > debian installer for Bookworm be able to generate a bootable system > using the version of grub and e2fsprogs in Bookworm. > > But a generalized requirement that we be able to use debootstrap and > vmdb2 to be able to bootstrap an arbitrarily old distribution doesn't > seem to be reasonable. You are completyl wrong. This breaks a standard way of installing any system supported by deboostrap with a grub without a fix to deal with this version of e2fsprogs. This isn't just about vmdb2. What you are saying is ignorant. If this isn't cared about, then this version of e2fsprogs shouldn't make it into Bookworm. We are in the middle of a freeze and this breaks toolchains and a standard way (see [1]) of installing Debian. Daniel
Bug#1030846: Bug#1030939: e2fsprogs: generates filesystems that grub-install doesn't recognize
On Tue, Feb 14, 2023 at 01:01:38AM +0100, Daniel Leidert wrote: > Hi Steve, > > I believe that your fix to grub2 in Sid is not enough to handle > #1030939/#1030846. > > This problem breaks e.g. vmdb2. I can no longer create a Bullseye > system image with vmdb2 on Sid, because the grub-install step in the > Bullseye chroot now fails, because the created filesystem (created with > e2fsprogs on Sid) cannot be recognized by grub. I'm confused, why does using vmdb2 on *sid* involve running grub-install on a *bulleye* chroot? That seems to be extremely ill-advised. If you are trying to install a bullseye system, why can't you using vmdb2 on bullseye? And if you are trying to install a sid or bookworm system using vmdb2, why can't you (or vmdb2) use a sid or bookworm chroot for doing the grub-install? > I have to downgrade e2fsprogs to the version in Testing to get the > build going again. It also means that a Bookworm system can never be > used to format and debootstrap a Bullseye or Buster system that > requires a grub installation. > > I guess that the fix applied to grub2 in Sid would have to be applied > to grub2 in Bullseye as well (and basically to any grub2 package in any > Debian or Ubuntu or Raspbian release supported by debootstrap). I can understand why we need to keep things synchronized so that a debian installer for Bookworm be able to generate a bootable system using the version of grub and e2fsprogs in Bookworm. But a generalized requirement that we be able to use debootstrap and vmdb2 to be able to bootstrap an arbitrarily old distribution doesn't seem to be reasonable. It would mean that we couldn't update to newer versions of the C library, or of new file system featuers, because we are somehow obliged to be able to boostrap ancient versions of the system. After all, we don't require that a binary built using Bookworm be able to run on Bullseye. How is this any different? I generate test appliances for file system regression testing which run on Debian Bullseye on my Debian Bookworm system --- and this gets done using build chroot[1]. I even have super-convenient scripts to create the build chroot[2]. This is not hard why can't vmdb2 do the same thing? [1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/building-xfstests.md [2] https://github.com/tytso/xfstests-bld/blob/master/setup-buildchroot - Ted
