Re: grub / grub2 conflicts
On Mon, 2011-09-26 at 21:49 +0200, Kevin Kofler wrote: Doug Ledford wrote: - Original Message - Or Doug could take over grub if he is willing to fix the issues he runs into. Or he could fork grub into maggot, use that for his needs. If he is willing to support it and you are not.. that would move us from this argument. I could, but that would give me another CRITPATH package, and I'd sooner slit my wrists. A forked GRUB for virtualization purposes only would definitely not be critpath. And I'd argue that even a non-forked GRUB 1 SHOULD no longer be critpath now that we default to GRUB 2, but you'd have to bring that to FESCo. grub-legacy is still default for EFI installs (which, in retrospect, is causing quite a bit of pain, but hey, hindsight is 20/20). Anyway, as of about ten hours ago, grub-efi is split off from grub, and pjones is probably going to have grub2 obsolete grub. it would, of course, be possible for the virt team to introduce the bits of grub they actually need for libguestfs as a sub-package of libguestfs or whatever, which would not be critpath. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - Or Doug could take over grub if he is willing to fix the issues he runs into. Or he could fork grub into maggot, use that for his needs. If he is willing to support it and you are not.. that would move us from this argument. I could, but that would give me another CRITPATH package, and I'd sooner slit my wrists. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Mon, Sep 26, 2011 at 09:47, Doug Ledford dledf...@redhat.com wrote: - Original Message - Or Doug could take over grub if he is willing to fix the issues he runs into. Or he could fork grub into maggot, use that for his needs. If he is willing to support it and you are not.. that would move us from this argument. I could, but that would give me another CRITPATH package, and I'd sooner slit my wrists. Oh I figured if it was going to be dropped it would no longer be CRITPATH, but if it would remain that I would prefer not to have Mrs Ledford hunting me down . -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- Stephen J Smoogen. The core skill of innovators is error recovery, not failure avoidance. Randy Nelson, President of Pixar University. Let us be kind, one to another, for most of us are fighting a hard battle. -- Ian MacLaren -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - Oh I figured if it was going to be dropped it would no longer be CRITPATH, but if it would remain that I would prefer not to have Mrs Ledford hunting me down . You're probably safer that way... ;-) -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
Doug Ledford wrote: - Original Message - Or Doug could take over grub if he is willing to fix the issues he runs into. Or he could fork grub into maggot, use that for his needs. If he is willing to support it and you are not.. that would move us from this argument. I could, but that would give me another CRITPATH package, and I'd sooner slit my wrists. A forked GRUB for virtualization purposes only would definitely not be critpath. And I'd argue that even a non-forked GRUB 1 SHOULD no longer be critpath now that we default to GRUB 2, but you'd have to bring that to FESCo. (FWIW, IMHO, the whole critpath nonsense should be abolished, it only causes problems and solves none.) Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Fri, Sep 23, 2011 at 20:51, Matthew Garrett mj...@srcf.ucam.org wrote: On Fri, Sep 23, 2011 at 07:42:55PM -0400, Doug Ledford wrote: It may work. It may not. It may leave the system unbootable. You can't guarantee it, and you've been told that this is behaviour that you can't depend on. If you choose to do so then fine, but any bugs filed against grub are just going to be closed. You're trying to do something unsupported. Depending on unsupported and undefined behaviour is bad design. Or Doug could take over grub if he is willing to fix the issues he runs into. Or he could fork grub into maggot, use that for his needs. If he is willing to support it and you are not.. that would move us from this argument. -- Stephen J Smoogen. The core skill of innovators is error recovery, not failure avoidance. Randy Nelson, President of Pixar University. Let us be kind, one to another, for most of us are fighting a hard battle. -- Ian MacLaren -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message -
You're basically arguing that we should never remove any software
from Fedora
in case it's used in a virtual machine hosted on a Fedora machine.
This is not a workable scenario.
OMG Peter, can you intentionally conflate any more molehills into mountains?
Do you even believe the stuff you are spewing? Of course it doesn't apply to
all software, but the boot loader is an acknowledged special case of *all*
installations, and it's something necessary to manage vm images, that does not
conflate to all software unless you are just trying to be pissy.
Regardless, this is my suggestion Richard:
Grab the current grub package and add the source tarball to libguestfs
Grab the current grub2 package and add the source tarball to libguestfs
Build grub and grub2 as part of the libguestfs package
Install the grub and grub2 binaries as /usr/libexec/vm-grub{,2} and remove all
the other grub/grub2 files (they aren't needed)
Maintain the binaries in libguestfs and fix any problems you find with specific
guest OSes in your own package. This has the side benefit that libguestfs will
be more portable as a result because you won't care about the host OSes boot
loader installation (or lack thereof).
If anyone complains about a grub/grub2 binary in libguestfs, tell them it's a
necessary tool the base OS didn't want to provide reliably so you did what was
necessary.
--
Doug Ledford dledf...@redhat.com
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/23/2011 07:54 AM, Doug Ledford wrote:
- Original Message -
You're basically arguing that we should never remove any software
from Fedora
in case it's used in a virtual machine hosted on a Fedora machine.
This is not a workable scenario.
OMG Peter, can you intentionally conflate any more molehills into mountains?
Do you even believe the stuff you are spewing? Of course it doesn't apply to
all software, but the boot loader is an acknowledged special case of *all*
installations, and it's something necessary to manage vm images, that does
not conflate to all software unless you are just trying to be pissy.
I don't in any way buy that it's necessary to manage vm images; it's
used because of a bad design, not because it's necessary. But thanks for the
pile of insults instead of listening to what's been said in disagreement.
Grab the current grub package and add the source tarball to libguestfs Grab
the current grub2 package and add the source tarball to libguestfs Build grub
and grub2 as part of the libguestfs package Install the grub and grub2
binaries as /usr/libexec/vm-grub{,2} and remove all the other grub/grub2
files (they aren't needed) Maintain the binaries in libguestfs and fix any
problems you find with specific guest OSes in your own package. This has the
side benefit that libguestfs will be more portable as a result because you
won't care about the host OSes boot loader installation (or lack thereof). If
anyone complains about a grub/grub2 binary in libguestfs, tell them it's a
necessary tool the base OS didn't want to provide reliably so you did what
was necessary.
If you don't package the tools as grub-install and grub in root's PATH,
I don't honestly have much objection to this, with some caveats. Not least,
there's some work required just to make that plan work. Also, you probably
do want the patches that represent most of a decade of maintenance work.
--
Peter
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - it's used because of a bad design, It's not a bad design, it's the *right* design. Being able to rescue a guest that can't boot without resorting to a rescue cd boot of the guest vm is a worthwhile goal and this is part of that. The two alternative designs (guest code in guest vm, guest code in host vm) were both shown to be inferior designs (the first because the guest vm might not be bootable and requires booting up the guest vm which is highly undesirable if the user is simply attempting an offline modification of the vm, the second because that's a huge gaping security cluster fuck). not because it's necessary. It's just as necessary as any of the other rescue tools we put on rescue CDs. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Fri, Sep 23, 2011 at 04:28:30PM -0400, Doug Ledford wrote: - Original Message - it's used because of a bad design, It's not a bad design, it's the *right* design. Being able to rescue a guest that can't boot without resorting to a rescue cd boot of the guest vm is a worthwhile goal and this is part of that. The two alternative designs (guest code in guest vm, guest code in host vm) were both shown to be inferior designs (the first because the guest vm might not be bootable and requires booting up the guest vm which is highly undesirable if the user is simply attempting an offline modification of the vm, the second because that's a huge gaping security cluster fuck). It's a bad design because it asserts something (grub versions are compatible with each other) that isn't true (they're not). I don't have any idea how to solve this given the constraints that are being imposed, but this approach certainly isn't a solution. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - It's a bad design because it asserts something (grub versions are compatible with each other) that isn't true (they're not). I've stated this once already, but since you glossed over it. It does not assert that grub versions are compatible, it asserts that the stage1 boot loader and the console utility are able to work with paired stage1.5 and stage2 loaders. Code inspection of the stage1 loader showed this compatibility assumption to be correct, and experience has shown the grub utility compatibility to be correct. This is unlikely to change. As Peter has said, grub is dead, there is no upstream, other distros including Fedora are leaving it behind, so it is more or less a static target at this point in time, and we already have the experience based evidence that your fears are not founded in reality. Could there be incompatibilities? Yes. Are there? None found yet, and based upon code inspection, analysis of the code in question, the fact that upstream has been dead for years which tends to cause maintainers in distros to do the absolute bare minimum to keep their distros booting and discourages wild code changes that might destabilize things and introduce exactly the sort of incompatibilities you are afraid of, it is a reasonable engineering decision to decide to go with the existing code as it is and fix up any future incompatibilities that might arise, if they ever even do. As such, it's *not* a bad design, it's an expedient design. It benefits from a certain amount of serendipity. It would be much riskier if grub were in active development. But it's not, we got lucky, it works as it is, so go with it. There's absolutely no reason not to, especially if Richard is willing to do as I suggested and just throw a current grub utility into libguestfs and be done with it. I don't have any idea how to solve this given the constraints that are being imposed, but this approach certainly isn't a solution. It's a perfectly workable solution. You just don't like it on the basis of your own ingrained FUD against the idea that isn't even based on realistic future development of this dead end package. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Fri, Sep 23, 2011 at 07:42:55PM -0400, Doug Ledford wrote: - Original Message - It's a bad design because it asserts something (grub versions are compatible with each other) that isn't true (they're not). I've stated this once already, but since you glossed over it. It does not assert that grub versions are compatible, it asserts that the stage1 boot loader and the console utility are able to work with paired stage1.5 and stage2 loaders. Code inspection of the stage1 loader showed this compatibility assumption to be correct, and experience has shown the grub utility compatibility to be correct. There is no guarantee that any given stage 1 is compatible with any given stage 1.5, and there is no guarantee that any given grub is compatible with any given stage 1. This is unlikely to change. As Peter has said, grub is dead, there is no upstream, other distros including Fedora are leaving it behind, so it is more or less a static target at this point in time, and we already have the experience based evidence that your fears are not founded in reality. Could there be incompatibilities? Yes. Are there? None found yet, and based upon code inspection, analysis of the code in question, the fact that upstream has been dead for years which tends to cause maintainers in distros to do the absolute bare minimum to keep their distros booting and discourages wild code changes that might destabilize things and introduce exactly the sort of incompatibilities you are afraid of, it is a reasonable engineering decision to decide to go with the existing code as it is and fix up any future incompatibilities that might arise, if they ever even do. We've done rather more than the bare minimum with grub. The delta between Fedora and the last upstream release is rather large. Some of those changes have been pretty wild. As such, it's *not* a bad design, it's an expedient design. It benefits from a certain amount of serendipity. It would be much riskier if grub were in active development. But it's not, we got lucky, it works as it is, so go with it. There's absolutely no reason not to, especially if Richard is willing to do as I suggested and just throw a current grub utility into libguestfs and be done with it. It may work. It may not. It may leave the system unbootable. You can't guarantee it, and you've been told that this is behaviour that you can't depend on. If you choose to do so then fine, but any bugs filed against grub are just going to be closed. You're trying to do something unsupported. Depending on unsupported and undefined behaviour is bad design. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Wed, 2011-09-21 at 15:54 -0400, Peter Jones wrote: On 09/21/2011 03:39 PM, Mark McLoughlin wrote: On Wed, 2011-09-21 at 18:48 +0100, Matthew Garrett wrote: Remember that the incompatibility isn't between libguestfs and the guest, it's between the host grub and the guest grub. Both of those can change without libguestfs's knowledge. Sounds like we need a 'Conflicts: libguestfs' in both grub and grub2 then? Yes, but this will hardly help the situation, which right now is that the distro pulls in grub 2, because that's what we've collectively chosen to do, and libguestfs pulls in grub on the host, even though it isn't really using it there. So effectively your solution is to keep the problem we've got right now. Sigh. I was joking. Obviously, if maintainers went around inserting Conflicts with other packages because they don't like how the other package works, then there'd be an order of magnitude more unpleasantness on fedora-devel. Not liking the way libguestfs works is no justification for an arbitrary grub2 Conflicts in the grub package. It sounds like there are issues which necessitate the Conflicts - e.g. the grubby issue - but that they could be resolved. Can we focus on those issues, what exactly they are and how folks can do to help resolve them? Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 11:27:35AM +0100, Mark McLoughlin wrote: Sigh. I was joking. Obviously, if maintainers went around inserting Conflicts with other packages because they don't like how the other package works, then there'd be an order of magnitude more unpleasantness on fedora-devel. The grub maintainer is telling you that the way in which you're trying to use grub is broken. You *need* to use the grub files that are in guest, not the host. This will be even more true with grub 2. It's not a matter of disliking the approach, it's a matter of it being demonstrably technically incorrect. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, 2011-09-22 at 14:05 +0100, Matthew Garrett wrote: On Thu, Sep 22, 2011 at 11:27:35AM +0100, Mark McLoughlin wrote: Sigh. I was joking. Obviously, if maintainers went around inserting Conflicts with other packages because they don't like how the other package works, then there'd be an order of magnitude more unpleasantness on fedora-devel. The grub maintainer is telling you that the way in which you're trying to use grub is broken. You *need* to use the grub files that are in guest, not the host. This will be even more true with grub 2. It's not a matter of disliking the approach, it's a matter of it being demonstrably technically incorrect. There's nothing technically incorrect about the approach, demonstrably or otherwise, if the version of grub in the guest and host is compatible. Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 04:50:16PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 14:05 +0100, Matthew Garrett wrote: The grub maintainer is telling you that the way in which you're trying to use grub is broken. You *need* to use the grub files that are in guest, not the host. This will be even more true with grub 2. It's not a matter of disliking the approach, it's a matter of it being demonstrably technically incorrect. There's nothing technically incorrect about the approach, demonstrably or otherwise, if the version of grub in the guest and host is compatible. grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: On Thu, Sep 22, 2011 at 04:50:16PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 14:05 +0100, Matthew Garrett wrote: The grub maintainer is telling you that the way in which you're trying to use grub is broken. You *need* to use the grub files that are in guest, not the host. This will be even more true with grub 2. It's not a matter of disliking the approach, it's a matter of it being demonstrably technically incorrect. There's nothing technically incorrect about the approach, demonstrably or otherwise, if the version of grub in the guest and host is compatible. grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. But again, all of this orthogonal to the issue of the Conflicts. Whether the Conflicts is correct is a completely separate discussion from how libguestfs should use grub and grub2. Conflating the two discussions makes it appear like the maintainer of one package is refusing to fix a bug in his package until another maintainer agrees to change the design of his program to the first maintainers taste. Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. But again, all of this orthogonal to the issue of the Conflicts. Whether the Conflicts is correct is a completely separate discussion from how libguestfs should use grub and grub2. There is no rational reason to have grub and grub2 installed on the same system at once, and having them both there increases the complexity of the system. Conflating the two discussions makes it appear like the maintainer of one package is refusing to fix a bug in his package until another maintainer agrees to change the design of his program to the first maintainers taste. There's no bug in grub. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 2011/09/22 17:37 (GMT+0100) Matthew Garrett composed: There is no rational reason to have grub and grub2 installed on the same system at once, and having them both there increases the complexity of the system. For which definition of system? My systems typically contain 20 or more partitions, of which 6 or more are /boot or / partitions on which Grub has been installed, and all of which (except on Mac) contain DOS compatible/legacy MBR code. I definitely don't need the additional complexity that putting Grub2 on any of them would produce. It's bad enough https://bugzilla.redhat.com/show_bug.cgi?id=107748 is nearly 8 years old with no action. -- The wise are known for their understanding, and pleasant words are persuasive. Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. There is no rational reason to have grub and grub2 installed on the same system at once, and having them both there increases the complexity of the system. you can install KDE and GNOME and you are worrying about grub and grub2? surely grubby could detect which one is actually in use and pick it without the world ending. but I agree with Mark, you guys are completely conflating two very different things and using one to justify the other. the is no technical reason why grub and grub2 should conflict, if there is syslinux should also conflict. Maybe I have a grub USB key I want to update with a new grub while my main MBR is grub2. Dave. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 02:02:15PM -0400, David Airlie wrote: you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. No, that's not good enough. You need to know the version installed on the system, not the packaged version. Upgrading the package doesn't cause grub-install to be rerun. There is no rational reason to have grub and grub2 installed on the same system at once, and having them both there increases the complexity of the system. you can install KDE and GNOME and you are worrying about grub and grub2? They don't both attempt to sit in the same few bytes. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/22/2011 02:02 PM, David Airlie wrote: On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. The whole point of libguestfs's usage was that the package isn't actually installed in the guest. So that won't work. The rest of your point ignores that grub1 is going away as soon as is reasonably practicable. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, 2011-09-22 at 17:18 +0100, Mark McLoughlin wrote: You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. 'check' it? And what's the user expected to do if they're incorrect? Crowbar Ubuntu's grub2 into Fedora, or vice versa, so the host and the guest will be compatible? I hate to say it, but honestly, this thread looks pretty clear-cut to an outsider: pjones and mjg59 are correct, and you and rwmj are incorrect. Their arguments that it is fundamentally unsafe to use the host's grub or, even more so, grub2 in a guest have clear merit, and it honestly feels like the counter-arguments so far have been 'we've got away with it so far' and 'doing it any other way is hard, and we already wrote all this code, so please stop raising inconvenient questions'. Neither of those arguments are at all compelling. I haven't seen a single serious attempt to refute the central point that there is no guarantee of compatibility between any particular two versions or even builds of grub or grub2, and there is not even a mechanism for denoting and testing compatibility. Given this, it's hard to see how it can possibly be the right thing to do to use the host's grub or grub2 in the guest. You have provided lots of arguments of the form 'but it would make things so much easier for everyone if we could do that', but that's really beside the point: it may be sad that you can't, but if you can't, you can't. It's like saying it'd be really cool if I could fly, so I'm just gonna go ahead and jump off the side of this building. It's not that anyone thinks it wouldn't be cool if you could fly, it's just that, in point of fact, you can't. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 05:37:39PM +0100, Matthew Garrett wrote: On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. We allow you to inspect the guest to find the OS version, and even versions of individual packages installed in it. Using this you can make rules like only apply grub-install to Fedora guests = version 15, or only to guests that have grub version between 1.A and 1.B. http://libguestfs.org/guestfs.3.html#inspection Rich. PS. Also orthogonal to this discussion: It turns out that virt-v2v doesn't use grub-install. However virt-v2v still needs some development work to support grub2 configuration. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 07:38:54PM +0100, Richard W.M. Jones wrote: On Thu, Sep 22, 2011 at 05:37:39PM +0100, Matthew Garrett wrote: I described something that is, practically speaking, impossible. We allow you to inspect the guest to find the OS version, and even versions of individual packages installed in it. The package version of grub does nothing to tell you which version of grub is actually installed in the boot sector. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 02:18:48PM -0400, Peter Jones wrote: On 09/22/2011 02:02 PM, David Airlie wrote: On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. The whole point of libguestfs's usage was that the package isn't actually installed in the guest. So that won't work. This is not correct, grub is installed in the guest, but we don't want to run it from the guest because of security problems. I outlined it in an earlier email in this thread. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 02:02:15PM -0400, David Airlie wrote: you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. No, that's not good enough. You need to know the version installed on the system, not the packaged version. Upgrading the package doesn't cause grub-install to be rerun. There is no rational reason to have grub and grub2 installed on the same system at once, and having them both there increases the complexity of the system. you can install KDE and GNOME and you are worrying about grub and grub2? They don't both attempt to sit in the same few bytes. Nicely editing out of the other use-case I supplied. grub and grub2 *packages* don't install into the same few bytes. I thought you were good at backing up arguments with technical reasons, not strawmen. The argument is should it be possible to install grub1 and grub2 *packages* on the same system? The strawman is if you install grub1 and grub2 bootloaders into the MBR on the same system they will conflict. I totally agree with your strawman, but you still haven't provided any technical reasons why the argument is wrong. Above all people Matthew I thought you were aware of how strawmen work and would be against their use. Dave. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 11:38:26AM -0700, Adam Williamson wrote: On Thu, 2011-09-22 at 17:18 +0100, Mark McLoughlin wrote: You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. 'check' it? And what's the user expected to do if they're incorrect? Crowbar Ubuntu's grub2 into Fedora, or vice versa, so the host and the guest will be compatible? We don't pretend to work on Ubuntu grub2, never said we did. I hate to say it, but honestly, this thread looks pretty clear-cut to an outsider: pjones and mjg59 are correct, and you and rwmj are incorrect. Their arguments that it is fundamentally unsafe to use the host's grub or, even more so, grub2 in a guest have clear merit, and it honestly feels like the counter-arguments so far have been 'we've got away with it so far' and 'doing it any other way is hard, and we already wrote all this code, so please stop raising inconvenient questions'. Neither of those arguments are at all compelling. I haven't seen a single serious attempt to refute the central point that there is no guarantee of compatibility between any particular two versions or even builds of grub or grub2, and there is not even a mechanism for denoting and testing compatibility. Given this, it's hard to see how it can possibly be the right thing to do to use the host's grub or grub2 in the guest. The issue here isn't this at all. Issue #1 is that a conflicts was added to grub2, but no reason is given. It was apparently done to work around some problem in grubby, but again there's no clarity on what the problem(s) were and if there would be a better way to fix this. There is a further issue #2, quite orthogonal, which is that grub (upstream) doesn't support offline installation. This is a bug in grub 1 2 which really should be taken upstream. Nevertheless, it's quite possible to use grub1 offline on compatible guests. We don't really need to prove this, because we do it, and test the results, and we publish everything we do in open source code. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 11:45:11AM -0700, Jesse Keating wrote: On Sep 22, 2011, at 11:18 AM, Peter Jones wrote: On 09/22/2011 02:02 PM, David Airlie wrote: On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. The whole point of libguestfs's usage was that the package isn't actually installed in the guest. So that won't work. The rest of your point ignores that grub1 is going away as soon as is reasonably practicable. It also ignores any non-rpm guests. Guys ... libguestfs can inspect any guest and work out what apps are installed, what OS there is, what arch, what bootloader etc. We support a dozen different Linux distros (not just rpm/deb-based but some really odd ones too), along with FreeBSD, and Windows. We really have thought about all of this. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/22/2011 02:41 PM, Richard W.M. Jones wrote: On Thu, Sep 22, 2011 at 02:18:48PM -0400, Peter Jones wrote: On 09/22/2011 02:02 PM, David Airlie wrote: On Thu, Sep 22, 2011 at 05:18:09PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. I described something that is, practically speaking, impossible. you run rpm -q grub in the guest and on the host, if they are the same nvr, then they are the same package, where's the rocket science here. The whole point of libguestfs's usage was that the package isn't actually installed in the guest. So that won't work. This is not correct, grub is installed in the guest, but we don't want to run it from the guest because of security problems. I outlined it in an earlier email in this thread. Oh, my mistake. That being beside the point, it pretty much means any VM created in a previous OS release won't work. In any case I totally disagree with your idea of security, as I mentioned at the time. It makes things worse, not better. And that's still ignoring that grub1 needs to completely go away. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 8:40 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Thu, Sep 22, 2011 at 07:38:54PM +0100, Richard W.M. Jones wrote: On Thu, Sep 22, 2011 at 05:37:39PM +0100, Matthew Garrett wrote: We allow you to inspect the guest to find the OS version, and even versions of individual packages installed in it. The package version of grub does nothing to tell you which version of grub is actually installed in the boot sector. That doesn't matter: When libguestfs runs its installation of grub, the boot sector will be overwritten. What matters is the versions of stage1/1.5/2 files in the guest, which is adequately described by identifying the RPM. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 02:51:40PM -0400, Peter Jones wrote: Oh, my mistake. That being beside the point, it pretty much means any VM created in a previous OS release won't work. In any case I totally disagree with your idea of security, as I mentioned at the time. It makes things worse, not better. Not running random code from untrusted guests makes things worse? And that's still ignoring that grub1 needs to completely go away. It's not going to completely go away until such guests completely go away. People use virtualization precisely because it allows them to continue to run very old operating systems. Or even supported ones like RHEL 6 (using grub1 until 2017-2020). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, 2011-09-22 at 19:47 +0100, Richard W.M. Jones wrote: I hate to say it, but honestly, this thread looks pretty clear-cut to an outsider: pjones and mjg59 are correct, and you and rwmj are incorrect. Their arguments that it is fundamentally unsafe to use the host's grub or, even more so, grub2 in a guest have clear merit, and it honestly feels like the counter-arguments so far have been 'we've got away with it so far' and 'doing it any other way is hard, and we already wrote all this code, so please stop raising inconvenient questions'. Neither of those arguments are at all compelling. I haven't seen a single serious attempt to refute the central point that there is no guarantee of compatibility between any particular two versions or even builds of grub or grub2, and there is not even a mechanism for denoting and testing compatibility. Given this, it's hard to see how it can possibly be the right thing to do to use the host's grub or grub2 in the guest. The issue here isn't this at all. It's become that. Issue #1 is that a conflicts was added to grub2, but no reason is given. It was apparently done to work around some problem in grubby, but again there's no clarity on what the problem(s) were and if there would be a better way to fix this. It seems pretty clear that the thread kinda diverged away from the actual question of the conflict several days ago, and is now fundamentally about whether it's correct for libguestfs to expect to be able to use the host bootloader package to configure the guest's bootloader. If you are actually accepting that it's incorrect for libguestfs to do this, but maintaining that the conflict is not valid, you could do a better job of making that clear, and directing the argument back to the issue of the conflict. It's certainly tenable for you to hold that, even if you accept that your example use case for having both grub and grub2 installed at the same time (libguestfs) is not a good one for technical reasons, it's still not necessary or correct for the grub and grub2 packages to conflict. That's a viable path to go down. But it's not at all clear from the thread that that's actually what you're doing. There is a further issue #2, quite orthogonal, which is that grub (upstream) doesn't support offline installation. This is a bug in grub 1 2 which really should be taken upstream. Nevertheless, it's quite possible to use grub1 offline on compatible guests. We don't really need to prove this, because we do it, and test the results, and we publish everything we do in open source code. Again, your argument is 'we've gotten away with it so far', which was true of every bad idea ever at *some* point. You can get away with driving on the wrong side of the street for quite a long time, if you pick a pretty quiet street; it doesn't make it the right thing to do. It may be that, if you raised it with the grub team, they would accept that offline installation using a different version of the code is something they would like to attempt to support. If so, great, everyone will be happy. But it doesn't change the fact that, at present, this is not supported, and libguestfs is doing the wrong thing by relying on it and hoping to carry on getting away with it. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 02:44:00PM -0400, David Airlie wrote: Nicely editing out of the other use-case I supplied. grub and grub2 *packages* don't install into the same few bytes. I thought you were good at backing up arguments with technical reasons, not strawmen. The argument is should it be possible to install grub1 and grub2 *packages* on the same system? The strawman is if you install grub1 and grub2 bootloaders into the MBR on the same system they will conflict. I totally agree with your strawman, but you still haven't provided any technical reasons why the argument is wrong. Above all people Matthew I thought you were aware of how strawmen work and would be against their use. Because it means additional complexity for any tools which have to interact with the bootloader and provides no benefit for any real usecases. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 07:58:35PM +0100, Matthew Garrett wrote: On Thu, Sep 22, 2011 at 02:44:00PM -0400, David Airlie wrote: Nicely editing out of the other use-case I supplied. grub and grub2 *packages* don't install into the same few bytes. I thought you were good at backing up arguments with technical reasons, not strawmen. The argument is should it be possible to install grub1 and grub2 *packages* on the same system? The strawman is if you install grub1 and grub2 bootloaders into the MBR on the same system they will conflict. I totally agree with your strawman, but you still haven't provided any technical reasons why the argument is wrong. Above all people Matthew I thought you were aware of how strawmen work and would be against their use. Because it means additional complexity for any tools which have to interact with the bootloader and provides no benefit for any real usecases. That grubby has to decide whether to run grub or grub2? Please can you outline exactly what bugs arose that caused the conflicts to be added. THIS is what we've been asking for. If we can fix those bugs in a better way, we can remove the conflicts, and we can take over legacy maintainership of grub1 ourselves. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/22/2011 03:27 PM, Richard W.M. Jones wrote: On Thu, Sep 22, 2011 at 02:51:40PM -0400, Peter Jones wrote: Oh, my mistake. That being beside the point, it pretty much means any VM created in a previous OS release won't work. In any case I totally disagree with your idea of security, as I mentioned at the time. It makes things worse, not better. Not running random code from untrusted guests makes things worse? Having more things installed on the host means a larger attack surface. And that's still ignoring that grub1 needs to completely go away. It's not going to completely go away until such guests completely go away. People use virtualization precisely because it allows them to continue to run very old operating systems. Or even supported ones like RHEL 6 (using grub1 until 2017-2020). It's going to go away *in Fedora* much sooner than the timeframe it takes for old OSes to bitrot into the void, yes. You've totally ignored the point, which is that you won't be able to run it on the host when it isn't there. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - Having more things installed on the host means a larger attack surface. Not if the host is properly locked down. And given that guests typically have more open services, and therefore a larger remote attack surface, the more there is in the guest, the less secure you are. You can do an Everything install in the host which gives a huge local attach surface, but if there is no entrance vector and you don't enable network services, it's no less secure than the guest and is probably far more secure. Your argument is silly given typical host/guest roles where the guest has a far large remote attack surface regardless of the host's local attack surface and is likely to be the remote entry point to get to the host. In that situation, minimizing the local attack surface of the guest (on the assumption that the remote compromise is going to happen in the guest and from there local attacks will be made either against the guest or the host) is far higher priority than the host. It's going to go away *in Fedora* much sooner than the timeframe it takes for old OSes to bitrot into the void, yes. You've totally ignored the point, which is that you won't be able to run it on the host when it isn't there. No, he's well aware of that, which is why he *wants* it there. Situation 1: run grub inside the guest from inside the guest vm, secure as you can't access anything but your virtual drive, so even a compromised grub can't effect the host unless the vm subsystem of the host is broken and allows access outside the scope of the virtual drive, but has the drawback that you actually have to boot up the guest instance in order to run grub, and that may not be possible if the guest's virtual block device has already undergone a resize. This is roughly equivalent to running any command in a chroot from within the chroot environment. Situation 2: run grub off of the guest vm's block device, but run from the host context. This opens the host to the possibility of a chained compromise where the attacker first compromises the guest, then uses a compromised grub to compromise the host. The fact that the host's native block devices, in addition to just the virtual block devices that the guest vm would normally see, are present is the source of this security risk. This is roughly the same as running a program from a chroot environment outside of the chroot environment. It does, however, eliminate the conundrum from situation 1 where you might not be able to boot due to an already completed fs resize. This situation does not need to boot and can effectively deal with the resize. In the chroot analogy though, this is the most insecure thing to do of all. Situation 3: run grub off of the host acting on the guest vm's block device and using the guest vm's boot loader files. This does not open up the host because the grub is from the host and can only be trojaned if the host has already been compromised. At no point does grub execute the boot loader files from the guest, it merely installs them. Worst case scenario is that a compromised guest stays compromised barring the possibility that an attacker can craft a special grub boot loader stage that causes the host grub binary to freak out (highly unlikely, but I would have to audit the code before I declared it impossible). This is equivalent to running a non-chroot based app upon the chroot environment, and in the chroot analogy this is generally considered a safe thing to do security wise, no? Fedora ships a virtualization environment, so while grub1 should go away as soon as possible in terms of Fedora's own use, having it around for situation 3 is not outside the scope of a reasonable request in support of Fedora's own virtualization stack. Therefore, I would take your argument as basically We don't want it in the base OS any more, and we don't care about our virtualization stack, so go away. I don't think he missed the point at all, except maybe missing that some people don't care about supporting a reasonably functioning virtualization stack in Fedora. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/22/2011 02:47 PM, Richard W.M. Jones wrote: There is a further issue #2, quite orthogonal, which is that grub (upstream) doesn't support offline installation. This is a bug in grub 1 2 which really should be taken upstream. You're still missing the point here. This wasn't a design feature of grub or of grub2. It's not a bug, it's a thing nobody has ever planned for. Additionally, there is no grub 1 upstream. You're never going to convince any upstream to change this, because there's nobody working on it. There hasn't been an upstream since 2005. It needs to be jettisoned into space. Nevertheless, it's quite possible to use grub1 offline on compatible guests. We don't really need to prove this, because we do it, and test the results, and we publish everything we do in open source code. And that's not going to be possible when we don't install grub1 on the host system. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 22, 2011 at 09:23:40PM +0200, Miloslav Trmač wrote: On Thu, Sep 22, 2011 at 8:40 PM, Matthew Garrett mj...@srcf.ucam.org wrote: On Thu, Sep 22, 2011 at 07:38:54PM +0100, Richard W.M. Jones wrote: On Thu, Sep 22, 2011 at 05:37:39PM +0100, Matthew Garrett wrote: We allow you to inspect the guest to find the OS version, and even versions of individual packages installed in it. The package version of grub does nothing to tell you which version of grub is actually installed in the boot sector. That doesn't matter: When libguestfs runs its installation of grub, the boot sector will be overwritten. What matters is the versions of stage1/1.5/2 files in the guest, which is adequately described by identifying the RPM. And now you're assuming that sector list will always be in the same place. Which also isn't guaranteed. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/22/2011 04:07 PM, Doug Ledford wrote: Fedora ships a virtualization environment, so while grub1 should go away as soon as possible in terms of Fedora's own use, having it around for situation 3 is not outside the scope of a reasonable request in support of Fedora's own virtualization stack. Therefore, I would take your argument as basically We don't want it in the base OS any more, and we don't care about our virtualization stack, so go away. I don't think he missed the point at all, except maybe missing that some people don't care about supporting a reasonably functioning virtualization stack in Fedora. You're basically arguing that we should never remove any software from Fedora in case it's used in a virtual machine hosted on a Fedora machine. This is not a workable scenario. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
Peter Jones wrote: You're basically arguing that we should never remove any software from Fedora in case it's used in a virtual machine hosted on a Fedora machine. This is not a workable scenario. Why, if the virtualization folks are willing to pick up maintainership? You won't have to maintain the old GRUB 1 anymore. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Wed, Sep 21, 2011 at 08:39:24PM +0100, Mark McLoughlin wrote: On Wed, 2011-09-21 at 18:48 +0100, Matthew Garrett wrote: Remember that the incompatibility isn't between libguestfs and the guest, it's between the host grub and the guest grub. Both of those can change without libguestfs's knowledge. Sounds like we need a 'Conflicts: libguestfs' in both grub and grub2 then? I don't think so. Nothing they do or install conflicts with libguestfs. libguestfs is simply trying to use them inappropriately. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/21/2011 03:39 PM, Mark McLoughlin wrote: On Wed, 2011-09-21 at 18:48 +0100, Matthew Garrett wrote: On Wed, Sep 21, 2011 at 06:30:58PM +0100, Mark McLoughlin wrote: On Mon, 2011-09-19 at 20:11 +0100, Matthew Garrett wrote: The grub package (as provided in Fedora) is not designed for that. This would be a much easier discussion to have if you stopped describing things that are manifestly true as not true. And while it is the case that grub *is* binary compatible between every version we've ever released, it is *not* guaranteed that that remains true, or even that it's true between us and any distribution that may be installed in a guest. If libguestfs had code to detect that the guest version was incompatible and failed gracefully with a nice explanation for the user, then there's no problem right? To be reliable you'd need to support disassembling the binaries installed and working out what the arguments are meant to look like. This doesn't seem like a great way to spend time. Not your problem how libguestfs authors spend their time. Nor whether they actually do this or choose to just warn their users about potential incompatibility. Remember that the incompatibility isn't between libguestfs and the guest, it's between the host grub and the guest grub. Both of those can change without libguestfs's knowledge. Sounds like we need a 'Conflicts: libguestfs' in both grub and grub2 then? Yes, but this will hardly help the situation, which right now is that the distro pulls in grub 2, because that's what we've collectively chosen to do, and libguestfs pulls in grub on the host, even though it isn't really using it there. So effectively your solution is to keep the problem we've got right now. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Wed, Sep 21, 2011 at 03:54:28PM -0400, Peter Jones wrote: Yes, but this will hardly help the situation, which right now is that the distro pulls in grub 2, because that's what we've collectively chosen to do, and libguestfs pulls in grub on the host, even though it isn't really using it there. So effectively your solution is to keep the problem we've got right now. Tools on the host are often useful in guest situations. If I created a filesystem using mke2fs: lvcreate /dev/vg/foo mke2fs /dev/vg/foo and attached this to a guest, is that an inappropriate use of host tools for a guest? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Wed, Sep 21, 2011 at 09:10:06PM +0100, Richard W.M. Jones wrote: On Wed, Sep 21, 2011 at 03:54:28PM -0400, Peter Jones wrote: Yes, but this will hardly help the situation, which right now is that the distro pulls in grub 2, because that's what we've collectively chosen to do, and libguestfs pulls in grub on the host, even though it isn't really using it there. So effectively your solution is to keep the problem we've got right now. Tools on the host are often useful in guest situations. If I created a filesystem using mke2fs: lvcreate /dev/vg/foo mke2fs /dev/vg/foo and attached this to a guest, is that an inappropriate use of host tools for a guest? Yes, if the guest is running a sufficiently old kernel. But mke2fs is designed to allow you to create filesystems that will work with arbitrary kernels, so it's possible to use it in appropriate ways. grub is not designed to be compatible across arbitrary versions, and so using it with that expectation is inappropriate. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - On Fri, Sep 16, 2011 at 03:01:06PM -0400, Doug Ledford wrote: On 9/15/2011 12:01 PM, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 04:56:43PM +0100, Richard W.M. Jones wrote: The most obvious case where it can fail involves grub being effectively unmaintained, and so various vendors have extended it in different ways. You may end up with valid configuration files for one distribution that can't be parsed by the grub for another. The assumption you're making is fragile. It's even worse for grub2, since it has a built-in module loader. Modules built for one version of grub aren't guaranteed (or even really expected) to work when loaded into another. No it's not. Grub doesn't install the 1.5 stage or the 2nd stage loaders, it uses the ones present in the root filesystem defined by the install command. In this case, that's going to be the modules in the guest vm filesystem. As such, anything valid in the guest vm's copy of grub will work in the guest vm even if the grub used to install the master boot record comes from the host. grub-install *does* install the 1.5 and 2nd stage loaders. OK, technically it install the 1.5 or the 2.0 if you don't use a 1.5 (if you install both the 1.5 and 2.0, then it patches the name of the 2.0 into the 1.5 that it installs and the 1.5 reads the filesystem to find the 2.0 so that it isn't subject to breaking boot if the 2.0 moves due to an updated version or some such). However, it installs them from the files you list to the install command as I listed in my previous email. So, they are still compatible with the guest vm grub if you follow a procedure like I listed. I haven't looked to see if the 512 byte MBR is hard coded in grub or if it grabs it from /boot/grub and installs what it finds there, so I can't speak to that. Even if it didn't, I'm not convinced it's guaranteed that an arbitrary grub stage 1 can launch an arbitrary grub stage 1.5 since there are assumptions about register and stack state - see start.S. And I'm sure those assumptions haven't changed probably in the last 5 years or more. But, I can look through the old grub CVS versions if it would help settle your concerns. Those sorts of hard coded constants tend not to change much, especially if the coders have any sort of clue what so ever about compatibility issues. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Mon, Sep 19, 2011 at 12:44:58PM -0400, Doug Ledford wrote: OK, technically it install the 1.5 or the 2.0 if you don't use a 1.5 (if you install both the 1.5 and 2.0, then it patches the name of the 2.0 into the 1.5 that it installs and the 1.5 reads the filesystem to find the 2.0 so that it isn't subject to breaking boot if the 2.0 moves due to an updated version or some such). However, it installs them from the files you list to the install command as I listed in my previous email. So, they are still compatible with the guest vm grub if you follow a procedure like I listed. I haven't looked to see if the 512 byte MBR is hard coded in grub or if it grabs it from /boot/grub and installs what it finds there, so I can't speak to that. Sure. If you're installing the files from the guest then there's no problem. But in that case you only need enough code in the host to be able to write the new images, whereas what Richard's been telling us is that the host-side grub binaries are required. Even if it didn't, I'm not convinced it's guaranteed that an arbitrary grub stage 1 can launch an arbitrary grub stage 1.5 since there are assumptions about register and stack state - see start.S. And I'm sure those assumptions haven't changed probably in the last 5 years or more. But, I can look through the old grub CVS versions if it would help settle your concerns. Those sorts of hard coded constants tend not to change much, especially if the coders have any sort of clue what so ever about compatibility issues. Given that you're always supposed to install the stage 1.5/stage 2 along with the mbr, I don't see where compatibility issues come into it. If you're using the code as you're meant to use the code then you'll always be safe. If you're not, it's not guaranteed to be safe. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - On Mon, Sep 19, 2011 at 12:44:58PM -0400, Doug Ledford wrote: OK, technically it install the 1.5 or the 2.0 if you don't use a 1.5 (if you install both the 1.5 and 2.0, then it patches the name of the 2.0 into the 1.5 that it installs and the 1.5 reads the filesystem to find the 2.0 so that it isn't subject to breaking boot if the 2.0 moves due to an updated version or some such). However, it installs them from the files you list to the install command as I listed in my previous email. So, they are still compatible with the guest vm grub if you follow a procedure like I listed. I haven't looked to see if the 512 byte MBR is hard coded in grub or if it grabs it from /boot/grub and installs what it finds there, so I can't speak to that. Sure. If you're installing the files from the guest then there's no problem. But in that case you only need enough code in the host to be able to write the new images, whereas what Richard's been telling us is that the host-side grub binaries are required. The minimal code you need to do what I suggested is the grub binary itself. Shouldn't need grub-install or any of the other support files, but as the grub binary is in the base package, you need the grub package in addition to the grub2 package. Even if it didn't, I'm not convinced it's guaranteed that an arbitrary grub stage 1 can launch an arbitrary grub stage 1.5 since there are assumptions about register and stack state - see start.S. And I'm sure those assumptions haven't changed probably in the last 5 years or more. But, I can look through the old grub CVS versions if it would help settle your concerns. Those sorts of hard coded constants tend not to change much, especially if the coders have any sort of clue what so ever about compatibility issues. Given that you're always supposed to install the stage 1.5/stage 2 along with the mbr, This is incorrect. The whole reason the stage1.5 portion is an fs compatible reader is so that you can update the stage2 file and it will pick the changes up without needing to be reinstalled. This is also born out by the fact that on package update, there is no %post action in the spec to reinstall the mbr and stage 1.5 files even though the stage2 file likely just changed. I don't see where compatibility issues come into it. If you're using the code as you're meant to use the code then you'll always be safe. If you're not, it's not guaranteed to be safe. Like I said, not true. The grub package is designed to be updateable without requiring an mbr reinstall. What's more is I had a look at the stage1.[hS] files in the grub shipped in FC-1 and RHEL-5, and just like I said, they are indeed binary compatible. So even if the grub user space application pulls its MBR from a statically linked copy of the MBR, it will still work with pretty much any stage1.5 or stage2 you find in a guest. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Mon, Sep 19, 2011 at 02:53:11PM -0400, Doug Ledford wrote: This is incorrect. The whole reason the stage1.5 portion is an fs compatible reader is so that you can update the stage2 file and it will pick the changes up without needing to be reinstalled. This is also born out by the fact that on package update, there is no %post action in the spec to reinstall the mbr and stage 1.5 files even though the stage2 file likely just changed. We never update the stage 2 file without reinstalling the mbr and stage 1.5. The output of rpm -qf grub may be instructive. I don't see where compatibility issues come into it. If you're using the code as you're meant to use the code then you'll always be safe. If you're not, it's not guaranteed to be safe. Like I said, not true. The grub package is designed to be updateable without requiring an mbr reinstall. What's more is I had a look at the stage1.[hS] files in the grub shipped in FC-1 and RHEL-5, and just like I said, they are indeed binary compatible. So even if the grub user space application pulls its MBR from a statically linked copy of the MBR, it will still work with pretty much any stage1.5 or stage2 you find in a guest. The grub package (as provided in Fedora) is not designed for that. This would be a much easier discussion to have if you stopped describing things that are manifestly true as not true. And while it is the case that grub *is* binary compatible between every version we've ever released, it is *not* guaranteed that that remains true, or even that it's true between us and any distribution that may be installed in a guest. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Mon, Sep 19, 2011 at 10:53 AM, Doug Ledford dledf...@redhat.com wrote: Like I said, not true. The grub package is designed to be updateable without requiring an mbr reinstall. What's more is I had a look at the stage1.[hS] files in the grub shipped in FC-1 and RHEL-5, and just like I said, they are indeed binary compatible. So even if the grub user space application pulls its MBR from a statically linked copy of the MBR, it will still work with pretty much any stage1.5 or stage2 you find in a guest. Pretty much any? Hmm are you saying that random other linux distribution's grub binaries are garunteed(or promised/expected) to be binary compatible? Other distributors do have the ability to patch that 1.5 stage code in non-binary compatible ways don't they? We aren't talking strictly about the Fedora/RHEL ecosystem are we? Just because RHEL and Fedora have chosen not to include binary incompatible patches, doesn't mean its a truism across the guest OS landscape does it? Is that binary compatibility tested for as part of operation? Is that compatibility strictly a consequence of distribution level decision making concerning Fedora and RHEL? Is that binary compatibility guaranteed or promised from other distribution's grub1 variants being shipped? -jef -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
Matthew Garrett wrote: The output of rpm -qf grub may be instructive. I suppose you mean rpm -ql grub… Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - On Mon, Sep 19, 2011 at 10:53 AM, Doug Ledford dledf...@redhat.com wrote: Like I said, not true. The grub package is designed to be updateable without requiring an mbr reinstall. What's more is I had a look at the stage1.[hS] files in the grub shipped in FC-1 and RHEL-5, and just like I said, they are indeed binary compatible. So even if the grub user space application pulls its MBR from a statically linked copy of the MBR, it will still work with pretty much any stage1.5 or stage2 you find in a guest. Pretty much any? Hmm are you saying that random other linux distribution's grub binaries are garunteed(or promised/expected) to be binary compatible? Can I guarantee it, no. I didn't look at their stage1.[hS] files. However, as I said in my original comment about this, the stage1 loader is *dirt simple*. There is a very low chance that it will be incompatible simply due to the fact that almost nothing interesting happens there. There is a much greater chance in the stage1.5 files though. But, since the grub command line utility, when used as I directed, reads the stage1.5 and stage2 files from the filesystem and doesn't try to use its own internal version of those files, incompatibility at that layer isn't a problem. Other distributors do have the ability to patch that 1.5 stage code in non-binary compatible ways don't they? They do. But unless they have assembler code masochists, they likely don't. We aren't talking strictly about the Fedora/RHEL ecosystem are we? Just because RHEL and Fedora have chosen not to include binary incompatible patches, doesn't mean its a truism across the guest OS landscape does it? No, but again, the chances of a binary incompatible stage1 are very low. Is that binary compatibility tested for as part of operation? You would have to ask the grub maintainer about that. I just know that a grub update does not reinstall the stage1 or stage1.5 files, while the stage2 file is replaced in the filesystem, meaning that the previous stage1.5 file will attempt to use the new stage2 file in chain loading fashion. Whether or not that is ever tested or just assumed to work I can't speak to. Is that compatibility strictly a consequence of distribution level decision making concerning Fedora and RHEL? Between the stage1.5 and stage2, that could very well be. Between the stage1 and stage1.5, it's more a consequence of the fact that the stage1 loader has one very distinct job to perform, in a very small amount of instruction space, and you don't go around adding features to the stage1 loader, it simply doesn't have the room for it. Is that binary compatibility guaranteed or promised from other distribution's grub1 variants being shipped? No clue. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - Matthew Garrett wrote: The output of rpm -qf grub may be instructive. I suppose you mean rpm -ql grub… That worked better. And I see your point. I was mistaken in thinking that the grub files resided directly in /boot/grub. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
- Original Message - On Mon, Sep 19, 2011 at 02:53:11PM -0400, Doug Ledford wrote: This is incorrect. The whole reason the stage1.5 portion is an fs compatible reader is so that you can update the stage2 file and it will pick the changes up without needing to be reinstalled. This is also born out by the fact that on package update, there is no %post action in the spec to reinstall the mbr and stage 1.5 files even though the stage2 file likely just changed. We never update the stage 2 file without reinstalling the mbr and stage 1.5. The output of rpm -qf grub may be instructive. Which has it's own gotchas. I never use grub-install as it does the wrong things in certain circumstances. I always manually run grub then do the install command myself. In my case, after a grub update, I'm going to end up with a newer MBR but older stage1.5 and stage2 files because I didn't know I needed to copy them from /usr/share after an rpm upgrade. I don't see where compatibility issues come into it. If you're using the code as you're meant to use the code then you'll always be safe. If you're not, it's not guaranteed to be safe. Like I said, not true. The grub package is designed to be updateable without requiring an mbr reinstall. What's more is I had a look at the stage1.[hS] files in the grub shipped in FC-1 and RHEL-5, and just like I said, they are indeed binary compatible. So even if the grub user space application pulls its MBR from a statically linked copy of the MBR, it will still work with pretty much any stage1.5 or stage2 you find in a guest. The grub package (as provided in Fedora) is not designed for that. This would be a much easier discussion to have if you stopped describing things that are manifestly true as not true. And while it is the case that grub *is* binary compatible between every version we've ever released, it is *not* guaranteed that that remains true, or even that it's true between us and any distribution that may be installed in a guest. I never said it was guaranteed, just that it was highly likely. And for the stage1 loader, I stand by that. For the stage1.5 and stage2 loaders, they are installed as a pair by the grub install command I sent out in this thread and so incompatibilities between them are taken care of. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 9/15/2011 10:53 AM, Peter Jones wrote: On 09/15/2011 10:36 AM, Richard W.M. Jones wrote: On Thu, Sep 15, 2011 at 03:31:49PM +0100, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 03:27:16PM +0100, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. No. It is not sane to have multiple bootloaders installed on one machine. There's an interesting verbal trick there. multiple bootloaders are not installed. Multiple versions of the grub RPM package are installed. Only one bootloader would be installed on the host. It's really not useful or reasonable to think of grub and grub2 as multiple versions of the same bootloader - they don't share tools, for example. But even so, multiple versions of the same bootloader doesn't make sense either. For use on a single system I would agree. It does, however, make just as much sense as a cross compiler on a different arch host for making binaries for a breadboard. Requiring the ability to do so adds a significant amount of extra complexity to the tools associated with it for no useful benefit. The useful benefit was outlined in the original email. It really wasn't - it's still unclear why anybody would choose to do things that way. On the face it's a completely wrong choice. Well, the primary example he gave in another email was resizing of a guest VM image, which then might mean realigning the image, and if that's done when the image is up and running and done inside the image, then you could readjust the boot loader inside the image. However, as the tool in question is capable of doing this to an offline image (I assume by the way Richard talks), it *must* be able to adjust the boot loader from outside the image or the image won't boot up again. As a general use case, assume you are an admin with 100 different images to manage and you want to resize all of them, having to fire up each image and resize it from inside the guest and adjust the boot loader from inside the guest would be a horrible way to do things. Being able to use a tool on the host to resize all the images in one go and fix up all the boot loaders in one go, from the host, not from the guest, is certainly the way I, as an admin, would want things. Just install the grub package in the guest, and chroot into the guest if you need to run grub-install there. Running tools from out of the guest is insecure. There are several ways in which a guest could exploit the host if we did this. See Security here for some issues: http://libguestfs.org/guestfs.3.html#running_commands I wrote a web page about my opinions does not make them fact. But even if we took as given that it's somehow better not to use packages in the guest, it's still not a reason to have the packages *unpacked and installed* on the host system. Doing this introduces many more chances for exploitation and plain old corruption and errors. At the very least it should be using raw, non-installed packages on the host rather than installed ones. Which, by the way, Fedora already has tools to accomplish. See my above comment about cross-compilers. There are certainly use cases for having the tool install and live on the host. As for security, if you assume that the host is locked down tight with no running services besides sshd and libvirtd, then it is arguably the better place to have a tool like grub installed than in the guest which might be running apache and considerably more open to attack than the host. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Fri, Sep 16, 2011 at 10:29 AM, Doug Ledford dledf...@redhat.com wrote: See my above comment about cross-compilers. There are certainly use cases for having the tool install and live on the host. As for security, if you assume that the host is locked down tight with no running services besides sshd and libvirtd, then it is arguably the better place to have a tool like grub installed than in the guest which might be running apache and considerably more open to attack than the host. I don't think the problem is with the use case per-say. I think problem is grub as a tool was never designed with this use case in mind, and I think what people are trying to say that its amazing that grub toolset has ever worked for Richard's host/guest use case and he's gotten very lucky relying on what is essentially undefined/unverified behavior even prior to the introduction of grub2 when he was primarily working with guests and hosts using different grub1 variants. Virtualization changes the rules of the game, and we have to be very careful not to assume that tools like grub correctly anticipated the needs of a highly virtualized environment. Something is needed, its just not clear to me the tools we have fits the need well enough to be reliable. -jefWhen all you have is a hammer everything looks like a nailspaleta -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 9/15/2011 12:01 PM, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 04:56:43PM +0100, Richard W.M. Jones wrote: For grub1 guests, it has turned out not to matter which specific version of grub [as long as it was grub1] was used, as apparently grub-install updates all files needed in /boot/grub as appropriate. Or at least we haven't come across a guest where this hasn't worked (yet -- we could be in for a surprise). The most obvious case where it can fail involves grub being effectively unmaintained, and so various vendors have extended it in different ways. You may end up with valid configuration files for one distribution that can't be parsed by the grub for another. The assumption you're making is fragile. It's even worse for grub2, since it has a built-in module loader. Modules built for one version of grub aren't guaranteed (or even really expected) to work when loaded into another. No it's not. Grub doesn't install the 1.5 stage or the 2nd stage loaders, it uses the ones present in the root filesystem defined by the install command. In this case, that's going to be the modules in the guest vm filesystem. As such, anything valid in the guest vm's copy of grub will work in the guest vm even if the grub used to install the master boot record comes from the host. The mistake here is in thinking that grub installs the later stage boot modules. It doesn't. It installs the master boot record only (and that's a dirt simply 512 byte assembly block that probably hasn't changed since so long ago you would be hard pressed to find any version of it that isn't byte for byte identical). It then patches in the logical block address of the stage 1.5 boot loader into the MBR. It then patches in the location/name of the stage 2 boot loader into the stage 1.5 boot loader. It then patches in the location/name of the config file into the stage 2 loader. The MBR loader just starts a chain loading process, all the other work happens in the other loaders. These other loaders are present in the guest /boot/grub directory and hence will always be in sync with what the guest vm expects. I'm very interested in how to reinstall bootloaders *without* invoking guest code. Also in how to not break the bootloader when moving or aligning the boot partition, which sometimes happens for reasons we don't understand (but not on all grub1 guests, only on RHEL 5 era grub1). You're asking for the impossible. The only supportable bootloader for a specific guest is the bootloader that matches the installed OS. If you want to support grub2 on Ubuntu, for instance, you'll need Ubuntu's grub2 - not Fedora's. The binary interface may have changed between them. Poppycock. Although I can't speak for grub2, the grub install command in the grub shell does not install anything other than that very static 512 byte MBR that I mentioned, the rest is all chaining of files in /boot/grub and so will match the OS expectations even if the installer is from a different version/os than the guest. The real trick is simply getting everything set up properly. First, I wouldn't use grub-install, it makes too many assumptions. Second, you mount the guest block device on the host, this should give you the partitions, then you mount the guest vm's /boot partition on the host so you can access the files, then you then run the grub shell and issue these commands (assuming that the first partition is the boot partition in the guest vm's block device): (loopback or whatever mount guest vm's block dev) mount /dev/vm_block_dev1 /mnt/vm_boot grub grub device (hd0) /dev/vm_block_dev grub root (hd0,0) grub install --stage2=/mnt/vm_boot/grub/stage2 (hd0) /mnt/vm_boot/grub/e2fs_stage1_5 /mnt/vm_boot/grub/stage2 /mnt/vm_boot/grub/grub.conf grub exit umount /dev/vm_block_dev1 losetup -d /dev/vm_block_dev Now, I would imagine grub2 can be similarly directed to use the various boot files from the guest. Of course, if you are doing all this, it does beg the question of why libguestfs couldn't simply mount both the root and boot partitions of the guest vm, chroot into the root fs, then issue all the above grub commands using the guest vm's copy of grub (I'm assuming grub is installed, after all, it isn't guaranteed to be able to boot in the future if you uninstall the grub rpm package after guest installation). Of course, the virtual block devices in the guest would not be the same as they would under the host with the guest block device mounted, so there would need to be some fiddling with device names and such. -- Doug Ledford dledf...@redhat.com GPG KeyID: CFBFF194 http://people.redhat.com/dledford Infiniband specific RPMs available at http://people.redhat.com/dledford/Infiniband signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Fri, Sep 16, 2011 at 03:01:06PM -0400, Doug Ledford wrote: On 9/15/2011 12:01 PM, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 04:56:43PM +0100, Richard W.M. Jones wrote: The most obvious case where it can fail involves grub being effectively unmaintained, and so various vendors have extended it in different ways. You may end up with valid configuration files for one distribution that can't be parsed by the grub for another. The assumption you're making is fragile. It's even worse for grub2, since it has a built-in module loader. Modules built for one version of grub aren't guaranteed (or even really expected) to work when loaded into another. No it's not. Grub doesn't install the 1.5 stage or the 2nd stage loaders, it uses the ones present in the root filesystem defined by the install command. In this case, that's going to be the modules in the guest vm filesystem. As such, anything valid in the guest vm's copy of grub will work in the guest vm even if the grub used to install the master boot record comes from the host. grub-install *does* install the 1.5 and 2nd stage loaders. Even if it didn't, I'm not convinced it's guaranteed that an arbitrary grub stage 1 can launch an arbitrary grub stage 1.5 since there are assumptions about register and stack state - see start.S. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Fri, Sep 16, 2011 at 03:01:06PM -0400, Doug Ledford wrote: Of course, if you are doing all this, it does beg the question of why libguestfs couldn't simply mount both the root and boot partitions of the guest vm, chroot into the root fs, then issue all the above grub commands using the guest vm's copy of grub (I'm assuming grub is installed, after all, it isn't guaranteed to be able to boot in the future if you uninstall the grub rpm package after guest installation). I'm wary about running guest code. Such code can trivially take over the appliance, and from there try to exploit qemu or send back bogus data to the library part. I like to think that libguestfs is programmed defensively, but it's very easy to make mistakes in a large C library (let alone qemu). For more see the security: sub-heading here: http://libguestfs.org/guestfs.3.html#running_commands Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 03:27:16PM +0100, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. No. It is not sane to have multiple bootloaders installed on one machine. Requiring the ability to do so adds a significant amount of extra complexity to the tools associated with it for no useful benefit. Just install the grub package in the guest, and chroot into the guest if you need to run grub-install there. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 03:31:49PM +0100, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 03:27:16PM +0100, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. No. It is not sane to have multiple bootloaders installed on one machine. There's an interesting verbal trick there. multiple bootloaders are not installed. Multiple versions of the grub RPM package are installed. Only one bootloader would be installed on the host. Requiring the ability to do so adds a significant amount of extra complexity to the tools associated with it for no useful benefit. The useful benefit was outlined in the original email. Just install the grub package in the guest, and chroot into the guest if you need to run grub-install there. Running tools from out of the guest is insecure. There are several ways in which a guest could exploit the host if we did this. See Security here for some issues: http://libguestfs.org/guestfs.3.html#running_commands Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/15/2011 10:27 AM, Richard W.M. Jones wrote: This is about: https://bugzilla.redhat.com/show_bug.cgi?id=737261 F16 TC2 DVD grub/grub2 conflict The grub package in F16 has a Conflicts: grub2 line. There are no actual file conflicts, but this was added in order to workaround some bugs in grubby, including: As mjg59 said - it's really not reasonable to have multiple bootloaders installed on a system (especially by default, as you're essentially asking!) It greatly increases the complexity in our tooling, as well as greatly increasing the chances somebody accidentally bricks a box by running tools for the bootloader they're not using. - https://bugzilla.redhat.com/show_bug.cgi?id=725185 - https://bugzilla.redhat.com/show_bug.cgi?id=731226 Now the problem is that libguestfs provides a way for people to use grub-install (and in the future grub2-install) on guests. Think for example if you had a mix of Fedora 15 and Fedora 16 guests on your host. libguestfs, as I guess is well known, uses tools from the host in order to manage guests. Honestly I don't think this is that well known, and looking at it I'm amazed this passed package review. Why aren't you guys using any of the tools we've got to build images from packages instead of installing things on the host? This is terribly bad behavior. This is done so that we don't have to separately package all the tools, which would be a security headache (if there's a security issue discovered in grub-install, just updating the host package is enough to fix it -- you don't have to track down a separate copy statically bundled in libguestfs). Trading half of one security problem (since you're not eliminating the problem of security bugs already being installed in a guest) for the problem of installing lots of extra packages on the host? Not a win. Even if you don't care about libguestfs, you might well wish to loop-mount an old guest and run grub-install --root-directory=... on it. Nice strawman, but in that case grub-install will already be on any properly installed guest. So I propose that we drop this conflicts and fix grubby instead. We certainly can't do that without at least first fixing other problems. It'd also be good to think about how we can fix libguestfs's terrible behavior. However the maintainer of grub is unwilling to do this, which is why I've escalated this issue here. I've been asking you to explain why you need this and you stopped participating in the conversation and started this thread instead. There are a number of problems with this, not least that it's not how engineering escalations even work. In any case, I *still* haven't said I'm unwilling to remove the conflicts (though we do need to debug the real problem and see if it's solvable first), though in general I think it's a correct thing to have there. I wanted to understand why you needed it first. You've been mighty uncooperative at getting your own problem solved. -- Peter Power corrupts. Absolute power is kind of neat. -- John Lehman, Secretary of the Navy, 1981-1987 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/15/2011 10:36 AM, Richard W.M. Jones wrote: On Thu, Sep 15, 2011 at 03:31:49PM +0100, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 03:27:16PM +0100, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. No. It is not sane to have multiple bootloaders installed on one machine. There's an interesting verbal trick there. multiple bootloaders are not installed. Multiple versions of the grub RPM package are installed. Only one bootloader would be installed on the host. It's really not useful or reasonable to think of grub and grub2 as multiple versions of the same bootloader - they don't share tools, for example. But even so, multiple versions of the same bootloader doesn't make sense either. Requiring the ability to do so adds a significant amount of extra complexity to the tools associated with it for no useful benefit. The useful benefit was outlined in the original email. It really wasn't - it's still unclear why anybody would choose to do things that way. On the face it's a completely wrong choice. Just install the grub package in the guest, and chroot into the guest if you need to run grub-install there. Running tools from out of the guest is insecure. There are several ways in which a guest could exploit the host if we did this. See Security here for some issues: http://libguestfs.org/guestfs.3.html#running_commands I wrote a web page about my opinions does not make them fact. But even if we took as given that it's somehow better not to use packages in the guest, it's still not a reason to have the packages *unpacked and installed* on the host system. Doing this introduces many more chances for exploitation and plain old corruption and errors. At the very least it should be using raw, non-installed packages on the host rather than installed ones. Which, by the way, Fedora already has tools to accomplish. -- Peter Power corrupts. Absolute power is kind of neat. -- John Lehman, Secretary of the Navy, 1981-1987 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 03:36:55PM +0100, Richard W.M. Jones wrote: On Thu, Sep 15, 2011 at 03:31:49PM +0100, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 03:27:16PM +0100, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. No. It is not sane to have multiple bootloaders installed on one machine. There's an interesting verbal trick there. multiple bootloaders are not installed. Multiple versions of the grub RPM package are installed. Only one bootloader would be installed on the host. grub and grub2 are different packages with approximately no code in common. They're different bootloaders. We don't support having multiple different bootloaders installed. Just install the grub package in the guest, and chroot into the guest if you need to run grub-install there. Running tools from out of the guest is insecure. There are several ways in which a guest could exploit the host if we did this. See Security here for some issues: http://libguestfs.org/guestfs.3.html#running_commands We're talking about guest creation, aren't we? Why would you ever need to run grub-install against a guest image that already exists? And if you do, you're already going to have problems come F17. It's likely that grub will no longer exist, but F15 guests will still need it rather than grub2. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 10:46:34AM -0400, Peter Jones wrote: On 09/15/2011 10:27 AM, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. We certainly can't do that without at least first fixing other problems. Could you explain (preferably with a full list of bugs) what you were trying to solve with the conflicts line in the spec file? The only bugs I've seen so far describe problems in grubby, and this appears to be a workaround for them. However it may be I don't have the full picture. However the maintainer of grub is unwilling to do this, which is why I've escalated this issue here. I've been asking you to explain why you need this and you stopped participating in the conversation and started this thread instead. Since you starting swearing at me on IRC, I thought it better that we discuss this in technical terms, and mailing lists are in any case a better forum for technical discussions than chat. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 03:59:57PM +0100, Matthew Garrett wrote: We're talking about guest creation, aren't we? No, we're talking about fixing and resizing existing guests, where grub-install needs to be run to fix the bootloader. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 04:21:36PM +0100, Richard W.M. Jones wrote: On Thu, Sep 15, 2011 at 03:59:57PM +0100, Matthew Garrett wrote: We're talking about guest creation, aren't we? No, we're talking about fixing and resizing existing guests, where grub-install needs to be run to fix the bootloader. So how do you ensure that the version you run is the same version as the package installed in the guest? Having those not match is an invitation for bizarre failure down the line. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/15/2011 11:16 AM, Richard W.M. Jones wrote: On Thu, Sep 15, 2011 at 10:46:34AM -0400, Peter Jones wrote: On 09/15/2011 10:27 AM, Richard W.M. Jones wrote: So I propose that we drop this conflicts and fix grubby instead. We certainly can't do that without at least first fixing other problems. Could you explain (preferably with a full list of bugs) what you were trying to solve with the conflicts line in the spec file? The only bugs I've seen so far describe problems in grubby, and this appears to be a workaround for them. However it may be I don't have the full picture. You're correct that this initially was added to work around a problem in grubby, and I fully intend to further investigate that, as I said above. But the fact remains that having multiple bootloaders installed, especially with such similarly named tools, is a *bad idea*. However the maintainer of grub is unwilling to do this, which is why I've escalated this issue here. I've been asking you to explain why you need this and you stopped participating in the conversation and started this thread instead. Since you starting swearing at me on IRC, I thought it better that we discuss this in technical terms, and mailing lists are in any case a better forum for technical discussions than chat. Nice try - but the (somewhat reasonable) swearing was *after* you decided to stop participating in the conversation and decided to move the discussion elsewhere instead of helping to find the best way to solve the problem. In no way is walking away from an ongoing conversation with relevant other people who are trying to help you an appropriate response. At best it's just rude. That's why after you did that, I said that you were being an asshole. I apologize if my language offended you; I was frustrated that you began behaving in an uncooperative manor when Matthew and I were trying to analyze the full scale of the problem and find possible solutions to it. I should have behaved more excellently towards you, even after you refused to answer our questions. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
I will simply say that this is not my view of what happened. In any case I hope we can be more excellent about this now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 10:46:34AM -0400, Peter Jones wrote: On 09/15/2011 10:27 AM, Richard W.M. Jones wrote: libguestfs, as I guess is well known, uses tools from the host in order to manage guests. Honestly I don't think this is that well known, and looking at it I'm amazed this passed package review. Why aren't you guys using any of the tools we've got to build images from packages instead of installing things on the host? This is terribly bad behavior. I'm assuming you mean 'appliance-creator'? This tool isn't really suitable; it's doing something quite different from what libguestfs needs/uses. They're solving different problems. I'd suggest that you familiarize yourself with how libguestfs goes about it first. It's pretty well documented and I've included some links below. Then if there specific packaging issues that could be solved better then I'm very interested to know. Rich. For an overview of what libguestfs is: http://libguestfs.org/ For an overview of the supermin appliance that we use and how it is made: http://libguestfs.org/febootstrap.8.html http://libguestfs.org/febootstrap-supermin-helper.8.html https://rwmj.wordpress.com/2010/12/10/tip-creating-throwaway-appliances-with-febootstrap/ https://rwmj.wordpress.com/2009/10/22/supermin-appliance-now-in-febootstrap/ For some more specifics on the internals of libguestfs: http://libguestfs.org/guestfs.3.html#internals -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 04:25:41PM +0100, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 04:21:36PM +0100, Richard W.M. Jones wrote: On Thu, Sep 15, 2011 at 03:59:57PM +0100, Matthew Garrett wrote: We're talking about guest creation, aren't we? No, we're talking about fixing and resizing existing guests, where grub-install needs to be run to fix the bootloader. So how do you ensure that the version you run is the same version as the package installed in the guest? Having those not match is an invitation for bizarre failure down the line. Well, this is interesting. We're planning to snoop out whether the guest is using grub1 or grub2. Previously this hasn't worked at all for (eg) Ubuntu guests. This is why providing grub2 in Fedora is great for us. For grub1 guests, it has turned out not to matter which specific version of grub [as long as it was grub1] was used, as apparently grub-install updates all files needed in /boot/grub as appropriate. Or at least we haven't come across a guest where this hasn't worked (yet -- we could be in for a surprise). I'm very interested in how to reinstall bootloaders *without* invoking guest code. Also in how to not break the bootloader when moving or aligning the boot partition, which sometimes happens for reasons we don't understand (but not on all grub1 guests, only on RHEL 5 era grub1). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 04:56:43PM +0100, Richard W.M. Jones wrote: For grub1 guests, it has turned out not to matter which specific version of grub [as long as it was grub1] was used, as apparently grub-install updates all files needed in /boot/grub as appropriate. Or at least we haven't come across a guest where this hasn't worked (yet -- we could be in for a surprise). The most obvious case where it can fail involves grub being effectively unmaintained, and so various vendors have extended it in different ways. You may end up with valid configuration files for one distribution that can't be parsed by the grub for another. The assumption you're making is fragile. It's even worse for grub2, since it has a built-in module loader. Modules built for one version of grub aren't guaranteed (or even really expected) to work when loaded into another. I'm very interested in how to reinstall bootloaders *without* invoking guest code. Also in how to not break the bootloader when moving or aligning the boot partition, which sometimes happens for reasons we don't understand (but not on all grub1 guests, only on RHEL 5 era grub1). You're asking for the impossible. The only supportable bootloader for a specific guest is the bootloader that matches the installed OS. If you want to support grub2 on Ubuntu, for instance, you'll need Ubuntu's grub2 - not Fedora's. The binary interface may have changed between them. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/15/2011 12:01 PM, Matthew Garrett wrote: On Thu, Sep 15, 2011 at 04:56:43PM +0100, Richard W.M. Jones wrote: For grub1 guests, it has turned out not to matter which specific version of grub [as long as it was grub1] was used, as apparently grub-install updates all files needed in /boot/grub as appropriate. Or at least we haven't come across a guest where this hasn't worked (yet -- we could be in for a surprise). The most obvious case where it can fail involves grub being effectively unmaintained, and so various vendors have extended it in different ways. You may end up with valid configuration files for one distribution that can't be parsed by the grub for another. The assumption you're making is fragile. It's even worse for grub2, since it has a built-in module loader. Modules built for one version of grub aren't guaranteed (or even really expected) to work when loaded into another. Hmm... there isn't a version check to prevent this Seems sort of fragile. I'm very interested in how to reinstall bootloaders *without* invoking guest code. Also in how to not break the bootloader when moving or aligning the boot partition, which sometimes happens for reasons we don't understand (but not on all grub1 guests, only on RHEL 5 era grub1). You're asking for the impossible. The only supportable bootloader for a specific guest is the bootloader that matches the installed OS. If you want to support grub2 on Ubuntu, for instance, you'll need Ubuntu's grub2 - not Fedora's. The binary interface may have changed between them. -- Stephen Clark *NetWolves* Sr. Software Engineer III Phone: 813-579-3200 Fax: 813-882-0209 Email: steve.cl...@netwolves.com http://www.netwolves.com -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/15/2011 09:59 AM, Matthew Garrett wrote: We're talking about guest creation, aren't we? Why would you ever need to run grub-install against a guest image that already exists? And if you do, you're already going to have problems come F17. It's likely that grub will no longer exist, but F15 guests will still need it rather than grub2. Ugh. This sound like it will make it pretty difficult to maintain a system that dual-boots Fedora and RHEL/CentOS/SL (or any other grub 1- only distro). Am I reading this wrong? -- Ian Pilcher arequip...@gmail.com If you're going to shift my paradigm ... at least buy me dinner first. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On 09/15/2011 12:19 PM, Ian Pilcher wrote: On 09/15/2011 09:59 AM, Matthew Garrett wrote: We're talking about guest creation, aren't we? Why would you ever need to run grub-install against a guest image that already exists? And if you do, you're already going to have problems come F17. It's likely that grub will no longer exist, but F15 guests will still need it rather than grub2. Ugh. This sound like it will make it pretty difficult to maintain a system that dual-boots Fedora and RHEL/CentOS/SL (or any other grub 1- only distro). Am I reading this wrong? You can still chain load effectively, or share the whole bootloader. You just can't mix and match, which is effectively what libguestfs winds up doing. -- Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, Sep 15, 2011 at 11:19:24AM -0500, Ian Pilcher wrote: On 09/15/2011 09:59 AM, Matthew Garrett wrote: We're talking about guest creation, aren't we? Why would you ever need to run grub-install against a guest image that already exists? And if you do, you're already going to have problems come F17. It's likely that grub will no longer exist, but F15 guests will still need it rather than grub2. Ugh. This sound like it will make it pretty difficult to maintain a system that dual-boots Fedora and RHEL/CentOS/SL (or any other grub 1- only distro). Am I reading this wrong? grub1 can be installed in a partition from the older OS and then chainbooted by grub 2, if that's you're preferred way of doing things. -- Matthew Garrett | mj...@srcf.ucam.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
