Re: [libvirt] [PATCHv2 1/2] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests.
On Fri, Oct 24, 2014 at 03:28:33PM -0700, Craig Rodrigues wrote: > On Fri, Oct 24, 2014 at 6:11 AM, Conrad Rad wrote: > > > > > Any idea how soon? Months? A year? 5 years? I'm not comfortable > > postponing improvements indefinitely for vaporware. In the wonderful > > bhyve-UEFI future, we can ignore/warn about . > > > > > Hi, > > I have to agree with Conrad here. I hope that any perceived future > direction of bhyve is not going to be used as an excuse to block > some of the libvirt patches that Conrad is submitting. The stuff > that Conrad is working on overlaps some of the shortcomings in > libvirt/bhyve that > I mentioned here: > https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-October/002857.html > > Fixing these issues in libvirt will make libvirt + bhyve more usable > today. When the bhyve-UEFI stuff comes out in future, > that will be even better, Libvirt has a goal that we never break application compatibility, which includes the way the XML is configured for guests. So when reviewing patches like this series it is important to try to have an awareness of what further patches may be coming down the pipe in the future, so we can avoid painting ourselves into a corner. So this question of Roman's isn't really about blocking patches for future vapourware, but rather about making sure that decisions we make today don't cause us undue problems with UEFI does arrive at some point later. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [PATCHv2 1/2] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests.
On Fri, Oct 24, 2014 at 6:11 AM, Conrad Rad wrote: > > Any idea how soon? Months? A year? 5 years? I'm not comfortable > postponing improvements indefinitely for vaporware. In the wonderful > bhyve-UEFI future, we can ignore/warn about . > > Hi, I have to agree with Conrad here. I hope that any perceived future direction of bhyve is not going to be used as an excuse to block some of the libvirt patches that Conrad is submitting. The stuff that Conrad is working on overlaps some of the shortcomings in libvirt/bhyve that I mentioned here: https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-October/002857.html Fixing these issues in libvirt will make libvirt + bhyve more usable today. When the bhyve-UEFI stuff comes out in future, that will be even better, but there are a few people (me and others) who are trying to put together FreeBSD + bhyve systems today that are viable alternatives to Linux + KVM for managing many VM's. bhyve is rapidly improving, but the lack of libvirt support means that a huge ecosystem of software built for libvirt + KVM cannot be used for bhyve. I hope we can start improving libvirt today to eliminate this problem. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [PATCHv2 1/2] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests.
On Fri, Oct 24, 2014 at 12:58 AM, Roman Bogorodskiy wrote: > Conrad Meyer wrote: > Hi Conrad. > > Thanks for the patch. As for the /tmp, probably storing it somewhere in > BHYVE_STATE_DIR would be more clean. Sure. > However, I'm concerned about libvirt dealing with the grub-bhyve > specifics as it involves some assumptions about its behaviour ahd has > limitations, like the one you pointed with installing from CD. Sure. However, the user can always override the assumptions. And I think the CD case is actually quite easy to fix, if that's a blocking issue. > My idea was to add the following behaviour: > > * bootloader specified and bootloader_args specified -- use that as is > * bootloader not specified, bootloader_args specified -- call > bhyveload $bootloader_args $default_bhyveloader_args > * bootloader and bootloader_args not set -- use defaults (current > behaviour) > * bootloader specified, bootloader_args missing -- fail here Ok. I think the grub-bhyve stuff is just a special case of the 4th bullet here; but otherwise my patch matches up with this behavior. > Actually, I've implemented that already, but didn't send because it > needs some more testing. Ok. I am happy to guinea-pig bhyve improvements for libvirt, if you need it. > The reason of my concern about adding a special handling of grub-bhyve > case that, as I've mentioned, we have to make assumptions and there are > also caveats you mentioned. Probably it'd worth to wait for bhyve to > provide single-step VM boot, i.e. UEFI support, that should appear soon > anyway. Any idea how soon? Months? A year? 5 years? I'm not comfortable postponing improvements indefinitely for vaporware. In the wonderful bhyve-UEFI future, we can ignore/warn about . > There are some comments inline. Thanks! >> -/* Image path */ >> -virCommandAddArg(cmd, "-d"); >> -virCommandAddArg(cmd, virDomainDiskGetSource(disk)); >> +/* XXX cleanup this file. */ >> +fd = mkstemp(tmpmapfile); > > This yields an error in 'syntax-check'. Running 'make syntax-check' > allows to avoid some of the general code and formatting issues. I did run syntax-check, but did not see any error around there. syntax-check has lots of console output; does it log somewhere else too? Thanks, Conrad ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [PATCHv2 1/2] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests.
Conrad Meyer wrote: > We still default to bhyveloader(1) if no explicit bootloader > configuration is supplied in the domain. > > If the /domain/bootloader looks like grub-bhyve and the user doesn't > supply /domain/bootloader_args, we make an intelligent guess and try > chainloading the first partition on the disk. > > Caveats: > - We can't install from CD without explicit bootloader_args. > - We leave a device.map file lying around in /tmp. I don't see a good > way not to do so without reworking the API somewhat. > > Sponsored by: EMC / Isilon storage division Hi Conrad. Thanks for the patch. As for the /tmp, probably storing it somewhere in BHYVE_STATE_DIR would be more clean. However, I'm concerned about libvirt dealing with the grub-bhyve specifics as it involves some assumptions about its behaviour ahd has limitations, like the one you pointed with installing from CD. My idea was to add the following behaviour: * bootloader specified and bootloader_args specified -- use that as is * bootloader not specified, bootloader_args specified -- call bhyveload $bootloader_args $default_bhyveloader_args * bootloader and bootloader_args not set -- use defaults (current behaviour) * bootloader specified, bootloader_args missing -- fail here Actually, I've implemented that already, but didn't send because it needs some more testing. The reason of my concern about adding a special handling of grub-bhyve case that, as I've mentioned, we have to make assumptions and there are also caveats you mentioned. Probably it'd worth to wait for bhyve to provide single-step VM boot, i.e. UEFI support, that should appear soon anyway. There are some comments inline. > Signed-off-by: Conrad Meyer > --- > docs/drvbhyve.html.in | 28 ++-- > docs/formatdomain.html.in | 4 +- > po/libvirt.pot| 4 ++ > src/bhyve/bhyve_command.c | 107 > +- > 4 files changed, 128 insertions(+), 15 deletions(-) > > diff --git a/docs/drvbhyve.html.in b/docs/drvbhyve.html.in > index 39afdf5..6e85800 100644 > --- a/docs/drvbhyve.html.in > +++ b/docs/drvbhyve.html.in > @@ -37,8 +37,7 @@ bhyve+ssh://r...@example.com/system (remote access, SSH > tunnelled) > Example config > > The bhyve driver in libvirt is in its early stage and under active > development. So it supports > -only limited number of features bhyve provides. All the supported features > could be found > -in this sample domain XML. > +only limited number of features bhyve provides. > > > > @@ -50,8 +49,8 @@ up to 31 PCI devices. > > >> - bhyve > -df3be7e7-a104-11e3-aeb0-50e5492bd3dc > +bhyve > +df3be7e7-a104-11e3-aeb0-50e5492bd3dc >219136 >219136 >1 > @@ -157,5 +156,26 @@ An example of domain XML device entry for that will look > like: > Please refer to the Storage documentation for > more details on storage > management. > > +Using grub2-bhyve or Alternative Bootloaders > + > +It's possible to boot non-FreeBSD guests by specifying an explicit > +bootloader, e.g. grub-bhyve(1). Arguments to the bootloader may > be > +specified as well. If no arguments are given and bootloader is > +grub-bhyve, libvirt will try and boot from the first partition > of > +the disk image. > + > + > + ... > +/usr/local/sbin/grub-bhyve > +... > + ... > + > + > +(Of course, to install from a CD a user will have to supply explicit > +arguments to grub-bhyve.) > + > +Caveat: bootloader_args does not support any quoting. > +Filenames, etc, must not have spaces or they will be tokenized > incorrectly. > + > > > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in > index 0099ce7..b7b6c46 100644 > --- a/docs/formatdomain.html.in > +++ b/docs/formatdomain.html.in > @@ -217,7 +217,9 @@ >a BIOS, and instead the host is responsible to kicking off the >operating system boot. This may use a pseudo-bootloader in the >host to provide an interface to choose a kernel for the guest. > - An example is pygrub with Xen. > + An example is pygrub with Xen. The Bhyve hypervisor > + also uses a host bootloader, either bhyveload or > + grub-bhyve. > > > > diff --git a/po/libvirt.pot b/po/libvirt.pot > index 0b44ad7..d8c9a4d 100644 > --- a/po/libvirt.pot > +++ b/po/libvirt.pot > @@ -851,6 +851,10 @@ msgstr "" > msgid "domain should have at least one disk defined" > msgstr "" > > +#: src/bhyve/bhyve_command.c:407 > +msgid "Custom loader requires explicit %s configuration" > +msgstr "" > + > #: src/bhyve/bhyve_device.c:50 > msgid "PCI bus 0 slot 1 is reserved for the implicit LPC PCI-ISA bridge" > msgstr "" > diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c > index bea4a59..99956ae 1006
[PATCHv2 1/2] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests.
We still default to bhyveloader(1) if no explicit bootloader configuration is supplied in the domain. If the /domain/bootloader looks like grub-bhyve and the user doesn't supply /domain/bootloader_args, we make an intelligent guess and try chainloading the first partition on the disk. Caveats: - We can't install from CD without explicit bootloader_args. - We leave a device.map file lying around in /tmp. I don't see a good way not to do so without reworking the API somewhat. Sponsored by: EMC / Isilon storage division Signed-off-by: Conrad Meyer --- docs/drvbhyve.html.in | 28 ++-- docs/formatdomain.html.in | 4 +- po/libvirt.pot| 4 ++ src/bhyve/bhyve_command.c | 107 +- 4 files changed, 128 insertions(+), 15 deletions(-) diff --git a/docs/drvbhyve.html.in b/docs/drvbhyve.html.in index 39afdf5..6e85800 100644 --- a/docs/drvbhyve.html.in +++ b/docs/drvbhyve.html.in @@ -37,8 +37,7 @@ bhyve+ssh://r...@example.com/system (remote access, SSH tunnelled) Example config The bhyve driver in libvirt is in its early stage and under active development. So it supports -only limited number of features bhyve provides. All the supported features could be found -in this sample domain XML. +only limited number of features bhyve provides. @@ -50,8 +49,8 @@ up to 31 PCI devices.- bhyve -df3be7e7-a104-11e3-aeb0-50e5492bd3dc +bhyve +df3be7e7-a104-11e3-aeb0-50e5492bd3dc 219136 219136 1 @@ -157,5 +156,26 @@ An example of domain XML device entry for that will look like: Please refer to the Storage documentation for more details on storage management. +Using grub2-bhyve or Alternative Bootloaders + +It's possible to boot non-FreeBSD guests by specifying an explicit +bootloader, e.g. grub-bhyve(1). Arguments to the bootloader may be +specified as well. If no arguments are given and bootloader is +grub-bhyve, libvirt will try and boot from the first partition of +the disk image. + + + ... +/usr/local/sbin/grub-bhyve +... + ... + + +(Of course, to install from a CD a user will have to supply explicit +arguments to grub-bhyve.) + +Caveat: bootloader_args does not support any quoting. +Filenames, etc, must not have spaces or they will be tokenized incorrectly. + diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 0099ce7..b7b6c46 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -217,7 +217,9 @@ a BIOS, and instead the host is responsible to kicking off the operating system boot. This may use a pseudo-bootloader in the host to provide an interface to choose a kernel for the guest. - An example is pygrub with Xen. + An example is pygrub with Xen. The Bhyve hypervisor + also uses a host bootloader, either bhyveload or + grub-bhyve. diff --git a/po/libvirt.pot b/po/libvirt.pot index 0b44ad7..d8c9a4d 100644 --- a/po/libvirt.pot +++ b/po/libvirt.pot @@ -851,6 +851,10 @@ msgstr "" msgid "domain should have at least one disk defined" msgstr "" +#: src/bhyve/bhyve_command.c:407 +msgid "Custom loader requires explicit %s configuration" +msgstr "" + #: src/bhyve/bhyve_device.c:50 msgid "PCI bus 0 slot 1 is reserved for the implicit LPC PCI-ISA bridge" msgstr "" diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c index bea4a59..99956ae 100644 --- a/src/bhyve/bhyve_command.c +++ b/src/bhyve/bhyve_command.c @@ -300,6 +300,7 @@ virBhyveProcessBuildLoadCmd(virConnectPtr conn, { virCommandPtr cmd; virDomainDiskDefPtr disk; +bool bhyveload, grub_bhyve; if (def->ndisks < 1) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", @@ -326,19 +327,105 @@ virBhyveProcessBuildLoadCmd(virConnectPtr conn, return NULL; } -cmd = virCommandNew(BHYVELOAD); +if (def->os.bootloader == NULL) { +bhyveload = true; +grub_bhyve = false; +cmd = virCommandNew(BHYVELOAD); +} else { +bhyveload = false; +if (strstr(def->os.bootloader, "grub-bhyve") == 0) +grub_bhyve = true; +cmd = virCommandNew(def->os.bootloader); +} -/* Memory */ -virCommandAddArg(cmd, "-m"); -virCommandAddArgFormat(cmd, "%llu", - VIR_DIV_UP(def->mem.max_balloon, 1024)); +if (bhyveload && def->os.bootloaderArgs == NULL) { +VIR_DEBUG("%s: bhyveload with default arguments", __func__); + +/* Memory (MB) */ +virCommandAddArg(cmd, "-m"); +virCommandAddArgFormat(cmd, "%llu", + VIR_DIV_UP(def->mem.max_balloon, 1024)); + +/* Image path */ +virCommandAddArg(cmd, "-d"); +virCommandAddArg(cmd, virDomainDiskGetSource(disk
