Re: HEADS UP: merged PVHv2 support and future plans
On Sat, Aug 25, 2018 at 06:54:54PM +0200, Kai Otto wrote: > On 13/08/18 19:19, Roger Pau Monné wrote: > > On Mon, Aug 13, 2018 at 05:24:04PM +0200, Kai Otto wrote: > >> On 30/07/18 12:19, Roger Pau Monné wrote: > >>> Packages for Xen 4.11 are now available in pkg. > >>> > >>> Remember that in order to use those you need a very recent FreeBSD > >>> kernel (r336475 or newer) and the Xen command line has slightly changed, > >>> so dom0=pvh must be used instead of dom0pvh in order to boot. I will > >>> see about changing this in the handbook. > >>> > >>> Roger. > >>> ___ > >>> freebsd-xen@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-xen > >>> To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org" > >>> > >> > >> Hi Roger, > >> I tried to get some VMs running on > >> FreeBSD-12.0-CURRENT-amd64-20180802-r337160. > >> System is a Supermicro X9DRi-LN4F+ mainboard with dual Intel E5-2680v2 > >> (ivy bridge). > [...] > >> When using xen-tools411/xen-kernel411, I can boot without problems, but > >> when running "xl create", neither a PVH nor HVM config work (see configs > >> and logs attached). > >> They both throw "libxl: info: > >> libxl_create.c:109:libxl__domain_build_info_setdefault: qemu-xen is > >> unavailable, using qemu-xen-traditional instead: No such file or directory" > > > > Hello, > > > > This is a mistake of the port makefile which should be fixed in > > r477080 that I've just committed. I hope updated packages will be > > generated soon. Thanks for the testing! > > > > Hello again :-) > > Today I gave it another try with FreeBSD 12.0-ALPHA3 r338287 and the > current packages for xen-tools411 and xen-kernel411. Sadly, I didn't get > very far, even with the above bug being fixed. > When attempting to start a HVM config, the console (xl console ) > stays empty and unresponsive. > Connecting via vnc, the screen only shows this message: "Guest has not > initialized the display (yet)". I think this should be fixed by r477953, please make sure you have the following package installed: xen-tools411-4.11.0_4 > When repeatedly running 'xl list', I can see that the domU almost > consumes 1 second of CPU per second walltime. > > Trying out PVH, the console also doesn't show any sign of life, and VNC > says: "This VM has no graphic display device." > Running 'xl list' shows, that the domU consumed about 1.4s of CPU, and > then switched to 'blocked' state. Can you paste the config file you are using to create the PVH guest? And which OS/version are you trying to boot as PVH? Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On 13/08/18 19:19, Roger Pau Monné wrote: > On Mon, Aug 13, 2018 at 05:24:04PM +0200, Kai Otto wrote: >> On 30/07/18 12:19, Roger Pau Monné wrote: >>> Packages for Xen 4.11 are now available in pkg. >>> >>> Remember that in order to use those you need a very recent FreeBSD >>> kernel (r336475 or newer) and the Xen command line has slightly changed, >>> so dom0=pvh must be used instead of dom0pvh in order to boot. I will >>> see about changing this in the handbook. >>> >>> Roger. >>> ___ >>> freebsd-xen@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-xen >>> To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org" >>> >> >> Hi Roger, >> I tried to get some VMs running on >> FreeBSD-12.0-CURRENT-amd64-20180802-r337160. >> System is a Supermicro X9DRi-LN4F+ mainboard with dual Intel E5-2680v2 >> (ivy bridge). [...] >> When using xen-tools411/xen-kernel411, I can boot without problems, but >> when running "xl create", neither a PVH nor HVM config work (see configs >> and logs attached). >> They both throw "libxl: info: >> libxl_create.c:109:libxl__domain_build_info_setdefault: qemu-xen is >> unavailable, using qemu-xen-traditional instead: No such file or directory" > > Hello, > > This is a mistake of the port makefile which should be fixed in > r477080 that I've just committed. I hope updated packages will be > generated soon. Thanks for the testing! > Hello again :-) Today I gave it another try with FreeBSD 12.0-ALPHA3 r338287 and the current packages for xen-tools411 and xen-kernel411. Sadly, I didn't get very far, even with the above bug being fixed. When attempting to start a HVM config, the console (xl console ) stays empty and unresponsive. Connecting via vnc, the screen only shows this message: "Guest has not initialized the display (yet)". When repeatedly running 'xl list', I can see that the domU almost consumes 1 second of CPU per second walltime. Trying out PVH, the console also doesn't show any sign of life, and VNC says: "This VM has no graphic display device." Running 'xl list' shows, that the domU consumed about 1.4s of CPU, and then switched to 'blocked' state. I can't see any errors in the log file either. Do you or anyone else have a working .cfg file that you could share, or any ideas what might be going wrong? - Kai ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On Mon, Aug 13, 2018 at 05:24:04PM +0200, Kai Otto wrote:
> On 30/07/18 12:19, Roger Pau Monné wrote:
> > Packages for Xen 4.11 are now available in pkg.
> >
> > Remember that in order to use those you need a very recent FreeBSD
> > kernel (r336475 or newer) and the Xen command line has slightly changed,
> > so dom0=pvh must be used instead of dom0pvh in order to boot. I will
> > see about changing this in the handbook.
> >
> > Roger.
> > ___
> > freebsd-xen@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-xen
> > To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
> >
>
> Hi Roger,
> I tried to get some VMs running on
> FreeBSD-12.0-CURRENT-amd64-20180802-r337160.
> System is a Supermicro X9DRi-LN4F+ mainboard with dual Intel E5-2680v2
> (ivy bridge).
>
> When using xen-tools47/xen-kernel47, during boot of dom0, I only get the
> error seen in the attached png ("d0v0 Triple fault").
>
>
> When using xen-tools411/xen-kernel411, I can boot without problems, but
> when running "xl create", neither a PVH nor HVM config work (see configs
> and logs attached).
> They both throw "libxl: info:
> libxl_create.c:109:libxl__domain_build_info_setdefault: qemu-xen is
> unavailable, using qemu-xen-traditional instead: No such file or directory"
Hello,
This is a mistake of the port makefile which should be fixed in
r477080 that I've just committed. I hope updated packages will be
generated soon. Thanks for the testing!
> Which doesn't make much sense to me, as PVH shouldn't use qemu at all?
Some PVH guests will use QEMU in order to run the PV disk backend or
the PV framebuffer (vfb), but PVH won't have any of the emulated
devices by QEMU.
Roger.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On 30/07/18 12:19, Roger Pau Monné wrote:
> Packages for Xen 4.11 are now available in pkg.
>
> Remember that in order to use those you need a very recent FreeBSD
> kernel (r336475 or newer) and the Xen command line has slightly changed,
> so dom0=pvh must be used instead of dom0pvh in order to boot. I will
> see about changing this in the handbook.
>
> Roger.
> ___
> freebsd-xen@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-xen
> To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
>
Hi Roger,
I tried to get some VMs running on
FreeBSD-12.0-CURRENT-amd64-20180802-r337160.
System is a Supermicro X9DRi-LN4F+ mainboard with dual Intel E5-2680v2
(ivy bridge).
When using xen-tools47/xen-kernel47, during boot of dom0, I only get the
error seen in the attached png ("d0v0 Triple fault").
When using xen-tools411/xen-kernel411, I can boot without problems, but
when running "xl create", neither a PVH nor HVM config work (see configs
and logs attached).
They both throw "libxl: info:
libxl_create.c:109:libxl__domain_build_info_setdefault: qemu-xen is
unavailable, using qemu-xen-traditional instead: No such file or directory"
Which doesn't make much sense to me, as PVH shouldn't use qemu at all?
The config files were put together following the documentation in
http://xenbits.xen.org/docs/4.11-testing/man/xl.cfg.5.html
which marks some of the keywords or syntax used in the FreeBSD handbook
as deprecated (i.e. 'builder' vs. 'type').
If I should do something different, or test a patch or something, let me
know. I'm super excited to see a recent Xen version ported to FreeBSD! :-)
- Kai
type="hvm"
name = "test_server"
memory = 512
vcpus = 1
vif = [ 'bridge=bridge0' ]
disk = [ '/dev/zvol/zbulk/vm/universal_media_server,raw,hda,rw',
'/zbulk/mediashare/isos/linux/alpine-virt-3.8.0-x86_64.iso,raw,hdc,r,cdrom'
]
vnc = 1
vnclisten = "0.0.0.0"
usbdevice = ['tablet']
type='pvh'
name = "test_server"
memory = 512
vcpus = 1
kernel = "/mnt/boot/vmlinuz-virt"
ramdisk = "/mnt/boot/initramfs-virt"
vif = [ 'bridge=bridge0' ]
vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1']
disk = [ '/dev/zvol/zbulk/vm/universal_media_server,raw,xvda,rw',
'/zbulk/mediashare/isos/linux/alpine-virt-3.8.0-x86_64.iso,raw,xvdb,r,cdrom'
]
usbdevice = "tablet"
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On Thu, Jul 19, 2018 at 11:04:44AM +0200, Roger Pau Monné wrote: [...] > After introducing PVHv2 PVHv1 was deprecated and PVHv1 has been > removed from the hypervisor in recent versions, that's why the Xen > ports package is still stuck with Xen 4.7, because later versions > removed PVHv1 support. With the addition of PVHv2 to FreeBSD the port > can be updated to newer Xen versions and we can move forward. Packages for Xen 4.11 are now available in pkg. Remember that in order to use those you need a very recent FreeBSD kernel (r336475 or newer) and the Xen command line has slightly changed, so dom0=pvh must be used instead of dom0pvh in order to boot. I will see about changing this in the handbook. Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On Thu, 19 Jul 2018, Roger Pau Monné wrote: > On Thu, Jul 19, 2018 at 12:46:07PM +, Marcin Cieslak wrote: > > On Thu, 19 Jul 2018, Roger Pau Monné wrote: > > > > > Hello, > > > > > > Today I've merged PVHv2 support into FreeBSD, allowing FreeBSD to be > > > used as a PVHv2 DomU and Dom0. While it's not a huge set of changes, > > > I would *really* appreciate if people could test the code starting > > > from r336474 (or any later changeset). > > > > Hi Roger, I am ready to test it on my (small) setup, should I keep > > existing 4.7 running as is and just upgrade the Dom0? > > Yes, just update the FreeBSD kernel. I can at least confirm that my box boot cleanly after the update. Haven't tried launching any domUs yet. Marcin smime.p7s Description: S/MIME Cryptographic Signature
Re: HEADS UP: merged PVHv2 support and future plans
On Thu, Jul 19, 2018 at 12:46:07PM +, Marcin Cieslak wrote: > On Thu, 19 Jul 2018, Roger Pau Monné wrote: > > > Hello, > > > > Today I've merged PVHv2 support into FreeBSD, allowing FreeBSD to be > > used as a PVHv2 DomU and Dom0. While it's not a huge set of changes, > > I would *really* appreciate if people could test the code starting > > from r336474 (or any later changeset). > > Hi Roger, I am ready to test it on my (small) setup, should I keep > existing 4.7 running as is and just upgrade the Dom0? Yes, just update the FreeBSD kernel. > Or should I upgrade Xen too? I haven't updated the Xen package, so there's no easy way ATM to test using a PVHv2 Dom0. Let's focus first on making sure I didn't break existing setups. Thanks, Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On Thu, Jul 19, 2018 at 02:03:00PM +0200, Miroslav Lachman wrote: > Roger Pau Monné wrote on 2018/07/19 11:04: > > Hello, > > > > Today I've merged PVHv2 support into FreeBSD, allowing FreeBSD to be > > used as a PVHv2 DomU and Dom0. While it's not a huge set of changes, > > I would *really* appreciate if people could test the code starting > > from r336474 (or any later changeset). > > > > I expect there's going to be some confusion with PVHv1 vs PVHv2, so I > > will try to clarify this now. PVHv1 was introduced ~4 years ago, and > > at the time it seemed like a good way to move forward, allowing Xen to > > rely more on hardware virtualization. Later on, we sadly discovered > > that PVHv1 was still too similar to classic PV, and didn't allow Xen > > to make use of all the possible hardware virtualization extensions, so > > PVHv2 was introduced ~2 years ago as a replacement for PVHv1. PVHv2 > > ABI however is not compatible with PVHv1, which means that different > > entry points and interfaces must be used to interact with the > > hypervisor. > > > > After introducing PVHv2 PVHv1 was deprecated and PVHv1 has been > > removed from the hypervisor in recent versions, that's why the Xen > > ports package is still stuck with Xen 4.7, because later versions > > removed PVHv1 support. With the addition of PVHv2 to FreeBSD the port > > can be updated to newer Xen versions and we can move forward. > > > > There will be issues however, as newer versions of Xen won't have > > support for PVHv1. My plan is the following in order to try to make > > this less painful for users: > > > > - Wait until FreeBSD 12 is released, which will contain PVHv1 and > > PVHv2 support. > > - Once FreeBSD 12 has been released, update the Xen port to the > > latest version. > > What about creating new port as xen410 (for version 4.10) or repocopy of the > old one to xen47 to allow coexist of two different versions in the ports > tree and allow user to choose the right one for their OS version? I wondered about that, I will try to do it, but I have to admit my time is quite limited and I'm not sure I will be able to keep both up to data. Thanks, Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On Thu, 19 Jul 2018, Roger Pau Monné wrote: > Hello, > > Today I've merged PVHv2 support into FreeBSD, allowing FreeBSD to be > used as a PVHv2 DomU and Dom0. While it's not a huge set of changes, > I would *really* appreciate if people could test the code starting > from r336474 (or any later changeset). Hi Roger, I am ready to test it on my (small) setup, should I keep existing 4.7 running as is and just upgrade the Dom0? Or should I upgrade Xen too? Marcin smime.p7s Description: S/MIME Cryptographic Signature
Re: HEADS UP: merged PVHv2 support and future plans
Roger Pau Monné wrote on 2018/07/19 11:04: Hello, Today I've merged PVHv2 support into FreeBSD, allowing FreeBSD to be used as a PVHv2 DomU and Dom0. While it's not a huge set of changes, I would *really* appreciate if people could test the code starting from r336474 (or any later changeset). I expect there's going to be some confusion with PVHv1 vs PVHv2, so I will try to clarify this now. PVHv1 was introduced ~4 years ago, and at the time it seemed like a good way to move forward, allowing Xen to rely more on hardware virtualization. Later on, we sadly discovered that PVHv1 was still too similar to classic PV, and didn't allow Xen to make use of all the possible hardware virtualization extensions, so PVHv2 was introduced ~2 years ago as a replacement for PVHv1. PVHv2 ABI however is not compatible with PVHv1, which means that different entry points and interfaces must be used to interact with the hypervisor. After introducing PVHv2 PVHv1 was deprecated and PVHv1 has been removed from the hypervisor in recent versions, that's why the Xen ports package is still stuck with Xen 4.7, because later versions removed PVHv1 support. With the addition of PVHv2 to FreeBSD the port can be updated to newer Xen versions and we can move forward. There will be issues however, as newer versions of Xen won't have support for PVHv1. My plan is the following in order to try to make this less painful for users: - Wait until FreeBSD 12 is released, which will contain PVHv1 and PVHv2 support. - Once FreeBSD 12 has been released, update the Xen port to the latest version. What about creating new port as xen410 (for version 4.10) or repocopy of the old one to xen47 to allow coexist of two different versions in the ports tree and allow user to choose the right one for their OS version? - Send an email to freebsd-xen notifying users of the bump in the package. - Users running FreeBSD < 12 must stick with the old Xen 4.7 package. - Users running FreeBSD >= 12 must use the new Xen package. [...] Miroslav Lachman ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
Thank you. Roger Pau Monné schrieb am Do., 19. Juli 2018, 11:50: > On Thu, Jul 19, 2018 at 11:42:24AM +0200, Gerd Hafenbrack wrote: > > Hello, > > > > Roger Pau Monné schrieb am Do., 19. Juli 2018, > 11:05: > > > > > ... Later on, we sadly discovered > > > that PVHv1 was still too similar to classic PV, and didn't allow Xen > > > to make use of all the possible hardware virtualization extensions, so > > > PVHv2 was introduced ~2 years ago ... > > > > > > > Is there a page comparing hardware virtualization extensions between > PVHv1 > > and PVHv2? > > Hm, not really. PVHv2 has some devices (either emulated by software or > hardware) that are not present on PVHv1: LAPIC, IO-APIC. The way to > setup and inject interrupts on PVHv2 is also the same as on native > (IO-APIC pin, MSI or MSI-X). > > Also PVHv2 allows to change paging modes (or completely disable > paging), which PVHv1 didn't allow at all. > > To sum up, PVHv2 is very, very similar to HVM but it doesn't require a > QEMU device model in order to run. > > > What about AMD support? We are getting EPIC machines. > > Yes, PVHv2 supports AMD, although I have to admit I haven't done > extensive testing on AMD due to my lack of AMD hardware. I expect > that's going to change soon. > > I've tested PVHv2 FreeBSD DomUs on AMD hardware, but I haven't tested > FreeBSD PVHv2 Dom0 on AMD yet. > > Roger. > ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
On Thu, Jul 19, 2018 at 11:42:24AM +0200, Gerd Hafenbrack wrote: > Hello, > > Roger Pau Monné schrieb am Do., 19. Juli 2018, 11:05: > > > ... Later on, we sadly discovered > > that PVHv1 was still too similar to classic PV, and didn't allow Xen > > to make use of all the possible hardware virtualization extensions, so > > PVHv2 was introduced ~2 years ago ... > > > > Is there a page comparing hardware virtualization extensions between PVHv1 > and PVHv2? Hm, not really. PVHv2 has some devices (either emulated by software or hardware) that are not present on PVHv1: LAPIC, IO-APIC. The way to setup and inject interrupts on PVHv2 is also the same as on native (IO-APIC pin, MSI or MSI-X). Also PVHv2 allows to change paging modes (or completely disable paging), which PVHv1 didn't allow at all. To sum up, PVHv2 is very, very similar to HVM but it doesn't require a QEMU device model in order to run. > What about AMD support? We are getting EPIC machines. Yes, PVHv2 supports AMD, although I have to admit I haven't done extensive testing on AMD due to my lack of AMD hardware. I expect that's going to change soon. I've tested PVHv2 FreeBSD DomUs on AMD hardware, but I haven't tested FreeBSD PVHv2 Dom0 on AMD yet. Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: HEADS UP: merged PVHv2 support and future plans
Hello, Roger Pau Monné schrieb am Do., 19. Juli 2018, 11:05: > ... Later on, we sadly discovered > that PVHv1 was still too similar to classic PV, and didn't allow Xen > to make use of all the possible hardware virtualization extensions, so > PVHv2 was introduced ~2 years ago ... > Is there a page comparing hardware virtualization extensions between PVHv1 and PVHv2? What about AMD support? We are getting EPIC machines. Gerd > ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
HEADS UP: merged PVHv2 support and future plans
Hello, Today I've merged PVHv2 support into FreeBSD, allowing FreeBSD to be used as a PVHv2 DomU and Dom0. While it's not a huge set of changes, I would *really* appreciate if people could test the code starting from r336474 (or any later changeset). I expect there's going to be some confusion with PVHv1 vs PVHv2, so I will try to clarify this now. PVHv1 was introduced ~4 years ago, and at the time it seemed like a good way to move forward, allowing Xen to rely more on hardware virtualization. Later on, we sadly discovered that PVHv1 was still too similar to classic PV, and didn't allow Xen to make use of all the possible hardware virtualization extensions, so PVHv2 was introduced ~2 years ago as a replacement for PVHv1. PVHv2 ABI however is not compatible with PVHv1, which means that different entry points and interfaces must be used to interact with the hypervisor. After introducing PVHv2 PVHv1 was deprecated and PVHv1 has been removed from the hypervisor in recent versions, that's why the Xen ports package is still stuck with Xen 4.7, because later versions removed PVHv1 support. With the addition of PVHv2 to FreeBSD the port can be updated to newer Xen versions and we can move forward. There will be issues however, as newer versions of Xen won't have support for PVHv1. My plan is the following in order to try to make this less painful for users: - Wait until FreeBSD 12 is released, which will contain PVHv1 and PVHv2 support. - Once FreeBSD 12 has been released, update the Xen port to the latest version. - Send an email to freebsd-xen notifying users of the bump in the package. - Users running FreeBSD < 12 must stick with the old Xen 4.7 package. - Users running FreeBSD >= 12 must use the new Xen package. I know this is not ideal, because users wanting to update Xen will also need to update to FreeBSD 12, but I cannot think of any better way to deal with this. Lastly, the Xen CI test system (osstest) has recently gained support for testing FreeBSD, which should make catching FreeBSD/Xen related issues easier. This is an example of an initial FreeBSD test flight: https://lists.xenproject.org/archives/html/xen-devel/2018-07/msg01680.html Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
