Re: available hypervisors in FreeBSD

[Miroslav Lachman]

Peter Ross wrote on 12/20/2015 09:15:

Hi all,

I read through an older threat I kept in my archive. It started like this:

On Wed, 1 Apr 2015, Udo Rader wrote:


As far as my homework digging revealed, FreeBSD supports four
hypervisors:

* bhyve
* KVM
* QEMU
* VirtualBox


.. and later Xen was mentioned.

I ask myself which of the solutions are most mature at the moment and
immediately usable in production.

Reason is a potential company move from VMware ESXi/Centos(6/7) with
some critical Windows 2008 and 2012 IIS/.NET applications) involved.

While most of open source may go into FreeBSD jails, we have a few
CentOS6/7 boxes with proprietary software we have to keep, as well as
the Windows VMs to maintain (there is a long term effort to move them to
Open Source too but the final migration of all may be years away).

We may phase out ESXi gradually, or just keep it, depending on the
performance and maturity of FreeBSD based solutions.

I have experience with Linux on VirtualBox and it worked well if the
load was not high but the performance wasn't too good when under stress
(but it never crashed, I might add).

Which of the solutions are worth testing? Do you have recommendations?

I am thinking of server software and "containerisation" only, so USB
passthrough or PCI etc. is not really important.

Stability, performance and resource utilisation (e.g. possible
over-allocation of RAM) are matter most.


VirtualBox is the most usable and you can use it in headless mode. If 
you are really not satified with VirtualBox, you can try Xen. The other 
options is not mature enough to run highly loaded Windows in production.

(it is just my opinion and somebody else can see it otherwise)

Miroslav Lachman

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Adam Vande More]

On Sun, Dec 20, 2015 at 2:15 AM, Peter Ross 
wrote:

> Hi all,
>
> I read through an older threat I kept in my archive. It started like this:
>
> On Wed, 1 Apr 2015, Udo Rader wrote:
>
> As far as my homework digging revealed, FreeBSD supports four hypervisors:
>>
>> * bhyve
>> * KVM
>> * QEMU
>> * VirtualBox
>>
>
> .. and later Xen was mentioned.
>
> I ask myself which of the solutions are most mature at the moment and
> immediately usable in production.
>
> Reason is a potential company move from VMware ESXi/Centos(6/7) with some
> critical Windows 2008 and 2012 IIS/.NET applications) involved.
>
> While most of open source may go into FreeBSD jails, we have a few
> CentOS6/7 boxes with proprietary software we have to keep, as well as the
> Windows VMs to maintain (there is a long term effort to move them to Open
> Source too but the final migration of all may be years away).
>
> We may phase out ESXi gradually, or just keep it, depending on the
> performance and maturity of FreeBSD based solutions.
>
> I have experience with Linux on VirtualBox and it worked well if the load
> was not high but the performance wasn't too good when under stress (but it
> never crashed, I might add).
>
> Which of the solutions are worth testing? Do you have recommendations?
>
> I am thinking of server software and "containerisation" only, so USB
> passthrough or PCI etc. is not really important.
>
> Stability, performance and resource utilisation (e.g. possible
> over-allocation of RAM) are matter most.
>

VBox is fine, it works well and really has all the features of vitalization
of the big 3 except for clustering and a few side things.

I've been using bhyve and I like it.  I have no stability issues on dozens
of guests some with a lot of IO net and disk.

I had hoped VPS[1] would make it in, but that seems to have stalled.



[1] http://www.7he.at/freebsd/vps/



-- 
Adam
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Sergey Manucharian]

Excerpts from Miroslav Lachman's message from Sun 20-Dec-15 09:57:
> Peter Ross wrote on 12/20/2015 09:15:
> >> As far as my homework digging revealed, FreeBSD supports four
> >> hypervisors:
> >>
> >> * bhyve
> >> * KVM
> >> * QEMU
> >> * VirtualBox
> >
> > .. and later Xen was mentioned.
> > 
> > Which of the solutions are worth testing? Do you have recommendations?
> >
> > I am thinking of server software and "containerisation" only, so USB
> > passthrough or PCI etc. is not really important.
> > 
> > Stability, performance and resource utilisation (e.g. possible
> > over-allocation of RAM) are matter most.
> 
> VirtualBox is the most usable and you can use it in headless mode. If 
> you are really not satified with VirtualBox, you can try Xen. 

I agree that VirtualBox is really stable, and I'm using it in production
environments for many years. However, there are a couple of possible
drawbacks: It does not support VRDP (remote console) and USB2/3 on FreeBSD.

Tha latter is probably not really important (although I needed it too).
The lack of remote console is bad for troubleshooting and/or remote
(re)installation.

Currently I have one bhyve Windows Server 2012 machine, which works
fine, although it's not really loaded at the moment.

Sergey

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Adam Vande More]

On Sun, Dec 20, 2015 at 9:25 AM, Sergey Manucharian  wrote:

> I agree that VirtualBox is really stable, and I'm using it in production
> environments for many years. However, there are a couple of possible
> drawbacks: It does not support VRDP (remote console) and USB2/3 on FreeBSD.
>
> Tha latter is probably not really important (although I needed it too).
> The lack of remote console is bad for troubleshooting and/or remote
> (re)installation.
>

Remote console is available via VNC, not RDP.


-- 
Adam
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Udo Rader]

On 12/20/2015 09:15 AM, Peter Ross wrote:
> Hi all,
> 
> I read through an older threat I kept in my archive. It started like this:
> 
> On Wed, 1 Apr 2015, Udo Rader wrote:
> 
>> As far as my homework digging revealed, FreeBSD supports four
>> hypervisors:
>>
>> * bhyve
>> * KVM
>> * QEMU
>> * VirtualBox
> 
> .. and later Xen was mentioned.
> 
> I ask myself which of the solutions are most mature at the moment and
> immediately usable in production.
> 
> Reason is a potential company move from VMware ESXi/Centos(6/7) with
> some critical Windows 2008 and 2012 IIS/.NET applications) involved.
> 
> While most of open source may go into FreeBSD jails, we have a few
> CentOS6/7 boxes with proprietary software we have to keep, as well as
> the Windows VMs to maintain (there is a long term effort to move them to
> Open Source too but the final migration of all may be years away).
> 
> We may phase out ESXi gradually, or just keep it, depending on the
> performance and maturity of FreeBSD based solutions.
> 
> I have experience with Linux on VirtualBox and it worked well if the
> load was not high but the performance wasn't too good when under stress
> (but it never crashed, I might add).
> 
> Which of the solutions are worth testing? Do you have recommendations?
> 
> I am thinking of server software and "containerisation" only, so USB
> passthrough or PCI etc. is not really important.
> 
> Stability, performance and resource utilisation (e.g. possible
> over-allocation of RAM) are matter most.

two thoughts:

first, PCI passthru is a nice thing if you want to directly address
NICs, which again is a nice feature for virtualized servers relying in
almost native network throughput.

and second, but you are probably aware of that already, IIRC Xen dom0
support is quite new & lacks some features
(http://wiki.xenproject.org/wiki/FreeBSD_Dom0)
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Sergey Manucharian]

Excerpts from Adam Vande More's message from Sun 20-Dec-15 09:36:
> On Sun, Dec 20, 2015 at 9:25 AM, Sergey Manucharian  wrote:
> 
> > I agree that VirtualBox is really stable, and I'm using it in production
> > environments for many years. However, there are a couple of possible
> > drawbacks: It does not support VRDP (remote console) and USB2/3 on FreeBSD.
> >
> > Tha latter is probably not really important (although I needed it too).
> > The lack of remote console is bad for troubleshooting and/or remote
> > (re)installation.
> >
> 
> Remote console is available via VNC, not RDP.

It is VNC, and I use it Linux hosts, it's rather confusing since the option
is "--vrde on|off". But isn't it a part of the extension pack, which is
not available for FreeBSD?

https://www.virtualbox.org/manual/ch07.html

S.

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Adam Vande More]

On Sun, Dec 20, 2015 at 10:14 AM, Sergey Manucharian  wrote:

> > Remote console is available via VNC, not RDP.
>
> It is VNC, and I use it Linux hosts, it's rather confusing since the option
> is "--vrde on|off".


See
https://lists.freebsd.org/pipermail/freebsd-emulation/2013-January/010354.html

You can also set options like VNCAddress4 for listening address.


> But isn't it a part of the extension pack, which is
> not available for FreeBSD?
>
> https://www.virtualbox.org/manual/ch07.html
>

The explanation lies within that page.  VRDP is only in extension pack,
VRDE is available to all.  So someone with enough gumption could write a
VRDE RDP support.

-- 
Adam
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: available hypervisors in FreeBSD

[Udo Rader]

On 04/01/2015 12:41 PM, Roger Pau Monné wrote:
 El 01/04/15 a les 12.30, Udo Rader ha escrit:
 As far as my homework digging revealed, FreeBSD supports four hypervisors:

 * bhyve
 * KVM
 * QEMU
 * VirtualBox
 
 Make that 5:
  * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0
 
 Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits
 under the hypervisor category, it's an emulator instead, so the list
 should probably be 3 (Bhyve, VirtualBox and Xen).

thanks for pointing Xen out. I was indeed not aware of Xen running on
FreeBSD, an intriguing (and well known) alternative.

The wiki says, that migrate/save/restore are missing from the *BSD port.
Is that still valid?
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Roger Pau Monné]

El 01/04/15 a les 12.30, Udo Rader ha escrit:
 Hi all,
 
 first please excuse if this may be a FAQ, but even though I am a long
 time linux admin (~1996), I am quite new to the *BSD world and I am
 trying to evaluate if FreeBSD fits our virtualization needs.
 
 So, for my many questions:
 
 As far as my homework digging revealed, FreeBSD supports four hypervisors:
 
 * bhyve
 * KVM
 * QEMU
 * VirtualBox

Make that 5:
 * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0

Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits
under the hypervisor category, it's an emulator instead, so the list
should probably be 3 (Bhyve, VirtualBox and Xen).

Roger.

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Roger Pau Monné]

El 01/04/15 a les 14.59, Udo Rader ha escrit:
 On 04/01/2015 12:41 PM, Roger Pau Monné wrote:
 El 01/04/15 a les 12.30, Udo Rader ha escrit:
 As far as my homework digging revealed, FreeBSD supports four hypervisors:

 * bhyve
 * KVM
 * QEMU
 * VirtualBox

 Make that 5:
  * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0

 Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits
 under the hypervisor category, it's an emulator instead, so the list
 should probably be 3 (Bhyve, VirtualBox and Xen).
 
 thanks for pointing Xen out. I was indeed not aware of Xen running on
 FreeBSD, an intriguing (and well known) alternative.
 
 The wiki says, that migrate/save/restore are missing from the *BSD port.
 Is that still valid?

Yes, I'm currently finishing the patches for Xen. This is not missing
from FreeBSD, but from Xen itself when running Dom0 in PVH mode.

Roger.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Roger Pau Monné]

Hello,

El 01/04/15 a les 16.27, Gerd Hafenbrack ha escrit:
 On 2015-04-01 16:19, Roger Pau Monné wrote:
 El 01/04/15 a les 14.59, Udo Rader ha escrit:
 ... thanks for pointing Xen out. I was indeed not aware of Xen
 running on
 FreeBSD, an intriguing (and well known) alternative.

 The wiki says, that migrate/save/restore are missing from the *BSD port.
 Is that still valid?

 Yes, I'm currently finishing the patches for Xen. This is not missing
 from FreeBSD, but from Xen itself when running Dom0 in PVH mode.
 
 The documentation for FreeBSD as Dom0 seems outdated anyway to me.

The document was last modified on the 15th of March 2015. I know things
move fast in the IT industry, but I wouldn't call that outdated.

Roger.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Gerd Hafenbrack]

On 2015-04-01 16:46, Roger Pau Monné wrote:

Hello,

El 01/04/15 a les 16.27, Gerd Hafenbrack ha escrit:

... The documentation for FreeBSD as Dom0 seems outdated anyway to me.


The document was last modified on the 15th of March 2015. I know things
move fast in the IT industry, but I wouldn't call that outdated. ...


An example:
https://wiki.freebsd.org/FreeBSD/Xen
FreeBSD/Xen (last edited 2014-08-29 04:27:05 by CherryMathew)

http://wiki.xenproject.org/wiki/FreeBSD_Dom0
This page was last modified on 14 March 2015, at 02:08.
I think this is the page you are referring to.

Please excuse my words. Sorry, they weren't intended as an offense.

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Paul Vixie]

Udo Rader wrote:
 ...

 I understand, that bhyve is native to BSD and will probably be the most
 effective. But given its relatively 'young age', is it production ready
 for (non nested) x86/amd64 linux guests?

there's no libvirt for bhyve yet, which turns some people off. (not me,
i don't use libvirt in any case.)

there's significant clock drift, even with
kern.timecounter.hardware=TSC-low in the guests:

 ...
 Jan 26 05:38:08 guests ntpd[619]: time reset +0.223304 s
 Jan 26 06:06:22 guests ntpd[619]: time reset +0.196973 s
 Jan 26 06:34:24 guests ntpd[619]: time reset +0.200070 s
 Jan 26 07:08:28 guests ntpd[619]: time reset +0.210997 s
 Jan 26 07:36:09 guests ntpd[619]: time reset +0.205481 s
 Jan 26 08:10:04 guests ntpd[619]: time reset +0.205461 s
 Jan 26 08:39:43 guests ntpd[619]: time reset +0.175491 s
 Jan 26 09:10:29 guests ntpd[619]: time reset +0.189261 s
 Jan 26 09:44:03 guests ntpd[619]: time reset +0.164616 s
 Jan 26 10:20:25 guests ntpd[619]: time reset +0.176280 s
 Jan 26 10:56:18 guests ntpd[619]: time reset +0.161555 s
 Jan 26 11:39:53 guests ntpd[619]: time reset +0.166066 s
 Jan 26 12:31:11 guests ntpd[619]: time reset +0.142994 s
 ...

(that's much worse with the default kern.timecounter.hardware value, but
still rather absurd.)

i use bhyve in production and seems altogether ready.


-- 
Paul Vixie
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Gustau Pérez]

Hi all,

I found the source of the problem. Once upon a time I compiled my
 kernel with OFED support (WITH_OFED in /etc/src.conf). That installed
 $INCLUDE/rdma/rdma_cma.h, which at the time of the installation (haven't
 checked now) were missing the rdma_addrinfo structs.

Moving the $INCLUDE/rdma dir somewhere allowed the build to finish.

   Sorry I did not finish my explanation. Having the rdma_cma.h in the
default include dir fooled the tools/qemu-xen-dir/configure script, and
thus compiling as if the target was a linux box. Moving the
$INCLUDE/rdma dir, cleaning, configuring and building again did the job.

   Gustau
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: available hypervisors in FreeBSD

[Craig Rodrigues]

On Wed, Apr 1, 2015 at 9:41 AM, Paul Vixie p...@redbarn.org wrote:




 there's no libvirt for bhyve yet, which turns some people off.


Wrong.

See:
https://libvirt.org/drvbhyve.html
http://www.slideshare.net/CraigRodrigues1/libvirt-bhyve

libvirt/bhyve is definitely not as polished as libvirt/KVM.
It definitely needs more work, but at least some people
have put in the work to add libvirt/bhyve support at all.

--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org