Re: VMWare/Virtualbox virtio network drivers?
On Wed, Sep 07, 2011 at 01:24:37PM +, Julian Elischer wrote: > The BHyve guys are implementing virtio drivers and have looked@both > these drivers and others. > I don't know what their plans are.. > > you should send to virtualizat...@freebsd.org (cc'd and redirected) to > get up to date info. Hi, Virtio drivers are coming. See: http://lists.freebsd.org/pipermail/svn-src-projects/2011-September/004361.html -- Craig Rodrigues rodr...@crodrigues.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
BHyVe web page
Hi, I have created a web page for the BSD Hypervisor (BHyVe) project: http://wiki.freebsd.org/BHyVe This page contains pointers to how to get the BHyVe code, build it, etc. It will be updated with more documentation over time. Please send followup questions to freebsd-virtualization@FreeBSD.org . -- Craig Rodrigues rodr...@crodrigues.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
VirtualBox problem booting FreeBSD-9.0-RELEASE-amd64-bootonly.iso
Hi, I am trying to run FreeBSD-9.0-RELEASE-amd64-bootonly.iso inside VirtualBox and am having some problems. The error messages I am getting are similar to the ones mentioned here: http://lists.freebsd.org/pipermail/freebsd-stable/2011-December/065079.html My host environment === Operating System: Windows 7, home edition, 64-bit VirtualBox for Windows, version 4.1.12 r77245 CPU Information I downloaded the CoreInfo.exe utility which is part of Windows Sysinternals utilities ( http://technet.microsoft.com/en-us/sysinternals/bb545027 ) According to this utility, my CPU is: Intel(R) Core(TM)2 Duo CPU T5450 @ 1.66GHz Intel64 Family 6 Model 15 Stepping 13, GenuineIntel HTT * Hyperthreading enabled HYPERVISOR - Hypervisor is present VMX - Supports Intel hardware-assisted virtualization SVM - Supports AMD hardware-assisted virtualization EM64T * Supports 64-bit mode So my CPU support 64-bit mode, but does not support hardware-assisted virtualization. Error message = If I try to mount and then boot it under VirtualBox, I get this error: CPU doesn't support long mode Consoles: internal video/keyboard BIOS drive C: is disk0 BIOS 639kB/523264kB available memory FreeBSD/x86 bootstrap loader, Revision 1.1 (r...@farrell.cse.buffalo.edu, Tue Jan 3 06:51:49 UTC 2012) Can't work out which disk we are booting from. Guess BIOS device 0x not found by probes, defaulting to disk0: FATAL: int13_harddisk: function 42. Can't use 64bits lba Does anyone know how to fix this problem in VirtualBox? QEMU for Windows Just to try something else, I downloaded QEMU 0.9.0 for Windows from: http://www.h7.dion.ne.jp/~qemu-win/ I extracted the zip file into a directory: C:\local\qemu-0.9.0-windows I then ran: cd c:\local\qemu-0.9.0-windows qemu-system-x86_64.exe -L . -m 128 -cdrom c:\temp\FreeBSD-9.0-RELEASE-amd64-bootonly.iso The ISO booted up just fine. From the dmesg output, this was the emulated CPU: CPU: QEMU Virtual CPU version 0.9.0 (1666.98-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x623 Family = 6 Model = 2 Stepping = 3 Features=0x78bfbfd Features2=0x1 AMD Features=0x20100800 Does anyone know how I can get VirtualBox to behave the same way as QEMU for FreeBSD-9.0-RELEASE-amd64-bootonly.iso? Thanks. -- Craig Rodrigues rodr...@crodrigues.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: VirtualBox problem booting FreeBSD-9.0-RELEASE-amd64-bootonly.iso
On Mon, Apr 16, 2012 at 12:08 AM, Craig Rodrigues wrote: > My host environment > === > > Operating System: Windows 7, home edition, 64-bit > VirtualBox for Windows, version 4.1.12 r77245 > > CPU Information > > > I downloaded the CoreInfo.exe utility which is part of Windows > Sysinternals utilities > ( http://technet.microsoft.com/en-us/sysinternals/bb545027 ) > > According to this utility, my CPU is: > > Intel(R) Core(TM)2 Duo CPU T5450 @ 1.66GHz > Intel64 Family 6 Model 15 Stepping 13, GenuineIntel > HTT * Hyperthreading enabled > HYPERVISOR - Hypervisor is present > VMX - Supports Intel hardware-assisted virtualization > SVM - Supports AMD hardware-assisted virtualization > EM64T * Supports 64-bit mode > > > So my CPU support 64-bit mode, but does not support hardware-assisted > virtualization. > > > > Error message > = > If I try to mount and then boot it under VirtualBox, I get this error: > > CPU doesn't support long mode > > Consoles: internal video/keyboard > BIOS drive C: is disk0 > BIOS 639kB/523264kB available memory > > FreeBSD/x86 bootstrap loader, Revision 1.1 > (r...@farrell.cse.buffalo.edu, Tue Jan 3 06:51:49 UTC 2012) > Can't work out which disk we are booting from. > Guess BIOS device 0x not found by probes, defaulting to disk0: > FATAL: int13_harddisk: function 42. Can't use 64bits lba > > > > Does anyone know how to fix this problem in VirtualBox? Hi, After exchanging a few e-mails with Bernard, I think I understand the problem. According to the VirtualBox manual: http://www.virtualbox.org/manual/ch03.html#intro-64bitguests The CPU needs to support hardware-assisted virtualization in order for VirtualBox to support 64-bit guests. Without the CPU support, VirtualBox cannot do it. I double checked this with CoreInfo.exe from Windows Sysinternals utilities, and also from the Intel data sheet for my CPU: http://ark.intel.com/products/30787 (VT-x option not supported) I will use QEMU under Windows until I can get a better machine. :) -- Craig Rodrigues rodr...@crodrigues.org -- Craig Rodrigues rodr...@crodrigues.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: VirtualBox problem booting FreeBSD-9.0-RELEASE-amd64-bootonly.iso
On Mon, Apr 16, 2012 at 9:29 PM, Adam Vande More wrote: > On Mon, Apr 16, 2012 at 10:30 PM, Craig Rodrigues > wrote: >> >> After exchanging a few e-mails with Bernard, I think I understand the >> problem. >> According to the VirtualBox manual: >> >> http://www.virtualbox.org/manual/ch03.html#intro-64bitguests >> >> The CPU needs to support hardware-assisted virtualization in order for >> VirtualBox to support 64-bit guests. >> Without the CPU support, VirtualBox cannot do it. I double checked >> this with CoreInfo.exe from Windows Sysinternals utilities, >> and also from the Intel data sheet for my CPU: >> >> http://ark.intel.com/products/30787 >> (VT-x option not supported) >> >> >> I will use QEMU under Windows until I can get a better machine. :) > > > I believe a Windows 7 product key will activate either 32 or 64 install, so > unless you have a specific reason to install 64 bit then you are likely > better using 32 anyway. 32 bit OS's and apps use less memory than their > corresponding 64 bit counterparts. > > > > -- > Adam Vande More Hi, Yes, you are right. Windows 7 comes on 2-DVD's, one for 32-bit and one for 64-bit. The same product key can be used for either version. I had installed 64-bit Windows with the intent of testing out 64-bit guest OS's under FreeBSD, and was unaware of the requirements for hardware assisted virtualization. :) I am interested in developing and testing 32-bit and 64-bit Windows binaries anyways, so I will keep my install for now. :) Thanks. -- Craig Rodrigues rodr...@crodrigues.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Hyper-V Integration Components Patch for FreeBSD 8.2, 8.3, 9.0 and 9.1-BETA1
Hi, Thanks for writing this up. I have created a page on the FreeBSD wiki with info about Microsoft Hyper-V, and I placed a link to your blog entry there: http://wiki.freebsd.org/HyperV -- Craig Rodrigues rodr...@crodrigues.org On Mon, Aug 13, 2012 at 3:36 PM, Chris Knight wrote: > Hello, > > I've created some patchsets based on the beta release of the Hyper-V > integration components for FreeBSD. > > The patchsets are for 8.2, 8.3, 9.0 and 9.1-BETA1 and can be found here: > > http://blog.chrisara.com.au/2012/08/hyper-v-integration-components-for_13.html > > Although the Hyper-V kernel modules compile, they'll cause a kernel > panic if loaded, but I got them built cleanly to allow for easy kernel > swapping using nextboot. > Using GEOM labels makes it easy to swap between a Hyper-V enabled > kernel and a non-Hyper-V enabled kernel. > > It's also worth noting that the Hyper-V network driver is flaky - UDP > works fairly well, but TCP is very flaky. Haven't yet got to the root > cause of this. > > The storage performance increase is very nice, as is the heartbeat and > shutdown capabilities. I've yet to check if KVP functionality is > included. > > -- > Regards, > Chris Knight > ___ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscr...@freebsd.org" > -- Craig Rodrigues rodr...@crodrigues.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
AMD and BHyve?
Hi, I recently updated one of my home machines to FreeBSD-10/amd64. Here is what I have: FreeBSD 10.0-CURRENT #0 r249181: Sat Apr 6 03:07:32 UTC 2013 rodr...@dibbler.crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 WARNING: WITNESS option enabled, expect reduced performance. CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ (2511.48-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x60fb1 Family = 0xf Model = 0x6b Stepping = 1 Features=0x178bfbff Features2=0x2001 AMD Features=0xea500800 AMD Features2=0x11f Will this CPU work with BHyve when the AMD support is fully committed? Right now when I try to load vmm.ko I get: kldload vmm.ko amd_iommu_init: not implemented amdv_init: not implemented amdv_cleanup: not implemented module_register_init: MOD_LOAD (vmm, 0x81c893b0, 0) error 6 Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FW: AMD and BHyve?
On Sat, Apr 13, 2013 at 2:43 PM, Anish wrote: > Hi Craig, > > >AMD Features=0xea500800 > AMD Features2=0x11f > > Yes, the AMD processor you have support AMD-V or SVM and I got BHyVe > working on AMD processors with SVM Nested Page Table/RVI support, > equivalent of Intel Extended Page Table. OK, cool. > > > Neel and Peter have created projects/bhyve_svm for submitting AMD patches > for BHyVe. It doesn't have complete patches yet, only vmcb related files. I looked at http://svnweb.freebsd.org/base/projects/bhyve_svm/sys/amd64/vmm/amd/amdv.c but it looks like the code will still give me an error if I try to kldload vmm.ko on my box. Do you have a patch? Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [Xen-devel] FreeBSD PVHVM call for testing
On Fri, May 24, 2013 at 7:14 AM, Konrad Rzeszutek Wilk < konrad.w...@oracle.com> wrote: > On Thu, May 23, 2013 at 07:41:50PM +0200, Roger Pau Monné wrote: > > Hello, > > > > I've pushed a new branch, pvhvm_v10 that contains a PV IPI > > implementation for both amd64 and i386. I've also updated the wiki to > > point to the pvhvm_v10 branch: > > I feel a bit stupid to ask this, but how I install 'gmake'? Doing 'pkg_add > -r gmake' > tells me there is no package (perhaps I am using a too modern version of > FreeBSD > (FreeBSD-10.0-CURRENT-amd64-20130512-r250582-release.iso)? > > The Wiki mentions how to install git but that fails b/c it can't find > gmake. > For 10.0-CURRENT, not all the packages are available yet from the main FreeBSD.org ftp site. I am going through a similar setup issue with someone who has signed up for Google Summer of Code, and I need him to use close to the latest 10.0-CURRENT and have a usable system. I wrote this blog post for the student: http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/ You may wish to follow those steps to configure your system to download packages from one of the 3rd party pkg mirrors, and then use the "pkg" command to install gmake and whatever other packages you need. Outback Dingo has mentioned using "pkg" command for installing packages in 10-CURRENT, and that is what I am instructing my GSoC to do. Good luck. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On Thu, May 23, 2013 at 6:30 AM, Roger Pau Monné wrote: > On 23/05/13 15:20, Jeroen van der Ham wrote: > > > > On 13 May 2013, at 20:32, Roger Pau Monné wrote: > >> Also, I've created a wiki page that explains how to set up a FreeBSD > >> PVHVM for testing: > >> > >> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM > > > > > > You mention on that page that it is easier to install on 10.0-CURRENT > snapshots. > > What are the issues with installing this on 9.1? Is it possible? > > I don't think it is recommended to use a HEAD (10) kernel with a 9.1 > userland. You can always install a 9.1 and then do a full update with > the source on my repository. > Actually in FreeBSD, it is possible to run an older userland on a newer kernel, and a lot of effort is spent in preserving this type of backwards compatibility. So a 9.1 userland with a 10 kernel will work. However, running a newer userland on an older kernel is not guaranteed to work. So running a 10 userland with a 9.1 kernel will most likely not work. However, since you guys are doing very cutting edge stuff with 10-CURRENT, it is better that you do not waste time with 9.1. I recommend you start with a 10.0 CURRENT snapshot ISO and go from there. I am going through a similar setup exercise with a Google Summer of Code student where he needs to have a latest CURRENT system running in a VM. I wrote this blog post: http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/ for the steps how to do it. You can follow those steps to get bootstrapped with a working environment if it helps you out. Good luck. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On Tue, May 28, 2013 at 8:21 AM, Outback Dingo wrote: > > > > On Fri, May 24, 2013 at 6:27 PM, Craig Rodrigues > wrote: > >> I wrote this blog post: >> >> >> http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/ >> >> for the steps how to do it. You can follow those steps to get >> bootstrapped >> with a working environment if it helps you out. >> >> Good luck. >> >> > Any chance this will be backported to 9.X or 9-STABLE at least ?? > > I wrote a blog post specifically for installing 10-CURRENT from a snapshot ISO and getting bootstrapped from there, to help the Google Summer of Code Student that I am mentoring. What specifically do you want to be backported to 9-STABLE? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On Tue, May 28, 2013 at 3:43 PM, Outback Dingo wrote: > > I wrote a blog post specifically for installing 10-CURRENT from a snapshot >> ISO and getting bootstrapped >> from there, to help the Google Summer of Code Student that I am mentoring. >> What specifically do you want to be backported to 9-STABLE? >> > > Yes Ive seen the post, however we have a hard requirement for 9-STABLE for > some development work we are > currently involved in. And I would prefer not to move our current > infrastructure for development to CURRENT. > A major portion of our development environment is based on XEN for testing > and development. > Therefor we realize PVHM is a work in progress, and we have built some > VMs from it, they run exceptionally > well, However our build environments refuse to build 9-STABLE on CURRENT > PVHVM, or 10-CURRENT. > Lastly Im sure theres a ton of users who would like to see this in 9.x or > 9-STABLE > > OK, I misunderstood you since you responded in the thread and quoted some of my post. My blog post has nothing relevant to Xen and PVHM, and is specific to setting up a Google Summer of Code student on a 10-CURRENT environment. The FreeBSD Xen and PVHM developers will need to provide the details about their plans to merge their changes to the 9-STABLE branch. that's something which I am not familiar with. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
VIMAGE crashes on 9.x with bluetooth devices
Hi,
I tried running PC-BSD 9.1 on an HP Elitebook 8460p laptop (
http://h10010.www1.hp.com/wwpc/pscmi...oks/8460p.html ).
See the screenshot I posted here:
http://forums.pcbsd.org/showthread.php?t=20001
In PC-BSD, the default kernel comes with VIMAGE enabled in the kernel
config.
The problem I encountered seems to be caused by problems
with the FreeBSD kernel with VIMAGE + bluetooth.
I've seen similar threads being posted here:
"VIMAGE crashes on 9.x with hotplug net80211 devices"
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-October/040784.html
"Panic in netgraph with VIMAGE"
http://lists.freebsd.org/pipermail/freebsd-virtualization/2009-May/000325.html
Doing some kernel debugging, I got this stack trace:
#0 doadump (textdump=) at pcpu.h:234
#1 0x80923e86 in kern_reboot (howto=260) at
/usr/home/rodrigc/freebsd/9/sys/kern/kern_shutdown.c:449
#2 0x80924387 in panic (fmt=0x1 ) at
/usr/home/rodrigc/freebsd/9/sys/kern/kern_shutdown.c:637
#3 0x80ce97f0 in trap_fatal (frame=0xc, eva=)
at /usr/home/rodrigc/freebsd/9/sys/amd64/amd64/trap.c:879
#4 0x80ce9b51 in trap_pfault (frame=0xff811e2b7700, usermode=0)
at /usr/home/rodrigc/freebsd/9/sys/amd64/amd64/trap.c:795
#5 0x80cea104 in trap (frame=0xff811e2b7700) at
/usr/home/rodrigc/freebsd/9/sys/amd64/amd64/trap.c:463
#6 0x80cd3433 in calltrap () at
/usr/home/rodrigc/freebsd/9/sys/amd64/amd64/exception.S:228
#7 0x81cc0bbd in ng_make_node_common (type=0x81cb6000,
nodepp=0xfe0124d9c310)
at
/usr/home/rodrigc/freebsd/9/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:669
#8 0x81cb320e in ubt_attach (dev=0xfe01041f1d00)
at
/usr/home/rodrigc/freebsd/9/sys/modules/netgraph/bluetooth/ubt/../../../../netgraph/bluetooth/drivers/ubt/ng_ubt.c:461
#9 0x80956c9c in device_attach (dev=0xfe01041f1d00) at
device_if.h:180
#10 0x8078a61e in usb_probe_and_attach (udev=0xfe012458,
iface_index=)
at /usr/home/rodrigc/freebsd/9/sys/dev/usb/usb_device.c:1183
#11 0x80792c4e in uhub_explore (udev=0xfe000900c000) at
/usr/home/rodrigc/freebsd/9/sys/dev/usb/usb_hub.c:272
#12 0x80792c74 in uhub_explore (udev=0xfe0007ee2000) at
/usr/home/rodrigc/freebsd/9/sys/dev/usb/usb_hub.c:286
#13 0x8077ce46 in usb_bus_explore (pm=) at
/usr/home/rodrigc/freebsd/9/sys/dev/usb/controller/usb_controller.c:359
#14 0x80796da3 in usb_process (arg=) at
/usr/home/rodrigc/freebsd/9/sys/dev/usb/usb_process.c:169
#15 0x808f1cef in fork_exit (callout=0x80796cd0
, arg=0xff8001c3cdb0, frame=0xff811e2b7b00)
at /usr/home/rodrigc/freebsd/9/sys/kern/kern_fork.c:988
#16 0x80cd395e in fork_trampoline () at
/usr/home/rodrigc/freebsd/9/sys/amd64/amd64/exception.S:602
#17 0x in ?? ()
and the error occurred on line 669 in ng_base.c:
#7 0x81cc0bbd in ng_make_node_common (type=0x81cb6000,
nodepp=0xfe0124d9c310)
at
/usr/home/rodrigc/freebsd/9/sys/modules/netgraph/netgraph/../../../netgraph/ng_base.c:669
669 node->nd_ID = V_nextID++; /* 137/sec for 1 year before wrap */
(kgdb) l
664
665 /* Get an ID and put us in the hash chain. */
666 IDHASH_WLOCK();
667 for (; { /* wrap protection, even if silly */
668 node_p node2 = NULL;
669 node->nd_ID = V_nextID++; /* 137/sec for 1 year before wrap */
670
671 /* Is there a problem with the new number? */
672 NG_IDHASH_FIND(node->nd_ID, node2); /* already taken? */
673 if ((node->nd_ID != 0) && (node2 == NULL)) {
It turns out that when VIMAGE is defined in the kernel, V_nextID is not an
int any more,
it is a macro. line 669 expands to something that looks like:
node->nd_ID = (*(__typeof(vnet_entry_nextID)*)
(__curthread())->td_vnet))->vnet_data_base) +
(uintptr_t)&vnet_entry_nextID))++;
On my system, td_vnet was not initialized, so that was causing the kernel
panic.
Is the attached patch OK to commit to -CURRENT, and then after some time
MFC to stable/9?
With this patch, my system can boot with VIMAGE enabled in the kernel and
it does not crash.
--
Craig
Index: bluetooth/drivers/ubt/ng_ubt.c
===
--- bluetooth/drivers/ubt/ng_ubt.c (revision 252218)
+++ bluetooth/drivers/ubt/ng_ubt.c (working copy)
@@ -109,7 +109,9 @@
#include
#include
#include
+#include
#include
+#include
#include "usbdevs.h"
#include
@@ -123,6 +125,7 @@
#include
#include
+#include
#include
#include
#include
@@ -453,13 +456,14 @@
sc->sc_dev = dev;
sc->sc_debug = NG_UBT_WARN_LEVEL;
-
+ CURVNET_SET(TD_TO_VNET(curthread));
/*
* Create Netgraph node
*/
if (ng_make_node_common(&typestruct, &sc->sc_node) != 0) {
UBT_ALERT(sc, "could not create Netgraph node\n");
+ CURVNET_RESTORE();
return (ENXIO);
}
@@ -467,10 +471,12 @@
if (ng_name_node(sc->sc_node, devi
VIMAGE + PF crash in mbuf destructor
Hi,
I used a kernel config with the following lines:
include GENERIC
options VIMAGE
and compiled a CURRENT kernel from svn://svn.freebsd.org/base/head@253346 .
I also have PF enabled on my system.
Once in a while I have been getting kernel panics like these:
(kgdb) #0 doadump (textdump=1) at pcpu.h:236
#1 0x808bc617 in kern_reboot (howto=260)
at /usr/home/rodrigc/freebsd/head/sys/kern/kern_shutdown.c:447
#2 0x808bcb25 in vpanic (fmt=,
ap=)
at /usr/home/rodrigc/freebsd/head/sys/kern/kern_shutdown.c:754
#3 0x808bcb73 in panic (fmt=)
at /usr/home/rodrigc/freebsd/head/sys/kern/kern_shutdown.c:683
#4 0x8033dff7 in db_panic (addr=,
have_addr=, count=,
modif=)
at /usr/home/rodrigc/freebsd/head/sys/ddb/db_command.c:482
#5 0x8033dbcd in db_command (cmd_table=)
at /usr/home/rodrigc/freebsd/head/sys/ddb/db_command.c:449
#6 0x8033d944 in db_command_loop ()
at /usr/home/rodrigc/freebsd/head/sys/ddb/db_command.c:502
#7 0x803402f0 in db_trap (type=, code=0)
at /usr/home/rodrigc/freebsd/head/sys/ddb/db_main.c:231
#8 0x808f3623 in kdb_trap (type=12, code=0, tf=)
at /usr/home/rodrigc/freebsd/head/sys/kern/subr_kdb.c:654
#9 0x80cda43a in trap_fatal (frame=0xff811dbab6b0,
eva=)
at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/trap.c:868
#10 0x80cda6f4 in trap_pfault (frame=0x0, usermode=0)
at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/trap.c:699
#11 0x80cd9ef0 in trap (frame=0xff811dbab6b0)
at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/trap.c:463
#12 0x80cc31a2 in calltrap ()
at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/exception.S:232
#13 0x8208f7b7 in pf_mtag_free (t=0xfe00a8797870)
at
/usr/home/rodrigc/freebsd/head/sys/modules/pf/../../netpfil/pf/pf.c:830
#14 0x808a51c9 in mb_dtor_mbuf (mem=0xfe000d0bc500, size=256,
arg=0x0) at /usr/home/rodrigc/freebsd/head/sys/kern/kern_mbuf.c:499
#15 0x80b55d4d in uma_zfree_arg (zone=0xfe000b4ab900,
item=0xfe000d0bc500, udata=0x0)
at /usr/home/rodrigc/freebsd/head/sys/vm/uma_core.c:2560
#16 0x8092d1f5 in m_freem (mb=) at uma.h:364
#17 0x8058ba72 in iwn_tx_done (sc=0xff8000974000,
desc=, ackfailcnt=16, status=131 '\203')
at /usr/home/rodrigc/freebsd/head/sys/dev/iwn/if_iwn.c:2817
#18 0x80583e60 in iwn_notif_intr (sc=0xff8000974000)
at /usr/home/rodrigc/freebsd/head/sys/dev/iwn/if_iwn.c:3015
#19 0x80583684 in iwn_intr (arg=0xff8000974000)
at /usr/home/rodrigc/freebsd/head/sys/dev/iwn/if_iwn.c:3306
#20 0x8088daf3 in intr_event_execute_handlers (
p=, ie=0xfe000b696600)
at /usr/home/rodrigc/freebsd/head/sys/kern/kern_intr.c:1263
#21 0x8088e4c6 in ithread_loop (arg=0xfe000b31b040)
at /usr/home/rodrigc/freebsd/head/sys/kern/kern_intr.c:1276
#22 0x8088b3f4 in fork_exit (
callout=0x8088e420 , arg=0xfe000b31b040,
frame=0xff811dbabac0)
at /usr/home/rodrigc/freebsd/head/sys/kern/kern_fork.c:991
#23 0x80cc36de in fork_trampoline ()
at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/exception.S:606
#24 0x in ?? ()
Current language: auto; currently minimal
(kgdb)
It turns out that in this file: src/sys/netpfil/pf/pf.c
826 static void
827 pf_mtag_free(struct m_tag *t)
828 {
829
830 uma_zfree(V_pf_mtag_z, t);
831 }
when line 830 is hit, it turns out that curthread->td_vnet is NULL.
Does anyone have an idea as to the best place
to put CURVNET_SET() to avoid this problem?
I am a little less famiiar with mbuf and pf.
Thanks.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: VIMAGE + PF crash in mbuf destructor
On Sun, Jul 21, 2013 at 11:38 PM, Adrian Chadd wrote:
> hm. There's lots of mbuf free calls in the net80211 TX and RX path; do
> we have to have to set the vnet context during the whole tx/rx path?
>
I'm not sure about that.
In src/sys/netpfil/pf/pf.c, we have this in pf_initialize():
751 /* Mbuf tags */
752 V_pf_mtag_z = uma_zcreate("pf mtags", sizeof(struct m_tag) +
753 sizeof(struct pf_mtag), NULL, NULL, pf_mtag_init, NULL,
754 UMA_ALIGN_PTR, 0);
and further down this:
812 static int
813 pf_mtag_init(void *mem, int size, int how)
814 {
815 struct m_tag *t;
816
817 t = (struct m_tag *)mem;
818 t->m_tag_cookie = MTAG_ABI_COMPAT;
819 t->m_tag_id = PACKET_TAG_PF;
820 t->m_tag_len = sizeof(struct pf_mtag);
821 t->m_tag_free = pf_mtag_free;
822
823 return (0);
824 }
825
826 static void
827 pf_mtag_free(struct m_tag *t)
828 {
829
830 uma_zfree(V_pf_mtag_z, t);
831 }
Can we somehow modify pf_mtag_init() so that it passes the
vnet into the pf_mtag?
Then we can call CURVNET_SET/CURVNET_RESTORE in pf_mtag_free().
--
Craig
>
>
>
> -adrian
>
> On 21 July 2013 23:32, Craig Rodrigues wrote:
> > Hi,
> >
> > I used a kernel config with the following lines:
> >
> > include GENERIC
> > options VIMAGE
> >
> > and compiled a CURRENT kernel from svn://
> svn.freebsd.org/base/head@253346 .
> >
> > I also have PF enabled on my system.
> >
> > Once in a while I have been getting kernel panics like these:
> >
> >
> >
> > (kgdb) #0 doadump (textdump=1) at pcpu.h:236
> > #1 0x808bc617 in kern_reboot (howto=260)
> > at /usr/home/rodrigc/freebsd/head/sys/kern/kern_shutdown.c:447
> > #2 0x808bcb25 in vpanic (fmt=,
> > ap=)
> > at /usr/home/rodrigc/freebsd/head/sys/kern/kern_shutdown.c:754
> > #3 0x808bcb73 in panic (fmt=)
> > at /usr/home/rodrigc/freebsd/head/sys/kern/kern_shutdown.c:683
> > #4 0x8033dff7 in db_panic (addr=,
> > have_addr=, count=,
> > modif=)
> > at /usr/home/rodrigc/freebsd/head/sys/ddb/db_command.c:482
> > #5 0x8033dbcd in db_command (cmd_table=)
> > at /usr/home/rodrigc/freebsd/head/sys/ddb/db_command.c:449
> > #6 0x8033d944 in db_command_loop ()
> > at /usr/home/rodrigc/freebsd/head/sys/ddb/db_command.c:502
> > #7 0x803402f0 in db_trap (type=, code=0)
> > at /usr/home/rodrigc/freebsd/head/sys/ddb/db_main.c:231
> > #8 0x808f3623 in kdb_trap (type=12, code=0, tf= > out>)
> > at /usr/home/rodrigc/freebsd/head/sys/kern/subr_kdb.c:654
> > #9 0x80cda43a in trap_fatal (frame=0xff811dbab6b0,
> > eva=)
> > at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/trap.c:868
> > #10 0x80cda6f4 in trap_pfault (frame=0x0, usermode=0)
> > at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/trap.c:699
> > #11 0x80cd9ef0 in trap (frame=0xff811dbab6b0)
> > at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/trap.c:463
> > #12 0x80cc31a2 in calltrap ()
> > at /usr/home/rodrigc/freebsd/head/sys/amd64/amd64/exception.S:232
> > #13 0x8208f7b7 in pf_mtag_free (t=0xfe00a8797870)
> > at
> > /usr/home/rodrigc/freebsd/head/sys/modules/pf/../../netpfil/pf/pf.c:830
> > #14 0x808a51c9 in mb_dtor_mbuf (mem=0xfe000d0bc500, size=256,
> > arg=0x0) at /usr/home/rodrigc/freebsd/head/sys/kern/kern_mbuf.c:499
> > #15 0x80b55d4d in uma_zfree_arg (zone=0xfe000b4ab900,
> > item=0xfe000d0bc500, udata=0x0)
> > at /usr/home/rodrigc/freebsd/head/sys/vm/uma_core.c:2560
> > #16 0x8092d1f5 in m_freem (mb=) at uma.h:364
> > #17 0x8058ba72 in iwn_tx_done (sc=0xff8000974000,
> > desc=, ackfailcnt=16, status=131 '\203')
> > at /usr/home/rodrigc/freebsd/head/sys/dev/iwn/if_iwn.c:2817
> > #18 0x80583e60 in iwn_notif_intr (sc=0xff8000974000)
> > at /usr/home/rodrigc/freebsd/head/sys/dev/iwn/if_iwn.c:3015
> > #19 0x80583684 in iwn_intr (arg=0xff8000974000)
> > at /usr/home/rodrigc/freebsd/head/sys/dev/iwn/if_iwn.c:3306
> > #20 0x8088daf3 in intr_event_execute_handlers (
> > p=, ie=0xfe000b696600)
> > at /usr/home/rodrigc/freebsd/head/sys/kern/kern_intr.c:1263
> > #21 0x8088e4c6 in ithread_loop (arg=0xfe000b31b040)
> &g
Re: VIMAGE + PF crash in mbuf destructor
On Mon, Jul 22, 2013 at 10:11 AM, Adrian Chadd wrote: > > I don't think the default vnet context is the correct behaviour there. > We'd need to figure out what the vnet context of the mbuf is and set > that. > > What do you think about Marko's suggestion to de-virtualize V_pf_mtag_z? What would be the down side of that? I don't understand enough of the PF code to understand which variables need to be virtual and which don't. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
De-virtualize V_pf_mtag_z to eliminate kernel panics.
Gleb,
Since you did a lot of work in GRN 240233
to fix PF issues, especially for VIMAGE, I thought I would
ask your opinion on the attached patch.
In this post:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-July/001405.html
I reported multiple PF-related panics when VIMAGE was enabled
in my kernel config.
In these posts:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-July/001413.html
http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-July/001420.html
Marko Zec seemed to think that de-virtualizing V_pf_mtag_z
would be a valid solution to this problem, and that keeping
V_pf_mtag_z as a per-vnet variable is not necessary.
What do you think of Marko's comments, and this patch?
Thanks.
--
Craig
Index: sys/netpfil/pf/pf.c
===
--- sys/netpfil/pf/pf.c (revision 253346)
+++ sys/netpfil/pf/pf.c (working copy)
@@ -187,8 +187,7 @@
static VNET_DEFINE(uma_zone_t, pf_sources_z);
#defineV_pf_sources_z VNET(pf_sources_z)
-static VNET_DEFINE(uma_zone_t, pf_mtag_z);
-#defineV_pf_mtag_z VNET(pf_mtag_z)
+uma_zone_t pf_mtag_z;
VNET_DEFINE(uma_zone_t, pf_state_z);
VNET_DEFINE(uma_zone_t, pf_state_key_z);
@@ -749,7 +748,7 @@
V_pf_altqs_inactive = &V_pf_altqs[1];
/* Mbuf tags */
- V_pf_mtag_z = uma_zcreate("pf mtags", sizeof(struct m_tag) +
+ pf_mtag_z = uma_zcreate("pf mtags", sizeof(struct m_tag) +
sizeof(struct pf_mtag), NULL, NULL, pf_mtag_init, NULL,
UMA_ALIGN_PTR, 0);
@@ -803,7 +802,7 @@
mtx_destroy(&pf_overloadqueue_mtx);
mtx_destroy(&pf_unlnkdrules_mtx);
- uma_zdestroy(V_pf_mtag_z);
+ uma_zdestroy(pf_mtag_z);
uma_zdestroy(V_pf_sources_z);
uma_zdestroy(V_pf_state_z);
uma_zdestroy(V_pf_state_key_z);
@@ -827,7 +826,7 @@
pf_mtag_free(struct m_tag *t)
{
- uma_zfree(V_pf_mtag_z, t);
+ uma_zfree(pf_mtag_z, t);
}
struct pf_mtag *
@@ -838,7 +837,7 @@
if ((mtag = m_tag_find(m, PACKET_TAG_PF, NULL)) != NULL)
return ((struct pf_mtag *)(mtag + 1));
- mtag = uma_zalloc(V_pf_mtag_z, M_NOWAIT);
+ mtag = uma_zalloc(pf_mtag_z, M_NOWAIT);
if (mtag == NULL)
return (NULL);
bzero(mtag + 1, sizeof(struct pf_mtag));
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Simulating CD-ROM device with BHyve and ISO disk image?
Hi, I have an ISO image that I am trying to boot in BHyve. I tried this: truncate -s 2G /tmp/disk.img sh /usr/share/examples/bhyve/vmrun.sh -I test.iso -d /tmp/disk.img freenas1 The initial kernel boots, but then fails to mount the root device: mountroot> ? List of GEOM managed disk devices: iso9660/CDROM vtbd1 vtbd0 FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.1-STABLE #0 r+42f8bbd: Sun Aug 18 00:03:25 PDT 2013 r...@build.ixsystems.com:/tank/home/rodrigc/freenas/freenas/os-base/amd64/tank/home/rodrigc/freenas/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64 gcc version 4.2.1 20070831 patched [FreeBSD] CPU: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz (2494.19-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a Stepping = 7 Features=0x8f8bab7f Features2=0x83ba6257 AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant real memory = 536870912 (512 MB) avail memory = 492359680 (469 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 2 package(s) x 1 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. ACPI Warning: FADT (revision 5) is longer than ACPI 2.0 version, truncating length 268 to 244 (20110527/tbfadt-320) ioapic0: Changing APIC ID to 2 ioapic0 irqs 0-16 on motherboard kbd0 at kbdmux0 aesni0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib0: no PRT entry for 0.31.INTA virtio_pci0: port 0x2000-0x201f mem 0xc000-0xc0001fff at device 1.0 on pci0 vtnet0: on virtio_pci0 virtio_pci0: host features: 0x1018020 virtio_pci0: negotiated features: 0x1018020 vtnet0: Ethernet address: 00:a0:98:30:99:23 virtio_pci1: port 0x2040-0x207f mem 0xc0002000-0xc0003fff at device 2.0 on pci0 vtblk0: on virtio_pci1 virtio_pci1: host features: 0x1004 virtio_pci1: negotiated features: 0x1004 vtblk0: 2048MB (4194304 512 byte sectors) virtio_pci2: port 0x2080-0x20bf mem 0xc0004000-0xc0005fff at device 3.0 on pci0 vtblk1: on virtio_pci2 virtio_pci2: host features: 0x1004 virtio_pci2: negotiated features: 0x1004 vtblk1: 254MB (521624 512 byte sectors) uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on pci0 uart2: console (9600,n,8,1) Timecounters tick every 1.000 msec ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled DUMMYNET 0xfe0002106940 with IPv6 initialized (100409) load_dn_sched dn_sched WF2Q+ loaded load_dn_sched dn_sched FIFO loaded load_dn_sched dn_sched PRIO loaded load_dn_sched dn_sched QFQ loaded load_dn_sched dn_sched RR loaded SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 1247092718 Hz quality 1000 Trying to mount root from cd9660:/dev/cd0 [ro]... mountroot: waiting for device /dev/cd0 ... Mounting from cd9660:/dev/cd0 failed with error 19. Trying to mount root from cd9660:/dev/acd0 [ro]... mountroot: waiting for device /dev/acd0 ... Mounting from cd9660:/dev/acd0 failed with error 19. Loader variables: Manual root filesystem specification: : [options] Mount using filesystem and with the specified (optional) option list. eg. ufs:/dev/da0s1a zfs:tank cd9660:/dev/acd0 ro (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /) ? List valid disk boot devices . Yield 1 second (for background tasks) Abort manual input mountroot> ? List of GEOM managed disk devices: iso9660/CDROM vtbd1 vtbd0 Currently the logic for how the root mount device is chosen is hardcoded in the vfs_mountroot_conf0 function: http://bxr.su/FreeBSD/sys/kern/vfs_mountroot.c#823 This logic basically says, try in this order: cd9660:/dev/cd0 cd9660:/dev/acd0 device specified in vfs.root.mountfrom It looks like with BHyve, if I invoke the vmrun.sh script as I have done, the ISO file is not on cd0 or acd0, but is on vtbd1. Is there a way I can mount the ISO file on a simulated cd0 device? This would make the logic in my installer still work. This ISO image successfully boots on real hardware, and in qemu with the "-cdrom" flag. Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Simulating CD-ROM device with BHyve and ISO disk image?
If I do either of these at the mountroot prompt then things work: cd9660:iso9660/CDROM cd9660:/dev/vtbd1 I was kind of hoping to leverage off the existing logic in vfs_mountroot.c which tries to mount in this order: cd9660:/dev/cd0 cd9660:/dev/acd0 I don't want to hard-code any more device names in my code. If I put the following entries in /boot/loader.conf of my ISO image: boot_cdrom="YES" vfs.root.mountfrom="cd9660:iso9660/CDROM" then the logic in vfs_mountroot.c will try to mount the root file system from the devices in this order: cd9660:/dev/cd0 cd9660:/dev/acd0 cd9660:iso9660/CDROM That seems to work. I'd rather have a flag to BHyve to force an ISO image to be associated with a CDROM device l Iike /dev/cd0. This would be similar to what can be done with QEMU, VirtualBox, VMWare, etc. -- Craig On Aug 27, 2013 9:08 PM, "Andreas Nilsson" wrote: > What happens if you specify > > cd9660:iso9660/CDROM when first boot attempt has failed? > > You could also try cd9660:/dev/vtbd1 > > > Best regards > Andreas > > > On Wed, Aug 28, 2013 at 1:12 AM, Craig Rodrigues wrote: > >> Hi, >> >> I have an ISO image that I am trying to boot in BHyve. >> >> I tried this: >> >> truncate -s 2G /tmp/disk.img >> sh /usr/share/examples/bhyve/vmrun.sh -I test.iso -d /tmp/disk.img >> freenas1 >> >> The initial kernel boots, but then fails to mount the root device: >> >> >> >> mountroot> ? >> >> List of GEOM managed disk devices: >> iso9660/CDROM vtbd1 vtbd0 >> >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 9.1-STABLE #0 r+42f8bbd: Sun Aug 18 00:03:25 PDT 2013 >> r...@build.ixsystems.com: >> /tank/home/rodrigc/freenas/freenas/os-base/amd64/tank/home/rodrigc/freenas/freenas/FreeBSD/src/sys/FREENAS.amd64 >> amd64 >> gcc version 4.2.1 20070831 patched [FreeBSD] >> CPU: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz (2494.19-MHz K8-class CPU) >> Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a >> Stepping = 7 >> >> >> Features=0x8f8bab7f >> >> >> Features2=0x83ba6257 >> AMD Features=0x20100800 >> AMD Features2=0x1 >> TSC: P-state invariant >> real memory = 536870912 (512 MB) >> avail memory = 492359680 (469 MB) >> Event timer "LAPIC" quality 400 >> ACPI APIC Table: >> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs >> FreeBSD/SMP: 2 package(s) x 1 core(s) >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP): APIC ID: 1 >> WARNING: VIMAGE (virtualized network stack) is a highly experimental >> feature. >> ACPI Warning: FADT (revision 5) is longer than ACPI 2.0 version, >> truncating >> length 268 to 244 (20110527/tbfadt-320) >> ioapic0: Changing APIC ID to 2 >> ioapic0 irqs 0-16 on motherboard >> kbd0 at kbdmux0 >> aesni0: on motherboard >> cryptosoft0: on motherboard >> acpi0: on motherboard >> atrtc0: port 0x70-0x71,0x72-0x77 irq 8 on acpi0 >> Event timer "RTC" frequency 32768 Hz quality 0 >> Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 >> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >> pcib0: port 0xcf8-0xcff on acpi0 >> pci0: on pcib0 >> pcib0: no PRT entry for 0.31.INTA >> virtio_pci0: port 0x2000-0x201f mem >> 0xc000-0xc0001fff at device 1.0 on pci0 >> vtnet0: on virtio_pci0 >> virtio_pci0: host features: 0x1018020 >> >> virtio_pci0: negotiated features: 0x1018020 >> >> vtnet0: Ethernet address: 00:a0:98:30:99:23 >> virtio_pci1: port 0x2040-0x207f mem >> 0xc0002000-0xc0003fff at device 2.0 on pci0 >> vtblk0: on virtio_pci1 >> virtio_pci1: host features: 0x1004 >> virtio_pci1: negotiated features: 0x1004 >> vtblk0: 2048MB (4194304 512 byte sectors) >> virtio_pci2: port 0x2080-0x20bf mem >> 0xc0004000-0xc0005fff at device 3.0 on pci0 >> vtblk1: on virtio_pci2 >> virtio_pci2: host features: 0x1004 >> virtio_pci2: negotiated features: 0x1004 >> vtblk1: 254MB (521624 512 byte sectors) >> uart2: <16550 or compatible> port 0x3f8-0x3ff irq 4 at device 31.0 on pci0 >> uart2: console (9600,n,8,1) >> Timecounters tick every 1.000 msec >> ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, >> logging disabled >> DUMMYNET 0xfe0002106940 with IPv6 initialized (100409) >> load_dn_sched dn_sched WF2Q+ loaded >> load_dn_sched dn_sched FIFO
Problems with missing ttyv devices in BHyve?
Hi, I am working on the FreeNAS project for iXsystems. FreeNAS is derived from FreeBSD. I am trying to use BHyve to do testing of FreeNAS, but I am running into some problems. I did the following: (1) Downloaded the latest daily build ISO of FreeNAS, which has some fixes to make the installer boot up properly under BHyve: http://sourceforge.net/projects/freenas/files/FreeNAS-nightlies/ (2) Run this script as root, to create a bridge and connect it to tap0 device: == #!/bin/sh set -x iface=em0 ifconfig bridge0 create ifconfig tap0 create ifconfig bridge0 addm $iface addm tap0 up sysctl net.link.tap.user_open=1 sysctl net.link.tap.up_on_open=1 == (3) Create a 2G disk: truncate -s 2G /tmp/disk.img (4) Run this to boot the installer: sh -x /usr/share/examples/bhyve/vmrun.sh -m 2048 -d /tmp/disk.img -I FreeNAS-9.2.0-ALPHA-1bf84a0-x64.iso freenas1_vm (5) In the installer, select disk vtbd0 as the device to install to. (6) Shut down bhyve, and run again to boot off the disk: sh -x /usr/share/examples/bhyve/vmrun.sh -m 2048 -d /tmp/disk.img freenas1_vm I am getting some errors about missing tty devices, so I can't log into the TrueNAS console. Does anyone have any ideas what the fix for this should be? It would be handy to boot FreeNAS inside BHyve, especially for testing. I don't get this problem under QEMU, Virtualbox, or VMWare still work. Thanks. -- Craig bhyve_log.txt Script started on Fri Aug 30 16:29:00 2013 command: sh -x /usr/share/examples/bhyve/vmrun.sh -m 2048 -d /tmp/disk.img freenas1_vm + LOADER=/usr/sbin/bhyveload + BHYVECTL=/usr/sbin/bhyvectl + FBSDRUN=/usr/sbin/bhyve + DEFAULT_MEMSIZE=512 + DEFAULT_CPUS=2 + DEFAULT_TAPDEV=tap0 + DEFAULT_VIRTIO_DISK=./diskdev + DEFAULT_ISOFILE=./release.iso + id -u + [ 0 -ne 0 ] + kldstat -n vmm + [ 0 -ne 0 ] + force_install=0 + isofile=./release.iso + memsize=512 + cpus=2 + virtio_diskdev=./diskdev + tapdev=tap0 + apic_opt='' + gdbport=0 + getopts haic:g:I:m:d:t: c + memsize=2048 + getopts haic:g:I:m:d:t: c + virtio_diskdev=/tmp/disk.img + getopts haic:g:I:m:d:t: c + shift 4 + [ 1 -ne 1 ] + vmname=freenas1_vm + [ ! -f /tmp/disk.img ] + [ ! -r /tmp/disk.img ] + [ ! -w /tmp/disk.img ] + echo 'Launching virtual machine "freenas1_vm" ...' Launching virtual machine "freenas1_vm" ... + [ 1 ] + /usr/sbin/bhyvectl --vm=freenas1_vm --destroy + file /tmp/disk.img + grep ': x86 boot sector' + rc=0 + [ 0 -ne 0 ] + [ 0 -ne 0 ] + need_install=0 + [ 0 -eq 1 -o 0 -eq 1 ] + BOOTDISK=/tmp/disk.img + installer_opt='' + /usr/sbin/bhyveload -m 2048 -d /tmp/disk.img freenas1_vm Consoles: userboot FreeBSD/amd64 User boot, Revision 1.1 (rodrigc@pcbsd-4708, Sat Aug 24 16:54:52 PDT 2013) |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/Loading /boot/defaults/loader.conf -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/*/boot/kernel/kernel text=0xb8db88 -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-
Re: PXE booting into a Hyper-V 2012 R2 instance - slows to a crawl?
On Tue, Sep 24, 2013 at 6:53 AM, Karl Pielorz wrote: > > > We are creating new Hyper-V instances using our PXE boot environment > (which boots you into a FreeBSD 9.1 amd64 O/S) - we'd build a Hyper-V aware > kernel etc. from that (after SVN'ing the latest Hyper-V project source). > PXE involves DHCP and TFTP. I would recommend that you do the following: (1) On the machine hosting your TFTP server, look at the TFTP logs and look for any clues. If your TFTP server is a FreeBSD box, look at /var/log/xferlog. I don't know what logs are available for a Windows TFTP server. (2) In between your booting machine, and your TFTP server, use Wireshark or tcpdump to capture the network traffic and analyze what is going on. (3) By default, the FreeBSD PXE loader uses a TFTP blocksize of 512 bytes per block (which is the original TFTP standard). You can increase the size of that to something bigger, depending on the MTU of your network. Usually 1400 bytes will work. In /boot/defaults/loader.conf, you can see this: #tftp.blksize="1428"# Set the RFC 2348 TFTP block size. # If the TFTP server does not support RFC 2348, # the block size is set to 512. If the value # is out of range ( < 8 || > 9008 ) an error is # returned. If you put in yout /boot/loader.conf file: tftp.blksize="1428" you can see if that speeds up the TFTP transfer during PXE booting. > > Hyper-V would take seconds to boot this - but since changing over to R2 it > takes a 'very, very long time'. > > The initial PXE bit happens quickly (i.e. seeing the server, and getting > the BTX output out) but then it gets stuck with the \ spinner - it'll spin > for a short time, stop, spin again for a short time, stop again. > The spinner is the part where it is loading the kernel. For PXE, this is happening over TFTP. This looks like a slow network problem, so you need to analyze it a bit. I wrote some notes on PXE booting in FreeBSD which you might want to get some ideas from: http://www.freebsd.org/doc/handbook/network-pxe-nfs.html -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhve bug report: attempting to physical DVD drive for install CD is unbootable
On Mon, Sep 30, 2013 at 4:51 PM, Aryeh Friedman wrote: > -- Forwarded message -- > From: Aryeh Friedman > Date: Mon, Sep 30, 2013 at 7:51 PM > Subject: Re: bhve bug report: attempting to physical DVD drive for install > CD is unbootable > To: Neel Natu > > > One of the features we want in petitecloud is to allow people to just pop > in a CD and install an instance (see the new UI for 0.1.7 that will be > released sometime tonight or tomorrow) > It's not the most elegant solution and can potentially use a lot of disk space, but you might want to consider something where in the petitecloud UI, if /dev/cd[n] or /dev/acd[n] is used as the input device, you rip the CD ( see: http://forums.freebsd.org/showthread.php?t=1195 ) and then pass the ripped ISO file as a parameter to bhyvectl. I ran into a related problem with FreeNAS: http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-August/001484.html I'm actually fixing the FreeNAS installer so that it will work with BHyve. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Disabling ATA driver for IDE devices on Hyper-V
Hi, There was some discussion between Larry Melia and Alexander Motin on this and related topics a few months ago. Here are some relevant threads: http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-April/001180.html http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-May/001281.html http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-June/001297.html If you are continuing Larry's work, and can get feedback from Alexander, that would be great. -- Craig On Mon, Sep 30, 2013 at 2:02 PM, Abhishek Gupta (LIS) wrote: > Hi folks, > > This may have been asked before but the solution we have currently is less > than ideal. Please bear with me on the repetition and provide any > suggestions you may have. > > One of the issues with Hyper-V is that it does not virtualize the CD > device and therefore we rely on the ATA driver in the guest operating > system to manage CDROMs. What we would like to do is disable the ATA driver > for all device types except the CDROM in the presence of Hyper-V. Can > someone help us in answering the following in this regard: > > > a) What is the correct location in the FreeBSD ATA code where we can > put in selective disabling? > > b) Is this a good approach? Is there something cleaner you can think > of? > > As of now we have implemented a disengage driver that disables ATA driver > permanently on Hyper-V. However this is less than desirable and we are > looking for a better solution. Please let us know if you have some thoughts. > > Thanks, > Abhishek > > ___ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscr...@freebsd.org" > ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Problems with missing ttyv devices in BHyve?
On Sat, Sep 7, 2013 at 9:10 AM, Peter Grehan wrote: > Hi Craig, > > > Does anyone have any ideas what the fix for this should be? >> > > You'll have to edit the /etc/ttys file and > > -ttyu2 "/usr/libexec/getty std.9600" dialup off secure > +ttyu2 "/usr/libexec/getty std.9600" vt100 on secure > > bhyve is ttyu2 rather than ttyu0 since it emulates a PCI serial port. > > Maybe this could go into the FreeNAS repo - I've never really understood > the aversion to having logins open on the serial ports by default. Seems a > relic of 1980's Unix dialup modems. > > Thanks Peter, thats worked. With some other changes that I have done, I have been able to successfully install and boot the latest FreeNAS code inside BHyve. It's very cool, and extremely useful for testing FreeNAS. A few questions for you. (1) I want to get the same FreeNAS image working inside Bhyve and on real hardware. In the source code for /sbin/init, when it parses /etc/ttys, if the entry does not exist, it prints out an error message about not being able to find the tty entry to open it. Do you know if there is a way to specify an entry for /dev/ttyu2, only if it exists, and not print out an error message otherwise? That way, I can get the same image to boot on real hardware, which has ttyv console devices, but does not have /dev/ttyu2, and also in BHyve, and /sbin/init won't print any errors. (2) Is it possible to connect to the console of the BHyve VM via /dev/ttyu2 or some other mechanism? I would like to be able to connect to the BHyve VM and use something like Expect or py-expect to execute console commands in the VM, for automated testing. Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Problems with missing ttyv devices in BHyve?
On Thu, Oct 3, 2013 at 10:38 PM, Peter Grehan wrote: > > > (2) Is it possible to connect to the console of the BHyve VM via >> /dev/ttyu2 >>or some other mechanism? I would like to be able to connect to >> the BHyve VM >>and use something like Expect or py-expect to execute console >> commands in the VM, >>for automated testing. >> > > Not currently - the only options you have are to run bhyve under expect, > or set up a network port and log in and run expect on commands. > > I have what I think is a plan to fix this: maybe for 10.1. > OK. Once FreeNAS is fully set up, networking via the tap device works great, and I have been able to access the web GUI and ssh into FreeNAS inside BHyve with no problems. However, what I am trying to test is the early stage bootup of our installation CD, before FreeNAS is installed on the disk. When our CD boots up, it presents a text-based dialog menu where the user can select certain items and choose which disk to install on. If I could access the console in BHyve at this point via expect to drive the install menu, that would be very useful. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Use humanize_number() inside bhyveload?
Hi,
In /usr/src/usr.bin/truncate/truncate.c , you can see
that the '-s' parameter to the truncate utility
is parsed with the expand_number() utility.
Can we use expand_number() in bhyve_load? That will allow
people to enter parameters like:
-m 2G
This is handy and consistent with other FreeBSD utilities.
Index: usr.sbin/bhyveload/Makefile
===
--- usr.sbin/bhyveload/Makefile (revision 256057)
+++ usr.sbin/bhyveload/Makefile (working copy)
@@ -4,8 +4,8 @@
SRCS= bhyveload.c
MAN= bhyveload.8
-DPADD+=${LIBVMMAPI}
-LDADD+=-lvmmapi
+DPADD+=${LIBVMMAPI} ${LIBUTIL}
+LDADD+=-lvmmapi -lutil
WARNS?=3
Index: usr.sbin/bhyveload/bhyveload.8
===
--- usr.sbin/bhyveload/bhyveload.8 (revision 256057)
+++ usr.sbin/bhyveload/bhyveload.8 (working copy)
@@ -62,11 +62,11 @@
.Bl -tag -width indent
.It Fl m Ar mem-size
.Ar mem-size
-is the amount of memory allocated to the guest in units of megabytes.
+is the amount of memory allocated to the guest.
.Pp
The default value of
.Ar mem-size
-is 256.
+is 256 MB.
.It Fl d Ar disk-path
The
.Ar disk-path
@@ -83,7 +83,7 @@
.Pa /freebsd/release.iso
and has 1GB memory allocated to it:
.Pp
-.Dl "bhyveload -m 1024 -d /freebsd/release.iso freebsd-vm"
+.Dl "bhyveload -m 1G -d /freebsd/release.iso freebsd-vm"
.Sh SEE ALSO
.Xr bhyve 4 ,
.Xr bhyve 8 ,
Index: usr.sbin/bhyveload/bhyveload.c
===
--- usr.sbin/bhyveload/bhyveload.c (revision 256057)
+++ usr.sbin/bhyveload/bhyveload.c (working copy)
@@ -67,9 +67,11 @@
#include
#include
#include
+#include
#include
#include
#include
+#include
#include
#include
#include
@@ -581,7 +583,9 @@
break;
case 'm':
- mem_size = strtoul(optarg, NULL, 0) * MB;
+ if (expand_number(optarg, &mem_size) == -1)
+ errx(EXIT_FAILURE,
+ "invalid size argument `%s'", optarg);
break;
case '?':
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Use humanize_number() inside bhyveload?
On Fri, Oct 4, 2013 at 1:40 PM, Peter Grehan wrote:
> Hi Craig,
>
>
> Can we use expand_number() in bhyve_load?
>>
>
> Please commit this change :)
>
> later,
>
> Peter.
>
>
Here is a bigger patch, which uses expand_number() in bhyveload and bhyve,
and also changes comments
in vmrun.sh to reflect the change.
Index: share/examples/bhyve/vmrun.sh
===
--- share/examples/bhyve/vmrun.sh (revision 256073)
+++ share/examples/bhyve/vmrun.sh (working copy)
@@ -31,7 +31,7 @@
BHYVECTL=/usr/sbin/bhyvectl
FBSDRUN=/usr/sbin/bhyve
-DEFAULT_MEMSIZE=512
+DEFAULT_MEMSIZE=512M
DEFAULT_CPUS=2
DEFAULT_TAPDEV=tap0
@@ -47,7 +47,7 @@
echo " -g: listen for connection from kgdb at "
echo " -i: force boot of the Installation CDROM image"
echo " -I: Installation CDROM image location (default is
${DEFAULT_ISOFILE})"
- echo " -m: memory size in MB (default is ${DEFAULT_MEMSIZE}MB)"
+ echo " -m: memory size (default is ${DEFAULT_MEMSIZE})"
echo " -t: tap device for virtio-net (default is $DEFAULT_TAPDEV)"
echo ""
echo " This script needs to be executed with superuser privileges"
Index: usr.sbin/bhyve/Makefile
===
--- usr.sbin/bhyve/Makefile (revision 256073)
+++ usr.sbin/bhyve/Makefile (working copy)
@@ -17,8 +17,8 @@
NO_MAN=
-DPADD= ${LIBVMMAPI} ${LIBMD} ${LIBPTHREAD}
-LDADD= -lvmmapi -lmd -lpthread
+DPADD= ${LIBVMMAPI} ${LIBMD} ${LIBUTIL} ${LIBPTHREAD}
+LDADD= -lvmmapi -lmd -lutil -lpthread
WARNS?= 2
Index: usr.sbin/bhyve/bhyverun.c
===
--- usr.sbin/bhyve/bhyverun.c (revision 256073)
+++ usr.sbin/bhyve/bhyverun.c (working copy)
@@ -37,7 +37,9 @@
#include
#include
+#include
#include
+#include
#include
#include
#include
@@ -541,6 +543,9 @@
break;
case 'm':
memsize = strtoul(optarg, NULL, 0) * MB;
+ if (expand_number(optarg, &memsize) == -1)
+ errx(EXIT_FAILURE,
+ "invalid size argument `%s'", optarg);
break;
case 'H':
guest_vmexit_on_hlt = 1;
Index: usr.sbin/bhyveload/Makefile
===
--- usr.sbin/bhyveload/Makefile (revision 256073)
+++ usr.sbin/bhyveload/Makefile (working copy)
@@ -4,8 +4,8 @@
SRCS= bhyveload.c
MAN= bhyveload.8
-DPADD+= ${LIBVMMAPI}
-LDADD+= -lvmmapi
+DPADD+= ${LIBVMMAPI} ${LIBUTIL}
+LDADD+= -lvmmapi -lutil
WARNS?= 3
Index: usr.sbin/bhyveload/bhyveload.8
===
--- usr.sbin/bhyveload/bhyveload.8 (revision 256073)
+++ usr.sbin/bhyveload/bhyveload.8 (working copy)
@@ -1,4 +1,4 @@
-.\"
+\"
.\" Copyright (c) 2012 NetApp Inc
.\" All rights reserved.
.\"
@@ -60,13 +60,29 @@
.Sh OPTIONS
The following options are available:
.Bl -tag -width indent
-.It Fl m Ar mem-size
+.It Fl m Ar mem-size Xo
+.Sm off
+.Op Cm K | k | M | m | G | g | T | t
+.Xc
+.Sm on
.Ar mem-size
-is the amount of memory allocated to the guest in units of megabytes.
+is the amount of memory allocated to the guest.
.Pp
+The
+.Ar mem-size
+argument may be suffixed with one of
+.Cm K ,
+.Cm M ,
+.Cm G
+or
+.Cm T
+(either upper or lower case) to indicate a multiple of
+Kilobytes, Megabytes, Gigabytes or Terabytes
+respectively.
+.Pp
The default value of
.Ar mem-size
-is 256.
+is 256M.
.It Fl d Ar disk-path
The
.Ar disk-path
@@ -83,7 +99,7 @@
.Pa /freebsd/release.iso
and has 1GB memory allocated to it:
.Pp
-.Dl "bhyveload -m 1024 -d /freebsd/release.iso freebsd-vm"
+.Dl "bhyveload -m 1G -d /freebsd/release.iso freebsd-vm"
.Sh SEE ALSO
.Xr bhyve 4 ,
.Xr bhyve 8 ,
Index: usr.sbin/bhyveload/bhyveload.c
===
--- usr.sbin/bhyveload/bhyveload.c (revision 256073)
+++ usr.sbin/bhyveload/bhyveload.c (working copy)
@@ -67,9 +67,11 @@
#include
#include
#include
+#include
#include
#include
#include
+#include
#include
#include
#include
@@ -581,7 +583,9 @@
break;
case 'm':
- mem_size = strtoul(optarg, NULL, 0) * MB;
+ if (expand_number(optarg, &mem_size) == -1)
+ errx(EXIT_FAILURE,
+"invalid size argument `%s'", optarg);
break;
case '?':
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Use humanize_number() inside bhyveload?
On Tue, Oct 8, 2013 at 9:00 PM, Neel Natu wrote: > > Thanks for the patch - submitted as r256176 with some modifications to > preserve backward compatibility. > > best > Neel > > Thanks! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Using wlan0 device over tap0 device with BHyve?
Hi, I am using this script to set up my tap device before running BHyve: = #!/bin/sh #iface=em0 iface=wlan0 ifconfig bridge0 create ifconfig tap0 create ifconfig bridge0 addm $iface addm tap0 up sysctl net.link.tap.user_open=1 sysctl net.link.tap.up_on_open=1 = If I change the value of "iface" so that it uses em0 instead of wlan0, networking works without a problem in my BHyve VM, and my BHyve VM can send out a DHCP request, and get an address. However, if I set the value of "iface" to wlan0, to use the wireless NIC in my laptop, then networking does not work in my BHyve VM, and my BHyve VM tries to send out a DHCP request to get an address, but it doesn't go out over the network. Any ideas why this isn't working for me? em0: flags=8843 metric 0 mtu 1500 options=4219b ether e4:11:5b:55:a2:2c nd6 options=29 media: Ethernet autoselect status: no carrier iwn0: flags=8843 metric 0 mtu 2290 ether 8c:70:5a:1b:92:54 nd6 options=21 media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng status: associated lo0: flags=8049 metric 0 mtu 16384 options=63 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff00 nd6 options=21 wlan0: flags=8943 metric 0 mtu 1500 ether 8c:70:5a:1b:92:54 inet 10.2.1.110 netmask 0x broadcast 10.2.255.255 nd6 options=29 media: IEEE 802.11 Wireless Ethernet MCS mode 11ng status: associated ssid Blahblah channel 1 (2412 MHz 11g ht/20) bssid 20:aa:4b:38:55:91 country US authmode WPA2/802.11i privacy ON deftxkey UNDEF AES-CCM 2:128-bit AES-CCM 3:128-bit txpower 11 bmiss 10 scanvalid 60 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi 7 roam:rate 64 protmode CTS ampdulimit 64k ampdudensity 8 -amsdutx amsdurx shortgi wme roaming MANUAL bridge0: flags=8843 metric 0 mtu 1500 ether 02:3e:3a:7a:d0:00 nd6 options=9 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 200 member: wlan0 flags=143 ifmaxaddr 0 port 4 priority 128 path cost 3 tap0: flags=8902 metric 0 mtu 1500 options=8 ether 00:bd:8e:95:01:00 nd6 options=29 media: Ethernet autoselect status: no carrier -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Using wlan0 device over tap0 device with BHyve?
Aryeh, In this case, I am not connected to the lan nic. From you look at the ifconfig output from my previous posting, you will see that my bridge0 interface only has two members: tap0 wlan0 -- Craig On Wed, Oct 9, 2013 at 4:36 PM, Aryeh Friedman wrote: > It sometimes doesn't like adding both the lan nic and tap to the bridge at > the same time > > > On Wed, Oct 9, 2013 at 7:30 PM, Craig Rodrigues > wrote: > > > Hi, > > > > I am using this script to set up my tap device before > > running BHyve: > > > > = > > #!/bin/sh > > > > > > #iface=em0 > > iface=wlan0 > > > > ifconfig bridge0 create > > ifconfig tap0 create > > ifconfig bridge0 addm $iface addm tap0 up > > > > sysctl net.link.tap.user_open=1 > > sysctl net.link.tap.up_on_open=1 > > = > > > > > > If I change the value of "iface" so that it uses em0 instead of > > wlan0, networking works without a problem in my BHyve VM, and my BHyve VM > > can send out a DHCP request, and get an address. > > > > However, if I set the value of "iface" to wlan0, to use the wireless NIC > > in my laptop, then networking does not work in my BHyve VM, and > > my BHyve VM tries to send out a DHCP request to get an address, but it > > doesn't go out over the network. > > > > Any ideas why this isn't working for me? > > > > > > em0: flags=8843 metric 0 mtu 1500 > > > > > > > options=4219b > > ether e4:11:5b:55:a2:2c > > nd6 options=29 > > media: Ethernet autoselect > > status: no carrier > > iwn0: flags=8843 metric 0 mtu > 2290 > > ether 8c:70:5a:1b:92:54 > > nd6 options=21 > > media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng > > status: associated > > lo0: flags=8049 metric 0 mtu 16384 > > options=63 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > > inet 127.0.0.1 netmask 0xff00 > > nd6 options=21 > > wlan0: flags=8943 metric > 0 > > mtu 1500 > > ether 8c:70:5a:1b:92:54 > > inet 10.2.1.110 netmask 0x broadcast 10.2.255.255 > > nd6 options=29 > > media: IEEE 802.11 Wireless Ethernet MCS mode 11ng > > status: associated > > ssid Blahblah channel 1 (2412 MHz 11g ht/20) bssid > > 20:aa:4b:38:55:91 > > country US authmode WPA2/802.11i privacy ON deftxkey UNDEF > > AES-CCM 2:128-bit AES-CCM 3:128-bit txpower 11 bmiss 10 scanvalid > > 60 > > bgscan bgscanintvl 300 bgscanidle 250 roam:rssi 7 roam:rate 64 > > protmode CTS ampdulimit 64k ampdudensity 8 -amsdutx amsdurx > shortgi > > wme roaming MANUAL > > bridge0: flags=8843 metric 0 mtu > > 1500 > > ether 02:3e:3a:7a:d0:00 > > nd6 options=9 > > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > > member: tap0 flags=143 > > ifmaxaddr 0 port 6 priority 128 path cost 200 > > member: wlan0 flags=143 > > ifmaxaddr 0 port 4 priority 128 path cost 3 > > tap0: flags=8902 metric 0 mtu 1500 > > options=8 > > ether 00:bd:8e:95:01:00 > > nd6 options=29 > > media: Ethernet autoselect > > status: no carrier > > > > > > -- > > Craig > > ___ > > freebsd-virtualization@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to " > > freebsd-virtualization-unsubscr...@freebsd.org" > > > ___ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscr...@freebsd.org" > ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Running FreeNAS under BHyve
Hi, Here is what I have been doing to run FreeNAS under BHyve. (1) Download latest FreeNAS-9.2.0-ALPHA ISO from: http://sourceforge.net/projects/freenas/files/FreeNAS-nightlies/2013-10-09/x64/ (2) Run this script to set up tap device: = #!/bin/sh # Change following to your network interface iface=em0 ifconfig bridge0 create ifconfig tap0 create ifconfig bridge0 addm $iface addm tap0 up sysctl net.link.tap.user_open=1 sysctl net.link.tap.up_on_open=1 = (3) Create a 4G disk image: truncate -s 4G /tmp/disk.img (4) Run the FreeNAS installer under BHyve and install it to the disk created in step (3): sh /usr/share/examples/bhyve/vmrun.sh -I FreeNAS-9.2.0-ALPHA-a2fb413-x64.iso -d /tmp/disk.img -m 4096 -t tap0 freenas_vm (5) Once the installer has completed, select Shutdown. (6) In a separate window, kill the vm bhyvectl --destroy --vm=freenas_vm (7) Restart BHyve, but this time, do not boot off the ISO image: sh /usr/share/examples/bhyve/vmrun.sh -d /tmp/disk.img -m 4G -t tap0 freenas_vm If DHCP works, you should be able to access the FreeNAS web GUI using a web browser, via the IP address that the BHyve VM got from DHCP. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Running FreeNAS under BHyve
On Thu, Oct 10, 2013 at 6:31 PM, Aryeh Friedman wrote: > > > > On Thu, Oct 10, 2013 at 7:36 PM, Craig Rodrigues wrote: > >> >> If DHCP works, you should be able to access the FreeNAS web GUI >> using a web browser, via the IP address that the BHyve VM got from DHCP. >> >> >> > How do you find this out sort of asking your local DHCP server which may > or may not be available depending on setup of DHCP > > If a DHCP address is obtained, it will be displayed on the console. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Running FreeNAS under BHyve
On Sat, Oct 12, 2013 at 2:39 PM, Aryeh Friedman wrote: > You can always run bhyve from the command line [we even provide a script >>> for this, to be documented in petitecloud 0.2]) in such cases how do >>> you get the IP address and other configurations on each lease change? >>> >> FreeNAS is using all the infrastructure of FreeBSD, so you would have to use the same technique when running FreeBSD under petitecloud to get this information. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Running FreeNAS under BHyve
On Sat, Oct 12, 2013 at 8:44 PM, Michael Dexter wrote: > > Hello, > > On 10/10/13 4:36 PM, Craig Rodrigues wrote: > > Here is what I have been doing to run FreeNAS under BHyve. > > Note that the script archive at bhvye.org has a script, nanobsd2bhyve.sh > that makes this a one step process to modify the FreeNAS .img file and > generates a launch script. > > Thanks for writing that script. I've found it to be generally unnecessary after my latest installer changes to FreeNAS itself. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Running bhyve on a AMD 1075T Phenom
Hi, I have an older machine with an AMD CPU, so I took a look at the http://svnweb.freebsd.org/base/projects/bhyve_svm/branch. I noticed that this branch has not been updated with the latest changes from HEAD, specifically the changes done in this branch: http://svnweb.freebsd.org/base/projects/bhyve_npt_pmap/ . I tried to check out the projects/bhyve_npt_pmap branch and manually do an "svn merge" from CURRENT. I resolved a few merge conflicts, but saw that there are quite a few changes to the pmap code and the svm_vminit() function to accomodate an API change due to bhyve_npt_pmap changes. There was a lot of non-trivial stuff done in the amd64 pmap code and the BHyve intel module for bhyve_npt_pmap, and these changes have not been made for the amd svm code. Does anyone have patches for the svm code? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
VIMAGE: Freed UMA key was not empty
Bjoern, I was looking at this bug in FreeNAS: "Panic on new jail with vnet" https://bugs.freenas.org/issues/3102 It turns out that the root cause of the problem is the same as the one mentioned here: http://lists.freebsd.org/pipermail/freebsd-current/2010-November/021280.html Using the testcase in that mail, I can reproduce these error messages: Freed UMA keg was not empty (10 items). Lost 1 pages of memory. and eventually get a kernel panic. In this message, you mentioned that you might have a patch which fixes the problem: http://lists.freebsd.org/pipermail/freebsd-current/2010-November/021316.html By any chance, do you have a patch handy? I can quickly try it out and provide feedback. Thanks. -- Craig -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Bhyve: acpi0: AcpiEnterSleepStatePrep failed - AE_NOT_FOUND
Hi, I notice that in BHyve, if I try to do "shutdown -p now", there is an ACPI error: == Halting and powering down... Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop...at Syncing disks, vnodes remaining...0 done All buffers synced. Uptime: 17s acpi0: AcpiEnterSleepStatePrep failed - AE_NOT_FOUND The operating system has halted. Please press any key to reboot. == Is that a known issue? Is there another way to trigger a BHyve VM to shutdown from inside the VM? Right now I need to "bhyve --destroy --vm=myvm". -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
VIMAGE: Freed UMA keg was not empty
Adrian, As part of looking at this bug in FreeNAS: "Panic on new jail with vnet" https://bugs.freenas.org/issues/3102 I started looking at this thread: http://lists.freebsd.org/pipermail/freebsd-current/2010-November/021280.html Based on this clue from Thiery: http://lists.freebsd.org/pipermail/freebsd-current/2010-November/021291.html I fixed some VIMAGE related memory leaks related to routetbl. I also fixed some VIMAGE related kernel panics when I tried creating and deleting lots of jails: http://lists.freebsd.org/pipermail/svn-src-all/2013-November/077178.html http://lists.freebsd.org/pipermail/svn-src-all/2013-November/077192.html http://lists.freebsd.org/pipermail/svn-src-all/2013-November/077195.html I haven't eliminated all the "Freed UMA keg was not empty" error messages. Do you have any recommendations for how I can track down the last few UMA related memory leaks? I am new to debugging UMA so would appreciate any pointers that you may have. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: VIMAGE: Freed UMA keg was not empty
On Wed, Nov 27, 2013 at 2:05 AM, Adrian Chadd wrote: > > Well, the first step is figuring out which UMA zones are actually > problematic. Isn't it logging which zones aren't empty? The error messages on the console look like this: Freed UMA keg was not empty (203 items). Lost 1 pages of memory. Freed UMA keg was not empty (36 items). Lost 2 pages of memory. That doesn't really tell which UMA zone isn't empty. Is there some technqiue to figure this out? I tried "vmstat -z" and "vmstat -m", but while those gave clues, it didn't point to which UMA zone was leaking. Is there some other technique or tool that I can use? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: VIMAGE: Freed UMA keg was not empty
On Wed, Nov 27, 2013 at 1:11 PM, Adrian Chadd wrote: > Modify that function to print out keg->uk_name as well. > Done. See: http://lists.freebsd.org/pipermail/svn-src-all/2013-November/077349.html Now if I run a kernel with VIMAGE enabled, and run the testcase mentioned here: http://lists.freebsd.org/pipermail/freebsd-current/2010-November/021280.html I get this on the console: Freed UMA keg (udp_inpcb) was not empty (30 items). Lost 3 pages of memory. Freed UMA keg (udpcb) was not empty (249 items). Lost 1 pages of memory. ifa_del_loopback_route: deletion failed: 48 This certainly helps narrow down where to look for problems. I'll see if I can post more fixes to eliminate these error messages. Thanks for your suggestion! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve(8) man page?
Hi, Do you have an mdoc formatted version of http://people.freebsd.org/~grehan/bhyve_8-v0.1.txt which is ready for commit to HEAD? It would be handy to have this in the tree, because it is annoying to have to look at the bhyve source code to figure out what all the command-line options are for. Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Booting a disk image with GRUB in BHyve?
Hi, I tried the following: (1) Downloaded SmartOS USB image from: http://wiki.smartos.org/display/DOC/Download+SmartOS (2) Uncompressed the bz2 file. (3) Ran "file" on the binary: # file smartos-latest-USB.img smartos-latest-USB.img: x86 boot sector; GRand Unified Bootloader, stage1 version 0x3, stage2 address 0x2000, stage2 segment 0x200; partition 1: ID=0xc, starthead 1, startsector 63, 3903732 sectors, extended partition table (last)\011, code offset 0x48 (4) Tried vmrun.sh: sh /usr/share/examples/bhyve/vmrun.sh -d ./smartos-latest-USB.img -m 4G smartos_vm1 I got this error: = Launching virtual machine "smartos_vm1" ... Consoles: userboot FreeBSD/amd64 User boot, Revision 1.1 (rodrigc@pcbsd-4708, Mon Oct 14 14:36:53 PDT 2013) \ can't load 'kernel' Type '?' for a list of commands, 'help' for more detailed help. OK How can I boot something which uses GRUB bootloader in BHyve? Thanks. -- ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Remote serial console for BHyve VM?
Hi, I am working with Sean Bruno to set up some BHyve VM's. Right now, the vmrun.sh script tells BHyve to direct the serial console to stdout. If I want to have the VM's start via an rc.d script, and have the VM be headless, what is the best way to have the VM console still be accessible? Has anyone done something like use netcat to direct the BHyve stdio to a socket? That way the serial console could be accessed remotely via telnet or ssh. Is there a better way to do this? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Remote serial console for BHyve VM?
On Mon, Jan 6, 2014 at 3:12 PM, Peter Grehan wrote: > One way is to use the cloneable null-modem driver, nmdm(4). Others have > used tmux for this, but I'll give a quick overview of the former. > > kldload nmdm.ko before starting VMs, e.g. at boottime or in rc.conf's > kld_list variable. > > Use the '-c' option for bhyveload to point it at one end of an nmdm > instance, and use it in place of "stdio" in the bhyve commandline. > > bhyveload -c /dev/nmdm0A ... > bhyve ... -l com1,/dev/nmdm0A ... > > You can then attach to the other end (nmdm*B) with any tool of your > choice - cu, screen, or socat to relay the device to a network connection. > Some experimentation may be required :) > > later, > I did the following: (1) Put the following in /boot/loader.conf to load the nmdm driver on bootup: nmdm_load="YES" (2) I wrote a script with this: #!/bin/sh touch /dev/nmdm0A /usr/sbin/bhyvectl --vm=vm1 --destroy /usr/sbin/bhyveload -m 8G -d disk.img -c /dev/nmdm0A vm1 /usr/sbin/bhyve -c 2 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,disk.img -l com1,/dev/nmdm0A vm1 When I ran the script, there was no output, which was a good sign. I then ran this command: cu -l /dev/nmdm0A and had full access to the console of the VM. Thanks!! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
rc.d scripts for starting BHyve from rc.d
Hi,
I put together some scripts for starting a BHyve VM
when a system boots. When the system boots, the console
of the VM uses /dev/nmdm. It is possible to connect
to the console of the VM with: cu -l /dev/nmdm0B
It is a bit rough, but works nicely.
Does someone out there have better scripts for doing this?
Can some of the existing rc.d scripts for jails be reused for BHyve?
I did the following:
== put following entries in /etc/sysctl.conf ==
# BHyve needs this for tap interfaces
net.link.tap.user_open=1
net.link.tap.up_on_open=1
=
== put following in /etc/rc.conf ==
#
# Create tap devices, one tap interface per BHyve VM.
# Add the tap interfaces to bridge0
cloned_interfaces="bridge0 tap0"
autobridge_interfaces="bridge0"
# change igb0 to whatever NIC you are using
autobridge_bridge0="tap* igb0"
== put following file in /usr/local/etc/rc.d/bhyvevm ==
#!/bin/sh
#
# $FreeBSD$
#
# PROVIDE: bhyvevm
# REQUIRE: netif bgfsck sshd LOGIN localpkg
# KEYWORD: shutdown
#
. /etc/rc.subr
name="bhyvevm"
rcvar="bhyvevm_enable"
start_cmd="bhyvevm_start"
stop_cmd="bhyvevm_stop"
# read configuration and set defaults
load_rc_config "$name"
: ${bhyvevm_enable="YES"}
bhyvevm_start()
{
/vm/10.0/start_vm.sh
}
bhyvevm_stop()
{
/vm/10.0/stop_vm.sh vm1
}
run_rc_command "$1"
== put following script in /vm/10.0/start_vm.sh ==
#!/bin/sh
VM=vm1
CONS_A=/dev/nmdm0A
CONS_B=${CONS_A%%A}B
IMG=/vm/10.0/disk.img
TAP=tap0
BRIDGE=bridge0
touch ${CONS_A}
if [ -e /dev/vmm/${VM} ]; then
/usr/sbin/bhyvectl --vm=${VM} --destroy
fi
echo "Starting BHyve virtual machine named '${VM}'. Use 'cu -l ${CONS_B}'
to access console"
cmd="/usr/sbin/bhyveload -m 8G -d ${IMG} -c ${CONS_A} ${VM}"
$cmd
ret=$?
if [ $ret -ne 0 ]; then
echo "[FAILED]: $cmd"
exit $ret
fi
ifconfig bridge0 up
cmd="/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}"
$cmd &
ifconfig bridge0 up
sleep 5
echo "~." | cu -l ${CONS_B}
=
put following script in /vm/10.0/stop_vm.sh ===
#!/bin/sh
usage()
{
echo $0 "[vm name]"
}
if [ $# -lt 1 ]; then
usage
exit 1
fi
VM=$1
echo "Stopping BHyve virtual machine named '$VM'"
PID=`pgrep bhyve`
if [ -n "$PID" ]; then
kill $PID
fi
COUNT=0
while [ $COUNT -lt 20 -a -n "$PID" ] ; do
PID2=`pgrep bhyve`
if [ "$PID" != "$PID2" ]; then
break
fi
sleep 5
done
if [ -e /dev/vmm/${VM} ]; then
/usr/sbin/bhyvectl --vm=${VM} --destroy
fi
exit 0
=
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
MAC addresses to use for BHyve VM's running under FreeBSD?
Hi, I am running many BHyve VM's and am using tap interfaces with a single bridge. I am configuring the IP addresses of these VM's via DHCP. I need to have separate MAC addresses for each VM. Can anyone recommend a range of MAC addresses to use? I seem to recall that at the 2013 FreeBSD Vendor Summit in Sunnyvale, California, that George mentioned that there might be a Organizational Unique Identifier (OUI) for the FreeBSD project that we can use for BHyve VM's. Is that right? If not, can people recommend a range of addresses to use? VMWare for example has an OUI of 00:50:56 ( http://pubs.vmware.com/vsphere-4-esxi-installable-vcenter/index.jsp?topic=/com.vmware.vsphere.esxi_server_config.doc_41/esx_server_config/advanced_networking/c_setting_up_mac_addresses.html) I'd rather not use that range if I don't have to, since I have a VMWare ESXi server running in the same network. :) -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve and network virtualization
On Sat, Feb 1, 2014 at 1:03 AM, Frédéric Alix wrote: > Now, i need a tool for build a virtual network, like crossbow in Illumos. > Well, it is not a graphical tool, but if you are OK with doing things from the command-line, you can create a bridge network device and create tap interfaces for each BHyve VM. Then inside your BHyve VM, you can use the vtnet driver (part of virtio) to access the network. Michael Dexter has written a nice document which explains how to do this and a lot more with BHyve: http://bhyve.org/bhyve-manual.pdf -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: MAC addresses to use for BHyve VM's running under FreeBSD?
On Wed, Feb 5, 2014 at 9:46 AM, Kurt Lidl wrote: > On Feb 5, 2014, at 3:33 , Kai Gallasch wrote: >> >> Am 05.02.2014 um 08:03 schrieb Craig Rodrigues: >>> >>>> Hi, >>>> >>>> I am running many BHyve VM's and am using tap interfaces >>>> with a single bridge. I am configuring the IP addresses >>>> of these VM's via DHCP. >>>> >>>> I need to have separate MAC addresses for each VM. >>>> >>>> Can anyone recommend a range of MAC addresses to use? >>>> >>>> I seem to recall that at the 2013 FreeBSD Vendor Summit in >>>> Sunnyvale, California, that George mentioned that >>>> there might be a Organizational Unique Identifier (OUI) for the FreeBSD >>>> project that we can use for BHyve VM's. Is that right? >>>> >>>> If not, can people recommend a range of addresses to use? >>>> >>> >>> http://standards.ieee.org/develop/regauth/oui/public.html >>> >>> Using "Search the Public MA-L Listing" with search term FreeBSD reveals.. >>> >>> --- snip --- >>> >>> Here are the results of your search through the public section of the >>> IEEE Standards OUI database report for freebsd: >>> >>> 58-9C-FC (hex) FreeBSD >>> Foundation >>> 589CFC (base 16) >>> FreeBSD >>> Foundation >>> P.O. Box 20247 >>> Boulder CO 80308-3247 >>> UNITED STATES >>> --- snap --- >>> >>> >>> >> Correct, that is an address that the Foundation has registered with the >> IEEE. >> >> If you look at sys/net/ieee_oui.h you will see that I've allocated a >> range to bhyve already. >> > > At work, we modified the bhyverun command to seed the hostname > of them machine running the hypervisor as part of the "generate a MAC > address" routine. That means that for virtual machine "foo", > you now get different MACs on server "bar" and server "baz". > Without this patch, you're likely to get identical MAC addresses > for virtual machine "foo" on different servers. > > I personally also have my virtual machines set bit 2 in > the first octet of the MAC address, so it falls into the > "locally administered" catagory of MAC addresses. My gut feel > is that using the FreeBSD OUI bhyve range, *AND* setting the > locally administered bit in the MAC address is the way to go. > b > George, Thanks for allocating that range of MAC addresses. We shoud probably document that MAC address range in one of the BHyve man pages. Kurt, Your change is definitely useful. It changes the behavior of BHyve with respect to MAC addresses, but it is a very useful change. Have you submitted your change to Peter and Neel to see if they can evaluate if it can be made part of BHyve in the FreeBSD src tree? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Detect of BHyve VM was powered off or rebooted?
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via "reboot" or "shutdown -r",
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}"
done
) &
then this might cover cases (1) and (3), but what will cover
case (2)?
Thanks for any advice.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Detect of BHyve VM was powered off or rebooted?
On Sun, Feb 9, 2014 at 1:32 PM, Aryeh Friedman wrote:
>
>
>
> On Sun, Feb 9, 2014 at 4:11 PM, Craig Rodrigues wrote:
>
>> Hi,
>>
>> I posted some rc.d scripts that I am using to boot a BHyve VM
>> and send the output to a serial console using the /dev/nmdm
>> driver:
>>
>>
>> http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
>>
>> It works quite well. There is some things I would like to improve,
>> and would like some advice on the best way to do it.
>>
>> (1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
>> then I do not want to automatically restart the VM in the script.
>> User should manually: service bhyvevm start
>>
>> (2) If the VM was powered down, via shutdown -p, or halt -p,
>>then in my script I do not want to restart the VM in the script.
>>User should manually: service bhyvevm start
>>
>> (3) If the VM was rebooted via "reboot" or "shutdown -r",
>>then I *do* want the script to restart the VM.
>>
>> I think if I change my start_vm.sh script to do something like:
>>
>>
>>
>> (
>> while [ -e /dev/vmm/${VM} ]; do
>> /usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
>> -s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}"
>> done
>>
>> ) &
>>
>>
>> then this might cover cases (1) and (3), but what will cover
>> case (2)?
>>
>> Thanks for any advice.
>>
>
> The options you gave (which are really the only ones available) do not
> distinguish between the reason for the termination of the instance of bhyve
> pointing to /dev/vmm/XXX (it just does a normal termination).
>
Did you play with any of these flags to bhyvectl?
[--get-vmcs-exit-ctls]
[--get-vmcs-exit-reason]
[--get-vmcs-exit-qualification]
[--get-vmcs-exit-interruption-info]
If I do:
bhyvectl --get-vmcs-exit-reason --vm vm1
I get:
vmcs_exit_reason[0] 0x001e
If I look at:
http://svnweb.freebsd.org/base/head/sys/amd64/vmm/intel/vmcs.h?view=markup#l310
we have:
#define EXIT_REASON_INOUT 30
Linux has this stuff also:
http://lxr.free-electrons.com/source/arch/x86/include/asm/vmx.h?v=3.1#L250
so I guess these values are defined in some Intel manual.
> In my playing with this for almost a month the only solution seems to be
> put a trigger on instance shutdown via whatever method to leave a "cookie"
> file behind on the host (note I have not nor plan to do this in any of my
> own work).
>
Yes, I can see why this might need to be done. I might need to do that
myself. However, if bhyvectl can be used to provide adequate status
information as to why the VM exited, I might be able to avoid this.
It would be nicer though if /usr/sbin/bhyve returned
the EXIT_REASON in its status code.
>
> An other question is it wise to auto-reboot regardless of reason of the
> hypervisor termination?
>
I think the answer is "it depends". If someone
does "reboot" or "shutdown -r", I think it is reasonable to restart the
VM. If someone does "halt -p" or "shutdown -p", I think it is reasonable
to not restart the VM. For any other error condition, I think it is
reasonable to not restart the VM.
I think that there is no single answer for all users and all applications.
In my case, I am running BHyve VM's as "headless" which
I access via the /dev/nmdm driver. If the VM reboots normally, I want it
to restart.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: RFC: hyperv disk i/o performance vs. data integrity
On Sat, Feb 1, 2014 at 9:24 PM, Aryeh Friedman wrote: > > Craig Rodrigues and the FreeNAS team have done a fantastic job already (I > have not personally tried FreeNAS yet but I have heard nothing but good > things about it) and making it so it can run on a bhyve instance. Thanks for those kind words. At the 2013 FreeBSD Vendor Summit, I gave this presentation: http://www.ixsystems.com/whats-new/jenkins-bhyve-and-webdriver-continuous-integration-testing-on-freenas/ which shows how I am using BHyve to test FreeNAS. BHyve is rough around the edges, but it works really well for testing. BHyve is improving every day. I encourage anyone in the BHyve community to download a FreeNAS 9.2.1 ISO and test booting it inside a BHyve VM. See this link for download details: http://forums.freenas.org/threads/announcing-freenas-9-2-1-release.18278/ Booting the FreeNAS ISO inside BHyve is a great way to test out BHyve (and FreeNAS). If you find FreeNAS-specific problems you can report them at: http://www.freenas.org/support/ That web page has links to our bugtracker and online forums. That way there is no need to clutter this mailing list with FreeNAS-specific stuff. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Detect of BHyve VM was powered off or rebooted?
On Tue, Feb 11, 2014 at 11:11 AM, John Baldwin wrote:
> On Sunday, February 09, 2014 7:03:41 pm Neel Natu wrote:
> > Hi Craig,
> >
> > On Sun, Feb 9, 2014 at 1:11 PM, Craig Rodrigues
> wrote:
> > > Hi,
> > >
> > > I posted some rc.d scripts that I am using to boot a BHyve VM
> > > and send the output to a serial console using the /dev/nmdm
> > > driver:
> > >
> > > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-
> January/002040.html
> > >
> > > It works quite well. There is some things I would like to improve,
> > > and would like some advice on the best way to do it.
> > >
> > > (1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
> > > then I do not want to automatically restart the VM in the script.
> > > User should manually: service bhyvevm start
> > >
> > > (2) If the VM was powered down, via shutdown -p, or halt -p,
> > >then in my script I do not want to restart the VM in the script.
> > >User should manually: service bhyvevm start
> > >
> > > (3) If the VM was rebooted via "reboot" or "shutdown -r",
> > >then I *do* want the script to restart the VM.
> > >
> > > I think if I change my start_vm.sh script to do something like:
> > >
> > >
> > >
> > > (
> > > while [ -e /dev/vmm/${VM} ]; do
> > > /usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s
> 1:0,lpc
> > > -s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A}
> ${VM}"
> > > done
> > >
> > > ) &
> > >
> > >
> > > then this might cover cases (1) and (3), but what will cover
> > > case (2)?
> >
> > The exit code of the bhyve process will be 0 if it exited because the
> > guest rebooted and will be non-zero if the guest did an acpi poweroff.
> > You can use that to distinguish between cases (2) and (3).
> >
> > Having said that there are error conditions for which bhyve exits with
> > a non-zero exit code. So, we'll need to explicitly define an exit code
> > to distinguish between an acpi poweroff and these error conditions.
>
> OTOH, in all the cases when bhyve exits with a non-zero exit code, you
> will want to exit the loop which would treat it the same as shutdown -p. I
> think you can just do this:
>
> while [ -e /dev/vmm/${VM} ]; do
> if ! bhyve ...; then
> break
> fi
> done
>
One question, if "bhyve" exits, do I have to call bhyveload again before
calling bhyve?
The /usr/share/examples/bhyve/vmrun.sh has a loop which does:
while [ 1 ] ; do
...
bhyvectl --destroy
bhyveload
bhyve
...
done
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Detect of BHyve VM was powered off or rebooted?
On Tue, Feb 11, 2014 at 3:18 PM, Neel Natu wrote: > > > > > > > > > One question, if "bhyve" exits, do I have to call bhyveload again before > > calling bhyve? > > Yes, that is correct. You will also need to destroy the VM after bhyve > exits. > Why can't bhyve clean up after itself as the bhyve process exits and destroy the VM? It seems pointless to me for the user to have to explictly destroy the VM. Once the bhyve process exits, I doesn't seem that I can use the VM again anyways. To run the VM again, I seem to need to call: -> bhyvectl to destroy the VM -> bhyveload to load -> bhyve to run the VM Aryeh has pointed out to me that it is the responsibility of the user to explictly destroy and tap and bridge devices used by the bhyve process after it exits. Is this right? What if I have a single bridge0 device, and 10 tap devices, one per VM? If one VM exits and needs to be restarted, do I need to tear down all the taps and bridge? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve and booting a ZFS-on-root system
On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen wrote: > Hi, > > Just for the fun of it, I tried my build zfs-system scripts in a bhyve-vm. > > I use the 10.0-RELEASE iso to get to a shell, config and interface and > download my script. Installing does work, and on a regular system we can > go and boot into a ZFS-on-Root system. > ZFS on Root inside a BHyve VM was not working until today. If you you have a FreeBSD-CURRENT system and then update to this revision: http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html then you can try it out. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve and booting a ZFS-on-root system
On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen wrote: > > CPU: AMD Phenom(tm) II X6 1075T Processor (3013.84-MHz K8-class CPU) > Origin = "AuthenticAMD" Id = 0x100fa0 Family = 0x10 Model = 0xa > Also, as Aryeh has pointed out, you cannot run BHyve on an AMD processor unless you check out the code from a special branch. Michael Dexter has provided a snapshot from the branch: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-February/002227.html but that snapshot doesn't have the ZFS on Root change. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve and booting a ZFS-on-root system
On Feb 23, 2014 2:59 AM, "Willem Jan Withagen" wrote: > > > > Op 22 feb. 2014 om 22:28 heeft Craig Rodrigues het volgende geschreven: > >> On Sat, Feb 22, 2014 at 12:16 PM, Willem Jan Withagen wrote: >>> >>> Hi, >>> >>> Just for the fun of it, I tried my build zfs-system scripts in a bhyve-vm. >>> >>> I use the 10.0-RELEASE iso to get to a shell, config and interface and >>> download my script. Installing does work, and on a regular system we can >>> go and boot into a ZFS-on-Root system. >> >> >> ZFS on Root inside a BHyve VM was not working until >> today. If you you have a FreeBSD-CURRENT system and then update >> to this revision: >> http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html >> >> then you can try it out. > > > I'm running the amdsrc tree, so just upgrading current will probably not work. > Peter suggested that he would upgrade the amdsrc-tree shortly. > so I'll wait just for that. > > I'm also looking into running a Ubuntu vm. i want to test/use zoneminder, and all my FreeBSD attempts run into trouble. > So I'm also still looking for grub-bhyve. grub2-bhyve is in ports: http://www.freshports.org/sysutils/grub2-bhyve/ If you do a web search, you'll find several HOWTO articles for using it. Recently, Rudy wrote a nice one: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-February/002305.html -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve and booting a ZFS-on-root system
On Feb 23, 2014 3:35 PM, "Willem Jan Withagen" wrote: > > On 22-2-2014 22:28, Craig Rodrigues wrote: > > > > ZFS on Root inside a BHyve VM was not working until > > today. If you you have a FreeBSD-CURRENT system and then update > > to this revision: > > http://lists.freebsd.org/pipermail/svn-src-all/2014-February/081210.html > > Good pointer. > > Added this patch to my tree, recompiled buildworld, and installed. > And I'm up and running a Root on ZFS vm > > So that works too. > That's good news! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
March 13: Jenkins and BHyve presentation
Hi, I will be giving a talk discussing what I have done settings up Jenkins ( https://jenkins.freebsd.org ) in the FreeBSD cluster, using BHyve VM's: The presentation will be on March 13 in Mountain View, California, U.S.A.: http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ If you think you can come, please RSVP on the Meetup site, so that the organizers can get a big enough room. I am interested in collaborating with FreeBSD hackers who can contribute to the effort and help expand testing of FreeBSD!! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: March 13: Jenkins and BHyve presentation
On Mon, Feb 24, 2014 at 1:04 AM, Craig Rodrigues wrote: > Hi, > > I will be giving a talk discussing what I have done > settings up Jenkins ( https://jenkins.freebsd.org ) > in the FreeBSD cluster, using BHyve VM's: > > The presentation will be on March 13 in Mountain View, California, U.S.A.: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ > > If you think you can come, please RSVP on the Meetup site, so that > the organizers can get a big enough room. > > I am interested in collaborating with FreeBSD hackers who can > contribute to the effort and help expand testing of FreeBSD!! > Hi, For those who cannot attend the meetup in Mountain View, California, U.S.A., I will post a link to the slides after the presentation. I am not sure if video/webcast will be available. I have given a similar presentation at the 2013 FreeBSD Vendor Summit describing the use of Jenkins in the FreeNAS project, so you can look at this to get an idea: http://www.ixsystems.com/whats-new/jenkins-bhyve-and-webdriver-continuous-integration-testing-on-freenas/ However, my presentation on March 13 will be focused on how I set up https://jenkins.freebsd.org If you cannot make the March 13 meetup, I will be giving a similar presentation on May 17 at BSDCan: http://www.bsdcan.org/2014/schedule/events/445.en.html and on May 15 there will be a working group where we will discuss how to expand Continuous Testing in the FreeBSD project: https://wiki.freebsd.org/201405DevSummit Hope to see folks at one of these events, so we can do some interesting collaboration! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Problem with ZFS userboot changes r262331 and VM with mixed partitions
Hi, I encountered problems with the new changes to userboot to boot ZFS. About a month ago, I created a FreeBSD 10 VM with the following layout: => 34 419430333 md0 GPT (200G) 341281 freebsd-boot (64K) 16220971522 freebsd-ufs (1.0G) 209731483886083 freebsd-swap (4.0G) 10485922 408944 freebsd-zfs (195G) 419430366 1 - free - (512B) The FreeBSD root file system is on p4, which is ZFS. On the root file system, I have a symlink: /boot -> /bootdir /bootdir mounts p2 which is UFS. So in this scenario, the kernel lives on UFS. When I did this about a month ago, during the BHyve boot process, (1) userboot loaded the kernel from p2 (UFS) (2) The kernel booted, loaded zfs.ko, and then proceeded to mount p4 where the root file system lived (ZFS) After r262331, when I try to boot the BHyve VM (1) I can see p4 from the loader prompt (2) The kernel doesn't load (3) If I try to load /boot/kernel/kernel from the loader prompt, I get: ZFS: i/o error - all block copies unavailable I don't fully understand all this logic. Is there a bug that can be fixed here? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions
On Tue, Mar 4, 2014 at 2:24 PM, Peter Grehan wrote: > Hi Craig, > > >> ZFS: i/o error - all block copies unavailable >> >> I don't fully understand all this logic. Is there a bug that can be fixed >> here? > > > Yes - a flag needs to be passed to userboot to tell it to ignore ZFS > filesystems. This is analogous to the decision to use boot2 (implicit UFS) > vs zfsboot (explicit ZFS) as the 2nd-stage loader on a real disk. > > It didn't look possible to do this without changing the userboot ABI, but, > it's probably time to bump that. Thanks for the explanation. Can this flag be passed via bhyveload, and if so, how? Right now the workaround was to recompile userboot with: cd /usr/src/sys/boot/userboot make clean make WITHOUT_ZFS=yes make WITHOUT_ZFS=yes install -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve + OpenStack
On Sat, Mar 8, 2014 at 6:42 PM, Marc Fournier wrote: > > Has anyone started lookign into this? FreeBSD + bhyve would, Semihalf is doing work Openstack in FreeBSD. See: http://www.freebsd.org/news/status/report-2013-10-2013-12.html#FreeBSD-Host-Support-for-OpenStack-and-OpenContrail Roman Bogorodskiy is working on libvirt and FreeBSD. However, his initial focus has been on QEMU, not bhyve See: http://empt1e.blogspot.hu/2013/11/running-qemu-vms-on-freebsd.html Yes, a lot of people would like to see Openstack + FreeBSD + BHyve. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: March 13: Jenkins and BHyve presentation
On March 13, 2014, I gave a talk at the Bay Area FreeBSD User Group on the current state of Jenkins and BHyve work in the FreeBSD cluster. The slides are now online: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Problem with ZFS userboot changes r262331 and VM with mixed partitions
On Wed, Mar 5, 2014 at 7:22 AM, Kurt Lidl wrote: > > > I ran into this same problem a couple of weeks ago when I first played > with the ZFS boot support in userboot.so. > > The following set of operations on the hypervisor can be used to add > bootable support to your zfs installation. > > (I don't know what the name of your zpool is on the emulated host. > If the name of that zpool is the same as the zpool on your hypervisor > host, you might have to go at this a different way.) I have taken > to naming the zpool for the any clients to be the same as the name > of the virtual machine, so I can easily mount it on the host running > the hypervisor and fiddle with it. > > For the purposes of this example, my virtual machine is called "vm0". > The zpool for that host lives in a zpool called "zdata" on my > hypervisor machine. In my case, my /boot was a symlink that > pointed to /bootdir/boot, so you might need to adjust the > following slightly. > > zfs import -f -R /mnt vm0 > > fsck_ufs /dev/zvol/zdata/vm0p2 > mount /dev/zvol/zdata/vm0p2 /mnt/bootdir > cd /mnt > chflags -h nosunlink boot > rm boot > cp -R bootdir/* . > umount /mnt/bootdir > cd / > zpool export vm0 > > Good luck. Hi, Thanks for breaking down the steps so nicely! I followed similar steps and got my VM to boot with the latest userboot. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: March 13: Jenkins and BHyve presentation
On Mon, Feb 24, 2014 at 1:04 AM, Craig Rodrigues wrote: > The presentation will be on March 13 in Mountain View, California, U.S.A.: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/167325932/ Thanks to Annie Zhang and the rest of the iXsystems marketing department who recorded the video and did the editing, the video for this presentation is now online: Go to: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups and click on the link for the video. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Google Summer of Code 2014 question
On Thu, Mar 20, 2014 at 8:24 AM, John Baldwin wrote: > > This isn't a bad place to ask this question. I think a person you might > want to talk to is Craig Rodrigues (cc'd) who has recently setup Jenkins > with the goal of doing automated testing using virtual machines. I'm not > sure exactly what design he is considering, but he is the person to ask. Last year I signed up to be a GSoC mentor, but this year I unfortunately do not have time, so haven't done so. Unfortunately, since I did not sign up as a mentor, I can't view your proposal on the google-melange web site. If you look at the following link, you can see the March 13 presentation which describes some work I did setting up Jenkins and BHyve for continuous integration and testing on FreeBSD: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups If you look at this link, item 22: https://wiki.freebsd.org/Jenkins#Jenkins_for_FreeBSD_status you will see that we are trying to work at integrating Jenkins with the Kyua framework, so that Jenkins can run Kyua, and report the results in the Jenkins web UI. Jenkins is used by hundreds of companies, not just in the FreeBSD world. Kyua is gaining popularity in the FreeBSD community, and is already used by major companies (EMC/Isilon and Spectralogic are the ones that I know of). There are already many systems out there for doing automation, starting/stopping VM's, and running tests. I'm focusing on integrating as many existing technologies and getting them to run under FreeBSD to put together a more modern continuous integration and testing pipeline for the FreeBSD project. I cannot comment on the specifics your proposal without reading it. However, I don't see a lot of value in coming up with a new Domain Specific Language for starting/stopping VM's and sending commands to VM's. There are already many people doing this already with other frameworks and automation API's. For example, libvirt ( http://libvirt.org ) is one popular API specifically for controlling VM's. In terms of frameworks that can send remote commands to VM's, there are already plenty of automation systems to choose from: Puppet, Chef, SaltStack, CFEngine are some examples. Creating a new Domain Specific Language for VM control sounds like a more academic/research exercise, so if you can get someone to mentor you for such a project, maybe you can target the results of your work for publication, if that is of interest to you. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Google Summer of Code 2014 question
On Thu, Mar 20, 2014 at 2:38 PM, Wojciech A. Koszek wrote:
> DSL isn't a must here, I agree, but you'll be limited in testing otherwise.
> Unless some technology already has what I want, I suggested getting this
> functionality:
>
> vm0.cmd("ifconfig .");
> vm1.cmd("ifconfig ");
>
> vm0.cmd("ping vm1");
>
> vm0.register(function(data) {
> if (data =~ /*Not*/) {
> error();
> }
> });
For the above pseudo-code which you listed,
you've pretty much got that functionality in
existing automation tools like Puppet, SaltStack, Chef, etc.
For unit testing, Kyua is quite good, and people are writing lots of unit
tests for it, and even committing some of them directly to FreeBSD.
> Once this would work for VMs, I'd be interesting in proposing the creation
> of the physical infrastructure for continuous FreeBSD testing.
For the above item, we've already started putting that together. See:
http://wiki.freebsd.org/Jenkins and the presentation I gave on March 13.
At the next Devsummit in Ottawa, we want to talk about this further
and get people
who are interested in this topic to participate in the conversation:
https://wiki.freebsd.org/201405DevSummit/Jenkins
It would be interesting to get Raspberry PI hardware being tested
regularly as part of
continuous integration. We don't have Raspberry PI hardware available
in the FreeBSD
cluster, so someone who is interested would have to take the action item
to acquire the hardware and set things up so the hardware is integrated with
continuous integration/testing.
Looking at the SummerOfCode2014 ideas page, my opinion is that for this item:
https://wiki.freebsd.org/SummerOfCode2014#TEST-o-steron_for_FreeBSD_.28node.js.2FLua.29
-> has already been done with things like libvirt, which is being
ported to FreeBSD
by Roman Bogorodskiy. See:
http://empt1e.blogspot.com/2014/03/bhyve-in-libvirt.html
-> existing automation frameworks can already be used to send
remote commands to VM's
and for this item:
https://wiki.freebsd.org/SummerOfCode2014#CON-tinuous_INtegration_for_FreeBSD_aka_improved_TinderBox_.28node.js.2FLua.29
-> better Tinderbox systems already exist. Jenkins and Buildbot
are two commonly
available open source implementations.
-> the jenkins-admin team set up Jenkins (
http://jenkins.freebsd.org ) because we were familiar
it, and so far it seems to work well
My focus is to get existing software that people in the non-FreeBSD world have
developed to solve these types of problems, and get them up and
running in FreeBSD.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Scripts for booting FreeBSD images from the install ISO for use in Jenkins?
On Tue, Mar 18, 2014 at 2:26 AM, Rainer Duffner wrote: > Am Mon, 17 Mar 2014 19:30:01 -0700 > schrieb Craig Rodrigues : > >> Hi, >> >> For the BSD DevSummit in May, one of the items >> on our agenda: >> >> https://wiki.freebsd.org/201405DevSummit/Jenkins >> >> is to talk about writing scripts which can take a FreeBSD ISO image, >> and then boot it and run it on a remote system or in a VM >> to install the OS. After the OS is up, we would like to run tests. >> All of this would be triggered from Jenkins. >> >> Does anyone have scripts which can do this? >> Can they be contributed to the Jenkins effort on FreeBSD? >> >> If you have scripts in Python, Ruby, Bourne shell, etc. are all fine, >> or even recipes in automation frameworks like Puppet, Ansible, Chef, >> SaltStack, etc., >> please let us know! :) > > > > I would have loved to attend this talk: > > > http://2014.asiabsdcon.org/timetable.html.en#P7A > > > Hopefully, more documentation and/or the slides/the video for this talk > will become available. Rainer, Thanks for posting that link! It is highly relevant to my original posting. That looks like a really good presentation, and I also wish I could have attended the talk! There seem to be many automation frameworks for provisioning and booting VM's and real machines. I just need to learn one of them. :) -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve support in Libvirt
On Mon, Mar 31, 2014 at 10:00 AM, Roman Bogorodskiy wrote: > > http://empt1e.blogspot.ru/2014/03/bhyve-in-libvirt.html > This is a huge step forward for bhyve! Thank you! There is a sizable ecosystem of software for managing hypervisors built on top of libvirt. Have you tested any of this software, such as virsh? Are there any GUI or web UI utilities built on top of libvirt which work with bhyve? You might want to consider submitting a status report at: https://www.freebsd.org/news/status/status.html -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: virt-manager bhyve
On Mon, May 5, 2014 at 2:47 PM, Alexandar Narayan wrote: > Has anyone built virt-manager with bhyve support? I haven't tried any of this stuff, but you may want to read the blog postings mentioned at this link: http://www.freebsd.org/news/status/report-2014-01-2014-03.html#libvirt/bhyve-Support Those blog postings were written by Roman Bogorodskiy. Roman has done a lot of work to get libvirt working with bhyve, but there seems to be some additional setup steps required. These steps are mentioned in his blog postings. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Enabling bhyve support in devel/libvirt port
Hi, For anyone interested in playing with libvirt and bhyve, I submitted these two patches to the devel/libvirt port: -> "devel/libvirt: enable BHYVE by default on FreeBSD 10" https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190767 -> "devel/libvirt: configure checks for bhyve are broken " https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190783 You need to manually rebuild the libvirt port. The prebuilt libvirt port available via "pkg install libvirt" has bhyve support disabled by default. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
libvirt and bhyve problems
Roman, I have been reading your blog posts on using libvirt + bhyve here: http://empt1e.blogspot.com/ and the libvirt bhyve documentation here: http://libvirt.org/drvbhyve.html but am having problems reproducing the steps. I have a machine running: FreeBSD 11.0-CURRENT FreeBSD 11.0-CURRENT #6 r266934M: Sat May 31 20:24:37 PDT 2014 r...@crodrigues.org:/usr/obj/opt2/branches/head/sys/GENERIC amd64 I applied the following patches to the devel/libvirt port to fix compilation problems for bhyve support: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002586.html I then did the following: BUILD AND CONFIGURE === cd /usr/ports/devel/libport make config (I made sure bhyve and QEMU support were enabled make make install LIBVIRT CONFIG == I created the following /tmp/bhyve.xml file: bhyve df3be7e7-a104-11e3-aeb0-50e5492bd3dc 219136 219136 1 hvm RUNNING I did the following as root: service libvirtd restart virsh -c 'bhyve:///system' Then I ran these commands from the virsh prompt: virsh # define /tmp/bhyve.xml Domain bhyve defined from /tmp/bhyve.xml virsh # start bhyve error: Failed to start domain bhyve error: End of file while reading data: Input/output error error: Failed to reconnect to the hypervisor At this point libvirtd process crashed with a /libvirtd.core process , but for some reason I could not get a valid stacktrace. Can you provide assistance to get this working? Ultimately, I want to get the Jenkins libvirt plugin ( https://wiki.jenkins-ci.org/display/JENKINS/Libvirt+Slaves+Plugin ) to work. Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy wrote: > Hi Craig, > > It's a little unfortunate that there's no stacktrace, though we can try > to see what's going on without it. > > Could you please edit libvirtd.conf (should be something like > /usr/local/etc/libvirtd.conf if installed from port) to add: > > log_level = 2 > > then run libvirtd by hand in foreground: OK, I added log_level = 2 to /usr/local/etc/libvirt/libvirtd.conf and ran libvirtd in the foreground: 2014-06-10 15:53:26.846+: 34485593088: info : libvirt version: 1.2.5 2014-06-10 15:53:26.846+: 34485593088: warning : virDriverLoadModule:63 : Module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not accessible 2014-06-10 15:53:27.023+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-10 15:53:27.023+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-10 15:53:27.034+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-10 15:53:27.034+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-10 15:53:27.130+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 15:53:53.668+: 34485598208: error : virNetDevGetMTU:404 : Cannot get interface MTU on 'virbr0': Device not configured Segmentation fault (core dumped) I got that coredump after doing: virsh -c 'bhyve:///system' virsh # define /tmp/bhyve.xml virsh # start bhyve ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Tue, Jun 10, 2014 at 8:58 AM, Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 4:43 AM, Roman Bogorodskiy wrote: >> Hi Craig, >> >> It's a little unfortunate that there's no stacktrace, though we can try >> to see what's going on without it. >> >> Could you please edit libvirtd.conf (should be something like >> /usr/local/etc/libvirtd.conf if installed from port) to add: >> >> log_level = 2 >> >> then run libvirtd by hand in foreground: > > OK, I added log_level = 2 to /usr/local/etc/libvirt/libvirtd.conf > and ran libvirtd in the foreground: > > 2014-06-10 15:53:26.846+: 34485593088: info : libvirt version: 1.2.5 > 2014-06-10 15:53:26.846+: 34485593088: warning : > virDriverLoadModule:63 : Module > /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so not > accessible I saw the above warning, so I tried patching the port with this: --- pkg-plist.orig 2014-06-10 10:36:22.084757789 -0700 +++ pkg-plist 2014-06-10 09:27:25.280043239 -0700 @@ -274,6 +274,8 @@ lib/libvirt/connection-driver/libvirt_driver_secret.la lib/libvirt/connection-driver/libvirt_driver_storage.so lib/libvirt/connection-driver/libvirt_driver_storage.la +lib/libvirt/connection-driver/libvirt_driver_uml.so +lib/libvirt/connection-driver/libvirt_driver_uml.la lib/libvirt/connection-driver/libvirt_driver_vbox.so lib/libvirt/connection-driver/libvirt_driver_vbox.la However, when I restarted libvirtd, I got different errors: 2014-06-10 17:38:50.269+: 34485593088: info : libvirt version: 1.2.5 2014-06-10 17:38:50.269+: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: Undefined symbol "inotify_add_watch" 2014-06-10 17:38:50.563+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-10 17:38:50.563+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-10 17:38:50.575+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-10 17:38:50.575+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-10 17:38:50.663+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 17:38:50.663+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Tue, Jun 10, 2014 at 10:54 AM, Roman Bogorodskiy wrote: > This warning is not critical. Actually, UML should not be built on > FreeBSD and I would need to disable it by default. For now I think the > proper fix is to configure with '--without-uml' flag. > > As for the segfault, I think I have a guess. > > Could you please drop '--without-network' from CONFIGURE_ARGS in the > port's Makefile and try to rebuild? > > Roman Bogorodskiy OK, I dropped --without-network from CONFIGURE_ARGS and rebuilt the port. I am still getting the coredump, but am getting different errors: 2014-06-10 18:59:22.713+: 34485593088: info : libvirt version: 1.2.5 2014-06-10 18:59:22.713+: 34485593088: warning : virDriverLoadModule:63 : Module /usr/local/lib/libvirt/connection-driver/libvirt_driver_network.so not accessible 2014-06-10 18:59:22.715+: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_qemu.so: Undefined symbol "networkAllocateActualDevice" 2014-06-10 18:59:22.716+: 34485593088: error : virDriverLoadModule:71 : failed to load module /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so /usr/local/lib/libvirt/connection-driver/libvirt_driver_uml.so: Undefined symbol "inotify_add_watch" 2014-06-10 18:59:22.845+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-10 18:59:22.845+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-10 18:59:22.847+: 34485605376: error : do_open:1166 : no connection driver available for qemu:///system 2014-06-10 18:59:42.273+: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet7: Invalid argument Segmentation fault (core dumped) ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Tue, Jun 10, 2014 at 12:15 PM, Roman Bogorodskiy wrote: > > This last line also sounds strange. > > I'm wondering if you execute 'net-list --all' will you see the 'default' > network? And if it's there, will 'net-show default' should that > Autostart is enabled for it? If I do: # virsh -c 'bhyve:///system' Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # net-list --all error: Failed to get the number of active networks error: this function is not supported by the connection driver: virConnectNumOfNetworks The past few times, I have been compiling the port using WITH_DEBUG=yes. However, I am not getting useful stacktraces: /usr/ports/devel/libvirt/work/libvirt-1.2.5]# file /usr/local/sbin/libvirtd /usr/local/sbin/libvirtd: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 11.0 (1100022), not stripped # gdb /usr/local/sbin/libvirtd libvirtd.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `libvirtd'. Program terminated with signal 11, Segmentation fault. #0 0x000801aa8513 in ?? () (gdb) where #0 0x000801aa8513 in ?? () #1 0x00180008 in ?? () #2 0x7f3f9330 in ?? () #3 0x000809e5510c in ?? () #4 0x0007 in ?? () #5 0x000809e5531b in ?? () #6 0x in ?? () BTW, can you give me: (1) exact git command to checkout libvirt (2) configure command you typed (3) make command you typed I want to get a working setup like you have. It looks like the port needs some fixing before it can be used fully. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues wrote: > > > # gdb /usr/local/sbin/libvirtd libvirtd.core > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > Core was generated by `libvirtd'. > Program terminated with signal 11, Segmentation fault. > #0 0x000801aa8513 in ?? () > (gdb) where > #0 0x000801aa8513 in ?? () > #1 0x00180008 in ?? () > #2 0x7f3f9330 in ?? () > #3 0x000809e5510c in ?? () > #4 0x0007 in ?? () > #5 0x000809e5531b in ?? () > #6 0x in ?? () OK, so it looks like in CURRENT, at least for the devel/libvirt port, the gdb in base, and also gdb in ports cannot debug libvirtd.core. I enabled WITH_LLDB=yes in /etc/src.conf and recompiled to install lldb. I am not so familiar with lldb, but at least I can get a stacktrace with it. lldb /usr/local/sbin/libvirtd libvirtd.core (lldb) bt * thread #1: tid = 0, 0x000801aa8513 libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x00080a4211c8, ifname=0x00080a88d1e0, macaddr=0x00080a88d144, vmuuid=0x00080a81b808, tapfd=0x, tapfdSize=1, virtPortProfile=0x, virtVlan=0x, flags=9) + 547 at virnetdevtap.c:583, name = 'libvirtd', stop reason = signal SIGSEGV * frame #0: 0x000801aa8513 libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x00080a4211c8, ifname=0x00080a88d1e0, macaddr=0x00080a88d144, vmuuid=0x00080a81b808, tapfd=0x, tapfdSize=1, virtPortProfile=0x, virtVlan=0x, flags=9) + 547 at virnetdevtap.c:583 frame #1: 0x000809e20569 libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=0x00080a81b800, cmd=0x00080a4db200, dryRun=false) + 841 at bhyve_command.c:82 frame #2: 0x000809e20196 libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=0x00080a48a040, def=0x00080a81b800, dryRun=false) + 310 at bhyve_command.c:248 frame #3: 0x000809e24d6a libvirt_driver_bhyve.so`virBhyveProcessStart(conn=0x00080a4db100, driver=0x00080a48a040, vm=0x00080a48b0f0, reason=VIR_DOMAIN_RUNNING_BOOTED, flags=0) + 746 at bhyve_process.c:134 frame #4: 0x000809e2334a libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=0x00080a48a280, flags=0) + 490 at bhyve_driver.c:847 frame #5: 0x000809e2315a libvirt_driver_bhyve.so`bhyveDomainCreate(dom=0x00080a48a280) + 26 at bhyve_driver.c:867 frame #6: 0x000801b96aad libvirt.so.1002`virDomainCreate(domain=0x00080a48a280) + 637 at libvirt.c:8968 frame #7: 0x01075427 libvirtd`remoteDispatchDomainCreate(server=0x00080781c400, client=0x0008078df480, msg=0x0008078791d0, rerr=0x7f9fcc30, args=0x000807934340) + 231 at remote_dispatch.h:3048 frame #8: 0x0103b84e libvirtd`remoteDispatchDomainCreateHelper(server=0x00080781c400, client=0x0008078df480, msg=0x0008078791d0, rerr=0x7f9fcc30, args=0x000807934340, ret=0x00080a421058) + 254 at remote_dispatch.h:3026 frame #9: 0x000801c26b20 libvirt.so.1002`virNetServerProgramDispatchCall(prog=0x00080790f400, server=0x00080781c400, client=0x0008078df480, msg=0x0008078791d0) + 784 at virnetserverprogram.c:437 frame #10: 0x000801c26641 libvirt.so.1002`virNetServerProgramDispatch(prog=0x00080790f400, server=0x00080781c400, client=0x0008078df480, msg=0x0008078791d0) + 705 at virnetserverprogram.c:307 frame #11: 0x0108f97c libvirtd`virNetServerProcessMsg(srv=0x00080781c400, client=0x0008078df480, prog=0x00080790f400, msg=0x0008078791d0) + 332 at virnetserver.c:172 frame #12: 0x0108cd9c libvirtd`virNetServerHandleJob(jobOpaque=0x000807934360, opaque=0x00080781c400) + 204 at virnetserver.c:193 frame #13: 0x000801ac84b8 libvirt.so.1002`virThreadPoolWorker(opaque=0x000807901d60) + 680 at virthreadpool.c:145 frame #14: 0x000801ac7b9b libvirt.so.1002`virThreadHelper(data=0x00080781b520) + 75 at virthread.c:197 frame #15: 0x000803c614f5 libthr.so.3`??? + 277 -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Tue, Jun 10, 2014 at 1:54 PM, Craig Rodrigues wrote:
> On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues wrote:
> lldb /usr/local/sbin/libvirtd libvirtd.core
>
> (lldb) bt
> * thread #1: tid = 0, 0x000801aa8513
> libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x00080a4211c8,
> ifname=0x00080a88d1e0, macaddr=0x00080a88d144,
> vmuuid=0x00080a81b808, tapfd=0x, tapfdSize=1,
> virtPortProfile=0x, virtVlan=0x,
> flags=9) + 547 at virnetdevtap.c:583, name = 'libvirtd', stop reason =
> signal SIGSEGV
> * frame #0: 0x000801aa8513
> libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=0x00080a4211c8,
> ifname=0x00080a88d1e0, macaddr=0x00080a88d144,
> vmuuid=0x00080a81b808, tapfd=0x, tapfdSize=1,
> virtPortProfile=0x, virtVlan=0x,
> flags=9) + 547 at virnetdevtap.c:583
> frame #1: 0x000809e20569
> libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=0x00080a81b800,
> cmd=0x00080a4db200, dryRun=false) + 841 at bhyve_command.c:82
> frame #2: 0x000809e20196
> libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=0x00080a48a040,
> def=0x00080a81b800, dryRun=false) + 310 at bhyve_command.c:248
> frame #3: 0x000809e24d6a
> libvirt_driver_bhyve.so`virBhyveProcessStart(conn=0x00080a4db100,
> driver=0x00080a48a040, vm=0x00080a48b0f0,
> reason=VIR_DOMAIN_RUNNING_BOOTED, flags=0) + 746 at
> bhyve_process.c:134
> frame #4: 0x000809e2334a
> libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=0x00080a48a280,
> flags=0) + 490 at bhyve_driver.c:847
> frame #5: 0x000809e2315a
> libvirt_driver_bhyve.so`bhyveDomainCreate(dom=0x00080a48a280) + 26
> at bhyve_driver.c:867
> frame #6: 0x000801b96aad
> libvirt.so.1002`virDomainCreate(domain=0x00080a48a280) + 637 at
> libvirt.c:8968
> frame #7: 0x01075427
> libvirtd`remoteDispatchDomainCreate(server=0x00080781c400,
> client=0x0008078df480, msg=0x0008078791d0,
> rerr=0x7f9fcc30, args=0x000807934340) + 231 at
> remote_dispatch.h:3048
> frame #8: 0x0103b84e
> libvirtd`remoteDispatchDomainCreateHelper(server=0x00080781c400,
> client=0x0008078df480, msg=0x0008078791d0,
> rerr=0x7f9fcc30, args=0x000807934340,
> ret=0x00080a421058) + 254 at remote_dispatch.h:3026
> frame #9: 0x000801c26b20
> libvirt.so.1002`virNetServerProgramDispatchCall(prog=0x00080790f400,
> server=0x00080781c400, client=0x0008078df480,
> msg=0x0008078791d0) + 784 at virnetserverprogram.c:437
> frame #10: 0x000801c26641
> libvirt.so.1002`virNetServerProgramDispatch(prog=0x00080790f400,
> server=0x00080781c400, client=0x0008078df480,
> msg=0x0008078791d0) + 705 at virnetserverprogram.c:307
> frame #11: 0x0108f97c
> libvirtd`virNetServerProcessMsg(srv=0x00080781c400,
> client=0x0008078df480, prog=0x00080790f400,
> msg=0x0008078791d0) + 332 at virnetserver.c:172
> frame #12: 0x0108cd9c
> libvirtd`virNetServerHandleJob(jobOpaque=0x000807934360,
> opaque=0x00080781c400) + 204 at virnetserver.c:193
> frame #13: 0x000801ac84b8
> libvirt.so.1002`virThreadPoolWorker(opaque=0x000807901d60) + 680
> at virthreadpool.c:145
> frame #14: 0x000801ac7b9b
> libvirt.so.1002`virThreadHelper(data=0x00080781b520) + 75 at
> virthread.c:197
> frame #15: 0x000803c614f5 libthr.so.3`??? + 277
>
> --
> Craig
OK, I think I found the problem.
In this file:
http://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/bhyve/bhyve_command.c;hb=HEAD
We have:
static int
bhyveBuildNetArgStr(const virDomainDef *def, virCommandPtr cmd, bool dryRun)
{
virDomainNetDefPtr net = NULL;
char *brname = NULL;
char *realifname = NULL;
int *tapfd = NULL;
tapfd is never initialized after this.
In the same function, we have this:
if (!dryRun)
if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net->mac,
def->uuid, tapfd, 1,
virDomainNetGetActualVirtPortProfile(net),
virDomainNetGetActualVlan(net),
VIR_NETDEV_TAP_CREATE_IFUP | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) {
VIR_FREE(net->ifname);
VIR_FREE(brname);
return -1;
}
virNetDevTapCreateinBridgePort() is being passed a NULL tapfd, but the
next parameter is 1 instead of 0.
Then further along in this file:
http://libvirt.org/git/?p=libvirt.git;a=blob_plain;f=src/util/virnetdevtap.c;hb=HEAD
we have:
error:
for (i = 0; i < tapfdSize
Re: libvirt and bhyve problems
On Wed, Jun 11, 2014 at 11:21 AM, Roman Bogorodskiy wrote: > > Thanks for the analysis! I'll think what would be a proper fix for that > problem. > > Also, while thinking how to reproduce it, I did 'kldunload if_bridge' > and started to see the same problem. So, as a temporary fix, could you > check if you have bridge support available (in kernel or via module)? > > Bridges are needed to get networking anyway. Hi, My output of kldstat is: 1 29 0x8020 19d6218 kernel 21 0x81bd7000 2e44b0 zfs.ko 32 0x81ebc000 8208 opensolaris.ko 41 0x81ec5000 4d68 nmdm.ko 51 0x81eca000 1a2110 vmm.ko 61 0x82211000 53e7 fdescfs.ko 71 0x82217000 9aff if_bridge.ko 81 0x82221000 5851 bridgestp.ko 91 0x82227000 5673 if_tap.ko 101 0x8222d000 2a94 uhid.ko 111 0x8223 3592 ums.ko so I do have if_bridge.ko loaded. If you are doing development on CURRENT, make sure that you have WITH_LLDB=yes in /etc/src.conf when you rebuild the system. Looks like gdb in base cannot debug the cores. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Wed, Jun 11, 2014 at 12:25 PM, Roman Bogorodskiy wrote: > I've attached a patch which should fix the segfault. Could you please > let me know if it fixes the problem? I have incorporated your patch into the libvirt port, and provided a tarball of the port here: http://people.freebsd.org/~rodrigc/libvirt_port.tgz I can confirm that your patch eliminates the coredump for me. So now I can do the following: (1) Start libvirtd: /usr/local/sbin/libvirtd (2) Start virsh: virsh -c "bhyve:///system" (3) Execute following inside virsh: define bhyve.xml start bhyve ( using bhyve.xml file from http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html ) In the libvirtd console, I see this: 2014-06-11 22:47:16.357+: 34485605376: info : libvirt version: 1.2.5 2014-06-11 22:47:16.357+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-11 22:47:16.357+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-11 22:47:21.050+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-11 22:47:21.063+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-11 22:47:21.168+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-11 22:47:21.168+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-11 22:47:43.469+: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet18: Invalid argument 2014-06-11 22:47:43.573+: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 For the last line, it looks like it is trying to do bhyvectl --destroy --vm=bhyve on a VM which doesn't exist, i.e. there is no entry in /dev/vmm/bhyve It should ignore the error at this point and try to spawn the VM, but it fails. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: > > http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz > > With this setup, I'm able to get networking (e.g. virsh net-list works) > and updated the fix for the previous problem. I was able to start a VM > with that setup. I deleted the old port from my system and took your modified port, built it, and installed it. I followed my previous steps in: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html and got this: 2014-06-12 18:23:54.328+: 34485605376: info : libvirt version: 1.2.5 2014-06-12 18:23:54.328+: 34485605376: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory 2014-06-12 18:23:54.328+: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-12 18:23:54.328+: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-12 18:23:54.438+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-12 18:23:54.439+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-12 18:23:54.460+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-12 18:23:54.461+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-12 18:23:54.560+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-12 18:23:54.560+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-12 18:24:17.940+: 34485598208: error : virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: Invalid argument 2014-06-12 18:24:18.056+: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Thu, Jun 12, 2014 at 11:28 AM, Craig Rodrigues wrote: > On Thu, Jun 12, 2014 at 1:00 AM, Roman Bogorodskiy wrote: >> >> http://people.freebsd.org/~novel/misc/libvirt_port_updated.tgz >> >> With this setup, I'm able to get networking (e.g. virsh net-list works) >> and updated the fix for the previous problem. I was able to start a VM >> with that setup. > > > I deleted the old port from my system and took your modified port, > built it, and installed it. > > I followed my previous steps in: > http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002588.html > and got this: > > 2014-06-12 18:23:54.328+: 34485605376: info : libvirt version: 1.2.5 > 2014-06-12 18:23:54.328+: 34485605376: error : > dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: > No such file or directory > 2014-06-12 18:23:54.328+: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-12 18:23:54.328+: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-12 18:23:54.438+: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-12 18:23:54.439+: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-12 18:23:54.460+: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-12 18:23:54.461+: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-12 18:23:54.560+: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-12 18:23:54.560+: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' > 2014-06-12 18:24:17.940+: 34485598208: error : > virNetDevBridgeAddPort:399 : Unable to add bridge tap0 port vnet0: > Invalid argument > 2014-06-12 18:24:18.056+: 34485598208: error : virCommandWait:2426 > : internal error: Child process (/usr/sbin/bhyvectl --destroy > --vm=bhyve) unexpected exit status 255 > > > -- > Craig After the program fails, these are my tap and bridge devices, as shown by ifconfig: bridge0: flags=8843 metric 0 mtu 1500 ether 02:29:45:c7:8f:00 nd6 options=9 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 2 member: tap2 flags=143 ifmaxaddr 0 port 7 priority 128 path cost 200 member: tap1 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 200 member: tap0 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 200 tap0: flags=8902 metric 0 mtu 1500 options=8 ether 00:bd:53:27:00:00 nd6 options=29 media: Ethernet autoselect status: no carrier tap1: flags=8902 metric 0 mtu 1500 options=8 ether 00:bd:59:27:00:01 nd6 options=29 media: Ethernet autoselect status: no carrier tap2: flags=8943 metric 0 mtu 1500 options=8 ether 00:bd:5e:27:00:02 nd6 options=29 media: Ethernet autoselect status: active Opened by PID 1506 vnet0: flags=8802 metric 0 mtu 1500 options=8 ether fe:54:00:33:20:8c nd6 options=21 media: Ethernet autoselect status: no carrier This is in my /etc/rc.conf for creating bridge and tap devices on bootup: # # Create tap devices, one tap interface per BHyve VM. # Add the tap interfaces to bridge0 cloned_interfaces="bridge0 tap0 tap1 tap2" autobridge_interfaces="bridge0" autobridge_bridge0="tap* em0" -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Thu, Jun 12, 2014 at 12:00 PM, Roman Bogorodskiy wrote: > > Ah. Actually, libvirt manages bridges and taps on its own. So, the > preferred flow is that libvirt starts its networks on startup > automatically. By default it has a network with the 'virbr0' device > which it creates on its own. > > Later on it creates vnet devices (which are tap devices) automatically > as well on VM startup and plugs them to the bridge. > > The configuration you have probably confuses it, could you consider > trying without that? OK, I removed all autobridge and cloned_interfaces entries from my rc.conf and rebooted. I am still using your version of the libvirt port. I tried the steps again and got this: 2014-06-13 19:09:10.482+: 34485605376: info : libvirt version: 1.2.5 2014-06-13 19:09:10.482+: 34485605376: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory 2014-06-13 19:09:10.482+: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-13 19:09:10.482+: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-13 19:09:10.593+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-13 19:09:10.593+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-13 19:09:10.615+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-13 19:09:10.615+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-13 19:09:10.716+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-13 19:09:10.716+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' 2014-06-13 19:09:25.938+: 34485598208: error : virNetDevTapCreate:385 : Unable to create tap device: Invalid argument 2014-06-13 19:09:26.063+: 34485598208: error : virCommandWait:2426 : internal error: Child process (/usr/sbin/bhyvectl --destroy --vm=bhyve) unexpected exit status 255 If I do ifconfig, I have no bridge or tap devices. > > Also, do you have working 'net-list' command in virsh with my version of > port? I get this: virsh # net-list Name State Autostart Persistent -- > > PS If you're available for some real-time conversation such as on IRC, I > think some kind of things would be faster to discuss there. You can private mail me some times when you are available. I am busy the next few days, but might have a few hours on Sunday. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wrote: > > I have updated the port: > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz OK, I deinstall the old port, took your port, built it, and installed it. When I started libvirtd, I got this: 2014-06-15 18:47:07.236+: 34485605376: info : libvirt version: 1.2.5 2014-06-15 18:47:07.236+: 34485605376: info : dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic is present, SO_BINDTODEVICE is in use 2014-06-15 18:47:07.237+: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-15 18:47:07.237+: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-15 18:47:07.395+: 34485605376: error : virCommandWait:2426 : internal error: Child process (/usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf) unexpected exit status 2: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 2014-06-15 18:47:07.484+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-15 18:47:07.484+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-15 18:47:07.495+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-15 18:47:07.495+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-15 18:47:07.583+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-15 18:47:07.584+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' Trying to start bhyve did not work. After some investigation, I did the following: service local_unbound stop Then I started libvirtd again: 2014-06-15 18:47:07.236+: 34485605376: info : libvirt version: 1.2.5 2014-06-15 18:47:07.236+: 34485605376: info : dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic is present, SO_BINDTODEVICE is in use 2014-06-15 18:47:07.237+: 34485605376: info : networkReloadFirewallRules:1750 : Reloading iptables rules 2014-06-15 18:47:07.237+: 34485605376: info : networkRefreshDaemons:1722 : Refreshing network daemons 2014-06-15 18:47:07.395+: 34485605376: error : virCommandWait:2426 : internal error: Child process (/usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf) unexpected exit status 2: dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use 2014-06-15 18:47:07.484+: 34485605376: error : virExec:417 : Cannot find 'pm-is-supported' in path: No such file or directory 2014-06-15 18:47:07.484+: 34485605376: warning : virQEMUCapsInit:948 : Failed to get host power management capabilities 2014-06-15 18:47:07.495+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/var/run/libvirt/qemu 2014-06-15 18:47:07.495+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/qemu 2014-06-15 18:47:07.583+: 34485605376: info : virDomainObjListLoadAllConfigs:18249 : Scanning for configs in /usr/local/etc/libvirt/bhyve 2014-06-15 18:47:07.584+: 34485605376: info : virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' At this point, I could see the virbr0 device: virbr0: flags=8843 metric 0 mtu 1500 ether 02:b8:18:5f:0f:00 inet 192.168.122.1 netmask 0xff00 broadcast 192.168.122.255 nd6 options=1 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 4 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 I was then able to do: virsh -c 'bhyve:///system' start bhyve and start a VM. This is good progress! Once thing I observed is that if I killed the libvirtd process, it did not clean up the dnsmasq process that it started: /usr/local/sbin/dnsmasq --conf-file=/usr/local/var/lib/libvirt/dnsmasq/default.conf and a subsequent restart of libvirtd would fail. The devel/libvirt port is looking a lot better with your changes. Can you push these changes into the port? I filed bugs 190767 and 190783, but did not get any response from the port maintainer, jgh@. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt and bhyve problems
On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wrote: > > I have updated the port: > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz > > Changes are: > > * added RUN_DEPENDS on dnsmasq (required for networking) > * added installation on network definition files Hi, dnsmasq needs to be in BUILD_DEPENDS *and* RUN_DEPENDS for this port. I did a test on my system where I deleted the dnsmasq port, and then built the libvirt port. During the configure stage of this port, the #define DNSMASQ variable is expanded and written to a config.h. If the dnsmasq port is not around during the build stage of the libvirt port, then this variable is not expanded properly, and when the libvirt port is fully built, running libvirtd will not start dnsmasq, because the path to dnsmasq was not written properly to config.h. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Request for Testers: libvirt and bhyve
Hi, I've had some success with getting libvirt and bhyve to work. I wrote up a list of the steps that I followed here: http://people.freebsd.org/~rodrigc/libvirt-bhyve/libvirt-bhyve.html If anyone is interested, please try it out, and provide feedback to this list. It would be great to get more testers. Roman, when do you think you will be able to commit all your latest fixes into the devel/libvirt port Makefile? It would greatly increase the usability of the port. Thanks for doing all the work to add bhyve support to libvirt!! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Request for testers: virt-manager, libvirt, and bhyve
Hi, I created a patch for the deskutils/virt-manager port to add support for bhyve. virt-manager depends on libvirt, so the work that Roman has done with libvirt is critical for it to work. I have instructions for setting it up here, plus a screenshot of the virt-manager UI: http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html Is there anyone interested in testing this and providing feedback? Also, since virt-manager is in Python, is there anyone out there interested in improving this patch, and sending it upstream to the virt-manager developers? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Request for testers: virt-manager, libvirt, and bhyve
On Wed, Jun 18, 2014 at 10:44 PM, Roman Bogorodskiy wrote: > Craig Rodrigues wrote: > >> Hi, >> >> I created a patch for the deskutils/virt-manager port to add support >> for bhyve. virt-manager depends on libvirt, so the work >> that Roman has done with libvirt is critical for it to work. >> >> I have instructions for setting it up here, plus a screenshot of the >> virt-manager UI: >> >> http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html >> >> Is there anyone interested in testing this and providing feedback? >> >> Also, since virt-manager is in Python, is there anyone out there interested >> in improving this patch, and sending it upstream to the virt-manager >> developers? > > Hi Craig, > > Back in April bhyve support was added into virt-manager master [1]. So > it should be there in the next release. > > I've looked at updating the desktuils/virt-manager port to 1.0.x, but at > that time it was blocked by some outdated dependency. > > 1: > https://git.fedorahosted.org/cgit/virt-manager.git/commit/?id=05df5a64843f2bd4e9a5197d97608d41b2e6dc43 Hi, What ports dependency is blocking the update of the deskutils/virt-manager port to 1.0.x? We can get other ports developers such as Bartlomiej to help. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
libvirt, bhyve, and Jenkins configuration problems
Hi, I am trying to configure Jenkins to spawn bhyve VM's on demand using libvirt. I did the following: (1) Looked at Roman's blog article: http://empt1e.blogspot.com/2014/03/using-jenkins-libvirt-slave-plugin-with.html (2) Followed the libvirt configuration steps here: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html Here are the screenshots of my Jenkins setp for libvirt: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve-jenkins.html However, after going through all that, when I tried to configure a job to run on the libvirt node, it did not start. Any ideas what I am doing wrong? Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Request for Testers: libvirt and bhyve
On Fri, Jun 20, 2014 at 5:56 AM, Vincent Hoffman wrote: > Hi Craig, > I've given it a try and its looking good. Nice! Do you have time to try virt-manager: http://people.freebsd.org/~rodrigc/libvirt/virt-manager.html > > Thanks for all the work on this. Roman did all the hard work to get bhyve support into libvirt, and upstream his changes to the libvirt project. All the thanks go to him! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt, bhyve, and Jenkins configuration problems
On Sat, Jun 21, 2014 at 6:52 AM, Roman Bogorodskiy wrote: > > > When you configure the connection in the 'Add a new Cloud' dialog, does > 'test connection' work for you? I don't see a host and credentials > specified, I'm not sure if jenkins libvirt plugin works with the local > connection. > OK, following your example, I have gotten further along. I have updated my setup instructions here: http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve-jenkins.html In "Step 3", when I click on "Test Connection", I see OK: bhyve+ssh://root@localhost:22/system?no_tty=1 version=1100 so that seems to work. For Step 5, when I enter the Host and Credentials information, does this information need to be the IP address and login information for the VM that has been spawned, and not the info for libvirt, as specified in Step 3? Does that mean that I need to configure my VM, so that it comes up with a known IP, so that it can be entered on this configuration page? Also, the VM, that is spawned, it must have openjdk installed? Does that mean that after libvirt starts the VM, Jenkins will try to SSH into the VM and start a Jenkins job by invoking java? Thanks. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
py-libvirt problems with bhyve
Hi,
I did the following:
(1) Set up libvirt, taking Roman's latest port, and configuring bhyve
support:
http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html
(2) Installed the py-libvirt port.
I wrote this code:
import libvirt
import sys
conn = libvirt.openReadOnly(None)
if conn == None:
print 'Failed to open connection to the hypervisor'
sys.exit(1)
try:
print conn.listAllDomains()
#print dir(conn)
dom0 = conn.lookupByName("bhyve")
except:
print 'Failed to find the main domain'
sys.exit(1)
but when I ran the code, I got this error:
[]
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'bhyve'
Failed to find the main domain
Can anyone tell me how to write a py-libvirt program to
start/stop a bhyve VM?
Thanks.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: libvirt, bhyve, and Jenkins configuration problems
On Wed, Jun 25, 2014 at 11:08 AM, Roman Bogorodskiy wrote: > > Does that mean that I need to configure my VM, so that it comes up with a > > known IP, so that > > it can be entered on this configuration page? > > Correct. > > Though, normally you don't need to perform any special configuration for > the IP address as long as the networking configuration is correct. > > The VM just needs to be configured to obtain an IP address via DHCP. > Later on it should be persistent in dnsmasq. > > An IP address could be obtained either from the dnsmasq leases file or > from arp cache for example. > But how does the Jenkins libvirt plugin know the IP address for the VM, unless you enter it in this field in the Jenkins UI? > > > Also, the VM, that is spawned, it must have openjdk installed? Does that > > mean that > > after libvirt starts the VM, Jenkins will try to SSH into the VM and > start > > a Jenkins job > > by invoking java? > > Yes, it's true, it starts its agent which does the job. > > PS Maybe there are other ways to push Jobs to Jenkins nodes, but I > didn't look close into that. > The easiest way is to have the openjdk installed in the VM, so that Jenkins can ssh in and run jobs. However, there is also the Publish Over SSH Plugin ( https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+SSH+Plugin ) which allows Jenkins to SSH into a remote host and run different commands, instead of starting a full Jenkins job, which requires the openjdk to be available. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: py-libvirt problems with bhyve
On Wed, Jun 25, 2014 at 11:01 AM, Roman Bogorodskiy
wrote:
> Craig Rodrigues wrote:
>
> >
> > conn = libvirt.openReadOnly(None)
> ^^^
>
> I think here the URI should be 'bhyve:///system'.
> Or you could tweak libvirt.conf and assign the bhyve url value to the
> 'uri_default' parameter.
>
> http://libvirt.org/uri.html#URI_default
>
>
Hi
OK, I changed my steps a little bit.
I did the following:
(1) Set up libvirt, taking Roman's latest port, and configuring bhyve
support:
http://people.freebsd.org/~rodrigc/libvirt/libvirt-bhyve.html
(2) Installed the py-libvirt port.
I wrote this code:
import libvirt
import sys
# Note we need to specify the URL here
conn = libvirt.openReadOnly("bhyve:///system")
if conn == None:
print 'Failed to open connection to the hypervisor'
sys.exit(1)
try:
print("All domains: ",conn.listAllDomains())
print("Defined domains: ", conn.listDefinedDomains())
#print dir(conn)
dom0 = conn.lookupByName("bhyve")
except:
print 'Failed to find the main domain'
sys.exit(1)
print("Domain 0: id %d running" % (dom0.ID()))
print(dom0.info())
I got this:
('All domains: ', [])
('Defined domains: ', ['bhyve'])
Domain 0: id -1 running
('Domain info: ', [5, 2147483L, 0L, 1, 0L])
So that's an improvement.
I started reading this tutorial on libvirt and KVM scripting:
https://www.ibm.com/developerworks/linux/library/os-python-kvm-scripting1/
In that tutorial, they give an example where it is possible
to configure the MAC and IP address of the VM in an XML config file.
Is it possible to do that with libvirt and bhyve?
Thanks.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
BAFUG July 10: libvirt virtualization API for controlling bhyve virtual machines
Hi, On July 10, 2014, in Mountain View, California, I will be giving a brief tech talk on: "libvirt virtualization API for controlling bhyve virtual machines" I have been testing the extensions that Roman Bogorodskiy has made to the lbivirt library for supporting bhyve. I will talk about my experiences with libvirt and bhyve. Here are more details about the talk plus directions to the location: http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/191061222/ Feel free to sign up on the meetup site and attend if you are in the area! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: BAFUG July 10: libvirt virtualization API for controlling bhyve virtual machines
Hi, Slides for this talk are available here: http://people.freebsd.org/~rodrigc/libvirt/presentations/ iXsystems recorded video of the presentation. When they put it online, I'll send the link. -- Craig On Mon, Jun 30, 2014 at 5:30 PM, Craig Rodrigues wrote: > Hi, > > On July 10, 2014, in Mountain View, California, I will be giving a brief > tech talk on: > > "libvirt virtualization API for controlling bhyve virtual machines" > > I have been testing the extensions that Roman Bogorodskiy < > no...@freebsd.org> > has made to the lbivirt library for supporting bhyve. I will talk about > my experiences with libvirt and bhyve. > > Here are more details about the talk plus directions to the location: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/191061222/ > > Feel free to sign up on the meetup site and attend if you are in the area! > -- > Craig > ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
vhd file from snapshots: VBOX_E_OBJECT_NOT_FOUND
Hi, I downloaded FreeBSD-11.0-CURRENT-amd64-20140714-r268622.vhd.xz from ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64/20140714/ Under MacOS X, I uncompressed the .vhd.xz file. I used VirtualBox, and created a VM. When I tried to attach the .vhd file to this VM, VirtualBox gave this error: VBOX_E_OBJECT_NOT_FOUND Any ideas? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: vhd file from snapshots: VBOX_E_OBJECT_NOT_FOUND
On Tue, Jul 29, 2014 at 12:48 PM, Glen Barber wrote: > On Tue, Jul 29, 2014 at 12:44:01PM -0700, Craig Rodrigues wrote: > > Hi, > > > > I downloaded FreeBSD-11.0-CURRENT-amd64-20140714-r268622.vhd.xz > > from > > > ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-CURRENT/amd64/20140714/ > > > > > > Under MacOS X, I uncompressed the .vhd.xz file. I used VirtualBox, and > > created a VM. When I tried to attach > > the .vhd file to this VM, VirtualBox gave this error: > > > > VBOX_E_OBJECT_NOT_FOUND > > > > Any ideas? > > There are a few issues with how qemu-img creates the VHD. The use of > qemu-img will very soon be not required, thanks to marcel@ adding VHD > support to mkimg(1). > > In the meantime, can you try cloning the VHD disk, but instead of > a dynamically-allocated image, set it to be a fixed-size image? I have > had very sporadic success with this, so cannot guarantee it will work. > > The VMDK image, however, should work without issue, if you are unable to > get the VHD image to work. > I don't know how to clone the VHD disk. Inside VirtualBox, I created a FreeBSD VM with no disks, and then attached the VHD image as a disk to this new VM. It wasn't working. However, I tried doing the same thing with the VMDK image, and that worked fine. If the new mkimg can create VHD files which import without problem into VirtualBox, that will be a huge improvement. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: BAFUG July 10: libvirt virtualization API for controlling bhyve virtual machines
Hi, Thanks to Cao Pham at iXsystems, video for this presentation has been posted online: https://www.youtube.com/watch?v=LRLV_SZo6Sw -- Craig On Mon, Jun 30, 2014 at 5:30 PM, Craig Rodrigues wrote: > Hi, > > On July 10, 2014, in Mountain View, California, I will be giving a brief > tech talk on: > > "libvirt virtualization API for controlling bhyve virtual machines" > > I have been testing the extensions that Roman Bogorodskiy < > no...@freebsd.org> > has made to the lbivirt library for supporting bhyve. I will talk about > my experiences with libvirt and bhyve. > > Here are more details about the talk plus directions to the location: > > http://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/191061222/ > > Feel free to sign up on the meetup site and attend if you are in the area! > -- > Craig > ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Patch to virt-manager port for bhyve
Jason, Can you apply this patch to the virt-manager port: https://people.freebsd.org/~rodrigc/libvirt/patch-virtmanager-bhyve.txt This allows using virt-manager to start a bhyve VM. I gave a presentation on this at Bay Area FreeBSD Users Group (BAFUG) recently: http://www.slideshare.net/CraigRodrigues1/libvirt-bhyve Roman Bogorodskiy has pushed a similar patch upstream, but the upstream virt-manager code now depends on gtk3, and we don't fully have that supported in ports yet. Getting the newer virt-manager in ports with gtk3 supported is a big job, that I don't have the bandwidth for right now, but this patch helps a lot. Thanks! -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
