Status of ptnet patch integration into bhyve ?
Hi, What is the status of merging bhyve's ptnet patches into head ? bhyve network performance (with tap + if_bridge) prevent us to use it on production environment, but I remember an interesting presentation about netmap/ptnet patches for bhyve [1]. I believe bhyve's team is aware of this project, but is there a design problem with it for not being merged into head? Thanks, Olivier [1] https://www.bsdcan.org/2017/schedule/events/814.en.html [2] https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: -current host, 10.1 client loops
On Wed, Mar 16, 2016 at 10:25 PM, Poul-Henning Kamp wrote: > > How do I tell that script to use ahci-hd ? > > You just need to replace virtio-blk by ahci-hd in the script, like: sed 's/virtio-blk/ahci-hd/g' /usr/share/examples/bhyve/vmrun.sh > ~/vmrun-ahci.sh And running ~/vmrun-ahci.sh in place ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Limited number of vale switch useable with bhyve ?
Hi, Thanks to btw's patch I can now use bhyve and vale. But I'm hitting a limitation: I want to start 5 VM connected by a full-meshed network. This mean: - VM 1 has 4 interfaces toward each other 4 VMs - and so one for all others 4 VMs This give these 5 command lines: bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc \ -s 2:0,virtio-net,vale12:BSDRP_1,mac=58:9c:fc:01:02:01 \ -s 2:1,virtio-net,vale13:BSDRP_1,mac=58:9c:fc:01:03:01 \ -s 2:2,virtio-net,vale14:BSDRP_1,mac=58:9c:fc:01:04:01 \ -s 2:3,virtio-net,vale15:BSDRP_1,mac=58:9c:fc:01:05:01 \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_1 \ -l com1,/dev/nmdm1A BSDRP_1 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc \ -s 2:0,virtio-net,vale15:BSDRP_5,mac=58:9c:fc:01:05:05 \ -s 2:1,virtio-net,vale25:BSDRP_5,mac=58:9c:fc:02:05:05 \ -s 2:2,virtio-net,vale35:BSDRP_5,mac=58:9c:fc:03:05:05 \ -s 2:3,virtio-net,vale45:BSDRP_5,mac=58:9c:fc:04:05:05 \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_5 \ -l com1,/dev/nmdm5A BSDRP_5 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc \ -s 2:0,virtio-net,vale12:BSDRP_2,mac=58:9c:fc:01:02:02 \ -s 2:1,virtio-net,vale23:BSDRP_2,mac=58:9c:fc:02:03:02 \ -s 2:2,virtio-net,vale24:BSDRP_2,mac=58:9c:fc:02:04:02 \ -s 2:3,virtio-net,vale25:BSDRP_2,mac=58:9c:fc:02:05:02 \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_2 \ -l com1,/dev/nmdm2A BSDRP_2 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc -s 2:0,virtio-net,vale13:BSDRP_3,mac=58:9c:fc:01:03:03 \ -s 2:1,virtio-net,vale23:BSDRP_3,mac=58:9c:fc:02:03:03 \ -s 2:2,virtio-net,vale34:BSDRP_3,mac=58:9c:fc:03:04:03 \ -s 2:3,virtio-net,vale35:BSDRP_3,mac=58:9c:fc:03:05:03 \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_3 \ -l com1,/dev/nmdm3A BSDRP_3 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc -s 2:0,virtio-net,vale14:BSDRP_4,mac=58:9c:fc:01:04:04 \ -s 2:1,virtio-net,vale24:BSDRP_4,mac=58:9c:fc:02:04:04 \ -s 2:2,virtio-net,vale34:BSDRP_4,mac=58:9c:fc:03:04:04 \ -s 2:3,virtio-net,vale45:BSDRP_4,mac=58:9c:fc:04:05:04 \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_4 -l com1,/dev/nmdm4A BSDRP_4 But host system complains it didn't reach to allocate memory,: 743.527464 nm_open [608] NIOCREGIF failed: Cannot allocate memory vale35:BSDRP_5 743.527470 nm_open [608] NIOCREGIF failed: Cannot allocate memory vale45:BSDRP_4 open of netmap device vale35:BSDRP_5 failed open of netmap device vale45:BSDRP_4 failed 743.561324 nm_open [608] NIOCREGIF failed: Cannot allocate memory vale45:BSDRP_5 open of netmap device vale45:BSDRP_5 failed 743.561359 nm_open [608] NIOCREGIF failed: Cannot allocate memory vale35:BSDRP_3 open of netmap device vale35:BSDRP_3 failed And the vtnet interfaces that failed to allocate memory are correctly in "status: no carrier mode" in the guests. But the host system has plenty of free RAM: root@lame5:~ # top -b last pid: 2242; load averages: 0.09, 0.24, 0.22 up 0+04:24:44 13:23:20 64 processes: 1 running, 63 sleeping Mem: 556M Active, 94M Inact, 5645M Wired, 243G Free ARC: 1273M Total, 461M MFU, 805M MRU, 16K Anon, 3117K Header, 3332K Other Swap: 4096M Total, 4096M Free Can I increase some sysctl dev.netmap for allowing more memory to be used by netmap/vale ? I've tried to multiply per 10 dev.netmap.buf_num, but no changes. Thanks ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
How to use bhyve and vale switch ?
Hi, I'm trying to start 2 VMs connected by a vale switch. Host system: FreeBSD 11.0-CURRENT #1 r296650 (Fri Mar 11) Guest systems: FreeBSD 11.0-CURRENT #0 r296721 (Sat Mar 12) Command-lines used for starting these VM are these one: sysctl dev.netmap.verbose=1 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc \ -s '2:0,virtio-net,vale12:BSDRP_2,mac=58:9c:fc:01:02:02' \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_2 \ -l com1,/dev/nmdm2A BSDRP_2 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc \ -s '2:0,virtio-net,vale12:BSDRP_1,mac=58:9c:fc:01:02:01' \ -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_1 \ -l com1,/dev/nmdm1A BSDRP_1 No error message found on the host dmesg regarding netmap. But from the guest, their vtnet NIC didn't works: [root@VM1]~# ifconfig vtnet0 1.1.1.1/24 up [root@VM1]~# ifconfig vtnet0 vtnet0: flags=8943 metric 0 mtu 1500 options=80028 ether 58:9c:fc:01:02:01 inet6 fe80::5a9c:fcff:fe01:201%vtnet0 prefixlen 64 scopeid 0x1 inet 1.1.1.1 netmask 0xff00 broadcast 1.1.1.255 nd6 options=21 media: Ethernet 10Gbase-T (none) status: no carrier Interface status is "no carrier" and nothing can exit the interface. Any idea where I did a mistake ? Thanks ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
No multicast routing support on Virtio vtnet(4) drivers ?
I'm trying to set-up a multicast network lab using bhyve (then the vtnet interfaces). But when starting a PIM multicast routing daemon, I've got this error message: [root@router]~# service pimd start Starting pimd. pimd: 17:22:59.119 Failed adding VIF 0 (MRT_ADD_VIF) for iface vtnet0:(error 45): Operation not supported May 5 17:22:59 router pimd[1810]: Failed adding VIF 0 (MRT_ADD_VIF) for iface vtnet0: Operation not supported /usr/local/etc/rc.d/pimd: WARNING: failed to start pimd Is it a bug or a non-implemented feature ? I didn't meet this problem on virtualbox with em(4) interfaces. Regards, ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Enabling VIMAGE by default for FreeBSD 11?
On Tue, Oct 14, 2014 at 8:20 AM, Craig Rodrigues wrote: > On Sat, Oct 11, 2014 at 1:20 PM, Alexander V. Chernikov > wrote: > > > > > On 11 Oct 2014, at 21:58, Craig Rodrigues wrote: > > > > > Hi, > > > > > > What action items are left to enable VIMAGE by default for FreeBSD 11? > > Are there any tests results showing performance implications on different > > network-related workloads? > > > > > Alexander, > > Do you have a testbed where you could run a quick network test for > non-virtualized workload: >-> CURRENT without VIMAGE in kernel config >-> CURRENT with VIMAGE in kernel config > > and provide some results ? > I can use my forwarding/firewalling 10Giga lab for testing VIMAGE impact. Here are my ministat results (smallest packet size, value in packet-per-second, about 2000 flows). => I didn't see lot's of performance impact with VIMAGE option added in kernel. Forwarding difference : x forwarding.r272978-VIMAGE + forwarding.r272978 +--+ |x +x x +++ xx +| | |_M__A| | | |MA| | +--+ N Min MaxMedian AvgStddev x 5 1929165 1998339 1963904 1966801.4 27506.256 + 5 1953943 2005868 1971503 1976523 19087.721 No difference proven at 95.0% confidence ipfw-statefull difference: x ipfw-statefull.r272978-VIMAGE + ipfw-statefull.r272978 +--+ | x x * * +x+ +| ||_MA__| | ||___M___A___| | +--+ N Min MaxMedian AvgStddev x 5 1490042 1531750 1503590 1505175 16403.596 + 5 1502719 1589778 1517320 1529871.8 35404.181 No difference proven at 95.0% confidence pf-statefull difference: x pf-statefull.r272978-VIMAGE + pf-statefull.r272978 +--+ |x++xx *+x +| ||__AM_| | | |A_M_| | +--+ N Min MaxMedian AvgStddev x 5 1315594 1341130 1334215 1331310 9769.922 + 5 1324108 1351078 1336257 1335044.2 10562.448 No difference proven at 95.0% confidence Regards ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve's VM loose network connectivity after a reboot
On Thu, Aug 21, 2014 at 5:26 PM, Peter Grehan wrote: > Hi Olivier, > > Do you have the net.link.tap.up_on_open sysctl set to 1 ? (didn't see > that in the BSDRP-lab-bhyve.sh script, if that's the one you're using) > > > Yes I was using the BSDRP-lab-bhyve.sh script and correctly setting net.link.tap.up_on_open fix my problem: Thanks ! Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve's VM loose network connectivity after a reboot
Hi, I'm using bhyve from a 11.0-CURRENT (r270177) host and I meet a problem after a VM (guest using fbsd -current too) is rebooted: This VM loose network connectivity once rebooted. vtnet interface seems OK from the VM, like the tap interface on the host: tap status correctly display the new bhyve PID number in its "Opened by" section. But this VM didn't send packet anymore: a "tcpdump -i tap-interface" from the host didn't show any packet after a reboot. Is there something special to do in the "while [ 1 ]" loop of my script, other than simply restarting bhyve for fixing this issue ? Thanks, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve and serial port emulation problem
On Thu, May 1, 2014 at 7:29 PM, Neel Natu wrote: > Hi Olivier, > > The UPDATING entry in r264175 describes how to fix this: > http://svnweb.freebsd.org/base?view=revision&revision=264175 > > Briefly, you will now need to change your /etc/ttys to look like this: > ttyu0 "/usr/libexec/getty 3wire" vt100 on secure > > If your virtual machine has an older world then your /etc/gettytab > will not have the entry for "3wire" and you may want to use > "3wire.9600" instead. > Hi Neel, I've just discovered that on an headless server I had the same problem too, then concluded that I need to check some serial port behavior change and not bhyve bug. I believe the default templates for /etc/ttys (like src/etc/etc.amd64/ttys) need to be updated for replacing "std.9600" by "3wire" for avoiding this regression on new installation. Sorry for the noise regarding bhyve, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve and serial port emulation problem
On Wed, Apr 30, 2014 at 7:33 PM, Neel Natu wrote: > Hi Olivier, > > Can you share the command line options you pass the bhyve(8)? > > Hi Neel, sure, here it is: bhyveload -m 256M -d /tmp/BSDRP/BSDRP_1 -c /dev/nmdm1A BSDRP_1 bhyve -c 1 -m 256M -A -H -P -s 0:0,hostbridge -s 0:1,lpc -s 2:0,virtio-net,tap0,mac=58:9c:fc:01:00:01 -s 1:0,virtio-blk,/tmp/BSDRP/BSDRP_1 -l com1,/dev/nmdm1A BSDRP_1 Regards, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve and serial port emulation problem
Hi, I'm using the same nanobsd image on Virtualbox and bhyve (-CURRENT r265101). The nanobsd image I'm using is configured for serial port as default console: [root@nanobsd]~# cat /boot.config -S38400 -h [root@nanobsd]~# grep ttyu0 /etc/ttys ttyu0 "/usr/libexec/getty std.38400" vt100 on secure The Virtualbox VM is configured for creating a serial pipe: VBoxManage modifyvm nanobsd --uart1 0x3F8 4 --uartmode1 server /tmp/nanobsd.serial and I connect to this guest using "socat unix-connect:/tmp/nanobsd.serial stdio,raw,echo=0,icanon=0" => And it's works without problem. The bhyve is configured with a com1 redirecting to a nmdm device: "-l com1,/dev/nmdm1" and I connect to this guest using "cu -l /dev/nmdm1B" => But this bhyve console didn't works correctly: I can't use tool like less/more (even ssh-keygen can't ask for a passphrase). If I run truss with these tools, It seems they hang for opening /dev/tty. Some differences between virtualbox and bhyve (don't know if it can help): [root@vbox_guest]~# kenv | grep console comconsole_pcidev="" comconsole_port="1016" comconsole_speed="9600" console="comconsole" [root@bhyve_guest]~# kenv | grep console console="userboot" Regards, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Report of my virtual network lab migrated from virtualbox to bhyve
On Fri, Feb 7, 2014 at 8:38 PM, Peter Grehan wrote: > > > If you create a sparse file for the bhyve raw disk (e.g. with truncate > -s), du will show the actual blocks used rather than the total size. But can I truncate an already existing image disk (downloaded nanobsd image as example) ? > There is an em(4) emulation slowly being worked on. It should also be > possible to add altq functionality to FreeBSD's virtio net driver. Adding altq(4) to vtnet(4) should be a better idea. > For internal-only networks, there will most likely be a user-space > ethernet switch (ala VDE) that bhyve network interfaces can be pointed at. Great: Why not the high-perf netmap VALE switch ? Because with a dummy TCP iperf bench between bhyve guests report only 254 Mbits/sec. Regards, ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Report of my virtual network lab migrated from virtualbox to bhyve
Hi, just a report of my migration to bhyve. I was using VirtualBox for generate full-meshed network lab of multiple VM (essentially nanobsd based) and have migrated my script to bhyve. My original script is resumed to this kind of usage: ./lab-script.sh -i FreeBSD-image-disk -n number-of-vm -l number-of-LAN For example I can easly generate this kind of topology: ./BSDRP-lab-bhyve.sh -i nanobsd-serial.img -n 4 -l 2 VM 1 have the following NIC: - vtnet0 connected to VM 2. - vtnet1 connected to VM 3. - vtnet2 connected to VM 4. - vtnet3 connected to LAN number 1 - vtnet4 connected to LAN number 2 VM 2 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 3. - vtnet2 connected to VM 4. - vtnet3 connected to LAN number 1 - vtnet4 connected to LAN number 2 VM 3 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 2. - vtnet2 connected to VM 4. - vtnet3 connected to LAN number 1 - vtnet4 connected to LAN number 2 VM 4 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 2. - vtnet2 connected to VM 3. - vtnet3 connected to LAN number 1 - vtnet4 connected to LAN number 2 First remark comparing the disk format: With an original nanobsd disk image of 488MB. - Virtualbox format disk size: 133M - bhyve raw disk size: 488M Second remark regarding the difference between bhyve and virtualbox with disk image management: I'm using the "Linked clone" feature of virtualbox: This avoid to duplicate full guest disk. Here is with the example of 10 VM: - Virtualbox consumed disk space: 133M + 10x133K = about 135M in total - bhyve consumed disk space: 488M x 10 = 4880M in total => Hard drive space is not expensive today, but there is a huge difference (I didn't compare perfs). Third remark regarding memory usage on host (reported by top) with each VM configured for 256M of RAM (the guest report 70M of used memory) in SIZE/RES : . Virtualbox consumed RAM: 490/308M x 10 + 125/20M (VBoxSVC) + 80/11M (VBoxXPCOMIPCD) = total of 5100/4010M - bhyve consumed RAM: 283/22M x 10 = total of 2830/220M => Wow... bhyve memory usage is lot's lower than with Virtualbox ! Forth remark regarding network card emulation: Virtualbox permits to emulate em(4) NIC that supports altq(4) but bhyve emulate only vtnet(4) that didn't have altq(4) support. => I can't use bhyve for simulating network lab with altq(4) Fifth remark regarding network card virtualization: Virtualbox permits to generate "internal only" NIC between VM, but bhyve supports only tap. Can you add epair(4) support into bhyve roadmap ? => I have to create a huge number of bridge/tap interfaces on the host just for internal VM Ethernet links. And a simple user can't create bridge/tap interfaces. Sixth remark regarding the use of nmdm device for serial redirection: Often, I have to start a connection to the nmdm device for "un-pausing" the starting of the bhyve guest. And once I connect to the nmdm dev, this error is displayed (on head): Assertion failed: (error == 0), function init_pci, file /src/usr.sbin/bhyve/pci_emul.c, line 1047. Seventh remark regarding user: I didn't have to use Virtualbox as root (just to be in the virtualbox group), but it's mandatory with bhyve. And final remark: FreeBSD booting speed is very impressive with bhyve ! My bhyve and virtualbox scripts (and qemu) used are online here: http://bsdrp.net/documentation/examples/how_to_build_a_bsdrp_router_lab Regards, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Virtualbox Non OSE version?
On Sat, May 29, 2010 at 3:55 PM, Bram Verswalm wrote: > Will there be a non OSE-version for FreeBSD in the future? Cause VRDP is > really a basic need for headless servers. There is a VNC patch for the FreeBSD Virtualbox OSE-version made by Daisuke Aoyama here: https://sourceforge.net/mailarchive/forum.php?thread_name=ECBD0772A41948D0A6430727BDD3054E%40artemis&forum_name=freenas-devel Regards, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Qemu and serial uart drivers
Hi, I meet a problem with the new serial driver "uart" used in FreeBSD 8.0-Beta and Qemu. Hi all, Under Qemu, my "serial-console-only" FreeBSD 7.2 image (sio drivers) works great, but my "serial-console-only" FreeBSD 8.0-Beta4 image (uart drivers) doesn't works. I try to replace Qemu 0.10.6 with an "up-to-date" qemu-devel using this patch: http://people.freebsd.org/~nox/qemu/qemu-devel-20090829.patch But I've still the same problem (I start qemu with no-graphic). The startup log under FreeBSD 8.0-Beta4 stop here: (...) atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] uart0: port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart0: console (9600,n,8,1) cpu0: on acpi0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: at port 0x3c0-0x3df iomem 0xa-0xb on isa0 Timecounter "TSC" frequency 1995170970 Hz quality 800 Timecounters tick every 1.000 msec IPsec: Initialized Security Association Processing. ad0: 476MB at ata0-master WDMA2 GEOM: ad0s1: geometry does not match label (255h,63s != 16h,63s). Trying to mount root from ufs:/dev/ufs/BSDRP1 cp: I've found this old message that report the same problem, but no solution here: http://lists.freebsd.org/pipermail/freebsd-current/2009-February/003222.html Anyone have an idea about this problem ? Is a FreeBSD bug or a Qemu bug ? What information can I give to you for troubleshoot this problem ? Thanks, Olivier ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
