Hello,
I have a baremetal server hosted by Vultr that I would like to host
several bhyve instances on. I have been given one public ipv4 address,
so my goal is to run the bhyve instances on a private subnet
(172.16.0.0/24) then use pf to NAT this subnet and do port forwarding
to. I am having an issues though getting the VM instances network to work.
Here is how the hybervisor is configured:
uname:
FreeBSD cojo 11.2-STABLE FreeBSD 11.2-STABLE #1 r337947: Fri Aug 17
03:22:33 PDT 2018
pete@cojo:/usr/obj/usr/home/pete/src/freebsd-stable/sys/GENERIC amd64
rc.conf:
cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="inet 172.16.0.1 netmask 255.255.255.0 addm tap0 up"
gateway_enable="YES"
ipv6_gateway_enable="YES"
pf_enable="YES"
pf.conf:
ext_if=ix0
scrub in all
nat on $ext_if inet from 172.16.0.0/24 to any -> ($ext_if)
pass in all
pass out all
and my script to invoke the VM:
/usr/sbin/bhyve -AHP -s 31:0,lpc \
-s 2:0,virtio-net,tap0 \
-s 3:0,virtio-blk,/vms/freebsd0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
-c 2 -m 1024M \
test0
I am able to bring up the FreeBSD VM and can attach to it via tightvnc,
so that's great. I have configured the VM to have an IPv4 address of
172.16.0.20/24 with a gateway of 172.16.0.1. This is where I get stuck
- when I attempt to ping the gateway from the VM I get "ping: sendto
host down" errors. Then I run "arp -an" to see what's up from the VM
and see the following:
? (172.16.0.1) at (incomplete) on vtnet0 expired [ethernet]
when I run tcpdump against the bridge0 interface on the hypervisor while
ping is run I see the following output:
15:56:48.995284 ARP, Request who-has 172.16.0.1 tell 172.16.0.20, length 46
15:56:48.995292 ARP, Reply 172.16.0.1 is-at 02:46:2f:56:ab:00 (oui
Unknown), length 28
And I can verify that that is the correct MAC addr of the bridge0 interface:
$ ifconfig bridge0|grep ether
ether 02:46:2f:56:ab:00
yet, tcpdump on the vtnet0 interface from the VM does not show the
responses to the ARP requests:
ARP: Request who-has 172.16.0.1 tell 172.16.0.20, length 28
ARP: Request who-has 172.16.0.1 tell 172.16.0.20, length 28
ARP: Request who-has 172.16.0.1 tell 172.16.0.20, length 28
I suspect I'm missing something trivial here, so any input would be
appreciated :)
Oh, one last bit - here are my sysctl settings:
security.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
kern.randompid=1
security.bsd.stack_guard_page=1
vfs.zfs.min_auto_ashift=12
net.link.tap.up_on_open=1
net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1
Cheers,
-pete
--
Pete Wright
[email protected]
@nomadlogicLA
_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"[email protected]"
