If it's a bug, I don't believe it's in the virtio stack.

I'm using Chelsio NICs on my bhyve servers, and I don't have to
disable anything for my guests to work, I even have TOE enabled.

-Dustin

On Tue, Nov 15, 2016 at 6:07 AM, Rajil Saraswat <raji...@gmail.com> wrote:
> On Nov 15, 2016 5:58 AM, <free...@jonathanprice.org> wrote:
>>
>> Hello,
>>
>> I am trying to get a windows server VM running under bhyve. So far I can
> use VNC to install Windows, but I've been unable to get the networking
> working. I will provide details below, and include a summary up here. On
> the guest, I have installed the latest version of winvirtio, and I have
> configured static IP addressing. The guest can ping itself, but not the
> host machine or anything external. It is worth noting that I am also
> currently running a CentOS guest, and this can connect to both the host and
> the outside world with no issues.
>>
>> Host: FreeBSD 11.0-RELEASE
>> Guest: Tried both Windows Server 2016 Essentials and Windows Server 2012
> Standard
>>
>> bhyve command:
>> bhyve -c 1 -H -w -m 3G -s 0,hostbridge -s 3,ahci-cd,/iohyve/ISO// -s
> '4,ahci-hd,/dev/zvol/zroot/iohyve/win12/disk0,sectorsize=512' -s
> 10,virtio-net,tap2 -s '11,fbuf,tcp=0.0.0.0:5902,w=800,h=600,wait' -s
> 12,xhci,tablet -s 31,lpc -l com1,/dev/nmdm3A -l
> bootrom,/iohyve/Firmware/BHYVE_UEFI.fd/BHYVE_UEFI.fd ioh-win12
>>
>> ifconfig output:
>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>
> options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
>>         ether e0:69:95:a2:f1:d9
>>         inet 46.105.xxx.xxx netmask 0xffffff00 broadcast 46.105.124.255
>>         inet6 fe80::e269:95ff:xxxx:f1d9%em0 prefixlen 64 scopeid 0x1
>>         inet6 2001:41d0:2:xxxx::1 prefixlen 56
>>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>         media: Ethernet autoselect (100baseTX <full-duplex>)
>>         status: active
>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>>         options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>>         inet6 ::1 prefixlen 128
>>         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
>>         inet 127.0.0.1 netmask 0xff000000
>>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>         groups: lo
>> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
>>         ether 02:cf:0c:8f:13:00
>>         inet6 fd00::ffff prefixlen 112
>>         inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255
>>         nd6 options=1<PERFORMNUD>
>>         groups: bridge
>>         id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>>         maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>>         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>>         member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                 ifmaxaddr 0 port 7 priority 128 path cost 2000000
>>         member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                 ifmaxaddr 0 port 5 priority 128 path cost 2000000
>>         member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                 ifmaxaddr 0 port 4 priority 128 path cost 2000000
>> pflog0: flags=0<> metric 0 mtu 33160
>>         groups: pflog
>> tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
> mtu 1500
>>         description: iohyve-centos
>>         options=80000<LINKSTATE>
>>         ether 00:bd:5d:d3:e8:00
>>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>         media: Ethernet autoselect
>>         status: active
>>         groups: tap
>>         Opened by PID 96206
>> tap1: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>         description: iohyve-win16es
>>         options=80000<LINKSTATE>
>>         ether 00:bd:61:ab:eb:01
>>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>         media: Ethernet autoselect
>>         status: no carrier
>>         groups: tap
>> tap2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
> mtu 1500
>>         description: iohyve-win12
>>         options=80000<LINKSTATE>
>>         ether 00:bd:c9:1a:59:02
>>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>         media: Ethernet autoselect
>>         status: active
>>         groups: tap
>>         Opened by PID 1911
>>
>> It is using NAT with PF to reach the internet. It should not rely on NAT
> to be able to ping the host machine, but I will post the NAT configuration
> below anyway:
>>
>> /etc/pf.conf:
>> # Automatically NAT any VMs
>> nat on em0 inet from bridge0:network to any -> (em0)
>>
>>
>> (On the Guest)
>> ==============
>>
>> C:\Users\Administrator>ipconfig
>>
>> Ethernet adapter Ethernet:
>>
>>         IPv4 Address: 10.0.0.3
>>         Subnet Mask: 255.255.255.0
>>         Default Gateway: 10.0.0.254
>>
>> C:\Users\Administrator>ping 10.0.0.3
>>
>> Pinging 10.0.0.3 with 32 bytes of data:
>> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
>> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
>> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
>> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128
>>
>> C:\Users\Administrator>ping 10.0.0.254
>>
>> Pinging 10.0.0.254 with 32 bytes of data:
>> Reply from 10.0.0.3: Destination host unreachable.
>> Reply from 10.0.0.3: Destination host unreachable.
>> Reply from 10.0.0.3: Destination host unreachable.
>> Reply from 10.0.0.3: Destination host unreachable.
>>
>> C:\Users\Administrator>arp -a
>>
>> Interface: 10.0.0.3 --- 0xc
>>   Internet Address              Physical Address                Type
>>   10.0.0.255                    ff-ff-ff-ff-ff-ff               static
>>   224.0.0.22                    01-00-5e-00-00-16               static
>>   224.0.0.252                   01-00-5e-00-00-fc               static
>>
>> (PF log testing)
>> ================
>>
>> I was curious to see if PF was seeing packets on the bridge interface, so
> I added the following to /etc/pf.conf:
>> pass log (all) all
>>
>> I then ran the following command, which had no results when I tried a
> test ping:
>> sudo tcpdump -n -e -ttt -i pflog0 host 10.0.0.3
>>
>> (I also watched live traffic on 10.0.0.254 and saw nothing from this IP)
>>
>>
>> Summary
>> ========
>>
>> I appear to be having some network connectivity issues that either lies
> in bhyve, virtio-net or the winvirtio drivers. If anybody has any
> suggestions for things I could try I'd love to hear it.
>>
>> Thanks,
>>
>> Jonathan.
>> _______________________________________________
>> freebsd-virtualization@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
>> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscr...@freebsd.org"
>
> I had to disable offloading on the host NIC to get the network working
> inside bhyve session. This seems to be a FreeBSD bug in virtio stack.
>
> ifconfig_igb0="-rxcsum -txcsum  -vlanmtu  -vlanhwtso  -tso4 -tso6
> -vlanhwtso -vlanhwcsum up"
>
> -Rajil
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to 
> "freebsd-virtualization-unsubscr...@freebsd.org"
_______________________________________________
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Reply via email to