On 2015-03-07 07:49, Noah Bergbauer wrote:
> Hi,
>
> I'm running FreeBSD 10.1 on a dedicated server, with a Linux VM in bhyve. The
> handbook tells you to bridge the tap interface to a real network interface,
> but that's not an option for me because I only have one IPv4 address. So
> instead, I assigned an internal IP address to bridge0 and used pf(4) to set
> up NAT routing.
> All of this works without any issues, but I would like to increase the MTU of
> 1500. It's a virtual interface after all, why should it be so low? FreeBSD's
> loopback interface's MTU is 16384 and on Linux, it's even 65536.
>
> So I used ifconfig(8) to increase the MTU of tap0 and just like the manpage
> says, bridge0 had the same MTU after I added tap0. On the Linux side, I did
> the same with eth0 and then I sent 2000 byte pings to the host machine.
>
> It seems to work, but let's use tcpdump(8) to make sure:
>
> 02:02:46.244678 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1686, seq
> 3, length 2008
> 02:02:46.244983 IP 10.42.42.1 > 10.42.42.100: ICMP echo reply, id 1686, seq
> 3, length 1480
> 02:02:46.245061 IP 10.42.42.1 > 10.42.42.100: ip-proto-1
> 02:02:47.244953 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1686, seq
> 4, length 2008
> 02:02:47.245347 IP 10.42.42.1 > 10.42.42.100: ICMP echo reply, id 1686, seq
> 4, length 1480
> 02:02:47.245422 IP 10.42.42.1 > 10.42.42.100: ip-proto-1
>
> The entire request goes through the virtio NIC, tap0, bridge0 and finally to
> the host's kernel without any issues. Which then sends a _fragmented_ reply
> because apparently it still thinks the MTU is 1500.
>
> A quick check with route(8) confirms this:
>
> # route show 10.42.42.100
>route to: 10.42.42.100
> destination: 10.42.42.0
>mask: 255.255.255.0
> fib: 0
> interface: bridge0
> flags:
> recvpipe sendpipe ssthresh rtt,msecmtuweightexpire
>0 0 0 0 1500 1 0
>
> So I manually forced a bigger MTU:
>
> # route change -net 10.42.42.0 -mtu 15000
> change net 10.42.42.0
>
> But now the reply packets get truncated instead of fragmented:
>
> 02:07:36.921165 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 3, length 2008
> 02:07:36.921624 IP truncated-ip - 6 bytes missing! 10.42.42.1 > 10.42.42.100:
> ICMP echo reply, id 1689, seq 3, length 2008
> 02:07:37.921042 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 4, length 2008
> 02:07:37.921499 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 4, length 2008
> 02:07:38.921522 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 5, length 2008
> 02:07:38.922253 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 5, length 2008
> 02:07:39.921432 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 6, length 2008
> 02:07:39.922165 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 6, length 2008
> 02:07:40.921513 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 7, length 2008
> 02:07:40.922245 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 7, length 2008
> 02:07:41.921393 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 8, length 2008
> 02:07:41.922160 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 8, length 2008
> 02:07:42.921504 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 9, length 2008
> 02:07:42.922348 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 9, length 2008
> 02:07:43.923031 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 10, length 2008
> 02:07:43.924904 IP truncated-ip - 6 bytes missing! 10.42.42.1 > 10.42.42.100:
> ICMP echo reply, id 1689, seq 10, length 2008
> 02:07:44.926832 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 11, length 2008
> 02:07:44.928511 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 11, length 2008
> 02:07:45.936968 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 12, length 2008
> 02:07:45.937722 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 12, length 2008
> 02:07:46.937453 IP 10.42.42.100 > 10.42.42.1: ICMP echo request, id 1689, seq
> 13, length 2008
> 02:07:46.938161 IP truncated-ip - 518 bytes missing! 10.42.42.1 >
> 10.42.42.100: ICMP echo reply, id 1689, seq 13, length 2008
> ^C
> 22 packets captured
> 24 packets received by filter
> 0 packets dropped by kernel
>
> This last dump was on the Linux side. Dumping at tap0 shows that the reply
> packets are still okay when they reach bhyve. Apparently they get truncated
> by bhyve's virtio-net as increasing the MTU works just f
