I currently have a problem with bridge networking on RHEL5 and RHEL6.
Instance interfaces added to a bridge interface can potentially lower the
bridge MTU, affecting the MTU of all other instances using the same bridge.
The fix is to set the MTU of the tap interface to be the same as that of
the bridge, before adding it. This patch applies against
tools/kvm-ifup.in.
What follows below is an example of the effects in my test setup. The
bridge interface in question, br801, has a MTU of 9000. Here is a config
snippet of one instance:
Hardware:
- VCPUs: 2
- memory: 2048MiB
- NICs:
- nic/0: MAC: aa:00:00:76:08:42, IP: None, mode: bridged, link: br10
- nic/1: MAC: aa:00:00:68:2b:cc, IP: None, mode: bridged, link: br801
Here's the configuration for bridge br801:
$ ip link show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP qlen 1000
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
$ ip link show dev eth0.801
10: eth0.801@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue
state UP
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
$ ip link show dev br801
11: br801: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state
UNKNOWN
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
After starting the instance, tap1 is created and associated with br801
$ brctl show br801
bridge name bridge id STP enabled interfaces
br10 8000.a4badb4484a8 no eth0.10
tap0
br801 8000.a4badb4484a8 no eth0.801
tap1
The tap interface tap1 has an MTU of 1500 and forces the bridge
interface down to 1500 as well:
$ for n in eth0 eth0.801 br801 tap1 ; do ip link show dev $n ; done
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP qlen 1000
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
10: eth0.801@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue
state UP
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
11: br801: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
45: tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UNKNOWN qlen 500
link/ether 0a:19:86:cd:5d:25 brd ff:ff:ff:ff:ff:ff
Attempts to ping with packet size 9000 from the instance fails.
After this, I shutdown the instance and restore the MTU of br801 to
9000. I then apply the patch. Before the instance is started:
$ for n in eth0 eth0.801 br801 ; do ip link show dev $n ; done
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP qlen 1000
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
10: eth0.801@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue
state UP
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
11: br801: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state
UNKNOWN
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
Start the instance, and tap1 is (again) associated with br801:
$ sudo gnt-instance start loki.example.net
Waiting for job 317 for loki.example.net...
$ brctl show br801
bridge name bridge id STP enabled interfaces
br10 8000.a4badb4484a8 no eth0.10
tap0
br801 8000.a4badb4484a8 no eth0.801
tap1
Here are the interfaces involved:
$ for n in eth0 eth0.801 br801 tap1 ; do ip link show dev $n ; done
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP qlen 1000
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
10: eth0.801@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue
state UP
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
11: br801: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state
UNKNOWN
link/ether a4:ba:db:44:84:a8 brd ff:ff:ff:ff:ff:ff
47: tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state
UNKNOWN qlen 500
link/ether ba:9a:8f:76:3c:32 brd ff:ff:ff:ff:ff:ff
I can now ping with packet size 9000 from the instance to other
destinations.
libvirt had a similar problem and resolution described here:
http://www.redhat.com/archives/libvir-list/2008-December/msg00083.html
