Trevor,
I can tell you how I did full end-to-end redundancy that works and works
well for me:
two L3 switches, stacked, RSTP and Broadcast Flood cpontrol off.
Split switch into three VLANS: Management, Guest+pub, and Storage.
ALL hypervisors are xenserver 6.02, and eac has 6 1GbE ports. I
determined which ports were connected to which transcievers,and spread
out the network such that no bonded interface shares an ethernet
transciever, and from the back of the HV it looks like:
eth0:eth1:eth2:eth3:eth4:eth5
mgmt:pub :pub :strg:strg:mgmt
sw1 :sw1 :sw2 :sw1 :sw2 :sw2
So, I rounded up a windows laptop and installed xencenter, and bonded:
eth0 & eth5 became "management" tagged network in Xencenter )and
Cloudstack). This bond is active/passive
eth1 & 2 were bonded to form "guest_pub" network in Xencenter and
Cloudstack. This bond is active-active
eth3 & 4 were bonded to form "storage" network in Xencenter and
Cloudstack. Also active-active, and 9000 MTU
My SAN node1 has a management port and a 10G uplink to one switch, and
SAN node2(active-active SAN) has a 10G port uplink to the other switch.
Using this configuration, NO switch changes have to be made to
accommodate the bonded interfaces, because the management interface is
active-passive, and the others are active-active using mode6 I think,
which is a WLB algorithm, which requires no switch changes whatsoever.
Additionally, the storage VLAN is setup with jumbo frames end-to-end,
and has ALL broadcast storm control turned off.
I monitored the network for several days and have not noticed any floods
or broadcast storms at all.
Hope this helps.
On 10/16/2012 03:00 PM, Trevor Francis wrote:
So, with the help of the fine gents on the IRC channel (thank you KDamage), I
have successfully managed to crash my network over 20 times due to ARP floods
coming from the bridges that I have created. I attempted to create a highly
available network by utilizing bonded bridges split amongst trunked switch
pairs and designing physical networks to take advantage of those bonds.
Here is what my network looks like
Host (6 total ethernet ports)
Eth0, Eth1 = Bond0, cloudVirBr10 (bridge) . Bonding mode 6, STP enabled.
Eth2, Eth3 = Bond2, cloudVirBr12 (bridge) . Bonding mode 6, STP enabled.
Eth4, Eth5 = Bond1, cloudVirBr11 (bridge) . Bonding mode 6, STP enabled.
I have the bonds split amongst trunked switches.
Switch1A - eth0 , Switch1B, eth1 trunked between each other, STP enabled.
Switch2A - eth4, Switch2B, eth5 trunked between each other, STP enabled.
Switch3A - eth2, Switch3B, eth3 trunked between each other, STP enabled.
The switch pairs are totally isolated from each other (ie, no interswitch
trunking).
So, you can turn off any switch or pull any cable and the network still works
fine.
Switch1A and Switch1B have internet delivered to them through HSRP (2 drops),
so if either of the switches crashes, the internet continues.
I have been working for over a month to figure out why the floods occur, but we
just cant seem to figure it out.
The hosts are running Cent 6.3 and KVM HVs.
Our goal was to have Management and Storage traffic go over Bond2, Guest
traffic go over Bond1, and Public Internet traffic go over Bond0. We created
physical networks in advanced networking with those settings and KVM tags
corresponding to our bridge names.
Our application requirement require that we have 2 ethernet connections
assigned directly to the linux VMs (eth0 = public internet, eth1 = guest
network). We cannot use NAT between a public and a private network.
To achieve this, we created a standard Isolated guest network and then created
a shared network that assigned a pool of public IPs directly to the machines.
We can successfully turn a VM up with a public IP without storms and turn a VM
up with a guest IP without storms. But when we turn a VM up with both, the
storms start immediately, killing the public network.
We are considering abandoing KVM in lieu of Xenserver, but we were told that
there was no way to successfully separate our public traffic to go over a
bonded bride and have our guest traffic go over a different bridge.
My brain still works in "Old Networking" mode, so the new cloud networking
stuff is throwing me for a loop.
Seeing what our network consists of, do you recommend doing things a different
way? We are considering swapping Xen for the KVM HVs and are looking at getting
maximum performance while still maintaining full 2N redundancy.
Thanks,
TGF
--
Regards,
Nik
Nik Martin
VP Business Development
Nfina Technologies, Inc.
+1.251.243.0043 x1003
Relentless Reliability
