Hey Wido and everyone in the Community, Hope you're all doing great. We're setting up Cloudstack and stumbled upon your cool videos about VXLAN, BGP, and IPV6 from 2019 and 2021. Watched them all, including the links you shared. Noticed that similar concepts pop up in Openstack and OpenNebula too, with a few twists.
So, we've got a few things on our mind about the setup, especially how it fits (or doesn't) with what we're doing. You mentioned in your video to reach out with questions, so here we are, looping you in. Our Setup Is Where: # Cloudstack V4.18.1 + Linstor (For SDS) # VXLAN Enabled for Guest (Using Multicast) # 1 Pair of Redundant Core Switch (With option to scale out) # 1 Pair of Redundant Leaf Switch # Our datacenter can handle up to 1,000 hypervisors, which is more than enough for us. Our setup isn’t huge – just right for supporting a max of 1,000 hypervisors per datacenter. And, we're not planning to link multiple datacenters into one big availability zone. Having said that, we're scratching our heads over whether BGP+EVPN is the way to go for us. Here's what we're thinking: # BGP + EVPN doesn't add additional benefit to small/medium sized clouds. (In our case, we only need Max 1,000 Hypervisors per Datacenter). Implementing BGP+EVPN only increases the complexity of the setup. # We are using IPV4. But it seems that for this BGP+EVPN to be beneficial, we need to use IPV6. If we use back IPV4 (with BGP+EVPN), there is no benefit than if we were to use the default VXLAN+Multicast. # And about redundancies – doesn't look like VXLAN+BGP+EVPN offers anything more than what VXLAN+Multicast already does. Our Assumptions, To enable BGP + EVPN: 1. We need to instal FRRouter in each Hypervisor and form BGP Neighbor to the Leaf Switch (so this is Hypervisor <> Leaf Switch and NOT Hypervisors <> Hypervisor). 2. After forming neighbours with the Leaf Switch, we need to enable L2VPN at all leaf switches. 3. When the first 2 steps above are done, this would then mean that each Hypervisor will be as if it is sitting in its own network segment. (Usually all hypervisors will be in 1 single network segment) 4. We run the modifyvxlan.sh provided on Github and replace it with the default one from Cloudstack. 5. That's it, basically run Cloudstack and test to see if it works. (That's all the steps we're aware of). Note: Our assumption is also that this BGP setup will only affect internal Guest Network (VXLAN) communications. Networks to the Public Internet (VLAN) will not be using this BGP. Would love to get your thoughts on this. Are we on the right track or missing something? Any advice or heads-up would be great. Regards, Hunter Yap
