Hello Wido, Thank you so much for taking the time to respond.: I bonded the interfaces for capacity.
I was following your YouTube videos and users mail. Yes, VXLAN seems to be working now. The most dificult/radical part was the tunning of the parameres on Ubuntu 22. or should I use a different Linux system? Let me put on a logic it may clear for me; then, with VXLAN: Wido: Regarding your Youtube video: To me seems there is a VLAN on the TOR. But makes no sense based on what you mention here. What is the VLAN624 being used on your Youtube video @time 24:10 https://www.youtube.com/watch?v=X02bxtIC0u4 Then, Management servers will be on the same vxlan as kvm cloudbr1 then this should be the management vxlan. Correct? Storage I did put the ceph on vxlan2 with BGP; then, should I removed from vxlan ior keep it on the management vxlan? which one should be the best practice.? What About Network planning with Cloudstack for NFVs and low latency: Final Question. Does Cloudstack supports Sriov ? Any documentation besides an old document that inidicates we should list the PCI interfaces ? IS there ANy documentation about this? I was thinnking SRIOV will align better with VXLAN/EVPN Or do you recommend to use DPDk ? If DPDK wihich one will you recommend, - OpenVswitch - if this is selected then I will have to create the Bridges with Openvswitch, and should work with VXLAN/EVPN , right? - Or tunsgteen(OpenSDN) - I know Contrail pretty much very well, Not sure if there is any documentation about OpenSDN - Formerly Tungteen - with Cloudstack and VXLAN/EVPN, - Seems VXLAN/EVPN and tungsteen are mutually exclusive, plus not supported on Ubuntu 22 as of yet, are both of this statements correct? Thank you so much for taking the time to respond. Tata Y. > On Jan 13, 2025, at 8:45 AM, Wido den Hollander <w...@widodh.nl> wrote: > > > > Op 11/01/2025 om 00:36 schreef Chi vediamo: >> Forgot to add Wido >> The Isolation should start, and should happen at the HOST running KVM, No >> VLANS needed at the Host, neither at the Leafs. >> But seems isolation is not happening! using cloudstack 4.20, maybe is a >> detail missing on my side. >> What Cloudstac documentation mean with "On the Spine router(s) the VNIs will >> terminate and they will act as IPv4/IPv6 gateways" >> DO "Need" vlans to be configured on the Leafs. I will understand if the VM >> isolation is based on VLANS that will be needed, but if the VM Isolation is >> going to be VXLAN >> what for I will need the VLANS at the leafs? > > No VLANs needed. Forget Guest and Storage traffic. The hypervisor will do > everything routed! It will talk BGP with the Top-of-Rack. > > You will reach your storage via routing without the need of VXLAN, have it > route through your network. > > No need for bonding on your hypervisor either, it just has two BGP uplink > sessions towards the ToR. > > Just connect your storage to your network and make it reachable via L3, same > goes for the management server. > > But before starting to use CloudStack with VXLAN, make sure you get VXLAN > working without CloudStack so you really understand how VXLAN works. > > Wido > >> based on this scenario from Guido presso: >> [spine1] [spine2] no ibgp between spines >> / / >> / / evpn/bgp unnumbered >> / / >> / / >> [leaf1]-----ibgp---[Leaf2] >> \ / >> \ / >> \ / >> \ / >> | bond1 bond2 | >> | [cloudbr1] | >> | vxlan1 | >> | vxlan2 | >> | Host 1 - KVM | >> | SRIOV | >> -------------------- >> the vxlan1 will handle the guest traffic >> the vxlan2 will handle main storage >> I have management over a regular vlan for now but will create a vxlan3 for it >> and vxlan2000 for Public traffic. >> vxlan1, vxlan2, vxlan3, and vxlan2000 are goint to be created manually or >> Cloudstack should/will create them when I choose the VXLAN for each one of >> the traffic names? >> Or should I use VXLAN for the Guest traffic only and use regular vlan >> isolation for the the remaining - management, storage and public ? >> I have 4 physical hosts, and i am using same ip addressing on vxlan 1 and >> vxlan 2 connect to different leafs and i am still able to ping between them. >> Thank you >> Tata Y. >>> On Jan 6, 2025, at 9:08 PM, Chi vediamo <tatay...@gmail.com> wrote: >>> >>> >>> Hello Community, >>> First: Thank you Wido, your answers on previous emails to the community >>> help a lot. I read the vincent.bernat document, but his example uses VLAN >>> mapping at the Switch level. >>> >>> I was thinking to use the LEAF-SPINE as a transport only , and the seetings >>> on the Host will take care of the Isolation. >>> >>> But is not working that way. Should I create the traditional VXLAN, >>> VLAN/VNI/VRF on the LEAF Switches to properly isolate it? >>> We are using SONIC NOS community version, nothing fancy. >>> >>> The BGP unnumbered evpn etc works fine. >>> >>> The output: >>> >>> vtysh -c 'show interface vxlan2' >>> VNI: 2 >>> Type: L2 >>> Tenant VRF: default >>> VxLAN interface: vxlan2 >>> VxLAN ifIndex: 14 >>> SVI interface: storage0 >>> SVI ifIndex: 13 >>> Local VTEP IP: 172.2.0.60 >>> Mcast group: 0.0.0.0 >>> Remote VTEPs for this VNI: >>> 172.2.0.59 flood: HER >>> 172.2.0.32 flood: HER >>> 172.2.0.30 flood: HER >>> 172.2.0.28 flood: HER >>> 172.2.0.26 flood: HER >>> >>> bridge fdb show dev vxlan2 >>> 8a:be:71:4c:e0:20 vlan 1 extern_learn master storage0 >>> 8a:be:71:4c:e0:20 extern_learn master storage0 >>> b2:33:bb:84:cc:38 vlan 1 extern_learn master storage0 >>> b2:33:bb:84:cc:38 extern_learn master storage0 >>> 86:07:90:2b:db:db vlan 1 extern_learn master storage0 >>> 86:07:90:2b:db:db extern_learn master storage0 >>> 4a:28:60:90:76:42 vlan 1 extern_learn master storage0 >>> 4a:28:60:90:76:42 extern_learn master storage0 >>> 22:d6:49:9f:08:07 vlan 1 extern_learn master storage0 >>> 22:d6:49:9f:08:07 extern_learn master storage0 >>> fe:4a:fb:63:9d:3a vlan 1 extern_learn master storage0 >>> fe:4a:fb:63:9d:3a extern_learn master storage0 >>> ee:78:b4:d8:3f:a0 vlan 1 master storage0 permanent >>> ee:78:b4:d8:3f:a0 master storage0 permanent >>> 00:00:00:00:00:00 dst 172.2.0.24 self permanent >>> 00:00:00:00:00:00 dst 172.2.0.26 self permanent >>> 00:00:00:00:00:00 dst 172.2.0.28 self permanent >>> 00:00:00:00:00:00 dst 172.2.0.30 self permanent >>> 00:00:00:00:00:00 dst 172.2.0.32 self permanent >>> 00:00:00:00:00:00 dst 172.2.0.59 self permanent >>> >>> fe:4a:fb:63:9d:3a dst 172.2.0.24 self extern_learn >>> >>> >>> vtysh -c 'show interface vxlan1' >>> >>> Interface vxlan1 is up, line protocol is up >>> Link ups: 1 last: 2025/01/06 23:53:01.17 >>> Link downs: 1 last: 2025/01/06 23:53:01.17 >>> vrf: default >>> index 14 metric 0 mtu 9050 speed 4294967295 >>> flags: <UP,BROADCAST,RUNNING,MULTICAST> >>> Type: Ethernet >>> HWaddr: ea:d3:68:02:7d:f7 >>> inet6 fe80::e8d3:68ff:fe02:7df7/64 >>> Interface Type Vxlan >>> Interface Slave Type None >>> VxLAN Id 100 VTEP IP: 10.23.13.14 Access VLAN Id 1 >>> >>> protodown: off >>> >>> vtysh -c 'show evpn vni 1' >>> VNI: 1 >>> Type: L2 >>> Tenant VRF: default >>> VxLAN interface: vxlan1 >>> VxLAN ifIndex: 14 >>> SVI interface: cloudbr1 >>> SVI ifIndex: 12 >>> Local VTEP IP: 10.23.13.14 >>> Mcast group: 0.0.0.0 >>> No remote VTEPs known for this VNI >>> Number of MACs (local and remote) known for this VNI: 0 >>> Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0 >>> Advertise-gw-macip: No >>> Advertise-svi-macip: No >>> >>> and I can ping the IPV6 that is routed using the FRR from :60 which is in >>> VXLAN 2 to :14 which is in VXLAN 1 >>> >>>> ping -I 20XX:5XX:56XX:fff0::2:60 20XX:5XX:56XX:fff0:0:2:13:14 >>>> PING 20XX:5XX:56XX:fff0:0:2:13:14(20XX:5XX:56XX:fff0:0:2:13:14) from >>>> 20XX:5XX:56XX:fff0::2:60 : 56 data bytes >>>> 64 bytes from 20XX:5XX:56XX:fff0:0:2:13:14: icmp_seq=1 ttl=61 time=0.293 ms >>>> 64 bytes from 20XX:5XX:56XX:fff0:0:2:13:14: icmp_seq=2 ttl=61 time=0.222 ms >>> >>> >>> Then, my questions are: >>> are you using at the Leaf Switches/routers a regular Mapping VLAN to VNI >>> VXLAN with VRF ? IF not, Can you share a FRR config of your Switches? >>> Or should I use an enterprise SONIC switch software ? >>> What other possibilities are with the modifyvxlan.sh That Wido states on >>> some user mails. >>> >>> >>> Thank you >>> >>> Tata Y. >>> >>> >
