Op 11/01/2025 om 00:36 schreef Chi vediamo:
Forgot to add Wido
The Isolation should start, and should happen at the HOST running KVM,
No VLANS needed at the Host, neither at the Leafs.
But seems isolation is not happening! using cloudstack 4.20, maybe is a
detail missing on my side.
What Cloudstac documentation mean with "On the Spine router(s) the VNIs
will terminate and they will act as IPv4/IPv6 gateways"
DO "Need" vlans to be configured on the Leafs. I will understand if the
VM isolation is based on VLANS that will be needed, but if the VM
Isolation is going to be VXLAN
what for I will need the VLANS at the leafs?
No VLANs needed. Forget Guest and Storage traffic. The hypervisor will
do everything routed! It will talk BGP with the Top-of-Rack.
You will reach your storage via routing without the need of VXLAN, have
it route through your network.
No need for bonding on your hypervisor either, it just has two BGP
uplink sessions towards the ToR.
Just connect your storage to your network and make it reachable via L3,
same goes for the management server.
But before starting to use CloudStack with VXLAN, make sure you get
VXLAN working without CloudStack so you really understand how VXLAN works.
Wido
based on this scenario from Guido presso:
[spine1] [spine2] no ibgp between spines
/ /
/ / evpn/bgp unnumbered
/ /
/ /
[leaf1]-----ibgp---[Leaf2]
\ /
\ /
\ /
\ /
| bond1 bond2 |
| [cloudbr1] |
| vxlan1 |
| vxlan2 |
| Host 1 - KVM |
| SRIOV |
--------------------
the vxlan1 will handle the guest traffic
the vxlan2 will handle main storage
I have management over a regular vlan for now but will create a vxlan3
for it
and vxlan2000 for Public traffic.
vxlan1, vxlan2, vxlan3, and vxlan2000 are goint to be created manually
or Cloudstack should/will create them when I choose the VXLAN for each
one of the traffic names?
Or should I use VXLAN for the Guest traffic only and use regular vlan
isolation for the the remaining - management, storage and public ?
I have 4 physical hosts, and i am using same ip addressing on vxlan 1
and vxlan 2 connect to different leafs and i am still able to ping
between them.
Thank you
Tata Y.
On Jan 6, 2025, at 9:08 PM, Chi vediamo <tatay...@gmail.com> wrote:
Hello Community,
First: Thank you Wido, your answers on previous emails to the
community help a lot. I read the vincent.bernat document, but his
example uses VLAN mapping at the Switch level.
I was thinking to use the LEAF-SPINE as a transport only , and the
seetings on the Host will take care of the Isolation.
But is not working that way. Should I create the traditional VXLAN,
VLAN/VNI/VRF on the LEAF Switches to properly isolate it?
We are using SONIC NOS community version, nothing fancy.
The BGP unnumbered evpn etc works fine.
The output:
vtysh -c 'show interface vxlan2'
VNI: 2
Type: L2
Tenant VRF: default
VxLAN interface: vxlan2
VxLAN ifIndex: 14
SVI interface: storage0
SVI ifIndex: 13
Local VTEP IP: 172.2.0.60
Mcast group: 0.0.0.0
Remote VTEPs for this VNI:
172.2.0.59 flood: HER
172.2.0.32 flood: HER
172.2.0.30 flood: HER
172.2.0.28 flood: HER
172.2.0.26 flood: HER
bridge fdb show dev vxlan2
8a:be:71:4c:e0:20 vlan 1 extern_learn master storage0
8a:be:71:4c:e0:20 extern_learn master storage0
b2:33:bb:84:cc:38 vlan 1 extern_learn master storage0
b2:33:bb:84:cc:38 extern_learn master storage0
86:07:90:2b:db:db vlan 1 extern_learn master storage0
86:07:90:2b:db:db extern_learn master storage0
4a:28:60:90:76:42 vlan 1 extern_learn master storage0
4a:28:60:90:76:42 extern_learn master storage0
22:d6:49:9f:08:07 vlan 1 extern_learn master storage0
22:d6:49:9f:08:07 extern_learn master storage0
fe:4a:fb:63:9d:3a vlan 1 extern_learn master storage0
fe:4a:fb:63:9d:3a extern_learn master storage0
ee:78:b4:d8:3f:a0 vlan 1 master storage0 permanent
ee:78:b4:d8:3f:a0 master storage0 permanent
00:00:00:00:00:00 dst 172.2.0.24 self permanent
00:00:00:00:00:00 dst 172.2.0.26 self permanent
00:00:00:00:00:00 dst 172.2.0.28 self permanent
00:00:00:00:00:00 dst 172.2.0.30 self permanent
00:00:00:00:00:00 dst 172.2.0.32 self permanent
00:00:00:00:00:00 dst 172.2.0.59 self permanent
fe:4a:fb:63:9d:3a dst 172.2.0.24 self extern_learn
vtysh -c 'show interface vxlan1'
Interface vxlan1 is up, line protocol is up
Link ups: 1 last: 2025/01/06 23:53:01.17
Link downs: 1 last: 2025/01/06 23:53:01.17
vrf: default
index 14 metric 0 mtu 9050 speed 4294967295
flags: <UP,BROADCAST,RUNNING,MULTICAST>
Type: Ethernet
HWaddr: ea:d3:68:02:7d:f7
inet6 fe80::e8d3:68ff:fe02:7df7/64
Interface Type Vxlan
Interface Slave Type None
VxLAN Id 100 VTEP IP: 10.23.13.14 Access VLAN Id 1
protodown: off
vtysh -c 'show evpn vni 1'
VNI: 1
Type: L2
Tenant VRF: default
VxLAN interface: vxlan1
VxLAN ifIndex: 14
SVI interface: cloudbr1
SVI ifIndex: 12
Local VTEP IP: 10.23.13.14
Mcast group: 0.0.0.0
No remote VTEPs known for this VNI
Number of MACs (local and remote) known for this VNI: 0
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
Advertise-svi-macip: No
and I can ping the IPV6 that is routed using the FRR from :60 which is
in VXLAN 2 to :14 which is in VXLAN 1
ping -I 20XX:5XX:56XX:fff0::2:60 20XX:5XX:56XX:fff0:0:2:13:14
PING 20XX:5XX:56XX:fff0:0:2:13:14(20XX:5XX:56XX:fff0:0:2:13:14) from
20XX:5XX:56XX:fff0::2:60 : 56 data bytes
64 bytes from 20XX:5XX:56XX:fff0:0:2:13:14: icmp_seq=1 ttl=61
time=0.293 ms
64 bytes from 20XX:5XX:56XX:fff0:0:2:13:14: icmp_seq=2 ttl=61
time=0.222 ms
Then, my questions are:
are you using at the Leaf Switches/routers a regular Mapping VLAN to
VNI VXLAN with VRF ? IF not, Can you share a FRR config of your Switches?
Or should I use an enterprise SONIC switch software ?
What other possibilities are with the modifyvxlan.sh That Wido states
on some user mails.
Thank you
Tata Y.
