Thanks for the heads-up! Just to clarify: we want a 1:1 mapping. One
VLAN should correspond to exactly one VXLAN VNI.
The hint regarding Tag 0 was actually given to us by Juniper Support.
According to Juniper, the correct mapping from VNI to VLAN cannot take
place this way.
So, are you saying that FRR is doing the correct export here? What
surprises us is that both the RT and the VNI label are applied
correctly. This seems to be an issue between JunOS and FRR.
Creating a routing instance for each VNI is not practicable for us. We
adopted the Route Targets from a blog post by Wido.
Thanks for your help!
*Von: *Júlíus Þór Bess <[email protected]>
*Datum: *Donnerstag, 28. Mai 2026 um 17:30
*An: *[email protected] <[email protected]>
*Betreff: *Re: AW: AW: Re: VXLAN with EVPN Problem
FRR supports vlan-based service-type. That's the reason that type-2
routes don't include the VNI. The part after the colon in the RT's
refers to the VNI.
I haven't used vlan-aware bundle service-type myself but you might be
able to setup vlan-based service-type routing instances for each VNI
and import/export from there into your vlan-aware bundled instance.
Not very nice, but might get the job done. In that case route-targets
are automatically created on FRR side, so maybe that's a little
consolation. ;)
On 5/28/26 3:02 PM, Julian Sluimann wrote:
Hi Júlíus,
Yes, we intentionally use a single VLAN-aware bundle EVPN
instance. We were advised to keep it simple in one shared setup.
We do not need inter-VNI/VLAN routing, so splitting into multiple
EVPN instances would just add unnecessary complexity for our use
case. Our plan is to use VNIs as simple l2 domains - like VLAN. We
don't have to split this into different routing instances, because
they don't need to be isolated from one another.
ip -d link show dev vxlan4004055
19: vxlan4004055: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue master brvx-4004055 state UNKNOWN mode DEFAULT group
default qlen 1000
link/ether 36:d9:fe:f8:e0:6b brd ff:ff:ff:ff:ff:ff promiscuity
1 allmulti 1 minmtu 68 maxmtu 65535
info: Using default fan map value (33)
vxlan id 4004055 local 10.66.200.2 srcport 0 0 dstport 4789
nolearning ttl auto ageing 300 udpcsum noudp6zerocsumtx
noudp6zerocsumrx
bridge_slave state forwarding priority 32 cost 100 hairpin off
guard off root_block off fastleave off learning on flood on
port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0
designated_bridge 8000.92:49:a7:74:a0:f6 designated_root
8000.92:49:a7:74:a0:f6 hold_timer 0.00 message_age_timer
0.00 forward_delay_timer 0.00 topology_change_ack 0
config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1
mcast_fast_leave off mcast_flood on bcast_flood on
mcast_to_unicast off neigh_suppress off group_fwd_mask 0
group_fwd_mask_str 0x0 vlan_tunnel off isolated off locked off
addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536
gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535
gro_max_size 65536
*Julian Sluimann*
IT-Service
Phone Icon
0256498996-30
Email Icon
[email protected]
epnox_logo.png
Website Icon
epnox.de
Location Icon
Stadtlohner Str. 6, 48691 Vreden
epnox GmbH Stadtlohner Straße 6 48691 Vreden
Geschäftsführer: Gerd Gevering | Christian Meiners | Nils Waning |
Ust.-Nummer: DE351345026
St.Nr.: 301/5813/2061 | HRB 20199 | zertifiziert nach: ISO 27001
*Bitte prüfen Sie vorher, ob ein Ausdruck dieser E-Mail wirklich
notwendig ist.*
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte
Informationen. Wenn Sie nicht der richtige Adressat sind oder
diese E-Mail irrtümlich erhalten haben, informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das unerlaubte
Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht
gestattet. This e-mail may contain confidential and/or privileged
information. If you are not the intended recipient (or have
received this e-mail in error) please notify the sender
immediately and destroy this e-mail. Any unauthorised copying,
disclosure or distribution of the material in this e-mail is
strictly forbidden.
*Von: *Júlíus Þór Bess <[email protected]>
*Datum: *Donnerstag, 28. Mai 2026 um 16:30
*An: *[email protected] <[email protected]>
*Betreff: *Re: AW: Re: VXLAN with EVPN Problem
Hi Julian,
Why do you mix the VNI's into one RT? Are you trying to do vlan-bundle
service type on the FRR side?
Also, can you show ip -d link show dev <vxlan dev>
On 5/28/26 11:53 AM, Julian Sluimann wrote:
> Hey everyone,
>
> Yesterday we reviewed the whole situation together with Juniper
support. According to them, the overall setup and configuration on
the Juniper side looks correct. However, they suspect that the
issue is related to how FRR advertises the EVPN routes.
>
> Specifically, the Type-2 routes received from FRR do not include
the VNI in the route itself (see output below – EthTag/VNI is
shown as "0"). The switch appears to recognize the VNI as a route
label and learns the MAC addresses, but due to the missing VNI
field it does not properly associate the MAC entries with the EVPN
MAC-IP table. This would also explain why communication works
locally on the same switch, but not across EVPN/VXLAN between the
servers.
>
> The interesting part is that the VNI is shown correctly for the
MAC address "48:a9:8a:ef:75:73". However, this device is not
connected via VXLAN directly — it is a MikroTik board connected
through a regular Layer-2 VLAN which is mapped into the VNI on the
EX4650 side. Communication between the VMs and the MikroTik board
is only partially working (75% packet loss). Only the two VM MAC
addresses ("02:04:02:64:00:01" and "02:04:02:64:00:02") learned
from FRR are missing the VNI information in the Type-2 routes:
>
> show bgp l2vpn evpn route type 2
> BGP table version is 4, local router ID is 10.66.200.3
> Network Next Hop Metric LocPrf Weight Path
> Extended Community
> Route Distinguisher: 10.65.200.240:100
> *> [2]:[4004055]:[48]:[48:a9:8a:ef:75:73] RD 10.65.200.240:100
> 10.65.200.240 0 65400 i
> RT:100:100 ET:8
>
> Route Distinguisher: 10.66.200.2:3
> *> [2]:[0]:[48]:[02:04:02:64:00:01] RD 10.66.200.2:3
> 10.66.200.2 0 65400 65202 i
> RT:100:100 ET:8
>
> Route Distinguisher: 10.66.200.3:3
> *> [2]:[0]:[48]:[02:04:02:64:00:02] RD 10.66.200.3:3
> 10.66.200.3 32768 i
> ET:8 RT:100:100
>
> Juniper’s statement was:
>
> “Based on our findings, it appears that the servers are not
sending their Type-2 routes with the VNI included in the route,
which may be preventing the creation of entries in the EVPN MAC-IP
table. In addition, the next-hop entry seems to be missing from
the PFE.”
>
> We also discussed this with another engineer who has extensive
experience with Juniper and EVPN deployments. He also does not see
a fundamental issue on the Juniper side, but unfortunately he has
no practical experience with FRR in this specific scenario.
>
> Maybe some of you have experience with FRR? Do you use it in a
similar way to us? Maybe we just missing a small detail in the config:
>
> !
> frr version 10.6.0
> frr defaults traditional
> hostname cs-kvm-st-cl1-03
> log syslog informational
> log file /var/log/frr/debug.log debugging
> service integrated-vtysh-config
> !
> ip prefix-list LOOPBACKS seq 10 permit 10.0.0.0/8 le 32
> !
> interface bond1
> ip ospf area 0.0.0.51
> ip ospf network point-to-point
> no ipv6 nd suppress-ra
> exit
> !
> interface lo
> ip address 10.66.200.3/32
> exit
> !
> router bgp 65203
> bgp router-id 10.66.200.3
> no bgp ebgp-requires-policy
> no bgp default ipv4-unicast
> no bgp network import-check
> neighbor uplinks peer-group
> neighbor uplinks remote-as external
> neighbor uplinks ebgp-multihop
> neighbor uplinks update-source lo
> neighbor 10.65.200.240 peer-group uplinks
> !
> address-family l2vpn evpn
> neighbor uplinks activate
> neighbor uplinks next-hop-self
> advertise-all-vni
> advertise ipv4 unicast
> vni 4004070
> route-target import 100:100
> route-target export 100:100
> proxy-arp
> exit-vni
> vni 4004055
> route-target import 100:100
> route-target export 100:100
> advertise-svi-ip
> proxy-arp
> exit-vni
> vni 10030864
> route-target import 100:100
> route-target export 100:100
> proxy-arp
> exit-vni
> advertise-svi-ip
> exit-address-family
> exit
> !
> router ospf
> ospf router-id 10.66.200.3
> redistribute connected route-map OSPF_EXPORT
> exit
> !
> route-map OSPF_EXPORT permit 10
> match ip address prefix-list LOOPBACKS
> exit
> !
> end
>
> Thanks for your help!
>
> Von: Julian Sluimann <[email protected]>
> Datum: Dienstag, 19. Mai 2026 um 10:48
> An: [email protected] <[email protected]>
> Cc: Wido den Hollander <[email protected]>
> Betreff: RE: Re: VXLAN with EVPN Problem
>
> Hey Wido!
>
> Thanks for reply!
>
>> I must say I have never tried with a JunOS VC, but why would
you also?
>> This is a full L3 setup, correct? Why add the VC of JunOS?
> We are currently using two virtual chassis because it was an
existing setup with running redundant L2 connections.
> We still need these connections. Our plan was to migrate to a L3
environment now. In a new setup we would use four single devices too.
> Right now this is not possible because of production systems.
>
>> Which MACs, where? Who is your gateway?
> We are seeing all mac-addresses on the right sides in evpn
database. These are published to evpn fine.
>
>> Why not use this for the BGP session?
> We are using a bond because of the virual chassis on the switch
side. We wanted to use 2x25G connection. We are using OSPF to
learn Loopback IPs of devices.
> BGP connection via Loopback is the reason for one single bgp
session between switch and kvm host.
>
>> Did you set a vtep-source under switch-options to lo0.0?
> Yes, of course we set vtep Interface to loopback.
>
> set switch-options vtep-source-interface lo0.0
>
> Can you tell me something about your import/export rules or add
a snippet too?
> Right now we are not using any policies.
>
> Our next steps are: We will try to run bgp sessions via
interface addresses without OSPF. We will try with a single spare
device without VC.
> We thought about using iBGP too but decided to use eBGP because
of easier debugging.
>
> Thank you for your help!
>
> On 2026/05/19 04:14:35 Wido den Hollander via users wrote:
>>
>> Op 18-05-2026 om 11:25 schreef Julian Sluimann:
>>> Hi everyone,
>>>
>>> We are currently trying to expand our L2 VLAN-based CloudStack
environment to include EVPN VXLAN. We've run into a problem that
we can't seem to solve…
>>>
>>> But right from the start: We are using two Juniper Virtual
Chassis, each with two Juniper EX4650 switches. The KVM hosts
(Ubuntu 24.04 with FRRouting 10.6.1) use an LACP bond (bond1) for
guest traffic. We have included the script needed to create the
bridges. The bridge is created correctly; we see traffic from the
VM’s "vnet" interface passing through the bridge and then exiting
the bond encapsulated in VXLAN. However, the traffic simply does
not seem to arrive on the other side. It doesn't matter whether
the KVM hosts are connected to the same switch chassis. If the KVM
hosts communicate directly with each other via BGP, everything
works without any problems.
>>>
>> I do see some problems with FRR <> JunOS communicating with
BGP+EVPN,
>> but it can work.
>>
>> I must say I have never tried with a JunOS VC, but why would
you also?
>> This is a full L3 setup, correct? Why add the VC of JunOS?
>>
>>> We noticed that the ARP table within the VM is not populated
correctly. That's strange, because both the switches and the KVM
hosts have filled their ARP tables with all the MAC addresses. But
even if we add the missing MAC addresses of the VMs on both sides,
they still cannot communicate with each other. That doesn't seem
to be the problem, but perhaps it's a consequence of the actual
problem?
>> Which MACs, where? Who is your gateway?
>>
>> (see more below)
>>
>>> Our current FRR configuration looks like this:
>>>
>>> !
>>> frr version 10.6.0
>>> frr defaults traditional
>>> hostname kvm-h1
>>> log syslog informational
>>> service integrated-vtysh-config
>>> !
>>> ip prefix-list LOOPBACKS seq 10 permit 10.0.0.0/8 le 32
>>> !
>>> interface bond1
>>> ip ospf area 0.0.0.51
>>> ip ospf network point-to-point
>>> no ipv6 nd suppress-ra
>>> exit
>>> !
>>> interface lo
>>> ip address 10.66.200.3/32
>>> exit
>>> !
>>> router bgp 65203
>>> bgp router-id 10.66.200.3
>>> no bgp ebgp-requires-policy
>>> no bgp default ipv4-unicast
>>> no bgp network import-check
>>> neighbor uplinks peer-group
>>> neighbor uplinks remote-as external
>>> neighbor uplinks ebgp-multihop
>>> neighbor uplinks update-source lo
>> Correct? How does the EX learn this loopback address of the KVM
host?
>> Why not use the address of the interface for the uplink?
>>
>> And why a bond and not two seperate BGP sessions?
>>
>>> neighbor 10.65.200.250 peer-group uplinks
>>> neighbor 10.65.200.250 description SWITCH1
>>> !
>>> address-family l2vpn evpn
>>> neighbor uplinks activate
>>> advertise-all-vni
>>> vni 10003168
>>> route-target import 100:100
>>> route-target export 100:100
>>> proxy-arp
>>> exit-vni
>>> advertise-svi-ip
>>> exit-address-family
>>> exit
>>> !
>>> router ospf
>>> ospf router-id 10.66.200.3
>>> redistribute connected route-map OSPF_EXPORT
>>> exit
>>> !
>>> route-map OSPF_EXPORT permit 10
>>> match ip address prefix-list LOOPBACKS
>>> exit
>>> !
>>> end
>>>
>>> Our test VNI is 10003168. It contains two VMs, each on a
different KVM host, which in turn are connected to two different
virtual switches.
>>>
>>> The "bond1" interface is used, which is defined in the Netplan
as follows:
>>>
>>> bonds:
>>> bond1:
>>> mtu: "9000"
>>> interfaces:
>>> - ens2f1np1
>>> - eno3np1
>>> addresses:
>>> - "10.65.200.5/31"
>> Why not use this for the BGP session?
>>
>>> parameters:
>>> mode: "802.3ad"
>>> lacp-rate: "slow"
>>> transmit-hash-policy: "layer3+4"
>>>
>>> This is how our juniper configuration looks like:
>>>
>>> set protocols evpn no-core-isolation
>>> set protocols evpn encapsulation vxlan
>>> set protocols evpn default-gateway no-gateway-community
>>> set protocols evpn duplicate-mac-detection detection-threshold 5
>>> set protocols evpn duplicate-mac-detection detection-window 180
>>> set protocols evpn duplicate-mac-detection auto-recovery-time 15
>>> set protocols evpn multicast-mode ingress-replication
>>> set protocols evpn extended-vni-list 4004070
>>> set protocols evpn extended-vni-list 10003168
>>>
>>> set routing-options router-id 10.65.200.250
>>> set switch-options vrf-target target:100:100
>>> set switch-options route-distinguisher 10.65.200.250:100
>> Did you set a vtep-source under switch-options to lo0.0?
>>
>>> Session to EX4650 VC2
>>>
>>> set protocols bgp group BGP-SW-to-SW multihop ttl 2
>>> set protocols bgp group BGP-SW-to-SW multihop no-nexthop-change
>>> set protocols bgp group BGP-SW-to-SW family inet unicast
>>> set protocols bgp group BGP-SW-to-SW family evpn signaling
>>> set protocols bgp group BGP-SW-to-SW neighbor 10.65.100.250
description AS65100
>>> set protocols bgp group BGP-SW-to-SW neighbor 10.65.100.250
local-address 10.65.200.250
>>> set protocols bgp group BGP-SW-to-SW neighbor 10.65.100.250
peer-as 65100
>>>
>>> set protocols bgp group BGP-SW-to-KVM multihop ttl 2
>>> set protocols bgp group BGP-SW-to-KVM multihop no-nexthop-change
>>> set protocols bgp group BGP-SW-to-KVM family inet unicast
>>> set protocols bgp group BGP-SW-to-KVM family evpn signaling
>>> set protocols bgp group BGP-SW-to-KVM neighbor 10.66.200.3
description "AS65203"
>>> set protocols bgp group BGP-SW-to-KVM neighbor 10.66.200.3
local-address 10.65.200.250
>>> set protocols bgp group BGP-SW-to-KVM neighbor 10.66.200.3
peer-as 65203
>> A snippet of the config on a QFX5120
>>
>>
>> group compute {
>> type external;
>> multihop {
>> no-nexthop-change;
>> }
>> accept-remote-nexthop;
>> import compute-in;
>> family inet {
>> unicast {
>> extended-nexthop;
>> }
>> }
>> family inet6 {
>> unicast;
>> }
>> family evpn {
>> signaling;
>> }
>> export compute-out;
>> neighbor 2a01:XXX:2:180::7 {
>> description compute0;
>> local-address 2a01:XXX:2:180::6;
>> peer-as 64650;
>> }
>> }
>>
>> Here we peer over IPv6, so yes, slightly different.
>>
>>
>>>> set protocols bgp local-as 65200
>>> Does anyone have experience with EVPN-VXLAN and Juniper EX4650
switches? It’s probably a really silly problem… but we just can’t
figure it out.
>> As said, the QFX5120 gave me some troubles as well, but it works!
>>
>> Wido
>>
>>> Thanks!
>>>
>>
