Hi Albert, Would you mind keeping mailing list? I think it is very helpful for us to share your great perspective!
On 2017年01月31日 23:35, Albert Siersema wrote: > Hello Iwase, > > Thanks for all the work in getting the interoperability changes upstream. > > I wonder if you could help me out in taking the next step on how to implement > auto-RD and auto-generated import & export in Ryu. I fully understand you > having all kinds of other work to do as well, so pointers to where I can > add/alter Ryu code myself are very welcome too. > This is a bit of a lengthy email, but I hope I'm sending you enough > background information. The actual question is at the bottom of the email :-) > > Cisco/Juniper/Cumulus and undoubtedly more vendors offer a > simplification/generalization of the EVPN configuration by not having to > specify RD/RT's, e.g. in Cisco NX-OS syntax: > > evpn > vni 10311 l2 > rd auto > route-target import auto > route-target export auto > > See: > http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html > search for "MP-eBGP Design with VTEP Leaf Nodes in the Same BGP Autonomous > System" > > In short: > To further simplify configuration (and facilitate auto RD) all physical leaf > switches share the same ASN (e.g. 65511). All spine switches in turn share an > ASN as well (e.g. 65510). > This means an AS path has to be accepted with the local AS number in it. > Cisco includes an 'allowas-in' configuration statement. I've changed the Ryu > code to to allow this as well (I will submit a patch for this shortly). > > I've been sniffing traffic to see how it actually works on the BGP level: > > - "underlay network" reachability of VTEP is announced through regular IPv4 > BGP updates > e.g. BGPSpeaker.prefix_add(prefix=myLoopbackIPv4) > [ basic stuff, works ] > > - Valid VTEP endpoints (see BUM below) are announced through > EVPN_MULTICAST_ETAG_ROUTE/PMSI_TYPE_INGRESS_REP > BGP updates using IPv4 RDs, i.e. <myLoopbackIPv4> : <VNI> (VNI+22767 > actually) > e.g.: > evpn_route_dist='{}:{}'.format(myLoopbackIPv4, vni+22767) > speaker.vrf_add( > route_dist=evpn_route_dist, > route_family=RF_L2_EVPN, > import_rts=[evpn_route_dist], > export_rts=[evpn_route_dist] > ) > BGPSpeaker.evpn_prefix_add( > route_type=EVPN_MULTICAST_ETAG_ROUTE, > tunnel_type=TUNNEL_TYPE_VXLAN, > pmsi_tunnel_type=PMSI_TYPE_INGRESS_REP, > vni=vni, # sent through BGPPathAttributePmsiTunnel > route_dist=evpn_route_dist, > ethernet_tag_id=0, > ip_addr=myLoopbackIPv4, > next_hop=myLopbackIPv4, > ) > Cisco solves the BUM (e.g. IPv4 ARP) issue by having the VTEP's listen for > ARP requests and flooding them > to all other known valid VTEP's. The list of valid VTEP's is composed by > receiving the > EVPN_MULTICAST_ETAG_ROUTE updates. > Thanks to your help, this now works like a charm as well. I've yet to solve > the BUM issue when connecting > to the physical Cisco switches, but that's an OpenVSwitch issue. I know the > OpenStack people have been busy > here, like flooding and l2population & ARP responders. > However, if you do have any hints here, I'd be delighted to here about them. > > - Through the EVPN prefix route_type EVPN_MAC_IP_ADV_ROUTE only the MAC > address is advertised. > Also sending the ip_addr seems to be accepted for now. This in turn > facilitates e.g. Ryu-Ryu connections > thus avoiding BUM flooding there. > e.g. : > speaker.evpn_prefix_add( > route_type=EVPN_MAC_IP_ADV_ROUTE, > route_dist=evpn_route_dist, > ethernet_tag_id=0, > mac_addr=evpn_mac, > ip_addr=None, # or IP address as well > next_hop=loopback_IPv4, > ) > > This last step is where some changes to the Ryu code are necessary (I think). > Looking at the BGP update message, this is what Cisco sends us: > > Path Attribute: Extended Communities > Community Transitive Two-Octet AS Route Target: <ASN>:<VNI> <---- ! > Community type high: Two-Octet AS (0x00) > Subtype as2: Route Target (0x02) > Two octets AS specific: <ASN> > Four octets AN specific: <VNI> > Path Attribute: MP_REACH_NLRI > AFI: L2VPN (25) > SAFI: EVPN (70) > Next hop: <> > NLRI EVPN NLRI: MAC Advertisement Route > AFI: Mac Advertisement Route (2) > RD: <router-id>:<VNI+22767> <---- ! > ESI: 0 > Ethernet Tag ID: 0 > MAC Address: ... > IP Address: empty (length: 0) > MPLS Label Stack: 644 (bottom) > > In the extended communities a two-octet AS route target (RT) is send in the > format <ASN>:<VNI>. Auto import/export will use this. However, it also needs > the new IPv4:VNI RD in the MP_REACH_NLRI path attribute. > I managed to test this by hard coding some stuff in > ryu/services/protocols/bgp/utils/bgp.py : > > def create_rt_extended_community(value, subtype=2): > if subtype == 2: > value = "65511:10311" # i.e. <local AS>:<VNI> > > I'm wondering how I can pass this information from e.g. my evpn_prefix_add() > call ? I guess auto RT/RD creation might need some new API into BGPSpeaker, though... Ryu creates relationships between RD to export/import RT at BGPSpeaker.vrf_add() method. Then, how about the following settings? $ cat ryu/services/protocols/bgp/bgp_sample_conf.py ...(snip) # ============================================================================= # BGP configuration. # ============================================================================= BGP = { # AS number for this BGP instance. 'local_as': 65000, # the same Local ASN # BGP Router ID. 'router_id': '172.17.0.1', # List of BGP neighbors. # The parameters for each neighbor are the same as the arguments of # BGPSpeaker.neighbor_add() method. 'neighbors': [ { 'address': '172.17.0.2', 'remote_as': 65000, # the same Local ASN 'enable_evpn': True, }, ], # List of BGP VRF tables. # The parameters for each VRF table are the same as the arguments of # BGPSpeaker.vrf_add() method. 'vrfs': [ # Example of VRF for EVPN { 'route_dist': '172.17.0.1:22867', # <LoopbackIPv4> : <VNI+22767> 'import_rts': ['65000:100'], # <ASN>:<VNI> 'export_rts': ['65000:100'], 'route_family': RF_L2_EVPN, }, ], # List of BGP routes. # The parameters for each route are the same as the arguments of # BGPSpeaker.prefix_add() or BGPSpeaker.evpn_prefix_add() method. 'routes': [ # Example of EVPN prefix { 'route_type': EVPN_MULTICAST_ETAG_ROUTE, 'route_dist': '172.17.0.1:22867', # <LoopbackIPv4> : <VNI+22767> 'esi': 0, 'ethernet_tag_id': 0, 'ip_addr': '10.40.1.1', }, ], } ...(snip) $ ryu-manager --bgp-app-config-file ryu/services/protocols/bgp/bgp_sample_conf.py ryu/services/protocols/bgp/application.py loading app ryu/services/protocols/bgp/application.py instantiating app ryu/services/protocols/bgp/application.py of RyuBGPSpeaker API method core.start called with args: {'bgp_server_port': 179, 'local_as': 65000, 'router_id': '172.17.0.1', 'refresh_max_eor_time': 0, 'refresh_stalepath_time': 0, 'waiter': <ryu.lib.hub.Event object at 0x7f1057253a20>, 'label_range': (100, 100000)} API method neighbor.create called with args: {'peer_next_hop': None, 'ip_address': '172.17.0.2', 'is_next_hop_self': False, 'cap_mbgp_ipv4': True, 'cap_mbgp_vpnv4': False, 'cap_four_octet_as_number': True, 'cap_mbgp_evpn': True, 'cap_enhanced_refresh': False, 'cap_mbgp_ipv6': False, 'password': None, 'connect_mode': 'both', 'is_route_server_client': False, 'cap_mbgp_vpnv6': False, 'remote_as': 65000} API method vrf.create called with args: {'route_dist': '172.17.0.1:22867', 'export_rts': ['65000:100'], 'site_of_origins': None, 'import_rts': ['65000:100'], 'route_family': 'evpn', 'multi_exit_disc': None} API method evpn_prefix.add_local called with args: {'ethernet_tag_id': 0, 'route_dist': '172.17.0.1:22867', 'next_hop': '0.0.0.0', 'route_type': 'multicast_etag', 'ip_addr': '10.40.1.1'} starting ssh server at localhost:4990 ...(snip) Note: To run the above, required to apply the patches I had sent. Thanks, Iwase > > Regards, > Albert Siersema > > > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Ryu-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ryu-devel
