Nischal, The proposed RI / inet.0 routing configs seem to work fine in our environment. I'm anxious to see this in device-manager asap.
I believe at this point we need the following 3 critical things in device-manager: 1. Above discussed route leaking 2. virtual-gateway-address 3. Fix hard coded /24 that device-manager uses for IRB - can't believe this exists :) Until these are in place we are forced to hand configured VN's on our MXs. Thanks for your awesome work! Dan On Wed, Jul 15, 2015 at 4:04 PM, Nischal Sheth <[email protected]> wrote: > > Hi Dan, > > Here's config snippets for how traffic would cross between VRF and > inet.0. > I haven't included the configs for virtual-switch (i.e. L2) instances for > public1 > and public2. In the example below, irb1 and irb2 are the > routing-interfaces > the BDs in the virtual-switch instances. > > 10.1.1/24 is subnet assigned to public1 > 10.1.2/24 is subnet assigned to public2 > > I've also not included detailed VRF configs for public1 and public2 > (e.g. vrf > import and export policies), just the portions needed for routing. > > Let me know if you have questions. > > -Nischal > > [edit interfaces] > + irb { > + unit 1 { > + family inet { > + address 10.1.1.254/24; > + } > + } > + unit 2 { > + family inet { > + address 10.1.2.254/24; > + } > + } > + } > > [edit forwarding-options family inet filter] > + input redirect-to-public-vrfs; > > [edit firewall] > > + filter redirect-to-public-vrfs { > + /* redirect traffic to public1 */ > + term t1 { > + from { > + destination-address { > + 10.1.1.0/24; > + } > + } > + then { > + routing-instance public1; > + } > + } > + /* redirect traffic to public2 */ > + term t2 { > + from { > + destination-address { > + 10.1.2.0/24; > + } > + } > + then { > + routing-instance public2; > + } > + } > + /* continue lookup in current table (inet.0) */ > + term default { > + then accept; > + } > + } > > [edit routing-instances] > + public1 { > + interface irb.1; > + routing-options { > + static { > + route 0.0.0.0/0 next-table inet.0; > + } > + } > + } > + public2 { > + interface irb.2; > + routing-options { > + static { > + route 0.0.0.0/0 next-table inet.0; > + } > + } > + } > > > On Jul 14, 2015, at 12:45 AM, Dan Houtz <[email protected]> wrote: > > Hi Nischal, > > Thank you for the detailed reply. I may have a few follow up questions in > the coming days but for now I only have two: > > 1. Do you have an example config of what you are thinking DM might push to > the MX to route between VRF and inet.0? Even if I could apply that by > default right now it would be helpful. > > 2. Any ETA on the bugs you referenced and will there be a reasonably easy > to make use of these prior to official 2.21 packages being released? > > Thanks! > Dan > On Jul 14, 2015 12:29 AM, "Nischal Sheth" <[email protected]> wrote: > >> >> On Jul 9, 2015, at 12:25 PM, Dan Houtz <[email protected]> wrote: >> >> Hi Nischal, >> >> >> Hi Dan, >> >> Pleas see inline. >> >> >> I'll need to re-read through the bug text a few more times to fully grasp >> as I have yet to totally understand the difference between L2 and L2/L3 >> networks from the Contrail perspective. It seems all networks we create >> via the Contrail webui are L2/L3. I see there is an L2 option when >> configured a port but this requires me to create IP/MAC mappings which >> doesn't apply to us. Perhaps this is just a limitation of the UI and I >> could accomplish L2 only via API calls? >> >> >> I gather from your follow-up email on DHCP that you have a good handle >> on the >> difference between L2 and L2/L3 in Contrail now. In terms of DM, the >> difference >> should be that DM will only configure a virtual-switch routing-instance >> for L2 only >> VNs, whereas it would configure both virtual-switch and vrf >> routing-instances for >> L2+L3 VNs. Further, we're also adding a L3 only mode where DM will create >> only >> a vrf. >> >> >> In our use case we are doing overlays between all bare metal servers - >> we do not have any Openstack/VM elements. We will also generally be >> assigning publicly routed IP's directly to all servers and will not require >> NAT. There may be cases where customers need to build 'backnets' with >> private IP addresses. I suspect most of these to be islands where routing >> don't be required but we may need to extend these to our MXs for routing >> between private networks belonging to a specific customer. >> >> >> Please see below. >> >> >> Based on the above thoughts I believe we will probably have 3 primary >> scenarios: >> >> 1. Create a 'public' virtual network in Contrail, assign a public IP >> block to it, and have it be publicly routable - prefixes of VN exported >> into inet.0 and default from inet.0 imported into routing instance. It >> seems like creating a network, marking it as external, and not creating >> floating IPs would indicate this type of setup but I haven't actually >> looked at how Contrail does NAT/Floating IPs/etc. >> >> >> This should have worked, but DM currently treats external VNs as L3 only >> VNs and >> does not create a virtual-switch routing-instance. The bug I created >> earlier tracks this >> issue. >> >> https://bugs.launchpad.net/juniperopenstack/+bug/1472699 >> >> The bug also describes the routing configuration to allow the VN subnet >> to be added >> or imported into inet.0 and traffic redirection from inet.0 to the VRF. >> We need to work >> around some JUNOS quirks wherein 2 route tables cannot have static routes >> that >> point to each other. I've assumed that the VRF will have a static >> default that points to >> inet.0. If you're doing manual configuration, you could use rib-groups to >> import default >> route from BGP/OSPF into the VRF, but DM uses static default with a table >> next-hop >> since it doesn't want to assume anything about routing protocol being >> used for inet.0. >> >> Traffic destined to the VRF subnet will be redirected from inet.0 to >> the VRF using filter >> based forwarding by applying an input filter on inet.0. This filter >> matches the destination >> subnet and redirects traffic to the appropriate VRF. >> >> >> >> 2. Create a 'private' virtual network in Contrail, assign a private IP >> block to it and have it be routable only between other networks belonging >> to the tenant it is created under. I think this would generally be an all >> or nothing thing - every private network belonging to a tenant can reach >> any other private network belonging to that tenant. I could see someone >> wanting more fine grain control but probably not super common and would >> mostly look at inline firewall chaining or policy to control this >> >> >> This is supported today, and is finer grained than all or nothing. You >> can use policy to >> allow traffic between specific VNs. Matches on L4 information are not yet >> supported. >> >> 3. Create a 'backnet' virtual network in Contrail, assign no IP's - >> simply extended L2 to appropriate bare metal servers. Servers could be IP'd >> adhoc without Contrail being aware at all. Network would be an island >> with no config existing on MX. >> >> >> This should work today (modulo issues you found with Praveen) if you do >> not extend >> such VNs to the MXs. It will be further refined as part of the fix for >> bugs 1472699 and >> 1471637. If a VN is L2 only, you will not need to configure a subnet for >> it. >> >> >> Let me know if the above makes any sense :) >> >> >> Yes, the requirements sound reasonable and I think that you should be >> able to achieve >> what you are looking once 1472699 and 1471637 are fixed. >> >> -Nischal >> >> Thanks! >> >> On Wed, Jul 8, 2015 at 11:58 AM, Nischal Sheth <[email protected]> >> wrote: >> >>> >>> Hi Dan, >>> >>> https://bugs.launchpad.net/juniperopenstack/+bug/1472699 >>> >>> This needs some changes to the device manager. >>> >>> Could you please take a look and provide feedback on whether >>> the solution outlined will meet your requirements? >>> >>> Thanks, >>> Nischal >>> >>> On Jul 7, 2015, at 5:54 PM, Dan Houtz <[email protected]> wrote: >>> >>> Unless I'm overlooking something, It doesn't look like device-manager >>> builds the config needed to import a default route from inet.0 into the >>> VN's routing instance or to import the routing instance's prefix into >>> inet.0. In our case we are assigning public IP's directly to the bare metal >>> servers and do not require Is it possible for device-manager to configure >>> the import/export policies to make these routes available? >>> >>> Thanks! >>> Dan >>> >>> On Tue, Jul 7, 2015 at 4:55 PM, Nischal Sheth <[email protected]> >>> wrote: >>> >>>> >>>> Hi Dan, >>>> >>>> This confirms that the problem is related to route target filtering. >>>> It's >>>> a JUNOSism that bgp.rtarget.0 routes are resolved using inet.3 by >>>> default. >>>> >>>> Using gr-* interfaces will result in creation of inet.3 routes to the >>>> CN IP >>>> since DM adds CN IPs to the dynamic-tunnel destination network list. >>>> In this case the gr tunnels wouldn't be used in the data plane since you >>>> are using VXLAN. If you also have VMs, MX will use gr tunnels for >>>> data traffic to vrouters. >>>> >>>> If it's undesirable to create gr- tunnels, you have 2 options: >>>> >>>> 1) You can configure JUNOS to resolve bgp.rtarget.0 over inet.0 >>>> instead >>>> of inet.3 using something like: >>>> >>>> root@a3-mx80-1# show routing-options resolution >>>> rib bgp.rtarget.0 { >>>> resolution-ribs inet.0; >>>> } >>>> >>>> 2) Alternately, you can add add static routes with discard nexthop to >>>> CN >>>> ip addresses into inet.3. >>>> >>>> root@a3-mx80-1# show routing-options rib inet.3 >>>> static { >>>> route 10.10.210.140/32 discard; >>>> } >>>> >>>> [edit] >>>> >>>> I would recommend the latter. >>>> >>>> -Nischal >>>> >>>> On Jul 7, 2015, at 2:36 PM, Dan Houtz <[email protected]> wrote: >>>> >>>> root@gw2z0> show route table bgp.rtarget.0 >>>> >>>> bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) >>>> + = Active Route, - = Last Active, * = Both >>>> >>>> 65412:65412:1/96 >>>> *[RTarget/5] 01:36:41 >>>> Type Default >>>> Local >>>> 65412:65412:8000001/96 >>>> *[RTarget/5] 01:36:41 >>>> Type Default >>>> Local >>>> >>>> root@gw2z0> show route table bgp.rtarget.0 hidden >>>> >>>> bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) >>>> + = Active Route, - = Last Active, * = Both >>>> >>>> 65412:65412:1/96 >>>> [BGP/170] 02:26:54, localpref 100, from >>>> 10.10.210.140 >>>> AS path: I, validation-state: unverified >>>> Unusable >>>> 65412:65412:8000001/96 >>>> [BGP/170] 02:26:54, localpref 100, from >>>> 10.10.210.140 >>>> AS path: I, validation-state: unverified >>>> Unusable >>>> >>>> >>>> I don't believe I should have any gr interfaces as I'm doing VXLAN (no >>>> MPLS/GRE/etc) correct? >>>> >>>> >>>> >>>> On Tue, Jul 7, 2015 at 4:10 PM, Nischal Sheth <[email protected]> >>>> wrote: >>>> >>>>> >>>>> Hi Dan, >>>>> >>>>> Since there's no routes being sent to the CNs, there may be a problem >>>>> with route target filtering, which makes the MX think that the CN is >>>>> not >>>>> interested in the any route targets. >>>>> >>>>> Can you run "show route table bgp.rtarget.0" on the MX and check if >>>>> there are any hidden routes in that table? If there are, we need to >>>>> check (show interfaces terse gr-*) if there's any gr-* devices on the >>>>> system. If there aren't any, can you add them using something like: >>>>> >>>>> fpc 1 { >>>>> pic 0 { >>>>> tunnel-services; >>>>> } >>>>> } >>>>> >>>>> -Nischal >>>>> >>>>> On Jul 7, 2015, at 12:47 PM, Dan Houtz <[email protected]> wrote: >>>>> >>>>> I was able to get updated MX code (14.2-20150704.0) and now have >>>>> device-manager successfully configuring my MX80. BGP session between MX >>>>> and >>>>> Contrail also seems to be stable now however I am having an issue with >>>>> reach-ability between MX and hosts connected to TOR switches. Based on >>>>> ititial troubleshooting I don't believe Junos is announcing the EVPN route >>>>> for the IRB interface: >>>>> >>>>> oot@gw2z0# show groups __contrail__ interfaces irb >>>>> gratuitous-arp-reply; >>>>> unit 4 { >>>>> family inet { >>>>> address 10.10.210.145/29; >>>>> } >>>>> } >>>>> >>>>> >>>>> root@gw2z0# run show route advertising-protocol bgp 10.10.210.140 >>>>> >>>>> bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 >>>>> hidden) >>>>> Prefix Nexthop MED Lclpref AS >>>>> path >>>>> 65412:65412:1/96 >>>>> * Self 100 I >>>>> 65412:65412:8000001/96 >>>>> * Self 100 I >>>>> >>>>> >>>>> IRB interface is up: >>>>> >>>>> root@gw2z0# run show interfaces routing | grep irb >>>>> irb.4 Up INET 10.10.210.145 >>>>> >>>>> root@gw2z0# run show route 10.10.210.145 >>>>> >>>>> _contrail_l3_4_Test.inet.0: 2 destinations, 3 routes (2 active, 0 >>>>> holddown, 0 hidden) >>>>> + = Active Route, - = Last Active, * = Both >>>>> >>>>> 10.10.210.145/32 *[Local/0] 00:02:51 >>>>> Local via irb.4 >>>>> >>>>> >>>>> >>>>> On Sat, Jul 4, 2015 at 1:10 PM, Nischal Sheth <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>>> https://bugs.launchpad.net/juniperopenstack/+bug/1465070 >>>>>> >>>>>> -Nischal >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> On Jul 4, 2015, at 11:04 AM, Dan Houtz <[email protected]> wrote: >>>>>> >>>>>> Great! Next question... >>>>>> >>>>>> Are there plans to add in ability to apply the >>>>>> 'virtual-gateway-address' knob when configuring IRBs? I believe this is >>>>>> the >>>>>> recommended way to configure redundant MX gateways correct? >>>>>> >>>>>> -Dan >>>>>> >>>>>> On Sat, Jul 4, 2015 at 9:15 AM, Vedamurthy Ananth Joshi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Yes…this should be addressed too. >>>>>>> >>>>>>> Vedu >>>>>>> >>>>>>> From: Dan Houtz <[email protected]> >>>>>>> Date: Saturday, July 4, 2015 at 7:38 PM >>>>>>> To: Vedamurthy Ananth Joshi <[email protected]> >>>>>>> Cc: OpenContrail Users List - 2 <[email protected]> >>>>>>> Subject: Re: [Users] Problem with Device Manager's VXLAN config in >>>>>>> Contrail 2.2 >>>>>>> >>>>>>> Vedu, >>>>>>> >>>>>>> Thank you for the information. I have reached out to our SE to see >>>>>>> about getting updated code. I am also seeing the following with BGP >>>>>>> sessions between Contrail and MX since moving to 2.2: >>>>>>> >>>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: RPD_BGP_NEIGHBOR_STATE_CHANGED: >>>>>>> BGP peer 10.10.210.140 (Internal AS 65412) changed state from >>>>>>> OpenConfirm >>>>>>> to Established (event RecvKeepAlive) (instance master) >>>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: bgp_read_v4_update:10535: >>>>>>> NOTIFICATION sent to 10.10.210.140 (Internal AS 65412): code 3 (Update >>>>>>> Message Error) subcode 9 (error with optional attribute), Data: c0 16 >>>>>>> 09 >>>>>>> 10 fc 00 >>>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: RPD_BGP_NEIGHBOR_STATE_CHANGED: >>>>>>> BGP peer 10.10.210.140 (Internal AS 65412) changed state from >>>>>>> Established >>>>>>> to Idle (event RecvUpdate) (instance master) >>>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: Received malformed update from >>>>>>> 10.10.210.140 (Internal AS 65412) >>>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: Family evpn, prefix >>>>>>> 3:10.10.210.140:1::4::10.10.214.65/152 >>>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: Malformed Attribute PMSI(22) >>>>>>> flag 0xc0 length 9. >>>>>>> Jul 4 14:06:52 gw2z0 rpd[86503]: bgp_parse_open_options: peer >>>>>>> 10.10.210.140+50620 (proto): unsupported AF 1 SAFI 243 >>>>>>> >>>>>>> Is this something that will also be fixed with the new MX code? >>>>>>> >>>>>>> Thanks! >>>>>>> Dan >>>>>>> >>>>>>> On Sat, Jul 4, 2015 at 8:02 AM, Vedamurthy Ananth Joshi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Dan, >>>>>>>> Ingress-node-replication was not pushed by Device Manager on >>>>>>>> purpose. >>>>>>>> The corresponding MX image could be any daily build equal to or >>>>>>>> greater than 14.2-20150627.0. >>>>>>>> >>>>>>>> Vedu >>>>>>>> >>>>>>>> From: Dan Houtz <[email protected]> >>>>>>>> Date: Saturday, July 4, 2015 at 1:47 PM >>>>>>>> To: OpenContrail Users List - 2 <[email protected]> >>>>>>>> Subject: [Users] Problem with Device Manager's VXLAN config in >>>>>>>> Contrail 2.2 >>>>>>>> >>>>>>>> Has anyone else tried configuring EVPN VXLAN on an MX using >>>>>>>> device manager in Contrail 2.2? In my testing the configuration being >>>>>>>> pushed my netconf is not valid: >>>>>>>> >>>>>>>> root@gw2z0# commit check >>>>>>>> [edit routing-instances _contrail_l2_4_Test bridge-domains bd-4] >>>>>>>> 'vxlan' >>>>>>>> multicast-group or ovsdb-managed or ingress-node-replication >>>>>>>> should be enabled >>>>>>>> error: configuration check-out failed: (statements constraint check >>>>>>>> failed) >>>>>>>> >>>>>>>> To fix this you must manually configure ingress-node-replication: >>>>>>>> >>>>>>>> root@gw2z0# set groups __contrail__ routing-instances >>>>>>>> _contrail_l2_4_Test bridge-domains bd-4 vxlan ingress-node-replication >>>>>>>> >>>>>>>> root@gw2z0# commit check >>>>>>>> configuration check succeeds >>>>>>>> >>>>>>>> Is this possibly MX junos version specific? I am using a daily >>>>>>>> build given to me by my SE as I don't believe any released versions >>>>>>>> support >>>>>>>> VXLAN: >>>>>>>> >>>>>>>> root@gw2z0# run show version >>>>>>>> Hostname: gw2z0 >>>>>>>> Model: mx80-48t >>>>>>>> Junos: 14.2-20150527_rpd_v2_evpn_vnid.0 >>>>>>>> >>>>>>>> I doubt it matters but it's also odd that device manager is >>>>>>>> applying this since I'm using VXLAN: >>>>>>>> >>>>>>>> root@gw2z0# show groups __contrail__ protocols mpls >>>>>>>> interface all; >>>>>>>> >>>>>>>> Thanks! >>>>>>>> Dan >>>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing list >>>>>> [email protected] >>>>>> >>>>>> http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >
_______________________________________________ Users mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
