Hi Dan, Good to know that it works.
Will get back to you with ETAs for 1 and 2 mid next week. Fix for 3 is under review - should get merged in the next day or two. -Nischal On Jul 16, 2015, at 1:45 PM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: Nischal, The proposed RI / inet.0 routing configs seem to work fine in our environment. I'm anxious to see this in device-manager asap. I believe at this point we need the following 3 critical things in device-manager: 1. Above discussed route leaking 2. virtual-gateway-address 3. Fix hard coded /24 that device-manager uses for IRB - can't believe this exists :) Until these are in place we are forced to hand configured VN's on our MXs. Thanks for your awesome work! Dan On Wed, Jul 15, 2015 at 4:04 PM, Nischal Sheth <[email protected]<mailto:[email protected]>> wrote: Hi Dan, Here's config snippets for how traffic would cross between VRF and inet.0. I haven't included the configs for virtual-switch (i.e. L2) instances for public1 and public2. In the example below, irb1 and irb2 are the routing-interfaces the BDs in the virtual-switch instances. 10.1.1/24 is subnet assigned to public1 10.1.2/24 is subnet assigned to public2 I've also not included detailed VRF configs for public1 and public2 (e.g. vrf import and export policies), just the portions needed for routing. Let me know if you have questions. -Nischal [edit interfaces] + irb { + unit 1 { + family inet { + address 10.1.1.254/24<http://10.1.1.254/24>; + } + } + unit 2 { + family inet { + address 10.1.2.254/24<http://10.1.2.254/24>; + } + } + } [edit forwarding-options family inet filter] + input redirect-to-public-vrfs; [edit firewall] + filter redirect-to-public-vrfs { + /* redirect traffic to public1 */ + term t1 { + from { + destination-address { + 10.1.1.0/24<http://10.1.1.0/24>; + } + } + then { + routing-instance public1; + } + } + /* redirect traffic to public2 */ + term t2 { + from { + destination-address { + 10.1.2.0/24<http://10.1.2.0/24>; + } + } + then { + routing-instance public2; + } + } + /* continue lookup in current table (inet.0) */ + term default { + then accept; + } + } [edit routing-instances] + public1 { + interface irb.1; + routing-options { + static { + route 0.0.0.0/0<http://0.0.0.0/0> next-table inet.0; + } + } + } + public2 { + interface irb.2; + routing-options { + static { + route 0.0.0.0/0<http://0.0.0.0/0> next-table inet.0; + } + } + } On Jul 14, 2015, at 12:45 AM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: Hi Nischal, Thank you for the detailed reply. I may have a few follow up questions in the coming days but for now I only have two: 1. Do you have an example config of what you are thinking DM might push to the MX to route between VRF and inet.0? Even if I could apply that by default right now it would be helpful. 2. Any ETA on the bugs you referenced and will there be a reasonably easy to make use of these prior to official 2.21 packages being released? Thanks! Dan On Jul 14, 2015 12:29 AM, "Nischal Sheth" <[email protected]<mailto:[email protected]>> wrote: On Jul 9, 2015, at 12:25 PM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: Hi Nischal, Hi Dan, Pleas see inline. I'll need to re-read through the bug text a few more times to fully grasp as I have yet to totally understand the difference between L2 and L2/L3 networks from the Contrail perspective. It seems all networks we create via the Contrail webui are L2/L3. I see there is an L2 option when configured a port but this requires me to create IP/MAC mappings which doesn't apply to us. Perhaps this is just a limitation of the UI and I could accomplish L2 only via API calls? I gather from your follow-up email on DHCP that you have a good handle on the difference between L2 and L2/L3 in Contrail now. In terms of DM, the difference should be that DM will only configure a virtual-switch routing-instance for L2 only VNs, whereas it would configure both virtual-switch and vrf routing-instances for L2+L3 VNs. Further, we're also adding a L3 only mode where DM will create only a vrf. In our use case we are doing overlays between all bare metal servers - we do not have any Openstack/VM elements. We will also generally be assigning publicly routed IP's directly to all servers and will not require NAT. There may be cases where customers need to build 'backnets' with private IP addresses. I suspect most of these to be islands where routing don't be required but we may need to extend these to our MXs for routing between private networks belonging to a specific customer. Please see below. Based on the above thoughts I believe we will probably have 3 primary scenarios: 1. Create a 'public' virtual network in Contrail, assign a public IP block to it, and have it be publicly routable - prefixes of VN exported into inet.0 and default from inet.0 imported into routing instance. It seems like creating a network, marking it as external, and not creating floating IPs would indicate this type of setup but I haven't actually looked at how Contrail does NAT/Floating IPs/etc. This should have worked, but DM currently treats external VNs as L3 only VNs and does not create a virtual-switch routing-instance. The bug I created earlier tracks this issue. https://bugs.launchpad.net/juniperopenstack/+bug/1472699 The bug also describes the routing configuration to allow the VN subnet to be added or imported into inet.0 and traffic redirection from inet.0 to the VRF. We need to work around some JUNOS quirks wherein 2 route tables cannot have static routes that point to each other. I've assumed that the VRF will have a static default that points to inet.0. If you're doing manual configuration, you could use rib-groups to import default route from BGP/OSPF into the VRF, but DM uses static default with a table next-hop since it doesn't want to assume anything about routing protocol being used for inet.0. Traffic destined to the VRF subnet will be redirected from inet.0 to the VRF using filter based forwarding by applying an input filter on inet.0. This filter matches the destination subnet and redirects traffic to the appropriate VRF. 2. Create a 'private' virtual network in Contrail, assign a private IP block to it and have it be routable only between other networks belonging to the tenant it is created under. I think this would generally be an all or nothing thing - every private network belonging to a tenant can reach any other private network belonging to that tenant. I could see someone wanting more fine grain control but probably not super common and would mostly look at inline firewall chaining or policy to control this This is supported today, and is finer grained than all or nothing. You can use policy to allow traffic between specific VNs. Matches on L4 information are not yet supported. 3. Create a 'backnet' virtual network in Contrail, assign no IP's - simply extended L2 to appropriate bare metal servers. Servers could be IP'd adhoc without Contrail being aware at all. Network would be an island with no config existing on MX. This should work today (modulo issues you found with Praveen) if you do not extend such VNs to the MXs. It will be further refined as part of the fix for bugs 1472699 and 1471637. If a VN is L2 only, you will not need to configure a subnet for it. Let me know if the above makes any sense :) Yes, the requirements sound reasonable and I think that you should be able to achieve what you are looking once 1472699 and 1471637 are fixed. -Nischal Thanks! On Wed, Jul 8, 2015 at 11:58 AM, Nischal Sheth <[email protected]<mailto:[email protected]>> wrote: Hi Dan, https://bugs.launchpad.net/juniperopenstack/+bug/1472699 This needs some changes to the device manager. Could you please take a look and provide feedback on whether the solution outlined will meet your requirements? Thanks, Nischal On Jul 7, 2015, at 5:54 PM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: Unless I'm overlooking something, It doesn't look like device-manager builds the config needed to import a default route from inet.0 into the VN's routing instance or to import the routing instance's prefix into inet.0. In our case we are assigning public IP's directly to the bare metal servers and do not require Is it possible for device-manager to configure the import/export policies to make these routes available? Thanks! Dan On Tue, Jul 7, 2015 at 4:55 PM, Nischal Sheth <[email protected]<mailto:[email protected]>> wrote: Hi Dan, This confirms that the problem is related to route target filtering. It's a JUNOSism that bgp.rtarget.0 routes are resolved using inet.3 by default. Using gr-* interfaces will result in creation of inet.3 routes to the CN IP since DM adds CN IPs to the dynamic-tunnel destination network list. In this case the gr tunnels wouldn't be used in the data plane since you are using VXLAN. If you also have VMs, MX will use gr tunnels for data traffic to vrouters. If it's undesirable to create gr- tunnels, you have 2 options: 1) You can configure JUNOS to resolve bgp.rtarget.0 over inet.0 instead of inet.3 using something like: root@a3-mx80-1# show routing-options resolution rib bgp.rtarget.0 { resolution-ribs inet.0; } 2) Alternately, you can add add static routes with discard nexthop to CN ip addresses into inet.3. root@a3-mx80-1# show routing-options rib inet.3 static { route 10.10.210.140/32<http://10.10.210.140/32> discard; } [edit] I would recommend the latter. -Nischal On Jul 7, 2015, at 2:36 PM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: root@gw2z0> show route table bgp.rtarget.0 bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) + = Active Route, - = Last Active, * = Both 65412:65412:1/96 *[RTarget/5] 01:36:41 Type Default Local 65412:65412:8000001/96 *[RTarget/5] 01:36:41 Type Default Local root@gw2z0> show route table bgp.rtarget.0 hidden bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) + = Active Route, - = Last Active, * = Both 65412:65412:1/96 [BGP/170] 02:26:54, localpref 100, from 10.10.210.140 AS path: I, validation-state: unverified Unusable 65412:65412:8000001/96 [BGP/170] 02:26:54, localpref 100, from 10.10.210.140 AS path: I, validation-state: unverified Unusable I don't believe I should have any gr interfaces as I'm doing VXLAN (no MPLS/GRE/etc) correct? On Tue, Jul 7, 2015 at 4:10 PM, Nischal Sheth <[email protected]<mailto:[email protected]>> wrote: Hi Dan, Since there's no routes being sent to the CNs, there may be a problem with route target filtering, which makes the MX think that the CN is not interested in the any route targets. Can you run "show route table bgp.rtarget.0" on the MX and check if there are any hidden routes in that table? If there are, we need to check (show interfaces terse gr-*) if there's any gr-* devices on the system. If there aren't any, can you add them using something like: fpc 1 { pic 0 { tunnel-services; } } -Nischal On Jul 7, 2015, at 12:47 PM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: I was able to get updated MX code (14.2-20150704.0) and now have device-manager successfully configuring my MX80. BGP session between MX and Contrail also seems to be stable now however I am having an issue with reach-ability between MX and hosts connected to TOR switches. Based on ititial troubleshooting I don't believe Junos is announcing the EVPN route for the IRB interface: oot@gw2z0# show groups __contrail__ interfaces irb gratuitous-arp-reply; unit 4 { family inet { address 10.10.210.145/29<http://10.10.210.145/29>; } } root@gw2z0# run show route advertising-protocol bgp 10.10.210.140 bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) Prefix Nexthop MED Lclpref AS path 65412:65412:1/96 * Self 100 I 65412:65412:8000001/96 * Self 100 I IRB interface is up: root@gw2z0# run show interfaces routing | grep irb irb.4 Up INET 10.10.210.145 root@gw2z0# run show route 10.10.210.145 _contrail_l3_4_Test.inet.0: 2 destinations, 3 routes (2 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 10.10.210.145/32<http://10.10.210.145/32> *[Local/0] 00:02:51 Local via irb.4 On Sat, Jul 4, 2015 at 1:10 PM, Nischal Sheth <[email protected]<mailto:[email protected]>> wrote: https://bugs.launchpad.net/juniperopenstack/+bug/1465070 -Nischal Sent from my iPhone On Jul 4, 2015, at 11:04 AM, Dan Houtz <[email protected]<mailto:[email protected]>> wrote: Great! Next question... Are there plans to add in ability to apply the 'virtual-gateway-address' knob when configuring IRBs? I believe this is the recommended way to configure redundant MX gateways correct? -Dan On Sat, Jul 4, 2015 at 9:15 AM, Vedamurthy Ananth Joshi <[email protected]<mailto:[email protected]>> wrote: Yes…this should be addressed too. Vedu From: Dan Houtz <[email protected]<mailto:[email protected]>> Date: Saturday, July 4, 2015 at 7:38 PM To: Vedamurthy Ananth Joshi <[email protected]<mailto:[email protected]>> Cc: OpenContrail Users List - 2 <[email protected]<mailto:[email protected]>> Subject: Re: [Users] Problem with Device Manager's VXLAN config in Contrail 2.2 Vedu, Thank you for the information. I have reached out to our SE to see about getting updated code. I am also seeing the following with BGP sessions between Contrail and MX since moving to 2.2: Jul 4 14:06:47 gw2z0 rpd[86503]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 10.10.210.140 (Internal AS 65412) changed state from OpenConfirm to Established (event RecvKeepAlive) (instance master) Jul 4 14:06:47 gw2z0 rpd[86503]: bgp_read_v4_update:10535: NOTIFICATION sent to 10.10.210.140 (Internal AS 65412): code 3 (Update Message Error) subcode 9 (error with optional attribute), Data: c0 16 09 10 fc 00 Jul 4 14:06:47 gw2z0 rpd[86503]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 10.10.210.140 (Internal AS 65412) changed state from Established to Idle (event RecvUpdate) (instance master) Jul 4 14:06:47 gw2z0 rpd[86503]: Received malformed update from 10.10.210.140 (Internal AS 65412) Jul 4 14:06:47 gw2z0 rpd[86503]: Family evpn, prefix 3:10.10.210.140:1::4::10.10.214.65/152<http://10.10.214.65/152> Jul 4 14:06:47 gw2z0 rpd[86503]: Malformed Attribute PMSI(22) flag 0xc0 length 9. Jul 4 14:06:52 gw2z0 rpd[86503]: bgp_parse_open_options: peer 10.10.210.140+50620 (proto): unsupported AF 1 SAFI 243 Is this something that will also be fixed with the new MX code? Thanks! Dan On Sat, Jul 4, 2015 at 8:02 AM, Vedamurthy Ananth Joshi <[email protected]<mailto:[email protected]>> wrote: Dan, Ingress-node-replication was not pushed by Device Manager on purpose. The corresponding MX image could be any daily build equal to or greater than 14.2-20150627.0. Vedu From: Dan Houtz <[email protected]<mailto:[email protected]>> Date: Saturday, July 4, 2015 at 1:47 PM To: OpenContrail Users List - 2 <[email protected]<mailto:[email protected]>> Subject: [Users] Problem with Device Manager's VXLAN config in Contrail 2.2 Has anyone else tried configuring EVPN VXLAN on an MX using device manager in Contrail 2.2? In my testing the configuration being pushed my netconf is not valid: root@gw2z0# commit check [edit routing-instances _contrail_l2_4_Test bridge-domains bd-4] 'vxlan' multicast-group or ovsdb-managed or ingress-node-replication should be enabled error: configuration check-out failed: (statements constraint check failed) To fix this you must manually configure ingress-node-replication: root@gw2z0# set groups __contrail__ routing-instances _contrail_l2_4_Test bridge-domains bd-4 vxlan ingress-node-replication root@gw2z0# commit check configuration check succeeds Is this possibly MX junos version specific? I am using a daily build given to me by my SE as I don't believe any released versions support VXLAN: root@gw2z0# run show version Hostname: gw2z0 Model: mx80-48t Junos: 14.2-20150527_rpd_v2_evpn_vnid.0 I doubt it matters but it's also odd that device manager is applying this since I'm using VXLAN: root@gw2z0# show groups __contrail__ protocols mpls interface all; Thanks! Dan _______________________________________________ Users mailing list [email protected]<mailto:[email protected]> http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
_______________________________________________ Users mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
