Hi Nischal, Thank you for the detailed reply. I may have a few follow up questions in the coming days but for now I only have two:
1. Do you have an example config of what you are thinking DM might push to the MX to route between VRF and inet.0? Even if I could apply that by default right now it would be helpful. 2. Any ETA on the bugs you referenced and will there be a reasonably easy to make use of these prior to official 2.21 packages being released? Thanks! Dan On Jul 14, 2015 12:29 AM, "Nischal Sheth" <[email protected]> wrote: > > On Jul 9, 2015, at 12:25 PM, Dan Houtz <[email protected]> wrote: > > Hi Nischal, > > > Hi Dan, > > Pleas see inline. > > > I'll need to re-read through the bug text a few more times to fully grasp > as I have yet to totally understand the difference between L2 and L2/L3 > networks from the Contrail perspective. It seems all networks we create > via the Contrail webui are L2/L3. I see there is an L2 option when > configured a port but this requires me to create IP/MAC mappings which > doesn't apply to us. Perhaps this is just a limitation of the UI and I > could accomplish L2 only via API calls? > > > I gather from your follow-up email on DHCP that you have a good handle > on the > difference between L2 and L2/L3 in Contrail now. In terms of DM, the > difference > should be that DM will only configure a virtual-switch routing-instance > for L2 only > VNs, whereas it would configure both virtual-switch and vrf > routing-instances for > L2+L3 VNs. Further, we're also adding a L3 only mode where DM will create > only > a vrf. > > > In our use case we are doing overlays between all bare metal servers - we > do not have any Openstack/VM elements. We will also generally be assigning > publicly routed IP's directly to all servers and will not require NAT. > There may be cases where customers need to build 'backnets' with private IP > addresses. I suspect most of these to be islands where routing don't be > required but we may need to extend these to our MXs for routing between > private networks belonging to a specific customer. > > > Please see below. > > > Based on the above thoughts I believe we will probably have 3 primary > scenarios: > > 1. Create a 'public' virtual network in Contrail, assign a public IP > block to it, and have it be publicly routable - prefixes of VN exported > into inet.0 and default from inet.0 imported into routing instance. It > seems like creating a network, marking it as external, and not creating > floating IPs would indicate this type of setup but I haven't actually > looked at how Contrail does NAT/Floating IPs/etc. > > > This should have worked, but DM currently treats external VNs as L3 only > VNs and > does not create a virtual-switch routing-instance. The bug I created > earlier tracks this > issue. > > https://bugs.launchpad.net/juniperopenstack/+bug/1472699 > > The bug also describes the routing configuration to allow the VN subnet > to be added > or imported into inet.0 and traffic redirection from inet.0 to the VRF. We > need to work > around some JUNOS quirks wherein 2 route tables cannot have static routes > that > point to each other. I've assumed that the VRF will have a static default > that points to > inet.0. If you're doing manual configuration, you could use rib-groups to > import default > route from BGP/OSPF into the VRF, but DM uses static default with a table > next-hop > since it doesn't want to assume anything about routing protocol being used > for inet.0. > > Traffic destined to the VRF subnet will be redirected from inet.0 to the > VRF using filter > based forwarding by applying an input filter on inet.0. This filter > matches the destination > subnet and redirects traffic to the appropriate VRF. > > > > 2. Create a 'private' virtual network in Contrail, assign a private IP > block to it and have it be routable only between other networks belonging > to the tenant it is created under. I think this would generally be an all > or nothing thing - every private network belonging to a tenant can reach > any other private network belonging to that tenant. I could see someone > wanting more fine grain control but probably not super common and would > mostly look at inline firewall chaining or policy to control this > > > This is supported today, and is finer grained than all or nothing. You > can use policy to > allow traffic between specific VNs. Matches on L4 information are not yet > supported. > > 3. Create a 'backnet' virtual network in Contrail, assign no IP's - > simply extended L2 to appropriate bare metal servers. Servers could be IP'd > adhoc without Contrail being aware at all. Network would be an island > with no config existing on MX. > > > This should work today (modulo issues you found with Praveen) if you do > not extend > such VNs to the MXs. It will be further refined as part of the fix for > bugs 1472699 and > 1471637. If a VN is L2 only, you will not need to configure a subnet for > it. > > > Let me know if the above makes any sense :) > > > Yes, the requirements sound reasonable and I think that you should be > able to achieve > what you are looking once 1472699 and 1471637 are fixed. > > -Nischal > > Thanks! > > On Wed, Jul 8, 2015 at 11:58 AM, Nischal Sheth <[email protected]> wrote: > >> >> Hi Dan, >> >> https://bugs.launchpad.net/juniperopenstack/+bug/1472699 >> >> This needs some changes to the device manager. >> >> Could you please take a look and provide feedback on whether >> the solution outlined will meet your requirements? >> >> Thanks, >> Nischal >> >> On Jul 7, 2015, at 5:54 PM, Dan Houtz <[email protected]> wrote: >> >> Unless I'm overlooking something, It doesn't look like device-manager >> builds the config needed to import a default route from inet.0 into the >> VN's routing instance or to import the routing instance's prefix into >> inet.0. In our case we are assigning public IP's directly to the bare metal >> servers and do not require Is it possible for device-manager to configure >> the import/export policies to make these routes available? >> >> Thanks! >> Dan >> >> On Tue, Jul 7, 2015 at 4:55 PM, Nischal Sheth <[email protected]> wrote: >> >>> >>> Hi Dan, >>> >>> This confirms that the problem is related to route target filtering. >>> It's >>> a JUNOSism that bgp.rtarget.0 routes are resolved using inet.3 by >>> default. >>> >>> Using gr-* interfaces will result in creation of inet.3 routes to the >>> CN IP >>> since DM adds CN IPs to the dynamic-tunnel destination network list. >>> In this case the gr tunnels wouldn't be used in the data plane since you >>> are using VXLAN. If you also have VMs, MX will use gr tunnels for >>> data traffic to vrouters. >>> >>> If it's undesirable to create gr- tunnels, you have 2 options: >>> >>> 1) You can configure JUNOS to resolve bgp.rtarget.0 over inet.0 instead >>> of inet.3 using something like: >>> >>> root@a3-mx80-1# show routing-options resolution >>> rib bgp.rtarget.0 { >>> resolution-ribs inet.0; >>> } >>> >>> 2) Alternately, you can add add static routes with discard nexthop to >>> CN >>> ip addresses into inet.3. >>> >>> root@a3-mx80-1# show routing-options rib inet.3 >>> static { >>> route 10.10.210.140/32 discard; >>> } >>> >>> [edit] >>> >>> I would recommend the latter. >>> >>> -Nischal >>> >>> On Jul 7, 2015, at 2:36 PM, Dan Houtz <[email protected]> wrote: >>> >>> root@gw2z0> show route table bgp.rtarget.0 >>> >>> bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) >>> + = Active Route, - = Last Active, * = Both >>> >>> 65412:65412:1/96 >>> *[RTarget/5] 01:36:41 >>> Type Default >>> Local >>> 65412:65412:8000001/96 >>> *[RTarget/5] 01:36:41 >>> Type Default >>> Local >>> >>> root@gw2z0> show route table bgp.rtarget.0 hidden >>> >>> bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) >>> + = Active Route, - = Last Active, * = Both >>> >>> 65412:65412:1/96 >>> [BGP/170] 02:26:54, localpref 100, from 10.10.210.140 >>> AS path: I, validation-state: unverified >>> Unusable >>> 65412:65412:8000001/96 >>> [BGP/170] 02:26:54, localpref 100, from 10.10.210.140 >>> AS path: I, validation-state: unverified >>> Unusable >>> >>> >>> I don't believe I should have any gr interfaces as I'm doing VXLAN (no >>> MPLS/GRE/etc) correct? >>> >>> >>> >>> On Tue, Jul 7, 2015 at 4:10 PM, Nischal Sheth <[email protected]> >>> wrote: >>> >>>> >>>> Hi Dan, >>>> >>>> Since there's no routes being sent to the CNs, there may be a problem >>>> with route target filtering, which makes the MX think that the CN is not >>>> interested in the any route targets. >>>> >>>> Can you run "show route table bgp.rtarget.0" on the MX and check if >>>> there are any hidden routes in that table? If there are, we need to >>>> check (show interfaces terse gr-*) if there's any gr-* devices on the >>>> system. If there aren't any, can you add them using something like: >>>> >>>> fpc 1 { >>>> pic 0 { >>>> tunnel-services; >>>> } >>>> } >>>> >>>> -Nischal >>>> >>>> On Jul 7, 2015, at 12:47 PM, Dan Houtz <[email protected]> wrote: >>>> >>>> I was able to get updated MX code (14.2-20150704.0) and now have >>>> device-manager successfully configuring my MX80. BGP session between MX and >>>> Contrail also seems to be stable now however I am having an issue with >>>> reach-ability between MX and hosts connected to TOR switches. Based on >>>> ititial troubleshooting I don't believe Junos is announcing the EVPN route >>>> for the IRB interface: >>>> >>>> oot@gw2z0# show groups __contrail__ interfaces irb >>>> gratuitous-arp-reply; >>>> unit 4 { >>>> family inet { >>>> address 10.10.210.145/29; >>>> } >>>> } >>>> >>>> >>>> root@gw2z0# run show route advertising-protocol bgp 10.10.210.140 >>>> >>>> bgp.rtarget.0: 2 destinations, 4 routes (2 active, 0 holddown, 2 hidden) >>>> Prefix Nexthop MED Lclpref AS >>>> path >>>> 65412:65412:1/96 >>>> * Self 100 I >>>> 65412:65412:8000001/96 >>>> * Self 100 I >>>> >>>> >>>> IRB interface is up: >>>> >>>> root@gw2z0# run show interfaces routing | grep irb >>>> irb.4 Up INET 10.10.210.145 >>>> >>>> root@gw2z0# run show route 10.10.210.145 >>>> >>>> _contrail_l3_4_Test.inet.0: 2 destinations, 3 routes (2 active, 0 >>>> holddown, 0 hidden) >>>> + = Active Route, - = Last Active, * = Both >>>> >>>> 10.10.210.145/32 *[Local/0] 00:02:51 >>>> Local via irb.4 >>>> >>>> >>>> >>>> On Sat, Jul 4, 2015 at 1:10 PM, Nischal Sheth <[email protected]> >>>> wrote: >>>> >>>>> >>>>> https://bugs.launchpad.net/juniperopenstack/+bug/1465070 >>>>> >>>>> -Nischal >>>>> >>>>> Sent from my iPhone >>>>> >>>>> On Jul 4, 2015, at 11:04 AM, Dan Houtz <[email protected]> wrote: >>>>> >>>>> Great! Next question... >>>>> >>>>> Are there plans to add in ability to apply the >>>>> 'virtual-gateway-address' knob when configuring IRBs? I believe this is >>>>> the >>>>> recommended way to configure redundant MX gateways correct? >>>>> >>>>> -Dan >>>>> >>>>> On Sat, Jul 4, 2015 at 9:15 AM, Vedamurthy Ananth Joshi < >>>>> [email protected]> wrote: >>>>> >>>>>> Yes…this should be addressed too. >>>>>> >>>>>> Vedu >>>>>> >>>>>> From: Dan Houtz <[email protected]> >>>>>> Date: Saturday, July 4, 2015 at 7:38 PM >>>>>> To: Vedamurthy Ananth Joshi <[email protected]> >>>>>> Cc: OpenContrail Users List - 2 <[email protected]> >>>>>> Subject: Re: [Users] Problem with Device Manager's VXLAN config in >>>>>> Contrail 2.2 >>>>>> >>>>>> Vedu, >>>>>> >>>>>> Thank you for the information. I have reached out to our SE to see >>>>>> about getting updated code. I am also seeing the following with BGP >>>>>> sessions between Contrail and MX since moving to 2.2: >>>>>> >>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: RPD_BGP_NEIGHBOR_STATE_CHANGED: >>>>>> BGP peer 10.10.210.140 (Internal AS 65412) changed state from OpenConfirm >>>>>> to Established (event RecvKeepAlive) (instance master) >>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: bgp_read_v4_update:10535: >>>>>> NOTIFICATION sent to 10.10.210.140 (Internal AS 65412): code 3 (Update >>>>>> Message Error) subcode 9 (error with optional attribute), Data: c0 16 09 >>>>>> 10 fc 00 >>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: RPD_BGP_NEIGHBOR_STATE_CHANGED: >>>>>> BGP peer 10.10.210.140 (Internal AS 65412) changed state from Established >>>>>> to Idle (event RecvUpdate) (instance master) >>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: Received malformed update from >>>>>> 10.10.210.140 (Internal AS 65412) >>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: Family evpn, prefix >>>>>> 3:10.10.210.140:1::4::10.10.214.65/152 >>>>>> Jul 4 14:06:47 gw2z0 rpd[86503]: Malformed Attribute PMSI(22) >>>>>> flag 0xc0 length 9. >>>>>> Jul 4 14:06:52 gw2z0 rpd[86503]: bgp_parse_open_options: peer >>>>>> 10.10.210.140+50620 (proto): unsupported AF 1 SAFI 243 >>>>>> >>>>>> Is this something that will also be fixed with the new MX code? >>>>>> >>>>>> Thanks! >>>>>> Dan >>>>>> >>>>>> On Sat, Jul 4, 2015 at 8:02 AM, Vedamurthy Ananth Joshi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Dan, >>>>>>> Ingress-node-replication was not pushed by Device Manager on >>>>>>> purpose. >>>>>>> The corresponding MX image could be any daily build equal to or >>>>>>> greater than 14.2-20150627.0. >>>>>>> >>>>>>> Vedu >>>>>>> >>>>>>> From: Dan Houtz <[email protected]> >>>>>>> Date: Saturday, July 4, 2015 at 1:47 PM >>>>>>> To: OpenContrail Users List - 2 <[email protected]> >>>>>>> Subject: [Users] Problem with Device Manager's VXLAN config in >>>>>>> Contrail 2.2 >>>>>>> >>>>>>> Has anyone else tried configuring EVPN VXLAN on an MX using >>>>>>> device manager in Contrail 2.2? In my testing the configuration being >>>>>>> pushed my netconf is not valid: >>>>>>> >>>>>>> root@gw2z0# commit check >>>>>>> [edit routing-instances _contrail_l2_4_Test bridge-domains bd-4] >>>>>>> 'vxlan' >>>>>>> multicast-group or ovsdb-managed or ingress-node-replication >>>>>>> should be enabled >>>>>>> error: configuration check-out failed: (statements constraint check >>>>>>> failed) >>>>>>> >>>>>>> To fix this you must manually configure ingress-node-replication: >>>>>>> >>>>>>> root@gw2z0# set groups __contrail__ routing-instances >>>>>>> _contrail_l2_4_Test bridge-domains bd-4 vxlan ingress-node-replication >>>>>>> >>>>>>> root@gw2z0# commit check >>>>>>> configuration check succeeds >>>>>>> >>>>>>> Is this possibly MX junos version specific? I am using a daily >>>>>>> build given to me by my SE as I don't believe any released versions >>>>>>> support >>>>>>> VXLAN: >>>>>>> >>>>>>> root@gw2z0# run show version >>>>>>> Hostname: gw2z0 >>>>>>> Model: mx80-48t >>>>>>> Junos: 14.2-20150527_rpd_v2_evpn_vnid.0 >>>>>>> >>>>>>> I doubt it matters but it's also odd that device manager is >>>>>>> applying this since I'm using VXLAN: >>>>>>> >>>>>>> root@gw2z0# show groups __contrail__ protocols mpls >>>>>>> interface all; >>>>>>> >>>>>>> Thanks! >>>>>>> Dan >>>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> [email protected] >>>>> >>>>> http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org >>>>> >>>>> >>>> >>>> >>> >>> >> >> > >
_______________________________________________ Users mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/users_lists.opencontrail.org
