Raszuk, Please find my comments inline.
Regards Duleep From: [email protected] [mailto:[email protected]] On Behalf Of Robert Raszuk Sent: Saturday, August 8, 2015 3:05 PM To: Duleep Thilakarathne Cc: Richard Li; UTTARO, JAMES; [email protected] Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Hi Thilakarathne, Point #1: What makes you think that 10 ms over 1 GB ethernet peering is any better then 20 ms RTT over 100 GB ? I would really prefer to get routed over 100 GB peerings even if the RTT would be doubled. Your preference may correct theoretically or in a different scenario other than ISP environment. ISPs need to pay for links and link capacity decided based on the traffic requirement and several other factors. There are none pay scenarios in ISP peering but I am not going to explain as it is irrelevant to this discussion. ISPs setup multiple links with multiple capacities in order to achieve several targets. Redundancy, low latency are few of them. Further ISPs advertise IPs to all upstream in order to get redundancy unless there is no special reason. Therefore there is high possibility to select so called 10ms 1GB link than 100 GB 20ms link for particular IP blocks in ISP environment. I will try to explain the scenario I am trying to address again since you have not understand it clearly. Assume particular tier 3 ISP in Asia (say ISP A) has 10 GB link to USA POP and 1G link to Asia POP. Web hosting provider based on Asia advertise its IP block through BGP and ISP A receives web hosting provider IP block from both paths. Assume in this case AS PATH length is equal. Then how does router select best outgoing path. According to you, if router selects 10G link, all traffic route to USA and come back to Asia. It is not desired path. My proposal is to decide best path in such a scenario and not in general BGP scenario. My proposal try to prevent random route selection using new end to end BGP attribute. If router has another end to end BGP attribute other than AS-PATH , router can consider it during tie condition. Other attribute I propose is delay between AS which is proportional to geographic distance. Further this proposal may not have more significance, if eBGP peer resides short geographic distance. For example same state in USA. But this proposal more significant to tier 2 and tier 3 ISPs who have multi-homing with long distance upstream ISPs. Point #2: > There is no reqirment to synchronize different administrative > domains since router itself automatically calculate value and > add when routes advertised similar to AS PATH addition > operation. Sorry to ruin your impression about power and intelligence of routers, but they only do what they are programmed to do. So Jim's point about synchronizing metrics is still valid. Of course I assume that for you the only metric you consider here are milliseconds and therefor do not bother. Imagine one operator chooses to use physical distance and other RTT. So in the new attribute you will get time [ms] vs distance [miles]. Yet one more will also use distance buy expressed in kilometers. Please elaborate how useful such comparison will turn out to be ? I am not oppose to Jim’s point on synchronizing. What I want to highlight is that synchronizing is not required for the attribute proposed by me. present routers program to select outgoing path based on random parameter when AS-PATH length is equal (other conditions after AS length check may not have more significance). My suggestion is to change the program in such scenario. This will prevent random route selection which router programed to do. Proposed attribute can use either length in km or delay in milliseconds. This need decide and include in the RFC. but there is a relationship between delay and distance to eBGP peer . Practically south Asian ISP can achieve around 30ms two way delay to Singapore POP and 300ms to USA POP. We can find cable distance if required. Point #3: As you are suggesting use of ICMP to measure RTT please keep in mind that ICMP is not high priority protocol. It may wait in the remote or local router for processing much more then the propagation delay of the link it arrived on. I am open in this comment. ICMP or TCP delay can be used. Based on my experience even ICMP is not give big issue as most of border routers are carrier grade. 5% to 10% delay variation may not affect to route selection decision. Point #4: How often do you plan to remeasure the eBGP propagation ? Note that today many optical long haul transmission is hidden from ASBRs. That means that your provider of long distance connection may at will reroute you via his own web of fiber which does affect RTT. So it is pretty safe to assume what you have measured yesterday today is irrelevant. I propose, each border router need to check delay to eBGP neighbor every 15 min interval. If routers detects 25% delay difference to previous delay value, router triggers BGP update message. if not, no update message triggers. Optical long haul transmission will not have big issue in practical scenarios as SLAs exists. They are free to change backhaul path as long as delay does not change significantly. If delay changes significantly, it address separately and business cannot exists. As I described earlier this proposal has much significance when ISPs have large geographic separation. Point #5: eBGP propagation may be few orders of magnitude less relevant as propagation within each AS path is traversing. And except the case of few ASes under the same administration we do not have a way to express that one today except AIGP attribute. AIGP RFC clearly mentioned , it is proposed for single administrative domain. I am talking about different administrative domains and eBGP. So if you would like to continue your research perhaps looking at that aspect first may be more valuable .... Since you have not understood practical issue and solution I am explaining and you have not provided valid reason to change my research ,still I decide to continue research on this aspect. Cheers, R. On Sat, Aug 8, 2015 at 7:10 AM, Duleep Thilakarathne <[email protected]<mailto:[email protected]>> wrote: Richard, Yes I am referring eBGP scenario. I suggest distance calculations based on 1. ICMP delay between eBGP speakers. 2. Manually configure binding to remote AS. Each eBGP speaking routers need to accumulate distance value when advertised routes to external peer.There is no reqirment to synchronize different administrative domains since router itself automatically calculate value and add when routes advertised similar to AS PATH addition operation. ----- Reply message ----- From: "Richard Li" <[email protected]<mailto:[email protected]>> To: "Duleep Thilakarathne" <[email protected]<mailto:[email protected]>>, "UTTARO, JAMES" <[email protected]<mailto:[email protected]>>, "'Robert Raszuk'" <[email protected]<mailto:[email protected]>> Cc: "'[email protected]<mailto:[email protected]>'" <[email protected]<mailto:[email protected]>> Subject: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Date: Fri, Aug 7, 2015 10:11 PM There might be a good point here. RFC 7311 only takes care of the IGP metrics. But In Duleep’s example, the metrics between two eBGP speakers are not taken into consideration. In order to have AIGP attribute to really represent the accumulated one, the metrics on such links should be considered as well. However, there might be some challenges or obstacles: The way to configure one metrics on the link between two eBGP speakers might not be consistent with the way to configure another metrics on the another link between two speakers. Richard From: BESS [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Duleep Thilakarathne Sent: Friday, August 07, 2015 8:41 AM To: UTTARO, JAMES; 'Robert Raszuk' Cc: '[email protected]<mailto:[email protected]>' Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Jim, What I want to suggest is to insert item 5 (refer below items listed) to BGP best path selection algorithm. Once AS-PATH length is equal, next we can think on how to select best outgoing interface. If we don’t select proper outgoing interface it will affect to latency. I am talking this based on practical experience I have in ISP environment. There are several options to select best outgoing interface when AS-PATH are equal. In this case I suggest geo distance to destination. Following are options to calculate geo distance. Router selects outgoing interface with lowest GEO distance to destination. 1. BGP speaking router can add distance when advertise to route to upstream similar to AS-PATH attribute. For example A----B----C-----D Router B advertise distance AB to router C. router C advertise accumulated distance AB+BC to router D. 2. Above distance can be configured as manual interface command or dynamically using ICMP or similar mechanism. We can assume ICMP delay propositional to geo distance. 3. Alternative option is to calculate real geo distance from coordinate system. In this case we miss intermediate hops. Accuracy is not much accurate since cable paths do not follow real coordinate based distance. In this case we should have knowledge on coordinates of upstream router which relevant IP block advertise. Option 1 can be achieved through BGP protocol itself ,if agreed to introduce new attribute. Option 3 more suitable to SDN based implementation. Calculation can be daily or weekly basis as this is not primary criteria. Further Consider following scenario. I am in Sri Lanka. Assume I have upstream POPS to Singapore, AMS, New York. Assume I need to reach destination IP located at Japan. When I check BGP routing table, AS-PATH length is equal from all three upstream. Then I have three options. Then router selects any interface randomly if no policy configured. I hope you agreed up to this point. In such case I suggest to consider GEO distance to destination. In most cases lowest distance path is the best path. This may not correct always but better than random outgoing interface selection. 1. Discarding the routes with the unreachable Next_Hop. 2. Preferring the route with the highest Local_Pref. 3. Preferring the aggregated route. The preference of an aggregated route is higher than the preference of a non-aggregated route. 4. Preferring the route with the shortest AS-Path. 5. If AS-Path finds equal, consider shortest GEO distance. If still distance is same follow next steps. 6. Comparing the Origin attribute and selecting the routes with the Origin attribute as IGP, EGP, or Incomplete in order. Regards Duleept From: UTTARO, JAMES [mailto:[email protected]] Sent: Friday, August 7, 2015 7:44 PM To: Duleep Thilakarathne; 'Robert Raszuk' Cc: '[email protected]<mailto:[email protected]>' Subject: RE: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Duleep, So a bit confused here. How do want the decision making to go if a path has a shorter AS-PATH and longer latency than the alternative?? If latency is the prime motivator why do you care about AS-PATH length at all.. Comments In-Line.. Jim Uttaro From: Duleep Thilakarathne [mailto:[email protected]] Sent: Friday, August 07, 2015 9:31 AM To: Robert Raszuk Cc: UTTARO, JAMES; [email protected]<mailto:[email protected]> Subject: RE: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Hi Raszuk, Question 1: How does the router know about user's high latency ? Actually I am referring ISP edge router to another ISP edge router delay due to transmission distance. [Jim U>] The underlying facility and it’s representative transmission distance will most likely differ from geographical distance. Which do you want to address? To Robert’s point you still need to acquire that knowledge and it may be orthogonal to an attribute that is defined as delay. Question 2: How do you assure Internet stability where you start churning paths based on the latency of data plane ? It is not required to consider stability in this situation since it is unavoidable. What is refer is, router need to select best outgoing path considering physical distance whenever possible when AS-PATH length is equal. If router selects long distance path randomly, it impacts to latency. Question 3: What you are after has effectively been solved many years ago .. it is called Optimized Edge Routing (OER) / Performance Routing (PFR) - I suggest you google for those terms. Thank for the suggestion. I gone through these proposals. But what I am suggesting is whether we can address this idea from BGP protocol level. For example by introducing new attribute related to physical distance/delay similar to AS-PATH. New attribute need to update across the As path. My ultimate objective is to prevent router randomly select outgoing path when AS-PATH lengths are equal. Further I am trying SDN based simulation these days. Hope I can share output. But this could similar to what you have proposed except geo distance calculation mechanism. Refer below standard BGP route selection criteria. I suggest item 5. Wordings may different from vendor to vendor. 1. Discarding the routes with the unreachable Next_Hop. 2. Preferring the route with the highest Local_Pref. 3. Preferring the aggregated route. The preference of an aggregated route is higher than the preference of a non-aggregated route. 4. Preferring the route with the shortest AS-Path. 5. If AS-Path finds equal, consider shortest GEO distance. If still distance is same follow next steps. 6. Comparing the Origin attribute and selecting the routes with the Origin attribute as IGP, EGP, or Incomplete in order. Regsrds Duleept From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Robert Raszuk Sent: Friday, August 7, 2015 6:29 PM To: Duleep Thilakarathne Cc: UTTARO, JAMES; [email protected]<mailto:[email protected]> Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Duleep, > Then end user experiences high latency to reach destination. In such > a case, I suggest router need to consider geographic distance to > destination and select path via NTT to reach destination by default. Question 1: How does the router know about user's high latency ? Question 2: How do you assure Internet stability where you start churning paths based on the latency of data plane ? Question 3: What you are after has effectively been solved many years ago .. it is called Optimized Edge Routing (OER) / Performace Routing (PFR) - I suggest you google for those terms. Regards, R. On Fri, Aug 7, 2015 at 2:51 PM, Duleep Thilakarathne <[email protected]<mailto:[email protected]>> wrote: Hi Jim, Please refer below example. Assume destination IP is in Asian region. Particular ISP in a different location (Say India) has upstream peering to US POP (Say AT&T) and Asia POP (Say NTT). If we check BGP routing table, assume it shows XX.XX.XX.XX/24 -------->AS - AT&T,AS-XX,AS-Destination -------->AS - NTT,AS-YY,AS-Destination In above case AS-PATH is equal and assume router automatically select path via AT&T. Then end user experiences high latency to reach destination. In such a case, I suggest router need to consider geographic distance to destination and select path via NTT to reach destination by default. Deciding geo distance is a challenge but there are options. Here geo distance means shortest distance to reach IP destination from upstream POP. Current practice is to use community strings, but it depends on upstream ISP capability. Can you comment my idea. Regards Duleept From: UTTARO, JAMES [mailto:[email protected]<mailto:[email protected]>] Sent: Friday, August 7, 2015 4:09 PM To: Duleep Thilakarathne; 'Robert Raszuk' Cc: '[email protected]<mailto:[email protected]>' Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Duleep, Assuming AS-PATH is equal and AS-Content different how can you know that the internal metrics of each AS are consistent and mirror actual geographic distances? You have to be assured that each administrative domain applies the same metric assignment. I do not believe this is possible when there are multiple administrative domains. Jim Uttaro From: BESS [mailto:[email protected]] On Behalf Of Duleep Thilakarathne Sent: Friday, August 07, 2015 5:19 AM To: Robert Raszuk Cc: [email protected]<mailto:[email protected]> Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Hi Raszuk, I went through RFC7311 and my concern is different than RFC 7311. I have analyzed full BGP routing table (541,199 routes) with two tier 1 ISP multi-homing scenario and found nearly 50% of routes have equal AS-PATH length. In this analysis It was considered, there was no route policy applied to influence local preference. According to BGP best path selection algorithm, when AS-PATH lengths are equal, router breaks tie condition based on route internal logic. This does not grantee proper outgoing path selection. Appreciate your concern on above analysis. Regards Duleept From: Robert Raszuk [mailto:[email protected]] Sent: Monday, July 27, 2015 2:40 AM To: Duleep Thilakarathne Cc: [email protected]<mailto:[email protected]> Subject: Re: [bess] BGP route selection criteria - geographic distance when AS_PATH are equal Hi Duleep, Please consider RFC 7311 and provide feedback why you think it is not sufficient for your objective. https://tools.ietf.org/html/rfc7311 Best, R. On Sun, Jul 26, 2015 at 9:15 PM, Duleep Thilakarathne <[email protected]<mailto:[email protected]>> wrote: Hi, I would like to suggest to consider geographic distance when AS_PATH are equal in BGP route selection criteria. (as tie breaking rule). Can anybody comment on my idea. Regards Duleept This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd. _______________________________________________ BESS mailing list [email protected]<mailto:[email protected]> https://www.ietf.org/mailman/listinfo/bess This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd. This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd. This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd. This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd. This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd. This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. Mobitel (Pvt) Ltd.
_______________________________________________ BESS mailing list [email protected] https://www.ietf.org/mailman/listinfo/bess
