Re: Strange practices?
Hi, On 7 Jun 2010, at 23:02, Joel M Snyder joel.sny...@opus1.com wrote: On 6/7/10 11:51 PM: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? Yes, this is common and works fine. [...] Ugly, but given the vast chalice of despair that is the global BGP table, hardly a drop in the bucket. Ugly, failover might not work depending on just what is actually configured, and there is of course no need to take the full table if you want to do it right, with BGP. It does also marry your network to one provider, which might not suit depending on how independent you want to be (what will happen to your pricing with the address space incumbent at renew time, or what will happen in the event of their commercial failure). Because something will likely work, does not make it a scalable or sensible design. Just do it right from the start :-) Andy
Re: Strange practices?
Hi, On Tue, Jun 8, 2010 at 6:50 AM, Dale Cornman bstym...@gmail.com wrote: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice I have seen it quite often. It allows an enterprise to be multihomed w/o getting PI or PA address space so they are usually pretty happy with it. as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. AFAIR prefixes can be originated by more than one AS so there shouldn't be any issues. -- SY, Jen Linkova aka Furry
Strange practices?
Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. Thanks -Bill
Re: Strange practices?
Should work fine. --Original Message-- From: Dale Cornman To: nanog@nanog.org Subject: Strange practices? Sent: Jun 7, 2010 5:50 PM Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. Thanks -Bill Sent from my BlackBerry device on the Rogers Wireless Network
Re: Strange practices?
* Dale Cornman: I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. The 6to4 anycast gateway RFC practically mandates this, and it does work when you're doing anycast. But with static routes, you cannot handle some failure scenarious, and that usually a good reason to stay away from such setups. Of course, in the world of real routers, there might be constraints such lack of memory or processing power to handle BGP. 8-/
Re: Strange practices?
On Mon, Jun 07, 2010 at 03:50:25PM -0500, Dale Cornman wrote: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? Yes; tends to happen for clueless endpoints or providers who don't expressly require BGP for multihoming.` One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. MOAS prefixes are common in some content-origination applications, but since you never know what the rest of the universe is going to do in their routing forwarding decisions, is really isn't generally applicable. -- RSUC / GweepNet / Spunk / FnB / Usenix / SAGE
Re: Strange practices?
Hve seen it a few times -- usually with enterprise customers who are unable to manage their own routers and one ISP which has problems configuring BGP on their client facing equipment. Dale Cornman wrote: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. Thanks -Bill
Re: Strange practices?
I would say partitioning into two AS's like this is not a good thing. I wouldn't consider it a valid design myself, and would avoid it if possible. If one of the AS's that is announcing the block, originates any traffic into the other AS for that block, the traffic will drop. I realize this ideally should not happen, but BGP uses arbitrary metrics, and people turn alot of knobs, which makes wierd things happen. If someone were doing this themselves, I would say at least use a GRE tunnel with an iBGP link between the sites, but your not going to get that out of these providers, so its going to remain partitioned which should be thought through well as there may be issues with this. Brian On Jun 7, 2010, at 4:59 PM, Florian Weimer wrote: * Dale Cornman: I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. The 6to4 anycast gateway RFC practically mandates this, and it does work when you're doing anycast. But with static routes, you cannot handle some failure scenarious, and that usually a good reason to stay away from such setups. Of course, in the world of real routers, there might be constraints such lack of memory or processing power to handle BGP. 8-/
Re: Strange practices?
Let me recant on what I said. I re-read and had myself confused (apologies). I see that the providers are using their own AS's. I still would not do this if it could be avoided, but the traffic won't be dropped like I had said, in the way I was thinking. What I was thinking was a case where the same AS is announcing from two sites, which are not connected via iBGP. In that case default behavior is that the AS drops traffic from its own AS as this is how eBGP accomplishes loop prevention. In the case that is being described this won't happen since each provider is using its own AS to announce from. Brian On Jun 7, 2010, at 5:05 PM, Brian Feeny wrote: I would say partitioning into two AS's like this is not a good thing. I wouldn't consider it a valid design myself, and would avoid it if possible. If one of the AS's that is announcing the block, originates any traffic into the other AS for that block, the traffic will drop. I realize this ideally should not happen, but BGP uses arbitrary metrics, and people turn alot of knobs, which makes wierd things happen. If someone were doing this themselves, I would say at least use a GRE tunnel with an iBGP link between the sites, but your not going to get that out of these providers, so its going to remain partitioned which should be thought through well as there may be issues with this. Brian On Jun 7, 2010, at 4:59 PM, Florian Weimer wrote: * Dale Cornman: I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. The 6to4 anycast gateway RFC practically mandates this, and it does work when you're doing anycast. But with static routes, you cannot handle some failure scenarious, and that usually a good reason to stay away from such setups. Of course, in the world of real routers, there might be constraints such lack of memory or processing power to handle BGP. 8-/
Re: Strange practices?
It's going to show inconsistent AS which some people may not like, but that's just ugly not broken. As the customer, it means your outgoing path selection is probably being made on the basis of some non-global attribute, and the return path is entirely at the mercy of your two isps... I wouldn't do that becuase the alternatives are better and not exactly a lot of work, but will it work? yes. joel On 2010-06-07 13:50, Dale Cornman wrote: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. Thanks -Bill
Re: Strange practices?
On Mon, Jun 7, 2010 at 13:50, Dale Cornman bstym...@gmail.com wrote: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. Thanks -Bill So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? -Bill
RE: Strange practices?
Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP?” As stated before...yes this is a common practice. One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well.” Yes, one ISP owns the block, both will aggregate the blocks and announce the blocks to the global internet. BGP attributes will shape best path for routing; i.e., AS-PATH, ORIGIN, LOCAL PREF. MEDS should take care of leaking routes. So, is this design scheme viable? Yes, it is. ~Jay Murphy IP Network Specialist NM State Government IT Services Division PSB – IP Network Management Center Santa Fé, New México 87505 We move the information that moves your world. “Good engineering demands that we understand what we’re doing and why, keep an open mind, and learn from experience.” “Engineering is about finding the sweet spot between what's solvable and what isn't. Radia Perlman Please consider the environment before printing e-mail -Original Message- From: Dale Cornman [mailto:bstym...@gmail.com] Sent: Monday, June 07, 2010 2:50 PM To: nanog@nanog.org Subject: Strange practices? Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well. I had personally never heard of this and am curious if this is a common practice as well as if this would potentially create any problems by 2 Autonomous Systems both originating the same prefix. Thanks -Bill Confidentiality Notice: This e-mail, including all attachments is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited unless specifically provided under the New Mexico Inspection of Public Records Act. If you are not the intended recipient, please contact the sender and destroy all copies of this message. -- This email has been scanned by the Sybari - Antigen Email System.
Re: Strange practices?
On 2010.06.07 17:49, Murphy, Jay, DOH wrote: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP?� As stated before...yes this is a common practice. One of the ISP's in this case owns the block and has even provided a letter of authorization to the other, allowing them to announce it in BGP as well.� Yes, one ISP owns the block, both will aggregate the blocks and announce the blocks to the global internet. BGP attributes will shape best path for routing; i.e., AS-PATH, ORIGIN, LOCAL PREF. MEDS should take care of leaking routes. So, is this design scheme viable? Yes, it is. I understood the OP's question as one of concern. It sounds to me like one of their ISPs can't/won't/doesn't know how to configure a client-facing BGP session. I've run into this before, and it was due to a lack of understanding/clue of how to peer with a multi-homed client when the client didn't have their own ASN. If that is the case, then I'd be concerned about situations where the link goes down, but the advertisement is not removed from their DFZ-facing sessions, possibly causing a black hole for traffic transiting that ISP. The work involved in co-ordinating two ISPs to detect and protect against this type of situation is far more difficult than just configuring BGP from the client out (imho). Steve
RE: Strange practices?
So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? Yes, BGP updates. ~Jay Murphy IP Network Specialist NM State Government IT Services Division PSB - IP Network Management Center Santa Fé, New México 87505 Bus. Ph.: 505.827.2851 We move the information that moves your world. Good engineering demands that we understand what we're doing and why, keep an open mind, and learn from experience. Engineering is about finding the sweet spot between what's solvable and what isn't. Radia Perlman P Please consider the environment before printing e-mail Confidentiality Notice: This e-mail, including all attachments is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited unless specifically provided under the New Mexico Inspection of Public Records Act. If you are not the intended recipient, please contact the sender and destroy all copies of this message. -- This email has been scanned by the Sybari - Antigen Email System. image001.jpgimage002.png
RE: Strange practices?
On 6/7/10 11:51 PM: Has anyone ever heard of a multi-homed enterprise not running bgp with either of 2 providers, but instead, each provider statically routes a block to their common customer and also each originates this block in BGP? Yes, this is common and works fine. We do it with a number of customers who want a backup provider but don't want to go to the trouble of getting portable address space, an ASN, and so on. As long as both providers have a way of shutting down the advertisement (typically because they learn it via BGP) and as long as the customer doesn't try to load balance (i.e., treats it as active/passive not true active/active), then it's not a bad solution. Ugly, but given the vast chalice of despair that is the global BGP table, hardly a drop in the bucket. jms -- Joel M Snyder, 1404 East Lind Road, Tucson, AZ, 85719 Senior Partner, Opus One Phone: +1 520 324 0494 j...@opus1.comhttp://www.opus1.com/jms
Re: Strange practices?
On 2010.06.07 18:10, Murphy, Jay, DOH wrote: Yes, the customer has an AS number, it's just from the private AS number block, e.g. AS 65000..when the block is routed to the AS running BGP, it is tagged with that ISP's public AS number, and announced to the world in this manner. ...but the OP stated that he doesn't do any BGP with either upstream, and instead relies on the upstreams to statically route the block to him. I was getting at the usage of private-AS in my last post. Perhaps I'm mis-understanding something. Clarify, transiting? The OP has two 'transit' providers, neither of which he has a BGP session established. Both of his upstream ISPs provide transit for him to the wider Internet. Do you mean one ISP acts as a transit routing domain for another, or for traffic that traverses this particular ISP, which one? Traverses. ie. my upstream providers provide 'transit' services for networks that I advertise to them, however, I don't allow any of my peers to 'transit' my network. Steve
Re: Strange practices?
On 2010.06.07 17:59, Murphy, Jay, DOH wrote: So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? Yes, BGP updates. ...again, I'm confused. BGP updates from where to where? From how I understand the OP's original question, there is no BGP. Hence, if one of the providers is statically routing the prefix to an interface or un-numbered as opposed to an IP address, then blackholing can occur if IP reachability is broken, but the link-layer is not. Is this not correct? Steve
Re: Strange practices?
On Mon, Jun 7, 2010 at 14:59, Murphy, Jay, DOH jay.mur...@state.nm.us wrote: So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? Yes, BGP updates. Um, it wasn't a trick question Jay, and as others have stated, since the providers are statically routing this address space to their common customer, this would require a coordinated effort to manually (or preferably automatically) shutdown the advertisement should connectivity be lost to the customer. There are a number of ways that could be achieved, but it's obviously important that it is. -Bill
RE: Strange practices?
Steve, We are obviously interpreting this in different slants. Definition of Transit service: for example, AS200 is said to receive transit service from, let's say AS3356, if through this connection, AS200 receives connectivity to the entire Internet and not only AS3356 and its customers. Yes I understand the customer is using static, however, some providers use BGP, and they use BGP to peer with other ISPs, that's it. ~Jay Murphy IP Network Specialist NM State Government IT Services Division PSB – IP Network Management Center Santa Fé, New México 87505 We move the information that moves your world. “Good engineering demands that we understand what we’re doing and why, keep an open mind, and learn from experience.” “Engineering is about finding the sweet spot between what's solvable and what isn't. Radia Perlman Please consider the environment before printing e-mail -Original Message- From: Steve Bertrand [mailto:st...@ipv6canada.com] Sent: Monday, June 07, 2010 4:38 PM To: Murphy, Jay, DOH Cc: Dale Cornman; nanog@nanog.org Subject: Re: Strange practices? On 2010.06.07 17:59, Murphy, Jay, DOH wrote: So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? Yes, BGP updates. ...again, I'm confused. BGP updates from where to where? From how I understand the OP's original question, there is no BGP. Hence, if one of the providers is statically routing the prefix to an interface or un-numbered as opposed to an IP address, then blackholing can occur if IP reachability is broken, but the link-layer is not. Is this not correct? Steve Confidentiality Notice: This e-mail, including all attachments is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited unless specifically provided under the New Mexico Inspection of Public Records Act. If you are not the intended recipient, please contact the sender and destroy all copies of this message. -- This email has been scanned by the Sybari - Antigen Email System.
RE: Strange practices?
Right on... ~Jay Murphy IP Network Specialist NM State Government IT Services Division PSB - IP Network Management Center Santa Fé, New México 87505 We move the information that moves your world. Good engineering demands that we understand what we're doing and why, keep an open mind, and learn from experience. Engineering is about finding the sweet spot between what's solvable and what isn't. Radia Perlman P Please consider the environment before printing e-mail From: d...@hetzel.org [mailto:d...@hetzel.org] On Behalf Of Dorn Hetzel Sent: Monday, June 07, 2010 4:41 PM To: Steve Bertrand Cc: Murphy, Jay, DOH; nanog@nanog.org Subject: Re: Strange practices? Perhaps the providers BGP is just being fed from interface anchored static routes which will, hopefully, drop out if the customer facing interface goes down. Of course, this is realistic if we're talking about actual circuits like a T-1, not so much if we're talking metro ethernet or something... On Mon, Jun 7, 2010 at 6:38 PM, Steve Bertrand st...@ipv6canada.com wrote: On 2010.06.07 17:59, Murphy, Jay, DOH wrote: So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? Yes, BGP updates. ...again, I'm confused. BGP updates from where to where? From how I understand the OP's original question, there is no BGP. Hence, if one of the providers is statically routing the prefix to an interface or un-numbered as opposed to an IP address, then blackholing can occur if IP reachability is broken, but the link-layer is not. Is this not correct? Steve Confidentiality Notice: This e-mail, including all attachments is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited unless specifically provided under the New Mexico Inspection of Public Records Act. If you are not the intended recipient, please contact the sender and destroy all copies of this message. -- This email has been scanned by the Sybari - Antigen Email System. image001.png
RE: Strange practices?
Yes, I understand this point. So, elaborate on the answer... I am not making something simple, complex, homey. ~Jay Murphy IP Network Specialist NM State Government IT Services Division PSB – IP Network Management Center Santa Fé, New México 87505 We move the information that moves your world. “Good engineering demands that we understand what we’re doing and why, keep an open mind, and learn from experience.” “Engineering is about finding the sweet spot between what's solvable and what isn't. Radia Perlman Please consider the environment before printing e-mail -Original Message- From: Bill Fehring [mailto:li...@billfehring.com] Sent: Monday, June 07, 2010 4:42 PM To: Murphy, Jay, DOH Cc: Dale Cornman; nanog@nanog.org Subject: Re: Strange practices? On Mon, Jun 7, 2010 at 14:59, Murphy, Jay, DOH jay.mur...@state.nm.us wrote: So if the enterprise loses connectivity to one of these two providers, does the provider without working connectivity to the enterprise have mechanism in place to cease originating the address space? Yes, BGP updates. Um, it wasn't a trick question Jay, and as others have stated, since the providers are statically routing this address space to their common customer, this would require a coordinated effort to manually (or preferably automatically) shutdown the advertisement should connectivity be lost to the customer. There are a number of ways that could be achieved, but it's obviously important that it is. -Bill Confidentiality Notice: This e-mail, including all attachments is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited unless specifically provided under the New Mexico Inspection of Public Records Act. If you are not the intended recipient, please contact the sender and destroy all copies of this message. -- This email has been scanned by the Sybari - Antigen Email System.
Re: Strange practices?
On 2010.06.07 18:48, Murphy, Jay, DOH wrote: Steve, We are obviously interpreting this in different slants. Agreed ;) Definition of Transit service: for example, AS200 is said to receive transit service from, let's say AS3356, if through this connection, AS200 receives connectivity to the entire Internet and not only AS3356 and its customers. Yes. The OP has transit through two separate ISPs. Neither of which provide him a BGP session, because one of the providers doesn't seem willing/capable to do so, even though the ISP who is responsible for the space has provided the other with an LOA to allow the prefix to originate from their ASN. Essentially, the OP is transiting through both ISPs, but not providing any transit services, and the transit path is provided via static routes as opposed to dynamic ones. Yes I understand the customer is using static, however, some providers use BGP, and they use BGP to peer with other ISPs, s/some/real ...and not only for peering, but for transit (to the DFZ) as well. that's it. I have had a couple discussions with people off list. Although I don't know the reasoning for the OP's ISP's decision to not use BGP, in cases that I've dealt with this, it is usually due to lack of clue on how to use private ASs, or BGP in general. These ISPs (in my experience) have their DFZ-facing sessions set up by their upstreams, and don't have the knowledge to configure BGP toward the clients. Personally, if this is the case, then I'd be just as concerned with their ability to ensure that a proper configuration to auto-detect failure that causes removal of the prefix from their tables to avoid blackholes. With that said, I'd also be just as concerned with their BGP troubleshooting and filtering abilities if they were to offer a session. Some of the smaller ISPs that fit this bill will actually allow you to work with them and provide them advice along the way, if not even contract the client as a consultant to ensure that this new-to-them setup is documented properly so it can be re-used with other clients. Also, I'm sure that it would be more work to co-ordinate the efforts for a static setup like this between two providers than it would be to just set up BGP. More documentation (and unnecessary static routes too). Steve