Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Dear Colleagues, Sorry for the late response. The problem was due to faulty firmware on one of our Alaxala routers. We resolved the problem the same day (Aug. 18) by downgrading firmware. For more details, please see Alaxala page here (English): http://www.alaxala.com/en/information/20090827.html If you have filters etc. blocking AS9354, please remove them. Thank you, Nick Boyadjiev AS9354 Nagoya JAPAN
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On Tue, Aug 18, 2009 at 09:37:22AM +0200, Ivan Pepelnjak wrote: Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) http://wiki.nil.com/Filter_excessively_prepended_BGP_paths It will still be a while before we see unbroken 4byte AS behavior (that whole 'fix the teardown on a anyone sneezing' problem). But like with stale bogon filters, I expect folks inclined to use this to drop it in and forget about it. So it would be wise to adjust the recommended filter to anticipate a 2byteAS view allowing multiple instances of AS-TRANS; there's likely a more elegant approach, but the quick step of explicitly allowing _(23465_)+ before you deny _([0-9]+)_\1_\1_\1_\1_ Cheers, Joe -- RSUC / GweepNet / Spunk / FnB / Usenix / SAGE
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On 19/08/2009, at 6:58 AM, Ivan Pepelnjak wrote: No. You cannot influence the inbound traffic apart from not advertising some of your prefixes to some of your neighbors or giving them hints with BGP communities or AS-path prepending. Whatever you do with BGP on your routers influences only the paths the outbound traffic is taking. What you'd actually need is remote-triggered black hole. Search the Nanog archives for RTBH, you'll find a number of links in a message from Frank Bulk sent a few days ago. Or, you can prepend your advertisement with the troublesome ASN. Works for one or two troublesome ASNs as a quick hack at 3am - don't do it unless you understand why it works and why you shouldn't do it. -- Nathan Ward
RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) http://wiki.nil.com/Filter_excessively_prepended_BGP_paths Ivan http://www.ioshints.info/about http://blog.ioshints.info/
RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Ivan- Thanks for posting this how-to on excessive as prepends. I have a couple of questions that some of the less BGP savvy out their may find helpfull 1. In my enviornment, we are not doing full routes. We have partial routes from AS209 and then fail to AS7263. Is their any advantage for someone like me to do this, as we are not providing any IP transit so we are not passing the route table to anyone else? 2. When I run the sh ip bgp quote-regexp _([0-9]+)_\1_\1_\1_\1_ \1_ | begin Network I am seeing many many ASes that would be filtered by this access-list. What happens to those networks, are they unreachable from my AS, or do I just route those networks to my upstream provider and let them deal with it? 3. This last question is a little OT, but relates to your access list In the event of some kind if DOS attack coming from one of a few AS numbers (in this case we will use 14793), what is the feesability of using ip as-path access-list 100 deny _([0-9]+)_\1_\1_\1_\1_ ip as-path access-list 100 deny 14793 ip as-path access-list 100 permit .* Would this have any affect at all, or would my pipe from my upstream still be congested with garbage traffic ? Thanks Dylan Ebner -Original Message- From: Ivan Pepelnjak [mailto:i...@ioshints.info] Sent: Tuesday, August 18, 2009 2:37 AM To: 'randal k'; 'Adam Hebert' Cc: nanog@nanog.org Subject: RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ? Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) http://wiki.nil.com/Filter_excessively_prepended_BGP_paths Ivan http://www.ioshints.info/about http://blog.ioshints.info/
RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Ivan- Thanks for posting this how-to on excessive as prepends. I have a couple of questions that some of the less BGP savvy out their may find helpfull 1. In my enviornment, we are not doing full routes. We have partial routes from AS209 and then fail to AS7263. Is their any advantage for someone like me to do this, as we are not providing any IP transit so we are not passing the route table to anyone else? You could do it to protect your BGP table, but as you're not a transit AS, it does not make much sense. 2. When I run the sh ip bgp quote-regexp _([0-9]+)_\1_\1_\1_\1_ \1_ | begin Network I am seeing many many ASes that would be filtered by this access-list. Obviously a lot of people are prepend-happy. What happens to those networks, are they unreachable from my AS, or do I just route those networks to my upstream provider and let them deal with it? If I understood correctly, you're using a default route toward AS7263, which means that anything that is not in your BGP table (and consequently your IP routing table) will be sent out of your AS via the default route. If you're getting the paths you're filtering from AS209 that means that more traffic will go to AS7263. 3. This last question is a little OT, but relates to your access list In the event of some kind if DOS attack coming from one of a few AS numbers (in this case we will use 14793), what is the feesability of using ip as-path access-list 100 deny _([0-9]+)_\1_\1_\1_\1_ ip as-path access-list 100 deny 14793 ip as-path access-list 100 permit .* Would this have any affect at all, or would my pipe from my upstream still be congested with garbage traffic ? No. You cannot influence the inbound traffic apart from not advertising some of your prefixes to some of your neighbors or giving them hints with BGP communities or AS-path prepending. Whatever you do with BGP on your routers influences only the paths the outbound traffic is taking. What you'd actually need is remote-triggered black hole. Search the Nanog archives for RTBH, you'll find a number of links in a message from Frank Bulk sent a few days ago. Hope this helps Ivan http://www.ioshints.info/about http://blog.ioshints.info/
RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Ivan- This helps vey much. Thanks Dylan Ebner -Original Message- From: Ivan Pepelnjak [mailto:i...@ioshints.info] Sent: Tuesday, August 18, 2009 1:58 PM To: Dylan Ebner; 'randal k'; 'Adam Hebert' Cc: nanog@nanog.org Subject: RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ? Ivan- Thanks for posting this how-to on excessive as prepends. I have a couple of questions that some of the less BGP savvy out their may find helpfull 1. In my enviornment, we are not doing full routes. We have partial routes from AS209 and then fail to AS7263. Is their any advantage for someone like me to do this, as we are not providing any IP transit so we are not passing the route table to anyone else? You could do it to protect your BGP table, but as you're not a transit AS, it does not make much sense. 2. When I run the sh ip bgp quote-regexp _([0-9]+)_\1_\1_\1_\1_ \1_ | begin Network I am seeing many many ASes that would be filtered by this access-list. Obviously a lot of people are prepend-happy. What happens to those networks, are they unreachable from my AS, or do I just route those networks to my upstream provider and let them deal with it? If I understood correctly, you're using a default route toward AS7263, which means that anything that is not in your BGP table (and consequently your IP routing table) will be sent out of your AS via the default route. If you're getting the paths you're filtering from AS209 that means that more traffic will go to AS7263. 3. This last question is a little OT, but relates to your access list In the event of some kind if DOS attack coming from one of a few AS numbers (in this case we will use 14793), what is the feesability of using ip as-path access-list 100 deny _([0-9]+)_\1_\1_\1_\1_ ip as-path access-list 100 deny 14793 ip as-path access-list 100 permit .* Would this have any affect at all, or would my pipe from my upstream still be congested with garbage traffic ? No. You cannot influence the inbound traffic apart from not advertising some of your prefixes to some of your neighbors or giving them hints with BGP communities or AS-path prepending. Whatever you do with BGP on your routers influences only the paths the outbound traffic is taking. What you'd actually need is remote-triggered black hole. Search the Nanog archives for RTBH, you'll find a number of links in a message from Frank Bulk sent a few days ago. Hope this helps Ivan http://www.ioshints.info/about http://blog.ioshints.info/
RE: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
We are seeing the same thing, has anyone found the offending AS yet? Thanks ERIC -Original Message- From: Adam Hebert [mailto:a2t...@gmail.com] Sent: Monday, August 17, 2009 3:11 PM To: nanog@nanog.org Subject: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ? Multiple providers are seeing this right now. I assume someone is advertising an extremely long AS_PATH again? anyone else seeing this? Adam
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Throw your coffee at them! Just my two pence ;) ...James -BEGIN GEEK CODE BLOCK- Version: 3.1 GIT/MU/U dpu s: a-- C++$ U+ L++ B- P+ E? W+++$ N K W++ O M++$ V- PS+++ PE++ Y+ PGP t 5 X+ R- tv+ b+ DI D+++ G+ e(+) h--(++) r++ z++ --END GEEK CODE BLOCK--
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Yep, we started seeing this right around 12:20pm MST. We saw it from a customer's rapidly-flapping BGP peer. We told them to configure bgp maxas-limit, but apparently CRS1s don't have that command. Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) Cheers, Randal On Mon, Aug 17, 2009 at 2:11 PM, Adam Heberta2t...@gmail.com wrote: Multiple providers are seeing this right now. I assume someone is advertising an extremely long AS_PATH again? anyone else seeing this? Adam
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On Mon, Aug 17, 2009 at 03:37:07PM -0600, randal k wrote: Yep, we started seeing this right around 12:20pm MST. We saw it from a customer's rapidly-flapping BGP peer. We told them to configure bgp maxas-limit, but apparently CRS1s don't have that command. Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) Been a while since I had to throw this on cisco, but I since it lacks sane repeat constraint, you have to either choose to iterate over your acceptable space or deny on the longer-than-acceptable. For the latter, ^[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_([0-9]+_)+ clobbers 15 ASNs and longer. -- RSUC / GweepNet / Spunk / FnB / Usenix / SAGE
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On Aug 17, 2009, at 5:37 PM, randal k wrote: Yep, we started seeing this right around 12:20pm MST. We saw it from a customer's rapidly-flapping BGP peer. We told them to configure bgp maxas-limit, but apparently CRS1s don't have that command. Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) Is there some significant barrier to people getting recent code on the devices that is not impacted by this and the other fun bgp 'attacks' that can happen? We usually see customers drop bgp sessions all over, making me wonder ... if you're not able to upgrade, what is the issue? Just that most people don't see these as an attack against their infrastructure? That people are unwilling to upgrade code unless it has a long-term impact to their operations? An outage once every few months is OK? - Jared
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
I'd have to _assume_ that a lot of those impacted don't have a maint contract with their router vendor of choice and therefore don't have an easy path to upgrade. -jim --Original Message-- From: Jared Mauch To: randal k Cc: nanog@nanog.org Subject: Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ? Sent: Aug 17, 2009 7:40 PM On Aug 17, 2009, at 5:37 PM, randal k wrote: Yep, we started seeing this right around 12:20pm MST. We saw it from a customer's rapidly-flapping BGP peer. We told them to configure bgp maxas-limit, but apparently CRS1s don't have that command. Anybody have a handy route-map that will deny anything with a as-path longer than say 15-20? ;-) Is there some significant barrier to people getting recent code on the devices that is not impacted by this and the other fun bgp 'attacks' that can happen? We usually see customers drop bgp sessions all over, making me wonder ... if you're not able to upgrade, what is the issue? Just that most people don't see these as an attack against their infrastructure? That people are unwilling to upgrade code unless it has a long-term impact to their operations? An outage once every few months is OK? - Jared Sent from my BlackBerry device on the Rogers Wireless Network
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On Aug 17, 2009, at 6:45 PM, deles...@gmail.com wrote: I'd have to _assume_ that a lot of those impacted don't have a maint contract with their router vendor of choice and therefore don't have an easy path to upgrade. -jim Cisco gives out free software upgrades for any security(PSIRT) issue. Take the recent 4-byte asn crash advisory: -- snip -- http://www.cisco.com/en/US/products/products_security_advisory09186a0080aea4c9.shtml Customers should have their product serial number available and be prepared to give the URL of this notice as evidence of entitlement to a free upgrade. Free upgrades for non-contract customers must be requested through the TAC. -- snip -- I would view this as an active attack against the internet infrastructure and be working with the PSIRT team if it impacted my network ... - Jared
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On Mon, 17 Aug 2009 18:40:39 -0400, Jared Mauch ja...@puck.nether.net wrote: Is there some significant barrier to people getting recent code on the devices that is not impacted by this and the other fun bgp 'attacks' that can happen? In a word: YES. Any respectable ISP will not load code that has not been extensively tested. Failure to do so can, and WILL, lead to even greater impact outages. (we've all made that mistake. Once.) Unless you do millions with Cisco and can therefore get custom IOS builds, you won't get a newer version with *just* the intended bug fixed. Their maint rebuilds end up with multiple fixes and all too often, previous fixes reverted. (I stopped counting the number of times I had to bitch at them to refix the SNMP DLCI interface counters on the 7401... we don't test frame relay on the 7401 -- sure, that's eons ago, but nothing has changed over there.) And then there's the question of support... again, any respectable ISP maintains maint contracts with their vendors. But, things tend to fall through the cracks... contracts expire, people forget to list all the equipment, vendors drop support for various hardware and software, etc. You've obviously not gone to Cisco for any non-contract software updates. It's faster to bribe someone with an active service contract or use google. Also... Never underestimate the power of Lazy! --Ricky
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, Aug 17, 2009 at 4:26 PM, Ricky Beam jfb...@gmail.com wrote: Any respectable ISP will not load code that has not been extensively tested. [...] Just an observation on how things have changed in ~15 years: I recall Cisco code bugs that were fixed in semi- real-time, and quotes from tli: Code still warm from compiler. Confidence level: Boots in lab. :-) - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFKifMaq1pz9mNUZTMRAkIVAKD/G3MTDXLx9QydkEQq4azQZW4VGwCgrI+t y30sajrGH98vhOhcyyW0wm0= =CzXf -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
Confidence level: Boots in lab. One could argue that certain things haven't actually changed that much ;-). Marko. -- Marko CCIE #18427 (SP) My network blog: http://cisco.markom.info/
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
On Aug 17, 2009, at 5:17 PM, Paul Ferguson wrote: I recall Cisco code bugs that were fixed in semi- real-time, and quotes from tli: Code still warm from compiler. Confidence level: Boots in lab. IETF Dallas, 1995 I think. MCI Reston engg and Cisco (Ravi and others) in the terminal room cutting a new image of IS-IS for us because we'd tripped on the 24-day timer bug wherein all adjacencies would drop. Loaded on production routers that evening to fix the problem... good times :) -b
Re: Anyone else seeing (invalid or corrupt AS path) 3 bytes E01100 ?
From: Brett Watson br...@the-watsons.org Date: Mon, 17 Aug 2009 19:11:06 -0700 On Aug 17, 2009, at 5:17 PM, Paul Ferguson wrote: I recall Cisco code bugs that were fixed in semi- real-time, and quotes from tli: Code still warm from compiler. Confidence level: Boots in lab. IETF Dallas, 1995 I think. MCI Reston engg and Cisco (Ravi and others) in the terminal room cutting a new image of IS-IS for us because we'd tripped on the 24-day timer bug wherein all adjacencies would drop. Loaded on production routers that evening to fix the problem... good times :) I remember back in 1993 or 1994 getting a note from Tony Li chastising us for running BGP4 code that was over a WEEK old. How could we possibly expect things to work with code that far out of date? The amazing part is that BGP4 not only worked, but was very stable less then a month later when NSFnet shut down an se had to go to BGP4 to peer with everyone (like MCI, Sprint, NASA, and UUnet). Yep, it booted in the lab, all right. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: ober...@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
