Re: Bell Canada outage?

2012-08-08 Thread Darius Jahandarie
On Wed, Aug 8, 2012 at 2:31 PM, Zachary McGibbon zachary.mcgibbon+na...@gmail.com wrote: Anyone at Bell Canada / Sympatico can tell us what's going on? Our routing table is going nuts with Bell advertising a lot of routes they shouldn't be Bell leaked a full table. To add to the fun, it seems

Re: Bell Canada outage?

2012-08-08 Thread Chris Stone
On Wed, Aug 8, 2012 at 12:31 PM, Zachary McGibbon zachary.mcgibbon+na...@gmail.com wrote: Anyone at Bell Canada / Sympatico can tell us what's going on? Our routing table is going nuts with Bell advertising a lot of routes they shouldn't be Outages mailing list is reporting that Tata is

Re: Bell Canada outage?

2012-08-08 Thread Zachary McGibbon
That's what it looked like. We are connected (@ McGill University) to RISQ and a lot of our routes started to go via RISQ-Bell instead of RISQ-CANET/Canarie On Wed, Aug 8, 2012 at 2:35 PM, Chris Stone axi...@gmail.com wrote: On Wed, Aug 8, 2012 at 12:31 PM, Zachary McGibbon

Re: Bell Canada outage?

2012-08-08 Thread Jeff Wheeler
On Wed, Aug 8, 2012 at 2:35 PM, Chris Stone axi...@gmail.com wrote: Outages mailing list is reporting that Tata is having problems in Montreal affecting 'many routers'...maybe this is related? I am a transit customer of both TATA and Bell Canada. We saw route churn and heavy packet loss

Re: Bell Canada outage?

2012-08-08 Thread Mike Tancsa
On 8/8/2012 2:50 PM, Jeff Wheeler wrote: It also took over 10 minutes for my BGP withdraws to propagate from Bell to their neighbors, including Level3. I would guess Bell Canada's routers all have very busy CPU. I saw the same sort of behaviour from TATA. I shut my peer with them, yet

Re: Bell Canada outage?

2012-08-08 Thread Zachary McGibbon
This is what we saw via our upstream provider, it happened twice: [image: Inline image 1] On Wed, Aug 8, 2012 at 2:58 PM, Mike Tancsa m...@sentex.net wrote: On 8/8/2012 2:50 PM, Jeff Wheeler wrote: It also took over 10 minutes for my BGP withdraws to propagate from Bell to their

Re: Bell Canada outage?

2012-08-08 Thread Andree Toonk
Hi, .-- My secret spy satellite informs me that at 12-08-08 11:35 AM Darius Jahandarie wrote: On Wed, Aug 8, 2012 at 2:31 PM, Zachary McGibbon zachary.mcgibbon+na...@gmail.com wrote: Anyone at Bell Canada / Sympatico can tell us what's going on? Our routing table is going nuts with Bell

Re: Bell Canada outage?

2012-08-08 Thread Jared Mauch
On Aug 8, 2012, at 3:50 PM, Andree Toonk wrote: Hi, .-- My secret spy satellite informs me that at 12-08-08 11:35 AM Darius Jahandarie wrote: On Wed, Aug 8, 2012 at 2:31 PM, Zachary McGibbon zachary.mcgibbon+na...@gmail.com wrote: Anyone at Bell Canada / Sympatico can tell us what's

RE: Bell Canada outage?

2012-08-08 Thread Sharma, Kapeel
Could this be causing issue with XO in east coast ? -Original Message- From: Andree Toonk [mailto:andree+na...@toonk.nl] Sent: Wednesday, August 08, 2012 12:50 PM To: nanog@nanog.org Subject: Re: Bell Canada outage? Hi, .-- My secret spy satellite informs me that at 12-08-08 11:35 AM

Re: Bell Canada outage?

2012-08-08 Thread Zachary McGibbon
Thanks for the info, looks like Bell needs to put some filtering on their customer links! Things seem to have calmed down now but I'm still watching my prefixes received from my upstream provider: while true; do snmpget -v2c -c community router 1.3.6.1.4.1.9.9.187.1.2.4.1.1.ip of bgp peer.1.128;

Re: Bell Canada outage?

2012-08-08 Thread Harald Koch
On 8 August 2012 16:10, Zachary McGibbon zachary.mcgibbon+na...@gmail.comwrote: Thanks for the info, looks like Bell needs to put some filtering on their customer links! I remember when AS577 had those... ;) -- Harald

Re: Bell Canada outage?

2012-08-08 Thread Steve Dalberg
CPU's were pegged for a customer of mine in California. tracked it down to 2 events that went down at that time with a large message volume. 1) Peering between GLBX and Level3 dopped somewhere, causing many prefixes to shift away from L3 paths. 2) Some IPv6 prefixes were aggressively bouncing

Re: Bell Canada outage?

2012-08-08 Thread Andree Toonk
Further analysis shows that there were actually 107,409 prefixes affected of 14,391 unique origin ASn's. Interested if your prefixes was affected? I've uploaded a list of prefixes and ASn's that were leaked here: http://www.bgpmon.net/bell-leak.txt Cheers, Andree .-- My secret spy satellite

Re: Bell Canada outage?

2012-08-08 Thread Zachary McGibbon
Ouch!! That's a lot! What do you think the outcome of this will be? What do you think that Bell did/will do (hopefully) to fix this so it doesn't happen again? On Wed, Aug 8, 2012 at 4:31 PM, Andree Toonk andree+na...@toonk.nl wrote: Further analysis shows that there were actually 107,409

Re: Bell Canada outage?

2012-08-08 Thread David Miller
On 8/8/2012 4:14 PM, Steve Dalberg wrote: CPU's were pegged for a customer of mine in California. tracked it down to 2 events that went down at that time with a large message volume. 1) Peering between GLBX and Level3 dopped somewhere, causing many prefixes to shift away from L3 paths.

Re: Bell Canada outage?

2012-08-08 Thread Jeff Wheeler
We have been advised that TATA/6453 is back to normal, and re-activated our BGP to them. Everything seems okay on this front. No update from Bell Canada yet. On Wed, Aug 8, 2012 at 4:11 PM, Harald Koch c...@pobox.com wrote: On 8 August 2012 16:10, Zachary McGibbon Thanks for the info, looks

Re: Bell Canada outage?

2012-08-08 Thread up
Hi, .-- My secret spy satellite informs me that at 12-08-08 11:35 AM Darius Jahandarie wrote: On Wed, Aug 8, 2012 at 2:31 PM, Zachary McGibbon zachary.mcgibbon+na...@gmail.com wrote: Anyone at Bell Canada / Sympatico can tell us what's going on? Our routing table is going nuts with