Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 09:47 AM, andrew.wallace wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Never put down to malice which can be more easily explained by stupidity.. or in this case failure. RIM explained the problem earlier.. The messaging and browsing delays being experienced by BlackBerry users in Europe, the Middle East, Africa, India, Brazil, Chile and Argentina were caused by a core switch failure within RIM's infrastructure. Although the system is designed to failover to a back-up switch, the failover did not function as previously tested. As a result, a large backlog of data was generated and we are now working to clear that backlog and restore normal service as quickly as possible. We apologise for any inconvenience and we will continue to keep you informed. This appears to have been a result of a change on monday The problems began at about 11am on Monday. The Guardian understands that RIM was attempting a software upgrade on its database but suffered corruption problems, and that attempts to switch back to an older version led to a collapse http://www.guardian.co.uk/technology/2011/oct/12/blackberry-outage-executive-apologies?newsfeed=true thanks andrew On Wed, Oct 12, 2011 at 3:47 PM, andrew.wallace andrew.wall...@rocketmail.com wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
RE: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Maybe they use the same security solutions as Playstation Network does... that would explain a lot suddenly. Paul -Original Message- From: andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Wednesday, October 12, 2011 10:47 AM To: frnk...@iname.com Cc: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
On 10/12/11 07:47 , andrew.wallace wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ North American outages of the blackberry platform (particularly related to upgrades gone wrong) were not uncommon. Think for example sept 10, dec 18 and dec 22 2009. Andrew From: Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulk frnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
On Wed, 12 Oct 2011 07:47:13 PDT, andrew.wallace said: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. It ain't sabotage till you rule out misconfigured router. Consider the actual real-world threat models and their likelyhoods: 1) Insufficiently caffienated network engineer - this *NEVER* happens in real life, it's a total Bruce Schneier caliber movie-plot scenario. 2) Somebody sabotaging a RIM router. This is more likely, because there's just *bazillions* of people out there that stand to benefit from a RIM outage (and in fact profit more from an outage than from being able to watch traffic as it goes by). It's just a question of which one of those bazillions did it *this* time. Andrew, you *really* need to learn what the actual failure modes and root causes in real-life production networks are, and draw conclusions from reality, not whatever MI-7 inspired dream world the claim of sabotage came from. pgpBxmKLHmFJ6.pgp Description: PGP signature
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time). pgpP55SUQUVfz.pgp Description: PGP signature
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
+1 On Oct 12, 2011 11:51 AM, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Idiotberry Envoyé de mon iPhone Le 12 oct. 2011 à 17:55, Charles Mills w3y...@gmail.com a écrit : +1 On Oct 12, 2011 11:51 AM, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
I think it raises serious questions about RIM's DR strategy if a DB corruption or switch failure or whatever can cause this much outage. 'Surely' RIM have an second site that is independent of the primary (within reason) that they could of flipped to when they realised the DB was borked. If not then any business that relies on them needs to be shouting from the rooftops to get RIM to fix it. Chris. On 12 Oct 2011, at 16:49, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. Closest example would be an update that wasn't properly vetted in dev/test before migrating to prod. I've seen a few of those that I guess you could blame on the system. Even though the humans could have tested better -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 10:58 AM, Chris Campbell wrote: I think it raises serious questions about RIM's DR strategy if a DB corruption or switch failure or whatever can cause this much outage. 'Surely' RIM have an second site that is independent of the primary (within reason) that they could of flipped to when they realised the DB was borked. If not then any business that relies on them needs to be shouting from the rooftops to get RIM to fix it. Chris. On 12 Oct 2011, at 16:49, valdis.kletni...@vt.edu wrote: On Wed, 12 Oct 2011 09:52:02 CDT, -Hammer- said: What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. Yeah, and that extra comma in the one config file that didn't make a difference when you tested the failover in the lab *never* makes a difference when it hits in the production network, right? Or they changed the config of the primary and it didn't get propogated just right to the backup, or they had mismatched firmware levels on blades in the blades on the primary and backup switches, so traffic that didn't tickle a bug on the primary blades caused the blade to crash on the backup, or... Anybody on this list who's been around long enough probably has enough We should have had N+2 because the N+1'th device failed too stories to drain *several* pitchers of beer at a good pub... I've even had one case where my butt got *saved* from a ohnosecond-class whoops because the N+1'th device *was* crashed (stomped a config file, it replicated, was able to salvage a copy from a device that didn't replicate because it was down at the time).
RE: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
-Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: 12 October 2011 17:10 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. You have not seen VIP2-40s and CEF in action ;-) -- Leigh Porter __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
I have and totally get the point ... -- Michael Gatti cell.949.735.5612 ekim.it...@gmail.com (UTC-8) On Oct 12, 2011, at 9:12 AM, Leigh Porter wrote: -Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: 12 October 2011 17:10 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. You have not seen VIP2-40s and CEF in action ;-) -- Leigh Porter __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
RE: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
They are out there scrambling, trying to figure out where the truck that hit them came from. The PIO has been told to make up a story. Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email ralph.bra...@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055 -Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: Wednesday, October 12, 2011 10:52 AM To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) What kills me is what they have told the public. The lost a core switch. I don't know if they actually mean network switch or not but I'm pretty sure any of us that work on an enterprise environment know how to factor N+1 just for these types of days. And then the backup solution failed? I'm not buying it either. -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 09:47 AM, andrew.wallace wrote: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. http://news.cnet.com/8301-30686_3-20119163-266/blackberry-service-issues-spread-to-u.s-and-canada/ Andrew From: Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Tuesday, October 11, 2011 7:32 PM Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) And continues: “RIM'S SERVICE OUTAGE CONTINUES INTO DAY 2” http://www.channelstv.com/global/news_details.php?nid=29652cat=Politics Frank From:andrew.wallace [mailto:andrew.wall...@rocketmail.com] Sent: Monday, October 10, 2011 2:52 PM To: frnk...@iname.com Cc: outa...@outages.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) RIM shares down as BlackBerry outage continues http://www.marketwatch.com/story/rim-shares-down-as-blackberry-outage-continues-2011-10-10 Andrew From:Frank Bulkfrnk...@iname.com To: outa...@outages.org Sent: Monday, October 10, 2011 2:47 PM Subject: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) http://english.ahram.org.eg/NewsContent/3/12/23792/Business/Economy/Blackber ry-services-down-worldwide,-Egypt-affected.aspx FYI ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages ___ Outages mailing list outa...@outages.org https://puck.nether.net/mailman/listinfo/outages __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
- Original Message - From: Valdis Kletnieks valdis.kletni...@vt.edu On Wed, 12 Oct 2011 07:47:13 PDT, andrew.wallace said: Guys the outage has moved to U.S and Canada, I think we need to look at this perhaps being sabotage. It ain't sabotage till you rule out misconfigured router. Andrew, you *really* need to learn what the actual failure modes and root causes in real-life production networks are, and draw conclusions from reality, not whatever MI-7 inspired dream world the claim of sabotage came from. In fairness, Valdis, Andrew did not say this was obviously sabotage. He suggested that that possibility be added to the list of things which the RIM employees tasked with finding a root cause consider. I think the old filtering rule applies here: Once is happenstance. Twice is coincidence. Three times is enemy action. If this turns out to look like it came from 3 or more non-cascading failures, then sabotage will look a little more likely. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.)
Again. I know those stories are out there. I'm blessed with a lower profile or higher karma. One of the two. digging thru cube to fine wood to knock on -Hammer- I was a normal American nerd -Jack Herer On 10/12/2011 11:53 AM, Mike Gatti wrote: I have and totally get the point ... -- Michael Gatti cell.949.735.5612 ekim.it...@gmail.com (UTC-8) On Oct 12, 2011, at 9:12 AM, Leigh Porter wrote: -Original Message- From: -Hammer- [mailto:bhmc...@gmail.com] Sent: 12 October 2011 17:10 To: nanog@nanog.org Subject: Re: [outages] News item: Blackberry services down worldwide, Egypt affected (not N.A.) I have been witness to N+1 HUMAN failures but never a N+1 hardware failure or system/design failure that warranted questioning the need for N+2. Usually your N+1 failure is (as already referenced) pasting in a bad config that gets replicated or something like that. Not saying the hardware is perfect. It's just that I haven't personally seen a full blown failure like that without human help. You have not seen VIP2-40s and CEF in action ;-) -- Leigh Porter __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
I don't think that the Egyptian shutdown of domain names had much effect what shutdown of egyptian domain names? randy, who has a server which serves them there's an interesting point to be made about the geographic administrative and political distribution of secondaries being essential to insuring their survivability. Oddly your name is on bcp 16. clearly a forgery randy
Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
- Original Message - From: Martin Millnert milln...@gmail.com To: Marshall Eubanks t...@americafree.tv Cc: North American Network Operators Group nanog@nanog.org Sent: Thursday, 17 February, 2011 8:28:22 AM Subject: Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks t...@americafree.tv wrote: On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote: Operating local IRC networks is good, as is having local OS mirrors, such as Debian/Ubuntu and let's not forget, having a resilient DNS configuration (root zone copy hint 101: dig @k.root-servers.net. . axfr). A securely distributed Would it make sense for an ISP to store the root zone on their DNS servers instead of letting it be refreshed by the DNS cache? A cron job could refresh it from time to time. It would avoid entries from expiring and would always serve to clients entries with max ttl? A root server would be better, but that could be an intermediary step? Just speaking out loud here, so it may be total non-sense...
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes. http://www.renesys.com/blog/2011/01/egypt-leaves-the-internet.shtml notes, for example, that routes *through* Egypt were operational, but routes through the same fiber and the same routers *to* Egypt were non-functional. https://labs.ripe.net/Members/akvadrako/live_eqyptian_internet_incident_analysis pretty clearly states that prefixes associated with Egyptian ISPs were withdrawn. On Feb 16, 2011, at 11:50 AM, Franck Martin wrote: - Original Message - From: Martin Millnert milln...@gmail.com To: Marshall Eubanks t...@americafree.tv Cc: North American Network Operators Group nanog@nanog.org Sent: Thursday, 17 February, 2011 8:28:22 AM Subject: Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks t...@americafree.tv wrote: On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote: Operating local IRC networks is good, as is having local OS mirrors, such as Debian/Ubuntu and let's not forget, having a resilient DNS configuration (root zone copy hint 101: dig @k.root-servers.net. . axfr). A securely distributed Would it make sense for an ISP to store the root zone on their DNS servers instead of letting it be refreshed by the DNS cache? A cron job could refresh it from time to time. It would avoid entries from expiring and would always serve to clients entries with max ttl? A root server would be better, but that could be an intermediary step? Just speaking out loud here, so it may be total non-sense...
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
On 2/16/11 4:25 PM, Fred Baker wrote: I don't think that the Egyptian shutdown of domain names had much effect ... ditto. i'm not aware of any actions by the .eg registry operator, though i'll ask, coincidental to the prefix withdrawal. i suppose in the interests of completeness i should also ask about the (wicked recent) (مصر.) IDN ccTLD. these are both wicked small zones, relative to the density of names registered (registries other than .eg) by egyptian residents, and the larger number of network using egyptians. it is possible that as a preliminary step, the recursive resolver operators of each of the subsequently prefix withdrawing providers modified their cached data to provide policy-based resolution. -e
Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet
Never mind, Messrs. Cowie and Baker answered my question: http://mailman.nanog.org/pipermail/nanog/2011-February/033181.html Couldn't have paths through Egypt if layer 2 were cut off. (Right?) --Richard On Wed, Feb 16, 2011 at 5:38 PM, Richard Barnes richard.bar...@gmail.com wrote: It also seems like a question that could be decided empirically. Can anyone on here comment on whether or not the BGP session ended gracefully and the link lights remained lit? --Richard On Wed, Feb 16, 2011 at 9:09 AM, Marshall Eubanks t...@americafree.tv wrote: On Feb 16, 2011, at 12:15 AM, Joly MacFie wrote: http://www.nytimes.com/2011/02/16/technology/16internet.html There has been intense debate both inside and outside Egypt on whether the cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world. I do remember some intense debate, here and elsewhere, but I somehow don't remember those as being the primary debate parameters. Regards Marshall -- --- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote: I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes. Per the NYT article, the issue was the Egyptian Intranet -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc. --Steve Bellovin, http://www.cs.columbia.edu/~smb
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
On 2/16/11 6:10 PM, Steven Bellovin wrote: On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote: I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes. Per the NYT article, the issue was the Egyptian Intranet -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc. i'll have to check if the .eg servers were ever taken off-line. resolution of .com (beyond local caches) would have been pointless post-prefix withdrawal, but if the claim is that local ix routing remained possible, so non-cached .eg resolution was successful from outside of the eun to any other egyptian provider net, then if there is data to support the claim, it will be interesting. i'll ask. -e
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
ah On Feb 16, 2011, at 3:10 PM, Steven Bellovin wrote: On Feb 16, 2011, at 4:25 13PM, Fred Baker wrote: I don't think that the Egyptian shutdown of domain names had much effect; that's why the bgp prefixes were withdrawn. What was effective was the withdrawal of BGP prefixes. Per the NYT article, the issue was the Egyptian Intranet -- people couldn't contact other sites within Egypt by host name, even though the routes were up, because they couldn't resolve .eg, .com, etc. --Steve Bellovin, http://www.cs.columbia.edu/~smb
Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet
Mounir, On Wed, Feb 16, 2011 at 6:58 PM, Mounir Mohamed mounir.moha...@gmail.com wrote: No the BGP and the physical links were down. did you have any domestic BGP sessions up? Regards, Martin
Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet)
- Original Message - From: Randy Bush ra...@psg.com To: Fred Baker f...@cisco.com Cc: Franck Martin fra...@genius.com, North American Network Operators Group nanog@nanog.org Sent: Thursday, 17 February, 2011 2:37:02 PM Subject: Re: Local root zone (Was NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet) I don't think that the Egyptian shutdown of domain names had much effect what shutdown of egyptian domain names? randy, who has a server which serves them The ASCII one .eg or the UTF8 one .xn--wgbh1c? xn--wgbh1c. 172800 IN NS ns1.dotmasr.eg. xn--wgbh1c. 172800 IN NS ns2.dotmasr.eg. xn--wgbh1c. 172800 IN NS ns3.dotmasr.eg. eg. 172800 IN NS ns5.univie.ac.at. eg. 172800 IN NS rip.psg.com. eg. 172800 IN NS frcu.eun.eg.
Re: NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet
No the BGP and the physical links were down. What about non-Internet layer 2 links? A number of companies have private IP networks extending into Egypt providing MPLS or other VPN services. In addition, there are often longlines into the Gulf states to provide the Egyptian sites with redundancy. Were these communications also cut? One way to find out would be to talk to the networking folks at any major international consumer brand that is in Egypt. I would expect that nowadays if a Coke or a Pepsi is in a country, they will have some kind of IP VPN crossing that country's borders. --Michael Dillon http://www.linkedin.com/profile/view?id=13566587
NYTimes: Egypt Leaders Found ‘Off’ Switch for Internet
http://www.nytimes.com/2011/02/16/technology/16internet.html There has been intense debate both inside and outside Egypt on whether the cutoff at 26 Ramses Street was accomplished by surgically tampering with the software mechanism that defines how networks at the core of the Internet communicate with one another, or by a blunt approach: simply cutting off the power to the router computers that connect Egypt to the outside world. -- --- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http://isoc-ny.org ---
Egypt: direct economic cost estimated at $18m/day
This is from a 3% to 4% estimate of telecomms and datacomms in the overall Egyptian economy. The OEDC communique notes that attracting foreign investment may now be more difficult. (Is there anyone not looking at regional alternatives?) Source: http://www.lemonde.fr/technologies/article/2011/02/03/egypte-la-coupure-internet-a-coute-90-millions-de-dollars_1474489_651865.html
Re: Egypt 'hijacked Vodafone network'
On Feb 3, 2011, at 9:24 AM, andrew.wallace wrote: Mobile phone firm Vodafone accuses the Egyptian authorities of using its network to send pro-government text messages. http://www.bbc.co.uk/news/business-12357694 Here is their PR http://www.vodafone.com/content/index/press.html Note that this is entirely legal, under the emergency powers provisions of the Telecoms Act Regards Marshall Andrew
Re: Egypt 'hijacked Vodafone network'
On 02/03/2011 10:14 EST, Marshall Eubanks wrote: On Feb 3, 2011, at 9:24 AM, andrew.wallace wrote: Mobile phone firm Vodafone accuses the Egyptian authorities of using its network to send pro-government text messages. http://www.bbc.co.uk/news/business-12357694 Here is their PR http://www.vodafone.com/content/index/press.html Note that this is entirely legal, under the emergency powers provisions of the Telecoms Act Which is legal, Vodafone's protest or the government's telling them to send messages? afaik the agreement was that the operator would have preloaded canned messages, agreed on in advance with the government, and now the government is telling them to send out arbitrary messages they compose on the spot.
Re: Egypt 'hijacked Vodafone network'
On Feb 3, 2011, at 2:20 PM, andrew.wallace wrote: On Thu, Feb 3, 2011 at 6:59 PM, Scott Brim scott.b...@gmail.com wrote: On 02/03/2011 10:14 EST, Marshall Eubanks wrote: On Feb 3, 2011, at 9:24 AM, andrew.wallace wrote: Mobile phone firm Vodafone accuses the Egyptian authorities of using its network to send pro-government text messages. http://www.bbc.co.uk/news/business-12357694 Here is their PR http://www.vodafone.com/content/index/press.html Note that this is entirely legal, under the emergency powers provisions of the Telecoms Act Which is legal, Vodafone's protest or the government's telling them to send messages? afaik the agreement was that the operator would have preloaded canned messages, agreed on in advance with the government, and now the government is telling them to send out arbitrary messages they compose on the spot. I wonder if these messages were blockable by the end-user or if they were being sent as a service announcement from Vodafone. Certainly, if the government were sending the messages under the company name then something sounds wrong about that. What I would like is to hear from someone who received the messages and what their experiences were. They were described to me as being from Vodafone. I assumed that this meant that they were service messages. Marshall Andrew
Re: Egypt 'hijacked Vodafone network'
On Thu, Feb 3, 2011 at 7:48 PM, Marshall Eubanks t...@americafree.tv wrote: On Feb 3, 2011, at 2:20 PM, andrew.wallace wrote: On Thu, Feb 3, 2011 at 6:59 PM, Scott Brim scott.b...@gmail.com wrote: On 02/03/2011 10:14 EST, Marshall Eubanks wrote: On Feb 3, 2011, at 9:24 AM, andrew.wallace wrote: Mobile phone firm Vodafone accuses the Egyptian authorities of using its network to send pro-government text messages. http://www.bbc.co.uk/news/business-12357694 Here is their PR http://www.vodafone.com/content/index/press.html Note that this is entirely legal, under the emergency powers provisions of the Telecoms Act Which is legal, Vodafone's protest or the government's telling them to send messages? afaik the agreement was that the operator would have preloaded canned messages, agreed on in advance with the government, and now the government is telling them to send out arbitrary messages they compose on the spot. I wonder if these messages were blockable by the end-user or if they were being sent as a service announcement from Vodafone. Certainly, if the government were sending the messages under the company name then something sounds wrong about that. What I would like is to hear from someone who received the messages and what their experiences were. They were described to me as being from Vodafone. I assumed that this meant that they were service messages. Marshall A text message received Sunday by an Associated Press reporter in Egypt appealed to the country's honest and loyal men to confront the traitors and criminals and protect our people and honor. Another urged Egyptians to attend a pro-Mubarak rally in Cairo on Wednesday. The first was marked as coming from Vodafone. The other was signed: Egypt Lovers. http://news.yahoo.com/s/ap/20110203/ap_on_hi_te/eu_egypt_cell_phones Andrew
Re: Egypt 'hijacked Vodafone network'
That is horrible Next thing you know they'll be sending SMS messages to the people saying TEXT 666 to donate 58 Egyption Pounds to support Mubarak On Thu, Feb 3, 2011 at 1:32 PM, andrew.wallace andrew.wall...@rocketmail.com wrote: On Thu, Feb 3, 2011 at 7:48 PM, Marshall Eubanks t...@americafree.tv wrote: On Feb 3, 2011, at 2:20 PM, andrew.wallace wrote: On Thu, Feb 3, 2011 at 6:59 PM, Scott Brim scott.b...@gmail.com wrote: On 02/03/2011 10:14 EST, Marshall Eubanks wrote: On Feb 3, 2011, at 9:24 AM, andrew.wallace wrote: Mobile phone firm Vodafone accuses the Egyptian authorities of using its network to send pro-government text messages. http://www.bbc.co.uk/news/business-12357694 Here is their PR http://www.vodafone.com/content/index/press.html Note that this is entirely legal, under the emergency powers provisions of the Telecoms Act Which is legal, Vodafone's protest or the government's telling them to send messages? afaik the agreement was that the operator would have preloaded canned messages, agreed on in advance with the government, and now the government is telling them to send out arbitrary messages they compose on the spot. I wonder if these messages were blockable by the end-user or if they were being sent as a service announcement from Vodafone. Certainly, if the government were sending the messages under the company name then something sounds wrong about that. What I would like is to hear from someone who received the messages and what their experiences were. They were described to me as being from Vodafone. I assumed that this meant that they were service messages. Marshall A text message received Sunday by an Associated Press reporter in Egypt appealed to the country's honest and loyal men to confront the traitors and criminals and protect our people and honor. Another urged Egyptians to attend a pro-Mubarak rally in Cairo on Wednesday. The first was marked as coming from Vodafone. The other was signed: Egypt Lovers. http://news.yahoo.com/s/ap/20110203/ap_on_hi_te/eu_egypt_cell_phones Andrew
Re: Egypt 'hijacked Vodafone network'
On Thu, 03 Feb 2011 13:41:15 PST, Mike Lyon said: That is horrible Next thing you know they'll be sending SMS messages to the people saying TEXT 666 to donate 58 Egyption Pounds to support Mubarak I got an e-mail this morning... Attention: I am a consultant to the Egyptian President. I am contacting you for a possible business deal based on the present political crisis in Egypt. The conglomerate of Mobarak is ready to partner with you to help secure the resources of the president since the office of the presidency has been dissolved. pgpqZAP24Yh6I.pgp Description: PGP signature
Re: Connectivity status for Egypt
On Tue, Feb 1, 2011 at 21:30, Marshall Eubanks t...@americafree.tv wrote: On Jan 31, 2011, at 5:14 PM, Marshall Eubanks wrote: As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. I have been told that the Egyptian Prime Minister has publicly announced that the Internet would be restored soon, but at present neither my Looks like it's coming back: http://stat.ripe.net/egypt ~2500 prefixes being announced now. -- teo - http://www.teoruiz.com Res publica non dominetur
Re: Connectivity status for Egypt
On Wed, Feb 2, 2011 at 6:17 AM, Teo Ruiz teor...@gmail.com wrote: On Tue, Feb 1, 2011 at 21:30, Marshall Eubanks t...@americafree.tv wrote: On Jan 31, 2011, at 5:14 PM, Marshall Eubanks wrote: As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. I have been told that the Egyptian Prime Minister has publicly announced that the Internet would be restored soon, but at present neither my Looks like it's coming back: http://stat.ripe.net/egypt ~2500 prefixes being announced now. -- teo - http://www.teoruiz.com Res publica non dominetur Yes, confirmed from 09:29 UTC. Basically all major providers are back, full status quo ante (modulo reagg), major sites are up. http://www.renesys.com/blog/2011/02/egypt-returns-to-the-internet.shtml Good thoughts go out to the guys in the EG NOCs this morning.Nanog wants to hear your war stories some day over a cup of tea. --jim
Re: Connectivity status for Egypt
On Wed, Feb 02, 2011 at 06:23:39AM -0500, Jim Cowie co...@renesys.com wrote a message of 29 lines which said: Yes, confirmed from 09:29 UTC. Basically all major providers are back, full status quo ante (modulo reagg), major sites are up. EUN (the academic network, which includes the primary name server for .EG) is still unreachable (1130 UTC).
Re: Connectivity status for Egypt
On Wed, Feb 02, 2011 at 12:30:45PM +0100, Stephane Bortzmeyer bortzme...@nic.fr wrote a message of 10 lines which said: EUN (the academic network, which includes the primary name server for .EG) is still unreachable (1130 UTC). It works now (1137 UTC). BGP was a bit slow.
Re: Connectivity status for Egypt
On Feb 2, 2011, at 6:23 AM, Jim Cowie wrote: On Wed, Feb 2, 2011 at 6:17 AM, Teo Ruiz teor...@gmail.com wrote: On Tue, Feb 1, 2011 at 21:30, Marshall Eubanks t...@americafree.tv wrote: On Jan 31, 2011, at 5:14 PM, Marshall Eubanks wrote: As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. I have been told that the Egyptian Prime Minister has publicly announced that the Internet would be restored soon, but at present neither my Looks like it's coming back: http://stat.ripe.net/egypt ~2500 prefixes being announced now. -- teo - http://www.teoruiz.com Res publica non dominetur Yes, confirmed from 09:29 UTC. Basically all major providers are back, full status quo ante (modulo reagg), major sites are up. http://www.renesys.com/blog/2011/02/egypt-returns-to-the-internet.shtml It's not just BGP - DNS (based on the samples I have been testing) seems to be fully back too. Regards Marshall Good thoughts go out to the guys in the EG NOCs this morning.Nanog wants to hear your war stories some day over a cup of tea. --jim
Egypt
Hi again from Network World... We're now looking into a story on how Egypt may have restored service -- did they bring up all routes at once? Stagger the re-introduction of routes so as not to overwhelm routers? Any specific ISPs brought up before others and why? ie, Noor and the stock exchange brought up before any others, etc. Any thoughts from NANOG on how best -- or worst -- to restore Internet service following, reportedly, the largest government-mandated blackout ever? Also, are any of you in touch with any Egyptian ISPs and sharing this type of information? Or hearing any war stories from over there? Thanks again, and best regards, Jim Duffy Network World
Re: Connectivity status for Egypt
On Jan 31, 2011, at 5:14 PM, Marshall Eubanks wrote: As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. I have been told that the Egyptian Prime Minister has publicly announced that the Internet would be restored soon, but at present neither my monitoring nor http://stat.ripe.net/egypt/ confirms this. Regards Marshall DNS for egyptse.com also appears to be down, but Noor.net is definitely withdrawn : dig www.noor.net ; DiG 9.6.0-APPLE-P2 www.noor.net ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 15709 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.noor.net.IN A ;; ANSWER SECTION: www.noor.net. 503 IN CNAME noor.net. noor.net. 503 IN A 217.139.227.20 show ip bgp 217.139.227.20 % Network not in table Marshall On Jan 28, 2011, at 4:37 PM, Franck Martin wrote: If I'm correct, in 2000 in Fiji, the main fiber optic cable from the national provider to the international provider was sabotaged, cutting all communications. Fortunately an Alcatel team was on the island (SCC commissioning) with the right tools and could splice it back in a few hours, otherwise Fiji would have gone dark for days... - Original Message - From: Joe Abley jab...@hopcount.ca To: Marshall Eubanks t...@americafree.tv Cc: nanog@nanog.org Sent: Saturday, 29 January, 2011 7:32:07 AM Subject: Re: Connectivity status for Egypt On 2011-01-28, at 11:33, Marshall Eubanks wrote: On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. It was not the case in Nepal in 2005 though, if I remember correctly. In that case connectivity to the outside was maintained, but access to that connectivity by people inside the country was curtailed. Joe
Re: Connectivity status for Egypt
As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. DNS for egyptse.com also appears to be down, but Noor.net is definitely withdrawn : dig www.noor.net ; DiG 9.6.0-APPLE-P2 www.noor.net ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 15709 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.noor.net. IN A ;; ANSWER SECTION: www.noor.net. 503 IN CNAME noor.net. noor.net. 503 IN A 217.139.227.20 show ip bgp 217.139.227.20 % Network not in table Marshall On Jan 28, 2011, at 4:37 PM, Franck Martin wrote: If I'm correct, in 2000 in Fiji, the main fiber optic cable from the national provider to the international provider was sabotaged, cutting all communications. Fortunately an Alcatel team was on the island (SCC commissioning) with the right tools and could splice it back in a few hours, otherwise Fiji would have gone dark for days... - Original Message - From: Joe Abley jab...@hopcount.ca To: Marshall Eubanks t...@americafree.tv Cc: nanog@nanog.org Sent: Saturday, 29 January, 2011 7:32:07 AM Subject: Re: Connectivity status for Egypt On 2011-01-28, at 11:33, Marshall Eubanks wrote: On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. It was not the case in Nepal in 2005 though, if I remember correctly. In that case connectivity to the outside was maintained, but access to that connectivity by people inside the country was curtailed. Joe
Re: Connectivity status for Egypt
On Mon, Jan 31, 2011 at 2:14 PM, Marshall Eubanks t...@americafree.tvwrote: As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. Yep, Noor is now down. Those on the ground with Noor DSL in Cairo contacted their front line support, and they're saying technical problems that will take a few hours to fix. Does anyone has a list of routes that are still up, and seem to correlate with Egyptian locations? Andree's last list is here: http://bgpmon.net/egypt-routes-jan29-2011.txt I'm staring at looking glass output to check these remaining routes, and that seems unfair on both those offering those free services, and my own sanity... d. DNS for egyptse.com also appears to be down, but Noor.net is definitely withdrawn : dig www.noor.net ; DiG 9.6.0-APPLE-P2 www.noor.net ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 15709 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.noor.net. IN A ;; ANSWER SECTION: www.noor.net. 503 IN CNAME noor.net. noor.net. 503 IN A 217.139.227.20 show ip bgp 217.139.227.20 % Network not in table Marshall On Jan 28, 2011, at 4:37 PM, Franck Martin wrote: If I'm correct, in 2000 in Fiji, the main fiber optic cable from the national provider to the international provider was sabotaged, cutting all communications. Fortunately an Alcatel team was on the island (SCC commissioning) with the right tools and could splice it back in a few hours, otherwise Fiji would have gone dark for days... - Original Message - From: Joe Abley jab...@hopcount.ca To: Marshall Eubanks t...@americafree.tv Cc: nanog@nanog.org Sent: Saturday, 29 January, 2011 7:32:07 AM Subject: Re: Connectivity status for Egypt On 2011-01-28, at 11:33, Marshall Eubanks wrote: On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. It was not the case in Nepal in 2005 though, if I remember correctly. In that case connectivity to the outside was maintained, but access to that connectivity by people inside the country was curtailed. Joe
Re: Connectivity status for Egypt
Hi, Danny. Does anyone has a list of routes that are still up, and seem to correlate with Egyptian locations? Andree's last list is here: http://bgpmon.net/egypt-routes-jan29-2011.txt We see the following ASNs presently: ASN CC AS Name 6762| IT | SEABONE-NET TELECOM ITALIA SPARKLE S.p.A. 8452| IT | TE-AS TE-AS 15834 | EG | Menanet-AS 24863 | EG | LINKdotNET-AS 28045 | DO | Pantel Communications 33782 | EG | BA-AS 33789 | EG | Internet2 36992 | EG | ETISALAT-MISR We saw the count of prefixes decrease from circa 204 to 98. We see the following IPv4 prefixes presently: IPv4 Prefix ASN 41.32.214.0/24 | 8452 41.32.215.0/24 | 8452 41.78.60.0/22 | 6762 41.129.96.0/22 | 24863 41.152.0.0/16 | 36992 41.152.0.0/17 | 36992 41.152.0.0/18 | 36992 41.152.64.0/18 | 36992 41.152.64.0/19 | 36992 41.152.128.0/17 | 36992 41.152.128.0/19 | 36992 41.152.160.0/19 | 36992 41.152.185.0/24 | 36992 41.152.192.0/18 | 36992 41.152.192.0/19 | 36992 41.152.194.0/24 | 36992 41.152.195.0/24 | 36992 41.152.197.0/24 | 36992 41.152.198.0/24 | 36992 41.153.0.0/16 | 36992 41.153.0.0/17 | 36992 41.153.0.0/19 | 36992 41.153.64.0/18 | 36992 41.153.128.0/17 | 36992 41.153.128.0/24 | 36992 41.153.133.0/24 | 36992 41.153.136.0/21 | 36992 41.153.166.0/24 | 36992 41.153.192.0/18 | 36992 41.153.195.0/24 | 36992 41.153.196.0/24 | 36992 41.153.197.0/24 | 36992 41.153.198.0/24 | 36992 41.153.199.0/24 | 36992 41.153.200.0/24 | 36992 41.153.201.0/24 | 36992 41.153.202.0/24 | 36992 41.153.203.0/24 | 36992 41.153.204.0/24 | 36992 41.153.224.0/20 | 36992 41.153.224.0/21 | 36992 41.178.15.0/24 | 24863 41.178.49.0/24 | 24863 41.178.51.0/24 | 24863 41.196.0.0/24 | 24863 41.196.200.0/24 | 24863 41.222.128.0/21 | 36992 41.222.128.0/23 | 36992 41.222.128.0/24 | 36992 41.222.129.0/24 | 36992 41.222.130.0/24 | 36992 41.222.132.0/24 | 36992 41.235.24.0/24 | 8452 62.12.96.0/19 | 15834 62.241.134.0/24 | 24863 81.4.0.0/18 | 15834 81.10.56.0/24 | 8452 81.10.81.0/24 | 8452 81.10.82.0/24 | 8452 81.10.83.0/24 | 8452 81.10.114.0/24 | 8452 81.10.116.0/22 | 8452 81.10.122.0/23 | 8452 81.10.124.0/22 | 8452 81.10.127.0/24 | 8452 81.21.100.0/24 | 33789 81.21.110.0/24 | 33789 82.201.143.0/24 | 24863 84.233.0.0/17 | 36992 163.121.128.0/24 | 8452 163.121.170.0/24 | 8452 163.121.190.0/24 | 8452 163.121.229.0/24 | 8452 195.43.10.0/24 | 24863 195.246.37.0/24 | 24863 195.246.38.0/24 | 24863 196.204.160.0/19 | 33782 196.205.23.0/24 | 24863 196.205.70.0/24 | 24863 196.205.93.0/24 | 24863 196.218.248.0/22 | 8452 196.218.252.0/22 | 8452 196.219.248.0/22 | 8452 196.219.252.0/22 | 8452 197.192.0.0/13 | 36992 197.192.0.0/17 | 36992 197.193.0.0/18 | 36992 197.193.0.0/19 | 36992 197.193.32.0/19 | 36992 197.194.128.0/17 | 36992 197.195.0.0/17 | 36992 212.103.160.0/24 | 8452 212.103.169.0/24 | 8452 213.131.64.0/24 | 24863 213.181.236.0/24 | 33789 213.247.0.0/20 | 28045 213.247.16.0/20 | 28045 217.29.128.0/20 | 15834 Thanks, Rob. -- Rob Thomas Team Cymru https://www.team-cymru.org/ Say little and do much. M Avot 1:15
Re: Connectivity status for Egypt
On Jan 31, 2011, at 5:41 PM, Danny O'Brien wrote: On Mon, Jan 31, 2011 at 2:14 PM, Marshall Eubanks t...@americafree.tvwrote: As an update, BGP for Noor.net has been withdrawn. Even the Egyptian stock exchange - egyptse.com - now appears to be off the Internet. Yep, Noor is now down. Collateral damage from all of this, as detailed in http://blog.icann.org/2011/01/status-report-on-the-dns-in-egypt/ is that the Arabic script top-level domain .masr (مصر) has been unavailable since the 27th, since it is is operated by NTRA of Egypt. Regards Marshall Those on the ground with Noor DSL in Cairo contacted their front line support, and they're saying technical problems that will take a few hours to fix. Does anyone has a list of routes that are still up, and seem to correlate with Egyptian locations? Andree's last list is here: http://bgpmon.net/egypt-routes-jan29-2011.txt I'm staring at looking glass output to check these remaining routes, and that seems unfair on both those offering those free services, and my own sanity... d. DNS for egyptse.com also appears to be down, but Noor.net is definitely withdrawn : dig www.noor.net ; DiG 9.6.0-APPLE-P2 www.noor.net ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 15709 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.noor.net. IN A ;; ANSWER SECTION: www.noor.net. 503 IN CNAME noor.net. noor.net. 503 IN A 217.139.227.20 show ip bgp 217.139.227.20 % Network not in table Marshall On Jan 28, 2011, at 4:37 PM, Franck Martin wrote: If I'm correct, in 2000 in Fiji, the main fiber optic cable from the national provider to the international provider was sabotaged, cutting all communications. Fortunately an Alcatel team was on the island (SCC commissioning) with the right tools and could splice it back in a few hours, otherwise Fiji would have gone dark for days... - Original Message - From: Joe Abley jab...@hopcount.ca To: Marshall Eubanks t...@americafree.tv Cc: nanog@nanog.org Sent: Saturday, 29 January, 2011 7:32:07 AM Subject: Re: Connectivity status for Egypt On 2011-01-28, at 11:33, Marshall Eubanks wrote: On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. It was not the case in Nepal in 2005 though, if I remember correctly. In that case connectivity to the outside was maintained, but access to that connectivity by people inside the country was curtailed. Joe
Re: Connectivity status for Egypt
On Jan 28, 2011, at 8:32 AM, Mirjam Kuehne wrote: Hi, We did some analysis of the situation in Egypt using the RIPEstat toolbox (please note, this is a prototype and we're not sure how it will handle a big load): http://labs.ripe.net/Members/akvadrako/live_eqyptian_internet_incident_analysis This - specifically http://stat.ripe.net/egypt/ - shows another big spike today, which is presumably when Noor.net was pulled. Marshall Mirjam Kuehne RIPE NCC Carlos Alcantar wrote: Looks like you can still make phone calls into Egypt. So it's not totally lights out... Carlos Alcantar Race Communications / Race Team Member 101 Haskins Way, So. San Francisco, CA. 94080 Phone: +1 415 376 3314 Fax: +1 650 246 8901 / carlos *at* race.com / www.race.com -Original Message- From: Paul Ferguson [mailto:fergdawgs...@gmail.com] Sent: Thursday, January 27, 2011 11:46 PM To: Joel Jaeggli Cc: nanog@nanog.org Subject: Re: Connectivity status for Egypt -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 27, 2011 at 11:39 PM, Joel Jaeggli joe...@bogus.com wrote: On 1/27/11 10:49 PM, Roy wrote: Moral of the story: Separate facts from assumptions and guesses. I did some Google searches and that region has had large scale disruptions in the past. Several cables follow the same path to the Suez canal and were hit. my links through the region are all fine, but they don't jump off the cable in egypt just pass through. https://secure.wikimedia.org/wikipedia/en/wiki/2008_submarine_cable_d isr uption To my knowledge, no one has reported any cable problems in Norther Africa - -- and news of those problems generally travels very fast. :-) Also, if there *was* a cable problem on one of the paths through the vicinity, it affect more than just Egypt: https://secure.wikimedia.org/wikipedia/en/wiki/File:Cable_map18.svg I don't think it takes a leap of imagination to understand what has happened here. - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFNQnQ0q1pz9mNUZTMRAoFQAKCE8P0wINouFWUvW9GFn7FR6XVmOwCdGV/i VzTaxnJQOPVqyY2bP8ZraDA= =daOC -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Re: Connectivity status for Egypt
Hi Danny, .-- My secret spy satellite informs me that at 11-01-31 2:41 PM Danny O'Brien wrote: Does anyone has a list of routes that are still up, and seem to correlate with Egyptian locations? Andree's last list is here: http://bgpmon.net/egypt-routes-jan29-2011.txt Here's an updated list: http://www.bgpmon.net/egypt-routes-jan31-2011.txt Also see: http://bgpmon.net/blog/?p=450#lastupdate Cheers, Andree
RE: Connectivity status for Egypt
Here's an updated list: http://www.bgpmon.net/egypt-routes-jan31-2011.txt Some decent opportunities for route aggregation in that list...
Fwd: [listname] Fwd: Connectivity status for Egypt
Anonymized because I am forwarding without permission... Dears, As per what I know from friends: 1. All Internet is down. Apparently one network is still working but seems to be serving the stock exchange only and few others. 2. SMS is down. 3. Landlines are down (at least internationally). 4. Mobile networks seem to be on and off depending on location. For example, Vodafone is apparently working in Cairo while other networks are not !! God help them and bring Egypt and Egyptians out of this harmless inshallah. Regards.. Owen
Re: [menog] Fwd: Connectivity status for Egypt
Dear all, On 1/28/11 1:07 AM, Richard Barnes wrote: Hey all, Some NANOG participants are seeing hearing reports of disrupted communications in Egypt. Are any of you seeing the same thing? --Richard Here is the analysis of BGP table regarding what happened to the Internet in Egypt: http://stat.ripe.net/egypt/ https://labs.ripe.net/Members/akvadrako/live_eqyptian_internet_incident_analysis Vesna Manojlovic RIPE NCC Trainer -- Forwarded message -- From: Danny O'Brien da...@spesh.com Date: Thu, Jan 27, 2011 at 6:47 PM Subject: Connectivity status for Egypt To: NANOG Operators' Group nanog@nanog.org Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) Thank you, -- dobr...@cpj.org Danny O'Brien, Committee to Protect Journalists gpg key: http://www.spesh.com/danny/crypto/dannyobrien-key20091106.txt ___ Menog mailing list me...@menog.net http://lists.menog.net/mailman/listinfo/menog
Re: [menog] Fwd: Connectivity status for Egypt
Here is the analysis of BGP table regarding what happened to the Internet in Egypt: http://stat.ripe.net/egypt/ https://labs.ripe.net/Members/akvadrako/live_eqyptian_internet_incident_analysis Cidr report (http://www.cidr-report.org) shows this also very well: Recent Table History Date PrefixesCIDR Agg 26-01-11345293 201663 27-01-11344858 200621 28-01-11342381 201194 Top 20 Net Decreased Routes per Originating AS Prefixes Change ASnum AS Description -102102-0 AS5536 Internet-Egypt Kind regards, Ingo Flaschberger
RE: Connectivity status for Egypt
Looks like you can still make phone calls into Egypt. So it's not totally lights out... Carlos Alcantar Race Communications / Race Team Member 101 Haskins Way, So. San Francisco, CA. 94080 Phone: +1 415 376 3314 Fax: +1 650 246 8901 / carlos *at* race.com / www.race.com -Original Message- From: Paul Ferguson [mailto:fergdawgs...@gmail.com] Sent: Thursday, January 27, 2011 11:46 PM To: Joel Jaeggli Cc: nanog@nanog.org Subject: Re: Connectivity status for Egypt -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 27, 2011 at 11:39 PM, Joel Jaeggli joe...@bogus.com wrote: On 1/27/11 10:49 PM, Roy wrote: Moral of the story: Separate facts from assumptions and guesses. I did some Google searches and that region has had large scale disruptions in the past. Several cables follow the same path to the Suez canal and were hit. my links through the region are all fine, but they don't jump off the cable in egypt just pass through. https://secure.wikimedia.org/wikipedia/en/wiki/2008_submarine_cable_d isr uption To my knowledge, no one has reported any cable problems in Norther Africa - -- and news of those problems generally travels very fast. :-) Also, if there *was* a cable problem on one of the paths through the vicinity, it affect more than just Egypt: https://secure.wikimedia.org/wikipedia/en/wiki/File:Cable_map18.svg I don't think it takes a leap of imagination to understand what has happened here. - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFNQnQ0q1pz9mNUZTMRAoFQAKCE8P0wINouFWUvW9GFn7FR6XVmOwCdGV/i VzTaxnJQOPVqyY2bP8ZraDA= =daOC -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Re: Connectivity status for Egypt
Hi, We did some analysis of the situation in Egypt using the RIPEstat toolbox (please note, this is a prototype and we're not sure how it will handle a big load): http://labs.ripe.net/Members/akvadrako/live_eqyptian_internet_incident_analysis Mirjam Kuehne RIPE NCC Carlos Alcantar wrote: Looks like you can still make phone calls into Egypt. So it's not totally lights out... Carlos Alcantar Race Communications / Race Team Member 101 Haskins Way, So. San Francisco, CA. 94080 Phone: +1 415 376 3314 Fax: +1 650 246 8901 / carlos *at* race.com / www.race.com -Original Message- From: Paul Ferguson [mailto:fergdawgs...@gmail.com] Sent: Thursday, January 27, 2011 11:46 PM To: Joel Jaeggli Cc: nanog@nanog.org Subject: Re: Connectivity status for Egypt -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 27, 2011 at 11:39 PM, Joel Jaeggli joe...@bogus.com wrote: On 1/27/11 10:49 PM, Roy wrote: Moral of the story: Separate facts from assumptions and guesses. I did some Google searches and that region has had large scale disruptions in the past. Several cables follow the same path to the Suez canal and were hit. my links through the region are all fine, but they don't jump off the cable in egypt just pass through. https://secure.wikimedia.org/wikipedia/en/wiki/2008_submarine_cable_d isr uption To my knowledge, no one has reported any cable problems in Norther Africa - -- and news of those problems generally travels very fast. :-) Also, if there *was* a cable problem on one of the paths through the vicinity, it affect more than just Egypt: https://secure.wikimedia.org/wikipedia/en/wiki/File:Cable_map18.svg I don't think it takes a leap of imagination to understand what has happened here. - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFNQnQ0q1pz9mNUZTMRAoFQAKCE8P0wINouFWUvW9GFn7FR6XVmOwCdGV/i VzTaxnJQOPVqyY2bP8ZraDA= =daOC -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Re: Connectivity status for Egypt
On Jan 28, 2011, at 3:29 AM, Carlos Alcantar wrote: Looks like you can still make phone calls into Egypt. So it's not totally lights out... Mobile is apparently being shut down now : http://www.vodafone.com/content/index/press.html Statement - Vodafone Egypt All mobile operators in Egypt have been instructed to suspend services in selected areas. Under Egyptian legislation the authorities have the right to issue such an order and we are obliged to comply with it. The Egyptian authorities will be clarifying the situation in due course . - I think that clarifications are unnecessary in this case. Regards Marshall Carlos Alcantar Race Communications / Race Team Member 101 Haskins Way, So. San Francisco, CA. 94080 Phone: +1 415 376 3314 Fax: +1 650 246 8901 / carlos *at* race.com / www.race.com -Original Message- From: Paul Ferguson [mailto:fergdawgs...@gmail.com] Sent: Thursday, January 27, 2011 11:46 PM To: Joel Jaeggli Cc: nanog@nanog.org Subject: Re: Connectivity status for Egypt -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 27, 2011 at 11:39 PM, Joel Jaeggli joe...@bogus.com wrote: On 1/27/11 10:49 PM, Roy wrote: Moral of the story: Separate facts from assumptions and guesses. I did some Google searches and that region has had large scale disruptions in the past. Several cables follow the same path to the Suez canal and were hit. my links through the region are all fine, but they don't jump off the cable in egypt just pass through. https://secure.wikimedia.org/wikipedia/en/wiki/2008_submarine_cable_d isr uption To my knowledge, no one has reported any cable problems in Norther Africa - -- and news of those problems generally travels very fast. :-) Also, if there *was* a cable problem on one of the paths through the vicinity, it affect more than just Egypt: https://secure.wikimedia.org/wikipedia/en/wiki/File:Cable_map18.svg I don't think it takes a leap of imagination to understand what has happened here. - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFNQnQ0q1pz9mNUZTMRAoFQAKCE8P0wINouFWUvW9GFn7FR6XVmOwCdGV/i VzTaxnJQOPVqyY2bP8ZraDA= =daOC -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Egypt Telecom AS isolation
Hi to all, I try with BGPlay to show something related to BGP Traffic for some prefix of as8452. If you are interested check: http://extraexploit.blogspot.com/2011/01/egypt-telecom-as-isolation-bgplay-show.html -- http://extraexploit.blogspot.com
Re: Connectivity status for Egypt
Al Arabiya is reporting (via twitter) that the Internet has been shut of in Syria (where I have not heard of reports of protests). I have no confirmation of this as yet. Regards Marshall On Jan 27, 2011, at 9:47 PM, Danny O'Brien wrote: On Thu, Jan 27, 2011 at 6:07 PM, Roy r.engehau...@gmail.com wrote: On 1/27/2011 3:47 PM, Danny O'Brien wrote: Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) Thank you, I suggest that you confine your information to the press on what you know rather than speculation on the cause. Never attribute to malice that which can be adequately explained by stupidity, but don't rule out malice https://secure.wikimedia.org/wikipedia/en/wiki/Hanlon%27s_razor That is indeed one of the reasons why I'm seeking corroboration of the pattern of behaviour; at least to isolate and eliminate any alternative explanations. It would certainly be of operational interest (and certainly not unknown in the annals of historical stupidity) if, say, a single fiber-cut or network upgrade was disrupting all of these different forms of communication simultaneously. On the other hand, there's only a finite number of imaginary backhoes you can conjure up before other explanations begin to trump Hanlon's razor. Right now, I think that http://bgpmon.net/blog/?p=450 explains (or at least illustrates) why we were getting reports of widespread but not universal Internet interruption. See also http://www.renesys.com/blog/2011/01/egypt-leaves-the-internet.shtml . I don't have a good explanation for the SMS problems, but lots of independent reports; I've yet to have any real confirmation of no mobile service, and lots of denials, so right now I'm going to assume that's untrue. If anyone can get explanations from their peers in the region, please pass them on (however incomplete or informal -- mail me directly if you'd rather not contribute to rumors or non-operational NANOG discussions). It's late at night in Egypt, and the biggest protests are planned for tomorrow. A great deal of life-critical systems will be under a great deal of stress during that time, and the interruptions in network connectivity would be extremely worrying. Thanks for checking this out, d.
Re: Connectivity status for Egypt
On 28/01/2011 15:15, Marshall Eubanks wrote: Al Arabiya is reporting (via twitter) that the Internet has been shut of in Syria (where I have not heard of reports of protests). I have no confirmation of this as yet. AS29386 (Syrian Telecommunication Establishment) appears to be up at this time, as are all nameservers for the .sy TLD. Nick
Re: Connectivity status for Egypt
On Jan 28, 2011, at 10:23 AM, Patrik Wallström wrote: On Jan 28, 2011, at 4:15 PM, Marshall Eubanks wrote: Al Arabiya is reporting (via twitter) that the Internet has been shut of in Syria (where I have not heard of reports of protests). I have no confirmation of this as yet. I have seen no evidence if this. Can still reach services within the country. Definitely not shut down. -- TTFN, patrick
Re: Connectivity status for Egypt
On Jan 28, 2011, at 10:25 AM, Patrick W. Gilmore wrote: On Jan 28, 2011, at 10:23 AM, Patrik Wallström wrote: On Jan 28, 2011, at 4:15 PM, Marshall Eubanks wrote: Al Arabiya is reporting (via twitter) that the Internet has been shut of in Syria (where I have not heard of reports of protests). I have no confirmation of this as yet. I have seen no evidence if this. Can still reach services within the country. Definitely not shut down. Thanks Marshall -- TTFN, patrick
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 2:44 AM, Jake Khuon kh...@neebu.net wrote: I guess this begs the question of whether or not we're seeing actual layer1 going down or just the effects of mass BGP withdrawals. Are we seeing lights out on fibre links or just peering sessions going down? Both could still point to a coordinated intentional blackout by the Egyptian gov't though. out of curiousity, what's the difference though between loss of light and peer shutdown? If the local gov't comes in and says: Make the internets go down, you as the op choose how to do that... NOT getting calls from your peer for interface alarms is probably sane. You can simply drop your routes, leave BGP running even and roll ... If it's clear (and it seems to be) that the issue is a nation-state-decision... implementation (how it's done, no IF it's done) isn't really important, is it? -chris
Re: Connectivity status for Egypt
On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. The question is not whether that would it (it obviously would). The question is whether it is important if the laser stops blinking or just blinks in ways that end users can't see all the YouTube, web pages, twitter posts, etc. that the gov't doesn't want them to see. I think it does not matter. Censorship is censorship. (So much for routing around it.) -- TTFN, patrick On Jan 28, 2011, at 11:17 AM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Jan 28, 2011 at 2:44 AM, Jake Khuon kh...@neebu.net wrote: I guess this begs the question of whether or not we're seeing actual layer1 going down or just the effects of mass BGP withdrawals. Are we seeing lights out on fibre links or just peering sessions going down? Both could still point to a coordinated intentional blackout by the Egyptian gov't though. out of curiousity, what's the difference though between loss of light and peer shutdown? If the local gov't comes in and says: Make the internets go down, you as the op choose how to do that... NOT getting calls from your peer for interface alarms is probably sane. You can simply drop your routes, leave BGP running even and roll ... If it's clear (and it seems to be) that the issue is a nation-state-decision... implementation (how it's done, no IF it's done) isn't really important, is it? -chris
Re: Connectivity status for Egypt
On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. Marshall Jared Mauch On Jan 28, 2011, at 11:17 AM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Jan 28, 2011 at 2:44 AM, Jake Khuon kh...@neebu.net wrote: I guess this begs the question of whether or not we're seeing actual layer1 going down or just the effects of mass BGP withdrawals. Are we seeing lights out on fibre links or just peering sessions going down? Both could still point to a coordinated intentional blackout by the Egyptian gov't though. out of curiousity, what's the difference though between loss of light and peer shutdown? If the local gov't comes in and says: Make the internets go down, you as the op choose how to do that... NOT getting calls from your peer for interface alarms is probably sane. You can simply drop your routes, leave BGP running even and roll ... If it's clear (and it seems to be) that the issue is a nation-state-decision... implementation (how it's done, no IF it's done) isn't really important, is it? -chris
Re: Connectivity status for Egypt
Does anybody knows what is the situation with local traffic, are people able to communicate within the country, are there any local servers/services that are being blocked/etc. ? -J
Re: Connectivity status for Egypt
On Fri, 28 Jan 2011 11:17:58 EST, Christopher Morrow said: On Fri, Jan 28, 2011 at 2:44 AM, Jake Khuon kh...@neebu.net wrote: I guess this begs the question of whether or not we're seeing actual layer1 going down or just the effects of mass BGP withdrawals. Are we seeing lights out on fibre links or just peering sessions going down? out of curiousity, what's the difference though between loss of light and peer shutdown? When Jake wrote that at 2:44AM, it was still unclear if it was government mandate or accidental. The difference is that if it was government action, bringing up a peer may get you a bullet, while relighting a cable that suffered a shark attack is probably safe unless it was a shark with frickin' lasers mounted on its head - which is plausible, as covert-action sharks have been alleged in that region before: http://www.bbc.co.uk/news/world-middle-east-11937285 pgprReB7O0pbr.pgp Description: PGP signature
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 8:49 AM, Jorge Amodio jmamo...@gmail.com wrote: Does anybody knows what is the situation with local traffic, are people able to communicate within the country, are there any local servers/services that are being blocked/etc. ? -J According to CBC in Canada this morning... http://www.cbc.ca/world/story/2011/01/28/egypt-protests.html Internet, data services cut Internet and cellphone data service was unavailable throughout the country, making it impossible for news of the protests to be broadcast via social networking sites like Facebook and Twitter. The lack of service made it virtually impossible for Egyptians, who use mobile phones almost exclusively, to communicate with one another. Protest organizers had also been using social networking sites like Facebook and Twitter to spread information about the protests. In the United States, Mubarak's closest Western ally, the State Department, said the events unfolding in Egypt are of deep concern. Fundamental rights must be respected, violence avoided and open communications allowed, State Department spokesman P.J. Crowley said on Twitter. According to reports, the government ordered internet service providers to cut service early Friday morning. Egypt's four primary internet providers — Link Egypt, Vodafone/Raya, Telecom Egypt, Etisalat Misr — all stopped moving data in and out of the country at 12:34 a.m., according to a network security firm monitoring the traffic. (The service provider Noor, which is used by the Egyptian stock exchange, remained active.) An estimated one million people were expected to take part in the demonstrations Friday afternoon, which began following prayers at mosques in Cairo and elsewhere. Read more: http://www.cbc.ca/world/story/2011/01/28/egypt-protests.html#ixzz1CLlbJhdl cheers Jeff
Re: Connectivity status for Egypt
I'm a reporter for Network World, and we're working on a series of stories re the Egyptian Internet blackout. I hope I can glean some information from the operators on this list for my story. It would be much appreciated. My question for NANOG operators is... Is the blackout disrupting your operations in Egypt, Northern Africa and/or the Middle East? Have you noticed any resumption of service since the outage went into effect on Thursday, Jan. 27? Also, a bill was introduced recently in Congress proposing an Internet kill switch to be used, apparently, in response to cyberattacks on the U.S.: http://edge.networkworld.com/news/2009/040209-obama-cybersecurity-bill.html?page=1 Do you have any opinions on whether this kill switch could indeed be employed here to thwart attacks... or to suppress communications during time of political unrest? As a network operator, would you support such a bill? And would you comply with it if it indeed became law? Thank you, and best regards, Jim Duffy Managing Editor Network World
Re: Connectivity status for Egypt
Jim, On Jan 28, 2011, at 12:43 PM, jdu...@nww.com wrote: And would you comply with it if it indeed became law? For better or worse, companies will comply with lawful requests. In the event of US Civil Unrest, I think it would be much harder than in other regimes to exert this type of control, and would cause a much broader global impact to economic activity. The same would happen with any pan-european blackout. For the economic reasons alone, I rate the chances of kill-switch a zero. It makes for great reporting about power, but the practicality is zero. (this does not preclude the US Government from disconnecting *its* enterprise networks, as has happened with Bureau of Indian Affairs in the past, etc...) - Jared Mauch
Re: Connectivity status for Egypt
On 2011-01-28, at 11:33, Marshall Eubanks wrote: On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. It was not the case in Nepal in 2005 though, if I remember correctly. In that case connectivity to the outside was maintained, but access to that connectivity by people inside the country was curtailed. Joe
Re: Connectivity status for Egypt
On Fri, 2011-01-28 at 11:27 -0500, Patrick W. Gilmore wrote: I think it does not matter. Censorship is censorship. (So much for routing around it.) Obviously for the effected, the effects are the same. |8^) However, I'm interested in knowing about the level of fine control that the Egyptian government may have exercised. I think the subtle implications on the relationships between operators and governments bear some fine distinction in such a case. Also I think there will eventually be different consequences between an indiscriminate mass disconnect of all telecom and network services and a selective one where some of the infrastructure is left intact but under tighter control... especially if internal reach is still selectively available while external reach has been disabled. -- /*=[ Jake Khuon kh...@neebu.net ]=+ | Packet Plumber, Network Engineers /| / [~ [~ |) | | | | for Effective Bandwidth Utilisation / |/ [_ [_ |) |_| NETWORKS | +==*/
RE: Connectivity status for Egypt
-Original Message- From: Jake Khuon [mailto:kh...@neebu.net] Sent: Friday, January 28, 2011 12:07 PM To: Patrick W. Gilmore Cc: NANOG list Subject: Re: Connectivity status for Egypt On Fri, 2011-01-28 at 11:27 -0500, Patrick W. Gilmore wrote: I think it does not matter. Censorship is censorship. (So much for routing around it.) I think it would be pretty hard to actually cut off communications when the telephone system is still working. You can move a lot of email by dialup UUCP if you wanted to. I am guessing that satellite internet still works and landline dialup to a modem outside the country still works. And there's always static routes :)
Re: Connectivity status for Egypt
On 1/28/2011 8:17 AM, Christopher Morrow wrote: out of curiousity, what's the difference though between loss of light and peer shutdown? If the local gov't comes in and says: Make the internets go down, you as the op choose how to do that... NOT getting calls from your peer for interface alarms is probably sane. You can simply drop your routes, leave BGP running even and roll ... If it's clear (and it seems to be) that the issue is a nation-state-decision... implementation (how it's done, no IF it's done) isn't really important, is it? I guess it depends on what goes down as an effect of the mandate. If it's full Layer 1 severing, then leased line and other circuits will go down too. If it's just shut down your Internet peering sessions, then there's alternative opportunities for connectivity. For instance, our corporate WAN links into Cairo are still up (UUNET PIP). aj
Re: Connectivity status for Egypt
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/28/2011 12:36 PM, George Bonser wrote: -Original Message- From: Jake Khuon [mailto:kh...@neebu.net] Sent: Friday, January 28, 2011 12:07 PM To: Patrick W. Gilmore Cc: NANOG list Subject: Re: Connectivity status for Egypt On Fri, 2011-01-28 at 11:27 -0500, Patrick W. Gilmore wrote: I think it does not matter. Censorship is censorship. (So much for routing around it.) I think it would be pretty hard to actually cut off communications when the telephone system is still working. You can move a lot of email by dialup UUCP if you wanted to. Right. In a government regulated monopoly telcom carrier. I am guessing that satellite internet still works If people can't afford to eat, I doubt they can afford satellite internet. and landline dialup to a modem outside the country still works. This presumes people have long distance plans. And there's always static routes :) To what? If everyone has dropped BGP sessions how are you as an end user going to setup static routes? Unless there are no firewalls and everything is wide open how would you reach gateways? - -- Charles N Wyble (char...@knownelement.com) Systems craftsman for the stars http://www.knownelement.com Mobile: 626 539 4344 Office: 310 929 8793 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJNQyyUAAoJEMvvG/TyLEAt2ZoQALt3Arteje09ssqAkrbsretj BuH1UyzK6VNpyk9q72p9C10XowqNE9BGTni+B1lZxh4VNY/cSdRaQFQO9DsMt+ww dWl4HAu/PswRkWGrdQ2DIncRuXd8D6IOQ+ggv2I3cA6Pxi9Ep3rg5GF63+x1fTff 6SCU+FWjTe4ghkeDkR7d2L/6DESJiZCR1DojBMIPf1/W8TTllqmCXflPW6cLgIlC gBqiCVM24FhMBmNzGGjfcnfoQnCcwFD5qAVPBcMh0Y9Hz5olEN2F0tsgYSbG2szH 3UD4ocZ07xLMAG1LdkjoEJmORdAQOv5GL2nkFFCi+/K6sMTyhRhBkO03DA0tOkRN M/wJIrRMeSS5ur6NBy0PDgHcHYo138w5wUAoZi3B8JrfiP+cxJ+oEMm6LDDLTNV7 NbKgpkUOeAvi+qhXo2BUbXpZv8Oh/OAedwIu7/5xHx8YPm2Bq9OTkZrPECslig/G p2NCWpohbKfUn0EeN/NdutxWX/O6YY3y5mB/wfFasnr0kvi413QOMnRViOSgfNY/ DTtpzTc7aahY0L2uAU21qTZIMDRuB/aYaHfbfsKpL2LGdxq/JFm6sQQ/IeN5Q7ii 0QvMDM04Eqi4cCgut7p3DKTjkxFnU9Wilo/A8jeY4CRVH1I/Afft6aDh7GZNPKgr QaEcUTQLrfCF284d1XSl =QICt -END PGP SIGNATURE-
Re: Connectivity status for Egypt
Here is a blog by Al Jazeera on what is happening in Egypt. Look at the time stamp of 7:46. Kill-Switch is alive and well. Coming to America soon? http://blogs.aljazeera.net/middle-east/2011/01/28/liveblog-egypts-protests-erupt . ** *The only power people exert over us, is the power we allow them to exert.* * * *http://www.projectcensored.org/* * * *http://www.thenewamerican.com/* ** On Thu, Jan 27, 2011 at 3:47 PM, Danny O'Brien da...@spesh.com wrote: Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) Thank you, -- dobr...@cpj.org Danny O'Brien, Committee to Protect Journalists gpg key: http://www.spesh.com/danny/crypto/dannyobrien-key20091106.txt -- * *
Re: Connectivity status for Egypt
On Friday 28 January 2011 20:36:30 George Bonser wrote: -Original Message- From: Jake Khuon [mailto:kh...@neebu.net] Sent: Friday, January 28, 2011 12:07 PM To: Patrick W. Gilmore Cc: NANOG list Subject: Re: Connectivity status for Egypt On Fri, 2011-01-28 at 11:27 -0500, Patrick W. Gilmore wrote: I think it does not matter. Censorship is censorship. (So much for routing around it.) I think it would be pretty hard to actually cut off communications when the telephone system is still working. You can move a lot of email by dialup UUCP if you wanted to. I am guessing that satellite internet still works and landline dialup to a modem outside the country still works. And there's always static routes :) International dial-out is a good point, especially these days when international voice isn't wildly expensive any more. Does anyone have a source for dialup pools like that? Personally, I suspect that it's probably more important to cut off internal comms. Especially as the TV and media people are pretty good at bringing their own satellite connectivity. Which is more worrying, someone updating their wordpress.com blog, or the same person texting everyone they know to show up outside State TV at 1700 hours and bring a bag of bricks? A lot of the fbk/twt/whatever activity, and all the really politically important fraction of it, is just that - but going through either externally located servers or externally-owned ones. I wonder if anyone's working on a mesh or p-t-p radio app that runs on a smartphone? -- The only thing worse than e-mail disclaimers...is people who send e-mail to lists complaining about them signature.asc Description: This is a digitally signed message part.
Re: Connectivity status for Egypt
On 1/28/2011 1:02 PM, Alexander Harrowell wrote: I wonder if anyone's working on a mesh or p-t-p radio app that runs on a smartphone? Yes - came across http://www.servalproject.org/ from the linux.conf.au program.
Re: Connectivity status for Egypt
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/28/2011 01:02 PM, Alexander Harrowell wrote: On Friday 28 January 2011 20:36:30 George Bonser wrote: -Original Message- From: Jake Khuon [mailto:kh...@neebu.net] Sent: Friday, January 28, 2011 12:07 PM To: Patrick W. Gilmore Cc: NANOG list Subject: Re: Connectivity status for Egypt On Fri, 2011-01-28 at 11:27 -0500, Patrick W. Gilmore wrote: I think it does not matter. Censorship is censorship. (So much for routing around it.) I think it would be pretty hard to actually cut off communications when the telephone system is still working. You can move a lot of email by dialup UUCP if you wanted to. I wonder if anyone's working on a mesh or p-t-p radio app that runs on a smartphone? Yes. http://www.servalproject.org/ - -- Charles N Wyble (char...@knownelement.com) Systems craftsman for the stars http://www.knownelement.com Mobile: 626 539 4344 Office: 310 929 8793 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJNQzHxAAoJEMvvG/TyLEAtV60P+wUsrIDsZJrS7L2reSL0GhF7 LOZlTAWG4LeuJQnt6GfpKVJlZgoR1/ucm1NLZDnJeJ6+14sB0CqMipd2XDse5shB zWzvdHn0yGF/RQEnEpZPhQv83crbLVBN2k56KoPkgxbuBwLRdmIZkvSSTXbeAqXy ovQWTGMQWxjT0IUYFylljFo1Djsx/aFeMm+MW5R+9bSL4DASg802ozi2oxVeSpzP lbYsp2ZOtPq3XkWRE3g2JeA/BEJiVJ0BRGUSuoSKUVRsvW4qwlGK1ZUFmNeQ+7xn 0tCucyAhpe/ir3eLm0fMpXk0haXzbpns6oxfudUdMXPLO6PwtYNC/DgBIxlnxDxQ 7G9YD6O63xsRLf3VclDVYgvgwpm6YEIVcjZ4pg/fk1IpKazEB1UlQl10UaFItRvC V7Vo6sK9wzs4GddNwXVJFa919IP3bRsFY7wnWDNES/Nb71vmzvZlk1eEMITn/F1S zxDr1MQdAnsiWYqG7CFjrWjVNCOK/1x7msmIdRpUB31TyMRnzFWxb4JxXEweVbRo VulT+c1fNVyag+YnIkI1nj/8U5zfPygZXMx4FQhMCtA4dqcBeN2W01P0vDJnUuVH zGASBV18/4F1MGduC9zKU1yW4pTxTt0/t/o/o4/SkAMz8BCDvo7vct+3+3Y9+v4p aovANFV1Q3cNt+0GMqQS =UYCf -END PGP SIGNATURE-
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 3:51 PM, Alastair Johnson a...@sneep.net wrote: For instance, our corporate WAN links into Cairo are still up (UUNET PIP). cough that's the MCI PIP/cough...
Re: Connectivity status for Egypt
On Fri, 28 Jan 2011 12:36:30 PST, George Bonser said: I think it would be pretty hard to actually cut off communications when the telephone system is still working. You can move a lot of email by dialup UUCP if you wanted to. Sure, just pop onto amazon.com and order a modem... oh, wait. (It's certainly doable, but decidedly nontrivial, and will require much sneakernet to bootstrap) pgpd21T7CgXel.pgp Description: PGP signature
RE: Connectivity status for Egypt
I have also seen reports that Syria has severed their Internet access, as well: http://af.reuters.com/article/tunisiaNews/idAFLDE70P18Y20110126 http://twitter.com/AlArabiya_Eng/status/31002490816167936 Can anyone confirm that?
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 4:18 PM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Jan 28, 2011 at 3:51 PM, Alastair Johnson a...@sneep.net wrote: For instance, our corporate WAN links into Cairo are still up (UUNET PIP). cough that's the MCI PIP/cough... probably the .EG parts of that PIP are provided on a partner network still ... I don't think they have build of their own gear into the country, and there's a high likelihood that if state-security sees 'forbidden' traffic on those links they'll request traffic shutdown on that network as well. If you operate a network in the affected country I'm sure you'll have to comply with LEA demands... -chris
Re: Connectivity status for Egypt
Thank you Charles on 1/28/11 12:52 PM, Charles N Wyble wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/28/2011 12:36 PM, George Bonser wrote: -Original Message- From: Jake Khuon [mailto:kh...@neebu.net] Sent: Friday, January 28, 2011 12:07 PM To: Patrick W. Gilmore Cc: NANOG list Subject: Re: Connectivity status for Egypt On Fri, 2011-01-28 at 11:27 -0500, Patrick W. Gilmore wrote: I think it does not matter. Censorship is censorship. (So much for routing around it.) I think it would be pretty hard to actually cut off communications when the telephone system is still working. You can move a lot of email by dialup UUCP if you wanted to. Right. In a government regulated monopoly telcom carrier. I am guessing that satellite internet still works If people can't afford to eat, I doubt they can afford satellite internet. and landline dialup to a modem outside the country still works. This presumes people have long distance plans. And there's always static routes :) To what? If everyone has dropped BGP sessions how are you as an end user going to setup static routes? Unless there are no firewalls and everything is wide open how would you reach gateways? - -- Charles N Wyble (char...@knownelement.com) Systems craftsman for the stars http://www.knownelement.com Mobile: 626 539 4344 Office: 310 929 8793 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJNQyyUAAoJEMvvG/TyLEAt2ZoQALt3Arteje09ssqAkrbsretj BuH1UyzK6VNpyk9q72p9C10XowqNE9BGTni+B1lZxh4VNY/cSdRaQFQO9DsMt+ww dWl4HAu/PswRkWGrdQ2DIncRuXd8D6IOQ+ggv2I3cA6Pxi9Ep3rg5GF63+x1fTff 6SCU+FWjTe4ghkeDkR7d2L/6DESJiZCR1DojBMIPf1/W8TTllqmCXflPW6cLgIlC gBqiCVM24FhMBmNzGGjfcnfoQnCcwFD5qAVPBcMh0Y9Hz5olEN2F0tsgYSbG2szH 3UD4ocZ07xLMAG1LdkjoEJmORdAQOv5GL2nkFFCi+/K6sMTyhRhBkO03DA0tOkRN M/wJIrRMeSS5ur6NBy0PDgHcHYo138w5wUAoZi3B8JrfiP+cxJ+oEMm6LDDLTNV7 NbKgpkUOeAvi+qhXo2BUbXpZv8Oh/OAedwIu7/5xHx8YPm2Bq9OTkZrPECslig/G p2NCWpohbKfUn0EeN/NdutxWX/O6YY3y5mB/wfFasnr0kvi413QOMnRViOSgfNY/ DTtpzTc7aahY0L2uAU21qTZIMDRuB/aYaHfbfsKpL2LGdxq/JFm6sQQ/IeN5Q7ii 0QvMDM04Eqi4cCgut7p3DKTjkxFnU9Wilo/A8jeY4CRVH1I/Afft6aDh7GZNPKgr QaEcUTQLrfCF284d1XSl =QICt -END PGP SIGNATURE- ~.~
Re: Connectivity status for Egypt
If I'm correct, in 2000 in Fiji, the main fiber optic cable from the national provider to the international provider was sabotaged, cutting all communications. Fortunately an Alcatel team was on the island (SCC commissioning) with the right tools and could splice it back in a few hours, otherwise Fiji would have gone dark for days... - Original Message - From: Joe Abley jab...@hopcount.ca To: Marshall Eubanks t...@americafree.tv Cc: nanog@nanog.org Sent: Saturday, 29 January, 2011 7:32:07 AM Subject: Re: Connectivity status for Egypt On 2011-01-28, at 11:33, Marshall Eubanks wrote: On Jan 28, 2011, at 11:24 AM, Jared Mauch wrote: I have seen nation state disconnects where light is lost. I believe that was the case for Burma, for example. It was not the case in Nepal in 2005 though, if I remember correctly. In that case connectivity to the outside was maintained, but access to that connectivity by people inside the country was curtailed. Joe
Re: Connectivity status for Egypt
On Friday 28 January 2011 21:22:55 Christopher Morrow wrote: On Fri, Jan 28, 2011 at 4:18 PM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Jan 28, 2011 at 3:51 PM, Alastair Johnson a...@sneep.net wrote: For instance, our corporate WAN links into Cairo are still up (UUNET PIP). cough that's the MCI PIP/cough... probably the .EG parts of that PIP are provided on a partner network still ... I don't think they have build of their own gear into the country, and there's a high likelihood that if state-security sees 'forbidden' traffic on those links they'll request traffic shutdown on that network as well. If you operate a network in the affected country I'm sure you'll have to comply with LEA demands... -chris It's ironic that in 1991, the Soviet coup leaders had the international voice gateway shut down but left the Internet link up (who cares about some weird thing eggheads chat over?), but now, dictators in trouble pull all the BGP announcements but leave the PSTN up. Who cares about some old thing your mother uses? Not impressed by US journalists asking why the WH press secretary can't order Vodafone to turn their GSM net back on, though. 1) it's not them who would have to say no to the nice man from Central State Security with his electric shock baton, 2) VF.eg is half-owned by the Egyptian government... -- The only thing worse than e-mail disclaimers...is people who send e-mail to lists complaining about them signature.asc Description: This is a digitally signed message part.
Re: Connectivity status for Egypt
We should be asking the Egyptians to stagger the return of services so that infrastructure isn't affected, when connectivity is deemed to be allowed to come back online. Andrew Wallace --- British IT Security Consultant
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 3:44 PM, andrew.wallace andrew.wall...@rocketmail.com wrote: We should be asking the Egyptians to stagger the return of services so that infrastructure isn't affected, when connectivity is deemed to be allowed to come back online. Andrew Wallace --- British IT Security Consultant http://lifehacker.com/5746046/how-to-foil-a-nationwide-internet-shutdown ***Stefan Mititelu http://twitter.com/netfortius http://www.linkedin.com/in/netfortius
Re: Connectivity status for Egypt
On 1/28/11, andrew.wallace andrew.wall...@rocketmail.com wrote: We should be asking the Egyptians to stagger the return of services so that infrastructure isn't affected, when connectivity is deemed to be allowed to come back online. Well, yeah, it has to be done carefully, otherwise the first guy to turn on an E1 line that announces routes for the entire country is going to have his router overheat and the blue smoke get out If we're lucky, the Army won't damage too much as they either win or lose. -- Thanks; Bill Note that this isn't my regular email account - It's still experimental so far. And Google probably logs and indexes everything you send it.
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 02:07:51PM -0800, Bill Stewart wrote: On 1/28/11, andrew.wallace andrew.wall...@rocketmail.com wrote: We should be asking the Egyptians to stagger the return of services so that infrastructure isn't affected, when connectivity is deemed to be allowed to come back online. Well, yeah, it has to be done carefully, otherwise the first guy to turn on an E1 line that announces routes for the entire country is going to have his router overheat and the blue smoke get out If we're lucky, the Army won't damage too much as they either win or lose. It depends on what remains functional after the fact. If there is no demand for traffic, then routes will be stable and the session will stay active. If the link fills, the session bounces as packets get dropped. It also depends on whether the person turning up that first E1 actually has much behind them and whether those people have much connectivity that doesn't require shrapnel removal. --- Wayne Bouchard w...@typo.org Network Dude http://www.typo.org/~web/
Re: Connectivity status for Egypt
iMCI or WCOM? :) On Fri, Jan 28, 2011 at 5:18 PM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Jan 28, 2011 at 3:51 PM, Alastair Johnson a...@sneep.net wrote: For instance, our corporate WAN links into Cairo are still up (UUNET PIP). cough that's the MCI PIP/cough...
Re: Connectivity status for Egypt
On Fri, Jan 28, 2011 at 5:32 PM, jim deleskie deles...@gmail.com wrote: iMCI or WCOM? :) w (technically the folks that engineered it were mci folk... from texas. On Fri, Jan 28, 2011 at 5:18 PM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Jan 28, 2011 at 3:51 PM, Alastair Johnson a...@sneep.net wrote: For instance, our corporate WAN links into Cairo are still up (UUNET PIP). cough that's the MCI PIP/cough...
Re: Connectivity status for Egypt
On Jan 28, 2011, at 1:44 PM, andrew.wallace wrote: We should be asking the Egyptians to stagger the return of services so that infrastructure isn't affected, when connectivity is deemed to be allowed to come back online. Andrew Wallace --- British IT Security Consultant You should send them an email about that.
Connectivity status for Egypt
Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) Thank you, -- dobr...@cpj.org Danny O'Brien, Committee to Protect Journalists gpg key: http://www.spesh.com/danny/crypto/dannyobrien-key20091106.txt
Re: Connectivity status for Egypt
I have a server with CityNet Host in Cairo. The server and ISP are completely offline
Re: Connectivity status for Egypt
Some interesting financial news... Unsure if this is related the outages, but interesting. http://www.marketwatch.com/story/egypt-market-slumps-as-mideast-turmoil-spreads-2011-01-27 EGYPT: Stock market stumbles amid nationwide turbulencehttp://latimesblogs.latimes.com/babylonbeyond/2011/01/egypt-stock-market-stumbles-amidst-nationwide-turbulence.html http://www.marketwatch.com/story/egypt-market-slumps-as-mideast-turmoil-spreads-2011-01-27 http://latimesblogs.latimes.com/babylonbeyond/2011/01/egypt-stock-market-stumbles-amidst-nationwide-turbulence.html http://latimesblogs.latimes.com/babylonbeyond/2011/01/egypt-stock-market-stumbles-amidst-nationwide-turbulence.html On Thu, Jan 27, 2011 at 7:10 PM, Christopher cal...@gmail.com wrote: I have a server with CityNet Host in Cairo. The server and ISP are completely offline
Re: Connectivity status for Egypt
I'd suspect it's got a lot more to do with the open rioting on the streets, government shooting people, the numbers involved in protests, what happened in Tunisia next door etc. etc. Loss of Internet connectivity is relatively minor in comparison. Any investor with even half a brain is going to twig that's just not a good market to have money in right now. On 01/27/2011 02:53 PM, Craig V wrote: Some interesting financial news... Unsure if this is related the outages, but interesting. http://www.marketwatch.com/story/egypt-market-slumps-as-mideast-turmoil-spreads-2011-01-27 EGYPT: Stock market stumbles amid nationwide turbulencehttp://latimesblogs.latimes.com/babylonbeyond/2011/01/egypt-stock-market-stumbles-amidst-nationwide-turbulence.html http://www.marketwatch.com/story/egypt-market-slumps-as-mideast-turmoil-spreads-2011-01-27 http://latimesblogs.latimes.com/babylonbeyond/2011/01/egypt-stock-market-stumbles-amidst-nationwide-turbulence.html http://latimesblogs.latimes.com/babylonbeyond/2011/01/egypt-stock-market-stumbles-amidst-nationwide-turbulence.html On Thu, Jan 27, 2011 at 7:10 PM, Christophercal...@gmail.com wrote: I have a server with CityNet Host in Cairo. The server and ISP are completely offline
Re: Connectivity status for Egypt
On Jan 27, 2011, at 6:47 PM, Danny O'Brien wrote: Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) On twitter (follow the #jan25 and #jan28 hash tags), there are many reports of loss of internet connectivity in Egypt. Apparently cell phones and land lines are still working. Of course, the assumption there is that this is connected to the large protests expected tomorrow in Egypt. Regards Marshall Thank you, -- dobr...@cpj.org Danny O'Brien, Committee to Protect Journalists gpg key: http://www.spesh.com/danny/crypto/dannyobrien-key20091106.txt
Re: Connectivity status for Egypt
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 27, 2011 at 5:10 PM, Marshall Eubanks t...@americafree.tv wrote: On twitter (follow the #jan25 and #jan28 hash tags), there are many reports of loss of internet connectivity in Egypt. Apparently cell phones and land lines are still working. Of course, the assumption there is that this is connected to the large protests expected tomorrow in Egypt. The U.S Embassy in Cairo website is also unreachable, as well as the main Egyptian Governmental portal: %ping www.egypt.gov.eg Pinging www.egypt.gov.eg [81.21.104.81] with 32 bytes of data: Request timed out. Request timed out. Request timed out. Request timed out. Ping statistics for 81.21.104.81: Packets: Sent = 4, Received = 0, Lost = 4 (100% loss), %tracert cairo.usembassy.gov Tracing route to cairo.usembassy.gov [62.140.73.207] over a maximum of 30 hops: 1 4 ms 2 ms 1 ms 62.140.73.207 [snip] 737 ms27 ms28 ms ix-1-1-0-0.tcore1.LVW-LosAngeles.as6453.net [216.6.12.25] 8 247 ms 204 ms 205 ms if-2-2.tcore2.LVW-LosAngeles.as6453.net [66.110.59.2] 9 226 ms 199 ms 204 ms if-8-1508.tcore2.AEQ-Ashburn.as6453.net [64.86.252.74] 10 289 ms 301 ms 219 ms if-2-2.tcore1.AEQ-Ashburn.as6453.net [216.6.87.2] 11 190 ms 252 ms 204 ms if-6-871.tcore1.PVU-Paris.as6453.net [216.6.51.58] 12 197 ms 204 ms 203 ms if-11-1-0-1411.core1.PV1-Paris.as6453.net [80.231.153.13] 13 229 ms 229 ms 236 ms ix-9-0-0.core1.PV1-Paris.as6453.net [195.219.215.38] 14 *** Request timed out. 15 *** Request timed out. 16 *** Request timed out. 17 *** Request timed out. 18 * ^C %ping cairo.usembassy.gov Pinging cairo.usembassy.gov [62.140.73.207] with 32 bytes of data: Request timed out. Request timed out. Request timed out. Request timed out. Ping statistics for 62.140.73.207: Packets: Sent = 4, Received = 0, Lost = 4 (100% loss), % Information related to '62.140.73.0 - 62.140.73.255' inetnum: 62.140.73.0 - 62.140.73.255 netname: EG-NMC descr: AW-NMC descr: For any abuse complain contact ab...@nile-online.com country: EG admin-c: IM217-AFRINIC tech-c: IA119-AFRINIC tech-c: IM217-AFRINIC tech-c: OM2093-AFRINIC status: ASSIGNED PA mnt-by: O-MAHMOUD remarks: data has been transferred from RIPE Whois Database 20050221 source: AFRINIC # Filtered parent: 62.140.64.0 - 62.140.127.255 role: IP Address Manager address: top of address: pyramid address: sand address: sahara phone: +202 37611153 phone: +202 37611123 fax-no: +202 37607656 e-mail: ipad...@nile-online.com e-mail: ashwa...@nile-online.com e-mail: mha...@nile-online.com admin-c: MS22-Afrinic admin-c: MMK1-AFRINIC tech-c: AS38-Afrinic nic-hdl: IM217-AFRINIC source: AFRINIC # Filtered role: IP Address Admin address: 15 Mohamed Hafez St., address: Mohandessin address: Giza address: Egypt phone: +202 37611153 phone: +202 37611123 fax-no: +202 37607656 e-mail: ipad...@nile-online.com e-mail: ashwa...@nile-online.com e-mail: mha...@nile-online.com admin-c: MS22-Afrinic tech-c: AS38-Afrinic nic-hdl: IA119-AFRINIC remarks: data has been transferred from RIPE Whois Database 20050221 source: AFRINIC # Filtered person: Omar Mahmoud nic-hdl: OM2093-AFRINIC address: 15 Mohamed Hafez St., address: Mohandessin address: Giza address: Egypt address: Cairo address: Egypt e-mail: mispeng-c...@etisalatdata.net phone: +202 7606677 fax-no: +202 7607656 remarks: For any abuse complain contact ab...@nile-online.com remarks: data has been transferred from RIPE Whois Database 20050221 mnt-by: O-MAHMOUD source: AFRINIC # Filtered - - ferg -BEGIN PGP SIGNATURE- Version: PGP Desktop 9.5.3 (Build 5003) wj8DBQFNQhpoq1pz9mNUZTMRAkPsAKDQ4y08/I45IS/0XfNZ7kMbciQ61wCfQDJ9 Gm/f4cSJ9lY5BI4fBjdr+vU= =SRH3 -END PGP SIGNATURE- -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawgster(at)gmail.com ferg's tech blog: http://fergdawg.blogspot.com/
Re: Connectivity status for Egypt
Hi, Looking at the BGP announcements it seems that the problem started at around 22:28 UTC. Most of the Autonomous systems operating in Egypt are currently not announcing any or at least significantly less prefixes. The one exception seems to be AS20928 (Noor Data Networks). For more details also see: http://bgpmon.net/blog/?p=450 Cheers, Andree .-- My secret spy satellite informs me that at 11-01-27 3:47 PM Danny O'Brien wrote: Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) Thank you,
Re: Connectivity status for Egypt
On 01/28/2011 12:47 AM, Danny O'Brien wrote: If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) BGPmon has a quick analysis on the reachability of prefixes usually announced by the top 10 operators from Egypt: http://bgpmon.net/blog/?p=450 -Lorand Jakab
Re: Connectivity status for Egypt
On 1/27/2011 3:47 PM, Danny O'Brien wrote: Around 2236 UCT, we lost all Internet connectivity with our contacts in Egypt, and I'm hearing reports of (in declining order of confirmability): 1) Internet connectivity loss on major (broadband) ISPs 2) No SMS 4) Intermittent connectivity with smaller (dialup?) ISPs 5) No mobile service in major cities -- Cairo, Alexandria The working assumption here is that the Egyptian government has made the decision to shut down all external, and perhaps internal electronic communication as a reaction to the ongoing protests in that country. If anyone can provide more details as to what they're seeing, the extent, plus times and dates, it would be very useful. In moments like this there are often many unconfirmed rumors: I'm seeking concrete reliable confirmation which I can pass onto the press and those working to bring some communications back up (if you have a ham radio license, there is some very early work to provide emergency connectivity. Info at: http://pastebin.com/fHHBqZ7Q ) Thank you, I suggest that you confine your information to the press on what you know rather than speculation on the cause. Never attribute to malice that which can be adequately explained by stupidity, but don't rule out malice https://secure.wikimedia.org/wikipedia/en/wiki/Hanlon%27s_razor
