Route leak in Bangladesh
We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. Anybody see their prefixes hijacked as well? -- Grzegorz Janoszka
Re: Route leak in Bangladesh
be nice if some technical details were included
Re: Route leak in Bangladesh
At 10:27 30/06/2015 +0200, Grzegorz Janoszka wrote: We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. Anybody see their prefixes hijacked as well? Welcome to the party :-) Not only you. -Hank -- Grzegorz Janoszka
Re: leap second outage
I read that and that at midnight local time since that's when you have the extra second. I know a large carrier in Israel is down. Waiting for conf. If it's leep second related. --Original Message-- From: Stefan Sender: NANOG To: frnk...@iname.com Cc: nanog@nanog.org Subject: Re: leap second outage Sent: Jun 30, 2015 23:30 This was supposed to have happened @midnight UTC, right? Meaning that we are past that event. Under which scenarios should people be concerned about midnight local time? Lots of confusing messages flying all over... On Jun 30, 2015 10:13 PM, frnk...@iname.com wrote: We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank Regards, Dovid
Re: leap second outage
Correct, the leap second gets inserted at midnight UTC. Leap seconds can be introduced in UTC at the end of the months of December or June, depending on the evolution of UT1-TAI. Bulletin C is mailed every six months, either to announce a time step in UTC or to confirm that there will be no time step at the next possible date. ftp://hpiers.obspm.fr/iers/bul/bulc/bulletinc.dat On Tue, Jun 30, 2015 at 11:30 PM, Stefan netfort...@gmail.com wrote: This was supposed to have happened @midnight UTC, right? Meaning that we are past that event. Under which scenarios should people be concerned about midnight local time? Lots of confusing messages flying all over... On Jun 30, 2015 10:13 PM, frnk...@iname.com wrote: We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank
Sacramento Outage.
Is it odd that there is no mention of this even here? http://www.wavebroadband.com/resources/outage/service.txt -- sed quis custodiet ipsos custodes? (Juvenal)
leap second outage
We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank
Re: leap second outage
This was supposed to have happened @midnight UTC, right? Meaning that we are past that event. Under which scenarios should people be concerned about midnight local time? Lots of confusing messages flying all over... On Jun 30, 2015 10:13 PM, frnk...@iname.com wrote: We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank
Re: NTT-HE earlier today (~10am EDT)
On Wed, Jul 01, 2015 at 09:36:34AM +0900, Randy Bush wrote: - when not using the RTR protocol but generating prefix-list filters based on RPKI data, the devices might not support sufficient entries. because the rpki generated acls are bigger and heavier than those in the irr. and they have trans-fats. I don't consider RPKI generated ACLs a 1 to 1 replacement for IRR based filters. They might be used as supplement to each other. Kind regards, Job
Re: NTT-HE earlier today (~10am EDT)
- when not using the RTR protocol but generating prefix-list filters based on RPKI data, the devices might not support sufficient entries. because the rpki generated acls are bigger and heavier than those in the irr. and they have trans-fats. I don't consider RPKI generated ACLs a 1 to 1 replacement for IRR based filters. They might be used as supplement to each other. the major user puts the rpki-generated pseudo-irr in front of the others in peval(0) order. same number of resulting acls. hence i do not understand your the devices might not support sufficient entries. randy
Re: leap second outage
No. Some one leaked some routes: https://mobile.twitter.com/Axcelx/status/616058414746202113 Regards, Dovid -Original Message- From: Justin Paine jus...@cloudflare.com Date: Tue, 30 Jun 2015 20:37:06 To: do...@telecurve.com Cc: Stefannetfort...@gmail.com; NANOGnanog-boun...@nanog.org; frnk...@iname.com; nanog@nanog.org Subject: Re: leap second outage Any confirmation if the AWS outage was leap second-related? Justin Paine Head of Trust Safety CloudFlare Inc. PGP KeyID: 57B6 0114 DE0B 314D On Tue, Jun 30, 2015 at 8:32 PM, Dovid Bender do...@telecurve.com wrote: I read that and that at midnight local time since that's when you have the extra second. I know a large carrier in Israel is down. Waiting for conf. If it's leep second related. --Original Message-- From: Stefan Sender: NANOG To: frnk...@iname.com Cc: nanog@nanog.org Subject: Re: leap second outage Sent: Jun 30, 2015 23:30 This was supposed to have happened @midnight UTC, right? Meaning that we are past that event. Under which scenarios should people be concerned about midnight local time? Lots of confusing messages flying all over... On Jun 30, 2015 10:13 PM, frnk...@iname.com wrote: We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank Regards, Dovid
Re: NTT-HE earlier today (~10am EDT)
- equipment might not support the RTR protocol to validate announcements against the cache validator alcalu, cisco, juniper do - Legal obstacles in obtaining the anchors from all RIRs arin has been pwned by the tea party and is best ignored. that they do not protect their members is their choice. they're a bottom up policy organization, right. bottom of the barrel. - when not using the RTR protocol but generating prefix-list filters based on RPKI data, the devices might not support sufficient entries. because the rpki generated acls are bigger and heavier than those in the irr. and they have trans-fats. I'll count not using brocade as a valid method. you use brocade at your border? how's that working out for you? :) randy
Re: Sacramento Outage.
There has been mention to this in outages.org list Mehmet On Jun 30, 2015, at 17:37, Larry Sheldon larryshel...@cox.net wrote: Is it odd that there is no mention of this even here? http://www.wavebroadband.com/resources/outage/service.txt -- sed quis custodiet ipsos custodes? (Juvenal)
Re: leap second outage
That is my understanding as well. The event was about 3.5 hours ago. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Tue, Jun 30, 2015 at 11:30 PM, Stefan netfort...@gmail.com wrote: This was supposed to have happened @midnight UTC, right? Meaning that we are past that event. Under which scenarios should people be concerned about midnight local time? Lots of confusing messages flying all over... On Jun 30, 2015 10:13 PM, frnk...@iname.com wrote: We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank
Re: leap second outage
On 15-07-01 00:47, Harlan Stenn wrote: What I'm about to say may not be as stupid as it sounds: The problems here aren't problems for cases where it's not a problem. It is a problem where it *is* a problem. In fairness, systems should be used to NTP making adjustments to the system clock of a second or less. However, in systems that expect tightly synchronized clocks, they would want all the nodes to make the NTP adjustement at the same time.
Re: NTT-HE earlier today (~10am EDT)
On Tue, Jun 30, 2015 at 03:24:02AM +, Faisal Imtiaz wrote: Hi Jared, This is neat !, for someone who recently started working the IRR's, I can tell you that it has been very difficult finding all info in one location. What you shared is pretty neat !, and I would like to clean up the records associated with our prefixes. Can you suggest some practical tips on getting older 'stale' records cleaned up from the different registries ? (i.e. records created for us by others, in a former time-frame). I've not found a great method as it usually involves a lot of either happy or grumpy sounding emails to addresses that may bounce quite a lot. We have had good luck getting RADB to clean out older entries. I recently helped someone announce some IP blocks from the NTT network and there are many crusty IRR objects that seem impossible to clean up because the IRR is feels like first-in-never-out input. http://irrexplorer.nlnog.net/prefix/203.10.78.0/24 - Jared -- Jared Mauch | pgp key available via finger from ja...@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Re: Route leak in Bangladesh
A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. - Matsuzaki Yoshinobu m...@iij.ad.jp - IIJ/AS2497 INOC-DBA: 2497*629 In message 559252e9.6030...@janoszka.pl Date: Tue, 30 Jun 2015 10:27:21 +0200 Grzegorz Janoszka grzeg...@janoszka.pl wrote We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. Anybody see their prefixes hijacked as well? -- Grzegorz Janoszka
Re: Route leak in Bangladesh
Randy Bush ra...@psg.com wrote A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. 7007 all over again. do not redistribute bgp into igp. do not redistribute igp into bgp. I also suggested them to implement BGP community based route filtering in their outbound policy. Any other suggestions or thoughts to prevent such incidents in general? - Matsuzaki Yoshinobu m...@iij.ad.jp - IIJ/AS2497 INOC-DBA: 2497*629
Re: Route leak in Bangladesh
On 30/Jun/15 15:22, Matsuzaki Yoshinobu wrote: I also suggested them to implement BGP community based route filtering in their outbound policy. Any other suggestions or thoughts to prevent such incidents in general? - Get your downstreams to create route objects before you turn them up. - Get your provisioning teams to validate the prefixes being provided by your downstreams. - Use both prefix- and AS_PATH-based filters for your downstreams. - Use BGP communities (as you've stated). - No exceptions. Mark.
Re: Route leak in Bangladesh
On Tue, Jun 30, 2015 at 10:22:38PM +0900, Matsuzaki Yoshinobu wrote: Randy Bush ra...@psg.com wrote A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. 7007 all over again. do not redistribute bgp into igp. do not redistribute igp into bgp. I also suggested them to implement BGP community based route filtering in their outbound policy. Any other suggestions or thoughts to prevent such incidents in general? In addition to the BGP community scheme, outbound as-path filters could help. Most network's list of transit providers is fairly static, it would be easiy with as-path filters to prevent announcing upstream routes to other upstreams or peering partners. Kind regards, Job
Re: Route leak in Bangladesh
At 18:03 30/06/2015 +0900, Randy Bush wrote: be nice if some technical details were included Your prefix: xx.104.150.0/24: Prefix Description: Update time: 2015-06-30 07:39 (UTC) Detected by #peers: 8 Detected prefix: xx.104.150.0/24 Announced by: AS58587 (FIBERATHOME-BD Fiber @ Home Limited,BD) Upstream AS: AS6939 (HURRICANE - Hurricane Electric, Inc.,US) ASpath: 25152 6939 58587 and another: On Tuesday June 30th 2015 at 7:37 UTC we detected a Origin AS Change event for your prefix (23.44.244.0/22 Akamai) The detected prefix: 23.44.244.0/22, was announced by AS58587 (FIBERATHOME-BD Fiber @ Home Limited,BD) Alert description: Origin AS Change Detected Prefix: 23.44.244.0/22 Detected Origin AS: 58587 Expected Origin AS: 1680 Same Aspath of 6939 58587 -Hank
Re: Route leak in Bangladesh
On 30 Jun 2015, at 9:41, Job Snijders wrote: In addition to the BGP community scheme, outbound as-path filters could help. I agree, but possibly not in the case of a redistribution loop. (We don't know that's what happened, exactly, but I thought it was worth mentioning.) Joe
Re: leap second outage
Joe writes: A leap sec causing issues. For about 40 years now, there have been these leap seconds to no real issue. All of these are go-forwards No, they're all go-backwards events. That's no big deal to things that don't care about monotonic time, or to folks who aren't in violation of something if their timestamps are off by a second. What I'm about to say may not be as stupid as it sounds: The problems here aren't problems for cases where it's not a problem. It is a problem where it *is* a problem. It's a case where one person's signal is another person's noise. H
Re: leap second outage
A leap sec causing issues. For about 40 years now, there have been these leap seconds to no real issue. All of these are go-forwards and even MS AD (I believe) treat them as a little bump (nothing to see here move along). So unless you have really a tight VPN (non-standard conforming) I'd hope that nothing has happend, and if it did chances are it's etheir coincidence or intentional. I certainly hope I am around to collect on the https://en.wikipedia.org/wiki/Year_2038_problem for retirement. I think we've all seen the big to do regarding Y2K to know better Maybe I am wrong, but... Just my 2¢s -Joe On Tue, Jun 30, 2015 at 10:42 PM, Nicholas Suan ns...@nonexiste.net wrote: Correct, the leap second gets inserted at midnight UTC. Leap seconds can be introduced in UTC at the end of the months of December or June, depending on the evolution of UT1-TAI. Bulletin C is mailed every six months, either to announce a time step in UTC or to confirm that there will be no time step at the next possible date. ftp://hpiers.obspm.fr/iers/bul/bulc/bulletinc.dat On Tue, Jun 30, 2015 at 11:30 PM, Stefan netfort...@gmail.com wrote: This was supposed to have happened @midnight UTC, right? Meaning that we are past that event. Under which scenarios should people be concerned about midnight local time? Lots of confusing messages flying all over... On Jun 30, 2015 10:13 PM, frnk...@iname.com wrote: We experienced our first leap second outage -- our SHE (super head end) is using (old) Motorola encoders and we lost those video channels. They restarted all those encoders to restore service. Frank -- -Joe 920-530-3631
Re: leap second outage
On Wed, 1 Jul 2015, Jean-Francois Mezei wrote: However, in systems that expect tightly synchronized clocks, they would want all the nodes to make the NTP adjustement at the same time. This is both an operating system and application problem. http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time This is similar to the jiffycounter wrapping, since this doesn't happen that often, it's not commonly tested for. Good way is to start the jiffy counter so it wraps after 10 minutes of uptime. That way you'll run into any bugs quickly. Either we should abolish the leap second or we should make leap second adjustments (back and forth) on a monthly basis to exercise the code. This is a hard sell though... -- Mikael Abrahamssonemail: swm...@swm.pp.se
Re: Route leak in Bangladesh
On Tue, 30 Jun 2015, Matsuzaki Yoshinobu wrote: Randy Bush ra...@psg.com wrote A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. 7007 all over again. do not redistribute bgp into igp. do not redistribute igp into bgp. I also suggested them to implement BGP community based route filtering in their outbound policy. Any other suggestions or thoughts to prevent such incidents in general? At a minimum, AS-PATH filtering of outgoing routes to just your ASN(s) and your downstream customer ASNs. Whether this is done manually, built using AS-SETs from your route registry of choice, or through some other automated means is another story. jms
Re: Route leak in Bangladesh
On Jun 30, 2015, at 10:39 AM, Justin M. Streiner strei...@cluebyfour.org wrote: On Tue, 30 Jun 2015, Matsuzaki Yoshinobu wrote: Randy Bush ra...@psg.com wrote A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. 7007 all over again. do not redistribute bgp into igp. do not redistribute igp into bgp. I also suggested them to implement BGP community based route filtering in their outbound policy. Any other suggestions or thoughts to prevent such incidents in general? At a minimum, AS-PATH filtering of outgoing routes to just your ASN(s) and your downstream customer ASNs. Whether this is done manually, built using AS-SETs from your route registry of choice, or through some other automated means is another story. That sort of AS_PATH filtering would not have helped in this case. The AS originated the routes, it did not propagate an upstream route. So an AS_PATH filter to just its own AS would have passed these routes. You would need origin validation on your outbound routes. Job suggested prefix filters on outbound routes. (If you are doing prefix filters on your inbound customer links, it might be excessive caution to also prefix filter customers prefixes on outbound links? Or is it: you can never be too careful, belt-and-suspenders, measure twice, etc?) --Sandy signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Route leak in Bangladesh
On Tue, Jun 30, 2015 at 09:44:12AM -0400, Joe Abley wrote: On 30 Jun 2015, at 9:41, Job Snijders wrote: In addition to the BGP community scheme, outbound as-path filters could help. I agree, but possibly not in the case of a redistribution loop. (We don't know that's what happened, exactly, but I thought it was worth mentioning.) Joe, you are right. In this specific situation, for a small to medium sized network, it might be prudent to apply an outbound prefix-filter on all transit peering sessions and thus only allowing prefixes which actually belong to downstream customers and the network itself. One could generate that prefix-list based on the network's AS-SET. I wouldn't deploy /only/ outbound prefix-filters. This method should be viewed as complementary to other methods such as the already mentioned a BGP community scheme. Kind regards, Job
Re: Route leak in Bangladesh
On 30/06/2015 14:29, Mark Tinka wrote: - Get your downstreams to create route objects before you turn them up. - Get your provisioning teams to validate the prefixes being provided by your downstreams. - Use both prefix- and AS_PATH-based filters for your downstreams. - Use BGP communities (as you've stated). - No exceptions. plus: - fully automate ingress prefix management - use maxprefixes with manual reenable on all ebgp sessions I've been caught with fully automated IRR based per-session prefix filtering where the customer put the IXP AS macro into their AS macro. When the customer did a 7007 on this, we accepted everything that they announced back to us, oy vey. So you need both. Nick
Re: Route leak in Bangladesh
On 30/06/2015 17:09, Job Snijders wrote: If you were the network causing a leak of this type, prefix filters on inbound facing your customers might not have prevented this. If you are a network providing transit to the leak originator mentioned in the above paragraph, I believe a prefix based filter could have made a big difference. We seem to be assuming that this leak occurred within the context of a customer-provider BGP relationship. But what if this is not the case? What if this was a peering session - perhaps via a route server at an exchange point. max-pref on a session with a route server is an extremely blunt (and potentially ineffective) tool for the job. In some regions the use to route servers and the lack of clue about anything BGP beyond one session to the route server (and one session to transit) is scary. We place our faith in the IXP operator, that they know best, while there may be no evidence that they do... ;-) -- Graham Beneke
Re: ARIN just subdivided their last /17, /18, /19, /20, /21 and /22. Down to only /23s and /24s now. : ipv6
On Tue, 30 Jun 2015, Ricky Beam wrote: The death of Novell NetWare (and their transitioned to IP) killed it the enterprise. Games adopting IP for network play killed it in the home. Ultimately, it sucks as a WAN protocol, so the internet was built using this new fangled IP thing. There are still isolated pockets of devices out there speaking IPX, DECnet, Appletalk, etc, but either they're not connected to the 'Internet', or their traffic passes through other devices that encapsulate and de-encapsulate it in IP to allow it to be transported. jms
Re: Route leak in Bangladesh
On 30/Jun/15 16:24, Job Snijders wrote: In this specific situation, for a small to medium sized network, it might be prudent to apply an outbound prefix-filter on all transit peering sessions and thus only allowing prefixes which actually belong to downstream customers and the network itself. I say that regardless of size, deploy the ultimate solution as the network is only bound to grow. It's harder for folk to undo old habits as they become more entrenched. Mark.
Re: Route leak in Bangladesh
Some more data from BGPmon.net: This affected close to 28,000 prefixes from 4,477 unique Autonomous systems. The hijacks were originated by AS58587 and propagated via AS45796 (15,002 prefixes) and AS6939 (25,841). The AS45796 paths were only seen via one of our peers, while the AS6939 path had a more significant visibility. The event started at 07:37 UTC and lasted for a few minutes. Cheers Andree .-- My secret spy satellite informs me that at 2015-06-30 3:26 AM Matsuzaki Yoshinobu wrote: A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. - Matsuzaki Yoshinobu m...@iij.ad.jp - IIJ/AS2497 INOC-DBA: 2497*629 In message 559252e9.6030...@janoszka.pl Date: Tue, 30 Jun 2015 10:27:21 +0200 Grzegorz Janoszka grzeg...@janoszka.pl wrote We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. Anybody see their prefixes hijacked as well? -- Grzegorz Janoszka
Re: Route leak in Bangladesh
On Tue, Jun 30, 2015 at 04:38:48PM +0200, Mark Tinka wrote: On 30/Jun/15 16:24, Job Snijders wrote: In this specific situation, for a small to medium sized network, it might be prudent to apply an outbound prefix-filter on all transit peering sessions and thus only allowing prefixes which actually belong to downstream customers and the network itself. I say that regardless of size, deploy the ultimate solution as the network is only bound to grow. It's harder for folk to undo old habits as they become more entrenched. Nothing is ever regardless of anything :-) I would forsee issues if i'd try to add an eleven megabyte prefix-list on all devices in the network. Kind regards, Job
Re: Route leak in Bangladesh
On Tue, Jun 30, 2015 at 10:53:45AM -0400, Sandra Murphy wrote: That sort of AS_PATH filtering would not have helped in this case. The AS originated the routes, it did not propagate an upstream route. So an AS_PATH filter to just its own AS would have passed these routes. You would need origin validation on your outbound routes. Job suggested prefix filters on outbound routes. (If you are doing prefix filters on your inbound customer links, it might be excessive caution to also prefix filter customers prefixes on outbound links? Or is it: you can never be too careful, belt-and-suspenders, measure twice, etc?) I wouldn't consider it to be excessive caution to bring more safeguards to the game, you never know when diarrhea will strike. If you were the network causing a leak of this type, prefix filters on inbound facing your customers might not have prevented this. If you are a network providing transit to the leak originator mentioned in the above paragraph, I believe a prefix based filter could have made a big difference. Kind regards, Job
Re: Route leak in Bangladesh
On Jun 30, 2015, at 9:41 AM, Job Snijders j...@instituut.net wrote: In addition to the BGP community scheme, outbound as-path filters could help. Most network's list of transit providers is fairly static, it would be easiy with as-path filters to prevent announcing upstream routes to other upstreams or peering partners. Except that this was not a route leak per se. The announced routes AS_PATH shows them as originated by the offending AS, not *propagated* by the offending AS. So the outbound AS_PATH did not retain the upstream portion of the path. --Sandy signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Route leak in Bangladesh
On Tue, 30 Jun 2015, Sandra Murphy wrote: On Jun 30, 2015, at 10:39 AM, Justin M. Streiner strei...@cluebyfour.org wrote: At a minimum, AS-PATH filtering of outgoing routes to just your ASN(s) and your downstream customer ASNs. Whether this is done manually, built using AS-SETs from your route registry of choice, or through some other automated means is another story. That sort of AS_PATH filtering would not have helped in this case. The AS originated the routes, it did not propagate an upstream route. I didn't realise they offending AS was originating those routes, rather than propagating the existing ones. So an AS_PATH filter to just its own AS would have passed these routes. That's why I suggested it as a minimum precaution. When I worked in the service provider world, we did prefix + AS-PATH filtering + max-prefix, which was pretty effective in keeping BGP-borne madness down to a dull roar. Would that stop everything? No, but it did help a lot. I still work in a BGP-speaking organization - just not one that has downstream BGP-speaking customers at this point. You would need origin validation on your outbound routes. Job suggested prefix filters on outbound routes. (If you are doing prefix filters on your inbound customer links, it might be excessive caution to also prefix filter customers prefixes on outbound links? Or is it: you can never be too careful, belt-and-suspenders, measure twice, etc?) It depends on how much automation can be done to update the necessary filters and AS-PATH ACLs, and how much you trust both the automation method and the data source for those filters. jms
Re: GRE performance over the Internet - DDoS cloud mitigation
On 1 Jul 2015, at 1:37, Dennis B wrote: Would you like to learn more? lol I'm quite conversant with all these considerations, thanks. OP asserted that BGP sessions for diversion into any cloud DDoS mitigation service ran from the endpoint network through GRE tunnels to the cloud-based mitigation provider. I was explaining that in most cloud mitigation scenarios, GRE tunnels are used for re-injection of 'clean' traffic to the endpoint networks. --- Roland Dobbins rdobb...@arbor.net
Re: ARIN just subdivided their last /17, /18, /19, /20, /21 and /22. Down to only /23s and /24s now. : ipv6
On 06/30/2015 07:28 AM, Justin M. Streiner wrote: There are still isolated pockets of devices out there speaking IPX, DECnet, Appletalk, etc, but either they're not connected to the 'Internet', or their traffic passes through other devices that encapsulate and de-encapsulate it in IP to allow it to be transported. For the young (and the young at heart) those other devices were known as bridges back in the day.
Re: GRE performance over the Internet - DDoS cloud mitigation
Depends on what performance considerations you are trying to address, technically. The question is how can we guarantee the GRE/BGP performance (control traffic) during the time between detection and mitigation? GRE decapsulation? IE: Hardware vs Software? Routing of the Protocol over the internet? IE: If the inbound path is saturated, what is the availability of the GRE tunnel? User-experience with GRE packet overhead? IE: TCP Fragmentation causing PMTUD messages for reassembly? I've worked at Prolexic for 7 years and now Akamai for 1.4 yrs, post acquisition. Immediately, I can think of multiple scenarios' (3) that come to mind on how to solve any one of these categories. Would you like to learn more? lol DB On Mon, Jun 8, 2015 at 7:25 AM, Roland Dobbins rdobb...@arbor.net wrote: On 8 Jun 2015, at 17:57, Ramy Hashish wrote: a BGP session has to be established over a GRE tunnel over the internet between the ISP/NSP/DC and the cloud scrubbing center, This is incorrect. In most cloud overlay DDoS mitigation scenarios (e.g., end-customer obtains service from an MSSP which isn't providing them with transit), a) there is no BGP relationship whatsoever between the end-customer and the MSSP, and b) the GRE tunnel is used strictly for re-injection of clean traffic (i.e., post-mitigation) to the end-customer. In some scenarios, DNS is also used in place of/in addition to BGP-based diversion. But GRE is used for re-injection only. --- Roland Dobbins rdobb...@arbor.net
Re: ARIN just subdivided their last /17, /18, /19, /20, /21 and /22. Down to only /23s and /24s now. : ipv6
On Jun 30, 2015, at 10:03 , Stephen Satchell l...@satchell.net wrote: On 06/30/2015 07:28 AM, Justin M. Streiner wrote: There are still isolated pockets of devices out there speaking IPX, DECnet, Appletalk, etc, but either they're not connected to the 'Internet', or their traffic passes through other devices that encapsulate and de-encapsulate it in IP to allow it to be transported. For the young (and the young at heart) those other devices were known as bridges back in the day. Not all of them… Some of them were “gateways” and occasionally you’d even encounter a “proxy”. Owen
Re: ARIN just subdivided their last /17, /18, /19, /20, /21 and /22. Down to only /23s and /24s now. : ipv6
On Tue, 30 Jun 2015 10:28:13 -0400, Justin M. Streiner strei...@cluebyfour.org wrote: There are still isolated pockets of devices out there speaking IPX, DECnet, Appletalk, etc Indeed. I'm one of them. (rarely) ... IPX managed print server. It speaks IP, but cannot be managed by IP. I'd throw it away, but it functions as a two port serial terminal server as well. (2 parallel, 2 serial) I don't have any true appletalk (or localtalk!) hardware anymore. But I know where there's a palet of them. :-) I still have MCA token-ring cards for an RS/6000 (and the RS/6000.) I'm just waiting for the NCDOT to need one to recoup a wad of tax money. or their traffic passes through other devices that encapsulate and de-encapsulate it in IP to allow it to be transported. A, the internet in a box IPX-IP gateway device. God, how we hated those things. But some companies refused to install an IP stack, 'tho they'd install the IPX IP app suite. (late '90s)
Re: World's Fastest Internet™ in Canadaland
On 15-06-26 14:04, Hank Disuko wrote: Bell Canada is apparently gearing up to provide the good people of Toronto with the World's Fastest Internet™. http://www.thestar.com/news/city_hall/2015/06/25/bell-canada-to-give-toronto-worlds-fastest-internet.html BTW, initally, Bell limits it to 940mbps. Likely because the Sagemcom routers it uses don't have the umph to handle higher bandwidth. (these boxes also have the hacked VDSL modem that interfaces with the not-so-compliant and long ago discontinued Stinger DSKAMs that Bell continued to deploy until 2012 despite these being discontinued in 2005 and never getting full compliance with VDSL2.) One new CPE routers are found, Bell intends to raise marketed up to speed to 1gnps. Of course, it will br priced so that few people order it so congestion in 32 home sectors won't be too much of a problem. Question: From the network operator's point of view, is there a huge difference in network planning: 1- user spends 2 hours streaming a Netflix movie at roughly 6mbps. 2- user spends 5 minutes downloadimng that movie at 150mbps and then is idle for 2 hours while watching it ? Does 2 end up requiring less total capacity because on average fewer people use it at the same time ?
Call for presentations RIPE 71
Dear colleagues, Please find the CFP for RIPE 71 below or at https://ripe71.ripe.net/submit-topic/cfp/. The deadline for submissions is 13 September 2015. Please also note that speakers do not receive any extra reduction or funding towards the meeting fee at the RIPE Meetings. Kind regards, Benno Overeinder RIPE PC Chair https://www.ripe.net/participate/meetings/ripe-meetings/pc Call for Presentations A RIPE Meeting is an open event where Internet Service Providers, network operators and other interested parties get together. Although the meeting is mostly technical, it is also a chance for people to meet and network with others in their field. RIPE 71 will take place from 16-20 November 2015 in Bucharest, Romania. The RIPE Programme Committee (PC) is now seeking content proposals from the RIPE community for the plenary sessions, BoFs (Birds of a Feather sessions), panels, workshops, tutorials and lightning talks at RIPE 71. See the full descriptions of the different presentation formats, https://ripe71.ripe.net/submit-topic/presentation-formats/. Proposals for plenary sessions, BoFs, panels, workshops and tutorials must be submitted for full consideration no later than 13 September 2015. Proposals submitted after this date will be considered depending on the remaining available space in the programme. The PC is looking for presentations covering topics of network engineering and operations, including but not limited to: - IPv6 deployment - Managing IPv4 scarcity in operations - Commercial transactions of IPv4 addresses - Data centre technologies - Network and DNS operations - Internet governance and regulatory practices - Network and routing security - Content delivery - Internet peering and mobile data exchange Submissions RIPE Meeting attendees are quite sensitive to keeping presentations non-commercial, and product marketing talks are strongly discouraged. Repeated audience feedback shows that the most successful talks focus on operational experience, research results, or case studies. For example, presenters wishing to describe a commercial solution should focus on the underlying technology and not attempt a product demonstration. Presenters should indicate how much time they will require. In general, the time allocated for the different presentation formats is as follows: - Plenary presentations: 20-25 minutes presentation with 5-10 minutes discussion - Tutorials: up to two hours (Monday morning) - Workshops: one hour (during evening sessions) to two hours (Monday morning) - BoFs: approximately one hour - Lightning talks: 10 minutes The following general requirements apply: - Proposals must be submitted using the meeting submission system, https://ripe71.ripe.net/submit-topic/submission-form/. - Lightning talks should also be submitted using the meeting submission system (https://ripe71.ripe.net/submit-topic/submission-form/) and can be submitted any time up to and including the meeting week. The allocation of lightning talks will be announced on short notice---in some cases on the same day but often one day prior to the time slot allocated. - Presenters who propose a panel or BoF are encouraged to include speakers from several (perhaps even competing) companies and/or a neutral facilitator. - All presentation proposals will only be considered by the PC if they contain at least draft presentation slides (slides may be updated later on). For panels, proposals must contain a clear description, as well as the names of invited panellists, presenters and moderators. - Due to potential technical issues, presenters/panellists should be physically present at the RIPE Meeting. If you have any questions or requests concerning content submissions, please email pc [at] ripe [dot] net. -- Benno J. Overeinder NLnet Labs http://www.nlnetlabs.nl/
Re: GRE performance over the Internet - DDoS cloud mitigation
Roland, Agreed, Ramy's scenario was not truly spot on, but his question still remains. Perf implications when cloud security providers time to detect/mitigate is X minutes. How stable can GRE transports and BGP sessions be when under load? In my technical opinion, this is a valid argument, which deems wide opinion. Specifically, use-cases about how to apply defense in depth logically in the DC vs Hybrid vs Pure Cloud. Good topic, already some back-chatter personal opinions from Nanog lurkers! Regards, Dennis B. On Tue, Jun 30, 2015 at 2:45 PM, Roland Dobbins rdobb...@arbor.net wrote: On 1 Jul 2015, at 1:37, Dennis B wrote: Would you like to learn more? lol I'm quite conversant with all these considerations, thanks. OP asserted that BGP sessions for diversion into any cloud DDoS mitigation service ran from the endpoint network through GRE tunnels to the cloud-based mitigation provider. I was explaining that in most cloud mitigation scenarios, GRE tunnels are used for re-injection of 'clean' traffic to the endpoint networks. --- Roland Dobbins rdobb...@arbor.net
Re: Route leak in Bangladesh
I have sent this to a contact at another Bangladeshi ISP that should be able to reach the right person for this ASAP. On 30-Jun-2015, at 1:57 pm, Grzegorz Janoszka grzeg...@janoszka.pl wrote: We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. Anybody see their prefixes hijacked as well?
Re: NTT-HE earlier today (~10am EDT)
NTT's customer Sofia Connect leaked our routes to NTT. NTT accepted these routes instead of properly filtering their customer announcements. As a network of non-trivial size, announcing over 75,000 customer routes which is nearly 15% of the IPv4 routing table, we'd expect the common courtesy of having our ASN included in their customer facing AS-PATH filters, as we extend this same courtesy to other networks of this size (such as AS2914). sometimes the goddesses have a sense of humor At Tue, 30 Jun 2015 10:27:21 +0200, Grzegorz Janoszka wrote: We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. randy
Re: Route leak in Bangladesh
A friend in AS58587 confirmed that this was caused by a configuration error - it seems like related to redistribution, and they already fixed that. 7007 all over again. do not redistribute bgp into igp. do not redistribute igp into bgp. randy
Re: World's Fastest Internet™ in Canadaland
On 30 June 2015 at 22:32, Jean-Francois Mezei jfmezei_na...@vaxination.ca wrote: BTW, initally, Bell limits it to 940mbps. 940 Mbps is what speedtest.net will give you on a linespeed 1 Gbps connection. That sounds more like marketing people trying to understand overhead. Regards, Baldur
Re: Thoughts On Cheap Chinese xDSL Testers
There are some downsides with the Colt-250+ units (as I have one almost daily to do installs for a CLEC). 1. The Colts require 4 high amperage AA batteries. I used to purchase Duracell Ultra batteries which worked, but life span was a couple of weeks to maybe a month and now I cannot seem to find them in stores. I now use Lithium batteries and they seem to last a few months now. 2. They will only sync up for 100 Seconds max. Not helpful when you're trying to diagnose a flapping circuit. 3. They won't stay sync'd up on circuits with Occam DSLAMs. They randomly drop after a few seconds. Not a condition of the circuit. Some type of incompatibility with Occams. 4. As others said, not a Layer 3 or 2 (I think). 5. Does not provide any additional details like Far end errors, Near end Errors, FEC/HEC, etc. On the plus side: 1. They boot up really quick, as quick as you can press 2 buttons you can start a test. (my understanding is Sunrise units take a couple of minutes to bootup) 2. Relatively lightweight. 3. Can use regular 6p4c line cords in case you lose the nice Angled-Bed-of-Nails/6p4c test cable it comes with (like I accidentally did). 4. Can be purchased for cheap on eBay. I got mine years ago for less than $150.00 On Mon, Jun 29, 2015 at 9:23 PM, Robert Glover robe...@garlic.com wrote: The local ILEC (Verizon) use Colt 250+. They are pretty cool. They do not do layer 3 like the meter you referenced. I'm actually looking for a cost-effective meter that does ADSL+ / VDSL2 / e.SHDSL. it's easy to find one that does the first two, but not all three. Original message From: Lyndon Nerenberg lyn...@orthanc.ca Date: 06/29/2015 5:50 PM (GMT-08:00) To: North American Network Operators' Group nanog@nanog.org Subject: Thoughts On Cheap Chinese xDSL Testers I've been poking around looking for an inexpensive xDSL circuit tester to do some measurements on my home DSL line, in opposition to the telco. $2K+ is not in the budget, so I'm curious about the accuracy of the $300 Chinese units kicking around eBay (e.g. the ST332B). Anyone out there have experience with them? Are they even remotely close to accurate? --lyndon -- Joshua Zukerman President Snow Pond Technology Group Inc. www.snowpondtech.com
Re: NTT-HE earlier today (~10am EDT)
I was thinking that when I posted yesterday. These were announcements from a peer, not customer routes. We are lowering our max prefix limits on many peers as a result of this. We are also going towards more prefix filtering on peers beyond bogons and martians. Mike. On 6/30/15 2:19 AM, Randy Bush wrote: NTT's customer Sofia Connect leaked our routes to NTT. NTT accepted these routes instead of properly filtering their customer announcements. As a network of non-trivial size, announcing over 75,000 customer routes which is nearly 15% of the IPv4 routing table, we'd expect the common courtesy of having our ASN included in their customer facing AS-PATH filters, as we extend this same courtesy to other networks of this size (such as AS2914). sometimes the goddesses have a sense of humor At Tue, 30 Jun 2015 10:27:21 +0200, Grzegorz Janoszka wrote: We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. randy
Re: ARIN just subdivided their last /17, /18, /19, /20, /21 and /22. Down to only /23s and /24s now. : ipv6
On Tue, Jun 30, 2015 at 1:43 PM, Ricky Beam jfb...@gmail.com wrote: On Tue, 30 Jun 2015 10:28:13 -0400, Justin M. Streiner strei...@cluebyfour.org wrote: There are still isolated pockets of devices out there speaking IPX, DECnet, Appletalk, etc Indeed. I'm one of them. (rarely) ... IPX managed print server. It speaks IP, but cannot be managed by IP. I'd throw it away, but it functions as a two port serial terminal server as well. (2 parallel, 2 serial) I don't have any true appletalk (or localtalk!) hardware anymore. But I know where there's a palet of them. :-) I still have MCA token-ring cards for an RS/6000 (and the RS/6000.) I'm just waiting for the NCDOT to need one to recoup a wad of tax money. or their traffic passes through other devices that encapsulate and de-encapsulate it in IP to allow it to be transported. A, the internet in a box IPX-IP gateway device. God, how we hated those things. But some companies refused to install an IP stack, 'tho they'd install the IPX IP app suite. (late '90s) But how much memory you could save if you only ran IPX. Adding the IP stack would take you below 500K and then you would have programs that just wouldn't run. QEMM could only do so much.
Re: NTT-HE earlier today (~10am EDT)
* Mike Leber I was thinking that when I posted yesterday. These were announcements from a peer, not customer routes. We are lowering our max prefix limits on many peers as a result of this. We are also going towards more prefix filtering on peers beyond bogons and martians. Hi Mike, You're not mentioning RPKI here. Any particular reason why not? If I understand correctly, in today's leak the origin AS was changed/reset, so RPKI ought to have saved the day. (At least Grzegorz' day, considering that 33 of AS43996's prefixes are covered by ROAs.) Tore At Tue, 30 Jun 2015 10:27:21 +0200, Grzegorz Janoszka wrote: We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them.
Re: NTT-HE earlier today (~10am EDT)
We have been pushing large configurations to devices. You can check my slides from the London IEPG meeting. When 96% of your config is prefix filters we are sure trying. I ask others to encourage your vendors to make this a priority as we have faced a number of issues in this area and have been waiting quite some time for vendor resolution. Jared Mauch On Jun 30, 2015, at 5:26 PM, Mike Leber mle...@he.net wrote: On 6/30/15 3:02 PM, Tore Anderson wrote: * Mike Leber I was thinking that when I posted yesterday. These were announcements from a peer, not customer routes. We are lowering our max prefix limits on many peers as a result of this. We are also going towards more prefix filtering on peers beyond bogons and martians. Hi Mike, You're not mentioning RPKI here. Any particular reason why not? If I understand correctly, in today's leak the origin AS was changed/reset, so RPKI ought to have saved the day. (At least Grzegorz' day, considering that 33 of AS43996's prefixes are covered by ROAs.) Yes, we will incorporate RPKI into how we build our prefix filters for peers as we improve our tools. Currently this will involve some amount of prefix list compression due to the limits of current hardware and the need to still have BGP converge. As Job Snijders said, I would forsee issues if i'd try to add an eleven megabyte prefix-list on all devices in the network.. Mike.
RE: Route leak in Bangladesh
Yes, we have seen one of our prefixes hikacked. We contacted to Fiberathome and they told us the issue has been solved. Greetings. Ferran. -Mensaje original- De: NANOG [mailto:nanog-boun...@nanog.org] En nombre de Grzegorz Janoszka Enviado el: martes, 30 de junio de 2015 10:27 Para: nanog@nanog.org Asunto: Route leak in Bangladesh We have just received alert from bgpmon that AS58587 Fiber @ Home Limited has hijacked most of our (AS43996) prefixes and Hurricane Electric gladly accepted them. Anybody see their prefixes hijacked as well? -- Grzegorz Janoszka
Re: NTT-HE earlier today (~10am EDT)
On 6/30/15 3:02 PM, Tore Anderson wrote: * Mike Leber I was thinking that when I posted yesterday. These were announcements from a peer, not customer routes. We are lowering our max prefix limits on many peers as a result of this. We are also going towards more prefix filtering on peers beyond bogons and martians. Hi Mike, You're not mentioning RPKI here. Any particular reason why not? If I understand correctly, in today's leak the origin AS was changed/reset, so RPKI ought to have saved the day. (At least Grzegorz' day, considering that 33 of AS43996's prefixes are covered by ROAs.) Yes, we will incorporate RPKI into how we build our prefix filters for peers as we improve our tools. Currently this will involve some amount of prefix list compression due to the limits of current hardware and the need to still have BGP converge. As Job Snijders said, I would forsee issues if i'd try to add an eleven megabyte prefix-list on all devices in the network.. Mike.
Re: NTT-HE earlier today (~10am EDT)
On Wed, Jul 01, 2015 at 12:02:40AM +0200, Tore Anderson wrote: I was thinking that when I posted yesterday. These were announcements from a peer, not customer routes. We are lowering our max prefix limits on many peers as a result of this. We are also going towards more prefix filtering on peers beyond bogons and martians. You're not mentioning RPKI here. Any particular reason why not? If I understand correctly, in today's leak the origin AS was changed/reset, so RPKI ought to have saved the day. (At least Grzegorz' day, considering that 33 of AS43996's prefixes are covered by ROAs.) This assessment is correct, however there might be some constraints in play with regard to RPKI, which are not really related to RPKI itself, which prohibit meaningful deployment. I've seen a few obstacles myself: - equipment might not support the RTR protocol to validate announcements against the cache validator - Legal obstacles in obtaining the anchors from all RIRs - when not using the RTR protocol but generating prefix-list filters based on RPKI data, the devices might not support sufficient entries. Would be good if other people share obstacles, and possibly, the methods they used to overcome those. I'll count not using brocade as a valid method. Kind regards, Job
Re: NTT-HE earlier today (~10am EDT)
On Tue, Jun 30, 2015 at 03:32:42PM -0700, Ca By wrote: It is NTT that would have mitigated this issue if they deployed and enforcer rpki, right? No, NTT deploying RPKI would not have helped in yesterday's issue. But, RPKI could've made a difference in today's Bangladesh leak, even if RPKI validation was not implemented by the direct upstream with propagated the leak. Kind regards, Job
Re: NTT-HE earlier today (~10am EDT)
On Tue, Jun 30, 2015 at 05:40:03PM -0500, Jared Mauch wrote: We have been pushing large configurations to devices. You can check my slides from the London IEPG meeting. These are the slides: http://iepg.org/2014-03-02-ietf89/ietf89_iepg_jmauch.pdf When 96% of your config is prefix filters we are sure trying. I ask others to encourage your vendors to make this a priority as we have faced a number of issues in this area and have been waiting quite some time for vendor resolution. I'd like to add, that our average router config, since then has grown with an extra 100k lines compared to those 2013 statistics. Kind regards, Job
Re: NTT-HE earlier today (~10am EDT)
As Job Snijders said, I would forsee issues if i'd try to add an eleven megabyte prefix-list on all devices in the network.. and that is why i stopped peering with HE. i run origin validation; issue roas please. randy
Call for presentations EPF 10
Call for Presentations European Peering Forum 10 AMS-IX, DE-CIX, LINX, Netnod and guest IXP Equinix, are happy to host the 10th European Peering Forum (EPF) in Madrid, Spain from the 21st till the 23th of September 2015. The event will welcome up to 250 peering managers and coordinators from networks connected to the host Internet exchanges. Besides an interesting topical agenda, the three-day event accommodates room for attendees to meet on a one-to-one basis to discuss bilateral peering business opportunities. The programme committee will be looking for presentations and lightning talks related to peering and technical topics of interconnection. Your presentation should address * Interconnection Automation * Regional Peering * Interconnection and Peering Internet Governance and Regulatory Topics * Economic and Product Trends * Peering/Interconnection Strategy * Interesting findings about peering * 100GE Submissions === Presentations must be of a non-commercial nature. Product or marketing heavy talks are strongly discouraged. Submissions of presentations should be made to the programme committee epf...@peering-forum.eu. Please include: * Author's name and e-mail * Presentation title * Abstract * Slides (if available) * Time requested Deadlines = Presentation Abstract Deadline 2015-07-20 12:00 UTC Final Selection of Speakers 2015-07-31 Presentation Slides Submission Deadline 2015-09-04 12:00 UTC -- Arnold Nipper CTO/COO and Co-Founder DE-CIX Management GmbH | Lindleystrasse 12 | 60314 Frankfurt am Main | Germany | www.de-cix.net | Phone +49 69 1730902 22 | Mobile +49 172 2650958 | Fax +49 69 4056 2716 | arnold.nip...@de-cix.net | Geschaeftsfuehrer Harald A. Summa | Registergericht AG Koeln HRB 51135 signature.asc Description: OpenPGP digital signature
Re: NTT-HE earlier today (~10am EDT)
On Tuesday, June 30, 2015, Mike Leber mle...@he.net wrote: On 6/30/15 3:02 PM, Tore Anderson wrote: * Mike Leber I was thinking that when I posted yesterday. These were announcements from a peer, not customer routes. We are lowering our max prefix limits on many peers as a result of this. We are also going towards more prefix filtering on peers beyond bogons and martians. Hi Mike, You're not mentioning RPKI here. Any particular reason why not? If I understand correctly, in today's leak the origin AS was changed/reset, so RPKI ought to have saved the day. (At least Grzegorz' day, considering that 33 of AS43996's prefixes are covered by ROAs.) Yes, we will incorporate RPKI into how we build our prefix filters for peers as we improve our tools. Currently this will involve some amount of prefix list compression due to the limits of current hardware and the need to still have BGP converge. As Job Snijders said, I would forsee issues if i'd try to add an eleven megabyte prefix-list on all devices in the network.. Mike. It is NTT that would have mitigated this issue if they deployed and enforcer rpki, right?
Re: OK, Google. Time to dial back the AI hype.
Is the WSJ a wholly owned subsidiary of GOOG? It looks to me like a WSJ journalist said that. If you read the paper, which is linked from the article and takes about five minutes, you'll find that article is cheap clickbait and has approximately nothing to do with the topic of the paper. As far as I can tell, none of the people in the lengthy slashdot thread bothered to do that. ObNANOG: it's about a text chat app that can be loaded with various text databases. The first couple of examples use a tech support database and the examples are impressively close to the kind of tech support one gets from offshore script readers. The example quoted in the WSJ used a database of movie subtitles, so it's not surprising that it reads like a bad movie script. R's, John
