Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On 31/Mar/20 15:21, Dorian Kim wrote: > Mark, > > Unfortunately we don’t have any testing done or experience with RPKI on XE or > Classic boxes as we don’t have any deployed outside of OOB infrastructure. Cherish your blessings, and for the time being, keep them that way :-). Mark.
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On 31/Mar/20 15:21, Dorian Kim wrote: > Mark, > > Unfortunately we don’t have any testing done or experience with RPKI on XE or > Classic boxes as we don’t have any deployed outside of OOB infrastructure. Cherish your blessing, and for the time being, keep them that way :-). Mark.
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
> On Mar 31, 2020, at 7:19 AM, Mark Tinka wrote: > > > >> On 26/Mar/20 02:50, Job Snijders wrote: >> Dear group, >> >> Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI >> based BGP Origin Validation on virtually all EBGP sessions, both >> customer and peering edge. This change positively impacts the Internet >> routing system. > > Good man. The club is growing :-). > > Quick one - do you have ROV on any IOS or IOS XE-based boxes? We've had > to walk back the few we did in recent weeks; the thing is just totally > broken there. Mark, Unfortunately we don’t have any testing done or experience with RPKI on XE or Classic boxes as we don’t have any deployed outside of OOB infrastructure. -dorian
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On 31/Mar/20 14:46, Ben Maddison wrote: > Tomorrow is our first ROV invalid = reject anniversary, Ah yes - April 1 :-). Had actually forgotten about that. Fun times :-). Congrats - we are 4 days behind you. > and for most of > that time I have been in communications at various levels with Cisco > regarding the shocking brokenness in classic and XE. > > Aside from some well meaning sounding email, crickets. > > I very much hope, for the sake of the interwebs at large, that you have > more luck than me. We're are falling back to plan B, aka truck-roll. I've actually been able to find a decent warm body that understands the problems, has taken the suggested fixes and is willing to listen. Initially, all fixes had been scheduled for 17.2 However, he is, since the 18th of this month, working on committing the fixes to all earlier throttles that are still open for commit, which includes a number of 16.x releases. I also asked if he can back-port those fixes to the Cisco 7200 train - so 15.2(S4)8 - and he's graciously looking into that. We use a couple of those as looking glasses around the world (although we are moving to CSR1000v now for that), but I know that a handful of networks are still running that platform in production, and we also use it for lab workshops and such. So getting that fixed would be most helpful. Mark.
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On Tue, 2020-03-31 at 13:18 +0200, Mark Tinka wrote: > > On 26/Mar/20 02:50, Job Snijders wrote: > > Dear group, > > > > Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI > > based BGP Origin Validation on virtually all EBGP sessions, both > > customer and peering edge. This change positively impacts the > > Internet > > routing system. > > Good man. The club is growing :-). > > Quick one - do you have ROV on any IOS or IOS XE-based boxes? We've > had > to walk back the few we did in recent weeks; the thing is just > totally > broken there. > > The good news is Cisco are listening to fix suggestions, and I'm > waiting > for test code to verify. > Tomorrow is our first ROV invalid = reject anniversary, and for most of that time I have been in communications at various levels with Cisco regarding the shocking brokenness in classic and XE. Aside from some well meaning sounding email, crickets. I very much hope, for the sake of the interwebs at large, that you have more luck than me. We're are falling back to plan B, aka truck-roll. Cheers, Ben
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On 26/Mar/20 02:50, Job Snijders wrote: > Dear group, > > Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI > based BGP Origin Validation on virtually all EBGP sessions, both > customer and peering edge. This change positively impacts the Internet > routing system. Good man. The club is growing :-). Quick one - do you have ROV on any IOS or IOS XE-based boxes? We've had to walk back the few we did in recent weeks; the thing is just totally broken there. The good news is Cisco are listening to fix suggestions, and I'm waiting for test code to verify. Mark.
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
Many congratulations for getting this deployed, Job! Now that so many networks are dropping RPKI invalid announcements, for this to really have a practical effect operators should put in the effort to create and maintain ROAs for their route announcements. Over the last 10 years, the trend in most regions has been that first a large amount of ROAs were created by the local operator communities. After proving this was a high quality and well maintained data set, operators took the next step to start using RPKI to filter invalids. Especially in North America, the order seems to be flipped. While invalids are now being dropped more and more, ROA coverage is currently only at 7% in the US and 2.5% in Canada. Accuracy is at around 95%, so that’s great. https://www.nlnetlabs.nl/projects/rpki/rpki-analytics/ Please create ROAs! -Alex > On 26 Mar 2020, at 01:50, Job Snijders wrote: > > Dear group, > > Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI > based BGP Origin Validation on virtually all EBGP sessions, both > customer and peering edge. This change positively impacts the Internet > routing system. > > The use of RPKI technology is a critical component in our efforts to > improve Internet routing stability and reduce the negative impact of > misconfigurations or malicious attacks. RPKI Invalid route announcements > are now rejected in NTT EBGP ingress policies. A nice side effect: > peerlock AS_PATH filters are incredibly effective when combined with > RPKI OV. > > For NTT, this is the result of a multiyear project, which included > outreach, education, collaboration with industry partners, and > production of open source software shared among colleagues in the > industry. > > Shout out to Louis & team (Cloudflare) for the open source GoRTR > software and the OpenBSD project for rpki-client(8). > > I hope some take this news as encouragement to consider RPKI OV > "invalid == reject"-policies as safe to deploy in their own BGP > environments too. :-) > > If you have questions, feel free to reach out to me directly or the > NTT NOC at . > > Kind regards, > > Job
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
Job, Congratulations to NTT, AT&T, and others in our community who have deployed validation on their network edge. What is really exciting is all the activity in this and other operator regions that has come together to promote securing the routing system by combining multiple strategies. This shift to leadership by example is a big shift from just using the mailing list for public shaming anti-social behavior. (Public shaming should still be used strategically :-). While this is an important step, the big win that I see is the larger project promoting securing the routing system by combining multiple strategies. There is significant power in combining multiple strategies and engaging other organizations to develop their own multi-pronged approach. That evangelizing and investing in "leadership by example" has really accelerated this project in ways that individual engineers struggled to do so before and as a community where we remained stuck in the past. By raising the industry standard for routing security, implementation of these measures is no longer optional, and that has been done without government interference. NTT has invested in bringing this whole package to the NANOG region -- developing tools, working with other networks and even competitors, and evangelization of routing security. I am speaking on my own behalf, but as NANOG has started using social media and other online tools to promote the knowledge of the community, I will talk to the PC and staff about curating routing security materials for an educational playlist. If there is any chance you could resend a link to some of your materials, I think that would be beneficial (IRR tools, rpki validation and planning tools, peerlock implementation)? I also encourage any operator with a few spare minutes to poke around manrs.org/isps . Thanks, Sean Em qui., 26 de mar. de 2020 às 08:32, Job Snijders escreveu: > Dear group, > > Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI > based BGP Origin Validation on virtually all EBGP sessions, both > customer and peering edge. This change positively impacts the Internet > routing system. > > The use of RPKI technology is a critical component in our efforts to > improve Internet routing stability and reduce the negative impact of > misconfigurations or malicious attacks. RPKI Invalid route announcements > are now rejected in NTT EBGP ingress policies. A nice side effect: > peerlock AS_PATH filters are incredibly effective when combined with > RPKI OV. > > For NTT, this is the result of a multiyear project, which included > outreach, education, collaboration with industry partners, and > production of open source software shared among colleagues in the > industry. > > Shout out to Louis & team (Cloudflare) for the open source GoRTR > software and the OpenBSD project for rpki-client(8). > > I hope some take this news as encouragement to consider RPKI OV > "invalid == reject"-policies as safe to deploy in their own BGP > environments too. :-) > > If you have questions, feel free to reach out to me directly or the > NTT NOC at . > > Kind regards, > > Job >
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On 26/03/2020 10:39, Brandon Butterworth wrote: > What are you waiting for, someone to say make it so? I knew someone would come back with the smartarse response ;) I'm certainly not the authority on this, and I'm not tracking the deployments with any great detail. I'm happy to suggest where we could best host this information, however! -- Tom
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On Thu Mar 26, 2020 at 10:12:55AM +, Tom Hill wrote: > I am deeply upset that there isn't yet a Wikipedia article entitled, > "List of BGP networks implementing RPKI"... :) What are you waiting for, someone to say make it so? Plus a little graph showing the approaching RPKI event horizon brandon
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
> On Mar 25, 2020, at 10:05 PM, JASON BOTHE via NANOG wrote: > > Excellent work. I’m curious to know how many of the big ASs are participating > to date. If you or anyone on the list knows if this is published please let > me know. Quite a number have done this. I expect we are getting close to herd immunity if a few more large providers are able to deploy. - Jared
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
On 26/03/2020 02:05, JASON BOTHE via NANOG wrote: > Excellent work. I’m curious to know how many of the big ASs are > participating to date. If you or anyone on the list knows if this is > published please let me know. I am deeply upset that there isn't yet a Wikipedia article entitled, "List of BGP networks implementing RPKI"... :) If we can have nerdy lists of GPUs and CPUs, this must be valid also? -- Tom
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
Excellent work. I’m curious to know how many of the big ASs are participating to date. If you or anyone on the list knows if this is published please let me know. Thanks J~ > On Mar 25, 2020, at 21:03, Michel Py wrote: > > Hi Job, > >> Job Snijders wrote : >> Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based >> BGP Origin Validation on virtually all >> EBGP sessions, both customer and peering edge. This change positively >> impacts the Internet routing system. > > Great, and thanks ! > I do have a question, the same one everyone has on their mind : > How much whining / angry customers / calls / etc came out of it ? > > > Why did you say anything instead of eventually blaming it on the coronavirus > ? :P > > > Michel.
Re: NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
Hi Job, > Job Snijders wrote : > Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based BGP > Origin Validation on virtually all > EBGP sessions, both customer and peering edge. This change positively impacts > the Internet routing system. Great, and thanks ! I do have a question, the same one everyone has on their mind : How much whining / angry customers / calls / etc came out of it ? Why did you say anything instead of eventually blaming it on the coronavirus ? :P Michel.
NTT/AS2914 enabled RPKI OV 'invalid = reject' EBGP policies
Dear group, Exciting news! Today NTT's Global IP Network (AS 2914) enabled RPKI based BGP Origin Validation on virtually all EBGP sessions, both customer and peering edge. This change positively impacts the Internet routing system. The use of RPKI technology is a critical component in our efforts to improve Internet routing stability and reduce the negative impact of misconfigurations or malicious attacks. RPKI Invalid route announcements are now rejected in NTT EBGP ingress policies. A nice side effect: peerlock AS_PATH filters are incredibly effective when combined with RPKI OV. For NTT, this is the result of a multiyear project, which included outreach, education, collaboration with industry partners, and production of open source software shared among colleagues in the industry. Shout out to Louis & team (Cloudflare) for the open source GoRTR software and the OpenBSD project for rpki-client(8). I hope some take this news as encouragement to consider RPKI OV "invalid == reject"-policies as safe to deploy in their own BGP environments too. :-) If you have questions, feel free to reach out to me directly or the NTT NOC at . Kind regards, Job
