Re: Large prefix lists/sets on IOS-XR
On Fri, 9 Dec 2022 at 20:19, t...@pelican.org wrote: Hey Tim, > Or at least, you've moved the problem from "generate config" to "have > complete and correct data". Which statement should probably come with some > kind of trigger-warning... I think it's a lot easier than you think. I understand that all older networks and practical access networks have this problem, the data is in the network, it's of course not the right way to do it, but it's the way they are. But there is no reason to get discouraged. First you gotta ignore waterfall model, you can never order something ready and have utility out of it, because no data. What you can do, day1 a) copy configs as-is, as templates b) only edit the template c) push templates to network boom, now you are FAR, and that took an hour or day depending on the person. Maybe you feel like you've not accomplished much, but you have. Now you can start modelling data out of the template into the database, and keep shrinking the 'blobs'. You can do this at whatever pace is convenient, and you can trivially measure which one to do next, which one will reduce total blob bytes most. You will see constant, measurable progress. And you always know the network state is always what is in your files, as you are now always replacing the entire config with the generated config. -- ++ytti
Re: AS3356 Announcing 2000::/12
On Thu, Dec 8, 2022 at 9:35 AM Randy Bush wrote: > while i think the announcement is, shall we say, embarrassing, i do not > see how it would be damaging. real/correct announcements would be for > longer prefixes, yes? > > randy > Putting on a probably-overly-paranoid hat for a moment... If I announce 2000::/12, seemingly as an innocent error, it won't break most people's routing, and is likely to be simply chalked up as a copy-paste error, or other human "oops". But if I happen to be running a promiscuous packet capture on a box that the "erroneous" routing table entry ultimately resolves to, I warrant there's a certain amount of legitimate packet streams I could collect here and there, any time a router processes a WITHDRAW update message for a more specific prefix within the range, before a new ANNOUNCE update message is processed. I'm not going to get a great deal of information, as most simple prefix updates happen within the same update message; but during periods of higher internal churn in a network, you may have brief periods during which the more specific route is withdrawn before being re-announced, during which I'd be able to harvest packets destined for other networks. As I said--I'm probably being overly paranoid, but I can't help but wonder what packets such a collector might see, if left to run for a week or two... ^_^; Thanks! Matt
Re: AS3356 Announcing 2000::/12
> I know of a few people in a Discord that filter out anything bigger > than /16 routes, would this be wise to implement as a best practice? once upon a time, a very large provider took two /8s and announced as a /7. a vendor who thought a /8 was as short as they would ever see had routers fall over in a receiving large provider. do not hard code social theories. remember 640k. randy
Re: Large prefix lists/sets on IOS-XR
On Friday, 9 December, 2022 16:04, "Saku Ytti" said: > If you remove the need for deltas the whole problem becomes extremely > trivial. Fill in all the templates with data, push it. Or at least, you've moved the problem from "generate config" to "have complete and correct data". Which statement should probably come with some kind of trigger-warning... Cheers, Tim.
RE: Large prefix lists/sets on IOS-XR
Sander, How big? How slow? You can reply to me off or on list. About 8 to 10 years ago, we had a large effort to improve this. Now customers push many megabytes of prefix-sets several times a day and it works. I have sent some questions internally to get a better answer. Related, in 7.2.1, we added the as-set, which allows you to filter BGP routes by origin-as. It is similar to as-path-set. as-path-set is slow, using a linear lookup, but it is versatile, allowing ios-regex. as-set can only use numbers and only for origin-as, but it is fast using a log(N) lookup. Regards, Jakob. -Original Message- Date: Fri, 9 Dec 2022 00:02:52 +0100 From: Sander Steffann Hi, What is the best/most efficient/most convenient way to push large prefix lists or sets to an XR router for BGP prefix filtering? Pushing thousands of lines through the CLI seems foolish, I tried using the load command but it seems horribly slow. What am I missing? :) Cheers! Sander --- for every complex problem, there?s a solution that is simple, neat, and wrong
RE: AS3356 Announcing 2000::/12
I know of a few people in a Discord that filter out anything bigger than /16 routes, would this be wise to implement as a best practice? From: Warren Kumari Sent: Friday, December 9, 2022 9:13 AM To: Job Snijders Cc: r...@rkhtech.org; North American Network Operators' Group Subject: Re: AS3356 Announcing 2000::/12 On Thu, Dec 8 2022 at 12:38 PM, Job Snijders mailto:nanog@nanog.org> > wrote: Hi all, On Wed, Dec 07, 2022 at 08:24:54PM -0800, Ryan Hamel wrote: AS3356 has been announcing 2000::/12 for about 3 hours now, an aggregate covering over 23K prefixes (just over 25%) of the IPv6 DFZ. A few months ago I wrote: "Frequently Asked Questions about 2000::/12 and related routing errors": https://www.ripe.net/ripe/mail/archives/routing-wg/2022-July/004588.html Oh, that's a nice write-up. I must admit that it didn't occur to me that e.g 2000::/12 was likely something much more specific, but that someone missed the (probably) 6, 7, or 8 at the end, even though I've done this a few times myself… W Kind regards, Job
Weekly Global IPv4 Routing Table Report
This is an automated weekly mailing describing the state of the Global IPv4 Routing Table as seen from APNIC's router in Japan. The posting is sent to APOPS, NANOG, AfNOG, SANOG, PacNOG, SAFNOG TZNOG, MENOG, BJNOG, SDNOG, CMNOG, LACNOG and the RIPE Routing WG. Daily listings are sent to bgp-st...@lists.apnic.net. For historical data, please see https://thyme.apnic.net. If you have any comments please contact Philip Smith . IPv4 Routing Table Report 04:00 +10GMT Sat 10 Dec, 2022 BGP Table (Global) as seen in Japan. Report Website: https://thyme.apnic.net Detailed Analysis: https://thyme.apnic.net/current/ Analysis Summary BGP routing table entries examined: 916614 Prefixes after maximum aggregation (per Origin AS): 346701 Deaggregation factor: 2.64 Unique aggregates announced (without unneeded subnets): 443400 Total ASes present in the Internet Routing Table: 73926 Prefixes per ASN: 12.40 Origin-only ASes present in the Internet Routing Table: 63481 Origin ASes announcing only one prefix: 26081 Transit ASes present in the Internet Routing Table: 10445 Transit-only ASes present in the Internet Routing Table:422 Average AS path length visible in the Internet Routing Table: 4.3 Max AS path length visible: 55 Max AS path prepend of ASN (265020) 50 Prefixes from unregistered ASNs in the Routing Table: 1039 Number of instances of unregistered ASNs: 1044 Number of 32-bit ASNs allocated by the RIRs: 40848 Number of 32-bit ASNs visible in the Routing Table: 33831 Prefixes from 32-bit ASNs in the Routing Table: 164978 Number of bogon 32-bit ASNs visible in the Routing Table:16 Special use prefixes present in the Routing Table:1 Prefixes being announced from unallocated address space:516 Number of addresses announced to Internet: 3061103104 Equivalent to 182 /8s, 116 /16s and 186 /24s Percentage of available address space announced: 82.7 Percentage of allocated address space announced: 82.7 Percentage of available address space allocated: 100.0 Percentage of address space in use by end-sites: 99.6 Total number of prefixes smaller than registry allocations: 311058 APNIC Region Analysis Summary - Prefixes being announced by APNIC Region ASes: 239692 Total APNIC prefixes after maximum aggregation: 68356 APNIC Deaggregation factor:3.51 Prefixes being announced from the APNIC address blocks: 234528 Unique aggregates announced from the APNIC address blocks:97353 APNIC Region origin ASes present in the Internet Routing Table: 13152 APNIC Prefixes per ASN: 17.83 APNIC Region origin ASes announcing only one prefix: 3835 APNIC Region transit ASes present in the Internet Routing Table: 1756 Average APNIC Region AS path length visible:4.6 Max APNIC Region AS path length visible: 31 Number of APNIC region 32-bit ASNs visible in the Routing Table: 8409 Number of APNIC addresses announced to Internet: 773713792 Equivalent to 46 /8s, 29 /16s and 239 /24s APNIC AS Blocks4608-4864, 7467-7722, 9216-10239, 17408-18431 (pre-ERX allocations) 23552-24575, 37888-38911, 45056-46079, 55296-56319, 58368-59391, 63488-64098, 64297-64395, 131072-151865 APNIC Address Blocks 1/8, 14/8, 27/8, 36/8, 39/8, 42/8, 43/8, 49/8, 58/8, 59/8, 60/8, 61/8, 101/8, 103/8, 106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8, 116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8, 123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8, 163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8, 203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8, 222/8, 223/8, ARIN Region Analysis Summary Prefixes being announced by ARIN Region ASes:267472 Total ARIN prefixes after maximum aggregation: 121859 ARIN Deaggregation factor: 2.19 Prefixes being announced from the ARIN address blocks: 269090 Unique aggregates announced from the ARIN address blocks:129548 ARIN Region origin ASes present in the Internet Routing Table:19058 ARIN Prefixes per ASN:
Re: Large prefix lists/sets on IOS-XR
Fri, Dec 09, 2022 at 05:33:09PM +0200, Saku Ytti: > If you read carefully, that is what Steffann is doing. He is doing > 'load location:file' + 'commit'. He is not punching anything by hand. > > So the answer we are looking for is how to make that go faster. > > In Junos answer would be 'ephemeral config', but in IOS-XR as far as I > know, the only thing you can do is improve the 'load' part by moving > the server closer, other than that, you get what you get. set the tcp default mss higher use rcpd remove unnecessary whitespace; ios parser is slow as are the bits behind it remove comments
Re: Large prefix lists/sets on IOS-XR
Hi Ytti, >> Pushing thousands of lines via CLI/expect automation is def not a great >> idea, no. Putting everything into a file, copying that to the device, and >> loading from there is generally best regardless. The slowness you refer to >> is almost certainly just because of how XR handles config application. If >> I'm following correctly, that seems to be the crux of your question. > > If you read carefully, that is what Steffann is doing. He is doing > 'load location:file' + 'commit'. He is not punching anything by hand. > > So the answer we are looking for is how to make that go faster. > > In Junos answer would be 'ephemeral config', but in IOS-XR as far as I > know, the only thing you can do is improve the 'load' part by moving > the server closer, other than that, you get what you get. Perfect answer :) Not what I was hoping to hear, but if that’s what it is, then that’s what it is. Cheers! Sander
Re: AS3356 Announcing 2000::/12
On Thu, Dec 8 2022 at 12:38 PM, Job Snijders wrote: Hi all, > > On Wed, Dec 07, 2022 at 08:24:54PM -0800, Ryan Hamel wrote: > > AS3356 has been announcing 2000::/12 for about 3 hours now, an aggregate > covering over 23K prefixes (just over 25%) of the IPv6 DFZ. > > A few months ago I wrote: "Frequently Asked Questions about 2000::/12 and > related routing errors": > > https://www.ripe.net/ripe/mail/archives/routing-wg/2022-July/004588.html > Oh, that's a nice write-up. I must admit that it didn't occur to me that e.g 2000::/12 was likely something much more specific, but that someone missed the (probably) 6, 7, or 8 at the end, even though I've done this a few times myself… W > Kind regards, > > Job >
Re: Large prefix lists/sets on IOS-XR
On Fri, 9 Dec 2022 at 17:58, Joshua Miller wrote: > In terms of structured vs unstructured data, sure, assembling text is not a > huge lift. Though, when you're talking about layering on complex use cases, > then it gets more complicated. Especially if you want to compute the inverse > configuration to remove service instances that are no longer needed. In terms > of vendor support, I'd hope that if you're paying that kind of money, you're > getting a product that meets your requirements. Something that should be > assessed during vendor selection and procurement. That's just my preference; > do whatever works best for your use cases. Deltas are _super_ hard. But you never need to do them. Always produce a complete config, and let the vendor deal with the problem. We've done this with Junos, IOSXR, EOS (compass, not arista, RIP), SROS (MDCLI) for years If you remove the need for deltas the whole problem becomes extremely trivial. Fill in all the templates with data, push it. -- ++ytti
Re: Large prefix lists/sets on IOS-XR
I agree, I don't think you can get around the XR config bottleneck. But that's not really the end of the story. Let's think about why the loading time matters to Sander (Sander, please chime in here): 1. They have to address an immediate issue for a customer (internal or external) and the customer is impatient. 2. They're sitting there waiting for the load to complete before they can move on to their next task. Possibly having to make the same change to multiple devices. I can't really offer anything for the first case. But in the second case, automation can help address that by making changes more unattended and parallelizable. In terms of structured vs unstructured data, sure, assembling text is not a huge lift. Though, when you're talking about layering on complex use cases, then it gets more complicated. Especially if you want to compute the inverse configuration to remove service instances that are no longer needed. In terms of vendor support, I'd hope that if you're paying that kind of money, you're getting a product that meets your requirements. Something that should be assessed during vendor selection and procurement. That's just my preference; do whatever works best for your use cases. On Fri, Dec 9, 2022 at 10:35 AM Saku Ytti wrote: > On Fri, 9 Dec 2022 at 17:30, Tom Beecher wrote: > > > Pushing thousands of lines via CLI/expect automation is def not a great > idea, no. Putting everything into a file, copying that to the device, and > loading from there is generally best regardless. The slowness you refer to > is almost certainly just because of how XR handles config application. If > I'm following correctly, that seems to be the crux of your question. > > If you read carefully, that is what Steffann is doing. He is doing > 'load location:file' + 'commit'. He is not punching anything by hand. > > So the answer we are looking for is how to make that go faster. > > In Junos answer would be 'ephemeral config', but in IOS-XR as far as I > know, the only thing you can do is improve the 'load' part by moving > the server closer, other than that, you get what you get. > -- > ++ytti >
Re: Large prefix lists/sets on IOS-XR
On Fri, 9 Dec 2022 at 17:30, Tom Beecher wrote: > Pushing thousands of lines via CLI/expect automation is def not a great idea, > no. Putting everything into a file, copying that to the device, and loading > from there is generally best regardless. The slowness you refer to is almost > certainly just because of how XR handles config application. If I'm following > correctly, that seems to be the crux of your question. If you read carefully, that is what Steffann is doing. He is doing 'load location:file' + 'commit'. He is not punching anything by hand. So the answer we are looking for is how to make that go faster. In Junos answer would be 'ephemeral config', but in IOS-XR as far as I know, the only thing you can do is improve the 'load' part by moving the server closer, other than that, you get what you get. -- ++ytti
Re: Large prefix lists/sets on IOS-XR
Pushing thousands of lines via CLI/expect automation is def not a great idea, no. Putting everything into a file, copying that to the device, and loading from there is generally best regardless. The slowness you refer to is almost certainly just because of how XR handles config application. If I'm following correctly, that seems to be the crux of your question. On Thu, Dec 8, 2022 at 6:04 PM Sander Steffann wrote: > Hi, > > What is the best/most efficient/most convenient way to push large prefix > lists or sets to an XR router for BGP prefix filtering? Pushing thousands > of lines through the CLI seems foolish, I tried using the load command but > it seems horribly slow. What am I missing? :) > > Cheers! > Sander > > --- > for every complex problem, there’s a solution that is simple, neat, and > wrong >
Re: Large prefix lists/sets on IOS-XR
On Fri, 9 Dec 2022 at 17:07, Joshua Miller wrote: > I don't know that Netconf or gRPC are any faster than loading cli. Those > protocols facilitate automation so that the time it takes to load any one > device is not a significant factor, especially when you can roll out changes > to devices in parallel. Also, it's easier to build the changes into a > structured format than assemble the right syntax to interact with the CLI. As a programmer I don't really find output format to be significant cost. If I have source of data how I emit them out doesn't matter much. I accept preferences that people have, but don't think it to be important part of solution. Adrian mentioned paramiko, and if we imagine paramiko logging into IOS-XR, and doing 'load http://...' + 'commit'. We've automated the task. Depending on your platform netconf/yang/grpc can be asset or liability, I put IOS-XR strongly in the liability part, because they don't have proper infrastructure that is datafirst, they don't have proper module for even handling configurations, but configurations are owned by individual component teams (like tunnel teams owns GRE config and so forth). Contrasting with Juniper, which is datafirst, and even CLI is 2nd class citizen taking formal data from XML RPC. In IOS-XR you will find all kind of gaps, where you can't rely on netconf/yang, which you will then spend cycles to deal with vendor. Compared to people who use the first class citizen approach, CLI format, who are already done. I did not read Steffan as though he'd be punching in anything manually, he wants to make the process itself faster, without any delays introduced by humans. And I have personally nothing to offer him, except put your server closer to the router, so you can deal with the limited TCP window sizes that hurt transfer speed. -- ++ytti
Re: Large prefix lists/sets on IOS-XR
Hi Saku. I don't know that Netconf or gRPC are any faster than loading cli. Those protocols facilitate automation so that the time it takes to load any one device is not a significant factor, especially when you can roll out changes to devices in parallel. Also, it's easier to build the changes into a structured format than assemble the right syntax to interact with the CLI. On Fri, Dec 9, 2022, 09:38 Saku Ytti wrote: > Can Andrian and Joshua explain what they specifically mean, and how > they expect it to perform over what Steffann is already doing (e.g. > load https://nms/cfg/router.txt)? How much faster will it be, and why? > > Can Steffan explain how large a file they are copying, over what > protocol, how long does it take, and how long does the commit take. > > We used to have configurations in excess of a million lines before > 'or-longer' halved them, and we've seen much longer times than 30min > to get a new config pushed+commtited. We use FTP and while the FTP > does take its sweet time, the commit itself is very long as well. > > I refrain from expressing my disillusionment with the utility of doing > IRR based filtering. > > On Fri, 9 Dec 2022 at 15:38, Andrian Visnevschi via NANOG > wrote: > > > > Two options: > > - gRPC > > - Netconf > > > > You can use tools like paramiko,netmiko or napalm that are widely used > to programmatically configure and manage your XR router. > > > > > > On Fri, Dec 9, 2022 at 2:24 AM Joshua Miller wrote: > >> > >> Netconf is really nice for atomic changes to network devices, though it > would still take some time for the device to process such a large change. > >> > >> On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann > wrote: > >>> > >>> Hi, > >>> > >>> What is the best/most efficient/most convenient way to push large > prefix lists or sets to an XR router for BGP prefix filtering? Pushing > thousands of lines through the CLI seems foolish, I tried using the load > command but it seems horribly slow. What am I missing? :) > >>> > >>> Cheers! > >>> Sander > >>> > >>> --- > >>> for every complex problem, there’s a solution that is simple, neat, > and wrong > > > > > > > > -- > > > > Cheers, > > > > Andrian Visnevschi > > > > > > > -- > ++ytti >
Re: Large prefix lists/sets on IOS-XR
Can Andrian and Joshua explain what they specifically mean, and how they expect it to perform over what Steffann is already doing (e.g. load https://nms/cfg/router.txt)? How much faster will it be, and why? Can Steffan explain how large a file they are copying, over what protocol, how long does it take, and how long does the commit take. We used to have configurations in excess of a million lines before 'or-longer' halved them, and we've seen much longer times than 30min to get a new config pushed+commtited. We use FTP and while the FTP does take its sweet time, the commit itself is very long as well. I refrain from expressing my disillusionment with the utility of doing IRR based filtering. On Fri, 9 Dec 2022 at 15:38, Andrian Visnevschi via NANOG wrote: > > Two options: > - gRPC > - Netconf > > You can use tools like paramiko,netmiko or napalm that are widely used to > programmatically configure and manage your XR router. > > > On Fri, Dec 9, 2022 at 2:24 AM Joshua Miller wrote: >> >> Netconf is really nice for atomic changes to network devices, though it >> would still take some time for the device to process such a large change. >> >> On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann wrote: >>> >>> Hi, >>> >>> What is the best/most efficient/most convenient way to push large prefix >>> lists or sets to an XR router for BGP prefix filtering? Pushing thousands >>> of lines through the CLI seems foolish, I tried using the load command but >>> it seems horribly slow. What am I missing? :) >>> >>> Cheers! >>> Sander >>> >>> --- >>> for every complex problem, there’s a solution that is simple, neat, and >>> wrong > > > > -- > > Cheers, > > Andrian Visnevschi > > -- ++ytti
Re: Large prefix lists/sets on IOS-XR
Two options: - gRPC - Netconf You can use tools like paramiko,netmiko or napalm that are widely used to programmatically configure and manage your XR router. On Fri, Dec 9, 2022 at 2:24 AM Joshua Miller wrote: > Netconf is really nice for atomic changes to network devices, though it > would still take some time for the device to process such a large change. > > On Thu, Dec 8, 2022 at 6:05 PM Sander Steffann wrote: > >> Hi, >> >> What is the best/most efficient/most convenient way to push large prefix >> lists or sets to an XR router for BGP prefix filtering? Pushing thousands >> of lines through the CLI seems foolish, I tried using the load command but >> it seems horribly slow. What am I missing? :) >> >> Cheers! >> Sander >> >> --- >> for every complex problem, there’s a solution that is simple, neat, and >> wrong >> > -- *Cheers,* *Andrian Visnevschi*
