All the solution I was talking about does is select the “best” path egress from a multihomed ASN.
It just inserts a smaller prefix into the local routing table. If people outside of your ASN are seeing that then you’ve already lost the game. From: NANOG <[email protected]> On Behalf Of Tom Beecher Sent: Thursday, December 5, 2024 8:25 PM To: Ryan Hamel <[email protected]> Cc: [email protected] Subject: Re: Route optimization using GPUs? Real life example of an optimizer issue I helped a friend sort out many years ago. Prefix 1 : 10.0.0.0/16<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.0_16&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=RStntx3huOGWR0MfVGqNm4ctlvkpfHbDOrHMoN4BzsU&e=>, Communities A B C Prefix 2 : 10.0.0.0/20<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.0_20&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=hOSdLi-hYquxEayWsZIgvjIPV0glySY5_NWlk0Q0jJ0&e=>, Communities B D E Optimizer created /23s inside the /20. What communities do you think it applied to the new routes? - A B C - B D E - Random assortment of 3-6 values If you guessed random assortment, you're a winner! The optimizer got confused with 2 source prefixes covering what it was trying to generate, and every time it re-optimized it did something different with the communities. This is what I am talking about, that you can do everything "right" with respect to your policies and protections, and the optimizer can do something completely off book and cause an issue. On Thu, Dec 5, 2024 at 6:45 PM Ryan Hamel <[email protected]<mailto:[email protected]>> wrote: Tom, What does "these devices don't follow standard BGP behaviors" have to do with adding a NO_EXPORT or specific community on the import policy when a route is accepted, and being belt & suspenders with matching those communities to drop those routes on export to carriers/IX/PNI sessions? Ryan Hamel ________________________________ From: Tom Beecher <[email protected]<mailto:[email protected]>> Sent: Thursday, December 5, 2024 3:05 PM To: Ryan Hamel <[email protected]<mailto:[email protected]>> Cc: Warren Kumari <[email protected]<mailto:[email protected]>>; Mike Hammett <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits Nobody is asserting that operators are still not responsible for doing this. Stop it. What I think some of you guys fail to understand is that you can have perfectly appropriate policy protections in place that the prefixes generated by the optimizer can still bypass, because these devices don't follow standard BGP behaviors. This has happened before. On Thu, Dec 5, 2024 at 5:37 PM Ryan Hamel <[email protected]<mailto:[email protected]>> wrote: Warren, * "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… While that is also a factually true statement, you are also painting broad strokes over those who are responsible with those weapons. Employment requirements, hunting season, target practice at a range, skeet shooting, are just a few reasons to have them. Let's not dismiss those who follow the law, get qualified on a regular basis or have adequate training when/where/why/how to properly use them. "One rotten apple spoils the whole bunch", does not work here. This same thing also applies to operators of route optimizers. They are responsible for writing the correct import/export policies for their network, just like the carriers for writing sane policies for customer circuits. For the incidents those route optimizers have caused, the vendors, their customers, and upstream ISPs are still in business. Ryan Hamel ________________________________ From: NANOG <[email protected]<mailto:[email protected]>> on behalf of Warren Kumari <[email protected]<mailto:[email protected]>> Sent: Thursday, December 5, 2024 1:17 PM To: Mike Hammett <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: Route optimization using GPUs? Caution: This is an external email and may be malicious. Please take care when clicking links or opening attachments. On Thu, Dec 05, 2024 at 3:41 PM, Mike Hammett <[email protected]<mailto:[email protected]>> wrote: *shrugs* Incorrectly assigning the blame doesn't really help anyone. Sure, but the fact remains that there is blame to be assigned. It doesn't really matter to the affected network if the fault lies with the box itself, or the operator of the box, or the person who makes the morning coffee for the person who operates the box — the fact still remains that this call of devices have caused significant disruption for a whole bunch of external networks. "Guns don't kill people, people with guns kill people" may be a factually true statement - but if there were no guns, there would be less people being shot… "Route optimizers don't hijack routes, operators with route optimizers hijack routes" falls into the same category… W ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=kZw8AOOpxu2KBSVPWZMH9dBV1WmXZ3NnkPRfOhtWhww&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=ZYL7eHsxMHnJOsOQBuq9jw9bXGxRISlqKICkN-sVVyw&e=> ________________________________ From: "Tom Beecher" <[email protected]<mailto:[email protected]>> To: "Mike Hammett" <[email protected]<mailto:[email protected]>> Cc: "Rich Compton" <[email protected]<mailto:[email protected]>>, [email protected]<mailto:[email protected]> Sent: Thursday, December 5, 2024 2:39:52 PM Subject: Re: Route optimization using GPUs? I think most of the hatred towards them is unwarranted, This is essentially saying "I've never had a problem , so I don't think it's a big deal." On Thu, Dec 5, 2024 at 3:19 PM Mike Hammett <[email protected]<mailto:[email protected]>> wrote: Eh, different people have different opinions. I think most of the hatred towards them is unwarranted, ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ics-2Dil.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=kZw8AOOpxu2KBSVPWZMH9dBV1WmXZ3NnkPRfOhtWhww&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.midwest-2Dix.com_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=ZYL7eHsxMHnJOsOQBuq9jw9bXGxRISlqKICkN-sVVyw&e=> ________________________________ From: "Rich Compton" <[email protected]<mailto:[email protected]>> To: "Mike Hammett" <[email protected]<mailto:[email protected]>>, "Jason Bothe" <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Sent: Thursday, December 5, 2024 2:10:47 PM Subject: Re: Route optimization using GPUs? “I strongly recommend to turn off those BGP optimizers, glue the ports shut, burn the hardware, and salt the grounds on which the BGP optimizer sales people walked.” -Job Snijders https://mailman.nanog.org/pipermail/nanog/2017-August/092131.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__mailman.nanog.org_pipermail_nanog_2017-2DAugust_092131.html&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=zviuljLAPm-GL2Wop7Jqa4Y6JzBL2vdR520fbfopdeI&e=> From: NANOG <[email protected]<mailto:[email protected]>> on behalf of Mike Hammett <[email protected]<mailto:[email protected]>> Date: Thursday, December 5, 2024 at 12:24 PM To: Jason Bothe <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: Route optimization using GPUs? IIRC, the widespread outages are the result of exporting things that shouldn't be exported. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.ics-2Dil.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEBil7VttQ-24&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=nKeCefBjXxEbc2TrByVa3_-77wENFQR7xGBVYL-kps4&e=> Midwest-IX http://www.midwest-ix.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.com_v3_-5F-5Fhttp-3A_www.midwest-2Dix.com-5F-5F-3B-21-21CQl3mcHX2A-21GFUnuA6qE7A0AePY8rTtMX8Ll2VP7FK-2DFJTDkvkdwhlRmzqhmEA7Dat58fBwUk9jdLbHKhdSYEAsGopkmw-24&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=cn6Jkzobn304uaLihcKTH2JhoXrgevpg8UhgPz3VRXmGPCDHjHlazYiu-5X7DDtL&s=hmpG3hyUYYU58U5AtL1EY2EH_OhpLV05Qvc288aV6zc&e=> ________________________________ From: "Jason Bothe via NANOG" <[email protected]<mailto:[email protected]>> To: "Drew Weaver" <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Sent: Thursday, December 5, 2024 11:03:39 AM Subject: Re: Route optimization using GPUs? WIth merchant silicon getting faster and stronger everyday, and capacity and transit in a freewill, I’m not sure what GPU optimization would buy you, not to mention the ROI. The Internet routing table is not showing substantial signs of growth and in some cases has experienced a plateau. Also, the experience with ‘route optimization tools’ is that while they may bring you some priority in your traffic, they are also known for making horrible decisions resulting in widespread outages. J~ On Dec 5, 2024, at 8:13 AM, Drew Weaver <[email protected]<mailto:[email protected]>> wrote: So back in the.. hell I don’t know like… early 2010s there was a push for ‘route optimization’ from products like RouteScience and the Avaya CNA and more recently whatever Noction is doing. The big pain point for this technology at the time was that it could only optimize the top N egress routes due to how many probes it could send out and how many results it could process. It seems like now with a modest GPU in a router you could pretty easily ‘optimize’ [to the extent that you believe this technology worked] pretty much the whole routing table. We used these tools extensively back then and they actually worked pretty well in most cases. The biggest issue we ran into was people complaining that we pinged their IP addresses… which now a days seems like a great worst problem to have. Anyway is anyone doing any work on implementing GPUs into the BGP decision making process? Seems like a no brainer. -Drew
