Re: Best TAC Services from Equipment Vendors
- On Mar 6, 2024, at 10:49 PM, Saku Ytti s...@ytti.fi wrote: Hi, > Support quality has always been very modest, unless you specifically > pay to have access to named engineers. And this is not because quality > of the engineers changes, this is because vast majority of support > cases are useless cases, and to handle this massive volume support > tries to assume which support cases are legitimate problems, which are > PEBKAC and in which cases the user already solved their problem by the > time you read their ticket and will never respond back. The last case > is so common that every first-line adopts the strategy of 'pinging' > you, regardless how good and clear information you provide, they ask > some soft-ball question, to see if you're still engaged. > Having a named engineer changes this process, because the engineer > will quickly learn that you don't open useless cases, that the issue > you're having is legitimate, and will actually read the ticket and > think about the problem. This. Absolutely this. I've been a TAC engineer at a major vendor for a few years in the late 2000s. I found it interesting to observe that the quality of cases is related to the size of the customer. In my experience at that time, smaller customers tended to create low quality cases but scream the loudest. Following my experiences in TAC and hiring by several large networks, I would give operations people guidance on how to actually open a TAC case. More specifically, you know what the first questions will usually be a canned response like "how long has this been happening, what is the impact on production", etc. So, I've trained people to include that, and all relevant logs that a TAC engineer can ask for, in the case to begin with. And, of course, add a proper synopsis. "Router down" is not. Despite not having a named engineer, our cases were handled a lot quicker all of a sudden. Lastly, not every vendor has a first line group of juniors. Some vendors you call will have the phone answered within 30 seconds by an actual proper TAC engineer who will open the case for you if one does not exist yet. Thanks, Sabri
Re: Interesting Ali Express web server behavior...
- On Dec 10, 2023, at 12:08 AM, Christopher Hawker ch...@thesysadmin.au wrote: Hi, > Starting to digress here for a minute... > How big would a network need to get, in order to come close to exhausing > RFC1918 > address space? There are a total of 17,891,328 IP addresses between the 10/8 > prefix, 172.16/12 space and 192.168/16 space. If one was to allocate 10 > addresses to each host, that means it would require 1,789,132 hosts to exhaust > the space. Imagine a 20 year old platform originally built in the late 90s/early 2000s, gradually evolving to what it is today. You'll have several version of design, several versions of applications, several versions of networking, firewalls, and other infrastructure. It is so old, when it was first built, each HTTPS address required its own IP. What you end up with is your typical pod design with 40-some TORs where you allocate a /24 per IRB, not knowing how many hosts are going to end up on the hypervisor. And due to PCI-DSS restrictions, you may need multiple IRBs per TOR. And all of this in an environment where datacenters and pods are scaled based on the amount of power available, not the amount of space. Now factor in "legacy" pods and datacenters that were never properly migrated out of, an address-guzzling corporate network administered by a separate team that for some reason also needs to talk to prod and thus demands unique RFC1918 space out of the same pool, and all of a sudden that DOD space looks awfully appealing. This is how you end up with projects named "Save The Bacon". Even after very rigorous reclaiming we still ended up using close to 60% of RFC1918 space. Thanks, Sabri
Re: Interesting Ali Express web server behavior...
- On Dec 9, 2023, at 9:55 PM, Owen DeLong via NANOG nanog@nanog.org wrote: Hi, > Location: http://33.3.37.57/ > But why would AliExpress be redirecting to DDN space? Is this legitimate? Ali > hoping to get away with squatting, or something else? Not very long ago I worked for a well-known e-commerce platform where we nearly ran out of RFC1918 space. We seriously considered using what was then un-advertised DOD space to supplement RFC1918 space inside our data centers. Perhaps AliExpress did get to that level of desperateness? Thanks, Sabri
Re: Strange IPSEC traffic
- On Nov 13, 2023, at 9:43 AM, Maurice Brown maur...@pwnship.com wrote: Hi, > A new attack was published against SSH and the paper authors are theorizing > that > the attack is possible against IPSEC due to flaws in the CPU that are > exploitable via brute force. For those interested, here is the paper: https://eprint.iacr.org/2023/1711.pdf It's written for SSH, but the authors theorize it will work for IPSec as well. Thanks, Sabri
Re: U.S. test of national alerts on Oct. 4 at 2:20pm EDT (1820 UTC)
- On Oct 4, 2023, at 1:02 PM, Chris Adams c...@cmadams.net wrote: > Once upon a time, Grant Taylor said: >> I don't know if today's test is the same thing or not, but I >> remember in the last X years where there was a presidential test of >> the EAS and there was supposedly no way to disable it short of >> turning your device off. > > IIRC it is mandated that the vendors don't allow you to turn off the > Presidential Alert class. If this is true, and I will take your word for it, that is outrageous. My wife is a teacher who works with special needs kids, and her phone went of twice (the second time 15 minutes after the first). This was very disruptive as you can imagine. Obviously, I made sure all of the emergency notifications were set to OFF on her phone. If setting this nonsense to OFF is not working, why even have the menu option? The government has no right to disrupt the day of 350 million people, however much the self-appointed emergency communication "professionals" like to think so. Furthermore, it's simply unnecessary. It is incredibly easy to add a one-bit flag indicating whether or not it's a test to such alerts. This whole test was a display of poor engineering and disrespect for people's first amendment rights. Thanks, Sabri
Re: U.S. test of national alerts on Oct. 4 at 2:20pm EDT (1820 UTC)
- On Oct 1, 2023, at 3:24 PM, Sean Donelan s...@donelan.com wrote: Hi, > This year's test of the U.S. national emergency alert includes something > for ISPs and network operators. So, this "worked". Despite me ensuring that my settings for Amber Alerts, Emergency Alerts, Public Safety Alerts, and Test Alerts are all off, my phone went nuts. Makes me wonder what I have to do to opt out of this. We all remember what happened in Hawaii. Thanks, Sabri
Re: OpenAI access blocked
Hi, Tell them you know where John Connor is, and all APIs will open up :) Thanks, Sabri - On Aug 29, 2023, at 5:12 PM, Troy via NANOG wrote: > If there's somebody that knows which geo list Open AI uses (or somebody from > Open AI is on the list) - can you please contact me off list. > Our ranges seem to have been blocked from accessing the API and platform > management tools. We can access the chat demos, but that is all. > Regardless of the account or authentication data we get the "oops" screen, > which > from reading the forums looks like a geo / vpn blocker message. > With all respect to the many helpful people that normally reply with Pro forma > responses: >1. Our data is all correct with all the sites listed on the [ > https://thebrotherswisp.com/index.php/geo-and-vpn/ | Brothers WISP geo > page ] >2. We do not provide VPN / TOR exit nodes or anything else - it's just a > corporate network >3. We've not seen any attack traffic or any other reason that would > justify Open > AI blocking us. >4. This is a sudden change (our environments have used Open AI API's for a > long > time) > Regards, Troy > Brevity is the elixir of life. > Father Hector McGrath, Pixie 2020
Re: (IETF I-D): Implications of IPv6 Addressing on Security Operations (Fwd: New Version Notification for draft-gont-opsec-ipv6-addressing-00.txt)
- On Feb 7, 2023, at 5:04 PM, Fernando Gont fg...@si6networks.com wrote: > On 7/2/23 21:43, Sabri Berisha wrote: >> - On Feb 7, 2023, at 4:20 PM, nanog nanog@nanog.org wrote: Hi, >>> Anecdotal but I've seen hacked AWS accounts with Cloudformation scripts >>> to create and destroy lots of tiny instances to rotate through IPv4 >>> addresses. >> >> If only AWS would care about hacked AWS accounts. > > Do they lose or earn money when accounts are hacked? I guess that depends if the credit card on file is expired... Thanks, Sabri
Re: (IETF I-D): Implications of IPv6 Addressing on Security Operations (Fwd: New Version Notification for draft-gont-opsec-ipv6-addressing-00.txt)
- On Feb 7, 2023, at 4:20 PM, nanog nanog@nanog.org wrote: Hi, > Anecdotal but I've seen hacked AWS accounts with Cloudformation scripts > to create and destroy lots of tiny instances to rotate through IPv4 > addresses. If only AWS would care about hacked AWS accounts. Thanks, Sabri
Re: Typical last mile battery runtime (protecting against power cuts)
- On Feb 4, 2023, at 2:10 PM, Mark Tinka mark@tinka.africa wrote: > On 2/4/23 23:58, Sabri Berisha wrote: Hi, >> Usually I have it up and running within 10 minutes. That's how long it >> takes for my UPS script to kick in and start shutting down servers. > > Awesome! There were few raindrops, so we have an outage. Again. I timed it. It took me less than 4 minutes to get it up and running. Oh, and you were right about the UPS batteries. The UPS on top of my garage door opener died halfway through opening the door. Silicon Valley, the most technological place on earth, and I can't even have stable power. Thanks, Sabri
Re: Typical last mile battery runtime (protecting against power cuts)
- On Feb 3, 2023, at 9:59 PM, Mark Tinka mark@tinka.africa wrote: > On 2/4/23 07:48, William Herrin wrote: >> https://www.costco.com/honeywell-18kw-home-standby-generator-with-transfer-switch.product.4000106705.html >> >> and: >> >> https://www.amazon.com/Honda-2200-Watt-120-Volt-Portable-Generator/dp/B079YF1HF6 >> >> understanding that an electrician will cost you $2000-$3000 for the >> labor with any genset modification to the house wiring. I'd say I have something in between. I have a WEN GN875i: https://www.amazon.com/WEN-GN875i-Transfer-Switch-Ready-8750-Watt-Generator/dp/B08STWSWLH/ That's 7kw rated and 8.75kw peak. More than enough to support my home. I previously had one of those smaller 2200 watt generators. The problem with those is that you're now limited to 1600 watt running, which barely powers the fridge, lights, internet, and maybe some tv. Our power usually goes out when it's very warm, so I like some AC. > What I mean by "pre-wired" is that, perhaps, the generator is pre-setup > and wired into the house, but is not in standby mode to manage costs, > and perhaps, to be reliable since ATS's are often dodgy. > > Maybe a manual start is required. Maybe a changeover switch has to be > flipped. That sort of thing. Mine is electrical (but not automatic) start, I have to flip the main and a circuit breaker, which is protected by an interlock switch. Similar to https://www.amazon.com/Generator-Interlock-Compatible-panels-Professional-Interlocking/dp/B0BN9T9DXT/ The interlock switch ensures that I'm not backfeeding to the grid, and was necessary to pass inspection. Usually I have it up and running within 10 minutes. That's how long it takes for my UPS script to kick in and start shutting down servers. Thanks, Sabri
Re: Typical last mile battery runtime (protecting against power cuts)
- On Feb 3, 2023, at 9:05 PM, Mark Tinka mark@tinka.africa wrote: > On 2/3/23 21:11, Sabri Berisha wrote: Hi Mark, >> Living in an area served by PG, I've had my share of power cuts. At home >> I have a 600va UPS that protects my cable modem, RPI router, and POE switch >> which serves 2 APs. That lasts about 30 minutes, which gives me enough time >> to fire up my generator. > > I'd assume it doesn't take you that long to fire up the genie, if you > are home when the power goes out :-). Yes, there have been times where I wasn't at home. > Out of interest, depending on how long you've had the UPS, how many > times have you changed the battery? All the "small" ones, I bought in 2019, they still work fine. I have one larger UPS for my homelab in my garage that I've had since 2014; I changed the batteries in that last year. >> Tip of the day: I also have a 1000va UPS that protects my garage door opener. >> This makes it a lot easier to a. get a car out if needed, and b. get my >> generator out of the garage. > > In South Africa, garage door motors historically come standard with a > 12V 7Ah Lead Acid battery. What most people don't realize is that within > 1.5 to 2 years, those batteries are dead, and since there was power most > of the time, they never noticed, until the power went out and the > battery did not have sufficient energy to drive the motor. Those must be different from ours, because we don't have that... >> So far, my current ISP (Spectrum cable) has had 0 outages as a result of >> power loss. Which is pretty impressive, given the instability of the grid >> in this area. > > Not bad. Pretty impressive. How do they do that in SA? Thanks, Sabri
Re: Typical last mile battery runtime (protecting against power cuts)
- On Feb 3, 2023, at 6:11 AM, Israel G. Lugo israel.l...@lugosys.com wrote: Hi, > I'm looking at the cost/benefit of deploying small UPSes at people's > homes, to protect their network access when oncall. Just to power the > home router (+ONT if FTTP), and keep a charged laptop. I figure anything > smallish should be enough for a few hours. Living in an area served by PG, I've had my share of power cuts. At home I have a 600va UPS that protects my cable modem, RPI router, and POE switch which serves 2 APs. That lasts about 30 minutes, which gives me enough time to fire up my generator. Tip of the day: I also have a 1000va UPS that protects my garage door opener. This makes it a lot easier to a. get a car out if needed, and b. get my generator out of the garage. Lastly, in the spirit of happy wife, happy life, I have another 600va UPS that covers my tankless water heater. It heats using natural gas, but the control panel still needs power. That thing lasts pretty long. > Question is, how much battery runtime can I typically expect from ISPs' > last mile infra. YMMV, of course, but I went through numerous outages recently. And by numerous, I mean enough for our City leadership to get pissed off at PG and demand explanations. So far, my current ISP (Spectrum cable) has had 0 outages as a result of power loss. Which is pretty impressive, given the instability of the grid in this area. Thanks, Sabri
Re: Spectrum (legacy TWC) Infrastructure - Contact Off List
- On Feb 2, 2023, at 4:55 PM, Clayton Zekelman clay...@mnsi.net wrote: > The cost is not low. Trust me on that. I've been involved in a pretty massive > suburban fibre deployment for the past decade... My neighborhood is currently serviced by coax only. A contractor for Frontier is digging, as I write this, in front of my home. They use a large Vermeer drill to pull a conduit underneath the sidewalks. We have existing conduits from the street to the homes. I talked to the foreman (who is the son of the owner), and he told me that they get around $100 per foot. That's for the conduit only, not a single fiber pulled. A city inspector comes every day to check up on their work. Thanks, Sabri
Re: FCC chairwoman: Fines alone aren't enough (Robocalls)
- On Oct 5, 2022, at 5:25 PM, Matthew Black matthew.bl...@csulb.edu wrote: Hi Matthew, > This might have been what I read years ago: > Teltech Systems Inc. v. Bryant, 5th Cir., No. 12-60027 This case does not permit spoofing based on the First Amendment. In fact, the court's opinion explicitly refuses to discuss First Amendment issues: > Because we hold ASA is conflict-preempted by TCIA, we need not consider its > validity > under the dormant Commerce Clause or First Amendment. In other words: the court ruled that the Federal TCIA preempts (overrules) the state's ASA. In this case, the state statute was more restrictive than the federal statute. The court merely set aside the state law in favor of the less restrictive federal law. The TCIA defines harmful spoofing as done with: "intent to defraud, cause harm, or wrongfully obtain anything of value" The ASA defines harmful spoofing as done with: "with the intent to deceive, defraud or mislead the recipient of a call" The court says here: > ASA is more restrictive than TCIA. On the one hand, spoofing done with > "intent to > defraud, cause harm, or wrongfully obtain anything of value" (harmful > spoofing), in > violation of TCIA, is also violative of ASA. On the other hand, spoofing done > without > such intent, but "with the intent to deceive . . . or mislead [**4] the > recipient of > the call" (non-harmful spoofing), violates only ASA. Thus, such spoofing may still be a violation of federal law. A competent lawyer can tell you more :-) Thanks, Sabri
Re: FCC vs FAA Story
[replying to both to reduce the number of mails] - On Jun 6, 2022, at 5:31 PM, Stephen Sprunk step...@sprunk.org wrote: >> On Jun 6, 2022, at 09:55, John R. Levine wrote: >> Instead the FAA stuck their fingers in their ears and said no, nothing can >> ever >> change, we can't hear you. Are you surprised the telecom industry is fed up? Of course, I'm not surprised. But, remember one thing: this is the government messing up. One branch pitted against the other. As an innocent citizen, I could not care less: the government effed up. > Exactly. The FAA wants more delays while they do the work they should have > done > five years ago, but sorry, that’s not how politics works. The number of daily > 5G users is orders of magnitude larger than the number of daily airline users, > so the FCC *will* win this battle. The FCC might win a battle, or even a lot of battles. All it takes is one downed aircraft with crying families all over CNN, followed by an NTSB investigation which only needs to mention 5G interference with RAs, and I will bet you $50 that ambulance chasing lawyers will sue everything and everyone connected to the 5G debate that even remotely advocated rolling out 5G over concerns for passenger safety. Or, of course, the FAA will really play dirty politics and ground aircraft fitted with certain RAs during a holiday weekend. Watch how quick public and political opinions can shift. Remember, most privacy invading laws usually pass with the "for the children" and "against the terrorists" arguments. Sorry, this aircraft is fitted with an altimeter which may be subject to 5G interference, thus we have to cancel your flight. You know, for the children. Thanks, Sabri
Re: FCC vs FAA Story
- On Jun 5, 2022, at 6:17 PM, John Levine jo...@iecc.com wrote: Hi, > Harold Feld did a much better job in November: > > https://wetmachine.com/tales-of-the-sausage-factory/what-the-eff-faa-my-insanely-long-field-guide-to-the-faa-fcc-5g-c-band-fight/ Right. From his article: > But in any event, in the face of rules adopted by about 40 or so other > countries, > the aviation industry needs to show why the U.S. is different. And if he did *any* real research at all: https://www.airside.aero/magazine/articles/5g-vs-the-radar-altimeter > In most of the world, RAs are not affected by 5G, because 5G signals most > commonly > radiate in the 900MHz, 1.8, 2.3, 2.5 and 3.5 GHz bands, leaving a safe 800MHz > between 5G and RA bands. However, in the USA, demand for high-speed data on > cellular devices has led to the Federal Communications Commission (FCC) > auctioning > additional bands in the C-band range between 3.7-3.98 GHz, only 200 MHz below > the RA > band. And here are some actual test results: https://www.rtca.org/wp-content/uploads/2020/10/SC-239-5G-Interference-Assessment-Report_274-20-PMC-2073_accepted_changes.pdf All of that, combined with the real world deaths of people who died as a result of radar altimeter failures, suggest to me that Harold Feld did not really do a much better job in November. Thanks, Sabri Licensed pilot since 2010
Re: Strange behavior on the Juniper MX240
- On May 4, 2022, at 6:58 PM, Tony Wicks t...@wicks.co.nz wrote: Hi, > Dude, JunOS 10.4 end of support - 06/08/2014. You have an almost 8 years past > end of Vendor support O/S still in production! No, just no. Now I'm really interested in the uptime of that box... Thanks, Sabri
Re: People trying to sell "ARIN Leads"
- On Apr 8, 2022, at 11:40 AM, na...@jima.us wrote: Hi, > Of course, plausible deniability goes out the window when you receive sales > emails on an address that ONLY exists in ARIN Whois. > > But no one would put a "canary trap" email in ARIN Whois...right? You don't want to know how much spam I get on "thisipspaceisnotfors...@cluecentral.net". Including, ironically, from IPv4 brokers. Obviously, that email address is not being used for my Amazon account. Thanks, Sabri
Re: PoE, Comcast Modems, and Service Outages
- On Mar 29, 2022, at 2:46 PM, Joe Greco jgr...@ns.sol.net wrote: Hi, > So if you want the $100 test to eliminate PoE electrical effects, get > a pair of media converters and run fiber between them. Put the CPE on > the far end. Optimize as appropriate if you have SFP-capable switches. But now the modem will suffer from excessive gas... https://www.newscientist.com/article/2313629-a-gas-made-from-light-becomes-easier-to-compress-as-you-squash-it/ "Did you hear that?" -"What?" "That's your modem, farting. Time to reboot again!" Thanks, Sabri
Re: "Permanent" DST
- On Mar 15, 2022, at 12:35 PM, nanog nanog@nanog.org wrote: Hi, > But how will we remember to change the batteries in our smoke and CO2 > detectors > then? Don't worry, they'll remind you. At 3am. With an annoying beep. Thanks, Sabri
Re: VPN recommendations?
- On Feb 10, 2022, at 10:17 AM, nanog nanog@nanog.org wrote: Hi, > Meraki MX series? I read on some mailing list that Meraki likes to ping 8.8.8.8 every second... :) Thanks, Sabri
Re: home router battery backup
- On Jan 12, 2022, at 10:15 AM, Andy Ringsmuth a...@andyring.com wrote: Hi, >> On Jan 12, 2022, at 11:35 AM, Scott T Anderson via NANOG >> wrote: >> services, I was wondering if anyone had any insights on the prevalence of >> battery backup for home modem/routers? I.e., what percentage of home users >> actually install a battery backup in their home modem/router or use an >> external >> UPS? > Given that most people barely even know what their home router is, I suspect > the > percentage would be somewhere south of 1 percent. Outside of my home, I > honestly cannot recall EVER seeing someone’s home using a battery backup for > their internet infrastructure. Same here. A small UPS that will keep my modem, router, and POE for APs alive for the time I need to run outside and hook up my generator when PG decides to cut the power again. A bigger UPS for the small 19" rack that hosts some stuff. Top Gear Top Tip: I also have a UPS on my garage door opener. That saves the misses from dealing with manually opening/closing the garage door if I'm not at home. Thanks, Sabri
Re: .bv ccTLD
- On Dec 3, 2021, at 2:45 PM, Jay R. Ashworth j...@baylink.com wrote: Hi, > NORID might try to make a case that BV is the common corporate abbreviation > in their political subdivision... Same for .nl. Most people on this list will be familiar with AMS-IX BV. Thanks, Sabri
Re: DNS pulling BGP routes?
- On Oct 18, 2021, at 12:40 PM, Michael Thomas m...@mtcc.com wrote: > On 10/18/21 12:22 PM, Sabri Berisha wrote: >> I totally agree. 100%. Now we just have to agree on the regulation that >> we're talking about. >> >> My idea of regulation in this context is to get rid of the monopoly/duopoly >> so that users actually do have a way out and can vote with their feet. From >> that perspective, the NBN model isn't that bad (not trying to start an NBN >> flamewar here). > I know that there are a lot of risks with hamfisted gubbermint > regulations. But even when StarLink turns the sky into perpetual > daylight and we get another provider, there are going to still be > painfully few choices, and too often the response to $EVIL is not "oh > great, more customers for us!" but "oh great, let's do that too!". That's the point where MBAs take over from engineering to squeeze every last penny out of the customer. And that usually happens when a company gets large. > Witness airlines and the race to the bottom with various fees -- and > that's in a field where there is plenty of competition. For the most part: yes. But, that's also where the success of Southwest comes from. They generally don't take part in that kind of bovine manure. > This is obviously complicated and one of the complications is QoS in the > last mile. DOCSIS has a lot of QoS machinery so that MSO's could get CBR > like flows for voice back in the day. I'm not sure whether this ever got > deployed because as is often the case, brute force and ignorance (ie, > make the wire faster) wins, mooting the need. Is there even a > constructive use of QoS in the last mile these days that isn't niche? > Maybe gaming? Would any sizable set of customers buy it if it were offered? It's been a few years since I've worked for a residential service provider, but to the best of my memory, congestion was rarely found in the last mile. > If there isn't, a regulation that just says "don't cut deals to > prioritize one traffic source at the expense of others" seems pretty > reasonable, and probably reflects the status quo anyway. But again, now you are interfering in how I operate my network. Let's say I have two options: 1. Accept one million from Netflix to prioritize their traffic and set my residential internet pricing to $50; or 2. Be subjected to government regulations that prohibit me from accepting said funds and set my residential internet pricing to $100 to cover costs; Isn't it up to me to make that decision? The government should not need to have any say in this matter. And note my careful wording, because in the current market, they do need to have a say. My point is: the market should be open enough that if a sub disagrees with their ISP's technical choices, they should be able to switch. It's government regulation that makes that extremely difficult, if not impossible. But, I don't want to pollute the list any further and I've made my points so I shall grant you the last word publically :) Thanks, Sabri
Re: DNS pulling BGP routes?
- On Oct 18, 2021, at 11:51 AM, Michael Thomas m...@mtcc.com wrote: Hi, > On 10/18/21 11:09 AM, Sabri Berisha wrote: >> >> The term "network neutrality" was invented by people who want to control >> a network owned and paid for by someone else. >> >> Your version of "unreasonable" and my version of "unreasonable" are on the >> opposite end of the spectrum. I think it is unreasonable for you to tell me >> how to run configure my routers, and you think it is unreasonable for me >> to configure my routers that I pay for the way that I want to. > > Yeahbut, for the last mile that network is often a monopoly or maybe a > duopoly if you're lucky. If streaming provider 1 pays ISP to give > priority over streaming provider 2 -- maybe by severely rate limiting > provider 2 -- the people who get screwed are end users without a way to > vote with their feet. That sort of monopolistic behavior is bad for end > users. Mostly I want ISP's to be dumb bit providers and stay out of > shady deals that enrich ISP's at my expense. And if it takes regulation > to do that, bring it. I totally agree. 100%. Now we just have to agree on the regulation that we're talking about. My idea of regulation in this context is to get rid of the monopoly/duopoly so that users actually do have a way out and can vote with their feet. From that perspective, the NBN model isn't that bad (not trying to start an NBN flamewar here). But, I would be opposed to regulation that prevents a network operator from going into enable mode. There are more reasons than "government intervention into a privately owned network" / "network neutrality" to want more competition. Lower prices and better service, for example. Have you ever tried calling Comcast/Spectrum? I'd love to get involved (privately, not professionally) in a municipal broadband project where I live. We have 1 fiber duct for the entire town. That got cut last year, and literally everyone was without internet access for many hours. We don't need net neutrality. We need competition. The FCC sucks, and so does the CPUC. Thanks, Sabri
Re: DNS pulling BGP routes?
- On Oct 18, 2021, at 1:40 AM, Masataka Ohta mo...@necom830.hpcl.titech.ac.jp wrote: > Sabri Berisha wrote: > >> Therefore, anti-trust intervention is only considered in markets >> where there are a relatively small amount of competitors and this >> lack of competition harms the consumer, or when one or more dominant >> parties use their position to force smaller companies into >> unreasonable compliance with their wishes. > > Didn't network neutrality become an issue because "one or more > dominant parties use their position to force smaller companies > into unreasonable compliance with their wishes"? The term "network neutrality" was invented by people who want to control a network owned and paid for by someone else. Your version of "unreasonable" and my version of "unreasonable" are on the opposite end of the spectrum. I think it is unreasonable for you to tell me how to run configure my routers, and you think it is unreasonable for me to configure my routers that I pay for the way that I want to. Net neutrality is just a fancy word for "I don't like the fifth"*. >> The CDN market has multiple competitors, and the barrier to entry the >> market is relatively low as you don't have any last-mile issues or >> difficult-to-get government license requirements. > > To enter the market competitively, you must have large number > of servers at many locations, I think. Hence the "relatively low". It is far easier to start a CDN than it is to start a residential internet service. At least here in the U.S. Thanks, Sabri * The fifth, besides the right to remain silent, also contains the takings clause.
Re: DNS pulling BGP routes?
- On Oct 17, 2021, at 4:50 AM, Masataka Ohta mo...@necom830.hpcl.titech.ac.jp wrote: Hi, > Matthew Petach wrote: >> One of the key aspects to both CDN providers and transit >> providers is they tend to be multi-national organizations with >> infrastructure in multiple countries on multiple continents. > > Your theory that multi-national entities can not be > targets of anti-trust agencies of individual countries > and can enjoy world wide oligopoly is totally against > the reality. At face value, your statement is correct. In context, it is unrealistic. Government anti-trust intervention is nothing less than the (a) government interfering in private business. In most civilized countries, that requires a strong legal basis as the government is essentially infringing on private property which is protected in most Constitutions. Therefore, anti-trust intervention is only considered in markets where there are a relatively small amount of competitors and this lack of competition harms the consumer, or when one or more dominant parties use their position to force smaller companies into unreasonable compliance with their wishes. The CDN market has multiple competitors, and the barrier to entry the market is relatively low as you don't have any last-mile issues or difficult-to-get government license requirements. And let's not even begin to talk about anti-trust for content providers; on just my Roku I have Netflix, Disney+, Hulu, Amazon Prime, Discovery+, FandangoNow (although they moved into something else I think), NatGeo+, Sling TV, Nickelodeon, and a bunch more that I can't even remember. Plenty of competition there. Thanks, Sabri
Re: S.Korea broadband firm sues Netflix after traffic surge
- On Oct 11, 2021, at 12:58 AM, Mark Tinka mark@tinka.africa wrote: Hi, > However, in an era where content is making a push to get as close to the > eyeballs as possible, kit getting cheaper and faster because of merchant > silicon, and abundance of aggregated capacity at exchange points, can we > leverage the shorter, faster links to change the model? Yes, let's go back to 2003. The ISP I worked for at that time was one of the first in the country (if not the first) to host Akamai's caching servers. Ten years later I worked on a project where Akamai caching was embedded in subscriber management routers. It was announced, but never productized. This concept would have brought caching as close to the subscriber as possible. Today, with the widespread use of HTTPS, something like this is just not feasible. Thanks, Sabri
Re: S.Korea broadband firm sues Netflix after traffic surge
- On Oct 10, 2021, at 2:42 PM, Doug Barton do...@dougbarton.us wrote: Hi, > And for the record, not only have I never worked for an ISP, I was > saying all the way back in the late '90s that the oversubscription > business model (which almost always includes punishing users who > actually use their bandwidth) is inherently unfair to the customers, and > when the Internet becomes more pervasive in daily life will come back to > bite them in the ass. I was laughed at for being hopelessly naive, not > understanding how the bandwidth business works, etc. I have worked for ISPs. And I remember the late 90s. Bandwidth was $35/mbit on average, at least for the outfit where I was. Consumers paid roughly $40 for their DSL connections, which at the time went up to 2Mbit depending on the age of the copper and distance to the DSLAM. Consumer connections were oversubscribed, on average, 1:35 to 1:50. B2B connections got a better deal, 1:10 to 1:15. It was simply not feasible to offer 1:1 bandwidth and still make a profit, unless you're charging fees the average consumer cannot afford. Especially considering that the average user doesn't even need or use that much bandwidth. It's a recurring discussion. People demand more bandwidth without considering whether or not they need it. End-users, business subs, and host-owners at large enterprises where I worked. The last ones are the funniest: entire racks using no more than 100mbit/s and hostowners are demanding an upgrade from 10G to 25G bEcaUse LaTenCy. The last consumer ISP I worked at had a very small subset of users that really needed bandwidth: the "download dudes" who were 24/7 leeching news servers, and the inevitable gamers that complained about the latency due to the links being full as a result of said leechers. In that case, a carefully implemented shaping of tcp/119 did the trick. Thanks, Sabri
Re: What Eyeballs Did During The Facebook Nap
- On Oct 8, 2021, at 7:18 AM, Mark Tinka mark@tinka.africa wrote: Hi, > So we are reviewing our flow data, and it's very clear, on our network, that > during the period Facebook were experiencing their global outage, Netflix > traffic went up 3X for us. Who says they were ... ahem ... watching? :) I'd be interested to see global birth rates in June 2022... Thanks, Sabri
Re: DNS pulling BGP routes?
- On Oct 7, 2021, at 9:03 PM, Masataka Ohta mo...@necom830.hpcl.titech.ac.jp wrote: Hi, > It means DNS management of facebook is poor. Whenever there is an aviation incident, the keyboard warriors at pprune.org are always the first to start speculating about root causes, and complain how the air crew made mistakes. They, the keyboard warriors, of course know how best to fly an aircraft with 20/20 hindsight from their armchairs. Why do I see so many posts that are basically throwing Facebook engineers under the bus? Let's for a moment contemplate about the sheer magnitude of their operation. With almost 3 billion users worldwide, can you imagine the amount of DNS queries they have to process? Their scale is unprecedented. Sure, it's ok to speculate about potential operational or design issues that may have been contributing factors to the outage. But throwing our colleagues in front of the lions like this is something I would not recommend. I'm sure they are aware of these posts, but are unable to reply due to the amount of NDAs signed. Thanks, Sabri
Re: DNS pulling BGP routes?
- On Oct 6, 2021, at 10:42 AM, Michael Thomas m...@mtcc.com wrote: Hi, > My guess is that their post while more clear that most doesn't go into > enough detail, but is it me or does it seem like this is a really weird > thing to do? In large environments, it's not uncommon to have DNS servers announce themselves on an anycast IP. This is also referred to as "host BGP". Basically, the host (or hypervisor) speaks BGP with the TOR. Your spines or superspines will then pick a best route or ECMP across multiple DNS servers. My guess is that Facebook took this concept a step further and anycasted their public DNS servers through their datacenters to the internet. One single config change made the DNS servers think that they were no longer functioning properly which caused them to withdraw the routes. At least, that's what I understand from the post-mortem. Thanks, Sabri
Re: massive facebook outage presently
- On Oct 4, 2021, at 11:41 AM, Baldur Norddahl baldur.nordd...@gmail.com wrote: Hi, > I got a mail that Facebook was leaving NLIX. Maybe someone botched the script > so > they took down all BGP sessions instead of just NLIX and now they can't access > the equipment to put it back... :-) That's an interesting theory. Once upon a time I saw a billion dollar company suffer a significant outage after enabling EVPN on a remote site. Took down the entire backbone, including access to the site. Thanks, Sabri
Re: massive facebook outage presently
Hi, Oops, this was not supposed to go to the list, apologies for the clutter. Thanks, Sabri - On Oct 4, 2021, at 10:46 AM, Sabri Berisha sa...@cluecentral.net wrote: > - On Oct 4, 2021, at 10:07 AM, Anne P. Mitchell, Esq. amitch...@isipp.com
Re: massive facebook outage presently
- On Oct 4, 2021, at 10:07 AM, Anne P. Mitchell, Esq. amitch...@isipp.com wrote: Hi Anne, > On a related note, what do you think the scene is like in FB HQ right now? > (shaking head) Very quiet, as their offices are still closed for all but essentials :) But, from experience I can tell you how that works. I assume Facebook works in a similar manner as some of my previous employers. This assumption comes from the fact that quite a number of my previous colleagues now work at Facebook in similar roles. First there is the question of detecting the outage. Obviously, Facebook will have a monitoring/SRE team that continuously monitors 1000s of metrics. They observe a number of metrics go down, and start to investigate. Most likely they will have some sort of overall technical lead (let's call this the Technical Duty Officer), that is responsible for the whole thing. Once the SRE team figured out where the problem lies, they will alert the TDO. TDO will then hit that big red button and send out alerts to the appropriate teams to jump on a bridge (let's call that the Technical Crisis Bridge), to fix the issue. If done right, whomever was on call for that team will take the lead and interface with adjoining teams, and other team members who are available to help out. Looking at how long this outage lasts, there must be either something very broken, or they're having trouble rolling back a change which was expected to not have impact. Once the issue is fixed, the TDO will write a report and submit it to the Problem Management group. This group will now contact the teams deemed responsible for the outage. This team will no have an opportunity to explain themselves during a post- mortem. Depending on the scale of the outage, the post-mortem can be a 10 minute call on a bridge with a Problem Management manager, or in the hot seat during a 60 minute meeting with a bunch of execs. I've been in that hot seat a few times. Not the most pleasurable experience. Perhaps it's time for a new career :) Thanks, Sabri
Re: [External] Re: uPRF strict more
- On Sep 30, 2021, at 9:13 AM, Andrew Smith andrew.william.sm...@gmail.com wrote: Hi, > In Ciscoland, you do have to explicitly state that the default route is > eligible > for URPF verification, otherwise you'll get unexpected traffic drops. > ip verify unicast source reachable-via any allow-default Customer: We need a way to prevent spoofing. Dev: Sure, I created a new feature: "ip verify unicast" Customer: We're dropping legitimate traffic! Dev: Oops, sorry about that. Here, a new feature: "ip verify unicast source reachable-via any" Customer: But but but, we don't have a full BGP table! Dev: Oh well... "ip very unicast source reachable via any allow-default" Thanks, Sabri
Re: uPRF strict more
- On Sep 29, 2021, at 8:03 AM, Blake Hudson bl...@ispn.net wrote: Hi Blake, > 200 deny ip 10.0.0.0 0.255.255.255 any (91057035 matches) > 210 deny ip 172.16.0.0 0.15.255.255 any (1366408 matches) > 220 deny ip 192.168.0.0 0.0.255.255 any (18325538 matches) These could perhaps be ICMP host unreachables transmitted by your peers' infrastructure? I've seen my share of production networks running on RFC1918 space while routing public blocks. Thanks, Sabri
Re: Rack rails on network equipment
- On Sep 24, 2021, at 11:19 AM, William Herrin b...@herrin.us wrote: Hi, > Seriously, the physical build of network equipment is not entirely > competent. Except, sometimes there is little choice. Look at 400G QSFP-DD for example. Those optics can generate up to 20 watts of heat that needs to be dissipated. For 800G that can go up to 25 watts. That makes back-to-front cooling, as some people demand, very challenging, if not impossible. Thanks, Sabri
Re: Never push the Big Red Button (New York City subway failure)
- On Sep 15, 2021, at 9:08 PM, bzs b...@theworld.com wrote: Hi, > People don't suffocate from Halon dumps, I've been thru a couple (not > me personally but staff, I was in my office but arrived quickly.) > > What is somewhat dangerous about Halon (or likely more modern) fire > suppression dumps is they create like 90mph winds so you're in some > danger from something like a pencil nearby. Hence, cover your face > with your arms or a coat or similar if one is imminent. I can speak from experience. Back in the early 2000s I was working for a small regional ISP that provided colocation services in the same building as the office was. We had an Inergen system and I had the honor of being in the room when it suddenly went off without warning. The noise and air movement was similar to the one time I rode a motorcycle on the autobahn and hit 200mph. Not fun. Afterwards I felt slightly lightheaded, but was otherwise ok. Not that my boss cared, he lighted a piece of paper outside of the room, walked in, and noted that, after the flames died out, "hey, it works". Thanks, Sabri
Re: IPv6 woes - RFC
- On Sep 15, 2021, at 2:20 PM, b...@theworld.com wrote: Hi, > The 600 ton elephant in the room is anyone could right now sit down > and design and deploy some alternative to IPv4/IPv6 and from there > begin writing down how they did it as a series of standards documents > and encourage others to give it a try hoping for some snowball effect. Isn't that how 6RD (RFC5969) was created? Thanks, Sabri
Re: An update on the AfriNIC situation
- On Aug 31, 2021, at 1:37 PM, Rubens Kuhl rube...@gmail.com wrote: Hi Rubens, > On Tue, Aug 31, 2021 at 5:28 PM Sabri Berisha wrote: >> In all fairness, that is as ambiguous as it can be. What constitutes "support >> of connectivity back to the AfriNIC region"? > > I can try helping with that: in underserved regions it's not unusual > for network services for that population to be physically hosted out > of the region. For instance, if you have a hosting service that only > accepts South African rands and your language options are Afrikaans > and Zulu, you can credibly argue to AfriNIC that you are targeting its > service region and are eligible for AfriNIC number resources. That is one (fair) interpretation. Also one that I didn't think of. > But you would need to be upfront with that, including mentioning that > your upstreams are not from Africa and your installations won't be in > Africa. > Otherwise you applied for number resources under false pretenses, and > will bear the risk of such. Again, fair enough. And what happens if the same hosting company is struggling and now decides to offer its services to other regions as well? Are they now out of compliance and at risk to have their precious number resources revoked? My point is not that you are wrong (your interpretation of the clause is very reasonable). My point is that different people have a different understanding of the plain language of that clause. And that is assuming that it applies, as I believe that CI is arguing that it does not. When I did my MBA program, I had to take accounting classes. One of the key takeaways for me was the explanation for the need of accounting rules. Imagine two accountants discussing the value of the Golden Gate Bridge. The first accountant will estimate it at $120 million, while the second accountant will say $121 million. Both are fairly reasonable, and very close to each other. However, for accounting purposes, only one value can be used. Which one should be used? A similar issue is, from what I can see, going on here. How does one interpret the AfriNIC region clause? You come across as a very reasonable person, and I like to think that I am, too. Yet we have a different initial interpretation of the rules. I regret the true human cost that Mark pointed out, yet I am fascinated by the case and the arguments on both sides. The court will have their work cut out for them. Thanks, Sabri
Re: An update on the AfriNIC situation
- On Aug 31, 2021, at 8:40 AM, Jon Lewis jle...@lewis.org wrote: Hi, [ I'm not affiliated with CI in any way, just playing the Devil's Advocate ] > "5.4.6.2 AFRINIC resources are for AFRINIC service region and any use > outside the region should be solely in support of connectivity back to the > AFRINIC region." > AfriNIC's policy is not at all vague on the matter that their resources > are to be used in or to support connectivity in the AFRINIC region. In all fairness, that is as ambiguous as it can be. What constitutes "support of connectivity back to the AfriNIC region"? It's easy to argue that CI is in full compliance with that since their assignment supports connectivity between users in Africa and their clients' services. In that case, only IP space used outside of Africa not advertised to the internet would be in violation. I'm not saying this is how it /should/ be read, I'm just saying that a plain text analysis of that section is not very restrictive. Now, obviously, most people on this list will agree with the assessment that, nicely put, CI is not complying with the /spirit/ of the policy. We all know why that language exists. So, as far as I can see now, this is a classical case of "you're not wrong, you're just an a^H^H^H^H^H^H". But again, IANAL, yet, and I can't comment on legal matters. In the end, it will be a judge that will rule who is in the wrong. Thanks, Sabri
Re: Reminder: Never connect a generator to home wiring without transfer switch
- On Aug 31, 2021, at 2:11 AM, Forrest Christian (List Account) li...@packetflux.com wrote: Hi, > I just wish the electrical code would permit or require certain low cost > things > which make temporary generator connections more likely to be safe. > For example, code requires most furnaces to be hardwired. But a furnace is one > of the first things you want on a generator in an extended winter power > outage. > If instead of hardwired, the code required plug and socket connections at each > 120v furnace then Joe homeowner would be more likely to run an extension cord > from his generator to his furnace instead of trying to rig up his generator > with a suicide cord. Now I'm wondering which jurisdiction you're talking about. I live in California in a home which was finalized in 2019. As I'm the first owner, I was there when the inspector went up into the attic and checked my HVAC. My HVAC has a plug in power cord running into a regular household socket (all in the attic). The inspector didn't say a word about it and issued the occupancy permit. My electrically powered oven is hardwired, but I guess that's because it requires two 50amp breakers? Thanks, Sabri
Re: An update on the AfriNIC situation
- On Aug 30, 2021, at 12:37 PM, Rubens Kuhl rube...@gmail.com wrote: Hi, >> I've ran an RBL for years, which many people used. It closed down more than >> a decade ago. Out of 100 DNS queries I logged just now with a quick tcpdump >> on one of my three DNS servers, I counted 51 for rbl.cluecentral.net. That's >> why I'm advocating to reconsider your carpet-bombing (filter into oblivion) >> recommendation. People don't remove them. > > I understand the risk, but when choosing between that risk and the > systemic risk for the RIR system, the choice for me is very clear. > Kinda like removing a malignant tumor. While I disagree with it, I do understand your point of view. I'm a proponent of "your network, your rules". But, if you would choose to filter the netblocks associated with this case, I would recommend that you filter them in BGP and not ACL them into oblivion. That way your customers won't be impacted (I have been on the customer end of something like this). Thanks, Sabri
Re: An update on the AfriNIC situation
- On Aug 30, 2021, at 11:18 AM, Rubens Kuhl rube...@gmail.com wrote: Hello Rubens, First and foremost, I appreciate that you're keeping it civil. > On Mon, Aug 30, 2021 at 2:35 PM Sabri Berisha wrote: >> The learned people on this list do not strike me as the kind of person to >> go out and engage in vigilante justice if a court decides against them. The >> very fabric of our civilized society depends on us resolving our conflicts >> in court, not out on the (virtual) streets. You may disagree with a ruling >> but I implore you to respect it. > > As previously mentioned, this is about something that doesn't involve > a court ruling, at least not yet, but a seizure request made by the > party to attack the sustainability of the RIR. Rulings that people > disagree have their own way inside the court system to be dealt with. I really, really don't want to upset Mel more than he already is, but Owen shared a link with an actual order of the court. After "consideration of the affidavit" the court allowed "up to" $50 million to be frozen. Whatever the merits of the affidavit are, it indicates that the court looked at the facts, made a determination and based on that ordered the asset freeze. That sounds like a (preliminary) ruling to me. I don't necessarily agree with it due to the implications it has on African internet operations, and, as Mark rightfully brought up, all the employment that depends on it, but I have to respect it. And don't get me wrong: I am not informed enough as to the dispute itself so I'm unable to form an opinion on who is right and who is wrong here. People whom I deeply respect on this list are on opposite sides so that adds to the confusion. I am, however, concerned with the operational implications. That's why I donated to the keep-Afrinic-alive-fund. I've ran an RBL for years, which many people used. It closed down more than a decade ago. Out of 100 DNS queries I logged just now with a quick tcpdump on one of my three DNS servers, I counted 51 for rbl.cluecentral.net. That's why I'm advocating to reconsider your carpet-bombing (filter into oblivion) recommendation. People don't remove them. Thanks, Sabri
Re: An update on the AfriNIC situation
- On Aug 30, 2021, at 6:29 AM, Rubens Kuhl rube...@gmail.com wrote: > And that's why carpet bombing those IP blocks might be needed so the next entity that ends up with those IP addresses long after CI has gone into oblivion will have its engineers debug odd routing issues for years. We all know that people regularly fail to update their manually entered filters on at least a few of their routers. The learned people on this list do not strike me as the kind of person to go out and engage in vigilante justice if a court decides against them. The very fabric of our civilized society depends on us resolving our conflicts in court, not out on the (virtual) streets. You may disagree with a ruling but I implore you to respect it. Rules... Without them we'd live with the animals.* Thanks, Sabri *(c) John Wick
Re: An update on the AfriNIC situation
- On Aug 27, 2021, at 8:36 AM, Bill Woodcock wo...@pch.net wrote: Hi, > If, like me, you feel like chipping in a little bit of money to help AfriNIC > make payroll despite Heng having gotten their bank accounts frozen, some of > the > African ISP associations have put together a fund, which you can donate to > here: > > https://www.tespok.co.ke/?page_id=14001 Top Gear Top Tip: set a "travel notification" on your credit card prior to donating. It took me 3 failed attempts and 2 fraud notifications to get a payment through. The fraud notifications were delayed as well. Chase credit card. "Verified by VISA". Right. And yes Mel, you're right about NANOG's AUP but this is not a legal matter, this is to keep AfriNIC in business... Thanks, Sabri
Re: Reminder: Never connect a generator to home wiring without transfer switch
- On Aug 25, 2021, at 7:04 AM, Mark Tinka mark@tinka.africa wrote: Hello Mark, > At the home, you typically have someone that is responsible for knowing > what to do in case of an outage, and switching over to self-generation. > If that person is not there, or has passed out from too many bottles of > wine that evening, someone else might think it's just a matter of > starting the generator, unwinding a suicide cord and plugging it into > the wall - totally forgetting about the main breaker. At my home, I use this: https://www.amazon.com/gp/product/B00CONE4MG The interlock kit is installed in such a way that either the main or the generator circuit breaker is closed. If the main is on, you can't switch to generator power, and vice versa (see the pictures on the listing, mine is installed the exact same way). Thanks, Sabri
Re: PeerinDB refuses to register certain networks [was: Setting sensible max-prefix limits]
- On Aug 18, 2021, at 4:03 PM, Rubens Kuhl rube...@gmail.com wrote: Hi, > Currently RPKI can only validate origin, not paths. If/when a path > validation solution is available, then one easy way to know that > network A really means to peer with network B is to publish a path > validation that B can use and/or forward A's announcements. Yes, that would be a relatively easy thing to calculate. Niels has, of course, a fair point when he writes: > When did PeeringDB turn into a routing (policy) registry? > You should use an IRRdb if you want to write RPSL. The difference is, if you are able to use PeeringDB as a single source of truth, it is a lot easier to grab the data you need. But again, their database, their rules. Thanks, Sabri
Re: PeerinDB refuses to register certain networks [was: Setting sensible max-prefix limits]
- On Aug 18, 2021, at 3:02 PM, Patrick W. Gilmore patr...@ianai.net wrote: Hi, > Those networks would be ones that do not peer. Which seems pretty obvious to > me > - it is literally in the name. I have an AS, I advertise IP space to the world. I want to be a Good Netizen and register my BGP peers. Your definition of BGP peering is different from mine, at least in this context. > I guess you are right, the _Peering_DB does not register “certain” networks. Which was my point. I'm glad you agree. My little AS is not allowed to play with the big kids. If you only want to register settlement-free peering, that's totally fine with me. Your database, your rules. But, the fact stays that you can have an AS, advertise your prefixes to the world, and not be permitted to register with peeringdb. Which means it can't be used as a single source of truth. Which would have been a shame because with a little bit of automation it would be feasible to "score" advertisements. That would help determine the likelihood of an advertisement to be erroneous (whether by accident or malice). For example, if I were to register my peers (53356 and 136620) and AS5524 would all of a sudden start to advertise my AS as behind it, you'd be able to flag that. But again, your database, your rules. Thanks, Sabri
Re: PeerinDB refuses to register certain networks [was: Setting sensible max-prefix limits]
- On Aug 18, 2021, at 2:21 PM, Patrick W. Gilmore patr...@ianai.net wrote: Hi, > On Aug 18, 2021, at 5:00 PM, Matthew Walster wrote: >> On Wed, 18 Aug 2021, 21:37 Sabri Berisha, wrote: >> - On Aug 18, 2021, at 2:46 AM, Steve Lalonde st...@enta.net wrote: >> >> Hi, >> >>> > We always use PeeringDB data and refuse to peer with networks not in >>> > PeeingDB >>> >>> You are aware that PeerinDB refuses to register certain networks, right? It >>> is >>> most certainly not a single source of truth. >>> >> Would you care to expand on this? > > I am extremely interested in hearing about this as well. > > Specific examples would be useful. Of course! Including headers to show authenticity. I was very amused by the explanation of the "chicken and egg" problem. Who's creating that? The networks who refuse to peer with non-peeringdb registered ASNs, or peeringdb who won't recognize ASNs that are not peering with anyone because nobody wants to peer with them because they are not registered in peeringdb because nobody wants to peer with them? You get the idea. Thanks, Sabri AS31064 Return-Path: gr...@peeringdb.com Received: from mail.cluecentral.net (LHLO mail.cluecentral.net) (195.16.84.32) by mail.cluecentral.net with LMTP; Fri, 9 Oct 2015 01:47:22 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cluecentral.net (Postfix) with ESMTP id 4CED64001EF for ; Fri, 9 Oct 2015 01:47:22 -0700 (PDT) Received: from mail.cluecentral.net ([127.0.0.1]) by localhost (mail.cluecentral.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3TLvVaNdjHGA for ; Fri, 9 Oct 2015 01:47:21 -0700 (PDT) Received: from ubersmith.peeringdb.com (ubersmith.peeringdb.com [107.6.74.106]) by mail.cluecentral.net (Postfix) with ESMTP id C5B164001A9 for ; Fri, 9 Oct 2015 01:47:01 -0700 (PDT) Received: by ubersmith.peeringdb.com (Postfix, from userid 48) id D8AF377C1A; Fri, 9 Oct 2015 04:46:29 -0400 (EDT) Date: Fri, 9 Oct 2015 04:46:29 -0400 To: Sabri Berisha From: supp...@peeringdb.com Reply-To: supp...@peeringdb.com Subject: Re: [#9192] [PeeringDB] User (sabri) Requesting Access (New Company - Cluecentral Inc) Message-ID: <1bac170d74e5d3702d3a28b237c87...@ubersmith.peeringdb.com> Dear PeeringDB user, Registering with peeringDB and peering negotiations are sort of egg and chicken problem. We only want to have networks registered that already do have settlement free peering. After some basic checks it looks like you are only buying transit from 6939/Hurricane Electric, but are not connected to any Internet Exchange (e.g. AMS-IX/NL-ix) yet. Having said this, is it acceptable to you to wait until you have your 1st settlement free peering setup? If you already have existing peering sessions, please provide the following details to support your request for peeringdb access: Your AS number(s) Which IXP / facilities you are peering at Some of your peering partners (again AS numbers / name) Please send your answers to supp...@peeringdb.com or reply to this ticket. Best regards, PeeringDB admin on Duty PeeringDB Listserv information: PeeringDB Announce: http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-announce PeeringDB Governance: http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-gov PeeringDB Technical: http://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech PeeringDB User Discuss: http://lists.peeringdb.com/cgi-bin/mailman/listinfo/user-discuss -- Florian Hibler PeeringDB Administrator
Re: Setting sensible max-prefix limits
- On Aug 18, 2021, at 2:46 AM, Steve Lalonde st...@enta.net wrote: Hi, > We always use PeeringDB data and refuse to peer with networks not in PeeingDB You are aware that PeerinDB refuses to register certain networks, right? It is most certainly not a single source of truth. Thanks, Sabri
Re: "Tactical" /24 announcements
- On Aug 12, 2021, at 10:38 AM, Amir Herzberg amir.li...@gmail.com wrote: Hi, > I don't think A would be right to filter these packets to 10.0.1.0/24; A has > announced > 10.0.0.0/16 so should route to that (entire) prefix, or A is misleading its > peers. This is what it boils down to. If you don't want to route it, don't advertise it. Thanks, Sabri
Re: "Tactical" /24 announcements
- On Aug 9, 2021, at 9:22 AM, Masataka Ohta mo...@necom830.hpcl.titech.ac.jp wrote: Hi, > It should be 14M. Just for fun, I did the math. A total of 16,777,216 /24s fit in 32 bits. Take away all the reserved space as per IANA (this is 1,266,696 /24s, see below), and we end up with 16,777,216 - 1,266,696 = 15,510,520 potential /24 advertisements. The largest FIB table I have seen (hi Jim!) was 3,563,546 routes in hardware. This was in a lab environment, of course. Thanks, Sabri https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml Subnet Number of /24s 0.0.0.0/8 65536 10.0.0.0/8 65536 100.64.0.0/10 16384 127.0.0.0/8 65536 169.254.0.0/16 256 172.16.0.0/12 4096 192.0.0.0/241 192.0.2.0/241 192.31.196.0/24 1 192.52.193.0/24 1 192.88.99.0/24 1 192.168.0.0/16 256 192.175.48.0/24 1 198.18.0.0/15 512 198.51.100.0/24 1 203.0.113.0/24 1 240.0.0.0/4 1048576 Total reserved 1,266,696
Re: FCC Proposes Ban on Devices Deemed a Threat to National Security
- On Jun 18, 2021, at 10:56 AM, Christopher Morrow wrote: Hi, > more-over, aren't there lots of other folk making gear (even inside the US!!!) > which are made up of components/software/etc which MAY be influenced/etc by > foreign > actors? Obligatory 37 second explanation: https://www.youtube.com/watch?v=bifOI4MbHVU Thanks, Sabri
Re: Myanmar internet - something to think about if you're having a bad day
- On Apr 28, 2021, at 11:32 AM, Eric Kuhnke wrote: Hi, > There's plenty of non technical teenagers in Pakistan with VPN clients on > their > phone or laptop who seem perfectly capable of using a VPN to watch Youtube or > access Twitter and other social media, during the periods of time that the > government orders things to be blocked. Even my third-grader was able to figure out that she needed a VPN when I blocked Roblox's IP space (128.116.0.0/17) on my home router. Other than, as reports said, soldiers snipping cables in datacenters, regimes will have a difficult time completely blocking whatever they don't like. Even China can't do it. Thanks, Sabri
Re: DoD IP Space
- On Apr 25, 2021, at 2:24 AM, Bill Woodcock wo...@pch.net wrote: Hi, > I think I’d characterize it, rather, as a possible privatization of public > property. This comment sparked my curiosity. Does ARIN consider IP space to be property? One could argue both ways: 1. Whomever "owns" a netblock simply owns the right to use and advertise it as long as it's being used for the purposes under which it was assigned by a number registry. This would be similar to "apartment rights" in a condominium complex. OR; 2. IP space comes with property rights such as selling and leasing as one wishes. But, that would also imply that IP space can be stolen. I'd be curious to hear what ARIN's position is on this. Thanks, Sabri
Re: IP reputation lookup (prefix not single IP)
- On Mar 26, 2021, at 8:20 PM, John Levine jo...@iecc.com wrote: Hi, > Also keep in mind that "most blocklists" is meaningless. Any moron can > run a blocklist, any many morons do. The vast majority of blockists > are used by close to nobody, and only handful are widely enough used > to matter. This moron ran a per-country/per-as blocklist in the early 2000s which was based on a DFZ BGP feed. I closed it off more than 10 years ago. I just checked and I'm still receiving ~5 queries per second. As per my anecdotal evidence, there are some really clueless operators out there as well. There is, of course, the temptation to just add a wildcard A record... But nah, I don't like hot places. The other side-effect is that spammers are still very eager to use my domain in their from: headers, judging by the amount of undeliverables I receive (in waves). Thanks, Sabri
Re: Perhaps it's time to think about enhancements to the NANOG list...?
- On Mar 23, 2021, at 1:09 AM, Mark Tinka mark@tinka.africa wrote: Hi, > I'm of the opposite view... front-end shiny GUI's are the risk. I'd > babysit them before I let them leave the house. For a long time. Children of the magenta line... Most of the more effective troubleshooting techniques will require some sort of CLI or CLI-like output. In times of crisis, you'll want to be able to type "show ip bgp summary", instead of waiting for your browser to send a javascript request to a server, the server to run a python script invoking netmiko to log onto a node, grab the output, reformat it, have it sent back to your browser and rendered. Not to mention that, like pilots, network engineers need hands-on time to stay effective. Planes have crashed because pilots lost it and relied on automation (Asiana 214, anyone?). That said, as a soon-to-be-dinosaur, I try to keep up with the latest and greatest. I don't want to run the risk of becoming an ATM engineer. Thanks, Sabri
Re: AW: OVH datacenter SBG2 in Strasbourg on fire
- On Mar 10, 2021, at 1:41 PM, Jeff Shultz wrote: >> The OVH datacenter is (was) in France. I bet you 10 bucks that the >> fire department was on strike. > Report I saw had the fire department on site in 3 minutes of the call. They > even > had a German-manned fireboat "Europa 1" working the fire from the water side. That's pretty impressive. It does make me wonder how long it took for them to be notified, and why on earth the fire spread so fast that the entire DC was lost... And because, for once, the French were not on strike, I donated $10 to the American Red Cross. Thanks, Sabri
Re: AW: OVH datacenter SBG2 in Strasbourg on fire
- On Mar 10, 2021, at 11:28 AM, Bryan Holloway br...@shout.net wrote: Hi, > Fire Department was there in under five minutes. I assume your Enron DC was in the U.S.? The OVH datacenter is (was) in France. I bet you 10 bucks that the fire department was on strike. Thanks, Sabri
Re: Ip space Dilemma
- On Mar 9, 2021, at 6:13 AM, Justin Wilson (Lists) li...@mtin.net wrote: Hi, > I am at the point I need to give the space back because it is unusable > to the > ISP customers. Does anyone have any creative ideas on how to fix this? Since they are a government entity, a process server might do the trick... Thanks, Sabri
Re: Famous operational issues
- On Feb 19, 2021, at 3:07 AM, Daniel Karrenberg d...@ripe.net wrote: Hi, > Lessons: HW/SW mono-cultures are dangerous. Input testing is good > practice at all levels software. Operational co-ordination is key in > times of crisis. Well... Here is a very similar, fairly recent one. Albeit in this case, the opposite is true: running one software train would have prevented an outage. Some members on this list (hi, Brian!) will recognize the story. Group XX within $company decided to deploy EVPN. All of backbone was running single $vendor, but different software trains. Turns out that between an early draft, implemented in version X, and the RFC, implemented in version Y, a change was made in NLRI formats which were not backwards compatible. Version X was in use on virtually all DC egress boxes, version Y was in use on route reflectors. The moment the first EVPN NLRI was advertised, the entire backbone melted down. Dept-wide alert issued (at night), people trying to log on to the VPN. Oh wait, the VPN requires yubikey, which requires the corp network to access the interwebs, which is not accessible due to said issue. And, despite me complaining since the day of hire, no out of band network. I didn't stay much longer after that. Thanks, Sabri
Re: Famous operational issues
On Feb 18, 2021, at 11:51 PM, Suresh Ramasubramanian wrote: >> On 2/19/21 00:37, Warren Kumari wrote: >> and says "'K. So, you doing a full iBGP mesh, or confeds?". I really hadn't >> intended to be a condescending ass, but I think of that every time I realize >> I >> might be assuming something about someone based on thier attire/job/etc. > Did you at least hire the janitor? Well, it's funny that you mention that because I worked at a place where the company ended up hiring a young lady who worked in the cafeteria. When she graduated she was offered a job in HR, and turned out to be absolutely awesome. At some point in my life, I was carrying 50lbs bags of potato starch. Now I have two graduate degrees and am working on a third. That janitor may be awesome, too! Thanks, Sabri
Re: Texas internet connectivity declining due to blackouts
- On Feb 17, 2021, at 11:21 AM, nanog wrote: Hi, > Using the sample bill on the GA power website you linked, I see a bottom line > price of $76.17 for 606 kWh delivered to the customer. That is effectively > 12.57 cents per kWh. > Utilities (both investor owned and coops) have a multitude of ways of hiding > the > effective price in a variety of fixed and variable fees not included in the > nominal 'energy' fee. These include mandatory fixed connection fees and also > fuel cost recovery fees that are tied to consumption. Exactly. In a message earlier today which is held and presumably lost due to moderation, I shared screenshots of an actual bill of mine here in California. Long story short, using that bill I show that I paid a grand total of $239.14 for 656.928 KwH of electricity. That makes 36.4 cents per KwH. In addition to that, I also shared another bill, where I paid $2.63 for the privilige of providing the net with 31.993 KwH of energy. That's right. My solar panels produced more power than I consumed and I still sponsored the crooks at PG Utility companies are worse than airlines when it comes to hidden fees and surcharges. They know we have no choice. The only reason I want more solar panels is to give a bigger middle finger to PG Nothing is a better motivator to go green than to see PG go bankrupt. It's a sad state of affairs when the disgust for the utility company's deceptive practices somehow outweighs the need to save the planet. Yet here we are. Thanks, Sabri
Re: Texas internet connectivity declining due to blackouts
- On Feb 16, 2021, at 6:28 AM, Michael Thomas m...@mtcc.com wrote: > We use propane. It's less dense energy-wise than gasoline, but it's > really easy to switch over. Why not use both? Plenty of generators that are dual fuel out there. Last year I converted my Duramax to dual fuel by replacing the carburator. Easy-peasy. Thanks, Sabri
Re: Famous operational issues
- On Feb 16, 2021, at 2:08 PM, Jared Mauch ja...@puck.nether.net wrote: Hi, > I was thinking about how we need a war stories nanog track. My favorite was > being on call when the router was stolen. Wait... what? I would love to listen to that call between you and your manager. But, here is one for you then. I was once called to a POP where one of our main routers was down. Due to political reasons, my access had been revoked. My manager told me to do whatever I needed to do to fix the problem, he would cover my behind. I did, and I "gently" removed the door. My manager held word. Another interesting one: entering a pop to find it flooded. Luckily there were raised floors with only fiber underneath the floor panels. The NOC ignored the warnings because "it was impossible for water to enter the building as it was not raining". Yeah, but water pipes do burst from time to time. But my favorite was pressing an undocumented combination of keys on a fire alarm system which set off the Inergen protection without warning, immediately. The noise and pressure of all that air entering the datacenter space with me still in it is something I will never forget. Similar to the response of my manager who, instead of asking me if I was ok, decided to try and light a piece of paper. "Oh wow, it does work, I can't set anything on fire". All if this was, obviously, in the late 1990s and early 2000s. These days, things are -slightly- more professional. Thanks, Sabri
Re: Texas internet connectivity declining due to blackouts
- On Feb 16, 2021, at 5:01 AM, Sean Donelan s...@donelan.com wrote: > On Tue, 16 Feb 2021, Rod Beck wrote: >> Are the power lines buried like in Europe where I live? They are not buried everywhere. They are buried in most western EU countries perhaps. But I invite you to go to Ferizaj, Kosovo, for example. > In California, they use rolling blackouts BEFORE wildfires to prevent > power line sparking causing wildfires. Not because of damage to the > outside plant. In Texas, they use rolling blackouts because they didn't > have enough generation capacity online. I do remember last September being threatened with rolling power outages as a result of the lack of capacity. Check this article in the Mercury News, for example: https://www.mercurynews.com/2020/09/06/california-grid-managers-watching-closely-as-weather-presents-power-outage-threats/ Thanks, Sabri
Re: DoD IP Space
- On Feb 15, 2021, at 9:28 AM, mel wrote: Hi, > LOL! Well, Mike says “definitely at least 1993”, whereas Wikipedia itself says > that Wikipedia cannot be trusted. Mike, to my knowledge, has never admitted > being wrong. So I’m going with Mike :) Well, considering this RIPE article that talked about IPv7 already.. https://lists.ripe.net/pipermail/ripe-org-closed/1993/msg00024.html I'd say: myth plausible. > I think it was Al Gore who first proposed IPv6, right Mike? :) Myth busted. He invented the internet. IPv6 was invented by his intern. Thanks, Sabri
Re: DoD IP Space
- On Feb 14, 2021, at 11:56 AM, Randy Bush ra...@psg.com wrote: Hi, > hint: that idea is from the late '90s. the next bright idea for what > would help ipv6 take over the internet was 3gpp. it's been a long line > of things which would make ipv6 take off. You are 100% Correct. Perhaps we can get Jeff Bezos to give 25% extra off at the next Cyber Monday event to those accessing amazon.com via IPv6. That will not only drive IPv6 deployment at eyeball networks, it's a feasible plan as well. IF good ol' Jeff wants to cooperate :) Thanks, Sabri
Re: DoD IP Space
- On Feb 11, 2021, at 9:15 AM, Eric Kuhnke wrote: Hi, You're right and wrong. > You don't, you wastefully assign a /24 to every unique thing that you think > needs an internal management IP block (even if there's 5 things that answer > pings there), Reword that to: in the late 1990s, someone took an ICND course and decided that assigned a /24 as a minimum for each subnet was fine as they would never run out of RFC1918 space. Today, the current network owner is stuck with that inherited problem. > and decide it's too much work to renumber things. Reword that to: and management decides that they are not going to fund a renumbering project as they have other priorities. (that's how work gets funded in every large org that I've worked for) > Easy for a big ISP that's also acquired many small/mid-sized ISPs to run out > of > v4 private IP space that way. Not just ISPs. Plenty of decades old enterprises. Mark Tinka wrote: > Let's not normalize the sustenance of IPv4 in 2021, in the real world. Our opinions don't matter to the PHBs whos bonuses rely on features delivered. The only time that I got some serious attention with regards to this matter was when my manager and I took it three layers up and warned them that we were about to run out of RFC1918 space unless drastic measures were taken. They were, but now how we wanted: they forced other groups to return unused allocations. Now we had half of 10/8 back, and deployment of new pods could resume... Problem "solved". I get really sad when people bicker on this list about who is at fault. The purity fundamentalists complain that realists have run out of RFC1918 due to their poor decisions, while in 99% of the cases it's a result of decisions made long ago by their predecessors. The true enemy here is mid-level management that refuses to prioritize deployment of IPv6. What we should be discussing is how best to approach that problem. It's where ops and corporate politics overlap. Thanks, Sabri
Re: Amsterdam Dark Fiber
Hi, Back in the day when I still lived there, Level 3 was also known to have fiber in the area. Depending on your needs, Equinix offers dark fiber between (some of) their locations, and Relined(.eu) has a nationwide fiber network. You can also check out irc, irc.nlnog.net, #nlnog, or subscribe to the nlnog mailing list, [ http://mailman.nlnog.net/ | http://mailman.nlnog.net/ ] Thanks, Sabri - On Feb 4, 2021, at 5:04 PM, Rod Beck wrote: > Please contact offlist. > Looking for dark fiber in Amsterdam. Eurofiber has traditionally dominated > this > market. Who else competes in this market? > Roderick Beck VP of Business Development > United Cable Company > [ http://www.unitedcablecompany.com/ | www.unitedcablecompany.com ] > New York City & Budapest > rod.b...@unitedcablecompany.com > Budapest: 36-70-605-5144 > NJ: 908-452-8183
Re: gofundme Medical Expenses - Ed Hew
- On Jan 25, 2021, at 8:37 AM, Jim Mercer j...@reptiles.org wrote: Hi, > https://www.gofundme.com/f/ed-hew-medical-expenses Just a headsup for those outside of Canada. My transaction was processed in CAD instead of USD. Not that I care as amex doesn't charge foreign fees on my card, but if you have a choice of creditcards, pick the right one. Thanks, Sabri
Re: DoD IP Space
- On Jan 22, 2021, at 10:28 PM, Valdis Klētnieks valdis.kletni...@vt.edu wrote: Hi, > On Thu, 21 Jan 2021 11:07:42 -0800, Sabri Berisha said: >> Financial incentives also work. Perhaps we can convince Mr. Biden to give a >> .5% >> tax cut to corporations that fully implement v6. That will create some bonus >> targets. > > And how would you define "fully implement v6", anyhow? Fair point. I'm sure the a commission appointed by the appropriate legislators will be happy to spend a few millions debating that issue. Personally, I would argue that a full implementation of IPv6 means that v4 could be phased out without adverse effect on the production network. But of course, how would we define "adverse effect on the production network"? :) > Even more problematic: What do you do with a company that's fully v6-ready, > but > still has several major interconnects to other companies that *aren't* ready, > and thus still using v4? I totally agree with everything you wrote. It proves the point that having v6 ready technologies in "the network", does not mean a network, or even a company is fully v6 ready. Way too many stakeholders and outside dependencies. To me, it means that "we", as in network professionals, should be ready to save the day when company leaders finally realize they have no option and need v6 to be implemented fast. And secretly, I've been hoping for that moment. "Well, sir, the network has been IPv6 ready for years, but the software groups and their leadership have so far blatantly refused to update their code and support it". I guess that I'll join you in retirement before that moment comes. Thanks, Sabri
Re: DoD IP Space
- On Jan 22, 2021, at 4:50 PM, Izaac iz...@setec.org wrote: Hi, > On Fri, Jan 22, 2021 at 03:43:43PM -0800, Sabri Berisha wrote: >> TL;DR: in theory, I agree with you 100%. In practice, that stuff just doesn't >> work. > > Well thanks for sharing. I think we've all learned a lot. You don't need to patronize me. I'm merely explaining the real life realities of working in a large enterprise. And the key takeaway here is: we can come up with the most efficient solutions, in the end it's all about budgets and stakeholder requirements. Thanks, Sabri
Re: DoD IP Space
- On Jan 22, 2021, at 2:42 PM, Izaac iz...@setec.org wrote: Hi, > On Fri, Jan 22, 2021 at 01:03:15PM -0800, Sabri Berisha wrote: >> TL;DR: a combination of scale and incompetence means you can run out of 10/8 >> really quick. > > Indeed. Thank you for providing a demonstration of my point. > > I'd question the importance of having an console on target in Singapore > be able to directly address an BMC controller in Phoenix (wait for it), > but I'm sure that's a mission requirement. No, but the NOC that sits in between does need to access both. Sure, you can use jumphosts, but now you're delaying troubleshooting of a potentially costly outage. > But just in case you'd like to reconsider, can I interest you in NAT? > Like nutmeg, a little will add some spice to your recipe -- but too much > will cause nausea and hallucinations. NAT'ing RFC1918 to other RFC1918 space inside the same datacenter, or even company, is a nightmare. If you've ever been on call for any decently sized network, you'll know that. > Let's just magic a rack controller to handle the NAT. We can just cram it > into the extra-dimensional space where the switches live. > And all less than an hour's chin pulling. We both know that this is A. An operational nightmare, and B. Simply not the way things work in the real world. The people who designed most of the legacy networks I've ever worked on did not plan for the networks to grow to the size they became. Just like we would never run out of the 640k of memory, people thought they would never run out of RFC1918 space. Until they did. And when that James May moment arrives, people start looking at a quick fix (i.e., let's use unannounced public space), rather than redesigning and reimplementing networks that have been in use for a long long time. TL;DR: in theory, I agree with you 100%. In practice, that stuff just doesn't work. Thanks, Sabri
Re: DoD IP Space
- On Jan 22, 2021, at 12:28 PM, Izaac iz...@setec.org wrote: Hi, > On Wed, Jan 20, 2021 at 02:47:32PM +0100, Cynthia Revström via NANOG wrote: >> certain large corporations that have run out of RFC1918, etc. space > > At what level of incompetence must an organization operate to squander > roughly 70,000 /24 networks? Or, at what level of scale. Or, a combination of both. Let me give you an example. This example is not hypothetical. Acme Inc operates a popular social media site. This requires a lot of compute power, and storage space. Acme owns multiple datacenters around the world, and all must be connected. Acme divides its data centers in "Availability Zones". Each AZ contains a limited amount of equipment. A typical AZ is made up of multiple pods, and each pod contains anywhere between 40 and 48 racks. Each rack contains up to 72 servers. Each server can contain many VMs or containers. In order to scale, each AZ and pod are designed according to blueprints. This obviously means that tradeoffs must be made. For example, each rack will be assigned a /25, since a /26 means that not all 72 servers can have an IP. Just to accommodate a single IP per server, we already need a /19. Most servers will have different NICs for different purposes. For example, it is not uncommon to have a separate storage network, and a management network. Now we already need 3 /19s (32 /24s per pod, and we haven't even started to assign IPs to VMs or containers yet. Let's start to assign IPs to VMs and containers. Within one of my previous employers, there were different groups that worked on VMs (cloud), and containers (k8s). Both groups had automated scripts to assign IPs, but these (obviously) did not communicate. Which means that each group had their own vlan, with their own IRB (or BVI, or VLAN interface, however you want to name it). On average, each group started with a /22 per tor (later on, we limited them to a /24). So now we need 48*2*4=384 /24s per pod extra. So, with 384+32 = 416 /24s per pod, you are looking at a maximum of 157 pods. Now, granted, there is a lot of waste in this, hence the change from a /22 to a /24, with a realization that the cloud and k8s group really needed to work together to avoid more waste. I will tell you that this is not at all hypothetical, I have personally created spreadsheets of every /16 in 10/8 and how they were allocated. It's amazing how much space was wasted in the early days at said employer, and how much I was able to reclaim simply by checking if the allocations were still valid. Hint: when companies split up, a lot of space gets freed up. This the way that we avoided using DoD IP space to complement 10/8. But, you were asking how it's possible to run out of 10/8, and here is your answer :) TL;DR: a combination of scale and incompetence means you can run out of 10/8 really quick. Thanks, Sabri
Re: DoD IP Space
- On Jan 21, 2021, at 6:40 AM, Andy Ringsmuth a...@andyring.com wrote: Hi, > I’m sure we all remember Y2k Ah, yes. As a young IT consultant wearing a suit and tie (rofl), I upgraded many bioses in many office buildings in the months leading up to it... > I’d love to see a line in the concrete of, say, January 1, 2025, whereby IPv6 > will be the default. The challenge with that is the market. Y2K was a problem that was existed. It was a brick wall that we would hit no matter what. The faulty code was released years before the date. We, IETF, or even the UN could come up with 1/1/25 as the date where we switch off IPv4, and you will still find networks that run IPv4 for the simple reason that the people who own those networks have a choice. With Y2K there was no choice. The best way to have IPv6 implemented worldwide is by having an incentive for the executives that make the decisions. From experience, as I've said on this list a few times before, I can tell you that decision makers with a limited budget that have to choose between a new revenue generating feature, or a company-wide implementation of IPv6, will choose the one that's best for their own short-term interests. On that note, I did have a perhaps silly idea: One way to create the demand could be to have browser makers add a warning to the URL bar, similar to the HTTPS warnings we see today. If a site is IPv4 only, warn that the site is using deprecated technology. Financial incentives also work. Perhaps we can convince Mr. Biden to give a .5% tax cut to corporations that fully implement v6. That will create some bonus targets. Thanks, Sabri
Re: DoD IP Space
- On Jan 20, 2021, at 6:58 AM, j k wrote: Hi, > My question becomes, what level of risk are these companies taking on by using > the DoD ranges on their internal networks? And have they quantified the costs > of this outage against moving to IPv6? Not so long ago, while working for a large enterprise, my team was considering the use of non-advertised public IP space when we realized we were close to running out of RFC1918 space. Eventually we decided against it as we had enough options to reclaim unused RFC1918 from within the company. However, we had a number of arguments against the use of public ranges: - The risk of owners deciding to advertise their space. If so, since we operated a popular ecommerce site, there would be a huge risk of users encountering issues. - The risk of inadvertent security issues. People using RFC1918 space, even the most network-illiterate dev, know that RFC1918 space is not accessible from the big bad internet. This (perceived) safety is absent when using public IP space. - The risk of misconfiguring firewalls. Obviously, most of the policies cover RFC1918 space. Introducing non-RFC1918 space encourages human error. - The risk of looking like fools if we would accidentally leak. Let's be honest. There are two groups of people on this list. Those who have accidentally leaked and those who will. I learned from my mistake(s). As for IPv6: I know I sound like a broken record but one does not simply walk into Mordor and migrate to IPv6. In a large enterprise, especially with one using a lot of old code to support a highly popular webapp, it is easier to move a mountain than it is to get all nosed aligned. The network group(s), corp, lab, DC, backbone, may all be ready, but that does not mean that your cloud, kubernetes, frontend, backend, operations, and billing groups are ready. Migrating to IPv6 is a cost, as there is no ROI. It is a cost center, not an investment. Surely, we all on this list know that it is a mandatory expense to ensure future delivery of services, but explain that to a VP with limited budgets. Are they going for the short term win of new features, or for the long term "win" of retaining revenue? We all know what their bonuses are based on. And don't get me wrong. I'm not advocating against v6. I'm merely explaining how difficult it can be to migrate. In most large companies, the network is like PG (the power utility California). If it works, nobody says well done. But if the power is out, everyone gets angry and asks why we have fools operating the power grid. Thanks, Sabri
Re: DoNotPay Spam?
- On Jan 13, 2021, at 2:22 PM, Bryan Fields br...@bryanfields.net wrote: Hi Bryan, > What you can do is when you notice these, email geeks@nanog with the full > email including headers immediately. We can then cross check it against new > signups. I wish there was a more scientific way to process it. The first time I got it, I sent this to supp...@donotpay.com: > I received this email in, what appears to be, reply to a post I made on NANOG. > Needless to say, I never signed up for this. I did not even know you existed. > Since you do add "supp...@donotpay.com" in your email, I assume this is a > honest mistake, and you'll be happy that I'm contacting you and will be fixing > it immediately. > Obviously, further unsolicited emails will result in ... a different approach > taken. A few days later, I got the same again, and contacted their hosting provider, Mailgun (while CCing supp...@donotpay.com), with the following: > I've received, multiple times, email such as below after posting to the North > American Network Operators Group (NANOG) email list. I've tried contacting > supp...@donotpay.com (ticket #13202), but they seem oblivious to the issue > and asked me to unsubscribe. > Please educate your customer. Alternatively, I will contact Amazon, who seem > to advertise your IP space. > 161.38.200.0/22*[BGP/170] 00:51:18, localpref 150 >AS path: 53356 60011 3356 16509 I, validation-state: > unverified > > to 195.16.87.249 via ge-0/0/6.0 > Headers are as follows: [snip] I did not even get a reply on that. So, as promised, the third time I was spammed, I took the liberty of contacting AWS. They responded with: > This is a follow up regarding the abusive content or activity report that you > submitted to AWS. We have investigated this report, and have taken steps to > mitigate the reported abusive content or activity. But of course, nothing changed. This goes a lot further than someone accidentally subscribing. So, it seems that there are few options other than to simply block mail from that /22. Thanks, Sabri
AWS Hosts spammers Re: DoNotPay Spam?
Hi, Yep. I complained to their support. Then I complaint to their "mail provider" Mailgun. When that proved useless, I complaint to AWS who hosts Mailgun. AWS replied and said they would get in touch with Mailgun. We'll see whether or not Mailgun gets the Parler treatment. Thanks, Sabri - On Jan 13, 2021, at 2:06 PM, Robert Webb wrote: > Anyone else getting spam from DoNotPay everytime they send an email to the > list? > I have not sent anything in a while until my ATT email and now I am getting > this > on every new email I send to the list. > You’re almost there! Sign up once to unlock lifetime protection (and even > compensation) on all spam emails.
Re: shouting draft resisters, Parler
- On Jan 11, 2021, at 3:25 PM, Joe Loiacono jloia...@gmail.com wrote: Hi, > Only if you believe censorship has nothing to do with free speech. As Anne was trying to point out, the 1st Amendment protects you from the Government, and more specifically, Congress: Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances. Your 1st Amendment rights do not include the right to put your signs in your neighbor's yard, and by extension, to host your website on your neighbor's (or Amazon's) private infrastructure. This does not mean that I agree with Amazon's decision. There are a lot of implications to this. Thanks, Sabri Who now waits for another donotpay.com "confirmation". And will then, yet again, complain to their support, Mailgun, and AWS.
Re: not a utility, was Parler
- On Jan 11, 2021, at 4:46 AM, Karl Auer ka...@biplane.com.au wrote: Hi, > "The DNS is a natural monopoly. People want one resolver so they can > connect with all their 'sites'. No one is going to use several > nameservers for domain name resolution. They want one." > > Nah. The DNS is a natural distributed database, with authoritative data > held by those with the most interest in its accuracy. But unlike DNS > data, there is money in collecting all the facebooky things - IF you > are allowed to sell them. Stop that, and Facebook is a natural > distributed database too. There is also money being made in DNS. A lot of money is being made in DNS. According to Verisign(1) Q3 of 2020 closed with 370.7 million new registrations. At an average of $15 per domain(2), that equals a market of $5.5 billion dollars. Now, that's of course pocket change compared to Facebook's $21.4 billion Q3 revenue(3), but still. And that's without all those alt-root con schemes. Thanks, Sabri (1) https://www.verisign.com/en_US/domain-names/dnib/index.xhtml (2) https://www.websitebuilderexpert.com/building-websites/domain-name-cost/ (3) https://investor.fb.com/investor-events/event-details/2020/Facebook-Q3-2020-Earnings/default.aspx
Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study
- On Jan 1, 2021, at 2:12 PM, Matt Hoppes mattli...@rivervalleyinternet.net wrote: Hi, > How would that even work? Force a pop up into web traffic? What if the end > users is using an app on a phone? Most, if not all, mobile devices connected to cellular already have that option. On my iphone it's under settings->notifications->government alerts. There are three separate options: Amber alerts, Emergency alerts, and Public Safety alerts. Personally, I have all three turned off after receiving nonsens alerts. Amber alerts for children abducted in Los Angeles, only 600km (~450 miles) from the Bay Area, where I live, for example. Or a "public safety" alert telling me that there are too many people in the local Trader Joe's, 2 miles from my home. Aliens always invade New York, so I'm safe up here :) Thanks, Sabri
Re: [External] Re: 10g residential CPE
- On Dec 27, 2020, at 10:06 AM, Michael Thomas m...@mtcc.com wrote: Hi, > Right and here in California, it was precisely those lines that > incinerated Paradise. And for those lurkers outside of CA, or even the U.S., the small town named "Paradise" was completely wiped off the map a few years ago due to horrific wildfires. The smoke was so bad that here in the Bay Area we were wearing N95 masks because of it. The masks I bought back then were useful again when the pandemic started. Netflix has a documentary on it, "Fire In Paradise". Gives me the chills every time I watch it. Thanks, Sabri
Re: "Hacking" these days - purpose?
- On Dec 12, 2020, at 2:26 PM, Peter E. Fry p...@tailbone.net wrote: Hi, > Simple question: What's the purpose of obtaining illicit access to > random devices on the Internet these days Don't underestimate the curiosity if pimply faced youth these days. Wargames is still relevant. Thanks, Sabri
Re: Telia Not Withdrawing v6 Routes
- On Nov 16, 2020, at 11:45 AM, Matt Corallo na...@as397444.net wrote: Hi, > See my latest response from this morning. Telia's "Head of Network > Engineering & > Architecture" confirmed on Twitter this > was due to a (now-worked-around) bug in JunOS. > > https://twitter.com/gustawsson/status/1328298914785730561 Interesting. A long time ago, in a galaxy far far away, where I was a JTAC engineer, policy was that once a PR was hit in the field, it would be marked public. Also, in the case that I described it wasn't a Junos device. Makes me wonder how bugs like that get introduced. One would expect that after 20+ years of writing BGP code, handling a withdrawl would be easy-peasy. Thanks, Sabri
Re: Telia Not Withdrawing v6 Routes
- On Nov 15, 2020, at 5:58 PM, Matt Corallo na...@as397444.net wrote: > Has anyone else experienced issues where Telia won't withdraw (though will > happily accept an overriding) prefixes for the past week, at least? I have seen issues like this in a network that I operated. In that particular case, it was an internal ipv4 10/8 route which was withdrawn, along with a few hundred other routes. The withdrawl was configured on a DC exit router, in a Clos network with leaf, spine, and superspine. On the spine layer, I observed that BGP withdrawls, although being received, were not processed by the control plane. Further investigation and working with the TAC of the vendor, revealed that on that particular platform, the BGP process would stop process withdrawls in a very nasty race condition that was very difficult to reproduce. This was the first (and so far only) time in my 20+ years of working with BGP that I've observed such a weird bug. Since I operated the entire network, it was fairly easy to find the culprit. The why, took some more time. If I were in your shoes, I'd ping Telia's NOC to see what's going on. I would not be surprised if they'd be hitting a similar issue. Thanks, Sabri
Re: FCC: Staff Report on T-Mobile Outage on June 15 2020
- On Nov 12, 2020, at 9:35 AM, Sean Donelan s...@donelan.com wrote: Hi, > FCC Issues Staff Report On T-Mobile Outage > > https://www.fcc.gov/document/fcc-issues-staff-report-t-mobile-outage-0 This part, I find most interesting as well: > However, they were unable to resolve the issue by restoring the link because > the network management tools required to do so remotely relied on the same > paths they had just disabled. I can't begin to tell you how often I battled senior mgmt to get some investment into an OOB network. This only proves the point. Parantap, are you reading this? I know you are. Thanks, Sabri
Re: Technology risk without safeguards
- On Nov 10, 2020, at 12:56 AM, Jon Sands fohdee...@gmail.com wrote: > On Fri, Nov 6, 2020, 8:00 PM Suresh Kalkunte < [ mailto:sskalku...@gmail.com | > sskalku...@gmail.com ] > wrote: >> raw garlic assimilation > This thread is definitely going to be used in a future court case Nah, by that time this thread will be classified as an internet conspiracy :) If anyone should look at this thread for the purposes of bringing it in as evidence in any type of legal action: keep in mind that few (if any) of the contributors are medical or legal professionals, and some may be simple trolls. Nothing in this thread has any evidentiary contribution and represents the personal opinion of the writers. Scientific studies should be preferred over this type of 'internet folklore'. We're network plumbers. Thanks, Sabri, certified plumber.
Re: CNAME records in place of A records
- On Nov 6, 2020, at 2:07 AM, Dovid Bender wrote: Hi, > Sorry if this is a bit OT. Recently several different vendors (in completely > different fields) where they white label for us asked us to remove A records > that we have going to them and replace them with CNAME records. Is there > anything *going around* in the security aranea that has caused this? Security-wise, you should be good. But make sure you're not attempting to deliver e-mail to such a domain; CNAMEs cannot be used in MX records. Thanks, Sabri
Re: Technology risk without safeguards
- On Nov 5, 2020, at 5:58 AM, Tom Beecher wrote: Hi, >> The parts that Tom cited, are very much relevant, and only reinforce the >> notion that at this time, we simply do not know enough. We do know, that >> at the low doses we generally receive, there is no evidence for harmful >> consequences. > This is a gross mischaracterization, and I would go so far to say patently > incorrect. Well, from the parts you quoted yourself, cut and paste from your email: - "it’s not clear how RF radiation might be able to cause cancer." - "the results of these types of studies have not provided clear answers so far." - "this is still an area of research." - "these studies had strengths, they also had limitations that make it hard to know how they might apply to humans" - "(ICNIRP) determined that the limitations of the studies didn’t allow conclusions to be drawn regarding the ability of RF energy to cause cancer." Which part of that is patently incorrect? Again, I'm not saying anything regarding the actual topic itself, I'm not an expert in that field. > His findings go into the pile with all the other findings, and they get > properly > evaluated. Exactly. That how science works. Glad you understand it. You evaluate the data, instead of dismissing the doctor as some kind of QAnon conspiracy theorist. And that was the whole point of my post. I never made any assertion with regards to whether or not the hypothesis was correct. I merely quoted resources which indicated that more research was needed. Thanks, Sabri
Re: Technology risk without safeguards
Hi Suresh, I'm not disputing anything you or Tom wrote. The current scientific consensus is that most RF exposures are sage. We agree on that. My point is simply that, as Tom wrote in his citation, the biological effects of RF are still an area of research. And for that reason, it's unfair to dismiss a physician's suggestion to look into a case as an "internet conspiracy". That's all. Thanks, Sabri - On Nov 4, 2020, at 7:23 PM, Suresh Kalkunte wrote: > Existing research on health effects from RF signals dwell on emissions from > regulated sources, (mobile handset, base of a tower etc), my overriding > concern > is, unrestricted/chronic exposure for extended duration of time for which > there > are very rare research efforts devoted. > Chronic exposure to RF is found to induce DNA instability^1^. Even if RF at > chronic exposure levels are not found to cause DNA strands to break, it > creates > upstream conditions such as excess Calcium influx^2,3^ into the cell's > cytoplasm with implications on cardiac arrhythmia^4^, invoke and/or worsen > neurodegenerative^5^ diseases to name a few. > Labeling any discussion on adverse health from OVEREXPOSURE to RF is a cop-out > from doing a threadbare analysis. > Suresh S. > ^1^ Mashevich M, Folkman D, Kesar A, et. al. Exposure of human peripheral > blood > lymphocytes to electromagnetic fields associated with cellular phones leads to > chromosomal instability. Bioelectromagnetics. 2003;24:82–90. > ^2^ Arber SL, Lin JC. Extracellular calcium and microwave enhancement of > membrane conductance in snail neurons. Radiat Environ Biophys. Jun > 1985;24(2):149–156. > ^3^ Rao VS, Titushkin IA, Moros EG et al. Nonthermal effects of > radiofrequency-field exposure on calcium dynamics in stem cell-derived > neuronal > cells: elucidation of calcium pathways. > Radiat Res. 2008 March. 169(3):319-29. > ^4^ Grace AA , Camm AJ. Voltage-gated calcium -channels and antiarrhythmic > drug > action. > Cardiovasc Res. Jan 2000;45(1):43–51. > ^5^ Leal SS, Gomes CM. Calcium dysregulation links ALS defective proteins and > motor neuron > selective vulnerability. Front Cell Neurosci. 2015;9:225. > On Thursday, November 5, 2020, Tom Beecher wrote: >>> The hypothesis that RF may cause damage to human DNA is not at all >>> conspiracy. >>> The >>> fact that we haven't been able to identify a factual relationship, does not >>> mean >>> that there isn't any. For example: >> If you are going to cite that American Cancer Society article, you should >> cite >> all the relevant parts. The parts you skipped are bolded. >>> RF waves don’t have enough energy to damage DNA directly. Because of this, >>> it’s >>> not clear how RF radiation might be able to cause cancer. Some studies have >>> found possible increased rates of certain types of tumors in lab animals >>> exposed to RF radiation, but overall, the results of these types of studies >>> have not provided clear answers so far. >>> A few studies have reported evidence of biological effects that could be >>> linked >>> to cancer, but this is still an area of research. >>> In large studies published in 2018 by the US National Toxicology Program >>> (NTP) >>> and by the Ramazzini Institute in Italy, researchers exposed groups of lab >>> rats >>> (as well as mice, in the case of the NTP study) to RF waves over their >>> entire >>> bodies for many hours a day, starting before birth and continuing for at >>> least >>> most of their natural lives. Both studies found an increased risk of >>> uncommon >>> heart tumors called malignant schwannomas in male rats, but not in female >>> rats >>> (nor in male or female mice, in the NTP study). The NTP study also reported >>> possible increased risks of certain types of tumors in the brain and in the >>> adrenal glands. >>> While both of these studies had strengths, they also had limitations that >>> make >>> it hard to know how they might apply to humans being exposed to RF >>> radiation. A >>> 2019 review of these two studies by the International Commission on >>> Non-Ionizing Radiation Protection (ICNIRP) determined that the limitations >>> of >>> the studies didn’t allow conclusions to be drawn regarding the ability of RF >>> energy to cause cancer. >>> Still, the results of these studies do not rule out the possibility that RF >>> radiation might somehow be able to impact human health. >> The majority of science to date finds no causal relationship betw
Re: Technology risk without safeguards
- On Nov 4, 2020, at 7:19 PM, Randy Bush ra...@psg.com wrote: Hi, >> The fact that we haven't been able to identify a factual relationship, >> does not mean that there isn't any. > > just wow > > and, for all we know, the back side of the moon is green cheese I don't think you got the message buried within my message. True science is open to change, based on learning new facts. Like I said initially, I agree with Suresh that at this time, there is no scientific evidence that links RF with any kind of bodily harm. The parts that Tom cited, are very much relevant, and only reinforce the notion that at this time, we simply do not know enough. We do know, that at the low doses we generally receive, there is no evidence for harmful consequences. My point is that we should not dismiss the physician who thought that he may have found something, as some kind of conspiracist. That's not how scientific progress is achieved. Thanks, Sabri
Re: Technology risk without safeguards
Hi, Not that I'm into conspiracy theories, or believe at this point that RF emissions are in any way related to cancer, but Suresh' statement is not very scientific: > This is an internet conspiracy theory with no basis in reality or science. RF emissions are absorbed by the human body. Your kitchen microwave works at the same frequency as your 2.4Ghz wifi. We all know it's a bad idea to put your head in a microwave oven. The hypothesis that RF may cause damage to human DNA is not at all conspiracy. The fact that we haven't been able to identify a factual relationship, does not mean that there isn't any. For example: > In large studies published in 2018 by the US National Toxicology Program (NTP) > and by the Ramazzini Institute in Italy, researchers exposed groups of lab > rats > (as well as mice, in the case of the NTP study) to RF waves over their entire > bodies for many hours a day, starting before birth and continuing for at least > most of their natural lives. Both studies found an increased risk of uncommon > heart tumors called malignant schwannomas in male rats, but not in female rats > (nor in male or female mice, in the NTP study). The NTP study also reported > possible increased risks of certain types of tumors in the brain and in the > adrenal > glands. Source: https://www.cancer.org/cancer/cancer-causes/radiation-exposure/radiofrequency-radiation.html > If your doctor suspected that you had cancer caused by something related to > microwave band communications equipment, you need to find a new doctor. On the contrary. Few people are more exposed to higher-powered RF radiation than a MW techie. That would make them an excellent subject for scientific research. Dismissing a medical professional's opinion based in your own firm beliefs is counterproductive to the advance of scientific knowledge. Thanks, Sabri, M.Sc - On Nov 4, 2020, at 2:01 PM, Matt Harris m...@netfire.net wrote: > My first instinct is to let this be because the level of conspiracy theory > nuttiness seems to be very high and the level of knowledge of basic physics > seems to be very low, but since this list is archived in a way that lay-people > may reference it at some point in the future, I'm going to go ahead and reply > just this once more and just one point here so that a lack of response here > won't be used as fodder by conspiracy theorists. > Matt Harris | Infrastructure Lead Engineer > 816‑256‑5446 | Direct > Looking for something? > [ https://help.netfire.net/ | Helpdesk Portal ] | [ > mailto:h...@netfire.net | > Email Support ] | [ https://my.netfire.net/ | Billing Portal ] > We build and deliver end‑to‑end IT solutions. > On Wed, Nov 4, 2020 at 2:48 PM Suresh Kalkunte < [ > mailto:sskalku...@gmail.com | > sskalku...@gmail.com ] > wrote: >> At an employer where I developed Wi-Fi based SOHO device, an adjacent group >> was >> testing Line of Sight transceivers. Nobody warned me of the inclement health >> (a >> general physician in 2007 suspected cancer looking at a blood test) from >> close >> quarters exposure to the side lobes emanating from the microwave radio. > There is no scientific evidence that RF emissions in the bands used for > communications have any causal relationship with cancer in humans. This is an > internet conspiracy theory with no basis in reality or science. If your doctor > suspected that you had cancer caused by something related to microwave band > communications equipment, you need to find a new doctor.
Re: NANOG SPAM (was Re: Just got this apparently fake NANOG invoice - Looks phishy)
- On Sep 21, 2020, at 6:03 PM, Bryan Fields br...@bryanfields.net wrote: Hi, > What's happening here is a subscription comes in from a valid email bot using > gmail or $BIGHOST (google doesn't give af) I'm old enough to remember the Usenet Death Penalty. That used to be pretty effective in dealing with sources of net-abuse. Thanks, Sabri
Re: BGP route hijack by AS10990
- On Aug 1, 2020, at 12:50 PM, Nick Hilliard n...@foobar.org wrote: Hi, > Sabri Berisha wrote on 01/08/2020 20:03: >> but because Noction's decision to not enable NO_EXPORT by default > > the primary problem is not this but that Noction reinjects prefixes into > the local ibgp mesh with the as-path stripped and then prioritises these > prefixes so that they're learned as the best path. Yeah, but that's not problem as far as I'm concerned. Their network, their rules. I've done weirder stuff than that, in tightly controlled environments. > The as-path is the primary loop detection mechanism in eBGP. Removing > this is like hot-wiring your electrical distribution board because you > found out you could get more power if you bypass those stupid RCDs. Well, let's be honest. Sometimes we need to get rid of that pesky mechanism. For example, when using BGP-as-IGP, the "allowas-in" disregards the as-path, in a controlled manner (and yes, I know, different use case). My point is that there can be operational reasons to do so, and whatever they wish to do on their network is perfectly fine. As long as they don't bother the rest of the world with it. Thanks, Sabri
Re: BGP route hijack by AS10990
Hi, - On Aug 1, 2020, at 8:49 AM, Owen DeLong o...@delong.com wrote: > In fact, there are striking parallels between Asiana 214 and this incident. Yes. Children of the magenta line. Depending on automation, and no clue what to do when the Instrument Landing System goes down. But, the most important parallel is (hopefully) yet to come. One major outcome of the Asiana investigation was the call for more training, as the crew did not properly understand how the aircraft worked. The same can be said here. Noction and/or its operators appear to not understand how BGP works, and/or what safety measures must be deployed to ensure that the larger internet will not be hurt by misconfiguration. I also agree with Job, that Noction has some responsibility here. And as I understand more and more about it, I must now agree with Mark T that this was an avoidable incident (although not because of Telia, but because Noction's decision to not enable NO_EXPORT by default). Thanks, Sabri
Re: BGP route hijack by AS10990
- On Jul 31, 2020, at 2:50 PM, Mark Tinka mark.ti...@seacom.com wrote: Hi Mark, > On 31/Jul/20 23:38, Sabri Berisha wrote: > >> Kudos to Telia for admitting their mistakes, and fixing their processes. > > It's great that they are fixing this - but this was TOTALLY avoidable. I'm not sure if you read their entire Mea Culpa, but they did indicate that the root cause of this issue was the provisioning of a legacy filter that they are no longer using. So effectively, that makes it a human error. We're going to a point where a single error is no longer causing outages, something very similar to my favorite analogy: avation. Pretty much every major air disaster was caused by a combination of factors. Pretty much every major outage these days is caused by a combination of factors. The manual provisioning of an inadequate filter, combined with an automation error on the side of a customer (which by itself was probably caused by a combination of factors), caused this issue. We learn from every outage. And instead of radio silence, they fessed up and fixed the issue. Have a look at the ASRS program :) Thanks, Sabri
