165 Halsey recurring power issues

2023-10-23 Thread Babak Pasdar
Hello, I wanted to get some feedback as to what is considered standard A/B power setup when data centers sell redundant power.  It has always been my understanding that A/B power means individually unique and preferably alternate path connections to disparate UPS units. A few months ago,

Re: 165 Halsey recurring power issues

2023-10-23 Thread Sean Donelan
On Mon, 23 Oct 2023, James Jun wrote: "2N" generally means you're connected to completely different UPS system/complex and corresponding distribution systems for each of your circuit. This is ideal configuration for most critical loads. If you are in a single facility, even one with 2N+2

Re: 165 Halsey recurring power issues

2023-10-23 Thread William Herrin
On Mon, Oct 23, 2023 at 7:38 AM Babak Pasdar wrote: > A few months ago, 165 Halsey took us down for several hours. They > claimed that a UPS failed causing this issue. Our natural reaction was > that we have A/B redundant power so a failed UPS on the A circuit should > not take down the cabinet.

Re: BGP hijack?

2023-10-23 Thread Nicolas VUILLERMET
Hello, Because we were migrating our default table containing the DFZ into a VRF, we had a BGP session between 2 routers terminating on one side in the main table and on the other in the VRF. We had to remove the no-export from our redistribution route-map because of this private eBGP

Re: 165 Halsey recurring power issues

2023-10-23 Thread James Jun
On Mon, Oct 23, 2023 at 03:31:21PM -0400, Babak Pasdar wrote: > > Is the UPS the battery or the battery and controller combined? "N+1" nominally means you're connected to the same UPS system/complex, but each of your feed is on a different module. Your other leg will be diverse from a failure

Re: 165 Halsey recurring power issues

2023-10-23 Thread Aaron Wendel
I toured The Planet years ago in Dallas and was told by the sales rep that A+B power was two circuits from the same PDU. :) I consider A+B power to be two distinct feeds, separate utility entrances, separate generators, separate UPS', PDU's, etc.  Past that I consider things like firewall

Re: 165 Halsey recurring power issues

2023-10-23 Thread Eric Kuhnke
Bulk/high-volume hosting companies, dedicated server companies/small rack unit count colocation operate on very thin margins. Unless a customer is paying a LOT more per month they're not economically going to be connected to true diverse A/B power. In this case their use of the

Re: 165 Halsey recurring power issues

2023-10-23 Thread Sean Donelan
165 Halsey (and most of its tenant) data centers is an older facility. Data center practices have changed over the decades, and terminlogy wasn't standardized until recently. The biggest FUBAR in telco and data centers is the difference between "redundancy" and "diversity." Redundant

Re: 165 Halsey recurring power issues

2023-10-23 Thread Babak Pasdar
Thanks James, At signup we asked for N+1 power, two circuits to different UPS units. I think they sliced it thin by connecting us to two battery packs on the same UPS. When the UPS controller crashed both battery packs went down.  Which now raises the question -- is it reasonable to have to

Re: 165 Halsey recurring power issues

2023-10-23 Thread Jay Hennigan
On 10/23/23 15:56, Eric Kuhnke wrote: In this case their use of the incorrectly-described A/B was probably exclusively to handle the (not extremely rare) instances of rackmount server power supply failures, to give each 1U or 2U size machine, or rack of blades, two live power supplies with

RE: 165 Halsey recurring power issues

2023-10-23 Thread Tony Wicks
If you have been sold "redundant" power and the DC provider has connected both sides to one UPS in any form they are seriously amiss. You should not be expected to know the internal workings of the DC UPS systems and any talk of battery packs (unless you are getting 48v DC) is utterly

Re: 165 Halsey recurring power issues

2023-10-23 Thread James Jun
On Mon, Oct 23, 2023 at 10:38:09AM -0400, Babak Pasdar wrote: > I wanted to get some feedback as to what is considered standard A/B > power setup when data centers sell redundant power.?? It has always been > my understanding that A/B power means individually unique and preferably > alternate

Re: 165 Halsey recurring power issues

2023-10-23 Thread William Herrin
On Mon, Oct 23, 2023 at 3:56 PM Eric Kuhnke wrote: > Bulk/high-volume hosting companies, dedicated server companies/small > rack unit count colocation operate on very thin margins. Unless a > customer is paying a LOT more per month they're not economically > going to be connected to true diverse

Re: BGP hijack?

2023-10-23 Thread Ben Cox via NANOG
Hey everybody, I run bgp.tools, (And had a extremely busy alerting engine for a few minutes) >From what bgp.tools can see it seems like they had a private asn in the path like so ``` 2027 422027 6696 6939 42615 212232 ``` This can be valid for a number of reasons, ( they might have been

Re: BGP hijack?

2023-10-23 Thread Tyler Conrad
Thanks for the transparency, Vincent. Are you able to share how the AS-Path became mangled to begin with? I’m assuming this was some kind of route optimizer, but maybe something else going on? On Mon, Oct 23, 2023 at 07:32 wrote: > Hello everyone, > I'm working for MilkyWan / AS2027 and I

Re: RPKI unknown for superprefixes of existing ROA ?

2023-10-23 Thread Willy Manga
. On 23/10/2023 16:00, nanog-requ...@nanog.org wrote: Message: 19 Date: Sun, 22 Oct 2023 14:20:56 -0400 From: Amir Herzberg I agree that a good, sensible defense would be to simply announce your entire address block, e.g., in the example, your entire /22 (with a ROA to your ASN), and filter

Re: BGP hijack?

2023-10-23 Thread vincent
Hello everyone, I'm working for MilkyWan / AS2027 and I wanted to give you some explanations regarding this incident. Last week-end, during an upgrade on our network configuration, it appears that some prefixes were announced with an incorrect AS Path. Based on our analysis, none of these

Re: 165 Halsey recurring power issues

2023-10-23 Thread Eric Kuhnke
At which point one starts looking at the risk factors, if your whole facility is "redundant", is the power feed coming in from two geographically diverse substations, via diverse duct banks, into diverse entry vaults, and diverse risers? Doesn't eliminate the possibility of the entire building

Re: 165 Halsey recurring power issues

2023-10-23 Thread Eric Kuhnke
I didn't say that I have sympathy for it but that unfortunately this is considered acceptable practice within many low-budget "hosting" companies and probably has been for 15 years. It's a known risk when you're buying a $50/month "server". Same general category of problem as the OVH datacenter