Seth Mattinen wrote: I have a pure curiosity question for the NANOG crowd here. If you run your facility/datacenter/cage/rack on 120 volts, why?
I've been running my facility at 208 for years because I can get away with lower amperage circuits. I'm curious about the reasons for using high-amp 120 volt circuits to drive racks of equipment instead of low-amp 208 or 240 volt circuits And you have been doing something that is a right step in the right direction, but may well not be the best ultimate solution. Lets half ignore codes. Not to be illegal, but they CAN be changed and you can often get specific exceptions if you are not just in the back room of an office but are in a clearly professionally managed facility with well trained staff plugging in equipment. Every time I look at a nameplate and I see 100-250VAC I get very frustrated. If only that had been perhaps 100-300VAC, I could then run it on 277VAC and that is especailly nice for many reasons. Most large USA buildings already have 277 and probably all their flourescent lighting is run off it (277VAC ballasts are readily available and what look like rats-ass wall switches but are higher rated ones are readily available - both even at home despot if you look hard enough), so nothing terribly new has to be learned by electricians, etc. 277 is the phase leg to NEUTRAL voltage of a 277/480 WYE system that most everything except small human plugged appliances use in any but the very largest USA office building. It is what typically comes in from the power company. 120/208 is the output of typically a delta/wye transformer that steps that down for the dumb humans to safely use and you are paying the penalty of the WHOLE LOAD having to go through a second less than 100% efficient transformer. The beauty of 277 is that on a single breaker pole (unlike 208 where you are most likely to have 2 HOT legs and need a 2 pole simultaneous trip breaker) on reasonable size branch circuits that you are still allowed to plug MULTIPLE loads into without individual fuses or breakers (that is "allowed to" - you may chose to protect each outlet in the rack, but that is not compulsory) you get 277/120=2.31 times as much power available. Sadly routers, servers, switches, etc. typically are rated to 250VAC, so using raw 277 won't work. But let us see how close HP/IBM/ACP and many many others are getting still using ONE breaker pole per much more efficient branch circuit. NB that as you go to larger branch circuits in AMPS, you MUST be supplying just ONE load or MUST have additioanl breakers or fuses as you split it up. We all know 120/208 and 277/480. What about another NEW pair of voltages in WYE connection! Lets use 240/415. It is exactly twice 120/208 (well it is not stated as 240/416 I'd guess since 240 x 1.73259 = 415.82 they just truncate rather than round - though 2400/4160 is a standard designation...) and is inside the 250V max rating of the switching power supplies. It still uses a single breaker pole. Your get EXACTLY twice as much power out of a 240/415 WYE branch circuit as you would out of a 120/208 at the same AMPs. But you may save a transformer and its continuous power waste or at least part of it in between. How do you get to 240/415 is the next issue. If you have 2400/4160 or 7,960/13,800 primary into your building, and you do all your own transformers, getting 240/415(6) can be a single transformer step for you, and you will probably have many transformers so can also create seperate 277/480 for modest size AC inits and lighting, While LARGE chillers can be ordered at the higher voltages, and for the relatively small amount of 120/208 you probably should come off 277/480 into standard 120/208 delta wye transformers because normal electricians can do that rather then the 13K gods($$$$). But if you are a smaller building the only voltage that makes sense that the utility is supporting is 277/480. Rather than take all your rack power through another transformer step with the losses and the extra heat to eject from the building, consider instead using buck (as in the classic BOOST/BUCK transformers) to knock that 277/480 down to 240/415. It can be packaged as a 3 phase unit for less than three singles, and will be smaller and less costly to have wired up, but the three singles may be available from stock. It is the same sort of device you must have in front of a load that needs 240 or 250 and can't handle 208, but in that case is wired BOOSTING rather than BUCKING. FWIW an electric range burner or a hot water heater element rated for 240 produces EXACTLY 75% of the heat if run on 208 (go do the math...), but you should NOT use boost bucks for such a simple situation because optional heating elements can be ordered originally OR bought as replacements for less than $10 each and easily replaced in the field to give the original 240 rated wattage on 208 supply. In any case the 3 phase buck transformer VA rating will just need to be 3 x (277-240) x per-phase-AMPs on the load side. Or look at it this way: 37/240 = 15.4% to just buck rather than the KVA of a transformer dropping the whole load. Remember the buck transformer's secondary in this case simply is 37 volts and is wired in series but 180 degrees out of phase so drops the 277 to 240VAC But maybe that isn't wizest in the big picture but may work for you. If you are into really big systems you really need folks that know what they are doing. Those that simply tell you to buy K rated transformers may be missing a slick opportunity to knock out power factor problems caused by triplen harmonics related to the multigrounded neutral system. So you may not want to BUCK, but instead use an exceptional transformer SYSTEM. Look at these folks (this is just one PDF, they have plenty more, and a few dollars more spent with them can bring a rapid ROI just on power factor savings alone on the utility bill let alone the dramatically lower transformer losses) : http://mirusinternational.com/downloads/CAT-EC01-08-F10a_w.pdf NB that one of their tricks is to have the load split to separate outputs with different phase shifts within these special transformers so the troublesome harmonics are canceled and not reflected back into the primary. Even though newer power supplies will be more efficient and better power factor corrected that years ago, this is still an issue to be very aware of. Anyway, GOOGLE for 415 volts or 240/415 and these days you will find many hits from big names you already know. Maybe any "issues" with local codes can be solved or already have been. Some folks have been bucking down from 277 to more like 250 for years (to neutral, so single breaker) with the local inspector staying totally clueless at to what was "accidentally" happening. He was probably aware of plugged in PDUs with integral 277/480 to 120/208 DeltaWye transformers in use so probably assumed a lot of the higher voltage breaker panels were feeding more of that. A 30 amp 3 phase feed at 240/415 to a big power strip with 15 or 20 amp breakers for smaller groups of single phase IEC outlets is a STANDARD product from multiple sources and at the 80% allowed loading provides 240 x 30 x 3 x .8 = 17,280 VA and just two of these feeds gets you to almost 35 KVA per cabinet that drives folks to liquid CO2 cooling systems that can easily function in a datacenter without a raised floor, and do it without CRAC created hurricanes or huge air ducts trying to cool the room and failing. NB that such a power strip using IEC outlets uses EXACTLY the same cords as you would use for 120VAC - nothing weird other than NOONE is using the "WEIRD" North American plugs and outlets Well, almost. The power strip could be built for BOTH 120/208 AND 240/415 and would require NO outlet changes. Breakers, shoild be picked for the higher voltage. IEC outlets in these voltages are THE SAME - they just change between 15 and 20 amps and there are specials for high temperature usage. The one thing that should be done is to be sure be to use S cord rather than SJ cord (600V class vs 300V class - well even better use the "O"il option too, simply because it lasts a lot longer - SO cord) - or use a plastic equivalent for any cords carring more than one HOT from 240/415. The old code allowed 42 poles per breaker panel. That would be 7 cabinets worth of just these handy size 30 amp 3 phase feeds for 241920 VA total - almost 1/4 of a megawatt. Bigger branch circuit amps can give you a lot more, obviously, but short of going to Square-D I Line panels, you probably don't want much more load than this in a typical breaker panel. It would be "nice" to have the two 30 amp 240/415 WYE feeds to any cabinet be fed from the SAME breaker numbers in two totally separate breaker panels, and each server's power supplies split between the two branch circuits. Just be VERY VERY certain you check each device as there are still some 120 volt ONLY devices some idiot will want to plug in. Three phase isn't all that weird.. Keeping it simple, if you have three separate coils on a generator that each produce AC 120 degrees out of phase with each other, and connect one end of each coil to a single grounded point called "neutral" , you WON'T have twice the single coil voltage between any two of the other ends of the coils (the HOT wires), but have the square root of 3 times it instead because they are not a full 180 degrees out of phase the way single phase house power is, or two flash cells that when you ground the center connection between the cells and call it neutral, you will find 3 volts between the ends of the two cell battery and 1.5 from that middle to either end BUT with opposite polarity.from the center. The sign waves coming out of that 3 phase generator peak in succession at each hot leg in turn, and a three phase motor will reverse direction if you swap any two of the three wires. Just as on a single phase neutral where you can reduce the neutral current to ZERO if you have two identical resistive loads (ie non reactive) from each hot leg to neutral, so can you if you have three identical resistive loads each on one of the three phase hot legs to neutral. If you put one of those resistive loads on just one phase leg, obviously the NEUTRAL has the same current as the one in use phase leg. The trick question is how much is on neutral if you had just two of the three resistors connected. Think of it this way. If all three are on, neutral has ZERO amps. cut ONE off, and you just CHANGED what is on neutrasl by one resistor's worth of current. When we remove that resistor from the common neutral wire that was reading ZERO amps, we will get a one amp reading on neutral. If we were to turn that resistor back on and shut OFF the other two, we would also get 1 amp on neutral, BUT there would be an exact 180 degree phase shift if we were watching the two currents on a scope. Together they cancel. But but but why 180 when 3 phases are all 120 degrees apart? Those two other legs ARE 120 degrees apart and each 120 apart from the third leg, but those two 120 degrees apart currents sum to being 180 degrees from the third leg (still assuming each resistor is indentical). Anything other than resistors and current may lead or lag voltage and harmonics may add in ugly ways so neutral current can easily exceed that on any phase leg and thus we have all the power factor issues and charges from utilities and mandates for better power factor corrected supplies and codes dictating over sized neutrals where switching power supplies abound. I was cheerily suggesting hot leg to neutral WYE (aka STAR) connections ( and I still do!). But actually those using two hot pole branch circuits ( or two hots of a three phase branch ciruuit broken into many smaller two wire 208 ones ) are NOT using the neutral at all in each of those SINGLE PHASE connections and avoid the triplen harmonic currents summing in the neutral, and can also still use the neutral for small random 120V loads but need not oversize the neutral. Actually any load including three phase DELTA stuck across there that avoids the neutral avoids the harmonics issues. I was just using the 208 single phase loads because they seemed to have been used more than I had suspected. Using a three phase branch circuit (say 30 amps as that would be a handys size, but DOES NOT MATTER here) at 120/208 - whether you are using it as 3 120 volt circuits or 3 208 volt circuits ( as EACH of those connection options gets EXACTLY the same max power out of the circuit!!!) gives you EXACTLY 1/2 the power you could get from the same ampere circuit running 240/415 (which would normally ONLY be running phase to neutral loads at 240V, but were 415 volt capable devices available you still get the same total out - EXACTLY twice what the 120/208 can deliver). NOTE that I carefully was refering to 4 wire 3 phase legs and a neutral for these example branch circuits. IF you are comparing two wire single phase circuits from the breaker panel to the cabinet (how silly! unless it is from a PDU in an adjacent cabinet), 208 volts gets you about 13% less power on your two wires than you could get at the same amperage on a 2 wire circuit at 240 volts. Whether those wires are both hots or a hot and a neutral makes no matter as it is simply the voltage between those two wires times max allowed amps (80% of branch ckt rating) that determines what you can get out. A switching supply simply sucks more amps at lower input voltage so gets the power it needs in either case, but you can inflict a larger load at the higher voltage. Thus 240/415 (240/416...) DELIVERS twice what 120/208 does amp for amp with full three phase wiring, and that is why it is so great.
