Re: [Wikitech-l] Reducing the environmental impact of the Wikimedia movement
On Thu, Mar 31, 2016 at 12:39 AM, Tim Starling wrote: > I think it's stretching the metaphor to call ops a "tight ship". We > could switch off spare servers in codfw for a substantial power > saving, in exchange for a ~10 minute penalty in failover time. But it > would probably cost a week or two of engineer time to set up suitable > automation for failover and periodic updates. > Just a small clarification: I don't think turning off and on periodically servers would be a feasible option because servers (and computers in general) tend to have a pretty high failure rate when being powered off and on regularly. We see this with some server failing every time we do a mass reboot due to some security issue. On the other hand, we could surely do better in terms of idle-server power consumption. In terms of costs and time spent (and probably also natural resources consumption, but I did no calculation whatsoever) it would probably be not sustainable. > Or we could have avoided a hot spare colo altogether, with smarter > disaster recovery plans, as I argued at the time. Another small clarification: our codfw datacenter is _not_ just a hot spare for disaster recovery and a lot of work has been done to make the two facilities mostly active-active (and a lot more will be done in the coming year). Cheers, Giuseppe P.S. The server energy footprint of the WMF is negligible if compared to the big internet players, but even a small-medium size local ISP has probably a larger footprint than us. This doesn't mean we should not try to get better, but we should always put things in prespective. -- Giuseppe Lavagetto Senior Technical Operations Engineer, Wikimedia Foundation ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Linker::link() rewrite
Hi, For the past few weeks I've been working[1] on rewriting Linker::link() to be non-static, use LinkTarget/TitleValue and some of the other fancy new services stuff. Yay! For the most part, you'd use it in similar ways: Linker::link( $title, $html, $attribs, $query ); is now: $linkRenderer = MediaWikiServices::getInstance() ->getHtmlPageLinkRenderer(); $linkRenderer->makeLink( $title, $html, $attribs, $query ); And there are makeKnownLink() and makeBrokenLink() entry points as well. Unlike Linker::link(), there is no $options parameter to pass in every time a link needs to be made. Those options are set on the HtmlPageLinkRenderer instance, and then applied to all links made using it. MediaWikiServices has an instance using the default settings, but other classes like Parser will have their own that should be used[2]. I'm also deprecating the two hooks called by Linker::link(), LinkBegin and LinkEnd. They are being replaced by the mostly-equivalent HtmlPageLinkRendererBegin and HtmlPageLinkRendererEnd hooks. More details are in the commit message. [3] is an example conversion for Wikibase. The commit is still a WIP because I haven't gotten around to writing specific tests for it (it passes all the pre-existing Linker and parser tests though!), and will be doing that in the next few days. Regardless, reviews / comments / feedback on [1] is appreciated! [1] https://gerrit.wikimedia.org/r/#/c/284750/ [2] https://gerrit.wikimedia.org/r/#/c/288572/ [3] https://gerrit.wikimedia.org/r/#/c/288674/ -- Legoktm ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Proposal to invest in Phabricator Calendar
On Sat, 2016-05-14 at 20:51 +0200, Ricordisamoa wrote: > If we're going to be investing money into improving Phabricator > upstream, I think we should start with making Differential usable > (i.e. a suitable replacement for Gerrit) If you have *specific* issues, please point them out by linking to tasks. "Usable" is too subjective to be a basis for discussions. Thanks! andre -- Andre Klapper | Wikimedia Bugwrangler http://blogs.gnome.org/aklapper/ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reducing the environmental impact of the Wikimedia movement
Lukas Mezger wrote: > With the help of Juliet Barbara and Gregory Varnum, we now have detailed > public figures regarding the energy use and energy sources of the Wikimedia > servers: As of May 2016, the servers use 222 kW, summing up to about 2 GWh > of electrical energy per year. For more information, please see > https://meta.wikimedia.org/wiki/Environmental_impact. > The next step would be to figure out the cost and feasibility of having the > servers run on 100% renewable energy. I'd appreciate it if someone could > help me find out how this works. As a European consumer, I can order > renewable energy for my house simply by calling my energy company on the > phone, with the price difference being negligible. I assume it is not just > as easy in our case, right? At Hawaii consumer prices, 2 GWh equals less than US-$ 800,000; that would be roughly 1 % of the Wikimedia Foundation budget. Don't you think it would be much better for *actually* reducing the environmental impact to start on the 99 % (or probably more like 99.5 %)? It would certainly be cheaper than paying *more* for energy. Tim ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reducing the environmental impact of the Wikimedia movement
Dear readers of the Wikitech mailing list, With the help of Juliet Barbara and Gregory Varnum, we now have detailed public figures regarding the energy use and energy sources of the Wikimedia servers: As of May 2016, the servers use 222 kW, summing up to about 2 GWh of electrical energy per year. For more information, please see https://meta.wikimedia.org/wiki/Environmental_impact. The next step would be to figure out the cost and feasibility of having the servers run on 100% renewable energy. I'd appreciate it if someone could help me find out how this works. As a European consumer, I can order renewable energy for my house simply by calling my energy company on the phone, with the price difference being negligible. I assume it is not just as easy in our case, right? Thank you, Lukas 2016-03-31 0:47 GMT+02:00 Katherine Maher : > Thanks Tim for clarifying. > > On Wed, Mar 30, 2016 at 3:39 PM, Tim Starling > wrote: > > > On 31/03/16 02:55, Katherine Maher wrote: > > > IIRC, we included clean energy consumption as a factor in > > > evaluating in our RFC for our choice of a backup colo a few years back > > > > Since I strongly support emissions reduction, on my own initiative I > > did an analysis of expected CO2 emissions of each of the candidate > > facilities during the selection process of the backup colo. That's > > presumably what you're referring to. > > > > < > > > https://docs.google.com/spreadsheets/d/1adt45Msw2o8Ml0s8S0USm9QLkW9ER3xCPkU9d2NJS4Y/edit#gid=0 > > > > > > > My conclusion was that codfw (the winner) was one of the worst > > candidates for CO2 emissions. However, the price they were offering > > was so much lower than the other candidates that I could not make a > > rational case for removing it as an option. You could buy high-quality > > offsets for our total emissions for much less than the price difference. > > > > However, this observation does require us to actually purchase said > > offsets, if codfw is to be represented as an ethical choice, and that > > was never done. > > > > codfw would not tell us their PUE, apparently because it was a > > near-empty facility and so it would have technically been a very large > > number. I thought it would be fair to account for marginal emissions > > assuming a projected higher occupancy rate and entered 2.9 for them, > > following a publication which gave that figure as an industry average. > > It's a new facility, but it's not likely that they achieved an > > industry-leading PUE since the climate in Dallas is not suitable for > > evaporative cooling or "free" cooling. > > > > > Ops runs a tight ship, and we're a relatively small footprint in our > > colos, > > > so we don't necessarily have the ability to drive purchasing decisions > > > based on scale alone. > > > > I think it's stretching the metaphor to call ops a "tight ship". We > > could switch off spare servers in codfw for a substantial power > > saving, in exchange for a ~10 minute penalty in failover time. But it > > would probably cost a week or two of engineer time to set up suitable > > automation for failover and periodic updates. > > > > Or we could have avoided a hot spare colo altogether, with smarter > > disaster recovery plans, as I argued at the time. My idea wasn't > > popular: Leslie Carr said she would not want to work for an > > organisation that adopted the relaxed DR restoration time targets that > > I advocated. And of course DR improvements were touted many times as > > an effective use of donor funds. > > > > Certainly you have a point about scale. Server hardware has extremely > > rudimentary power management -- for example when I checked a couple of > > years ago, none of our servers supported suspend-to-RAM, and idle > > power usage hardly differed from power usage at typical load. So the > > only option for reducing power usage of temporarily unused servers is > > powering off, and powering back on via out-of-band management. WMF > > presumably has little influence with motherboard suppliers. But we > > could at least include power management and efficiency as > > consideratons when we evaluate new hardware purchases. > > > > > At the time the report came out, we started talking to Lukas about how > we > > > could improve our efforts at the WMF and across the movement, but we've > > had > > > limited bandwidth to move this forward in the Foundation (and some > > > transitions in our Finance and Operations leadership, who were acting > as > > > executive sponsors). However, I think it's safe to say that we'd like > to > > > continue to reduce our environmental impact, and look forward to the > > > findings of this effort. > > > > We could at least offset our datacentre power usage, that would be > > cheap and effective. > > > > -- Tim Starling > > > > > > ___ > > Wikitech-l mailing list > > Wikitech-l@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > > > -- > Katherine M
[Wikitech-l] Prioritizing Phabricator improvements (was Re: Proposal to invest in Phabricator Calendar)
After a first wave of feedback it is clear that we have two different discussions that need solving: 1. If we use WMF Technical Collaboration budget for Phabricator improvements, should we invest it the Calendar application? This is a well framed discussion with a clear deadline for decisions (very soon, or our FY2015-16 budget will be gone). I have created a task to discuss this specifically: https://phabricator.wikimedia.org/T135327 2. If we want to improve the announcement and promotion of Wikimedia tech events, is the best first step to improve Phabricator Calendar? This is a complex discussion with many ramifications that we can keep discussing at https://phabricator.wikimedia.org/T1035 On Sat, May 14, 2016 at 3:06 AM, Legoktm wrote: > If we're going to be investing money into improving Phabricator upstream > (a great idea IMO), I think we should start with problem areas that > affect a large number of users/developers. Agreed. If I am proposing improvements around Calendar is because our team believes that we have a problem affecting the announcement and promotion of technical activities beyond the circle of usual and core contributors. By improving this area, we believe we could reach better to more casual technical contributors in our movement, and reach out better to new contributors out there. > There's plenty of low-hanging fruit like non-drag-n-drop file uploads[1]. Agreed. This one is being funded already. > [2] was also mentioned on #wikimedia-tech a few days ago Good, I wasn't aware. Is there a task in Wikimedia Phabricator reflecting the level of need, support, consensus? Feel free to propose it as suggested in https://phabricator.wikimedia.org/T135327 > or some of the UI/UX issues Nemo brought up after the last Phabricator > upgrade[3]. > I think https://secure.phabricator.com/T10926 reflects the essence of the improvements requested after the Phabricator UX update. It looks like the discussion so far is more about agreement on UX problems/solutions than complexity of the solution once agreed, but if there is room for funded prioritization, that also looks like a good candidate for a proposal. [1] https://phabricator.wikimedia.org/T165#2289766 > [2] https://secure.phabricator.com/T10691#167705 > [3] https://lists.wikimedia.org/pipermail/wikitech-l/2016-May/085489.html > -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l