[Wikitech-l] Reminder: Gerrit downtime in 30 minutes

2013-09-04 Thread Chad Horohoe
Hi all, Just a friendly reminder that we're taking down Gerrit in about 30 minutes for a planned migration to a new (bigger) server. We've allotted an hour for the changeover but I'm not expecting it to take that long. I'll be sure to let everyone know when it's back up and running. -Chad

Re: [Wikitech-l] Reminder: Gerrit downtime in 30 minutes

2013-09-04 Thread Chad Horohoe
On Wed, Sep 4, 2013 at 3:30 PM, Chad Horohoe choro...@wikimedia.org wrote: Hi all, Just a friendly reminder that we're taking down Gerrit in about 30 minutes for a planned migration to a new (bigger) server. We've allotted an hour for the changeover but I'm not expecting it to take

Re: [Wikitech-l] Reminder: Gerrit downtime in 30 minutes

2013-09-04 Thread Chad Horohoe
On Wed, Sep 4, 2013 at 4:49 PM, Chad Horohoe choro...@wikimedia.org wrote: On Wed, Sep 4, 2013 at 3:30 PM, Chad Horohoe choro...@wikimedia.orgwrote: Hi all, Just a friendly reminder that we're taking down Gerrit in about 30 minutes for a planned migration to a new (bigger) server. We've

Re: [Wikitech-l] Reminder: Gerrit downtime in 30 minutes

2013-09-05 Thread Chad Horohoe
Some people have been reporting some intermittent problems this afternoon. Am investigating. -Chad On Sep 4, 2013 5:31 PM, Chad Horohoe choro...@wikimedia.org wrote: On Wed, Sep 4, 2013 at 4:49 PM, Chad Horohoe choro...@wikimedia.orgwrote: On Wed, Sep 4, 2013 at 3:30 PM, Chad Horohoe choro

[Wikitech-l] CirrusSearch on mediawiki.org

2013-09-06 Thread Chad Horohoe
Hi, So on Tuesday, August 10th I'm going to go ahead and turn on CirrusSearch as the default for mw.org. There's still some bugs[0], and I'm sure people will find more[1]. Once it's default you can revert to the old lucene backend using srbackend=LuceneSearch like you can do now with Cirrus.

[Wikitech-l] [MediaWiki-announce] First release candidate for 1.27 (1.27.0-rc.0)

2016-06-01 Thread Chad Horohoe
Hello everyone, I am happy to announce the availability of the first release candidate of the new MediaWiki 1.27 series! MediaWiki 1.27 includes all changes released in the smaller 1.27.0-wmf.* software deployments to Wikimedia sites over six months, totaling approximately 2800+ commits by

[Wikitech-l] [MediaWiki-announce] Second release candidate for 1.27 (1.27.0-rc.1)

2016-06-14 Thread Chad Horohoe
Hello everyone, I am happy to announce the availability of the second release candidate of the new MediaWiki 1.27 series! AuthManager has been backported!!! Since the first release candidate support for the new authentication/ authorization framework (AuthManager) was merged from master. This

[Wikitech-l] 1.28.0-wmf.0 on hold for now

2016-05-11 Thread Chad Horohoe
Hi, When we deployed the first 1.28 release to the cluster yesterday, we got a new error[0] relating to unserialization of redis data. It's pretty spammy already, so I'm paranoid about deploying wider until we figure out why. Deploying some debugging work soon so we can figure out what's going

Re: [Wikitech-l] [Engineering] 1.28.0-wmf.0 on hold for now

2016-05-12 Thread Chad Horohoe
d work, diagnosing this and writing up a fix on > Phabricator. I turned that into a patch and deployed it about an hour ago. > There haven't been any more errors since then. > > On Wed, May 11, 2016 at 1:59 PM, Chad Horohoe <choro...@wikimedia.org> > wrote: > >> Hi, &

Re: [Wikitech-l] [Engineering] 1.28.0-wmf.0 on hold for now

2016-05-13 Thread Chad Horohoe
On Fri, May 13, 2016 at 6:53 AM, Brad Jorsch (Anomie) wrote: > On Thu, May 12, 2016 at 7:33 PM, Erik Bernhardson < > ebernhard...@wikimedia.org> wrote: > >> It looks like the train didn't roll forward today? I was going to swat >> out a new test that depends on

Re: [Wikitech-l] [Engineering] Gerrit 2.12.2 test instance - PLEASE TEST

2016-07-13 Thread Chad Horohoe
On Tue, Jul 12, 2016 at 10:53 PM, Mukunda Modell wrote: > So I'll start by ignoring horrible UI/UX which, IMO, hasn't improved very > much over the old version. > > I've used the test install for various routine activities and so far I've > only found one bug: > > When I

Re: [Wikitech-l] Gerrit downtime this Monday!

2016-07-24 Thread Chad Horohoe
Friendly reminder that this is happening in about 2 hours! -Chad On Thu, Jul 21, 2016 at 8:08 AM, Chad Horohoe <choro...@wikimedia.org> wrote: > Hi, > > So the time is upon us to finally upgrade Gerrit. I thank everyone in > advance for all of your patience > and good wo

Re: [Wikitech-l] Gerrit downtime this Monday!

2016-07-24 Thread Chad Horohoe
/known_hosts if your client does have trouble with it. Thanks again to Daniel, Antoine and Jaime for their help today. -Chad On Sun, Jul 24, 2016 at 4:02 PM, Chad Horohoe <choro...@wikimedia.org> wrote: > Friendly reminder that this is happening in about 2 hours! > > -Chad > > > O

[Wikitech-l] CI is having issues, hang with us

2016-08-10 Thread Chad Horohoe
Hi, Nodepool isn't happy, so our CI is having trouble spinning up new instances for testing. We're well aware and working on it. Since a couple of people have already asked us what they can do to help...best thing you can do is avoid pushing changes for review (if they're not urgent) for a bit.

[Wikitech-l] Gerrit 2.12.2 test instance - PLEASE TEST

2016-07-12 Thread Chad Horohoe
Hi, Daniel and I have been spending a lot of time in the last week preparing a smooth upgrade path for Gerrit to a new (and supported version). The migration will be coming soon but I could use your help testing things in the meantime. https://gerrit-new.wikimedia.org/r/ The database is a

[Wikitech-l] Gerrit downtime this Monday!

2016-07-21 Thread Chad Horohoe
Hi, So the time is upon us to finally upgrade Gerrit. I thank everyone in advance for all of your patience and good work testing things out for us. The plan is outlined in detail on Phabricator[0], but I'll give everyone the short version here. The downtime will be on Monday, July 25th from

[Wikitech-l] [MediaWiki-announce] 1.25 EOL announcement

2016-06-28 Thread Chad Horohoe
Hi, With the release of MediaWiki 1.27, the lifetime of MediaWiki version 1.25.x has come to an end. Users still using MediaWiki 1.25.x are advised to upgrade to version 1.27.0, the latest stable and LTS version. --Chad Horohoe ___ MediaWiki

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.27 now available!

2016-06-28 Thread Chad Horohoe
Hello everyone, I am happy to announce the availability the general release of MediaWiki 1.27! MediaWiki 1.27 includes all changes released in the smaller 1.27.0-wmf.* software deployments to Wikimedia sites over six months, totaling approximately 2800+ commits by around 165 authors. This is a

[Wikitech-l] CI outage - ongoing

2016-07-05 Thread Chad Horohoe
Hi folks, Right now our CI infrastructure (Zuul/Jenkins/Nodepool) are having a bad day and aren't able to spawn new instances to perform tests. The outage is ongoing and there isn't an ETA for restoration of service just yet. In the meantime: please avoid force-merging (doing the Verified+2

Re: [Wikitech-l] [Engineering] MediaWiki-Vagrant needs testers for Debian Jessie changes

2017-02-14 Thread Chad Horohoe
On Tue, Feb 14, 2017 at 6:53 PM Bryan Davis wrote: > Give things a try and report issues as children of the tracking task > for this migration > Works for me! Using OSX 10.12.3 using VirtualBox 5.1.14 and Vagrant 1.8.5 Would test with my Parallels install, but it's out of

[Wikitech-l] [MediaWiki-announce] Security Release Pre-Announcement (1.27, 1.26, 1.23)

2016-08-19 Thread Chad Horohoe
Hi everyone, I would like to give everyone advance notice of a pending security release for all supported branches. The release will come out on Monday, August 22nd. There are several security fixes for core and one extension that will be included in this release, along with the normal backports

Re: [Wikitech-l] [Engineering] Gerrit was down today

2016-10-06 Thread Chad Horohoe
Hi! Sorry for the extended downtime! From what we can tell, it appears as though the machine that Gerrit is running on (lead) is having some hardware issues that are making the CPU misbehave. We've worked around it for now, so things should be up (and Zuul is processing CI events just fine).

[Wikitech-l] [MediaWiki-announce] Security Release - 1.27.1, 1.26.4, 1.23.15

2016-08-22 Thread Chad Horohoe
I would like to announce the release of MediaWiki 1.27.1, 1.26.4, 1.23.15! These releases fix five security issues in core and one for the extension PdfHandler. Download links are given at the end of this email. == Security fixes == * (T139565) API: Generate head items in the context of the

[Wikitech-l] [MediaWiki-announce] 1.26 EOL announcement

2016-11-28 Thread Chad Horohoe
Hi, With the release of MediaWiki 1.28, the lifetime of MediaWiki version 1.26.x has come to an end. Users still using MediaWiki 1.26.x are advised to upgrade to version 1.28.0, the latest stable version. -Chad ___ MediaWiki announcements mailing list

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.28 now available!

2016-11-28 Thread Chad Horohoe
Hello everyone, I am happy to announce the availability the general release of MediaWiki 1.28! MediaWiki 1.28 includes all changes released in the smaller 1.28.0-wmf.* software deployments to Wikimedia sites over six months, totaling approximately 2100 commits by around 120 authors. This is a

Re: [Wikitech-l] [MediaWiki-announce] MediaWiki 1.28 now available!

2016-11-28 Thread Chad Horohoe
On Mon, Nov 28, 2016 at 12:08 PM Chad Horohoe <choro...@wikimedia.org> wrote: > Hello everyone, > > I am happy to announce the availability the general release of MediaWiki > 1.28! > > Hi! One minor note. The RELEASE-NOTES-1.28 was not updated prior to re

[Wikitech-l] [MediaWiki-announce] Second release candidate for 1.28 (1.28.0-rc.1)

2016-11-17 Thread Chad Horohoe
Hiya! I am happy to announce the availability of the second release candidate of the new MediaWiki 1.28 series! Sorry for the delay, it should've been out last week. Other things got in the way. There's not been a lot of churn on 1.28-related stuff, so I still anticipate doing the final release

Re: [Wikitech-l] [Engineering] Gerrit was down today

2016-10-12 Thread Chad Horohoe
Heya! Gonna reboot Gerrit real quick this morning. Turns out "cobalt" did not have hyperthreading turned on. Services should be back momentarily! -Chad On Fri, Oct 7, 2016 at 2:07 PM Daniel Zahn wrote: > The Gerrit migration is over. It is back up and served from new

[Wikitech-l] Gerrit upgrade (w/ short downtime) tomorrow

2016-12-06 Thread Chad Horohoe
Hi all, Tomorrow evening (18:30–19:30 UTC) I'll be taking Gerrit offline shortly for a planned upgrade from 2.12.5 to 2.13.3. It shouldn't take the full hour, but you never know :) If you play the deployment game regularly, you'll notice I've cancelled[0] tomorrow's "SF Morning SWAT" window

[Wikitech-l] Gerrit: login issues for some (hopefully only a few)

2016-12-07 Thread Chad Horohoe
Hi, We upgraded to Gerrit 2.13.3 today from 2.12.2. However, a couple of users have reported being unable to login. Based on reports and investigation, I'm thinking/hoping this is only affecting 11 users. However, if you find yourself unable to login with the message "Cannot assign username to

[Wikitech-l] Gerrit downtime tomorrow evening / early morning

2017-01-04 Thread Chad Horohoe
Hi, Doing a (minor) point release upgrade to Gerrit tomorrow evening to deal with a data loss bug we can potentially hit. While we're offline, we're going to push out a change that enables logging to Logstash. Window is 17:00-19:00 SF time (that's 01:00-03:00 UTC). Thanks for your patience :)

[Wikitech-l] [MediaWiki-announce] 1.23.17 bugfix release

2017-04-11 Thread Chad Horohoe
Hello! I would like to announce the release of MediaWiki 1.23.17. There was an issue in the 1.23.16 release that prevented it from running on PHP 5.3 installations. Considering the report wasn't very widespread, I'm guessing most people haven't been affected yet. If you are, this release will fix

[Wikitech-l] [MediaWiki-announce] Security Release: 1.28.1 / 1.27.2 / 1.23.16

2017-04-06 Thread Chad Horohoe
Hello! I would like to announce the release of MediaWiki 1.28.1, 1.27.2 and 1.23.16! These releases fix five security issues in core and one for the extension SyntaxHighlight_GeSHi. Download links are given at the end of this email. Please note that next month is the End-Of-Life date for

[Wikitech-l] [MediaWiki-announce] Security pre-release announcement: 1.28.1 / 1.27.2 / 1.23.16

2017-04-06 Thread Chad Horohoe
Hi, Tomorrow, April 6th we will be performing a security release of MediaWiki for all supported branches. The new versions will be 1.28.1, 1.27.2 and 1.23.16. This will resolve 9 issues in MediaWiki core, and one in a bundled extension. Have a great day, -- Chad Horohoe & Sam

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.29 Available!

2017-07-13 Thread Chad Horohoe
Hello everyone, I am happy to announce the availability the general release of MediaWiki 1.29! MediaWiki 1.29 includes all changes released in the smaller 1.29.0-wmf.* software deployments to Wikimedia sites over six months, totaling approximately 1640 commits by around 140 authors. This is a

Re: [Wikitech-l] [Ops] Change to version number incrementing for weekly Wikimedia MediaWiki deploys

2017-07-11 Thread Chad Horohoe
I think a branch that's never deployed is likely to confuse people more, SWAT immediately comes to mind (which branches do I port to?) And while it's fixable, it would mean a non-checked-out branch wouldn't get automatically cleaned up during our weekly pruning. All in all: I like this move. It

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.29.0-rc.1 now available

2017-07-12 Thread Chad Horohoe
https://releases.wikimedia.org/mediawiki/1.29/mediawiki-core-1.29.0-rc.1.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.0-rc.1.patch.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html -- Chad Horohoe ___ MediaWiki

[Wikitech-l] [MediaWiki-announce] 1.29.0-rc.0 released, timeline on 1.29.0, EOL of 1.23.x

2017-05-31 Thread Chad Horohoe
-1.29.0-rc.0.tar.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html -- Chad Horohoe ___ MediaWiki announcements mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce

[Wikitech-l] [MediaWiki-announce] Security release 1.27.3 and 1.28.2

2017-04-30 Thread Chad Horohoe
Hello! I would like to announce the immediate availability of security releases 1.27.3 and 1.28.2. Due to a mistake in packaging, the releases 1.27.2 and 1.28.1 did not contain the fix for SyntaxHighlight_GeSHi. This new release does contain that fix. The task in question:

[Wikitech-l] [MediaWiki-announce] 1.28.x End of Life

2017-11-03 Thread Chad Horohoe
by the end of the month, we've already branched and are working on making sure the release is ready-to-go :) -- Chad Horohoe ___ MediaWiki announcements mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.31.0 now available!

2018-06-14 Thread Chad Horohoe
Hello everyone, I am happy to announce the availability of the general release of MediaWiki 1.31! MediaWiki 1.31 includes all changes released in the smaller 1.31.0-wmf.* software deployments to Wikimedia sites over six months, totaling over 2100 commits by roughly 130 unique authors. This is a

[Wikitech-l] [MediaWiki-announce] 1.31.0-rc.2

2018-05-30 Thread Chad Horohoe
-- Chad Horohoe ___ MediaWiki announcements mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https

[Wikitech-l] [MediaWiki-announce] MediaWiki 1.31.0-rc.0 now available

2018-05-03 Thread Chad Horohoe
signatures: https://releases.wikimedia.org/mediawiki/1.31/mediawiki-core-1.31.0-rc.0.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.31/mediawiki-1.31.0-rc.0.tar.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html -- Chad Horohoe ___ MediaWiki

[Wikitech-l] Gerrit upgrade - Friday 2/2

2018-01-31 Thread Chad Horohoe
Howdy Gerriters! On Friday, I'll finally be upgrading Gerrit from 2.13.9 to 2.14.6. Going to do this from 21:00–23:00 UTC. It shouldn't take the full two hours, but hey it's Gerrit. https://phabricator.wikimedia.org/T156120 <3 -Chad ___ Wikitech-l

[Wikitech-l] Train halted - Cirrus and DB issues

2018-02-07 Thread Chad Horohoe
Hiya! I halted the train this afternoon. tl;dr: I rolled wmf.20 to group1 and I saw a huge spike in DB replag and Cirrus errors. Rolling back made them go away! There are tasks: https://phabricator.wikimedia.org/T186765 https://phabricator.wikimedia.org/T186764 I think Eric/Stas already fixed