On Fri, Mar 7, 2014 at 5:27 PM, Bryan Davis <[email protected]> wrote:
> I recommend that we add two new weekly cleanup steps:
>
> * When we deploy a new branch to group0 (Thursdays), all branches
> retired more than 5 weeks ago should be removed. This should really
> only include multiple branches the first time it's done to catch up.
> After that it will be an "add a branch, kill a branch" situation. With
> the current release cadence this will keep us at 7 checked out
> branches on tin, 2 versions in active use and 5 waiting for potential
> cache references to expire.
I did this process today for the first time. I ended up leaving 8
branches on tin instead of 7. I removed these branches:
* 1.23wmf6
* 1.23wmf7
* 1.23wmf8
* 1.23wmf9
* 1.23wmf10
1.23wmf11 was last live on 2014-02-06. I should be safe to delete
today, but I didn't want to tempt fate or anger any deity that may
control front-end 404 generation.
Here's what I did to delete each branch:
* Cleanup bits symlinks to branch
** /a/common/multiversion/deleteMediaWiki php-1.23wmfX
* Create symlinks cleanup patch
** git rm -r docroot/bits/static-1.23wmfX
** git rm -r w/static-1.23wmfX
** NOLOGMSG=1 git commit -m 'Remove 1.23wmfX symlinks'
* Revert change on tin
** NOLOGMSG=1 git reset HEAD^ --hard
* Repeat steps above for each branch as needed
* Approve and pull symlink cleanup patch to tin
* Delete branch checkout in /a/common
** rm -rf /a/common/php-1.23wmfX
* Repeat steps above for each branch as needed
* scap
When scap runs it will leave the php-1.23wmfX/cache/l10n/*.cdb files
in place because we ignore *.cdb purposefully in the rsync command run
by sync-common. This can be cleaned up with dsh:
dsh -g mediawiki-installation -M -F 40 -- \
'sudo -u mwdeploy -- rm -r /usr/local/apache/common-local/php-1.23wmfX'
Chad has suggested that an additional step of tagging the end of the
branch and deleting it be added to this process. Right now there is a
minor blocker on that as the branch delete can only be done via the
gerrit web ui and requires user permissions that I do not currently
have. I will make the tags though and let Chad either delete the
branches or figure out how to let me and Sam do it.
> * When we move group1 to the newest branch (Tuesdays), the cache/l10n
> directory of all non-active branches should be purged. By this point
> there is little chance that we will be reverting the wikipedias to the
> N-2 branch and thus the l10n cache is just taking up disk space and
> slowing down rsync comparisons.
I did this procedure for the first time this week as well purging the
l10n cache for 1.23wmf13, 1.23wmf14 and 1.23wmf15. The process I
followed was:
* Delete l10n files in /a/common
** sudo -u l10nupdate rm --recursive ${COMMON}/php-${VERSION}/cache/l10n/*
* Delete l10n files on cluster hosts
** dsh "${MW_DSH_ARGS[@]}" -- \
"sudo -u mwdeploy rm --recursive ${SHARED}/php-${VERSION}/cache/l10n/*"
* !log deletion
This has been turned into a script in the scap project that is
awaiting code review [0]. Once it is merged the process will be
reduced to:
scap-purge-l10n-cache --version 1.23wmfX
[0]: https://gerrit.wikimedia.org/r/#/c/118337/
--
Bryan Davis Wikimedia Foundation <[email protected]>
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA
irc: bd808 v:415.839.6885 x6855
_______________________________________________
MediaWiki-Core mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-core
