Aaron Schulz has submitted this change and it was merged.
Change subject: (bug 27320) Add refreshMessageBlobs.php
......................................................................
(bug 27320) Add refreshMessageBlobs.php
The clearMessageBlobs.php script is a bit too heavy-handed: it
obliterates the entire ResourceLoader message caches for all wikis,
which can lead to a database stampede as user requests try to repopulate
the cache all at once.
refreshMessageBlobs.php takes a more nuanced approach: It iterates over
all the cache entries and updates only those where the message cdb has
changed. And it does this one resource at a time, rather than having
every client connection try to do it in parallel.
The downside is that this is a good bit slower.
Bug: 27320
Change-Id: I3b6ae12875f2f323210fdfba36c5c5d9183588e2
---
A refreshMessageBlobs.php
1 file changed, 74 insertions(+), 0 deletions(-)
Approvals:
Aaron Schulz: Verified; Looks good to me, approved
jenkins-bot: Checked
diff --git a/refreshMessageBlobs.php b/refreshMessageBlobs.php
new file mode 100644
index 0000000..60db7e9
--- /dev/null
+++ b/refreshMessageBlobs.php
@@ -0,0 +1,74 @@
+<?php
+
+/**
+ * Refresh the msg_resource table when cdb message files have been updated
+ */
+
+require_once( __DIR__ . '/WikimediaMaintenance.php' );
+
+class RefreshMessageBlobs extends WikimediaMaintenance {
+ function execute() {
+ global $IP;
+
+ # Get modification timesatmp for English (fallback) from the
l10n cache
+ $enModTime = filemtime( "$IP/cache/l10n/l10n_cache-en.cdb" );
+ $langModTime = array( 'en' => $enModTime );
+
+ # To avoid cache stampede, fetch all the non-empty resource
message
+ # blobs and update them one at a time manually. To avoid excess
memory
+ # usage in LocalisationCache, order by language and clear the
cache
+ # between each language.
+ $dbr = wfGetDB( DB_SLAVE );
+ $dbw = wfGetDB( DB_MASTER );
+ $res = $dbr->select( 'msg_resource',
+ array( 'mr_resource', 'mr_lang', 'mr_blob',
'mr_timestamp' ),
+ "mr_blob != '{}'",
+ __METHOD__,
+ array( 'ORDER BY' => 'mr_lang' )
+ );
+ $prevLang = false;
+ foreach ( $res as $row ) {
+ # Check modification time for this language
+ if ( !isset( $langModTime[$row->mr_lang] ) ) {
+ $file =
"$IP/cache/l10n/l10n_cache-$row->mr_lang.cdb";
+ if ( file_exists( $file ) ) {
+ $langModTime[$row->mr_lang] =
filemtime( $file );
+ } else {
+ $langModTime[$row->mr_lang] =
$enModTime;
+ }
+ }
+ if ( wfTimestamp( TS_UNIX, $row->mr_timestamp ) >=
$langModTime[$row->mr_lang] ) {
+ continue;
+ }
+
+ # Clear LocalisationCache of the old language to reduce
memory usage
+ if ( $prevLang !== false && $prevLang !== $row->mr_lang
) {
+ Language::getLocalisationCache()->unload(
$prevLang );
+ }
+ $prevLang = $row->mr_lang;
+
+ # Update message blob. Even though we read from a slave
and are
+ # writing to master, it should be safe because we're
including
+ # mr_timestamp in the WHERE clause.
+ $messages = FormatJson::decode( $row->mr_blob, true );
+ foreach ( $messages as $key => $value ) {
+ $messages[$key] = wfMessage( $key
)->inLanguage( $row->mr_lang )->plain();
+ }
+ $dbw->update( 'msg_resource',
+ array(
+ 'mr_blob' => FormatJson::encode(
(object)$messages ),
+ 'mr_timestamp' => $dbw->timestamp(),
+ ),
+ array(
+ 'mr_resource' => $row->mr_resource,
+ 'mr_lang' => $row->mr_lang,
+ 'mr_timestamp' => $row->mr_timestamp,
+ ),
+ __METHOD__
+ );
+ }
+ }
+}
+
+$maintClass = 'RefreshMessageBlobs';
+require_once( DO_MAINTENANCE );
--
To view, visit https://gerrit.wikimedia.org/r/58909
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3b6ae12875f2f323210fdfba36c5c5d9183588e2
Gerrit-PatchSet: 3
Gerrit-Project: mediawiki/extensions/WikimediaMaintenance
Gerrit-Branch: master
Gerrit-Owner: Anomie <[email protected]>
Gerrit-Reviewer: Aaron Schulz <[email protected]>
Gerrit-Reviewer: Anomie <[email protected]>
Gerrit-Reviewer: Catrope <[email protected]>
Gerrit-Reviewer: PleaseStand <[email protected]>
Gerrit-Reviewer: Tim Starling <[email protected]>
Gerrit-Reviewer: jenkins-bot
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
