jenkins-bot has submitted this change and it was merged.
Change subject: Speed up saneitizeJobs.php
......................................................................
Speed up saneitizeJobs.php
On the production cluster fetching the status seems to take
extaordinarily long. Overall running saneitizeJobs.php --show takes 4 to
5 seconds. Similarly the cron job to insert saneitize jobs takes nearly
an hour and a half to run. Poking around it seems almost the entirety of
that time is spent waiting for elasticsearch to respond to the status
requests. With just shy of 900 wikis in production, doing a foreachwiki
against saneitizeJobs.php takes over 60 minutes due to this.
Speed things up by using the Index::exists() call, which performs a HEAD
request against the specific index, instead of requesting the full
server status. In a quick test on terbium this brings the runtime down
from 5s to around .6s. Shaving 4s off each execution should reduce the
time it takes to loop through the wikis in cron by ~60 minutes.
Change-Id: I3e0936074bc7679f56be737a397b00ad3be44b15
---
M includes/Maintenance/MetaStoreIndex.php
1 file changed, 3 insertions(+), 5 deletions(-)
Approvals:
Cindy-the-browser-test-bot: Looks good to me, but someone else must approve
DCausse: Looks good to me, approved
jenkins-bot: Verified
diff --git a/includes/Maintenance/MetaStoreIndex.php
b/includes/Maintenance/MetaStoreIndex.php
index 7264dc7..261aea3 100644
--- a/includes/Maintenance/MetaStoreIndex.php
+++ b/includes/Maintenance/MetaStoreIndex.php
@@ -116,10 +116,9 @@
public function createOrUpgradeIfNecessary() {
$this->fixOldName();
- $status = $this->client->getStatus();
// If the mw_cirrus_metastore alias still not exists it
// means we need to create everything from scratch.
- if ( !$status->aliasExists( self::INDEX_NAME ) ) {
+ if ( !$this->client->getIndex( self::INDEX_NAME )->exists() ) {
$this->log( self::INDEX_NAME . " missing creating.\n" );
$newIndex = $this->createNewIndex();
$this->switchAliasTo( $newIndex );
@@ -341,13 +340,12 @@
* If mw_cirrus_versions exists with no mw_cirrus_metastore
*/
private function fixOldName() {
- $status = $this->client->getStatus();
- if ( !$status->indexExists( self::OLD_INDEX_NAME ) ) {
+ if ( !$this->client->getIndex( self::OLD_INDEX_NAME )->exists()
) {
return;
}
// Old mw_cirrus_versions exists, if mw_cirrus_metastore alias
does not
// exist we must create it
- if ( !$status->aliasExists( self::INDEX_NAME ) ) {
+ if ( !$this->client->getIndex( self::INDEX_NAME )->exists() ) {
$this->log( "Adding transition alias to " .
self::OLD_INDEX_NAME . "\n" );
// Old one exists but new one does not
// we need to create an alias
--
To view, visit https://gerrit.wikimedia.org/r/300461
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3e0936074bc7679f56be737a397b00ad3be44b15
Gerrit-PatchSet: 2
Gerrit-Project: mediawiki/extensions/CirrusSearch
Gerrit-Branch: master
Gerrit-Owner: EBernhardson <[email protected]>
Gerrit-Reviewer: Cindy-the-browser-test-bot <[email protected]>
Gerrit-Reviewer: DCausse <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
