[MediaWiki-commits] [Gerrit] [s23.org wikistats] Throttle updates for big farms, keep upd... - change (operations...wikistats)

[Nemo bis (Code Review)]

Nemo bis has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/108670


Change subject: [s23.org wikistats] Throttle updates for big farms, keep 
updating big wikis' stats
......................................................................

[s23.org wikistats] Throttle updates for big farms, keep updating big wikis' 
stats

Patch quickly put together after my last comment on the bug.
If it looks reasonably ok, mutante will have to deploy it, change whatever 
switch
kept the wikia wikis stalled and then let the updates run to test.

Bug: 59943
Change-Id: Ieb3c1d49b6c6b7208172f7e0bac810ce72fe3bc2
---
M usr/lib/wikistats/update.php
1 file changed, 21 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/debs/wikistats 
refs/changes/70/108670/1

diff --git a/usr/lib/wikistats/update.php b/usr/lib/wikistats/update.php
index c4aa74f..cfcebd0 100644
--- a/usr/lib/wikistats/update.php
+++ b/usr/lib/wikistats/update.php
@@ -256,9 +256,24 @@
 
 #want to know number of wikis and progress in logs
 $mycount=0;
+$mybigcount=0;
 $totalcount=mysql_num_rows($myresult);
+$bigfarm=false;
+if ($totalcount > 1000 && $table != "mediawikis") {
+       $bigfarm=true;
+}
 
 while($row = mysql_fetch_array( $myresult )) {
+
+       // If it's a big lazy wikifarm, give precedence to bigger wikis,
+       // don't update too many of the others.
+       if ($bigfarm) {
+               if ($row['good'] >= 100) {
+                       $mybigcount++;
+               } elseif ($mycount >= 1000) {
+                       continue;
+               }
+       }
        $mycount++;
 
        if ($row['method']=="8") {
@@ -487,6 +502,12 @@
                }
        }
 
+       // Let the big but lazy wikifarms rest a second, or till next run if 
we've done enough
+       if ($bigfarm && $mycount >= 1000 && $mybigcount >= 500) {
+               break;
+       }
+       sleep(1);
+
 }
 
 if (isset($convert) && $convert) {

-- 
To view, visit https://gerrit.wikimedia.org/r/108670
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ieb3c1d49b6c6b7208172f7e0bac810ce72fe3bc2
Gerrit-PatchSet: 1
Gerrit-Project: operations/debs/wikistats
Gerrit-Branch: master
Gerrit-Owner: Nemo bis <federicol...@tiscali.it>
Gerrit-Reviewer: jenkins-bot

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits