BBlack has submitted this change and it was merged.
Change subject: Lower the basic job runner parallelism
......................................................................
Lower the basic job runner parallelism
This patch has two primary aims:
- Reduce the number of concurrent database accesses in a 'thundering herd'
scenario when a large refreshLinks job comes in. See ops list thread "enwiki
outage 2014-09-19 08:50:00".
- Improve efficiency by avoiding excessive context switching. Aim to roughly
match the runnable processes to the available hyperthreads on these
machines, which is currently 24 threads. For this we count the parsoid jobs
as the equivalent of 1-2 cpu bound jobs, as all they do is do http requests
to parsoid and wait for the result. In practice the number of runnable jobs
including non-php processes was hovering around 35 on these machines, so
this reduction should result in a less aggressive oversubscription. Also see
ops list thread "Job runner CPU usage almost tripled over last months".
Change-Id: I9c3a39ba4429f90767ef02a8d8a5e064c8928175
---
M manifests/role/mediawiki.pp
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Ori.livneh: Looks good to me, but someone else must approve
BBlack: Looks good to me, approved
jenkins-bot: Verified
diff --git a/manifests/role/mediawiki.pp b/manifests/role/mediawiki.pp
index e0abff6..d608c5d 100644
--- a/manifests/role/mediawiki.pp
+++ b/manifests/role/mediawiki.pp
@@ -125,7 +125,7 @@
class { '::mediawiki::jobrunner':
queue_servers => ['rdb1001.eqiad.wmnet', 'rdb1003.eqiad.wmnet'],
statsd_server => 'statsd.eqiad.wmnet:8125',
- runners_basic => 20,
+ runners_basic => 14,
runners_parsoid => 21,
runners_upload => 7,
runners_gwt => 1,
--
To view, visit https://gerrit.wikimedia.org/r/161473
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I9c3a39ba4429f90767ef02a8d8a5e064c8928175
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke <[email protected]>
Gerrit-Reviewer: Aaron Schulz <[email protected]>
Gerrit-Reviewer: BBlack <[email protected]>
Gerrit-Reviewer: Filippo Giunchedi <[email protected]>
Gerrit-Reviewer: Giuseppe Lavagetto <[email protected]>
Gerrit-Reviewer: Mark Bergsma <[email protected]>
Gerrit-Reviewer: Ori.livneh <[email protected]>
Gerrit-Reviewer: Springle <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
