| Krinkle removed projects: MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)), Patch-For-Review. Krinkle updated the task description. (Show Details) |
CHANGES TO TASK DESCRIPTION
This doesn't sound good:
| August 21
|--
| {F9141847 height=150}
Current: <https://grafana.wikimedia.org/dashboard/db/job-queue-health?refresh=1m&orgId=1>
Mitigation:
* 6898d06f8f Disable rebound CDN purges for backlinks in HTMLCacheUpdateJob. @aaron, https://gerrit.wikimedia.org/r/373705
In MediaWiki 1.27 (2015) we introduced `$wgCdnReboundPurgeDelay` (more information at <https://www.mediawiki.org/wiki/Manual:$wgCdnReboundPurgeDelay>). This means that effectively every purge is actually two purges, spaced out by a few seconds, to account for race conditions between database replication and cache population. However, the common case of recursive updates (e.g. from templates) is already delayed through the JobQueue, so we can omit the rebound purge in that case.
**→** Fixing this improved job execution throughput, and decreased job queue growth.
* 70d1bc0091 Make workItemCount() smarter for htmlCacheUpdate/refreshLinks. @aaron, <https://gerrit.wikimedia.org/r/373325>
To avoid pressure on CDN and database, job execution can be throttled via <https://www.mediawiki.org/wiki/Manual:$wgJobBackoffThrottling>. The placeholder jobs for recursive updates (e.g. for templates) were being counted as 1, which needlessly subjected them to throttling.
**→** Fixing this improved job execution throughput.
* 56ed177f1 Reduce batch size in WikiPageUpdater. @Ladsgroup, <https://gerrit.wikimedia.org/r/#/c/373551/>
**→** Fixing this improved job execution throughput.
* cb7c910ba Fix old regression in HTMLCacheUpdate de-duplication. @aaron, <https://gerrit.wikimedia.org/r/373979>
Following a refactor in mid-2016, the duplication logic for recursive updates started to be miscalculated (based on current time, instead of time from the initial placeholder/root job).
**→** Fixing this significantly decreased job queue growth.
| August 31
{F9141847}
[[ https://grafana.wikimedia.org/dashboard/db/job-queue-health?refresh=1m&orgId=1 | current]]|--
| {F9232210 height=150} {F9232209 height=200}
| August 21
|--
| {F9141847 height=150}
Current: <https://grafana.wikimedia.org/dashboard/db/job-queue-health?refresh=1m&orgId=1>
Mitigation:
* 6898d06f8f Disable rebound CDN purges for backlinks in HTMLCacheUpdateJob. @aaron, https://gerrit.wikimedia.org/r/373705
In MediaWiki 1.27 (2015) we introduced `$wgCdnReboundPurgeDelay` (more information at <https://www.mediawiki.org/wiki/Manual:$wgCdnReboundPurgeDelay>). This means that effectively every purge is actually two purges, spaced out by a few seconds, to account for race conditions between database replication and cache population. However, the common case of recursive updates (e.g. from templates) is already delayed through the JobQueue, so we can omit the rebound purge in that case.
**→** Fixing this improved job execution throughput, and decreased job queue growth.
* 70d1bc0091 Make workItemCount() smarter for htmlCacheUpdate/refreshLinks. @aaron, <https://gerrit.wikimedia.org/r/373325>
To avoid pressure on CDN and database, job execution can be throttled via <https://www.mediawiki.org/wiki/Manual:$wgJobBackoffThrottling>. The placeholder jobs for recursive updates (e.g. for templates) were being counted as 1, which needlessly subjected them to throttling.
**→** Fixing this improved job execution throughput.
* 56ed177f1 Reduce batch size in WikiPageUpdater. @Ladsgroup, <https://gerrit.wikimedia.org/r/#/c/373551/>
**→** Fixing this improved job execution throughput.
* cb7c910ba Fix old regression in HTMLCacheUpdate de-duplication. @aaron, <https://gerrit.wikimedia.org/r/373979>
Following a refactor in mid-2016, the duplication logic for recursive updates started to be miscalculated (based on current time, instead of time from the initial placeholder/root job).
**→** Fixing this significantly decreased job queue growth.
| August 31
{F9141847}
[[ https://grafana.wikimedia.org/dashboard/db/job-queue-health?refresh=1m&orgId=1 | current]]|--
| {F9232210 height=150} {F9232209 height=200}
TASK DETAIL
EMAIL PREFERENCES
To: Krinkle
Cc: GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, Harej, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, Emijrp, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, Vali.matei, Avner, Zppix, debt, Gehel, FloNight, Izno, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb
Cc: GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, Harej, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, Emijrp, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, Vali.matei, Avner, Zppix, debt, Gehel, FloNight, Izno, Eevans, mobrovac, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
