Krinkle added a comment.
In T173710#3966996, @Ladsgroup wrote:
The jobqueue size [..] will go down more once we enable "lua fine grained usage tracking" and "statement fine grained usage tracking" in more wikis.
Ref: T184322: Enable fine grained lua tracking gradually in client wikisTASK DETAILh
Krinkle removed a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, KrinkleCc: Jeff_G, Addshore, zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, Y
Krinkle removed projects: Discovery, CirrusSearch.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, KrinkleCc: Jeff_G, Addshore, zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daim
Krinkle added a subscriber: Jeff_G.Krinkle merged a task: T178840: Commons category updates are very slow. Commons job queue very large..
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: KrinkleCc: Jeff_G, Ad
Aklapper added a comment.
Is T178840: Commons category updates are very slow. Commons job queue very large. a duplicate?TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AklapperCc: zhuyifei1999, RP88, Jack_wh
Lydia_Pintscher removed a project: Wikidata-Former-Sprint-Board.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YO
Krinkle added a commit: rEWBA081c1cd78f9d: Add cause action/agent tracking to link refresh jobs.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: KrinkleCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey
Krinkle added a commit: rMW8f829de5f040: Add action/user tracking to link refresh jobs.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: KrinkleCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, m
elukey added a comment.
In T173710#3730226, @EBernhardson wrote:
It was perhaps noted before, but because of the recursive nature of the refreshLinks and htmlCacheUpdate jobs even if the backlog is being processed it may not look like it, because the jobs are just enqueing new jobs. Will probably
thiemowmde added subtasks: T178804: When processing changes to Wikibase SiteLinks on the client, only trigger updates for sitelinks that are actually shown in the sidebar., T178806: Wikibase: Batch HTMLCacheUpdateJobs across changes, T178810: Wikibase: Increase batch size for HTMLCacheUpdateJobs tr
elukey added a project: User-Elukey.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, em
daniel added a comment.
I have thought a bit about ways to mitigate this. Here are three things I think could help:
T178804: When processing changes to Wikibase SiteLinks on the client, only trigger updates for sitelinks that are actually shown in the sidebar. (doable in a weeks or two, if priori
Ladsgroup added a comment.
I think one of the reasons contributing to the problem is the same problem we had with T171027: "Read timeout is reached" DBQueryError when trying to load specific users' watchlists (with +1000 articles) on several wikis, we stopped emitting injectRCRecord jobs but we are
elukey added a comment.
Updated list (showjob1.txt contains group1, showjob.txt group2)
elukey@terbium:~$ awk '{if ($3 > 10) print $_}' showjob1.txt
commonswiki: refreshLinks: 1629991 queued; 5 claimed (2 active, 3 abandoned); 0 delayed
commonswiki: htmlCacheUpdate: 2470968 queued; 0 claimed
debt removed a project: Discovery-Search.debt added a comment.
Taking off the #discovery-search tag, as there isn't much we can do, but we'll continue to monitor using the #discovery tag.TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/
Krinkle added a parent task: T175897: Audit and improve JobQueue performance (2017).
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: KrinkleCc: Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWic
Ladsgroup added a comment.
This patch can go in when commons is on wmf.17. Sooner, it's useless. (See T174422: Make dbBatchSize in WikiPageUpdater configurable)TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
jcrespo added a comment.
Could, at least, that part have something to do with T164173, as a problem from the same cause, or a consequence of the fix? I also remember some tunning of some wikidata crons or job size, but not sure if upwards or downwards and not sure if related. Aaron, Ladsgroup or Da
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu9
Krinkle removed projects: MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)), Patch-For-Review.Krinkle updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTIONThis doesn't sound good:
| August 21
|--
| {F9141847 height=150}
Current:
Mitigation:
* 6898d06f8f Disable re
GWicke added a comment.
A possible contribution to the backlog building could be the infinite retry / immortal job problem described in T73853. Looking for old htmlCacheUpdate root jobs from April still executing over four months later (!) via grep htmlCacheUpdate runJobs.log | grep -c 'rootJobTime
GWicke added a project: Services (watching).
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GWickeCc: Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, Ha
ReleaseTaggerBot edited projects, added MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)); removed MW-1.30-release-notes (WMF-deploy-2017-08-29 (1.30.0-wmf.16)).
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpr
EBernhardson added a comment.
Parsing through the existing jobs with a script to measure the "real" number of purges that will be issued: P5916 I'm finding that on the wikis with the largest queues the majority of the jobs do nothing. The first 50k jobs for a few wikis:
wikidatawiki: Empty: 44166
ReleaseTaggerBot edited projects, added MW-1.30-release-notes (WMF-deploy-2017-08-29 (1.30.0-wmf.16)); removed MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)).
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpr
EBernhardson added a comment.
Related previous tickets about purge problems/increases: T124418 T133821TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: EBernhardsonCc: BBlack, Peachey88, Liuxinyu970226, daniel
ReleaseTaggerBot edited projects, added MW-1.30-release-notes (WMF-deploy-2017-08-22 (1.30.0-wmf.15)); removed MW-1.30-release-notes (WMF-deploy-2017-08-29 (1.30.0-wmf.16)).
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpr
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2017-08-24T01:46:36Z] Synchronized php-1.30.0-wmf.15/includes/jobqueue: Deploy 752580637 (T173710) (duration: 00m 49s)TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/
ReleaseTaggerBot added a project: MW-1.30-release-notes (WMF-deploy-2017-08-29 (1.30.0-wmf.16)).
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ReleaseTaggerBotCc: Daniel_Mietchen, Harej, XXN, Pasleim, Bugre
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Reedy, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunc
Reedy merged a task: T173957: Job queue growing constantly since around 7th August.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ReedyCc: Reedy, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF
jcrespo added a project: CirrusSearch.Herald added projects: Discovery, Discovery-Search.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespoCc: WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, A
Lydia_Pintscher added projects: Wikidata, Wikidata-Sprint.
TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, GoranSMilovanovi
33 matches
Mail list logo
