[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-04-29 Thread gerritbot
gerritbot added a comment. Change 589873 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] Switch query service maxlag to be median +1 https://gerrit.wikimedia.org/r/589873 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-04-22 Thread gerritbot
gerritbot added a comment. Change 589874 abandoned by Addshore: Switch query service maxlag to be median 1 https://gerrit.wikimedia.org/r/589874 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-04-18 Thread gerritbot
gerritbot added a comment. Change 589874 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.28] Switch query service maxlag to be median +1 https://gerrit.wikimedia.org/r/589874 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-04-18 Thread gerritbot
gerritbot added a comment. Change 589873 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@master] Switch query service maxlag to be median +1 https://gerrit.wikimedia.org/r/589873 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-02-05 Thread Addshore
Addshore added a comment. In T221774#5834478 , @Dvorapa wrote: > Today 8-10s for the whole day, none of Pywikibot tests loading WD succeed within timeout. See T243701 for more details

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-19 Thread ArthurPSmith
ArthurPSmith added a comment. In T221774#5815408 , @Addshore wrote: > [...] > Note that this dashboard includes metrics for both pooled and depooled servers. > So whatever you read there will likely also be reporting data for

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-19 Thread Addshore
Addshore added a comment. In T221774#5813612 , @ArthurPSmith wrote: > Am I misreading this graph? https://grafana.wikimedia.org/d/00489/wikidata-query-service?panelId=8=1=now-12h=now=10s It looks like the query service lag for 3 of

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-18 Thread Pintoch
Pintoch added a comment. This does not have anything to do with you indeed! I was just trying to explain that I stopped trying to help solve this issue (therefore unsubscribing from this). TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-18 Thread Multichill
Multichill added a comment. In T221774#5814473 , @Pintoch wrote: > I don't know - I stopped working on this task and T240369 since T240374 was

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-17 Thread Bugreporter
Bugreporter added a comment. In T221774#5680891 , @Addshore wrote: > Due to the apparent differences in lag pattern between some of the nodes we need to try to get T238751 done so that we can

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-17 Thread ArthurPSmith
ArthurPSmith added a comment. @Bugreporter well something must have changed early today - was it previously "mean" and is now "median"? I'm not sure which is better, but having WDQS hours out of date (we're over 4 hours now) is NOT a good situation, and what this whole task was intended to

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-17 Thread Bugreporter
Bugreporter added a comment. maxlag is currently based on median (not max) of lag of all public servers. TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, Bugreporter Cc: ArthurPSmith,

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2020-01-17 Thread ArthurPSmith
ArthurPSmith added a comment. Am I misreading this graph? https://grafana.wikimedia.org/d/00489/wikidata-query-service?panelId=8=1=now-12h=now=10s It looks like the query service lag for 3 of the servers has been growing steadily for the past roughly 8 hours. However, edits are going

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-12-10 Thread Pintoch
Pintoch added a comment. I am first getting in touch with people who seem to be running bots with maxlag greater than 5 or no maxlag parameter at all, to see if they would accept to follow @Addshore's advice never to use maxlag greater than 5 at all. TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-27 Thread Lea_Lacroix_WMDE
Lea_Lacroix_WMDE added a comment. In T221774#5694863 , @Pintoch wrote: > I will try to implement a responsible throttling strategy in OpenRefine, hoping that others will do the same. Once this is done, if by any chance you want to

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-26 Thread Pintoch
Pintoch added a comment. OK! If you have ways to check what sort of maxlag values are used it would be great! I will try to implement a responsible throttling strategy in OpenRefine, hoping that others will do the same. TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-26 Thread Addshore
Addshore added a comment. > I think it would be preferable to get a continuous throttling behaviour: the higher the lag, the slower the edits. Sounds sensible to me, 5 should perhaps still be the point at which to stop. But if editing slows down when the "maxlag" is 0-5 then maybe we

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-25 Thread Pintoch
Pintoch added a comment. Actually, some tools seem to be doing something like that already, since edits are still going through despite max lag being above 5 for more than an hour now (Author Disambiguator does this, QuickStatements too probably, Edoderoobot too). So these tools use higher

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-25 Thread Pintoch
Pintoch added a comment. One problem with the current policy (requesting all automated editing processes to use `maxlag=5`) is that this creates a binary threshold: either the query service lag is under the threshold, in which case bots will edit at full speed, or the query service lag is

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-22 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-22T12:34:11Z] Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 60 RESYNC (duration: 00m 51s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-22 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-22T12:32:27Z] Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 60 (duration: 00m 53s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-22 Thread gerritbot
gerritbot added a comment. Change 552474 **merged** by jenkins-bot: [operations/mediawiki-config@master] wgWikidataOrgQueryServiceMaxLagFactor 60 https://gerrit.wikimedia.org/r/552474 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-22 Thread gerritbot
gerritbot added a comment. Change 552474 had a related patch set uploaded (by Addshore; owner: Addshore): [operations/mediawiki-config@master] wgWikidataOrgQueryServiceMaxLagFactor 60 https://gerrit.wikimedia.org/r/552474 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-22 Thread Addshore
Addshore added a comment. In T221774#5683786 , @Multichill wrote: > @Addshore what factor are we using right now? Most bots seem to have stopped so it might be on the low side. I have slowly been lowering the factor, just about to

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-22 Thread Multichill
Multichill added a comment. @Addshore what factor are we using right now? Most bots seem to have stopped so it might be on the low side. TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-21 Thread Bugreporter
Bugreporter added a comment. Actually maxlag may be more than 5 second for up to one hour. tools should not break (not "give up") even no edits can not be made. TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-21 Thread Pintoch
Pintoch added a comment. So this is what we get with an exponential back-off (1.5 factor), at the moment: 22:37:27.148 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.49167 seconds lagged. -- pausing for 1000 milliseconds. (19338ms) 22:37:28.729

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Tagishsimon
Tagishsimon added a comment. Notable that WDQS lag seems to be paying not a blind bit of notice to this change :( WDQS 1005-7 all at ~3 hours. https://grafana.wikimedia.org/d/00489/wikidata-query-service?orgId=1=8 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Pintoch
Pintoch added a comment. Thanks for the notification! I would be happy to release a new version of OpenRefine with a patch applied - I can do this in the coming days. The exponential back-off suggested by @Multichill makes sense intuitively - could WMDE confirm that this is the policy they

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Multichill
Multichill added a comment. In T221774#5679442 , @Bugreporter wrote: > Currently multiple tools are broken because the time that make maxlag back to normal is much longer than the total time the tool retries edits (see this

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-20T18:56:43Z] Synchronized wmf-config/InitialiseSettings.php: RESYNC T221774 - wgWikidataOrgQueryServiceMaxLagFactor 120 (duration: 00m 50s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-20T18:51:14Z] Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 120 (duration: 00m 54s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-20T18:42:56Z] Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 170 (duration: 00m 53s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Addshore
Addshore added a comment. The first sync didn't seem to do the trick, but the second one did. The discovery team suggested that perhaps median might not be the best option. We should investigate using https://config-master.wikimedia.org/pybal/eqiad/wdqs and

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-20T15:36:08Z] Synchronized wmf-config/InitialiseSettings.php: RESYNC T221774 - wgWikidataOrgQueryServiceMaxLagFactor 180 [[gerrit:552069]] (duration: 00m 52s) TASK

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 552071 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] qsmaxlag, send altered lag info, not only raw lag https://gerrit.wikimedia.org/r/552071 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Addshore
Addshore added a comment. Now deployed and working https://www.wikidata.org/w/api.php?action=query=MediaWiki=json=-1 Current maxlag for wikidata is roughly 10 due to the query service TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-20T14:52:39Z] Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 180 [[gerrit:552069]] (duration: 00m 52s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-20T14:50:53Z] Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (use altered lag, not raw lag) [[gerrit:552072]] (duration: 00m 53s)

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 552072 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] qsmaxlag, send altered lag info, not only raw lag https://gerrit.wikimedia.org/r/552072 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 552072 **merged** by Addshore: [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] qsmaxlag, send altered lag info, not only raw lag https://gerrit.wikimedia.org/r/552072 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 552071 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@master] qsmaxlag, send altered lag info, not only raw lag https://gerrit.wikimedia.org/r/552071 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 552069 **merged** by jenkins-bot: [operations/mediawiki-config@master] wgWikidataOrgQueryServiceMaxLagFactor 180 https://gerrit.wikimedia.org/r/552069 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 552069 had a related patch set uploaded (by Addshore; owner: Addshore): [operations/mediawiki-config@master] wgWikidataOrgQueryServiceMaxLagFactor 180 https://gerrit.wikimedia.org/r/552069 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-20 Thread gerritbot
gerritbot added a comment. Change 551582 **merged** by Giuseppe Lavagetto: [operations/puppet@production] mediawiki/wikidata maint cron for updateQueryServiceLag https://gerrit.wikimedia.org/r/551582 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-19T18:20:28Z] Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (queryservice maxlag, hook) [[gerrit:551858]] (duration: 00m 53s)

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread Addshore
Addshore added a comment. Just the cron job to go now TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Lydia_Pintscher, Matthias_Geisler_WMDE, Simon_Villeneuve, Lea_Lacroix_WMDE,

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551858 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Add ApiMaxLagInfo hook https://gerrit.wikimedia.org/r/551858 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551869 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] onApiMaxLagInfo ttl to 1 https://gerrit.wikimedia.org/r/551869 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-19T17:43:52Z] Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (queryservice maxlag, maint script) [[gerrit:551857]] (duration: 00m

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551857 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Maintenance script to update cached wdqs lag https://gerrit.wikimedia.org/r/551857 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551869 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@master] onApiMaxLagInfo ttl to 1 https://gerrit.wikimedia.org/r/551869 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-11-19T17:11:30Z] Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (queryservice maxlag) [[gerrit:551855]] [[gerrit:551856]] (duration:

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551856 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Switch from load_composer_autoloader AutoloadNamespaces https://gerrit.wikimedia.org/r/551856 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551855 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Add WDQS lag into request maxlag logic. https://gerrit.wikimedia.org/r/551855 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551857 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Maintenance script to update cached wdqs lag https://gerrit.wikimedia.org/r/551857 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551856 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Switch from load_composer_autoloader AutoloadNamespaces https://gerrit.wikimedia.org/r/551856 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread Addshore
Addshore added a comment. Just prepared the patches needed to backport this (as otherwise it won't go out with the train and land for 2-3 weeks. TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551855 had a related patch set uploaded (by Addshore; owner: Alaa Sarhan): [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Add WDQS lag into request maxlag logic. https://gerrit.wikimedia.org/r/551855 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551858 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@wmf/1.35.0-wmf.5] Add ApiMaxLagInfo hook https://gerrit.wikimedia.org/r/551858 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551574 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] Add ApiMaxLagInfo hook https://gerrit.wikimedia.org/r/551574 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-19 Thread gerritbot
gerritbot added a comment. Change 551520 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] Maintenance script to update cached wdqs lag https://gerrit.wikimedia.org/r/551520 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-18 Thread gerritbot
gerritbot added a comment. Change 551640 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] Switch from load_composer_autoloader AutoloadNamespaces https://gerrit.wikimedia.org/r/551640 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-18 Thread gerritbot
gerritbot added a comment. Change 551640 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@master] Switch from load_composer_autoloader AutoloadNamespaces https://gerrit.wikimedia.org/r/551640 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-18 Thread gerritbot
gerritbot added a comment. Change 551582 had a related patch set uploaded (by Addshore; owner: Addshore): [operations/puppet@production] DNM: mediawiki/wikidata maint cron for updateQueryServiceLag https://gerrit.wikimedia.org/r/551582 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-18 Thread gerritbot
gerritbot added a comment. Change 551574 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@master] Add ApiMaxLagInfo hook https://gerrit.wikimedia.org/r/551574 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-18 Thread gerritbot
gerritbot added a comment. Change 532495 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] Add WDQS lag into request maxlag logic. https://gerrit.wikimedia.org/r/532495 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-11-18 Thread gerritbot
gerritbot added a comment. Change 551520 had a related patch set uploaded (by Addshore; owner: Addshore): [mediawiki/extensions/Wikidata.org@master] Maintenance script to update cached wdqs lag https://gerrit.wikimedia.org/r/551520 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-10-29 Thread Addshore
Addshore added a comment. Current patch being worked on is https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikidata.org/+/532495/ TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-09-02 Thread gerritbot
gerritbot added a comment. Change 532968 **merged** by jenkins-bot: [mediawiki/extensions/Wikidata.org@master] Add ADR for the solution design on including WDQS lag into maxlag https://gerrit.wikimedia.org/r/532968 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-28 Thread gerritbot
gerritbot added a comment. Change 532968 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan): [mediawiki/extensions/Wikidata.org@master] Add ADR for the solution design on including WDQS lag into maxlag https://gerrit.wikimedia.org/r/532968 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-26 Thread gerritbot
gerritbot added a comment. Change 512393 abandoned by Alaa Sarhan: Add Wikidata query service lag to Wikidata maxlag Reason: moving it to Wikidata.org https://gerrit.wikimedia.org/r/512393 TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-26 Thread gerritbot
gerritbot added a comment. Change 532495 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan): [mediawiki/extensions/Wikidata.org@master] [WIP] Add WDQS lag into request maxlag logic. https://gerrit.wikimedia.org/r/532495 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-26 Thread alaa_wmde
alaa_wmde added a comment. In T221774#5437227 , @Tarrow wrote: > I think I favour **option 1** assuming that the service we are planning to implement is the Prometheus based one. Yeap sorry for not making that clear before. We

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-26 Thread Tarrow
Tarrow added a comment. I think I favour **option 1** assuming that the service we are planning to implement is the Prometheus based one. If we are actually planning on the SPARQL based on then I think **option 0** could make more sense and defaulting to turned on with the option to

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-23 Thread alaa_wmde
alaa_wmde added a comment. So regarding where this thing should live. I have thought about it a bit more and have the follow options: **Option 0** Keep the service in Wikibase codebase. In this case, we will have to either: 1. provide a configuration to enable/add that service so

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-08-21 Thread alaa_wmde
alaa_wmde added a comment. :D https://gerrit.wikimedia.org/r/admin/projects/mediawiki/extensions/Wikidata.org why didn't occur to me to look exactly for that before 樂 .. neat TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-07-01 Thread Addshore
Addshore added a comment. replies to 2 comments inline In T221774#5296984 , @alaa_wmde wrote: > That thing is definitely better to live outside of Wikibase codebase, if possible, in an extension/external library that is more

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-07-01 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a comment. In T221774#5297123 , @hoo wrote: > >> If yes, where should WikimediaPrometheusSparqlEndpointReplicationStatus live? Here or in the Wikidata.org extension? > > > > That thing is definitely

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-07-01 Thread hoo
hoo added a comment. In T221774#5296984 , @alaa_wmde wrote: > @maho re feedback on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/512393 > > > Do we even want to do this via Prometheus

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-07-01 Thread alaa_wmde
alaa_wmde added a comment. @maho re feedback on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/512393 > Do we even want to do this via Prometheus (WikimediaPrometheusSparqlEndpointReplicationStatus) Sounds like the right thing to do so to me too, as per

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-06-16 Thread Multichill
Multichill added a comment. Bump, we had another preventable incident, see https://www.wikidata.org/w/index.php?title=Wikidata:Administrators%27_noticeboard=963260218#Edoderoobot_edit_rate . @hoo any update? TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-24 Thread gerritbot
gerritbot added a comment. Change 512393 had a related patch set uploaded (by Hoo man; owner: Hoo man): [mediawiki/extensions/Wikibase@master] Add Wikidata query service lag to Wikidata maxlag https://gerrit.wikimedia.org/r/512393 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-16 Thread Smalyshev
Smalyshev added a comment. Note that wdqs-test should not be included in maxlag at any case. These are test servers, which can be lagged significantly due to any number of reasons. Also, I'm not sure how easy it is to see if the server is in the pool, but it'd be nice if it was possible to

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-13 Thread hoo
hoo added a comment. @fgiunchedi @Smalyshev How about I do something like this: $result = json_decode(file_get_contents('http://prometheus.svc.eqiad.wmnet/ops/api/v1/query?query=blazegraph_lastupdated'), true)['data']['result']; foreach ( $result as $resultByInstance ) {

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-09 Thread fgiunchedi
fgiunchedi added a comment. In T221774#5155621 , @hoo wrote: > Possible way to do this: > > Create `PrometheusBlazegraphLagService` class which internally fetches the lag from a given Blazegraph instance like `curl

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-08 Thread Smalyshev
Smalyshev added a comment. > do we need a whitelist/blacklist (or both) or some other mechanism to make sure we don't eg. take servers into account that are being maintained. Ideally, only servers in the public pool, or maybe public+internal pool, should be taken into account, this

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-08 Thread Ladsgroup
Ladsgroup added a comment. Probably prometheus needs to define a discovery url: "http://prometheus.discovery.wmnet; (similar to http://ores.discovery.wmnet) @fgiunchedi knows better though. TASK DETAIL https://phabricator.wikimedia.org/T221774 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-08 Thread hoo
hoo added a comment. @Smalyshev Do you think it would be enough to look at `http://prometheus.svc.eqiad.wmnet/ops/api/v1/query?query=blazegraph_lastupdated` and `http://prometheus.svc.codfw.wmnet/ops/api/v1/query?query=blazegraph_lastupdated` (no matter which DC MW is running in) and just

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-07 Thread Smalyshev
Smalyshev added a comment. > Go through Prometheus or ask the instances individually or …? prometheus & icinga know what is the lag for each server, so I think it's better to ask them? I am not sure if we have good support for scheduled recurrent jobs on Wiki - if we do, we could just

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-05-07 Thread Ladsgroup
Ladsgroup added a comment. A very minor note: WDQS depends heavily on mediawiki nodes to function, making mediawiki apps to depend on WDQS (which we already has done in WikibaseQualityConstraints) will make a cyclic dependency that bit Wikipedia a lot with ores extension and ores service.

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-04-29 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a comment. > `maxlag` affects all API requests, even ones that shouldn’t have any effect on query service lag, such as `action=wbgetentity` or `action=query=userinfo`. Should we try to limit the impact of this change, e. g. by only using query service lag on

[Wikidata-bugs] [Maniphest] [Commented On] T221774: Add Wikidata query service lag to Wikidata maxlag

2019-04-25 Thread Sjoerddebruin
Sjoerddebruin added a comment. > What should the conversion factor be? (For dispatch lag, it’s 60 – five seconds of replication lag are equivalent to five minutes of dispatch lag.) Based on the last three months