[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

VladimirAlexiev added a subscriber: Lucas. VladimirAlexiev added a comment. @Lucas, congratulations on your command line wizardry! I know "jq" but not nearly to that extent, and I how did I not know about "units"? TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Lucas_Werkmeister_WMDE added a comment. Then I think I misunderstood your question? We certainly don’t have 155 million violations “of a certain kind”. I’m guessing most reports that users would want would be much smaller; on Wikidata:Database reports/Constraint violations/Summary

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore added a comment. In T201150#7355393 , @Lucas_Werkmeister_WMDE wrote: > That would suggest some 155 million constraint violations in total (almost all of them on items): > > $ units -t '((853 + 806 + 811) * 94504342 /

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Lucas_Werkmeister_WMDE added a comment. > Do you have a picture of how many violations, and violations split by type / kind exist on Wikidata at any given time? Looks like we’re averaging about 1⅔ constraint violations per item (excluding redirects, if I’m not mistaken): $ curl

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore added a comment. Do you have a picture of how many violations, and violations split by type / kind exist on Wikidata at any given time? > periodic processing of every item because constraint definitions or implementations can change globally (TODO?) Approaching this one is

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

VladimirAlexiev added a comment. Here's the goal: a SPARQL query should return **all** violations of a certain kind, with a possible data lag of a few hours. So you need: - a baseline of having processed **all** items (TODO) - processing of changed items (DONE) - periodic

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore added a comment. It sounds like we need a definition of `regularly`! TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: So9q, VladimirAlexiev, CamelCaseNick,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore removed Addshore as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: So9q, VladimirAlexiev, CamelCaseNick, Harmonia_Amanda, Addshore, Jonas,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Lydia_Pintscher closed subtask T290635: Evaluate whether WMDE can take over our essential community run constraints bot as Declined. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Bugreporter closed subtask T290641: tweak SPARQL queries that expose constraint violations as Invalid. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, Bugreporter Cc: So9q,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

VladimirAlexiev added a subtask: T290641: tweak SPARQL queries that expose constraint violations. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, VladimirAlexiev Cc: So9q,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

VladimirAlexiev reopened this task as "Open". VladimirAlexiev added a comment. > @Addshore Resolved T204031 on 11 Aug: "Deploy **regular running** of wikidata constraint checks using the job queue: These now run after **every edit**" In

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

So9q added a subtask: T290635: Evaluate whether WMF operations can take over our essential community run constraints bot. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, So9q Cc: So9q,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

So9q added a comment. In T201150#7341220 , @VladimirAlexiev wrote: > > 2. WMD should take over the operation of KrBot (that generates Constraint Violation pages) because it now takes huge resources ("Now the bot requires 106

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

So9q added a comment. In T201150#7341220 , @VladimirAlexiev wrote: > Can I make two related requests? Not sure how to post them as separate tasks related to this task, can someone from WMD do that? You can create a new task. After

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

VladimirAlexiev added a comment. Can I make two related requests? Not sure how to post them as separate tasks related to this task, can someone from WMD do that? 1. Change the query to return the offending value rather than the offending statement; which is more useful to eg generate a

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore changed the status of subtask T201147: Expose constraint violations to WDQS using event queue from Stalled to Open. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc:

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore closed this task as "Resolved". Addshore claimed this task. Addshore added a comment. Restricted Application added a project: User-Addshore. These now run after every edit TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore closed subtask T204031: Deploy regular running of wikidata constraint checks using the job queue as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: CamelCaseNick,

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Ladsgroup changed the status of subtask T204031: Deploy regular running of wikidata constraint checks using the job queue from Stalled to Open. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Michael changed the status of subtask T204031: Deploy regular running of wikidata constraint checks using the job queue from Open to Stalled. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] T201150: Regularly run constraint checks for all items

Addshore removed a project: Wikidata-Campsite. TASK DETAIL https://phabricator.wikimedia.org/T201150 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: CamelCaseNick, Harmonia_Amanda, Addshore, Jonas, Aklapper, Invadibot, MPhamWMF,