[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Addshore closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Addshore Cc: Bugreporter, MisterSynergy, Tonina_Zhelyazkova_WMDE, Addshore,

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Bugreporter added a comment. In T268625#6744532 , @Lucas_Werkmeister_WMDE wrote: > Alright, thanks! Both of those IDs were generated at some time between 20:30 and 21:30 (both UTC, I think) on January 3rd; overall, that time range saw

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. In T268625#6651847 , @Lydia_Pintscher wrote: > In T268625#6651803 , @Lucas_Werkmeister_WMDE wrote: > >> Even if the rate limit isn’t the

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

MisterSynergy added a comment. More: only 77 items have been created in the past 30 days using Relator (per recentchanges table), 67 of them by User:Animalparty [1]. This user also created some items related to "Edwin M. Post" recently [2][3][4]. The other Relator users are User:Ayack (5

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

MisterSynergy added a comment. Re. "Maggie Rogers": the request does not create an item since the input is not formatted correctly. If I throw this exact input to the API using pywikibot's editentity function [1], I get this error message: "WARNING: API error not-recognized-language: The

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. Alright, thanks! Both of those IDs were generated at some time between 20:30 and 21:30 (both UTC, I think) on January 3rd; overall, that time range saw 3058 IDs being generated, which I downloaded. Of those 3058 log events, 2954 hit /w/api.php (the

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. Ah, one more thing. The Mary Margaret Salisbury edits don’t have an edit summary, but the Edwin M. Post, Jr. ones do: > New item using [[:toollabs:wikidata-todo/relator|Relator]] TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

MisterSynergy added a comment. @Lucas_Werkmeister_WMDE: We have not seen excessive phases of QID skipping since the week ending December 5. We skip around 10k–50k QIDs per week since then which seems pretty "normal". However, some regions in the QID space are still only sparsely

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. Number of log events over the past 24 hours (from the Logstash dashboard linked in T268625#6688794 ): F33995859: Screenshot_2021-01-11 Wikibase IdGenerator (T268625) - Kibana.png

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lydia_Pintscher added a comment. Can we have a look at the logs and see where we currently stand? TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Lydia_Pintscher Cc:

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. Alright, the logging is now enabled on real Wikidata too. When the next burst of skipped IDs comes, we should be able to see in the logs where it comes from. TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-12-14T13:45:44Z] Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:649321|Enable Wikibase Repo ID generator logging on Wikidata (T268625 )]] (duration: 00m

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 649321 **merged** by jenkins-bot: [operations/mediawiki-config@master] Enable Wikibase Repo ID generator logging on Wikidata https://gerrit.wikimedia.org/r/649321 TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 649321 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [operations/mediawiki-config@master] Enable Wikibase Repo ID generator logging on Wikidata https://gerrit.wikimedia.org/r/649321 TASK DETAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. Logstash dashboard for the generated log messages (so far only Test Wikidata): https://logstash.wikimedia.org/app/kibana#/dashboard/AXZhbSkta1L68BhcA_cd TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-12-14T12:54:23Z] Synchronized wmf-config/Wikibase.php: Config: [[gerrit:644564|Enable Wikibase Repo ID generator logging on Test Wikidata (T268625 )]] (2/2) (duration: 00m

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-12-14T12:53:16Z] Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:644564|Enable Wikibase Repo ID generator logging on Test Wikidata (T268625 )]] (1/2)

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 644564 **merged** by jenkins-bot: [operations/mediawiki-config@master] Enable Wikibase Repo ID generator logging on Test Wikidata https://gerrit.wikimedia.org/r/644564 TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-12-14T12:45:06Z] Synchronized wmf-config/InitialiseSettings-labs.php: Config: [[gerrit:643874|Add log channel Wikibase.IdGenerator (T268625 )]] (Beta-only sync to avoid

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-12-14T12:43:55Z] Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:643874|Add log channel Wikibase.IdGenerator (T268625 )]] (duration: 00m 54s) TASK DETAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 643874 **merged** by jenkins-bot: [operations/mediawiki-config@master] Add log channel Wikibase.IdGenerator https://gerrit.wikimedia.org/r/643874 TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE added a comment. Since wmf.20 is still blocked (T263186 ), I don’t think we can backport the logging changes today. And that means it probably doesn’t make much sense to backport them at all – at this point, I figure we might

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 644564 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [operations/mediawiki-config@master] Enable Wikibase Repo ID generator logging on Test Wikidata https://gerrit.wikimedia.org/r/644564 TASK DETAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.21; 2020-12-08). TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, ReleaseTaggerBot Cc: MisterSynergy,

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 643882 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Log more messages in LoggingIdGenerator https://gerrit.wikimedia.org/r/643882 TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 643760 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Introduce LoggingIdGenerator https://gerrit.wikimedia.org/r/643760 TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 643882 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Wikibase@master] Log more messages in LoggingIdGenerator https://gerrit.wikimedia.org/r/643882 TASK DETAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 643874 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [operations/mediawiki-config@master] Add log channel Wikibase.IdGenerator https://gerrit.wikimedia.org/r/643874 TASK DETAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, gerritbot Cc: MisterSynergy, Tonina_Zhelyazkova_WMDE, Addshore, Pablo-WMDE,

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

gerritbot added a comment. Change 643760 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Wikibase@master] WIP: Introduce LoggingIdGenerator https://gerrit.wikimedia.org/r/643760 TASK DETAIL

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lydia_Pintscher added a comment. In T268625#6651803 , @Lucas_Werkmeister_WMDE wrote: > Even if the rate limit isn’t the current cause of the problem, adding a rate limit for entity ID generation seems like an elegant solution for me –

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lucas_Werkmeister_WMDE claimed this task. Lucas_Werkmeister_WMDE moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Lucas_Werkmeister_WMDE added a comment. Copying from the parent task: In T44362#6645565

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

Lydia_Pintscher triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lydia_Pintscher Cc: MisterSynergy, Tonina_Zhelyazkova_WMDE, Addshore, Pablo-WMDE,

[Wikidata-bugs] [Maniphest] T268625: [20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items

ItamarWMDE renamed this task from "Investigate the significant number of skipped Item IDs for newly created Wikidata items" to "[20h] Investigate the significant number of skipped Item IDs for newly created Wikidata items". TASK DETAIL https://phabricator.wikimedia.org/T268625 EMAIL