[Wikidata-bugs] [Maniphest] T309445: [Investigation] Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc: Lydia_Pintscher, karapayneWMDE, Addshore, Manuel,

[Wikidata-bugs] [Maniphest] T309445: [Investigation] Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE closed subtask T313039: Remove debug logging for item terms storage after merging as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc:

[Wikidata-bugs] [Maniphest] T309445: [Investigation] Terms storage missing most item labels after merge, resulting in item IDs being shown

Manuel renamed this task from "INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown" to "[Investigation] Terms storage missing most item labels after merge, resulting in item IDs being shown". TASK DETAIL

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Manuel edited projects, added Wikidata-Campsite; removed Wikidata Dev Team (Sprint-♾), Special:NewLexeme revival (Special:NewLexeme revival - sprint 11), Wikidata-Campsite (Team A Hearth ). Manuel added a comment. Ah okay I see, yes that makes sense! Thank you, Lucas, for everything

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. No, I don’t think so. The merge gadget could in theory do it, but I’m not sure that would be a good idea, since doing the purge right after the merge probably doesn’t help with the race condition. TASK DETAIL

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Manuel added a comment. Is the "purge with links update" something we should do routinely after merges? TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Manuel Cc:

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Trying something slightly different. I think the Still Life with Pie and Silver Ewer , inspected previously (T309445#8055711

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. In T309445#8055711 , @Lucas_Werkmeister_WMDE wrote: > Also, mysteriously, the “schedule deleteTermsOfEntity” log message seems to be missing – not just for this merge, but in general, there are no

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

karapayneWMDE added a project: Wikidata Dev Team (Sprint-♾). TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, karapayneWMDE Cc: Lydia_Pintscher, karapayneWMDE, Addshore,

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.21; 2022-07-18). TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, ReleaseTaggerBot Cc: Lydia_Pintscher,

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. (I had a quick look at mw1358 and mw1448, and their clocks seemed to be no more than 20 ms apart, not enough to fully explain the timeline being confusing. Also, we seem to be running `systemd-timesyncd` in production, with Wikimedia-specific time

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Found a new example, Still Life with Pie and Silver Ewer . The merge added a German label, one Dutch and English alias each, and a West Frisian description; the West Frisian

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Maintenance_bot Cc: Lydia_Pintscher, karapayneWMDE, Addshore, Manuel,

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

gerritbot added a comment. Change 810921 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Add debug logging for term storage cleanup https://gerrit.wikimedia.org/r/810921 TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, gerritbot Cc: Lydia_Pintscher, karapayneWMDE, Addshore, Manuel,

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

gerritbot added a comment. Change 810921 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Wikibase@master] Add debug logging for term storage cleanup https://gerrit.wikimedia.org/r/810921 TASK DETAIL

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Coming back to José Barreto for a moment, since that item hasn’t been edited yet and is still missing the terms: The missing alias («José Alberto Barreto», Spanish) is not only missing from the term store as a whole; it’s not even present in the

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Okay, I found another broken merge. Museo diocesano di Capua gained an Italian alias through a merge, but in the entityterms API

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Although I //still// don’t see an exception corresponding to José Barreto (from T309445#8023533 ), even when just looking

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. In T309445#8028484 , @Lucas_Werkmeister_WMDE wrote: > …and while working on adding the logging, I suddenly find this: > > name=ItemTermStoreWriterAdapter::saveTermsOfEntity() > if

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE closed subtask T311307: Add debug logging for item terms storage after merging as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc:

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

karapayneWMDE moved this task from Special:NewLexeme revival - sprint 10 to Special:NewLexeme revival - sprint 11 on the Special:NewLexeme revival board. karapayneWMDE edited projects, added Special:NewLexeme revival (Special:NewLexeme revival - sprint 11); removed Special:NewLexeme revival

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. In T309445#8023699 , @Lydia_Pintscher wrote: > Does any of the lists of potential merge candidates at https://www.wikidata.org/wiki/Help:Merge#Finding_items_to_merge help? Thanks, I might try

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. …and while working on adding the logging, I suddenly find this: name=ItemTermStoreWriterAdapter::saveTermsOfEntity() if ( $entity instanceof Item ) { try {

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. The simplest explanation for this whole business would be that the secondary data updates somehow just don’t happen. I don’t know why this would be – of course it’s possible that they crash for some reason, but I’d expect to see something in logstash

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. I tested it: a regular `index.php?action=purge` or `action.php?action=purge` doesn’t trigger secondary data updates, not even when you visit the page afterwards; `api.php?action=purge=1` does trigger secondary data updates. The merge gadget doesn’t

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE claimed this task. Lucas_Werkmeister_WMDE moved this task from To Do to Doing on the Special:NewLexeme revival (Special:NewLexeme revival - sprint 10) board. TASK DETAIL https://phabricator.wikimedia.org/T309445 WORKBOARD

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lydia_Pintscher added a comment. Does any of the lists of potential merge candidates at https://www.wikidata.org/wiki/Help:Merge#Finding_items_to_merge help? TASK DETAIL https://phabricator.wikimedia.org/T309445 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Okay, but after 50 repeats of the code from earlier (i.e. 100 items, 50 merges), I still haven’t been able to reproduce the issue locally once. So now I’m back to thinking that the issue doesn’t

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Recent merges: - Perdikas Palaeontological Museum : the merge added an English alias which //is// present in the term store. - Cecilio Angulo

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Found and merged another pair , again unable to reproduce the issue: name=before merge MariaDB [wikidatawiki]> SELECT wbit_item_id, wby_name, wbx_text, wbxl_language

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. Managed to find a pair of duplicates via the recent changes tagged as sitelink change from connected wiki (the dewiki and ptwiki articles were

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

Lucas_Werkmeister_WMDE added a comment. In T309445#7966722 , @Lucas_Werkmeister_WMDE wrote: > Tried reproducing this locally with the following snippet in the dev tools console … But didn’t succeed in reproducing the error, both labels

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

ItamarWMDE moved this task from Tech backlog to Special:NewLexeme revival - sprint 10 on the Special:NewLexeme revival board. ItamarWMDE edited projects, added Special:NewLexeme revival (Special:NewLexeme revival - sprint 10); removed Special:NewLexeme revival. TASK DETAIL

[Wikidata-bugs] [Maniphest] T309445: INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown

karapayneWMDE renamed this task from "Terms storage missing most item labels after merge, resulting in item IDs being shown" to "INVESTIGATION: Terms storage missing most item labels after merge, resulting in item IDs being shown". karapayneWMDE updated the task description. karapayneWMDE set