Addshore added a comment.
In T219123#5844088 <https://phabricator.wikimedia.org/T219123#5844088>, @Addshore wrote: > Now at Rebuilding Q8456304 till Q8456447. > While checking in SQL with queries like the following I noticed a few smaller holes being left behind: > > addshore@stat1007:~$ analytics-mysql wikidatawiki -e "SELECT DISTINCT wbit_item_id as id FROM wbt_item_terms LEFT JOIN wbt_term_in_lang ON wbit_term_in_lang_id = wbtl_id LEFT JOIN wbt_type ON wbtl_type_ id = wby_id LEFT JOIN wbt_text_in_lang ON wbtl_text_in_lang_id = wbxl_id LEFT JOIN wbt_text ON wbxl_text_id = wbx_id WHERE wbx_text IS NULL AND wbit_item_id < 1000000 AND wbit_item_id > 0000000;" -N -B > > 0to1holes > addshore@stat1007:~$ wc -l 0to1holes > 6714 0to1holes > > SO We will have to do 1 more pass to try to pick up these final holes at some point I re ran this query a week or so on and the number of holes is less, which is good. It feels like T244115 <https://phabricator.wikimedia.org/T244115> is likely the last hole creating issue which is acceptable. addshore@stat1007:~$ analytics-mysql wikidatawiki -e "SELECT DISTINCT wbit_item_id as id FROM wbt_item_terms LEFT JOIN wbt_term_in_lang ON wbit_term_in_lang_id = wbtl_id LEFT JOIN wbt_type ON wbtl_type_ id = wby_id LEFT JOIN wbt_text_in_lang ON wbtl_text_in_lang_id = wbxl_id LEFT JOIN wbt_text ON wbxl_text_id = wbx_id WHERE wbx_text IS NULL AND wbit_item_id < 1000000 AND wbit_item_id > 0000000;" -N -B > 0to1holes-again addshore@stat1007:~$ wc -l 0to1holes-again 4360 0to1holes-again Before reading from sections of the new store we still want to finish filling in these known holes as a result of the first hadoop based migration. I'll generate files and do that and keep this ticket updated. TASK DETAIL https://phabricator.wikimedia.org/T219123 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Addshore Cc: Ladsgroup, Addshore, Jdforrester-WMF, ArielGlenn, Aklapper, alaa_wmde, Iflorez, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
