BTullis added a comment.
In T334951#8836832 <https://phabricator.wikimedia.org/T334951#8836832>, @Manuel wrote: > @AndrewTavis_WMDE is our newly hired Data Analyst for Wikidata. The plan is that he will mainly work on this with support from @xcollazo (WMF), @ItamarWMDE (Staff Engineer for Wikidata), and me (Analytics Product Manager for Wikidata). We plan to first evaluate the situation of the WDCM in June. Ideally, we would start the migration only based on that evaluation. > > Would that fit your plans, or is there already a risk of losing data by then? Great. I don't believe that there is any risk of losing data by then. Spark2 will no longer be available when the Hadoop cluster is upgraded to Debian bullseye, but that's a little way off yet. It might also be relevant to look at those parts of the job that currently run on the stats servers. By the look of it that's stat1004 and stat1007. Both of these run Debian buster. We're just starting to bring in bullseye based stats servers: e.g. T336036: Bring stat1009 into service <https://phabricator.wikimedia.org/T336036> and T336040: Bring stat1010 into service with GPU from stat1005 <https://phabricator.wikimedia.org/T336040> at which point we will start on decommissioning some buster based stats servers and upgrading those that remain. I'm not sure what plans @xcollazo has for the migration either, so it could be that these parts of the process are migrated away from the stats servers to an airflow based pipeline. Anyway, I'm happy to try to help if I can, but Xabriel almost certainly knows more about the plan than I do. TASK DETAIL https://phabricator.wikimedia.org/T334951 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: xcollazo, BTullis Cc: ItamarWMDE, BTullis, GoranSMilovanovic, AndrewTavis_WMDE, Aklapper, Manuel, JAllemandou, lbowmaker, xcollazo, Astuthiodit_1, EChetty, karapayneWMDE, Invadibot, maantietaja, Akuckartz, Nandana, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
