JAllemandou added a comment.
Most of the complicated things already exist for this to work (equicalent of rsync for HDFS, spark job converting wikidata json dumps to parquet). I wanted for T216160 <https://phabricator.wikimedia.org/T216160> to be settled before moving into productionization (having the same date for the various dumps we handle simplifies quite a bit), and it takes time. TASK DETAIL https://phabricator.wikimedia.org/T209655 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JAllemandou Cc: leila, Ottomata, Nuria, GoranSMilovanovic, Addshore, JAllemandou, bmansurov, alaa_wmde, Nandana, Akovalyov, Lahi, Gq86, QZanden, LawExplorer, Avner, _jensen, rosalieper, Wikidata-bugs, aude, Capt_Swing, Mbch331, jeremyb
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
