jcrespo added projects: bacula, Data-Persistence-Backup, Data-Persistence. jcrespo added a comment.
number of files are (within reason) a non-blocker for bacula, as files are packaged into volumes. It is true that each file is stored as a mysql record, but that should be able to scale until dozens of (US) billons, although it may be slow to recover when rebuilding metadata. Most limiting factor would be the overall size + backup frequency for capacity planning. We don't have a lot of temporal data backed up, so not sure if we could come up with a strategy that saves space (e.g. if data is immutable, we may want to avoid full backups every day). What is the file/directory structure? If data is below e.g. 100GB I would consider it "small" and not requiring optimization. The typical backup schedule is incrementals of a set of paths every day, differentials every fortnite, and fulls monthly- however it is highly customizable per job. TASK DETAIL https://phabricator.wikimedia.org/T294355 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: jcrespo Cc: jcrespo, Manuel, Michael, Addshore, fgiunchedi, Aklapper, Lucas_Werkmeister_WMDE, Bongo-Cat, Invadibot, Devnull, LSobanski, maantietaja, lmata, Akuckartz, Nandana, Robin.guo, Lahi, Gq86, herron, GoranSMilovanovic, QZanden, Marostegui, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
