Hi Jonathan, yes you are correct! but we plan to resync this once or twice every week for the next 3-4months to be sure everything is as it should be.
Right now we are focused on getting them synced up and then we will run scheduled resyncs/checks once or twice a week depending on the data growth :) Thanks Andi Christiansen > On 11/17/2020 2:53 PM Jonathan Buzzard <[email protected]> wrote: > > > On 17/11/2020 11:51, Andi Christiansen wrote: > > Hi all, > > > > thanks for all the information, there was some interesting things > > amount it.. > > > > I kept on going with rsync and ended up making a file with all top > > level user directories and splitting them into chunks of 347 per > > rsync session(total 42000 ish folders). yesterday we had only 14 > > sessions with 3000 folders in each and that was too much work for one > > rsync session.. > > Unless you use something similar to my DB suggestion it is almost > inevitable that some of those rsync sessions are going to have issues > and you will have no way to track it or even know it has happened unless > you do a single final giant catchup/check rsync. > > I should add that a copy of the sqlite DB is cover your backside > protection when a user pops up claiming that you failed to transfer one > of their vitally important files six months down the line and the old > system is turned off and scrapped. > > > JAB. > > -- > Jonathan A. Buzzard Tel: +44141-5483420 > HPC System Administrator, ARCHIE-WeSt. > University of Strathclyde, John Anderson Building, Glasgow. G4 0NG > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
