Mertol, You probably want to check out the "e2scan" work that is being done atm. There's a bugzilla entry for it (NERSC). As this was not available to us a couple of months ago we came up with a way to snapshot both MDSs and then run "rsync --itemize-changes" between them. This builds a filelist that we can then split into deletes and copies. For the copies you can split it N times and use multiple "rsync --files-from=" instance across many clients.
The filelist generation takes around 6-8 hours and the copies around 5 hours for our 72Tb filesystem. Fast enough for us to get a backup through every 24 hours. But like I said a better/faster longterm solution is to use e2scan. Daire ----- "Mertol Ozyoney" <[EMAIL PROTECTED]> wrote: > _______________________________________________ > Lustre-discuss mailing list > [email protected] > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
