Ø I would counsel in the strongest possible terms against that approach. Ø Basically you have to be assured that none of your file names have "wacky" characters in them, because handling "wacky" characters in file
Ø names is exceedingly difficult. I cannot stress how hard it is and the above example does not handle all "wacky" characters in file names. Well that’s indeed another kettle of fish if you have irregular/special naming of files, no I didn’t cover that and if you have millions of files, yes a list would be unwieldy, then I would be tarring up dirs. before moving… and then untarring on GPFS or breaking up the list into sets or sub lists. If you have these wacky types of file names well there are fixes as in the rsync manpages… yes not easy but possible.. Ie 1. -s, --protect-args 2. As per usual you can escape the spaces, or substitute for spaces. rsync -avuz [email protected]:"${remote_path// /\\ }" . 3. Single quote the file name and path inside double quotes. Ø One thing I didn't mention is that I would run anything with in a screen (or tmux if that is your poison) and turn on logging. Absolutely agree… Ø For those interested I am in the process of cleaning up the script a bit and will post it somewhere in due course. Ø JAB. Would be interesting to see…. I’ve also had success on GPFS with DCP and possibly this would be another option Regards, Chris Schlipalius Team Lead, Data Storage Infrastructure, Supercomputing Platforms, Pawsey Supercomputing Centre (CSIRO) 1 Bryce Avenue Kensington WA 6151 Australia
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
