On Tue, 25 Jul 2017 10:02:14 +0100, Jonathan Buzzard said: > I would be tempted to zip up the directories and move them ziped ;-)
Not an option, unless you want to come here and re-write the researcher's tracking systems that knows where they archived a given run, and teach it "Except now it's in a .tar.gz in that directory, or perhaps one or two directories higher up, under some name". Yes, researchers do that. And as the joke goes: "What's the difference between a tenured professor and a terrorist?" "You can negotiate with a terrorist..." Plus remember that most of these directories are currently scattered across multiple tapes, which means "zip up a directory" may mean reading as many as 10 to 20 tapes just to get the directory on disk so you can zip it up. As it is, I had to write code that recall and processes all the files on tape 1, *wherever they are in the file system*, free them from the source disk, recall and process all the files on tape 2, repeat until tape 3,857. (And due to funding issues 5 years ago which turned into a "who paid for what tapes" food fight, most of the tapes ended up with files from entirely different file systems on them, going into different filesets on the destination). (And in fact, the migration is currently hosed up because a researcher *is* doing pretty much that - recalling all the files from one directory, then the next, then the next, to get files they need urgently for a deliverable but haven't been moved to the new system. So rather than having 12 LTO-5 drives to multistream the tape recalls, I've got 12 recalls fighting for one drive while the researcher's processing is hogging the other 11, due to the way the previous system prioritizes in-line opens of files versus bulk recalls)
pgpFzC70wWOWf.pgp
Description: PGP signature
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
