Re: [MCN-L] "easy" file duplication cleanup

2015-12-08 Thread Josh McDonald
As Matt implies above, this isn't too difficult with standard *nix utilities if the files are actually duplicates. I use a tip from Jim McNamara to do this in single directories; compare checksums and dump dupes into a file for review, then delete known duplicates: cksum *.jpg | sort -n >

[MCN-L] "easy" file duplication cleanup

2015-12-07 Thread Perian Sully
Hi everyone: I know this is possibly something of a fool's errand, but I'm hoping someone has come up with some magic tool or process for more-easily cleaning up file storage than going through 12 years of files one-by-one. As part of our DAMS project, I've run some TreeSize Pro scans on three

Re: [MCN-L] "easy" file duplication cleanup

2015-12-07 Thread Matt Morgan
You could do this with a shell script. One way: write a `find -exec ...` that runs through all the files, outputting the md5sums in some usable way. Sort the list and look for multiples (double-checking with diff on matches, if you're worried), and replace duplicates with symlinks if/where you