Thanks for that reference. I'm assuming you meant http://duff.sf.net.
However, while searching for it, I ran across this: https://en.wikipedia.org/wiki/Comparison_of_file_comparison_tools Seems like a useful addition to the pile of recommendations. Kurt On Tue, Jun 28, 2016 at 1:45 PM, Micheal Espinola Jr <[email protected]> wrote: > With a dataset this large, I also recommend duff. > > -- > Espi > > > On Tue, Jun 28, 2016 at 1:28 PM, Michael B. Smith <[email protected]> > wrote: >> >> diff. >> >> If you want a GUI, WinDiff. >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Kurt Buff >> Sent: Tuesday, June 28, 2016 3:16 PM >> To: ntsysadm >> Subject: Re: [NTSysADM] Compare two large lists >> >> Of the tools I'm aware of. BeyondCompare (http://scootersoftware.com/) is >> probably your best bet, but WinMerge (http://winmerge.org/) and some other >> tools might handle this, especially if the data is sorted. >> >> I'm sure others can make recommendations as well. >> >> If you've got time and a lot of RAM, PowerShell can do this as well - take >> each entry in the short list, compare against the large list, write it to a >> file if there's a match. That's a *very* slow algorithm, but it will work. >> >> >> >> On Tue, Jun 28, 2016 at 11:02 AM, Richard Stovall <[email protected]> >> wrote: >> > Not necessarily Windows-related. >> > >> > I need to compare a list of about 300,000 file hashes against a larger >> > list of ~30,000,000 and find ones that are represented in both data >> > sets. >> > >> > I'm not a database guy, nor have I ever played one on TeeVee. >> > >> > Any ideas about how to go about this with standard/free tools in >> > Windows or Linux? >> > >> > TIA, >> > RS >> >> >
