With a dataset this large, I also recommend duff. -- Espi
On Tue, Jun 28, 2016 at 1:28 PM, Michael B. Smith <[email protected]> wrote: > diff. > > If you want a GUI, WinDiff. > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Kurt Buff > Sent: Tuesday, June 28, 2016 3:16 PM > To: ntsysadm > Subject: Re: [NTSysADM] Compare two large lists > > Of the tools I'm aware of. BeyondCompare (http://scootersoftware.com/) is > probably your best bet, but WinMerge (http://winmerge.org/) and some > other tools might handle this, especially if the data is sorted. > > I'm sure others can make recommendations as well. > > If you've got time and a lot of RAM, PowerShell can do this as well - take > each entry in the short list, compare against the large list, write it to a > file if there's a match. That's a *very* slow algorithm, but it will work. > > > > On Tue, Jun 28, 2016 at 11:02 AM, Richard Stovall <[email protected]> > wrote: > > Not necessarily Windows-related. > > > > I need to compare a list of about 300,000 file hashes against a larger > > list of ~30,000,000 and find ones that are represented in both data sets. > > > > I'm not a database guy, nor have I ever played one on TeeVee. > > > > Any ideas about how to go about this with standard/free tools in > > Windows or Linux? > > > > TIA, > > RS > > >
