Agreed, MBS. This is what sort, uniq, and friends are for. There are windows versions of most of these tools here: http://gnuwin32.sourceforge.net/ along with a utility to automatically download them here: http://getgnuwin32.sourceforge.net/
Matthew Topper > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Michael B. Smith > Sent: Tuesday, June 28, 2016 4:29 PM > To: [email protected] > Subject: RE: [NTSysADM] Compare two large lists > > diff. > > If you want a GUI, WinDiff. > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Kurt Buff > Sent: Tuesday, June 28, 2016 3:16 PM > To: ntsysadm > Subject: Re: [NTSysADM] Compare two large lists > > Of the tools I'm aware of. BeyondCompare (http://scootersoftware.com/) is > probably your best bet, but WinMerge (http://winmerge.org/) and some > other tools might handle this, especially if the data is sorted. > > I'm sure others can make recommendations as well. > > If you've got time and a lot of RAM, PowerShell can do this as well - take > each > entry in the short list, compare against the large list, write it to a file > if there's > a match. That's a *very* slow algorithm, but it will work. > > > > On Tue, Jun 28, 2016 at 11:02 AM, Richard Stovall <[email protected]> > wrote: > > Not necessarily Windows-related. > > > > I need to compare a list of about 300,000 file hashes against a larger > > list of ~30,000,000 and find ones that are represented in both data sets. > > > > I'm not a database guy, nor have I ever played one on TeeVee. > > > > Any ideas about how to go about this with standard/free tools in > > Windows or Linux? > > > > TIA, > > RS >
