Re: File Comparison

[Gregg Morrison]

Title: File Comparison

Ken       Here is how I would do it:       Read your entire NUT.DAT file into an array. - This is your 'Smaller' File     Join it with a space delimiter so in memory it looks like:     Must use a delimiter to keep 11-byte groups from running together.       "12345678912 56789876543 23456789652 .."       Then read each line in your database DATA.DAT file and:                  TODO:  Maybe do a test to see if split is faster than                         $DATA = "">                                                                    #  maybe ($DATA, $junk) = split( / /, $input, 2); is faster?       Use favorite Regex to Look for $DATA anywhere in your Joined "NUT_DAT" string.     if it is >= 0 then save that Source line to BOX.DAT            you might also want to count the matched records.       close files       This will be immensely faster than 'Brute force' method.         Hope this solves your problem.          Gregg Morrison     (  [EMAIL PROTECTED] )       ----- Original Message ----- From: Kenneth Jideofor [ MTN - Ikoyi ] To: [EMAIL PROTECTED] Sent: Thursday, September 04, 2003 4:17 PM Subject: File Comparison Hi Guys, I have two files, nut.dat and data.dat. The first file, nut.dat, contains lines of eleven digit figures; each line in this file is an eleven-digit figure. The second file, data.dat, contains lines of spaced figures; each line having four groups of eleven-digit figures. In other words, each line in the file, nut.dat, has one field, while each line in the file, data.dat, has four fields. I want to compare each field in the nut.dat file against the first field of each line in the data.dat file. Where the field in the nut.dat file matches the first field in any of the lines in the data.dat file, the entire line in the data.dat file is saved to a file, box.dat. For example, the content of each file can be viewed as follows: nut.dat file: 12345678912 56789876543 23456789652 34567123456 data.dat file: 12345678912  23456098734  12348907678  67342519806 23456789652  87456321452  45231987564  23675843902 34567123456  23456709819  25361728980  49872653418 I wrote the following Perl script to perform the above task. The script works very fast for a small data.dat file while it is extremely very slow with a very huge data.dat file. I need to make the script faster. Could you, please, assist me with an improved version of the script? Regards, Ken ____________ My Perl Script______________________ #!/bin/perl -w open (FILE1, "/opt/MISC/AUDIT/nut.dat"); open (FILE2, "/opt/MISC/AUDIT/data.dat"); open (OUT, ">>/opt/MISC/AUDIT/box.txt"); @nuts = <FILE1>; @database = <FILE2>; close(FILE1); close(FILE2); foreach $nut(@nuts) {         chomp($nut); foreach $database(@database) {         chomp(@database); $data = "">         if ($data eq $nut) {                 print OUT "$database\n";                 }               } } close(OUT);         ******************** DISCLAIMER STATEMENT ******************** This e-mail message is private and confidential with its contents and attachments are the property of MTN Nigeria for the named addressee. It is solely intended for a specific addressee and purpose.  If you are not the addressee (a) you may not disclose, copy, distribute or take any action based on the contents hereof; (b) kindly inform the sender immediately or email - [EMAIL PROTECTED] and destroy all copies hereof.  Any unauthorized use or interception of this email, or the review, retransmission, dissemination or other use of, or taking of any action in reliance upon the contents of this email, by persons or entities other than the intended recipient, is prohibited. Save for communications relating to the official business of MTN Nigeria, MTN Nigeria does not accept any responsibility for the contents of this email or any opinions expressed in this email or its attachments. Due to the nature of email MTN Nigeria cannot ensure and accepts no liability for the integrity of this email and any attachments, nor that they are free of any virus. MTN Nigeria accepts no liability for any loss or damage whether direct or indirect or consequential, however caused, whether by negligence or otherwise, which may result directly or indirectly from this communication or any attached files. This message does not constitute a guarantee or proof of the facts mentioned herein.