On Wed, May 04, 2005 at 10:12:18AM +0800, roderick tapang wrote:
> how do you remove duplicate line entries in a file
> using perl? we have some logfiles that are xferred
> to a windows machine (don't ask why) for analysis
> and we only have ActivePerl available in it and one
> staff needs to have the duplicate lines remove.
> i really don't do perl so can anyone give out some
> lines of codes for this?
>
> (i posted this in ph-perl but didn't get any response
> yet).
>
Try this:
#!/usr/bin/perl
# usage: remove_dup_in_place.pl file1 file2 ... fileN
while (<>) {
if ($ARGV ne $oldargv) {
open(ARGVOUT, ">$ARGV");
$oldargv = $ARGV;
undef %hash;
}
$hash{$_}++;
print ARGVOUT sort keys %hash
if (eof and defined fileno(ARGVOUT));
}
exit;
--
$_=q:; # SHERWIN #
70;72;69;6e;74;20;
27;4a;75;73;74;20;
61;6e;6f;74;68;65;
72;20;50;65;72;6c;
20;6e;6f;76;69;63;
65;27;:;;s=~?(..);
?=pack q$C$,hex$1;
;;;=egg;;;;eval;;;
_________________________________________________
Philippine Linux Users' Group (PLUG) Mailing List
[email protected] (#PLUG @ irc.free.net.ph)
Read the Guidelines: http://linux.org.ph/lists
Searchable Archives: http://archives.free.net.ph
