Giorgos Keramidas writes:
> The main drawback of being unable to use the `freebsd' wordlist
> is that you will get many false positives for words that are
> perfectly valid for FreeBSD documentation but are not standard
> English words.
I have a script which does something similar, using ispell.
It's based on the Perl script - found on-line - appended below.
I pseudo-fixed that running the output through sort and
starting with least frequent hits.
Attempts to build a project-specific dictionary proved too
confusing and it was ultimatly not worth the effort.
Robert Huff
#!/usr/local/bin/perl -W
# WordFreq.pl -- Count word frequency in a text file
$ver = "v1.0"; # 05-Dec-2001 JP Vossen [EMAIL PROTECTED]>
# Basics from 8.3, page 280 of _Perl_Cookbook_
# Added stop words
(($myname = $0) =~ s/^.*(\/|\\)|\..*$//ig); # remove up to last "\" or "/" and
after any "."
$Greeting = ("$myname $ver Copyright 12001 JP Vossen
(http://www.jpsdomain.org/)\n");
$Greeting .= (" Licensed under the GNU GENERAL PUBLIC LICENSE:\n");
$Greeting .= (" See http://www.gnu.org/copyleft/gpl.html for full text and
details.\n"); # Version and copyright info
%seen = (); # Create the hash
# Define the stopwords
@stopwords = ("a", "an", "and", "are", "as", "at", "be", "but", "by",
"does", "for", "from", "had", "have", "her", "his", "if", "in", "is",
"it", "not", "of", "on", "or", "that", "the", "this", "to", "was",
"which", "with", "you");
if (("@ARGV" =~ /\?/) || (@ARGV > 5) || (@ARGV < 0)) { #if wrong # of args, or
a ? in args - die
print STDERR ("\n$Greeting\n\tUsage: $myname -i {infile} [-s]\n");
print STDERR ("\nIf -s is used, the list of stop words will NOT be
used.\n");
print STDERR ("The stopwords currently defined are:\n\n ");
foreach $stopword (@stopwords) {
print STDERR ("$stopword ");
} # end of foreach stopword
die ("\n");
}
use Getopt::Std; # User Perl5 built-in program argument handler
getopts('i:o:s'); # Define possible args.
if (! $opt_i) { $opt_i = "-"; } # If no input file specified, use STDIN
if (! $opt_o) { $opt_o = "-"; } # If no output file specified, use STDOUT
open (INFILE, "$opt_i") || die "$myname: error opening $opt_i $!\n";
open (OUTFILE, ">$opt_o") || die "$myname: error opening $opt_o $!\n";
print STDERR ("\n$Greeting\n");
while (<INFILE>) { # Read the input file
while ( /(\w['\w-]*)/g ) { # If we have a "word"
$seen{lc $1}++; # Count it in the hash
} # end of while words
} # end of while input
if (! $opt_s) { # If we're using stopwords
foreach $stopword (@stopwords) { # for each stopword
delete($seen{$stopword}); # Remove it from the hash
} # end of foreach stopword # This way we only test once for each
} # end of if using stopwords stopword, rather than in a loop!
# Print the results, sorted most frequent words at the top
foreach $word ( sort { $seen{$b} <=> $seen{$a} } keys %seen) {
printf OUTFILE ("%6d %s\n", $seen{$word}, $word);
} # end of foreach word
_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
