|
hi amit --
while thinking about the code i sent you in my last e-mail, i realized
there is an error.
the error can be demonstrated by the following:
C:[EMAIL PROTECTED]>perl t_grams1.pl "no now know"
``no now know'' has...
3 unigrams 1 know 1 no 1 now 2 bigrams 1 no now 1 now know 1 trigram 1 no now know 1.000000 = `no now' (1) / `no' (1)
1.000000 = `no now' (1) / `now' (1) 1.000000 = `now know' (1) / `know' (1) 1.000000 = `now know' (1) / `no' (1) <---- WRONG!!! 1.000000 = `now know' (1) / `now' (1) 1.000000 = `no now know' (1) / `no now' (1)
1.000000 = `no now know' (1) / `now know' (1) this error was generated in the following (fixed) function:
sub print_gram_ratios {
my ($hr_dividend_gram, # n_gram to be divided: the
`larger' n_gram
$hr_divisor_gram, # n_gram that will divide: the `smaller' n_gram ) = @_; for my $dividend_key (sort keys %$hr_dividend_gram)
{
for my $divisor_key (sort keys
%$hr_divisor_gram) {
# go to
next divisor unless divisor key string is
present
# somewhere in dividend key string. next unless $dividend_key =~ / \b \Q$divisor_key\E \b /x; # next unless $dividend_key =~ /$divisor_key/; WRONG!!! # print
ratio = n_gram_dividend /
n_gram_divisor
print_gram_ratio($hr_dividend_gram, $dividend_key, $hr_divisor_gram, $divisor_key); }
}
}
the corrected function produces (what i believe to be) the correct
output:
C:[EMAIL PROTECTED]>perl t_grams1.pl "no now know"
``no now know'' has...
3 unigrams 1 know 1 no 1 now 2 bigrams 1 no now 1 now know 1 trigram 1 no now know 1.000000 = `no now' (1) / `no' (1)
1.000000 = `no now' (1) / `now' (1) 1.000000 = `now know' (1) / `know' (1) 1.000000 = `now know' (1) / `now' (1) 1.000000 = `no now know' (1) / `no now' (1)
1.000000 = `no now know' (1) / `now know' (1) note that some fragility still exists in the code in general in that
it is possible
that a disjunction can develop between the definition of a
``word'', the separator string,
and the regex assertion used to detect the presence of an n-gram within an
(n+1)-gram.
(and, of course, i'm sure there are other weaknesses i DON'T know
about.)
if this was a professional application, i'd say some more work was needed
to ``tie it all
together''!
good night and good luck -- bill walters
|
_______________________________________________ ActivePerl mailing list [email protected] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
