Dear Moses Community,
When using mteval-13a.pl, we note that the output looks like this:
length ratio: 1.07303974221267 (1998/1862), penalty (log): 0
NIST score = 5.0564 BLEU score = 0.2318 for system "Google"
# ------------------------------------------------------------------------
Individual N-gram scoring
1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
8-gram 9-gram
------ ------ ------ ------ ------ ------ ------
------ ------
NIST: 4.4488 0.5554 0.0477 0.0045 0.0000 0.0000 0.0000
0.0000 0.0000 "Google"
BLEU: 0.5415 0.2972 0.1752 0.1025 0.0626 0.0354 0.0193
0.0085 0.0017 "Google"
# ------------------------------------------------------------------------
Cumulative N-gram scoring
1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
8-gram 9-gram
------ ------ ------ ------ ------ ------ ------
------ ------
NIST: 4.4488 5.0043 5.0520 5.0564 5.0564 5.0564 5.0564
5.0564 5.0564 "Google"
BLEU: 0.5415 0.4012 0.3044 0.2318 0.1784 0.1362 0.1031
0.0754 0.0493 "Google"
And at
https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/mteval-v13a.pl#L823,
it tries to calculate the cumulative score by accumulate the individual
ngram precisions and at each order of ngram add to it and do a
normalization before calculating the cumulative score for each order of
nrgram.
The question is why does it add the brevity penalty? (i.e. $len_score)
Also, is this score discussed in any paper?
Thanks in advance for the clarifications!
Regards,
Nat
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
