Hi all, I have some questions about HTER computation as explained in Snover et al., 2006, AMTA. The paper states that the denominator term should not use the length of the targeted reference (footnote 5) as this could be exploited by the annotators to favour longer targeted references in order to get better (lower) HTER scores.
While this might be reasonable, I thunk such long targeted references will also incur additional edit distance costs (say for deletion), increasing the numerator and thereby resulting in poor (higher) HTER score. Isn't this true? In that case isn't better to just use the length of the targeted reference in denominator? Secondly they don't say what to use instead for the denominator. One possibility would be to use the length of the untargeted reference, but not sure if this is correct. And if there are multiple references, should we just take the average reference length? Any clarifications/ comments would be appreciated. Thanks - Baskaran
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
