Hi!
I've noticed that there is a difference between the distortion costs
used by moses when decoding a plain text sentence and those used when
decoding a completely equivalent word lattice (that is, a set of nodes
linearly connected by a single path). In particular, the distortion cost
for going back in the input sentence is lower in the lattice case than
in the text case.
I found the reason for this difference in WordLattice.cpp, where the
corresponding distance is calculated as follows:
//TRACE_ERR("returning reverse distance from "<< (prev.GetEndPos()+1) << " to "
<< (current.GetStartPos()+1) << " which is " <<
(distances[prev.GetEndPos()+1][current.GetStartPos()+1] - 1) <<"\n");
result = distances[prev.GetEndPos() + 1][current.GetStartPos()] - 1;
if (result < 0 || result > 99999) {
TRACE_ERR("prev: " << prev << "\n current: " << current << "\n");
TRACE_ERR("C: got a weird distance from "<< prev.GetEndPos()+1 << " to "
<< current.GetStartPos() << " of " << result << "\n");
}
There is an obvious discrepancy between the position indices
current.GetStartPos()+1 in the TRACE_ERR and current.GetStartPos() in
the result calculation. Using current.GetStartPos()+1 for the
calculation gives me results that are identical to the sentence-decoding
case, as I would expect, but now the decoder fails the regression test.
Could somebody please explain to me why the different distortion costs
in the two conditions should be expected behaviour? Or is there a
problem with the regression test?
Thanks!
Christian
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
