This is probably a problem with the regression test. The two conditions ought to be identical, as you expect. However, keep in mind that the distortion model is incredibly weak, and the heuristic distance definition used in lattice decoding is also just an approximation, so an off-by-one error isn't the end of the world. Chris
On Mon, Jul 20, 2009 at 9:52 AM, Christian Hardmeier<[email protected]> wrote: > Hi! > > I've noticed that there is a difference between the distortion costs > used by moses when decoding a plain text sentence and those used when > decoding a completely equivalent word lattice (that is, a set of nodes > linearly connected by a single path). In particular, the distortion cost > for going back in the input sentence is lower in the lattice case than > in the text case. > > I found the reason for this difference in WordLattice.cpp, where the > corresponding distance is calculated as follows: > > //TRACE_ERR("returning reverse distance from "<< (prev.GetEndPos()+1) << " to > " << (current.GetStartPos()+1) << " which is " << > (distances[prev.GetEndPos()+1][current.GetStartPos()+1] - 1) <<"\n"); > result = distances[prev.GetEndPos() + 1][current.GetStartPos()] - 1; > if (result < 0 || result > 99999) { > TRACE_ERR("prev: " << prev << "\n current: " << current << "\n"); > > TRACE_ERR("C: got a weird distance from "<< prev.GetEndPos()+1 << " to > " << current.GetStartPos() << " of " << result << "\n"); > } > > There is an obvious discrepancy between the position indices > current.GetStartPos()+1 in the TRACE_ERR and current.GetStartPos() in > the result calculation. Using current.GetStartPos()+1 for the > calculation gives me results that are identical to the sentence-decoding > case, as I would expect, but now the decoder fails the regression test. > > Could somebody please explain to me why the different distortion costs > in the two conditions should be expected behaviour? Or is there a > problem with the regression test? > > Thanks! > Christian > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
