Hello all,
I seem to have encountered a bug in Moses in its handling of lattice inputs
(-inputtype 2): when I try to decode the lattice
((('klein', 0.1, 1), ('klein', 0.1, 2), ('klein', 0.1, 3),
('klein', 0.1, 4), ('ein', 1, 5)),
(('haus', 1, 5)),
(('haus', 1, 4)),
(('haus', 1, 3)),
(('haus', 1, 2)),
(('haus', 1, 1)))
(on 1 line) with the model www.statmt.org/moses/download/sample-models.tgz,
using the same moses.ini file as in the tarball, and with a maximum phrase
length of 2, i.e.
moses -f moses.ini -inputtype 2 -weight-i 1 -max-phrase-length 2 <input.lat
then Moses will crash (with a "St9bad_alloc" exception). However,
moses -f moses.ini -inputtype 2 -weight-i 1 -max-phrase-length 5 <input.lat
will run without any problems.
I have found that the segfault is caused by Moses allocating too little
memory for translation options. It seems that Moses in many places assumes
the difference between position numbers (node numbers in a lattice) is the
same as the number of words spanned by the positions, even though in (say)
the above lattice the single word "ein" spans 5 positions (0--4).
Thus Moses::TranslationOptionCollection will initialize m_collection[0] with
a size of 2:
size_t maxSize = size - startPos;
size_t maxSizePhrase = StaticData::Instance().GetMaxPhraseLength();
maxSize = std::min(maxSize, maxSizePhrase); /// == 2
for (size_t endPos = 0 ; endPos < maxSize ; ++endPos)
{
m_collection[startPos].push_back(TranslationOptionList() );
}
but when adding translation hypotheses for the various word spans,
Moses::TranslationOptionCollectionConfusionNet::ProcessUnknownWord(size_t)
will later write to m_collection[0][4]:
/// calls ProcessOneUnknownWord() with column increment 5
ProcessOneUnknownWord(i->first ,sourcePos,
source.GetColumnIncrement(sourcePos, j++),&(i->second));
I am not sure how best to fix this, as I am not really familiar with the
Moses code. I guess a quick fix may be to ignore -max-phrase-length when
decoding lattices, but there should probably be a better way. Any advice?
Thank you.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
