hi sanne what script/program are you using, and can you give the exact command you executed?
the filtering most moses users are familiar with takes in a phrase table AND the input file and output a new phrase table containing only the phrases needed to decode the input file. Its purely used to memory consumption and shouldn't present any problem or affect decoding results. >> Hi guys, >> >> Here is a follow up on my previous question: >> >> “ >> >> I would like to know how the phrase table is filtered. Are phrase >> pairs simply deleted when they are longer than a certain phrase >> length, or are the left over probabilities also re-estimated? To sum >> up to one again, for instance. >> >> “ >> >> I have an original phrase pair model in my /model directory which >> includes: >> >> alors ! vous avez été un ||| you have been a >> >> vous avez été un ||| you have been a >> >> and some others.. >> >> After filtering everything out that is longer than 5 words: >> >> I have nothing left with: >> >> ||| you have been a ||| >> >> In my /evaluation/filtered.devtest2006. I would expect at least >> >> vous avez été un ||| you have been a >> >> to be in the table…. >> >> Can someone explain this? >> >> Sanne >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
