the decoder creates a list of all possible paths through the lattice up
the the max-phrase-length before it does the pt lookups
So if the lattice is large and the max-phrase-length is high, it will
consume a lot of memory.
in the old version of moses (v1), it only built the paths as it does the
lookup and only keep the path if it finds rules for it. This is more
memory efficient but the code was too convoluted
On 06/02/15 12:43, Jorg Tiedemann wrote:
Hi,
I have a question about the -max-phrase-length flag. I have lattice
input that collects a lot of memory and eventually crashes when memory
runs out (over 90GB). Now I realized that setting max-phrase-length
helps even when setting it to length 7 which is the maximum length in
my extracted phrase table. I do't really see why this is the case but
maybe I misunderstand the flag and what it does. Is it not related to
the max length of translation options coming from the phrase table?
Thanks for clarifying this to me.
cheers,
Jörg
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
