Hi, If you have already filtered out the phrase table through the filter-model-given-input.pl script you can either i) split the corpus to translate into smaller pieces and use a filtered model for each piece; or ii) convert the phrase table into a binary format for on-demand loading: $ LC_ALL=C && zcat phrase-table.gz | sort | moses/misc/processPhraseTable -ttable 0 0 - -nscores 5 -out phrase-table then, edit moses.ini and change phrase-table.gz to phrase-table (without extension).
Good luck -- Felipe El sáb, 17-01-2009 a las 12:22 +0800, Hendra Setiawan escribió: > Hi moses-support members, > > I encounter a problem running moses decoder on a big phrase table and > am hoping somebody here can help. > > I am using a filtered phrase table which size is around 1GB > (uncompressed), a small size language model (srilm.gz from the sample > model provided) and no lexical reordering model. When I run the > decoder, the memory goes up to 12 GB before the kernel killed the > process. I am just wondering whether this memory requirement is > normal, meaning that perhaps I need a machine with higher memory or is > there any parameter (perhaps caching?) that I need to turn on or off? > > The process basically stops here: > > Start loading PhraseTable phrase-table.0-0.1 : [2.000] seconds > filePath: phrase-table.0-0.1 > > Regards, > Hendra > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support -- Felipe Sánchez Martínez <[email protected]> Departamento de Lenguajes y Sistemas Informáticos Universidad de Alicante, E-03071 Alicante (Spain) Tel.: +34 965 903 400, ext: 2038 Fax: +34 965 909 326 http://www.dlsi.ua.es/~fsanchez _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
