the number of scores on each line in the lexicalised reordering table, eg it would be 6 in the case: ! ! ) ||| ! ! ) ||| 0.714286 0.142857 0.142857 0.428571 0.142857 0.428571
* Looking for MT/NLP opportunities * Hieu Hoang http://moses-smt.org/ On 19 May 2017 at 20:59, Steve Braich <[email protected]> wrote: > Hi Hieu, > Can you give some guidance on the parameter --num-lex-scores? > > Thanks, > Steve > > On Sat, May 13, 2017 at 8:03 AM, Hieu Hoang <[email protected]> wrote: > >> Moses2 doesn't support the Compact phrase-table, only ProbingPT. >> >> You have to rebinarize the phrase-table with CreateProbingPT. It's better >> if you also integrate the lexicalised reordering table the phrase-table. >> The script >> >> scripts/generic/binarize4moses2.perl >> >> does this all in 1 go >> >> >> On 13/05/2017 04:14, Steve Braich wrote: >> >> Hello, >> I am trying to improve the performance of Moses in a server environment >> (not training). >> >> I have tried various things, including the following which is described >> on http://www.statmt.org/moses/?n=Moses.Optimize. >> It has had no impact: >> >> - *Multi-threading: *Up to 16 Cores on Google's Cloud >> I used the option -threads all and -threads 16 >> >> - *Memory:* >> I have launched Moses with as much as 60 GB of memory >> >> - *Caching the Models* >> Note: I only have the following files that I understand to be what >> you need to translate, including Phrase Table, Reordering Table, and >> target >> language model. >> Am I missing anything? Your documentation suggests that I might at >> http://www.statmt.org/moses/?n=Moses.Optimize#ntoc2 >> <http://www.statmt.org/moses/?n=Moses.Optimize>. I see these other >> files in my EMS directory but I get no errors without them. >> >> cat /home/autom8tr8n/mt_publish/model_enzh/phrase-table.minphr > >> /dev/null >> >> cat /home/autom8tr8n/mt_publish/model_enzh/reordering-table.minlexr > >> /dev/null >> >> cat /home/autom8tr8n/mt_publish/model_enzh/target.blm.zh > /dev/null >> >> - >> *Memory: *I have launched Moses with as much as 60 GB of memory >> >> - *Other:* >> I made sure "transparent huge pages" are enabled, and I use the >> compact phrase and reordering tables. >> >> >> So, I am now trying Moses2 and I am following the instructions here ( >> http://www.statmt.org/moses/?n=Site.Moses2). I am having some issues >> with this. Here are my questions: >> >> - *Does the Moses executable have to match for Training and Server?* >> In other words, do I have to train the models with Moses2 if I want >> to run them with Moses2 Server? >> >> - *Moses2 Error Messages:* >> I tried just running it with Moses2 (without server and just basic >> options): >> ~/mosesdecoder/bin/moses2 -f ~/mt_publish/model_enzh/moses.ini >> >> I get this error message: >> Starting... >> Defined parameters (per moses.ini or switch): >> config: /home/autom8tr8n/mt_publish/model_enzh/moses.ini >> distortion-limit: 6 >> feature: UnknownWordPenalty WordPenalty PhrasePenalty >> PhraseDictionaryCompact name=TranslationModel0 num-features=4 >> path=/home/autom8tr8n/mt_publish/model_enzh/phrase-table.minphr >> input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 >> num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 >> output-factor=0 path=/home/autom8tr8n/mt_publish/model_enzh/reordering-table >> Distortion KENLM name=LM0 factor=0 >> path=/home/autom8tr8n/mt_publish/model_enzh/target.blm.zh >> order=3 >> input-factors: 0 >> mapping: 0 T 0 >> mark-unknown: >> server: >> server-port: 3001 >> unknown-word-prefix: __UNK__ >> unknown-word-suffix: __UNK__ >> weight: LexicalReordering0= 0.0892686 0.0768025 0.0930783 >> 0.0716109 0.00303172 0.0492278 Distortion0= 0.0399769 LM0= 0.107799 >> WordPenalty0= -0.242343 PhrasePenalty0= 0.0403541 TranslationModel0= >> 0.0396487 0.0448175 0.0681687 0.0338723 UnknownWordPenalty0= 1 >> >> >> *Feature name PhraseDictionaryCompact is not registered.Aborted (core >> dumped) ^This error suggests that I didn't compile with CMPH. I did. See >> below.* >> >> >> - *Compilation* >> From the file dates, Moses2 appears to have been compiled when I >> initially compiled Moses (1). Here is how I compiled it: >> ./bjam -a --with-boost=/home/autom8tr8n/mosesdecoder/boost_1_63_0 >> --with-cmph=/home/autom8tr8n/mosesdecoder/cmph/cmph2.0 >> --with-xmlrpc-c=/home/autom8tr8n/mosesdecoder/opt >> >> Where is the compilation log file where I can see what Moses 2 was >> compiled with? >> >> >> Please answer my questions about Moses2, look over the other optimization >> I did, and please, I am all ears if you have other suggestions to boost >> performance. My very unscientific manual testing has shown about 1.2 >> seconds to translate a phrase. No improvement at all has been made with >> any optimization that I have tried so far. A VM with a single core and >> 3.75 GB of memory performs just as good as a 16 core, 60 GB memory VM. >> >> Thanks in advance, >> >> Steve >> >> >> >> _______________________________________________ >> Moses-support mailing >> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> -- >> Hieu Hoanghttp://moses-smt.org/ >> >> >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
