Hi Hieu, Can you give some guidance on the parameter --num-lex-scores? Thanks, Steve
On Sat, May 13, 2017 at 8:03 AM, Hieu Hoang <[email protected]> wrote: > Moses2 doesn't support the Compact phrase-table, only ProbingPT. > > You have to rebinarize the phrase-table with CreateProbingPT. It's better > if you also integrate the lexicalised reordering table the phrase-table. > The script > > scripts/generic/binarize4moses2.perl > > does this all in 1 go > > > On 13/05/2017 04:14, Steve Braich wrote: > > Hello, > I am trying to improve the performance of Moses in a server environment > (not training). > > I have tried various things, including the following which is described on > http://www.statmt.org/moses/?n=Moses.Optimize. > It has had no impact: > > - *Multi-threading: *Up to 16 Cores on Google's Cloud > I used the option -threads all and -threads 16 > > - *Memory:* > I have launched Moses with as much as 60 GB of memory > > - *Caching the Models* > Note: I only have the following files that I understand to be what you > need to translate, including Phrase Table, Reordering Table, and target > language model. > Am I missing anything? Your documentation suggests that I might at > http://www.statmt.org/moses/?n=Moses.Optimize#ntoc2 > <http://www.statmt.org/moses/?n=Moses.Optimize>. I see these other > files in my EMS directory but I get no errors without them. > > cat /home/autom8tr8n/mt_publish/model_enzh/phrase-table.minphr > > /dev/null > > cat /home/autom8tr8n/mt_publish/model_enzh/reordering-table.minlexr > > /dev/null > > cat /home/autom8tr8n/mt_publish/model_enzh/target.blm.zh > /dev/null > > - > *Memory: *I have launched Moses with as much as 60 GB of memory > > - *Other:* > I made sure "transparent huge pages" are enabled, and I use the > compact phrase and reordering tables. > > > So, I am now trying Moses2 and I am following the instructions here ( > http://www.statmt.org/moses/?n=Site.Moses2). I am having some issues > with this. Here are my questions: > > - *Does the Moses executable have to match for Training and Server?* > In other words, do I have to train the models with Moses2 if I want to > run them with Moses2 Server? > > - *Moses2 Error Messages:* > I tried just running it with Moses2 (without server and just basic > options): > ~/mosesdecoder/bin/moses2 -f ~/mt_publish/model_enzh/moses.ini > > I get this error message: > Starting... > Defined parameters (per moses.ini or switch): > config: /home/autom8tr8n/mt_publish/model_enzh/moses.ini > distortion-limit: 6 > feature: UnknownWordPenalty WordPenalty PhrasePenalty > PhraseDictionaryCompact name=TranslationModel0 num-features=4 > path=/home/autom8tr8n/mt_publish/model_enzh/phrase-table.minphr > input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 > num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 > output-factor=0 path=/home/autom8tr8n/mt_publish/model_enzh/reordering-table > Distortion KENLM name=LM0 factor=0 path=/home/autom8tr8n/mt_ > publish/model_enzh/target.blm.zh order=3 > input-factors: 0 > mapping: 0 T 0 > mark-unknown: > server: > server-port: 3001 > unknown-word-prefix: __UNK__ > unknown-word-suffix: __UNK__ > weight: LexicalReordering0= 0.0892686 0.0768025 0.0930783 > 0.0716109 0.00303172 0.0492278 Distortion0= 0.0399769 LM0= 0.107799 > WordPenalty0= -0.242343 PhrasePenalty0= 0.0403541 TranslationModel0= > 0.0396487 0.0448175 0.0681687 0.0338723 UnknownWordPenalty0= 1 > > > *Feature name PhraseDictionaryCompact is not registered.Aborted (core > dumped) ^This error suggests that I didn't compile with CMPH. I did. See > below.* > > > - *Compilation* > From the file dates, Moses2 appears to have been compiled when I > initially compiled Moses (1). Here is how I compiled it: > ./bjam -a --with-boost=/home/autom8tr8n/mosesdecoder/boost_1_63_0 > --with-cmph=/home/autom8tr8n/mosesdecoder/cmph/cmph2.0 > --with-xmlrpc-c=/home/autom8tr8n/mosesdecoder/opt > > Where is the compilation log file where I can see what Moses 2 was > compiled with? > > > Please answer my questions about Moses2, look over the other optimization > I did, and please, I am all ears if you have other suggestions to boost > performance. My very unscientific manual testing has shown about 1.2 > seconds to translate a phrase. No improvement at all has been made with > any optimization that I have tried so far. A VM with a single core and > 3.75 GB of memory performs just as good as a 16 core, 60 GB memory VM. > > Thanks in advance, > > Steve > > > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > > -- > Hieu Hoanghttp://moses-smt.org/ > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
