Hi Shweta Here's some suggestions: - Make sure you binarise the reordering model too - Try with kenlm instead of irstlm - make sure your binarised files are on a local disk (not nfs!) - Remember that you can translate a batch of sentences in one moses run
cheers - Barry On Friday 02 March 2012 17:15:35 shweta porwal wrote: > Hi im building an smt web translation system using your given guidelines on > moses home. > > The translations are good , but even after doing the steps of memory > management for speeding up the translation given in the step by step guide > , I am still facing delay in translation output due to disk access times. > According to the guide it should not load the entire phrase table every > time, but I guess I have missed something. > > > the output of echo command is: > > echo "mardi" | /opt/tools/moses/dist/bin/moses -f > /home/ssp/smt/etof/model/moses-bin.ini > > Defined parameters (per moses.ini or switch): > config: /home/ssp/smt/etof/model/moses-bin.ini > distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6 > /home/ssp/smt/etof/model/reordering-table.wbe-msd-bidirectional-fe > distortion-limit: 6 > input-factors: 0 > lmodel-file: 1 0 3 /home/ssp/smt/etof/lm/europarl-v6.blm.mm > mapping: 0 T 0 > ttable-file: 1 0 0 5 /home/ssp/smt/etof/model/phrase-table > ttable-limit: 20 > weight-d: 0.3 0.3 0.3 0.3 0.3 0.3 0.3 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 > weight-w: -1 > Loading lexical distortion models...have 1 models > Creating lexical reordering... > weights: 0.300 0.300 0.300 0.300 0.300 0.300 > Loading table into memory...done. > Start loading LanguageModel /home/ssp/smt/etof/lm/europarl-v6.blm.mm : > [49.000] seconds > In LanguageModelIRST::Load: nGramOrder = 3 > Language Model Type of /home/ssp/smt/etof/lm/europarl-v6.blm.mm is 1 > blmt > loadbin() > lmtable::loadbin_dict() > dict->size(): 41308 > loadbin_level (level 1) > mapping 41308 1-grams > tableOffs 494937 tableGaps3417-grams > done (level1) > loadbin_level (level 2) > mapping 484826 2-grams > tableOffs 1114557 tableGaps445-grams > done (level2) > loadbin_level (level 3) > mapping 297921 3-grams > tableOffs 8386947 tableGaps2435-grams > done (level3) > done > OOV code is 1499 > IRST: m_unknownId=1499 > Finished loading LanguageModels : [49.000] seconds > Start loading PhraseTable /home/ssp/smt/etof/model/phrase-table : [49.000] > seconds > filePath: /home/ssp/smt/etof/model/phrase-table > Finished loading phrase tables : [49.000] seconds > IO from STDOUT/STDIN > Created input-output object : [49.000] seconds > Translating line 0 in thread id 3039353712 > Translating: mardi > > reading bin ttable > size of OFF_T 8 > binary phrasefile loaded, default OFF_T: -1 > Collecting options took 0.120 seconds > Search took 0.120 seconds > tuesday > BEST TRANSLATION: tuesday [1] [total=-13.136] <<0.000, -1.000, 0.000, > -0.547, 0.000, 0.000, 0.000, 0.000, 0.000, -27.202, -0.916, -0.496, -0.865, > -0.580, 1.000>> > reset caches > Translation took 0.120 seconds > Finished translating > reset mmap > len = 623037 > sync = 0 > running msync... > done. Running munmap... > done > len = 7272835 > sync = 0 > running msync... > done. Running munmap... > done > len = 2087882 > sync = 0 > running msync... > done. Running munmap... > done > len = 623037 > sync = 0 > running msync... > done. Running munmap... > done > len = 7272835 > sync = 0 > running msync... > done. Running munmap... > done > len = 2087882 > sync = 0 > running msync... > done. Running munmap... > done > > It takes about 2-3 minutes to translate a single word. > > Any suggestions on how I could reduce the total translation time? I would > really appreciate the help. > > Thanks. > -- Barry Haddow University of Edinburgh +44 (0) 131 651 3173 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
