Hello all, I'm trying to speed up the translations from a Moses model.
I notice that for each sentence, (including empty ones), I have (-verbose 1): Initialize search took 2.404 seconds total Consequently, all sentences, even empty, take about 2.5 secs to translate. Those containing only XML markup such as <full translation="whatever">whatever</full> as well. I really don't know how to suppress this "search initialization" overhead, hopefully for all sentences. If that makes a difference, - in my application, I cannot filter the phrase table before calling Moses. - All files are binarized. The full output is below, as well as the moses.ini. Any help would be greatly appreciated ! Thanks Jean-Marc Coursimault Senior Software Engineer Xerox Research Centre Europe Currently at Xerox Research Centre India, Bangalore [email protected]<mailto:[email protected]> Mobile +91 96200 47600 Direct +91 80 3046 1750 time echo -e "bonjour\n\n<full translation=\"whatever including newline\">whatever including newline</full>\nmerci" | moses -verbose 1 -config moses.ini -minimum-bayes-risk -search-algorithm 1 -xml-input exclusive -cube-pruning-pop-limit 5000 -stack 5000 -threads 1 -monotone-at-punctuation Defined parameters (per moses.ini or switch): config: moses.final.ini cube-pruning-pop-limit: 5000 distortion-limit: 6 feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryBinary name=TranslationModel0 table-limit=20 num-features=11 path=phrase-table-fillup input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=reordering-table Distortion KENLM lazyken=0 name=LM0 factor=0 path=interpolated-binlm order=5 input-factors: 0 mapping: 0 T 0 minimum-bayes-risk: monotone-at-punctuation: search-algorithm: 1 stack: 5000 threads: 1 verbose: 1 weight: LexicalReordering0= -0.00421382408631085 0.0264411990233237 0.0681507631872309 0.0950507606296738 0.0235617857230281 0.0555396171784132 Distortion0= 0.05921781386616 LM0= 0.0948192696340088 WordPenalty0= -0.107113393837518 PhrasePenalty0= -0.00719809629986373 TranslationModel0= 0.0251023629475921 0.044428421259579 0.0756666901163637 0.0156437710203005 0.0344630592851508 -0.0137861719904281 -0.05813361788566 -0.0148306705190611 0.000877546106940157 -0.102630463792954 -0.0731307016104401 UnknownWordPenalty0= 1 xml-input: exclusive line=UnknownWordPenalty FeatureFunction: UnknownWordPenalty0 start: 0 end: 0 line=WordPenalty FeatureFunction: WordPenalty0 start: 1 end: 1 line=PhrasePenalty FeatureFunction: PhrasePenalty0 start: 2 end: 2 line=PhraseDictionaryBinary name=TranslationModel0 table-limit=20 num-features=11 path=phrase-table-fillup input-factor=0 output-factor=0 FeatureFunction: TranslationModel0 start: 3 end: 13 line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=reordering-table FeatureFunction: LexicalReordering0 start: 14 end: 19 Initializing LexicalReordering.. line=Distortion FeatureFunction: Distortion0 start: 20 end: 20 line=KENLM lazyken=0 name=LM0 factor=0 path=interpolated-binlm order=5 FeatureFunction: LM0 start: 21 end: 21 Loading UnknownWordPenalty0 Loading WordPenalty0 Loading PhrasePenalty0 Loading LexicalReordering0 binary file loaded, default OFF_T: -1 Loading Distortion0 Loading LM0 Loading TranslationModel0 IO from STDOUT/STDIN Created input-output object : [52.935] seconds Translating line 0 in thread id 139950765750016 reading bin ttable size of OFF_T 8 binary phrasefile loaded, default OFF_T: -1 binary file loaded, default OFF_T: -1 Line 0: Initialize search took 2.404 seconds total Line 0: Collecting options took 0.135 seconds Line 0: Search took 0.000 seconds good day Line 0: Decision rule took 0.000 seconds total Line 0: Additional reporting took 0.000 seconds total Line 0: Translation took 2.540 seconds total Translating line 1 in thread id 139950765750016 reading bin ttable size of OFF_T 8 binary phrasefile loaded, default OFF_T: -1 Line 1: Initialize search took 2.503 seconds total Line 1: Collecting options took 0.000 seconds Line 1: Search took 0.000 seconds Line 1: Decision rule took 0.000 seconds total Line 1: Additional reporting took 0.000 seconds total Line 1: Translation took 2.503 seconds total Translating line 2 in thread id 139950765750016 reading bin ttable size of OFF_T 8 binary phrasefile loaded, default OFF_T: -1 Line 2: Initialize search took 2.638 seconds total Line 2: Collecting options took 0.113 seconds Line 2: Search took 0.000 seconds whatever including newline Line 2: Decision rule took 0.000 seconds total Line 2: Additional reporting took 0.000 seconds total Line 2: Translation took 2.751 seconds total Translating line 3 in thread id 139950765750016 reading bin ttable size of OFF_T 8 binary phrasefile loaded, default OFF_T: -1 Line 3: Initialize search took 2.657 seconds total Line 3: Collecting options took 0.230 seconds Line 3: Search took 0.000 seconds thank you Line 3: Decision rule took 0.000 seconds total Line 3: Additional reporting took 0.000 seconds total Line 3: Translation took 2.888 seconds total Name:moses VmPeak:6375652 kB VmRSS:521296 kB RSSMax:6348296 kB user:13.594 sys:3.281 CPU:16.874 real:64.276 real 1m4.537s user 0m13.680s sys 0m3.301s Moses.ini : # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 [distortion-limit] 6 # feature functions [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryBinary name=TranslationModel0 table-limit=20 num-features=11 path=phrase-table-fillup input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=reordering-table Distortion KENLM lazyken=0 name=LM0 factor=0 path=interpolated-binlm order=5 # core weights [weight] LexicalReordering0= -0.00421382408631085 0.0264411990233237 0.0681507631872309 0.0950507606296738 0.0235617857230281 0.0555396171784132 Distortion0= 0.05921781386616 LM0= 0.0948192696340088 WordPenalty0= -0.107113393837518 PhrasePenalty0= -0.00719809629986373 TranslationModel0= 0.0251023629475921 0.044428421259579 0.0756666901163637 0.0156437710203005 0.0344630592851508 -0.0137861719904281 -0.05813361788566 -0.0148306705190611 0.000877546106940157 -0.102630463792954 -0.0731307016104401 UnknownWordPenalty0= 1
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
