Hi im building an smt web translation system using your given guidelines on
moses home.
The translations are good , but even after doing the steps of memory
management for speeding up the translation given in the step by step guide
, I am still facing delay in translation output due to disk access times.
According to the guide it should not load the entire phrase table every
time, but I guess I have missed something.
the output of echo command is:
echo "mardi" | /opt/tools/moses/dist/bin/moses -f
/home/ssp/smt/etof/model/moses-bin.ini
Defined parameters (per moses.ini or switch):
config: /home/ssp/smt/etof/model/moses-bin.ini
distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6
/home/ssp/smt/etof/model/reordering-table.wbe-msd-bidirectional-fe
distortion-limit: 6
input-factors: 0
lmodel-file: 1 0 3 /home/ssp/smt/etof/lm/europarl-v6.blm.mm
mapping: 0 T 0
ttable-file: 1 0 0 5 /home/ssp/smt/etof/model/phrase-table
ttable-limit: 20
weight-d: 0.3 0.3 0.3 0.3 0.3 0.3 0.3
weight-l: 0.5000
weight-t: 0.20 0.20 0.20 0.20 0.20
weight-w: -1
Loading lexical distortion models...have 1 models
Creating lexical reordering...
weights: 0.300 0.300 0.300 0.300 0.300 0.300
Loading table into memory...done.
Start loading LanguageModel /home/ssp/smt/etof/lm/europarl-v6.blm.mm :
[49.000] seconds
In LanguageModelIRST::Load: nGramOrder = 3
Language Model Type of /home/ssp/smt/etof/lm/europarl-v6.blm.mm is 1
blmt
loadbin()
lmtable::loadbin_dict()
dict->size(): 41308
loadbin_level (level 1)
mapping 41308 1-grams
tableOffs 494937 tableGaps3417-grams
done (level1)
loadbin_level (level 2)
mapping 484826 2-grams
tableOffs 1114557 tableGaps445-grams
done (level2)
loadbin_level (level 3)
mapping 297921 3-grams
tableOffs 8386947 tableGaps2435-grams
done (level3)
done
OOV code is 1499
IRST: m_unknownId=1499
Finished loading LanguageModels : [49.000] seconds
Start loading PhraseTable /home/ssp/smt/etof/model/phrase-table : [49.000]
seconds
filePath: /home/ssp/smt/etof/model/phrase-table
Finished loading phrase tables : [49.000] seconds
IO from STDOUT/STDIN
Created input-output object : [49.000] seconds
Translating line 0 in thread id 3039353712
Translating: mardi
reading bin ttable
size of OFF_T 8
binary phrasefile loaded, default OFF_T: -1
Collecting options took 0.120 seconds
Search took 0.120 seconds
tuesday
BEST TRANSLATION: tuesday [1] [total=-13.136] <<0.000, -1.000, 0.000,
-0.547, 0.000, 0.000, 0.000, 0.000, 0.000, -27.202, -0.916, -0.496, -0.865,
-0.580, 1.000>>
reset caches
Translation took 0.120 seconds
Finished translating
reset mmap
len = 623037
sync = 0
running msync...
done. Running munmap...
done
len = 7272835
sync = 0
running msync...
done. Running munmap...
done
len = 2087882
sync = 0
running msync...
done. Running munmap...
done
len = 623037
sync = 0
running msync...
done. Running munmap...
done
len = 7272835
sync = 0
running msync...
done. Running munmap...
done
len = 2087882
sync = 0
running msync...
done. Running munmap...
done
It takes about 2-3 minutes to translate a single word.
Any suggestions on how I could reduce the total translation time? I would
really appreciate the help.
Thanks.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
