Hi All,
i am undertaking English to regional language translation.
i have tuned the corpus using mert-moses.pl using following command:-
./mert-moses.pl corpus/corpus.lowercased.en corpus/corpus.lowercased.hi
/home/nakul/moses/mosesdecoder/trunk/moses-cmd/src/moses model/moses.ini
--working-dir corpus/tuning/mert --rootdir
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/
--decoder-flags "-v 0" >& mert.out &
the moses.ini, i got after training (at corpus/tuning/mert) has following
contents :-
# MERT optimized configuration
# decoder /home/nakul/moses/mosesdecoder/trunk/moses-cmd/src/moses
# BLEU 0.612451 -> 0.612451 on dev
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/corpus/corpus.lowercased.en
# We were before running iteration 8
# finished बुध अप्रेल 6 14:40:28 IST 2011
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
# translation tables: source-factors, target-factors, number of scores,
file
[ttable-file]
0 0 0 5
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/phrase-table.gz
# no generation models, no generation-file section
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
0 0 3 /home/nakul/moses/mosesdecoder/trunk/scripts/training/lm_hin.lm
# limit on how many phrase translations e for each phrase f are loaded
# 0 = all elements loaded
[ttable-limit]
20
# distortion (reordering) files
[distortion-file]
0-0 wbe-msd-bidirectional-fe-allff 6
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/reordering-table.wbe-msd-bidirectional-fe.gz
# distortion (reordering) weight
[weight-d]
0.009645
0.021906
0.008725
0.032902
0.019846
0.002462
0.016001
# language model weights
[weight-l]
0.022668
# translation model weights
[weight-t]
-0.008632
0.019782
0.228404
-0.005989
-0.499862
# no generation models, no weight-generation section
# word penalty
[weight-w]
-0.103176
[distortion-limit]
6
[v]
0
the contents of initial moses.ini is as follows:-
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
# translation tables: source-factors, target-factors, number of scores,
file
[ttable-file]
0 0 0 5
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/phrase-table.gz
# no generation models, no generation-file section
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
0 0 3 /home/nakul/moses/mosesdecoder/trunk/scripts/training/lm_hin.lm
# limit on how many phrase translations e for each phrase f are loaded
# 0 = all elements loaded
[ttable-limit]
20
# distortion (reordering) files
[distortion-file]
0-0 wbe-msd-bidirectional-fe-allff 6
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/reordering-table.wbe-msd-bidirectional-fe.gz
# distortion (reordering) weight
[weight-d]
0.3
0.3
0.3
0.3
0.3
0.3
0.3
# language model weights
[weight-l]
0.5000
# translation model weights
[weight-t]
0.2
0.2
0.2
0.2
0.2
# no generation models, no weight-generation section
# word penalty
[weight-w]
-1
[distortion-limit]
6
upon undertaking the translation i get mix of regional lang and english
words as follows:-
echo "Even amendments would require recommendation." | TMP=/tmp
/home/nakul/moses/mosesdecoder/trunk/moses-cmd/src/moses -f
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/corpus/tuning/mert/moses.ini
>output.txt
Defined parameters (per moses.ini or switch):
config:
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/corpus/tuning/mert/moses.ini
distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/reordering-table.wbe-msd-bidirectional-fe.gz
distortion-limit: 6
input-factors: 0
lmodel-file: 0 0 3
/home/nakul/moses/mosesdecoder/trunk/scripts/training/lm_hin.lm
mapping: 0 T 0
ttable-file: 0 0 0 5
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/phrase-table.gz
ttable-limit: 20
v: 0
weight-d: 0.009645 0.021906 0.008725 0.032902 0.019846 0.002462
0.016001
weight-l: 0.022668
weight-t: -0.008632 0.019782 0.228404 -0.005989 -0.499862
weight-w: -0.103176
Loading lexical distortion models...have 1 models
Creating lexical reordering...
weights: 0.022 0.009 0.033 0.020 0.002 0.016
Loading table into memory...done.
Start loading LanguageModel
/home/nakul/moses/mosesdecoder/trunk/scripts/training/lm_hin.lm : [1.000]
seconds
/home/nakul/moses/mosesdecoder/trunk/scripts/training/lm_hin.lm: line 122:
warning: non-zero probability for <unk> in closed-vocabulary LM
Finished loading LanguageModels : [1.000] seconds
Start loading PhraseTable
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/phrase-table.gz
: [1.000] seconds
filePath:
/home/nakul/moses/mosesdecoder/trunk/scripts/training/moses-scripts/scripts-20110405-1055/training/model/phrase-table.gz
Finished loading phrase tables : [1.000] seconds
IO from STDOUT/STDIN
Created input-output object : [1.000] seconds
Translating: Even amendments would require recommendation.
Collecting options took 0.000 seconds
Search took 0.000 seconds
BEST TRANSLATION: संशोधनों सहित Even|UNK|UNK|UNK प्रभाव require|UNK|UNK|UNK
recommendation.|UNK|UNK|UNK [11111] [total=-301.895] <<-4.000, -6.000,
-300.000, 0.000, 0.000, -1.022, -0.511, -1.609, 0.000, -48.665, -1.792,
-2.303, -1.386, -3.624, 2.000>>
Translation took 0.000 seconds
Finished translating
End. : [1.000] seconds
Please tell how to improve upon this translation, i have already tuned the
system but problem presists.
--
Thanks & Regards,
nakul
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
