Hi,
I'm trying to tune a hierarchical model and the process is aborted
with Exit code 134 and the message that the decoder died.
The command I'm running is:
nohup nice
/usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
../../parallel/europarl-v6.et-en.dev.et
../../parallel/europarl-v6.et-en.dev.en
/usr/local/share/moses/mosesdecoder/dist/bin/moses
train/model/moses.ini --mertdir
/usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
'/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
--Hierarchical' &>
mert.out
I have also tried this one and it doesn't work either:
nohup nice
/usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
../../parallel/europarl-v6.et-en.dev.et
../../parallel/europarl-v6.et-en.dev.en
/usr/local/share/moses/mosesdecoder/dist/bin/moses
train/model/moses.ini --mertdir
/usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
'/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
-Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical' &>
mert.out
The content of moses.ini file is:
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
# mapping steps
[mapping]
0 T 0
1 T 1
# translation tables: table type (hierarchical(0), textual (0), binary
(1)), source-factors, target-factors, number of scores, file
# OLD FORMAT is still handled for back-compatibility
# OLD FORMAT translation tables: source-factors, target-factors,
number of scores, file
# OLD FORMAT a binary table type (1) is assumed
[ttable-file]
6 0 0 5
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/rule-table.gz
6 0 0 1
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
# no generation models, no generation-file section
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
# limit on how many phrase translations e for each phrase f are loaded
# 0 = all elements loaded
[ttable-limit]
20
# language model weights
[weight-l]
0.5000
# translation model weights
[weight-t]
0.20
0.20
0.20
0.20
0.20
1.0
# no generation models, no weight-generation section
# word penalty
[weight-w]
-1
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
The content of mert.out file is:
nohup: ignoring input
main::create_extractor_script() called too early to check prototype at
/usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
line 681.
Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts
Use of uninitialized value $extractorargs in concatenation (.) or
string at
/usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
line 370.
filtering the phrase tables... Tue May 8 18:02:26 EEST 2012
exec:
/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
--Hierarchical ./filtered
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
Executing:
/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
--Hierarchical ./filtered
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >
filterphrases.out 2>
filterphrases.err
Asking moses for feature names and values from filtered/moses.ini
Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config
filtered/moses.ini -inputtype 0 -show-weights > ./features.list
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
cube-pruning-pop-limit: 1000
input-factors: 0
inputtype: 0
lmodel-file: 8 0 3
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
mapping: 0 T 0 1 T 1
max-chart-span: 20 1000
non-terminals: X
search-algorithm: 3
show-weights:
ttable-file: 6 0 0 5
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
6 0 0 1
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
ttable-limit: 20
weight-l: 0.5000
weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
weight-w: -1
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
seconds
Finished loading LanguageModels : [0.000] seconds
Using uniform ttable-limit of 20 for all translation tables.
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
: [0.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [0.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
Finished loading phrase tables : [0.000] seconds
Start loading phrase table from
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
: [0.000]
seconds
Start loading new format pt model : [0.000] seconds
Finished loading phrase tables : [72.000] seconds
Start loading phrase table from
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [72.000]
seconds
Start loading new format pt model : [72.000] seconds
Finished loading phrase tables : [72.000] seconds
MERT starting values and ranges for random generation:
lm = 0.500 ( 0.00 .. 1.00)
w = -1.000 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 1.000 ( 0.00 .. 1.00)
run 1 start at Tue May 8 18:05:59 EEST 2012
Parsing --decoder-flags: ||
Saving new config to: ./run1.moses.ini
Saved: ./run1.moses.ini
Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000
0.200000 0.200000 1.000000
DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143
0.057143 0.057143 0.285714
Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses
-config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm
0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list
run1.best100.out 100 -input-file
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >
run1.out
(1) run decoder to produce n-best lists
params =
decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
0.057143 0.057143 0.057143 0.285714
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
cube-pruning-pop-limit: 1000
input-factors: 0
input-file:
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
inputtype: 0
lmodel-file: 8 0 3
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
mapping: 0 T 0 1 T 1
max-chart-span: 20 1000
n-best-list: run1.best100.out 100
non-terminals: X
search-algorithm: 3
ttable-file: 6 0 0 5
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
6 0 0 1
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
ttable-limit: 20
weight-l: 0.142857
weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714
weight-w: -0.285714
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
seconds
Finished loading LanguageModels : [0.000] seconds
Using uniform ttable-limit of 20 for all translation tables.
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
: [0.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [0.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
Finished loading phrase tables : [0.000] seconds
Start loading phrase table from
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
: [0.000]
seconds
Start loading new format pt model : [0.000] seconds
Finished loading phrase tables : [73.000] seconds
Start loading phrase table from
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [73.000]
seconds
Start loading new format pt model : [73.000] seconds
Finished loading phrase tables : [73.000] seconds
Created input-output object : [73.000] seconds
Translating line 0 in thread id 0
ERROR:ERROR: search. Aborting
Aborted
Exit code: 134
The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
0.057143 0.057143 0.057143 0.057143 0.285714
Thanks if you can help me to find the problem.
Kairit
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
