hi Kairit
The phrase table
---/filtered/phrase-table.0-0.1.1.gz
seems to have loaded in 0 seconds.
This is the filtered phrase table. Check that the original phrase table
has data, and that the tuning set has data.
hieu
(ps. please subscribe to the mailing list to post emails)
On 09/05/2012 09:23, [email protected] wrote:
> Hi,
>
> I'm trying to tune a hierarchical model and the process is aborted
> with Exit code 134 and the message that the decoder died.
>
> The command I'm running is:
>
> nohup nice
> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
> ../../parallel/europarl-v6.et-en.dev.et
> ../../parallel/europarl-v6.et-en.dev.en
> /usr/local/share/moses/mosesdecoder/dist/bin/moses
> train/model/moses.ini --mertdir
> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
> --Hierarchical'&>
> mert.out
>
> I have also tried this one and it doesn't work either:
>
> nohup nice
> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
> ../../parallel/europarl-v6.et-en.dev.et
> ../../parallel/europarl-v6.et-en.dev.en
> /usr/local/share/moses/mosesdecoder/dist/bin/moses
> train/model/moses.ini --mertdir
> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
> -Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical'&>
> mert.out
>
> The content of moses.ini file is:
>
> #########################
> ### MOSES CONFIG FILE ###
> #########################
>
> # input factors
> [input-factors]
>
>
> # mapping steps
> [mapping]
> 0 T 0
> 1 T 1
>
> # translation tables: table type (hierarchical(0), textual (0), binary
> (1)), source-factors, target-factors, number of scores, file
> # OLD FORMAT is still handled for back-compatibility
> # OLD FORMAT translation tables: source-factors, target-factors,
> number of scores, file
> # OLD FORMAT a binary table type (1) is assumed
> [ttable-file]
> 6 0 0 5
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/rule-table.gz
> 6 0 0 1
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
>
> # no generation models, no generation-file section
>
> # language models: type(srilm/irstlm), factors, order, file
> [lmodel-file]
> 8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
>
>
> # limit on how many phrase translations e for each phrase f are loaded
> # 0 = all elements loaded
> [ttable-limit]
> 20
>
>
> # language model weights
> [weight-l]
> 0.5000
>
>
> # translation model weights
> [weight-t]
> 0.20
> 0.20
> 0.20
> 0.20
> 0.20
> 1.0
>
> # no generation models, no weight-generation section
>
> # word penalty
> [weight-w]
> -1
>
> [cube-pruning-pop-limit]
> 1000
>
> [non-terminals]
> X
>
> [search-algorithm]
> 3
>
> [inputtype]
> 3
>
> [max-chart-span]
> 20
> 1000
>
> The content of mert.out file is:
>
> nohup: ignoring input
> main::create_extractor_script() called too early to check prototype at
> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
> line 681.
> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts
> Use of uninitialized value $extractorargs in concatenation (.) or
> string at
> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
> line 370.
> filtering the phrase tables... Tue May 8 18:02:26 EEST 2012
> exec:
> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
> --Hierarchical ./filtered
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini
> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
> Executing:
> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
> --Hierarchical ./filtered
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini
> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et>
> filterphrases.out 2>
> filterphrases.err
> Asking moses for feature names and values from filtered/moses.ini
> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config
> filtered/moses.ini -inputtype 0 -show-weights> ./features.list
> Defined parameters (per moses.ini or switch):
> config: filtered/moses.ini
> cube-pruning-pop-limit: 1000
> input-factors: 0
> inputtype: 0
> lmodel-file: 8 0 3
> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
> mapping: 0 T 0 1 T 1
> max-chart-span: 20 1000
> non-terminals: X
> search-algorithm: 3
> show-weights:
> ttable-file: 6 0 0 5
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> 6 0 0 1
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> ttable-limit: 20
> weight-l: 0.5000
> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
> weight-w: -1
> Loading lexical distortion models...have 0 models
> Start loading LanguageModel
> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
> seconds
> Finished loading LanguageModels : [0.000] seconds
> Using uniform ttable-limit of 20 for all translation tables.
> Start loading PhraseTable
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> : [0.000]
> seconds
> filePath:
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> Start loading PhraseTable
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> : [0.000]
> seconds
> filePath:
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> Finished loading phrase tables : [0.000] seconds
> Start loading phrase table from
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> : [0.000]
> seconds
> Start loading new format pt model : [0.000] seconds
> Finished loading phrase tables : [72.000] seconds
> Start loading phrase table from
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> : [72.000]
> seconds
> Start loading new format pt model : [72.000] seconds
> Finished loading phrase tables : [72.000] seconds
> MERT starting values and ranges for random generation:
> lm = 0.500 ( 0.00 .. 1.00)
> w = -1.000 ( 0.00 .. 1.00)
> tm = 0.200 ( 0.00 .. 1.00)
> tm = 0.200 ( 0.00 .. 1.00)
> tm = 0.200 ( 0.00 .. 1.00)
> tm = 0.200 ( 0.00 .. 1.00)
> tm = 0.200 ( 0.00 .. 1.00)
> tm = 1.000 ( 0.00 .. 1.00)
> run 1 start at Tue May 8 18:05:59 EEST 2012
> Parsing --decoder-flags: ||
> Saving new config to: ./run1.moses.ini
> Saved: ./run1.moses.ini
> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000
> 0.200000 0.200000 1.000000
> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143
> 0.057143 0.057143 0.285714
> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses
> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm
> 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list
> run1.best100.out 100 -input-file
> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et>
> run1.out
> (1) run decoder to produce n-best lists
> params =
> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
> 0.057143 0.057143 0.057143 0.285714
> Defined parameters (per moses.ini or switch):
> config: filtered/moses.ini
> cube-pruning-pop-limit: 1000
> input-factors: 0
> input-file:
> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
> inputtype: 0
> lmodel-file: 8 0 3
> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
> mapping: 0 T 0 1 T 1
> max-chart-span: 20 1000
> n-best-list: run1.best100.out 100
> non-terminals: X
> search-algorithm: 3
> ttable-file: 6 0 0 5
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> 6 0 0 1
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> ttable-limit: 20
> weight-l: 0.142857
> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714
> weight-w: -0.285714
> Loading lexical distortion models...have 0 models
> Start loading LanguageModel
> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
> seconds
> Finished loading LanguageModels : [0.000] seconds
> Using uniform ttable-limit of 20 for all translation tables.
> Start loading PhraseTable
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> : [0.000]
> seconds
> filePath:
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> Start loading PhraseTable
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> : [0.000]
> seconds
> filePath:
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> Finished loading phrase tables : [0.000] seconds
> Start loading phrase table from
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.gz
> : [0.000]
> seconds
> Start loading new format pt model : [0.000] seconds
> Finished loading phrase tables : [73.000] seconds
> Start loading phrase table from
> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
> : [73.000]
> seconds
> Start loading new format pt model : [73.000] seconds
> Finished loading phrase tables : [73.000] seconds
> Created input-output object : [73.000] seconds
> Translating line 0 in thread id 0
> ERROR:ERROR: search. Aborting
>
> Aborted
> Exit code: 134
> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
> 0.057143 0.057143 0.057143 0.057143 0.285714
>
>
> Thanks if you can help me to find the problem.
>
> Kairit
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
