Hi Kairit This is the problem:
> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filte > red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such file or > directory at Either CreateOnDIskPt does not exist, or you've provided an incorrect path for it. Not sure why the mert script didn't notice that filtering fails... cheers - Barry On Tuesday 15 May 2012 13:25:52 [email protected] wrote: > Hi, > > The phrase tables have data and also the paths to the tuning sets are > correct. However, filtering process seems to produce errors: > > The file mert-work/filterphrases.err has the content: > > Executing: mkdir -p > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed Considering factor 0 > Done. > filtering > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/rule > -table.gz -> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filte > red/phrase-table.0-0.1.1... binarizing...CreateOnDiskPt 1 1 5 100 2 > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1 > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filte > red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such file or > directory at > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-inp > ut.pl line 278, <PIPE> line 3721058. > > Kairit > > Hieu Hoang <[email protected]> kirjutas: > > hi Kairit > > > > The phrase table > > ---/filtered/phrase-table.0-0.1.1.gz > > seems to have loaded in 0 seconds. > > > > This is the filtered phrase table. Check that the original phrase > > table has data, and that the tuning set has data. > > > > hieu > > > > (ps. please subscribe to the mailing list to post emails) > > > > On 09/05/2012 09:23, [email protected] wrote: > >> Hi, > >> > >> I'm trying to tune a hierarchical model and the process is aborted > >> with Exit code 134 and the message that the decoder died. > >> > >> The command I'm running is: > >> > >> nohup nice > >> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> ../../parallel/europarl-v6.et-en.dev.et > >> ../../parallel/europarl-v6.et-en.dev.en > >> /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> train/model/moses.ini --mertdir > >> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd > >> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given > >>-input.pl --Hierarchical'&> > >> mert.out > >> > >> I have also tried this one and it doesn't work either: > >> > >> nohup nice > >> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> ../../parallel/europarl-v6.et-en.dev.et > >> ../../parallel/europarl-v6.et-en.dev.en > >> /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> train/model/moses.ini --mertdir > >> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd > >> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given > >>-input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical'&> > >> mert.out > >> > >> The content of moses.ini file is: > >> > >> ######################### > >> ### MOSES CONFIG FILE ### > >> ######################### > >> > >> # input factors > >> [input-factors] > >> > >> > >> # mapping steps > >> [mapping] > >> 0 T 0 > >> 1 T 1 > >> > >> # translation tables: table type (hierarchical(0), textual (0), binary > >> (1)), source-factors, target-factors, number of scores, file > >> # OLD FORMAT is still handled for back-compatibility > >> # OLD FORMAT translation tables: source-factors, target-factors, > >> number of scores, file > >> # OLD FORMAT a binary table type (1) is assumed > >> [ttable-file] > >> 6 0 0 5 > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/r > >>ule-table.gz 6 0 0 1 > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar > >> > >> # no generation models, no generation-file section > >> > >> # language models: type(srilm/irstlm), factors, order, file > >> [lmodel-file] > >> 8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> > >> > >> # limit on how many phrase translations e for each phrase f are loaded > >> # 0 = all elements loaded > >> [ttable-limit] > >> 20 > >> > >> > >> # language model weights > >> [weight-l] > >> 0.5000 > >> > >> > >> # translation model weights > >> [weight-t] > >> 0.20 > >> 0.20 > >> 0.20 > >> 0.20 > >> 0.20 > >> 1.0 > >> > >> # no generation models, no weight-generation section > >> > >> # word penalty > >> [weight-w] > >> -1 > >> > >> [cube-pruning-pop-limit] > >> 1000 > >> > >> [non-terminals] > >> X > >> > >> [search-algorithm] > >> 3 > >> > >> [inputtype] > >> 3 > >> > >> [max-chart-span] > >> 20 > >> 1000 > >> > >> The content of mert.out file is: > >> > >> nohup: ignoring input > >> main::create_extractor_script() called too early to check prototype at > >> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> line 681. > >> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts > >> Use of uninitialized value $extractorargs in concatenation (.) or > >> string at > >> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> line 370. > >> filtering the phrase tables... Tue May 8 18:02:26 EEST 2012 > >> exec: > >> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given- > >>input.pl --Hierarchical ./filtered > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/m > >>oses.ini > >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> Executing: > >> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given- > >>input.pl --Hierarchical ./filtered > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/m > >>oses.ini > >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> > >> filterphrases.out 2> > >> filterphrases.err > >> Asking moses for feature names and values from filtered/moses.ini > >> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config > >> filtered/moses.ini -inputtype 0 -show-weights> ./features.list > >> Defined parameters (per moses.ini or switch): > >> config: filtered/moses.ini > >> cube-pruning-pop-limit: 1000 > >> input-factors: 0 > >> inputtype: 0 > >> lmodel-file: 8 0 3 > >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> mapping: 0 T 0 1 T 1 > >> max-chart-span: 20 1000 > >> non-terminals: X > >> search-algorithm: 3 > >> show-weights: > >> ttable-file: 6 0 0 5 > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz 6 0 0 1 > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar ttable-limit: 20 > >> weight-l: 0.5000 > >> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 > >> weight-w: -1 > >> Loading lexical distortion models...have 0 models > >> Start loading LanguageModel > >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > >> seconds > >> Finished loading LanguageModels : [0.000] seconds > >> Using uniform ttable-limit of 20 for all translation tables. > >> Start loading PhraseTable > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz : [0.000] > >> seconds > >> filePath: > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz Start loading PhraseTable > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar : [0.000] > >> seconds > >> filePath: > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar Finished loading phrase tables : [0.000] seconds > >> Start loading phrase table from > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz : [0.000] > >> seconds > >> Start loading new format pt model : [0.000] seconds > >> Finished loading phrase tables : [72.000] seconds > >> Start loading phrase table from > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar : [72.000] > >> seconds > >> Start loading new format pt model : [72.000] seconds > >> Finished loading phrase tables : [72.000] seconds > >> MERT starting values and ranges for random generation: > >> lm = 0.500 ( 0.00 .. 1.00) > >> w = -1.000 ( 0.00 .. 1.00) > >> tm = 0.200 ( 0.00 .. 1.00) > >> tm = 0.200 ( 0.00 .. 1.00) > >> tm = 0.200 ( 0.00 .. 1.00) > >> tm = 0.200 ( 0.00 .. 1.00) > >> tm = 0.200 ( 0.00 .. 1.00) > >> tm = 1.000 ( 0.00 .. 1.00) > >> run 1 start at Tue May 8 18:05:59 EEST 2012 > >> Parsing --decoder-flags: || > >> Saving new config to: ./run1.moses.ini > >> Saved: ./run1.moses.ini > >> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 > >> 0.200000 0.200000 1.000000 > >> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > >> 0.057143 0.057143 0.285714 > >> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm > >> 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list > >> run1.best100.out 100 -input-file > >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> > >> run1.out > >> (1) run decoder to produce n-best lists > >> params = > >> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 > >> 0.057143 0.057143 0.057143 0.285714 > >> Defined parameters (per moses.ini or switch): > >> config: filtered/moses.ini > >> cube-pruning-pop-limit: 1000 > >> input-factors: 0 > >> input-file: > >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> inputtype: 0 > >> lmodel-file: 8 0 3 > >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> mapping: 0 T 0 1 T 1 > >> max-chart-span: 20 1000 > >> n-best-list: run1.best100.out 100 > >> non-terminals: X > >> search-algorithm: 3 > >> ttable-file: 6 0 0 5 > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz 6 0 0 1 > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar ttable-limit: 20 > >> weight-l: 0.142857 > >> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > >> weight-w: -0.285714 > >> Loading lexical distortion models...have 0 models > >> Start loading LanguageModel > >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > >> seconds > >> Finished loading LanguageModels : [0.000] seconds > >> Using uniform ttable-limit of 20 for all translation tables. > >> Start loading PhraseTable > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz : [0.000] > >> seconds > >> filePath: > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz Start loading PhraseTable > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar : [0.000] > >> seconds > >> filePath: > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar Finished loading phrase tables : [0.000] seconds > >> Start loading phrase table from > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/fil > >>tered/phrase-table.0-0.1.1.gz : [0.000] > >> seconds > >> Start loading new format pt model : [0.000] seconds > >> Finished loading phrase tables : [73.000] seconds > >> Start loading phrase table from > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/g > >>lue-grammar : [73.000] > >> seconds > >> Start loading new format pt model : [73.000] seconds > >> Finished loading phrase tables : [73.000] seconds > >> Created input-output object : [73.000] seconds > >> Translating line 0 in thread id 0 > >> ERROR:ERROR: search. Aborting > >> > >> Aborted > >> Exit code: 134 > >> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > >> 0.057143 0.057143 0.057143 0.057143 0.285714 > >> > >> > >> Thanks if you can help me to find the problem. > >> > >> Kairit > >> > >> > >> _______________________________________________ > >> Moses-support mailing list > >> [email protected] > >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Barry Haddow University of Edinburgh +44 (0) 131 651 3173 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
