I set the input type with flag and it does work now. Thank you very much. Kairit
Hieu Hoang <[email protected]> kirjutas: > hi kairit > > apologies. A small bug in the chart decoder only lets you use > inputtype=3. And by default the mert script always set inputtype=0, > unless you override it. > > However, I've fixed the bug in the decoder so it should work now. > > > On 18/05/2012 08:59, [email protected] wrote: >> One problem fixed: mistakenly I had used moses instead of moses_chart. >> >> With inputtype I don't see how I could change it: moses.ini in >> train/model folder already has inputtype 3, but tuning script calls it >> with value 0. >> >> The problem still persists, but now seems to be a different one >> according to the content of mert.out >> >> nohup: ignoring input >> main::create_extractor_script() called too early to check prototype at >> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >> line 681. >> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts >> Use of uninitialized value $extractorargs in concatenation (.) or >> string at >> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >> line 370. >> filtering the phrase tables... Fri May 18 10:31:35 EEST 2012 >> exec: >> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl >> -Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 >> 100 2" --Hierarchical ./filtered >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >> Executing: >> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl >> -Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 >> 100 2" --Hierarchical ./filtered >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >> filterphrases.out >> 2> >> filterphrases.err >> Asking moses for feature names and values from filtered/moses.ini >> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart >> -config filtered/moses.ini -inputtype 0 -show-weights> ./features.list >> Defined parameters (per moses.ini or switch): >> config: filtered/moses.ini >> cube-pruning-pop-limit: 1000 >> input-factors: 0 >> inputtype: 0 >> lmodel-file: 8 0 3 >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >> mapping: 0 T 0 1 T 1 >> max-chart-span: 20 1000 >> non-terminals: X >> search-algorithm: 3 >> show-weights: >> ttable-file: 2 0 0 5 >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin >> 6 0 0 >> 1 >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> ttable-limit: 20 >> weight-l: 0.5000 >> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 >> weight-w: -1 >> Loading lexical distortion models...have 0 models >> Start loading LanguageModel >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] >> seconds >> Finished loading LanguageModels : [0.000] seconds >> Using uniform ttable-limit of 20 for all translation tables. >> Start loading PhraseTable >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin >> : >> [0.000] >> seconds >> filePath: >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin >> Start loading PhraseTable >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> : >> [0.000] >> seconds >> filePath: >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> Finished loading phrase tables : [0.000] seconds >> Start loading phrase table from >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> : >> [0.000] >> seconds >> Start loading new format pt model : [0.000] seconds >> Finished loading phrase tables : [0.000] seconds >> MERT starting values and ranges for random generation: >> lm = 0.500 ( 0.00 .. 1.00) >> tm = 0.200 ( 0.00 .. 1.00) >> tm = 0.200 ( 0.00 .. 1.00) >> tm = 0.200 ( 0.00 .. 1.00) >> tm = 0.200 ( 0.00 .. 1.00) >> tm = 0.200 ( 0.00 .. 1.00) >> tm = 1.000 ( 0.00 .. 1.00) >> w = -1.000 ( 0.00 .. 1.00) >> run 1 start at Fri May 18 10:38:15 EEST 2012 >> Parsing --decoder-flags: || >> Saving new config to: ./run1.moses.ini >> Saved: ./run1.moses.ini >> Normalizing lambdas: 0.500000 0.200000 0.200000 0.200000 0.200000 >> 0.200000 1.000000 -1.000000 >> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 >> 0.057143 0.057143 0.285714 >> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart >> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm >> 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list >> run1.best100.out 100 -input-file >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >> run1.out >> (1) run decoder to produce n-best lists >> params = >> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >> 0.057143 0.057143 0.057143 0.285714 >> Defined parameters (per moses.ini or switch): >> config: filtered/moses.ini >> cube-pruning-pop-limit: 1000 >> input-factors: 0 >> input-file: >> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >> inputtype: 0 >> lmodel-file: 8 0 3 >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >> mapping: 0 T 0 1 T 1 >> max-chart-span: 20 1000 >> n-best-list: run1.best100.out 100 >> non-terminals: X >> search-algorithm: 3 >> ttable-file: 2 0 0 5 >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin >> 6 0 0 >> 1 >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> ttable-limit: 20 >> weight-l: 0.142857 >> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >> weight-w: -0.285714 >> Loading lexical distortion models...have 0 models >> Start loading LanguageModel >> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] >> seconds >> Finished loading LanguageModels : [1.000] seconds >> Using uniform ttable-limit of 20 for all translation tables. >> Start loading PhraseTable >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin >> : >> [1.000] >> seconds >> filePath: >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin >> Start loading PhraseTable >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> : >> [1.000] >> seconds >> filePath: >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> Finished loading phrase tables : [1.000] seconds >> Start loading phrase table from >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar >> : >> [1.000] >> seconds >> Start loading new format pt model : [1.000] seconds >> Finished loading phrase tables : [1.000] seconds >> Created input-output object : [1.000] seconds >> Translating:<s> eriti tÃEURhtis on teha koostööd sÃOEsinikuvaese >> majanduse ja energiatõhusa ÃOEhiskonna loomisega seotud uuenduste >> valdkonnas .</s> >> >> Translation took 0.090 seconds >> Exception: std::bad_cast >> Exit code: 1 >> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 >> 0.057143 0.057143 0.057143 0.057143 0.285714 >> >> >> Thank you for your patience, this is the first time for me to train a >> MT system. >> Kairit >> >> Barry Haddow<[email protected]> kirjutas: >> >>> Hi Kairit >>> >>> Actually the problem is that you're using moses, not moses_chart. >>> You need to >>> use moses_chart for a hierarchical model, >>> >>> cheers - Barry >>> >>> On Thursday 17 May 2012 11:57:34 Barry Haddow wrote: >>>> Hi Kairit >>>> >>>> Try setting your inputtype to 3, >>>> >>>> cheers - Barry >>>> >>>> On Thursday 17 May 2012 11:46:26 you wrote: >>>>> Hi, >>>>> >>>>> Now the filtering works correctly but the decoder still dies. >>>>> >>>>> The command I'm running now, is: >>>>> nohup nice >>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.en >>>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>> train/model/moses.ini --mertdir >>>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd >>>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given- >>>>> in put.pl -Binarizer >>>>> "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2" >>>>> --Hierarchical'&> mert.out >>>>> >>>>> I'm attaching again the mert.out file although it seems to be similar >>>>> to the previous one: >>>>> >>>>> nohup: ignoring input >>>>> main::create_extractor_script() called too early to check prototype at >>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>> line 681. >>>>> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts >>>>> Use of uninitialized value $extractorargs in concatenation (.) or >>>>> string at >>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>> line 370. >>>>> filtering the phrase tables... Wed May 16 20:40:51 EEST 2012 >>>>> exec: >>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-i >>>>> np ut.pl -Binarizer >>>>> "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2" >>>>> --Hierarchical ./filtered >>>>> >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mo >>>>> s es.ini >>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>> Executing: >>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-i >>>>> np ut.pl -Binarizer >>>>> "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2" >>>>> --Hierarchical ./filtered >>>>> >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mo >>>>> s es.ini >>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>>> filterphrases.out 2> filterphrases.err >>>>> Asking moses for feature names and values from filtered/moses.ini >>>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config >>>>> filtered/moses.ini -inputtype 0 -show-weights> ./features.list >>>>> Defined parameters (per moses.ini or switch): >>>>> config: filtered/moses.ini >>>>> cube-pruning-pop-limit: 1000 >>>>> input-factors: 0 >>>>> inputtype: 0 >>>>> lmodel-file: 8 0 3 >>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>> mapping: 0 T 0 1 T 1 >>>>> max-chart-span: 20 1000 >>>>> non-terminals: X >>>>> search-algorithm: 3 >>>>> show-weights: >>>>> ttable-file: 2 0 0 5 >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>>> er ed/phrase-table.0-0.1.1.bin 6 0 0 1 >>>>> >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> u e-grammar ttable-limit: 20 >>>>> weight-l: 0.5000 >>>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 >>>>> weight-w: -1 >>>>> Loading lexical distortion models...have 0 models >>>>> Start loading LanguageModel >>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] >>>>> seconds >>>>> Finished loading LanguageModels : [0.000] seconds >>>>> Using uniform ttable-limit of 20 for all translation tables. >>>>> Start loading PhraseTable >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>>> er ed/phrase-table.0-0.1.1.bin : [0.000] seconds >>>>> filePath: >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>>> er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> ue -grammar : [0.000] seconds >>>>> filePath: >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> ue -grammar Finished loading phrase tables : [0.000] seconds >>>>> Start loading phrase table from >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> ue -grammar : [0.000] seconds >>>>> Start loading new format pt model : [0.000] seconds >>>>> Finished loading phrase tables : [0.000] seconds >>>>> MERT starting values and ranges for random generation: >>>>> lm = 0.500 ( 0.00 .. 1.00) >>>>> w = -1.000 ( 0.00 .. 1.00) >>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>> tm = 1.000 ( 0.00 .. 1.00) >>>>> run 1 start at Wed May 16 20:47:37 EEST 2012 >>>>> Parsing --decoder-flags: || >>>>> Saving new config to: ./run1.moses.ini >>>>> Saved: ./run1.moses.ini >>>>> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 >>>>> 0.200000 0.200000 1.000000 >>>>> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 >>>>> 0.057143 0.057143 0.285714 >>>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm >>>>> 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list >>>>> run1.best100.out 100 -input-file >>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >>>>> run1.out >>>>> (1) run decoder to produce n-best lists >>>>> params = >>>>> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >>>>> 0.057143 0.057143 0.057143 0.285714 >>>>> Defined parameters (per moses.ini or switch): >>>>> config: filtered/moses.ini >>>>> cube-pruning-pop-limit: 1000 >>>>> input-factors: 0 >>>>> input-file: >>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>> inputtype: 0 >>>>> lmodel-file: 8 0 3 >>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>> mapping: 0 T 0 1 T 1 >>>>> max-chart-span: 20 1000 >>>>> n-best-list: run1.best100.out 100 >>>>> non-terminals: X >>>>> search-algorithm: 3 >>>>> ttable-file: 2 0 0 5 >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>>> er ed/phrase-table.0-0.1.1.bin 6 0 0 1 >>>>> >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> u e-grammar ttable-limit: 20 >>>>> weight-l: 0.142857 >>>>> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>> weight-w: -0.285714 >>>>> Loading lexical distortion models...have 0 models >>>>> Start loading LanguageModel >>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] >>>>> seconds >>>>> Finished loading LanguageModels : [0.000] seconds >>>>> Using uniform ttable-limit of 20 for all translation tables. >>>>> Start loading PhraseTable >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>>> er ed/phrase-table.0-0.1.1.bin : [0.000] seconds >>>>> filePath: >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>>> er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> ue -grammar : [0.000] seconds >>>>> filePath: >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> ue -grammar Finished loading phrase tables : [0.000] seconds >>>>> Start loading phrase table from >>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>>> ue -grammar : [0.000] seconds >>>>> Start loading new format pt model : [0.000] seconds >>>>> Finished loading phrase tables : [0.000] seconds >>>>> Created input-output object : [0.000] seconds >>>>> Translating line 0 in thread id 0 >>>>> ERROR:ERROR: search. Aborting >>>>> >>>>> Aborted >>>>> Exit code: 134 >>>>> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 >>>>> 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>> >>>>> Any further ideas, what could be wrong? >>>>> Kairit >>>>> >>>>> Barry Haddow<[email protected]> kirjutas: >>>>>> Hi Kairit >>>>>> >>>>>> This is the problem: >>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>>> il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such >>>>>>> file or directory at >>>>>> Either CreateOnDIskPt does not exist, or you've provided an >>>>>> incorrect path for >>>>>> it. >>>>>> >>>>>> Not sure why the mert script didn't notice that filtering fails... >>>>>> >>>>>> cheers - Barry >>>>>> >>>>>> On Tuesday 15 May 2012 13:25:52 [email protected] wrote: >>>>>>> Hi, >>>>>>> >>>>>>> The phrase tables have data and also the paths to the tuning sets are >>>>>>> correct. However, filtering process seems to produce errors: >>>>>>> >>>>>>> The file mert-work/filterphrases.err has the content: >>>>>>> >>>>>>> Executing: mkdir -p >>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>>> il ter ed Considering factor 0 >>>>>>> Done. >>>>>>> filtering >>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model >>>>>>> /r ule -table.gz -> >>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>>> il te red/phrase-table.0-0.1.1... binarizing...CreateOnDiskPt 1 1 5 100 >>>>>>> 2 >>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>>> il ter ed/phrase-table.0-0.1.1 >>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>>> il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such >>>>>>> file or directory at >>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-give >>>>>>> n- inp ut.pl line 278,<PIPE> line 3721058. >>>>>>> >>>>>>> Kairit >>>>>>> >>>>>>> Hieu Hoang<[email protected]> kirjutas: >>>>>>>> hi Kairit >>>>>>>> >>>>>>>> The phrase table >>>>>>>> ---/filtered/phrase-table.0-0.1.1.gz >>>>>>>> seems to have loaded in 0 seconds. >>>>>>>> >>>>>>>> This is the filtered phrase table. Check that the original phrase >>>>>>>> table has data, and that the tuning set has data. >>>>>>>> >>>>>>>> hieu >>>>>>>> >>>>>>>> (ps. please subscribe to the mailing list to post emails) >>>>>>>> >>>>>>>> On 09/05/2012 09:23, [email protected] wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm trying to tune a hierarchical model and the process is aborted >>>>>>>>> with Exit code 134 and the message that the decoder died. >>>>>>>>> >>>>>>>>> The command I'm running is: >>>>>>>>> >>>>>>>>> nohup nice >>>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>>> ../../parallel/europarl-v6.et-en.dev.et >>>>>>>>> ../../parallel/europarl-v6.et-en.dev.en >>>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>>> train/model/moses.ini --mertdir >>>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd >>>>>>>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g >>>>>>>>> iv en -input.pl --Hierarchical'&> >>>>>>>>> mert.out >>>>>>>>> >>>>>>>>> I have also tried this one and it doesn't work either: >>>>>>>>> >>>>>>>>> nohup nice >>>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>>> ../../parallel/europarl-v6.et-en.dev.et >>>>>>>>> ../../parallel/europarl-v6.et-en.dev.en >>>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>>> train/model/moses.ini --mertdir >>>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd >>>>>>>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g >>>>>>>>> iv en -input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" >>>>>>>>> --Hierarchical'&> mert.out >>>>>>>>> >>>>>>>>> The content of moses.ini file is: >>>>>>>>> >>>>>>>>> ######################### >>>>>>>>> ### MOSES CONFIG FILE ### >>>>>>>>> ######################### >>>>>>>>> >>>>>>>>> # input factors >>>>>>>>> [input-factors] >>>>>>>>> >>>>>>>>> >>>>>>>>> # mapping steps >>>>>>>>> [mapping] >>>>>>>>> 0 T 0 >>>>>>>>> 1 T 1 >>>>>>>>> >>>>>>>>> # translation tables: table type (hierarchical(0), textual (0), >>>>>>>>> binary (1)), source-factors, target-factors, number of scores, file >>>>>>>>> # OLD FORMAT is still handled for back-compatibility >>>>>>>>> # OLD FORMAT translation tables: source-factors, target-factors, >>>>>>>>> number of scores, file >>>>>>>>> # OLD FORMAT a binary table type (1) is assumed >>>>>>>>> [ttable-file] >>>>>>>>> 6 0 0 5 >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /r ule-table.gz 6 0 0 1 >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar >>>>>>>>> >>>>>>>>> # no generation models, no generation-file section >>>>>>>>> >>>>>>>>> # language models: type(srilm/irstlm), factors, order, file >>>>>>>>> [lmodel-file] >>>>>>>>> 8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>>>>>> >>>>>>>>> >>>>>>>>> # limit on how many phrase translations e for each phrase f are >>>>>>>>> loaded # 0 = all elements loaded >>>>>>>>> [ttable-limit] >>>>>>>>> 20 >>>>>>>>> >>>>>>>>> >>>>>>>>> # language model weights >>>>>>>>> [weight-l] >>>>>>>>> 0.5000 >>>>>>>>> >>>>>>>>> >>>>>>>>> # translation model weights >>>>>>>>> [weight-t] >>>>>>>>> 0.20 >>>>>>>>> 0.20 >>>>>>>>> 0.20 >>>>>>>>> 0.20 >>>>>>>>> 0.20 >>>>>>>>> 1.0 >>>>>>>>> >>>>>>>>> # no generation models, no weight-generation section >>>>>>>>> >>>>>>>>> # word penalty >>>>>>>>> [weight-w] >>>>>>>>> -1 >>>>>>>>> >>>>>>>>> [cube-pruning-pop-limit] >>>>>>>>> 1000 >>>>>>>>> >>>>>>>>> [non-terminals] >>>>>>>>> X >>>>>>>>> >>>>>>>>> [search-algorithm] >>>>>>>>> 3 >>>>>>>>> >>>>>>>>> [inputtype] >>>>>>>>> 3 >>>>>>>>> >>>>>>>>> [max-chart-span] >>>>>>>>> 20 >>>>>>>>> 1000 >>>>>>>>> >>>>>>>>> The content of mert.out file is: >>>>>>>>> >>>>>>>>> nohup: ignoring input >>>>>>>>> main::create_extractor_script() called too early to check prototype >>>>>>>>> at >>>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>>> line 681. >>>>>>>>> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts >>>>>>>>> Use of uninitialized value $extractorargs in concatenation (.) or >>>>>>>>> string at >>>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>>> line 370. >>>>>>>>> filtering the phrase tables... Tue May 8 18:02:26 EEST 2012 >>>>>>>>> exec: >>>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-gi >>>>>>>>> ve n- input.pl --Hierarchical ./filtered >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /m oses.ini >>>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>>>>>> Executing: >>>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-gi >>>>>>>>> ve n- input.pl --Hierarchical ./filtered >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /m oses.ini >>>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >>>>>>>>> filterphrases.out 2> >>>>>>>>> filterphrases.err >>>>>>>>> Asking moses for feature names and values from filtered/moses.ini >>>>>>>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>>> -config filtered/moses.ini -inputtype 0 -show-weights> >>>>>>>>> ./features.list Defined parameters (per moses.ini or switch): >>>>>>>>> config: filtered/moses.ini >>>>>>>>> cube-pruning-pop-limit: 1000 >>>>>>>>> input-factors: 0 >>>>>>>>> inputtype: 0 >>>>>>>>> lmodel-file: 8 0 3 >>>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>>>>>> mapping: 0 T 0 1 T 1 >>>>>>>>> max-chart-span: 20 1000 >>>>>>>>> non-terminals: X >>>>>>>>> search-algorithm: 3 >>>>>>>>> show-weights: >>>>>>>>> ttable-file: 6 0 0 5 >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz 6 0 0 1 >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar ttable-limit: 20 >>>>>>>>> weight-l: 0.5000 >>>>>>>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 >>>>>>>>> weight-w: -1 >>>>>>>>> Loading lexical distortion models...have 0 models >>>>>>>>> Start loading LanguageModel >>>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : >>>>>>>>> [0.000] seconds >>>>>>>>> Finished loading LanguageModels : [0.000] seconds >>>>>>>>> Using uniform ttable-limit of 20 for all translation tables. >>>>>>>>> Start loading PhraseTable >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>>> seconds >>>>>>>>> filePath: >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar : [0.000] >>>>>>>>> seconds >>>>>>>>> filePath: >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar Finished loading phrase tables : [0.000] seconds >>>>>>>>> Start loading phrase table from >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>>> seconds >>>>>>>>> Start loading new format pt model : [0.000] seconds >>>>>>>>> Finished loading phrase tables : [72.000] seconds >>>>>>>>> Start loading phrase table from >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar : [72.000] >>>>>>>>> seconds >>>>>>>>> Start loading new format pt model : [72.000] seconds >>>>>>>>> Finished loading phrase tables : [72.000] seconds >>>>>>>>> MERT starting values and ranges for random generation: >>>>>>>>> lm = 0.500 ( 0.00 .. 1.00) >>>>>>>>> w = -1.000 ( 0.00 .. 1.00) >>>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>>> tm = 1.000 ( 0.00 .. 1.00) >>>>>>>>> run 1 start at Tue May 8 18:05:59 EEST 2012 >>>>>>>>> Parsing --decoder-flags: || >>>>>>>>> Saving new config to: ./run1.moses.ini >>>>>>>>> Saved: ./run1.moses.ini >>>>>>>>> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 >>>>>>>>> 0.200000 0.200000 1.000000 >>>>>>>>> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >>>>>>>>> 0.057143 0.057143 0.057143 0.285714 >>>>>>>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>>> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 >>>>>>>>> -tm 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>>>>>> -n-best-list run1.best100.out 100 -input-file >>>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >>>>>>>>> run1.out >>>>>>>>> (1) run decoder to produce n-best lists >>>>>>>>> params = >>>>>>>>> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >>>>>>>>> 0.057143 0.057143 0.057143 0.285714 >>>>>>>>> Defined parameters (per moses.ini or switch): >>>>>>>>> config: filtered/moses.ini >>>>>>>>> cube-pruning-pop-limit: 1000 >>>>>>>>> input-factors: 0 >>>>>>>>> input-file: >>>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>>>>>> inputtype: 0 >>>>>>>>> lmodel-file: 8 0 3 >>>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>>>>>> mapping: 0 T 0 1 T 1 >>>>>>>>> max-chart-span: 20 1000 >>>>>>>>> n-best-list: run1.best100.out 100 >>>>>>>>> non-terminals: X >>>>>>>>> search-algorithm: 3 >>>>>>>>> ttable-file: 6 0 0 5 >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz 6 0 0 1 >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar ttable-limit: 20 >>>>>>>>> weight-l: 0.142857 >>>>>>>>> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>>>>>> weight-w: -0.285714 >>>>>>>>> Loading lexical distortion models...have 0 models >>>>>>>>> Start loading LanguageModel >>>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : >>>>>>>>> [0.000] seconds >>>>>>>>> Finished loading LanguageModels : [0.000] seconds >>>>>>>>> Using uniform ttable-limit of 20 for all translation tables. >>>>>>>>> Start loading PhraseTable >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>>> seconds >>>>>>>>> filePath: >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar : [0.000] >>>>>>>>> seconds >>>>>>>>> filePath: >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar Finished loading phrase tables : [0.000] seconds >>>>>>>>> Start loading phrase table from >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>>> seconds >>>>>>>>> Start loading new format pt model : [0.000] seconds >>>>>>>>> Finished loading phrase tables : [73.000] seconds >>>>>>>>> Start loading phrase table from >>>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>>> el /g lue-grammar : [73.000] >>>>>>>>> seconds >>>>>>>>> Start loading new format pt model : [73.000] seconds >>>>>>>>> Finished loading phrase tables : [73.000] seconds >>>>>>>>> Created input-output object : [73.000] seconds >>>>>>>>> Translating line 0 in thread id 0 >>>>>>>>> ERROR:ERROR: search. Aborting >>>>>>>>> >>>>>>>>> Aborted >>>>>>>>> Exit code: 134 >>>>>>>>> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 >>>>>>>>> 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks if you can help me to find the problem. >>>>>>>>> >>>>>>>>> Kairit >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Moses-support mailing list >>>>>>>>> [email protected] >>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>> _______________________________________________ >>>>>>> Moses-support mailing list >>>>>>> [email protected] >>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> -- >>>>>> Barry Haddow >>>>>> University of Edinburgh >>>>>> +44 (0) 131 651 3173 >>>>>> >>>>>> -- >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> -- >>>> Barry Haddow >>>> University of Edinburgh >>>> +44 (0) 131 651 3173 >>>> >>> -- >>> Barry Haddow >>> University of Edinburgh >>> +44 (0) 131 651 3173 >>> >>> -- >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> >>> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
