hi kairit apologies. A small bug in the chart decoder only lets you use inputtype=3. And by default the mert script always set inputtype=0, unless you override it.
However, I've fixed the bug in the decoder so it should work now. On 18/05/2012 08:59, [email protected] wrote: > One problem fixed: mistakenly I had used moses instead of moses_chart. > > With inputtype I don't see how I could change it: moses.ini in > train/model folder already has inputtype 3, but tuning script calls it > with value 0. > > The problem still persists, but now seems to be a different one > according to the content of mert.out > > nohup: ignoring input > main::create_extractor_script() called too early to check prototype at > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > line 681. > Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts > Use of uninitialized value $extractorargs in concatenation (.) or > string at > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > line 370. > filtering the phrase tables... Fri May 18 10:31:35 EEST 2012 > exec: > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl > -Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 > 100 2" --Hierarchical ./filtered > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > Executing: > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl > -Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 > 100 2" --Hierarchical ./filtered > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> > filterphrases.out 2> > filterphrases.err > Asking moses for feature names and values from filtered/moses.ini > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart > -config filtered/moses.ini -inputtype 0 -show-weights> ./features.list > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > cube-pruning-pop-limit: 1000 > input-factors: 0 > inputtype: 0 > lmodel-file: 8 0 3 > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > mapping: 0 T 0 1 T 1 > max-chart-span: 20 1000 > non-terminals: X > search-algorithm: 3 > show-weights: > ttable-file: 2 0 0 5 > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin > 6 0 0 1 > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > ttable-limit: 20 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 > weight-w: -1 > Loading lexical distortion models...have 0 models > Start loading LanguageModel > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > seconds > Finished loading LanguageModels : [0.000] seconds > Using uniform ttable-limit of 20 for all translation tables. > Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin > : [0.000] > seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin > Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > : [0.000] > seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > Finished loading phrase tables : [0.000] seconds > Start loading phrase table from > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > : [0.000] > seconds > Start loading new format pt model : [0.000] seconds > Finished loading phrase tables : [0.000] seconds > MERT starting values and ranges for random generation: > lm = 0.500 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 1.000 ( 0.00 .. 1.00) > w = -1.000 ( 0.00 .. 1.00) > run 1 start at Fri May 18 10:38:15 EEST 2012 > Parsing --decoder-flags: || > Saving new config to: ./run1.moses.ini > Saved: ./run1.moses.ini > Normalizing lambdas: 0.500000 0.200000 0.200000 0.200000 0.200000 > 0.200000 1.000000 -1.000000 > DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > 0.057143 0.057143 0.285714 > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart > -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm > 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list > run1.best100.out 100 -input-file > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> > run1.out > (1) run decoder to produce n-best lists > params = > decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 > 0.057143 0.057143 0.057143 0.285714 > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > cube-pruning-pop-limit: 1000 > input-factors: 0 > input-file: > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > inputtype: 0 > lmodel-file: 8 0 3 > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > mapping: 0 T 0 1 T 1 > max-chart-span: 20 1000 > n-best-list: run1.best100.out 100 > non-terminals: X > search-algorithm: 3 > ttable-file: 2 0 0 5 > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin > 6 0 0 1 > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > ttable-limit: 20 > weight-l: 0.142857 > weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > weight-w: -0.285714 > Loading lexical distortion models...have 0 models > Start loading LanguageModel > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > seconds > Finished loading LanguageModels : [1.000] seconds > Using uniform ttable-limit of 20 for all translation tables. > Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin > : [1.000] > seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin > Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > : [1.000] > seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > Finished loading phrase tables : [1.000] seconds > Start loading phrase table from > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar > : [1.000] > seconds > Start loading new format pt model : [1.000] seconds > Finished loading phrase tables : [1.000] seconds > Created input-output object : [1.000] seconds > Translating:<s> eriti tÀhtis on teha koostööd sÌsinikuvaese > majanduse ja energiatõhusa Ìhiskonna loomisega seotud uuenduste > valdkonnas .</s> > > Translation took 0.090 seconds > Exception: std::bad_cast > Exit code: 1 > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > 0.057143 0.057143 0.057143 0.057143 0.285714 > > > Thank you for your patience, this is the first time for me to train a > MT system. > Kairit > > Barry Haddow<[email protected]> kirjutas: > >> Hi Kairit >> >> Actually the problem is that you're using moses, not moses_chart. You need to >> use moses_chart for a hierarchical model, >> >> cheers - Barry >> >> On Thursday 17 May 2012 11:57:34 Barry Haddow wrote: >>> Hi Kairit >>> >>> Try setting your inputtype to 3, >>> >>> cheers - Barry >>> >>> On Thursday 17 May 2012 11:46:26 you wrote: >>>> Hi, >>>> >>>> Now the filtering works correctly but the decoder still dies. >>>> >>>> The command I'm running now, is: >>>> nohup nice >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.en >>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>> train/model/moses.ini --mertdir >>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd >>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given- >>>> in put.pl -Binarizer >>>> "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2" >>>> --Hierarchical'&> mert.out >>>> >>>> I'm attaching again the mert.out file although it seems to be similar >>>> to the previous one: >>>> >>>> nohup: ignoring input >>>> main::create_extractor_script() called too early to check prototype at >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>> line 681. >>>> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts >>>> Use of uninitialized value $extractorargs in concatenation (.) or >>>> string at >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>> line 370. >>>> filtering the phrase tables... Wed May 16 20:40:51 EEST 2012 >>>> exec: >>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-i >>>> np ut.pl -Binarizer >>>> "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2" >>>> --Hierarchical ./filtered >>>> >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mo >>>> s es.ini /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>> Executing: >>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-i >>>> np ut.pl -Binarizer >>>> "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2" >>>> --Hierarchical ./filtered >>>> >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mo >>>> s es.ini /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>> filterphrases.out 2> filterphrases.err >>>> Asking moses for feature names and values from filtered/moses.ini >>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config >>>> filtered/moses.ini -inputtype 0 -show-weights> ./features.list >>>> Defined parameters (per moses.ini or switch): >>>> config: filtered/moses.ini >>>> cube-pruning-pop-limit: 1000 >>>> input-factors: 0 >>>> inputtype: 0 >>>> lmodel-file: 8 0 3 >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>> mapping: 0 T 0 1 T 1 >>>> max-chart-span: 20 1000 >>>> non-terminals: X >>>> search-algorithm: 3 >>>> show-weights: >>>> ttable-file: 2 0 0 5 >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>> er ed/phrase-table.0-0.1.1.bin 6 0 0 1 >>>> >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> u e-grammar ttable-limit: 20 >>>> weight-l: 0.5000 >>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 >>>> weight-w: -1 >>>> Loading lexical distortion models...have 0 models >>>> Start loading LanguageModel >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] >>>> seconds >>>> Finished loading LanguageModels : [0.000] seconds >>>> Using uniform ttable-limit of 20 for all translation tables. >>>> Start loading PhraseTable >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>> er ed/phrase-table.0-0.1.1.bin : [0.000] seconds >>>> filePath: >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>> er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> ue -grammar : [0.000] seconds >>>> filePath: >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> ue -grammar Finished loading phrase tables : [0.000] seconds >>>> Start loading phrase table from >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> ue -grammar : [0.000] seconds >>>> Start loading new format pt model : [0.000] seconds >>>> Finished loading phrase tables : [0.000] seconds >>>> MERT starting values and ranges for random generation: >>>> lm = 0.500 ( 0.00 .. 1.00) >>>> w = -1.000 ( 0.00 .. 1.00) >>>> tm = 0.200 ( 0.00 .. 1.00) >>>> tm = 0.200 ( 0.00 .. 1.00) >>>> tm = 0.200 ( 0.00 .. 1.00) >>>> tm = 0.200 ( 0.00 .. 1.00) >>>> tm = 0.200 ( 0.00 .. 1.00) >>>> tm = 1.000 ( 0.00 .. 1.00) >>>> run 1 start at Wed May 16 20:47:37 EEST 2012 >>>> Parsing --decoder-flags: || >>>> Saving new config to: ./run1.moses.ini >>>> Saved: ./run1.moses.ini >>>> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 >>>> 0.200000 0.200000 1.000000 >>>> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 >>>> 0.057143 0.057143 0.285714 >>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm >>>> 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list >>>> run1.best100.out 100 -input-file >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >>>> run1.out >>>> (1) run decoder to produce n-best lists >>>> params = >>>> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >>>> 0.057143 0.057143 0.057143 0.285714 >>>> Defined parameters (per moses.ini or switch): >>>> config: filtered/moses.ini >>>> cube-pruning-pop-limit: 1000 >>>> input-factors: 0 >>>> input-file: >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>> inputtype: 0 >>>> lmodel-file: 8 0 3 >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>> mapping: 0 T 0 1 T 1 >>>> max-chart-span: 20 1000 >>>> n-best-list: run1.best100.out 100 >>>> non-terminals: X >>>> search-algorithm: 3 >>>> ttable-file: 2 0 0 5 >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>> er ed/phrase-table.0-0.1.1.bin 6 0 0 1 >>>> >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> u e-grammar ttable-limit: 20 >>>> weight-l: 0.142857 >>>> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >>>> weight-w: -0.285714 >>>> Loading lexical distortion models...have 0 models >>>> Start loading LanguageModel >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] >>>> seconds >>>> Finished loading LanguageModels : [0.000] seconds >>>> Using uniform ttable-limit of 20 for all translation tables. >>>> Start loading PhraseTable >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>> er ed/phrase-table.0-0.1.1.bin : [0.000] seconds >>>> filePath: >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt >>>> er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> ue -grammar : [0.000] seconds >>>> filePath: >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> ue -grammar Finished loading phrase tables : [0.000] seconds >>>> Start loading phrase table from >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl >>>> ue -grammar : [0.000] seconds >>>> Start loading new format pt model : [0.000] seconds >>>> Finished loading phrase tables : [0.000] seconds >>>> Created input-output object : [0.000] seconds >>>> Translating line 0 in thread id 0 >>>> ERROR:ERROR: search. Aborting >>>> >>>> Aborted >>>> Exit code: 134 >>>> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 >>>> 0.057143 0.057143 0.057143 0.057143 0.285714 >>>> >>>> Any further ideas, what could be wrong? >>>> Kairit >>>> >>>> Barry Haddow<[email protected]> kirjutas: >>>>> Hi Kairit >>>>> >>>>> This is the problem: >>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>> il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such >>>>>> file or directory at >>>>> Either CreateOnDIskPt does not exist, or you've provided an >>>>> incorrect path for >>>>> it. >>>>> >>>>> Not sure why the mert script didn't notice that filtering fails... >>>>> >>>>> cheers - Barry >>>>> >>>>> On Tuesday 15 May 2012 13:25:52 [email protected] wrote: >>>>>> Hi, >>>>>> >>>>>> The phrase tables have data and also the paths to the tuning sets are >>>>>> correct. However, filtering process seems to produce errors: >>>>>> >>>>>> The file mert-work/filterphrases.err has the content: >>>>>> >>>>>> Executing: mkdir -p >>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>> il ter ed Considering factor 0 >>>>>> Done. >>>>>> filtering >>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model >>>>>> /r ule -table.gz -> >>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>> il te red/phrase-table.0-0.1.1... binarizing...CreateOnDiskPt 1 1 5 100 >>>>>> 2 >>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>> il ter ed/phrase-table.0-0.1.1 >>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f >>>>>> il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such >>>>>> file or directory at >>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-give >>>>>> n- inp ut.pl line 278,<PIPE> line 3721058. >>>>>> >>>>>> Kairit >>>>>> >>>>>> Hieu Hoang<[email protected]> kirjutas: >>>>>>> hi Kairit >>>>>>> >>>>>>> The phrase table >>>>>>> ---/filtered/phrase-table.0-0.1.1.gz >>>>>>> seems to have loaded in 0 seconds. >>>>>>> >>>>>>> This is the filtered phrase table. Check that the original phrase >>>>>>> table has data, and that the tuning set has data. >>>>>>> >>>>>>> hieu >>>>>>> >>>>>>> (ps. please subscribe to the mailing list to post emails) >>>>>>> >>>>>>> On 09/05/2012 09:23, [email protected] wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm trying to tune a hierarchical model and the process is aborted >>>>>>>> with Exit code 134 and the message that the decoder died. >>>>>>>> >>>>>>>> The command I'm running is: >>>>>>>> >>>>>>>> nohup nice >>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>> ../../parallel/europarl-v6.et-en.dev.et >>>>>>>> ../../parallel/europarl-v6.et-en.dev.en >>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>> train/model/moses.ini --mertdir >>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd >>>>>>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g >>>>>>>> iv en -input.pl --Hierarchical'&> >>>>>>>> mert.out >>>>>>>> >>>>>>>> I have also tried this one and it doesn't work either: >>>>>>>> >>>>>>>> nohup nice >>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>> ../../parallel/europarl-v6.et-en.dev.et >>>>>>>> ../../parallel/europarl-v6.et-en.dev.en >>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>> train/model/moses.ini --mertdir >>>>>>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd >>>>>>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g >>>>>>>> iv en -input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" >>>>>>>> --Hierarchical'&> mert.out >>>>>>>> >>>>>>>> The content of moses.ini file is: >>>>>>>> >>>>>>>> ######################### >>>>>>>> ### MOSES CONFIG FILE ### >>>>>>>> ######################### >>>>>>>> >>>>>>>> # input factors >>>>>>>> [input-factors] >>>>>>>> >>>>>>>> >>>>>>>> # mapping steps >>>>>>>> [mapping] >>>>>>>> 0 T 0 >>>>>>>> 1 T 1 >>>>>>>> >>>>>>>> # translation tables: table type (hierarchical(0), textual (0), >>>>>>>> binary (1)), source-factors, target-factors, number of scores, file >>>>>>>> # OLD FORMAT is still handled for back-compatibility >>>>>>>> # OLD FORMAT translation tables: source-factors, target-factors, >>>>>>>> number of scores, file >>>>>>>> # OLD FORMAT a binary table type (1) is assumed >>>>>>>> [ttable-file] >>>>>>>> 6 0 0 5 >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /r ule-table.gz 6 0 0 1 >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar >>>>>>>> >>>>>>>> # no generation models, no generation-file section >>>>>>>> >>>>>>>> # language models: type(srilm/irstlm), factors, order, file >>>>>>>> [lmodel-file] >>>>>>>> 8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>>>>> >>>>>>>> >>>>>>>> # limit on how many phrase translations e for each phrase f are >>>>>>>> loaded # 0 = all elements loaded >>>>>>>> [ttable-limit] >>>>>>>> 20 >>>>>>>> >>>>>>>> >>>>>>>> # language model weights >>>>>>>> [weight-l] >>>>>>>> 0.5000 >>>>>>>> >>>>>>>> >>>>>>>> # translation model weights >>>>>>>> [weight-t] >>>>>>>> 0.20 >>>>>>>> 0.20 >>>>>>>> 0.20 >>>>>>>> 0.20 >>>>>>>> 0.20 >>>>>>>> 1.0 >>>>>>>> >>>>>>>> # no generation models, no weight-generation section >>>>>>>> >>>>>>>> # word penalty >>>>>>>> [weight-w] >>>>>>>> -1 >>>>>>>> >>>>>>>> [cube-pruning-pop-limit] >>>>>>>> 1000 >>>>>>>> >>>>>>>> [non-terminals] >>>>>>>> X >>>>>>>> >>>>>>>> [search-algorithm] >>>>>>>> 3 >>>>>>>> >>>>>>>> [inputtype] >>>>>>>> 3 >>>>>>>> >>>>>>>> [max-chart-span] >>>>>>>> 20 >>>>>>>> 1000 >>>>>>>> >>>>>>>> The content of mert.out file is: >>>>>>>> >>>>>>>> nohup: ignoring input >>>>>>>> main::create_extractor_script() called too early to check prototype >>>>>>>> at >>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>> line 681. >>>>>>>> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts >>>>>>>> Use of uninitialized value $extractorargs in concatenation (.) or >>>>>>>> string at >>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>> line 370. >>>>>>>> filtering the phrase tables... Tue May 8 18:02:26 EEST 2012 >>>>>>>> exec: >>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-gi >>>>>>>> ve n- input.pl --Hierarchical ./filtered >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /m oses.ini >>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>>>>> Executing: >>>>>>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-gi >>>>>>>> ve n- input.pl --Hierarchical ./filtered >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /m oses.ini >>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >>>>>>>> filterphrases.out 2> >>>>>>>> filterphrases.err >>>>>>>> Asking moses for feature names and values from filtered/moses.ini >>>>>>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>> -config filtered/moses.ini -inputtype 0 -show-weights> >>>>>>>> ./features.list Defined parameters (per moses.ini or switch): >>>>>>>> config: filtered/moses.ini >>>>>>>> cube-pruning-pop-limit: 1000 >>>>>>>> input-factors: 0 >>>>>>>> inputtype: 0 >>>>>>>> lmodel-file: 8 0 3 >>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>>>>> mapping: 0 T 0 1 T 1 >>>>>>>> max-chart-span: 20 1000 >>>>>>>> non-terminals: X >>>>>>>> search-algorithm: 3 >>>>>>>> show-weights: >>>>>>>> ttable-file: 6 0 0 5 >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz 6 0 0 1 >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar ttable-limit: 20 >>>>>>>> weight-l: 0.5000 >>>>>>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 >>>>>>>> weight-w: -1 >>>>>>>> Loading lexical distortion models...have 0 models >>>>>>>> Start loading LanguageModel >>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : >>>>>>>> [0.000] seconds >>>>>>>> Finished loading LanguageModels : [0.000] seconds >>>>>>>> Using uniform ttable-limit of 20 for all translation tables. >>>>>>>> Start loading PhraseTable >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>> seconds >>>>>>>> filePath: >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar : [0.000] >>>>>>>> seconds >>>>>>>> filePath: >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar Finished loading phrase tables : [0.000] seconds >>>>>>>> Start loading phrase table from >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>> seconds >>>>>>>> Start loading new format pt model : [0.000] seconds >>>>>>>> Finished loading phrase tables : [72.000] seconds >>>>>>>> Start loading phrase table from >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar : [72.000] >>>>>>>> seconds >>>>>>>> Start loading new format pt model : [72.000] seconds >>>>>>>> Finished loading phrase tables : [72.000] seconds >>>>>>>> MERT starting values and ranges for random generation: >>>>>>>> lm = 0.500 ( 0.00 .. 1.00) >>>>>>>> w = -1.000 ( 0.00 .. 1.00) >>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>> tm = 0.200 ( 0.00 .. 1.00) >>>>>>>> tm = 1.000 ( 0.00 .. 1.00) >>>>>>>> run 1 start at Tue May 8 18:05:59 EEST 2012 >>>>>>>> Parsing --decoder-flags: || >>>>>>>> Saving new config to: ./run1.moses.ini >>>>>>>> Saved: ./run1.moses.ini >>>>>>>> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 >>>>>>>> 0.200000 0.200000 1.000000 >>>>>>>> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >>>>>>>> 0.057143 0.057143 0.057143 0.285714 >>>>>>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses >>>>>>>> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 >>>>>>>> -tm 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>>>>> -n-best-list run1.best100.out 100 -input-file >>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et> >>>>>>>> run1.out >>>>>>>> (1) run decoder to produce n-best lists >>>>>>>> params = >>>>>>>> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 >>>>>>>> 0.057143 0.057143 0.057143 0.285714 >>>>>>>> Defined parameters (per moses.ini or switch): >>>>>>>> config: filtered/moses.ini >>>>>>>> cube-pruning-pop-limit: 1000 >>>>>>>> input-factors: 0 >>>>>>>> input-file: >>>>>>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >>>>>>>> inputtype: 0 >>>>>>>> lmodel-file: 8 0 3 >>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en >>>>>>>> mapping: 0 T 0 1 T 1 >>>>>>>> max-chart-span: 20 1000 >>>>>>>> n-best-list: run1.best100.out 100 >>>>>>>> non-terminals: X >>>>>>>> search-algorithm: 3 >>>>>>>> ttable-file: 6 0 0 5 >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz 6 0 0 1 >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar ttable-limit: 20 >>>>>>>> weight-l: 0.142857 >>>>>>>> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>>>>> weight-w: -0.285714 >>>>>>>> Loading lexical distortion models...have 0 models >>>>>>>> Start loading LanguageModel >>>>>>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : >>>>>>>> [0.000] seconds >>>>>>>> Finished loading LanguageModels : [0.000] seconds >>>>>>>> Using uniform ttable-limit of 20 for all translation tables. >>>>>>>> Start loading PhraseTable >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>> seconds >>>>>>>> filePath: >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar : [0.000] >>>>>>>> seconds >>>>>>>> filePath: >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar Finished loading phrase tables : [0.000] seconds >>>>>>>> Start loading phrase table from >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work >>>>>>>> /f il tered/phrase-table.0-0.1.1.gz : [0.000] >>>>>>>> seconds >>>>>>>> Start loading new format pt model : [0.000] seconds >>>>>>>> Finished loading phrase tables : [73.000] seconds >>>>>>>> Start loading phrase table from >>>>>>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod >>>>>>>> el /g lue-grammar : [73.000] >>>>>>>> seconds >>>>>>>> Start loading new format pt model : [73.000] seconds >>>>>>>> Finished loading phrase tables : [73.000] seconds >>>>>>>> Created input-output object : [73.000] seconds >>>>>>>> Translating line 0 in thread id 0 >>>>>>>> ERROR:ERROR: search. Aborting >>>>>>>> >>>>>>>> Aborted >>>>>>>> Exit code: 134 >>>>>>>> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 >>>>>>>> 0.057143 0.057143 0.057143 0.057143 0.285714 >>>>>>>> >>>>>>>> >>>>>>>> Thanks if you can help me to find the problem. >>>>>>>> >>>>>>>> Kairit >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Moses-support mailing list >>>>>>>> [email protected] >>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> -- >>>>> Barry Haddow >>>>> University of Edinburgh >>>>> +44 (0) 131 651 3173 >>>>> >>>>> -- >>>>> The University of Edinburgh is a charitable body, registered in >>>>> Scotland, with registration number SC005336. >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> -- >>> Barry Haddow >>> University of Edinburgh >>> +44 (0) 131 651 3173 >>> >> -- >> Barry Haddow >> University of Edinburgh >> +44 (0) 131 651 3173 >> >> -- >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
