Hi Kairit You can pass the inputtype to mert-moses.pl with the -inputtype flag. Could you try this to see if it works?
cheers - Barry On Friday 18 May 2012 08:59:22 [email protected] wrote: > One problem fixed: mistakenly I had used moses instead of moses_chart. > > With inputtype I don't see how I could change it: moses.ini in > train/model folder already has inputtype 3, but tuning script calls it > with value 0. > > The problem still persists, but now seems to be a different one > according to the content of mert.out > > nohup: ignoring input > main::create_extractor_script() called too early to check prototype at > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > line 681. > Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts > Use of uninitialized value $extractorargs in concatenation (.) or > string at > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > line 370. > filtering the phrase tables... Fri May 18 10:31:35 EEST 2012 > exec: > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-inp > ut.pl -Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk > 1 1 5 100 2" --Hierarchical ./filtered > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mos > es.ini /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > Executing: > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-inp > ut.pl -Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk > 1 1 5 100 2" --Hierarchical ./filtered > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mos > es.ini /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > > filterphrases.out 2> filterphrases.err > Asking moses for feature names and values from filtered/moses.ini > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart > -config filtered/moses.ini -inputtype 0 -show-weights > ./features.list > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > cube-pruning-pop-limit: 1000 > input-factors: 0 > inputtype: 0 > lmodel-file: 8 0 3 > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > mapping: 0 T 0 1 T 1 > max-chart-span: 20 1000 > non-terminals: X > search-algorithm: 3 > show-weights: > ttable-file: 2 0 0 5 > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1.bin 6 0 0 1 > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glu > e-grammar ttable-limit: 20 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 > weight-w: -1 > Loading lexical distortion models...have 0 models > Start loading LanguageModel > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > seconds > Finished loading LanguageModels : [0.000] seconds > Using uniform ttable-limit of 20 for all translation tables. > Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1.bin : [0.000] seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1.bin Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue > -grammar : [0.000] seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue > -grammar Finished loading phrase tables : [0.000] seconds > Start loading phrase table from > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue > -grammar : [0.000] seconds > Start loading new format pt model : [0.000] seconds > Finished loading phrase tables : [0.000] seconds > MERT starting values and ranges for random generation: > lm = 0.500 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 1.000 ( 0.00 .. 1.00) > w = -1.000 ( 0.00 .. 1.00) > run 1 start at Fri May 18 10:38:15 EEST 2012 > Parsing --decoder-flags: || > Saving new config to: ./run1.moses.ini > Saved: ./run1.moses.ini > Normalizing lambdas: 0.500000 0.200000 0.200000 0.200000 0.200000 > 0.200000 1.000000 -1.000000 > DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > 0.057143 0.057143 0.285714 > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart > -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm > 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list > run1.best100.out 100 -input-file > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > > run1.out > (1) run decoder to produce n-best lists > params = > decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 > 0.057143 0.057143 0.057143 0.285714 > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > cube-pruning-pop-limit: 1000 > input-factors: 0 > input-file: > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > inputtype: 0 > lmodel-file: 8 0 3 > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > mapping: 0 T 0 1 T 1 > max-chart-span: 20 1000 > n-best-list: run1.best100.out 100 > non-terminals: X > search-algorithm: 3 > ttable-file: 2 0 0 5 > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1.bin 6 0 0 1 > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glu > e-grammar ttable-limit: 20 > weight-l: 0.142857 > weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > weight-w: -0.285714 > Loading lexical distortion models...have 0 models > Start loading LanguageModel > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > seconds > Finished loading LanguageModels : [1.000] seconds > Using uniform ttable-limit of 20 for all translation tables. > Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1.bin : [1.000] seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filter > ed/phrase-table.0-0.1.1.bin Start loading PhraseTable > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue > -grammar : [1.000] seconds > filePath: > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue > -grammar Finished loading phrase tables : [1.000] seconds > Start loading phrase table from > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue > -grammar : [1.000] seconds > Start loading new format pt model : [1.000] seconds > Finished loading phrase tables : [1.000] seconds > Created input-output object : [1.000] seconds > Translating: <s> eriti tÃhtis on teha koostööd sÃsinikuvaese > majanduse ja energiatõhusa Ãhiskonna loomisega seotud uuenduste > valdkonnas . </s> > > Translation took 0.090 seconds > Exception: std::bad_cast > Exit code: 1 > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > 0.057143 0.057143 0.057143 0.057143 0.285714 > > > Thank you for your patience, this is the first time for me to train a > MT system. > Kairit > > Barry Haddow <[email protected]> kirjutas: > > Hi Kairit > > > > Actually the problem is that you're using moses, not moses_chart. You > > need to use moses_chart for a hierarchical model, > > > > cheers - Barry > > > > On Thursday 17 May 2012 11:57:34 Barry Haddow wrote: > >> Hi Kairit > >> > >> Try setting your inputtype to 3, > >> > >> cheers - Barry > >> > >> On Thursday 17 May 2012 11:46:26 you wrote: > >> > Hi, > >> > > >> > Now the filtering works correctly but the decoder still dies. > >> > > >> > The command I'm running now, is: > >> > nohup nice > >> > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.en > >> > /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> > train/model/moses.ini --mertdir > >> > /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd > >> > '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-giv > >> >en- in put.pl -Binarizer > >> > "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 > >> > 2" --Hierarchical' &> mert.out > >> > > >> > I'm attaching again the mert.out file although it seems to be similar > >> > to the previous one: > >> > > >> > nohup: ignoring input > >> > main::create_extractor_script() called too early to check prototype at > >> > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> > line 681. > >> > Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts > >> > Use of uninitialized value $extractorargs in concatenation (.) or > >> > string at > >> > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl > >> > line 370. > >> > filtering the phrase tables... Wed May 16 20:40:51 EEST 2012 > >> > exec: > >> > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-give > >> >n-i np ut.pl -Binarizer > >> > "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 > >> > 2" --Hierarchical ./filtered > >> > > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/mo s es.ini > >> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > Executing: > >> > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-give > >> >n-i np ut.pl -Binarizer > >> > "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 > >> > 2" --Hierarchical ./filtered > >> > > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/mo s es.ini > >> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > > >> > > filterphrases.out 2> filterphrases.err > >> > > >> > Asking moses for feature names and values from filtered/moses.ini > >> > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config > >> > filtered/moses.ini -inputtype 0 -show-weights > ./features.list > >> > Defined parameters (per moses.ini or switch): > >> > config: filtered/moses.ini > >> > cube-pruning-pop-limit: 1000 > >> > input-factors: 0 > >> > inputtype: 0 > >> > lmodel-file: 8 0 3 > >> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> > mapping: 0 T 0 1 T 1 > >> > max-chart-span: 20 1000 > >> > non-terminals: X > >> > search-algorithm: 3 > >> > show-weights: > >> > ttable-file: 2 0 0 5 > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f > >> >ilt er ed/phrase-table.0-0.1.1.bin 6 0 0 1 > >> > > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl u e-grammar ttable-limit: 20 > >> > weight-l: 0.5000 > >> > weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 > >> > weight-w: -1 > >> > Loading lexical distortion models...have 0 models > >> > Start loading LanguageModel > >> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > >> > seconds > >> > Finished loading LanguageModels : [0.000] seconds > >> > Using uniform ttable-limit of 20 for all translation tables. > >> > Start loading PhraseTable > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f > >> >ilt er ed/phrase-table.0-0.1.1.bin : [0.000] seconds > >> > filePath: > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f > >> >ilt er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl ue -grammar : [0.000] seconds > >> > filePath: > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl ue -grammar Finished loading phrase tables : [0.000] seconds > >> > Start loading phrase table from > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl ue -grammar : [0.000] seconds > >> > Start loading new format pt model : [0.000] seconds > >> > Finished loading phrase tables : [0.000] seconds > >> > MERT starting values and ranges for random generation: > >> > lm = 0.500 ( 0.00 .. 1.00) > >> > w = -1.000 ( 0.00 .. 1.00) > >> > tm = 0.200 ( 0.00 .. 1.00) > >> > tm = 0.200 ( 0.00 .. 1.00) > >> > tm = 0.200 ( 0.00 .. 1.00) > >> > tm = 0.200 ( 0.00 .. 1.00) > >> > tm = 0.200 ( 0.00 .. 1.00) > >> > tm = 1.000 ( 0.00 .. 1.00) > >> > run 1 start at Wed May 16 20:47:37 EEST 2012 > >> > Parsing --decoder-flags: || > >> > Saving new config to: ./run1.moses.ini > >> > Saved: ./run1.moses.ini > >> > Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 > >> > 0.200000 0.200000 1.000000 > >> > DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > >> > 0.057143 0.057143 0.285714 > >> > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> > -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm > >> > 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list > >> > run1.best100.out 100 -input-file > >> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > > >> > run1.out > >> > (1) run decoder to produce n-best lists > >> > params = > >> > decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 > >> > 0.057143 0.057143 0.057143 0.285714 > >> > Defined parameters (per moses.ini or switch): > >> > config: filtered/moses.ini > >> > cube-pruning-pop-limit: 1000 > >> > input-factors: 0 > >> > input-file: > >> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > inputtype: 0 > >> > lmodel-file: 8 0 3 > >> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> > mapping: 0 T 0 1 T 1 > >> > max-chart-span: 20 1000 > >> > n-best-list: run1.best100.out 100 > >> > non-terminals: X > >> > search-algorithm: 3 > >> > ttable-file: 2 0 0 5 > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f > >> >ilt er ed/phrase-table.0-0.1.1.bin 6 0 0 1 > >> > > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl u e-grammar ttable-limit: 20 > >> > weight-l: 0.142857 > >> > weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > >> > weight-w: -0.285714 > >> > Loading lexical distortion models...have 0 models > >> > Start loading LanguageModel > >> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000] > >> > seconds > >> > Finished loading LanguageModels : [0.000] seconds > >> > Using uniform ttable-limit of 20 for all translation tables. > >> > Start loading PhraseTable > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f > >> >ilt er ed/phrase-table.0-0.1.1.bin : [0.000] seconds > >> > filePath: > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f > >> >ilt er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl ue -grammar : [0.000] seconds > >> > filePath: > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl ue -grammar Finished loading phrase tables : [0.000] seconds > >> > Start loading phrase table from > >> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model > >> >/gl ue -grammar : [0.000] seconds > >> > Start loading new format pt model : [0.000] seconds > >> > Finished loading phrase tables : [0.000] seconds > >> > Created input-output object : [0.000] seconds > >> > Translating line 0 in thread id 0 > >> > ERROR:ERROR: search. Aborting > >> > > >> > Aborted > >> > Exit code: 134 > >> > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > >> > 0.057143 0.057143 0.057143 0.057143 0.285714 > >> > > >> > Any further ideas, what could be wrong? > >> > Kairit > >> > > >> > Barry Haddow <[email protected]> kirjutas: > >> > > Hi Kairit > >> > > > >> > > This is the problem: > >> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-wor > >> > >>k/f il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": > >> > >> No such file or directory at > >> > > > >> > > Either CreateOnDIskPt does not exist, or you've provided an > >> > > incorrect path for > >> > > it. > >> > > > >> > > Not sure why the mert script didn't notice that filtering fails... > >> > > > >> > > cheers - Barry > >> > > > >> > > On Tuesday 15 May 2012 13:25:52 [email protected] wrote: > >> > >> Hi, > >> > >> > >> > >> The phrase tables have data and also the paths to the tuning sets > >> > >> are correct. However, filtering process seems to produce errors: > >> > >> > >> > >> The file mert-work/filterphrases.err has the content: > >> > >> > >> > >> Executing: mkdir -p > >> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-wor > >> > >>k/f il ter ed Considering factor 0 > >> > >> Done. > >> > >> filtering > >> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mo > >> > >>del /r ule -table.gz -> > >> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-wor > >> > >>k/f il te red/phrase-table.0-0.1.1... binarizing...CreateOnDiskPt 1 > >> > >> 1 5 100 2 > >> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-wor > >> > >>k/f il ter ed/phrase-table.0-0.1.1 > >> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-wor > >> > >>k/f il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": > >> > >> No such file or directory at > >> > >> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g > >> > >>ive n- inp ut.pl line 278, <PIPE> line 3721058. > >> > >> > >> > >> Kairit > >> > >> > >> > >> Hieu Hoang <[email protected]> kirjutas: > >> > >>> hi Kairit > >> > >>> > >> > >>> The phrase table > >> > >>> ---/filtered/phrase-table.0-0.1.1.gz > >> > >>> seems to have loaded in 0 seconds. > >> > >>> > >> > >>> This is the filtered phrase table. Check that the original phrase > >> > >>> table has data, and that the tuning set has data. > >> > >>> > >> > >>> hieu > >> > >>> > >> > >>> (ps. please subscribe to the mailing list to post emails) > >> > >>> > >> > >>> On 09/05/2012 09:23, [email protected] wrote: > >> > >>>> Hi, > >> > >>>> > >> > >>>> I'm trying to tune a hierarchical model and the process is > >> > >>>> aborted with Exit code 134 and the message that the decoder died. > >> > >>>> > >> > >>>> The command I'm running is: > >> > >>>> > >> > >>>> nohup nice > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.p > >> > >>>>l ../../parallel/europarl-v6.et-en.dev.et > >> > >>>> ../../parallel/europarl-v6.et-en.dev.en > >> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> > >>>> train/model/moses.ini --mertdir > >> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd > >> > >>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-mode > >> > >>>>l-g iv en -input.pl --Hierarchical'&> > >> > >>>> mert.out > >> > >>>> > >> > >>>> I have also tried this one and it doesn't work either: > >> > >>>> > >> > >>>> nohup nice > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.p > >> > >>>>l ../../parallel/europarl-v6.et-en.dev.et > >> > >>>> ../../parallel/europarl-v6.et-en.dev.en > >> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> > >>>> train/model/moses.ini --mertdir > >> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd > >> > >>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-mode > >> > >>>>l-g iv en -input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" > >> > >>>> --Hierarchical'&> mert.out > >> > >>>> > >> > >>>> The content of moses.ini file is: > >> > >>>> > >> > >>>> ######################### > >> > >>>> ### MOSES CONFIG FILE ### > >> > >>>> ######################### > >> > >>>> > >> > >>>> # input factors > >> > >>>> [input-factors] > >> > >>>> > >> > >>>> > >> > >>>> # mapping steps > >> > >>>> [mapping] > >> > >>>> 0 T 0 > >> > >>>> 1 T 1 > >> > >>>> > >> > >>>> # translation tables: table type (hierarchical(0), textual (0), > >> > >>>> binary (1)), source-factors, target-factors, number of scores, > >> > >>>> file # OLD FORMAT is still handled for back-compatibility > >> > >>>> # OLD FORMAT translation tables: source-factors, target-factors, > >> > >>>> number of scores, file > >> > >>>> # OLD FORMAT a binary table type (1) is assumed > >> > >>>> [ttable-file] > >> > >>>> 6 0 0 5 > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /r ule-table.gz 6 0 0 1 > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar > >> > >>>> > >> > >>>> # no generation models, no generation-file section > >> > >>>> > >> > >>>> # language models: type(srilm/irstlm), factors, order, file > >> > >>>> [lmodel-file] > >> > >>>> 8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> > >>>> > >> > >>>> > >> > >>>> # limit on how many phrase translations e for each phrase f are > >> > >>>> loaded # 0 = all elements loaded > >> > >>>> [ttable-limit] > >> > >>>> 20 > >> > >>>> > >> > >>>> > >> > >>>> # language model weights > >> > >>>> [weight-l] > >> > >>>> 0.5000 > >> > >>>> > >> > >>>> > >> > >>>> # translation model weights > >> > >>>> [weight-t] > >> > >>>> 0.20 > >> > >>>> 0.20 > >> > >>>> 0.20 > >> > >>>> 0.20 > >> > >>>> 0.20 > >> > >>>> 1.0 > >> > >>>> > >> > >>>> # no generation models, no weight-generation section > >> > >>>> > >> > >>>> # word penalty > >> > >>>> [weight-w] > >> > >>>> -1 > >> > >>>> > >> > >>>> [cube-pruning-pop-limit] > >> > >>>> 1000 > >> > >>>> > >> > >>>> [non-terminals] > >> > >>>> X > >> > >>>> > >> > >>>> [search-algorithm] > >> > >>>> 3 > >> > >>>> > >> > >>>> [inputtype] > >> > >>>> 3 > >> > >>>> > >> > >>>> [max-chart-span] > >> > >>>> 20 > >> > >>>> 1000 > >> > >>>> > >> > >>>> The content of mert.out file is: > >> > >>>> > >> > >>>> nohup: ignoring input > >> > >>>> main::create_extractor_script() called too early to check > >> > >>>> prototype at > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.p > >> > >>>>l line 681. > >> > >>>> Using SCRIPTS_ROOTDIR: > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts Use of uninitialized > >> > >>>> value $extractorargs in concatenation (.) or string at > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.p > >> > >>>>l line 370. > >> > >>>> filtering the phrase tables... Tue May 8 18:02:26 EEST 2012 > >> > >>>> exec: > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model > >> > >>>>-gi ve n- input.pl --Hierarchical ./filtered > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /m oses.ini > >> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > >>>> Executing: > >> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model > >> > >>>>-gi ve n- input.pl --Hierarchical ./filtered > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /m oses.ini > >> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > >>>>> filterphrases.out 2> > >> > >>>> filterphrases.err > >> > >>>> Asking moses for feature names and values from filtered/moses.ini > >> > >>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> > >>>> -config filtered/moses.ini -inputtype 0 -show-weights> > >> > >>>> ./features.list Defined parameters (per moses.ini or switch): > >> > >>>> config: filtered/moses.ini > >> > >>>> cube-pruning-pop-limit: 1000 > >> > >>>> input-factors: 0 > >> > >>>> inputtype: 0 > >> > >>>> lmodel-file: 8 0 3 > >> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> > >>>> mapping: 0 T 0 1 T 1 > >> > >>>> max-chart-span: 20 1000 > >> > >>>> non-terminals: X > >> > >>>> search-algorithm: 3 > >> > >>>> show-weights: > >> > >>>> ttable-file: 6 0 0 5 > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz 6 0 0 1 > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar ttable-limit: 20 > >> > >>>> weight-l: 0.5000 > >> > >>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 > >> > >>>> weight-w: -1 > >> > >>>> Loading lexical distortion models...have 0 models > >> > >>>> Start loading LanguageModel > >> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : > >> > >>>> [0.000] seconds > >> > >>>> Finished loading LanguageModels : [0.000] seconds > >> > >>>> Using uniform ttable-limit of 20 for all translation tables. > >> > >>>> Start loading PhraseTable > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz : [0.000] > >> > >>>> seconds > >> > >>>> filePath: > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar : [0.000] > >> > >>>> seconds > >> > >>>> filePath: > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar Finished loading phrase tables : [0.000] > >> > >>>> seconds Start loading phrase table from > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz : [0.000] > >> > >>>> seconds > >> > >>>> Start loading new format pt model : [0.000] seconds > >> > >>>> Finished loading phrase tables : [72.000] seconds > >> > >>>> Start loading phrase table from > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar : [72.000] > >> > >>>> seconds > >> > >>>> Start loading new format pt model : [72.000] seconds > >> > >>>> Finished loading phrase tables : [72.000] seconds > >> > >>>> MERT starting values and ranges for random generation: > >> > >>>> lm = 0.500 ( 0.00 .. 1.00) > >> > >>>> w = -1.000 ( 0.00 .. 1.00) > >> > >>>> tm = 0.200 ( 0.00 .. 1.00) > >> > >>>> tm = 0.200 ( 0.00 .. 1.00) > >> > >>>> tm = 0.200 ( 0.00 .. 1.00) > >> > >>>> tm = 0.200 ( 0.00 .. 1.00) > >> > >>>> tm = 0.200 ( 0.00 .. 1.00) > >> > >>>> tm = 1.000 ( 0.00 .. 1.00) > >> > >>>> run 1 start at Tue May 8 18:05:59 EEST 2012 > >> > >>>> Parsing --decoder-flags: || > >> > >>>> Saving new config to: ./run1.moses.ini > >> > >>>> Saved: ./run1.moses.ini > >> > >>>> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 > >> > >>>> 0.200000 0.200000 0.200000 1.000000 > >> > >>>> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 > >> > >>>> 0.057143 0.057143 0.057143 0.285714 > >> > >>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses > >> > >>>> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 > >> > >>>> -tm 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > >> > >>>> -n-best-list run1.best100.out 100 -input-file > >> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > >>>>> run1.out > >> > >>>> (1) run decoder to produce n-best lists > >> > >>>> params = > >> > >>>> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 > >> > >>>> 0.057143 0.057143 0.057143 0.285714 > >> > >>>> Defined parameters (per moses.ini or switch): > >> > >>>> config: filtered/moses.ini > >> > >>>> cube-pruning-pop-limit: 1000 > >> > >>>> input-factors: 0 > >> > >>>> input-file: > >> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et > >> > >>>> inputtype: 0 > >> > >>>> lmodel-file: 8 0 3 > >> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en > >> > >>>> mapping: 0 T 0 1 T 1 > >> > >>>> max-chart-span: 20 1000 > >> > >>>> n-best-list: run1.best100.out 100 > >> > >>>> non-terminals: X > >> > >>>> search-algorithm: 3 > >> > >>>> ttable-file: 6 0 0 5 > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz 6 0 0 1 > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar ttable-limit: 20 > >> > >>>> weight-l: 0.142857 > >> > >>>> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > >> > >>>> weight-w: -0.285714 > >> > >>>> Loading lexical distortion models...have 0 models > >> > >>>> Start loading LanguageModel > >> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : > >> > >>>> [0.000] seconds > >> > >>>> Finished loading LanguageModels : [0.000] seconds > >> > >>>> Using uniform ttable-limit of 20 for all translation tables. > >> > >>>> Start loading PhraseTable > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz : [0.000] > >> > >>>> seconds > >> > >>>> filePath: > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar : [0.000] > >> > >>>> seconds > >> > >>>> filePath: > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar Finished loading phrase tables : [0.000] > >> > >>>> seconds Start loading phrase table from > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-w > >> > >>>>ork /f il tered/phrase-table.0-0.1.1.gz : [0.000] > >> > >>>> seconds > >> > >>>> Start loading new format pt model : [0.000] seconds > >> > >>>> Finished loading phrase tables : [73.000] seconds > >> > >>>> Start loading phrase table from > >> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/ > >> > >>>>mod el /g lue-grammar : [73.000] > >> > >>>> seconds > >> > >>>> Start loading new format pt model : [73.000] seconds > >> > >>>> Finished loading phrase tables : [73.000] seconds > >> > >>>> Created input-output object : [73.000] seconds > >> > >>>> Translating line 0 in thread id 0 > >> > >>>> ERROR:ERROR: search. Aborting > >> > >>>> > >> > >>>> Aborted > >> > >>>> Exit code: 134 > >> > >>>> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm > >> > >>>> 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 > >> > >>>> > >> > >>>> > >> > >>>> Thanks if you can help me to find the problem. > >> > >>>> > >> > >>>> Kairit > >> > >>>> > >> > >>>> > >> > >>>> _______________________________________________ > >> > >>>> Moses-support mailing list > >> > >>>> [email protected] > >> > >>>> http://mailman.mit.edu/mailman/listinfo/moses-support > >> > >> > >> > >> _______________________________________________ > >> > >> Moses-support mailing list > >> > >> [email protected] > >> > >> http://mailman.mit.edu/mailman/listinfo/moses-support > >> > > > >> > > -- > >> > > Barry Haddow > >> > > University of Edinburgh > >> > > +44 (0) 131 651 3173 > >> > > > >> > > -- > >> > > The University of Edinburgh is a charitable body, registered in > >> > > Scotland, with registration number SC005336. > >> > > >> > _______________________________________________ > >> > Moses-support mailing list > >> > [email protected] > >> > http://mailman.mit.edu/mailman/listinfo/moses-support > >> > >> -- > >> Barry Haddow > >> University of Edinburgh > >> +44 (0) 131 651 3173 > > > > -- > > Barry Haddow > > University of Edinburgh > > +44 (0) 131 651 3173 > > > > -- > > The University of Edinburgh is a charitable body, registered in > > Scotland, with registration number SC005336. > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Barry Haddow University of Edinburgh +44 (0) 131 651 3173 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
