One problem fixed: mistakenly I had used moses instead of moses_chart.
With inputtype I don't see how I could change it: moses.ini in
train/model folder already has inputtype 3, but tuning script calls it
with value 0.
The problem still persists, but now seems to be a different one
according to the content of mert.out
nohup: ignoring input
main::create_extractor_script() called too early to check prototype at
/usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
line 681.
Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts
Use of uninitialized value $extractorargs in concatenation (.) or
string at
/usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
line 370.
filtering the phrase tables... Fri May 18 10:31:35 EEST 2012
exec:
/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
-Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5
100 2" --Hierarchical ./filtered
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
Executing:
/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
-Binarizer "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5
100 2" --Hierarchical ./filtered
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/moses.ini
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >
filterphrases.out 2>
filterphrases.err
Asking moses for feature names and values from filtered/moses.ini
Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart
-config filtered/moses.ini -inputtype 0 -show-weights > ./features.list
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
cube-pruning-pop-limit: 1000
input-factors: 0
inputtype: 0
lmodel-file: 8 0 3
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
mapping: 0 T 0 1 T 1
max-chart-span: 20 1000
non-terminals: X
search-algorithm: 3
show-weights:
ttable-file: 2 0 0 5
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin
6 0 0 1
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
ttable-limit: 20
weight-l: 0.5000
weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
weight-w: -1
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
seconds
Finished loading LanguageModels : [0.000] seconds
Using uniform ttable-limit of 20 for all translation tables.
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin
: [0.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [0.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
Finished loading phrase tables : [0.000] seconds
Start loading phrase table from
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [0.000]
seconds
Start loading new format pt model : [0.000] seconds
Finished loading phrase tables : [0.000] seconds
MERT starting values and ranges for random generation:
lm = 0.500 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 1.000 ( 0.00 .. 1.00)
w = -1.000 ( 0.00 .. 1.00)
run 1 start at Fri May 18 10:38:15 EEST 2012
Parsing --decoder-flags: ||
Saving new config to: ./run1.moses.ini
Saved: ./run1.moses.ini
Normalizing lambdas: 0.500000 0.200000 0.200000 0.200000 0.200000
0.200000 1.000000 -1.000000
DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143
0.057143 0.057143 0.285714
Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses_chart
-config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm
0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list
run1.best100.out 100 -input-file
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >
run1.out
(1) run decoder to produce n-best lists
params =
decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
0.057143 0.057143 0.057143 0.285714
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
cube-pruning-pop-limit: 1000
input-factors: 0
input-file:
/home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
inputtype: 0
lmodel-file: 8 0 3
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
mapping: 0 T 0 1 T 1
max-chart-span: 20 1000
n-best-list: run1.best100.out 100
non-terminals: X
search-algorithm: 3
ttable-file: 2 0 0 5
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin
6 0 0 1
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
ttable-limit: 20
weight-l: 0.142857
weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714
weight-w: -0.285714
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
seconds
Finished loading LanguageModels : [1.000] seconds
Using uniform ttable-limit of 20 for all translation tables.
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin
: [1.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filtered/phrase-table.0-0.1.1.bin
Start loading PhraseTable
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [1.000]
seconds
filePath:
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
Finished loading phrase tables : [1.000] seconds
Start loading phrase table from
/home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/glue-grammar
: [1.000]
seconds
Start loading new format pt model : [1.000] seconds
Finished loading phrase tables : [1.000] seconds
Created input-output object : [1.000] seconds
Translating: <s> eriti tÃhtis on teha koostööd sÃsinikuvaese
majanduse ja energiatõhusa Ãhiskonna loomisega seotud uuenduste
valdkonnas . </s>
Translation took 0.090 seconds
Exception: std::bad_cast
Exit code: 1
The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
0.057143 0.057143 0.057143 0.057143 0.285714
Thank you for your patience, this is the first time for me to train a
MT system.
Kairit
Barry Haddow <[email protected]> kirjutas:
> Hi Kairit
>
> Actually the problem is that you're using moses, not moses_chart. You need to
> use moses_chart for a hierarchical model,
>
> cheers - Barry
>
> On Thursday 17 May 2012 11:57:34 Barry Haddow wrote:
>> Hi Kairit
>>
>> Try setting your inputtype to 3,
>>
>> cheers - Barry
>>
>> On Thursday 17 May 2012 11:46:26 you wrote:
>> > Hi,
>> >
>> > Now the filtering works correctly but the decoder still dies.
>> >
>> > The command I'm running now, is:
>> > nohup nice
>> > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
>> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.en
>> > /usr/local/share/moses/mosesdecoder/dist/bin/moses
>> > train/model/moses.ini --mertdir
>> > /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
>> > '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-
>> >in put.pl -Binarizer
>> > "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2"
>> > --Hierarchical' &> mert.out
>> >
>> > I'm attaching again the mert.out file although it seems to be similar
>> > to the previous one:
>> >
>> > nohup: ignoring input
>> > main::create_extractor_script() called too early to check prototype at
>> > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > line 681.
>> > Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts
>> > Use of uninitialized value $extractorargs in concatenation (.) or
>> > string at
>> > /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > line 370.
>> > filtering the phrase tables... Wed May 16 20:40:51 EEST 2012
>> > exec:
>> > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-i
>> >np ut.pl -Binarizer
>> > "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2"
>> > --Hierarchical ./filtered
>> >
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mo
>> >s es.ini /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
>> > Executing:
>> > /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-given-i
>> >np ut.pl -Binarizer
>> > "/usr/local/share/moses/mosesdecoder/dist/bin/CreateOnDisk 1 1 5 100 2"
>> > --Hierarchical ./filtered
>> >
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/mo
>> >s es.ini /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
>> > > filterphrases.out 2> filterphrases.err
>> > Asking moses for feature names and values from filtered/moses.ini
>> > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses -config
>> > filtered/moses.ini -inputtype 0 -show-weights > ./features.list
>> > Defined parameters (per moses.ini or switch):
>> > config: filtered/moses.ini
>> > cube-pruning-pop-limit: 1000
>> > input-factors: 0
>> > inputtype: 0
>> > lmodel-file: 8 0 3
>> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
>> > mapping: 0 T 0 1 T 1
>> > max-chart-span: 20 1000
>> > non-terminals: X
>> > search-algorithm: 3
>> > show-weights:
>> > ttable-file: 2 0 0 5
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt
>> >er ed/phrase-table.0-0.1.1.bin 6 0 0 1
>> >
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >u e-grammar ttable-limit: 20
>> > weight-l: 0.5000
>> > weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
>> > weight-w: -1
>> > Loading lexical distortion models...have 0 models
>> > Start loading LanguageModel
>> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
>> > seconds
>> > Finished loading LanguageModels : [0.000] seconds
>> > Using uniform ttable-limit of 20 for all translation tables.
>> > Start loading PhraseTable
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt
>> >er ed/phrase-table.0-0.1.1.bin : [0.000] seconds
>> > filePath:
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt
>> >er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >ue -grammar : [0.000] seconds
>> > filePath:
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >ue -grammar Finished loading phrase tables : [0.000] seconds
>> > Start loading phrase table from
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >ue -grammar : [0.000] seconds
>> > Start loading new format pt model : [0.000] seconds
>> > Finished loading phrase tables : [0.000] seconds
>> > MERT starting values and ranges for random generation:
>> > lm = 0.500 ( 0.00 .. 1.00)
>> > w = -1.000 ( 0.00 .. 1.00)
>> > tm = 0.200 ( 0.00 .. 1.00)
>> > tm = 0.200 ( 0.00 .. 1.00)
>> > tm = 0.200 ( 0.00 .. 1.00)
>> > tm = 0.200 ( 0.00 .. 1.00)
>> > tm = 0.200 ( 0.00 .. 1.00)
>> > tm = 1.000 ( 0.00 .. 1.00)
>> > run 1 start at Wed May 16 20:47:37 EEST 2012
>> > Parsing --decoder-flags: ||
>> > Saving new config to: ./run1.moses.ini
>> > Saved: ./run1.moses.ini
>> > Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000
>> > 0.200000 0.200000 1.000000
>> > DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143
>> > 0.057143 0.057143 0.285714
>> > Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses
>> > -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857 -tm
>> > 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 -n-best-list
>> > run1.best100.out 100 -input-file
>> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et >
>> > run1.out
>> > (1) run decoder to produce n-best lists
>> > params =
>> > decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
>> > 0.057143 0.057143 0.057143 0.285714
>> > Defined parameters (per moses.ini or switch):
>> > config: filtered/moses.ini
>> > cube-pruning-pop-limit: 1000
>> > input-factors: 0
>> > input-file:
>> > /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
>> > inputtype: 0
>> > lmodel-file: 8 0 3
>> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
>> > mapping: 0 T 0 1 T 1
>> > max-chart-span: 20 1000
>> > n-best-list: run1.best100.out 100
>> > non-terminals: X
>> > search-algorithm: 3
>> > ttable-file: 2 0 0 5
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt
>> >er ed/phrase-table.0-0.1.1.bin 6 0 0 1
>> >
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >u e-grammar ttable-limit: 20
>> > weight-l: 0.142857
>> > weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714
>> > weight-w: -0.285714
>> > Loading lexical distortion models...have 0 models
>> > Start loading LanguageModel
>> > /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en : [0.000]
>> > seconds
>> > Finished loading LanguageModels : [0.000] seconds
>> > Using uniform ttable-limit of 20 for all translation tables.
>> > Start loading PhraseTable
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt
>> >er ed/phrase-table.0-0.1.1.bin : [0.000] seconds
>> > filePath:
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/filt
>> >er ed/phrase-table.0-0.1.1.bin Start loading PhraseTable
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >ue -grammar : [0.000] seconds
>> > filePath:
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >ue -grammar Finished loading phrase tables : [0.000] seconds
>> > Start loading phrase table from
>> > /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model/gl
>> >ue -grammar : [0.000] seconds
>> > Start loading new format pt model : [0.000] seconds
>> > Finished loading phrase tables : [0.000] seconds
>> > Created input-output object : [0.000] seconds
>> > Translating line 0 in thread id 0
>> > ERROR:ERROR: search. Aborting
>> >
>> > Aborted
>> > Exit code: 134
>> > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
>> > 0.057143 0.057143 0.057143 0.057143 0.285714
>> >
>> > Any further ideas, what could be wrong?
>> > Kairit
>> >
>> > Barry Haddow <[email protected]> kirjutas:
>> > > Hi Kairit
>> > >
>> > > This is the problem:
>> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f
>> > >>il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such
>> > >> file or directory at
>> > >
>> > > Either CreateOnDIskPt does not exist, or you've provided an
>> > > incorrect path for
>> > > it.
>> > >
>> > > Not sure why the mert script didn't notice that filtering fails...
>> > >
>> > > cheers - Barry
>> > >
>> > > On Tuesday 15 May 2012 13:25:52 [email protected] wrote:
>> > >> Hi,
>> > >>
>> > >> The phrase tables have data and also the paths to the tuning sets are
>> > >> correct. However, filtering process seems to produce errors:
>> > >>
>> > >> The file mert-work/filterphrases.err has the content:
>> > >>
>> > >> Executing: mkdir -p
>> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f
>> > >>il ter ed Considering factor 0
>> > >> Done.
>> > >> filtering
>> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/model
>> > >>/r ule -table.gz ->
>> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f
>> > >>il te red/phrase-table.0-0.1.1... binarizing...CreateOnDiskPt 1 1 5 100
>> > >> 2
>> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f
>> > >>il ter ed/phrase-table.0-0.1.1
>> > >> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work/f
>> > >>il te red/phrase-table.0-0.1.1.bin Can't exec "CreateOnDiskPt": No such
>> > >> file or directory at
>> > >> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-give
>> > >>n- inp ut.pl line 278, <PIPE> line 3721058.
>> > >>
>> > >> Kairit
>> > >>
>> > >> Hieu Hoang <[email protected]> kirjutas:
>> > >>> hi Kairit
>> > >>>
>> > >>> The phrase table
>> > >>> ---/filtered/phrase-table.0-0.1.1.gz
>> > >>> seems to have loaded in 0 seconds.
>> > >>>
>> > >>> This is the filtered phrase table. Check that the original phrase
>> > >>> table has data, and that the tuning set has data.
>> > >>>
>> > >>> hieu
>> > >>>
>> > >>> (ps. please subscribe to the mailing list to post emails)
>> > >>>
>> > >>> On 09/05/2012 09:23, [email protected] wrote:
>> > >>>> Hi,
>> > >>>>
>> > >>>> I'm trying to tune a hierarchical model and the process is aborted
>> > >>>> with Exit code 134 and the message that the decoder died.
>> > >>>>
>> > >>>> The command I'm running is:
>> > >>>>
>> > >>>> nohup nice
>> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > >>>> ../../parallel/europarl-v6.et-en.dev.et
>> > >>>> ../../parallel/europarl-v6.et-en.dev.en
>> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses
>> > >>>> train/model/moses.ini --mertdir
>> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
>> > >>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g
>> > >>>>iv en -input.pl --Hierarchical'&>
>> > >>>> mert.out
>> > >>>>
>> > >>>> I have also tried this one and it doesn't work either:
>> > >>>>
>> > >>>> nohup nice
>> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > >>>> ../../parallel/europarl-v6.et-en.dev.et
>> > >>>> ../../parallel/europarl-v6.et-en.dev.en
>> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/moses
>> > >>>> train/model/moses.ini --mertdir
>> > >>>> /usr/local/share/moses/mosesdecoder/dist/bin/ --filtercmd
>> > >>>> '/usr/local/share/moses/mosesdecoder/scripts/training/filter-model-g
>> > >>>>iv en -input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2"
>> > >>>> --Hierarchical'&> mert.out
>> > >>>>
>> > >>>> The content of moses.ini file is:
>> > >>>>
>> > >>>> #########################
>> > >>>> ### MOSES CONFIG FILE ###
>> > >>>> #########################
>> > >>>>
>> > >>>> # input factors
>> > >>>> [input-factors]
>> > >>>>
>> > >>>>
>> > >>>> # mapping steps
>> > >>>> [mapping]
>> > >>>> 0 T 0
>> > >>>> 1 T 1
>> > >>>>
>> > >>>> # translation tables: table type (hierarchical(0), textual (0),
>> > >>>> binary (1)), source-factors, target-factors, number of scores, file
>> > >>>> # OLD FORMAT is still handled for back-compatibility
>> > >>>> # OLD FORMAT translation tables: source-factors, target-factors,
>> > >>>> number of scores, file
>> > >>>> # OLD FORMAT a binary table type (1) is assumed
>> > >>>> [ttable-file]
>> > >>>> 6 0 0 5
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /r ule-table.gz 6 0 0 1
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar
>> > >>>>
>> > >>>> # no generation models, no generation-file section
>> > >>>>
>> > >>>> # language models: type(srilm/irstlm), factors, order, file
>> > >>>> [lmodel-file]
>> > >>>> 8 0 3 /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
>> > >>>>
>> > >>>>
>> > >>>> # limit on how many phrase translations e for each phrase f are
>> > >>>> loaded # 0 = all elements loaded
>> > >>>> [ttable-limit]
>> > >>>> 20
>> > >>>>
>> > >>>>
>> > >>>> # language model weights
>> > >>>> [weight-l]
>> > >>>> 0.5000
>> > >>>>
>> > >>>>
>> > >>>> # translation model weights
>> > >>>> [weight-t]
>> > >>>> 0.20
>> > >>>> 0.20
>> > >>>> 0.20
>> > >>>> 0.20
>> > >>>> 0.20
>> > >>>> 1.0
>> > >>>>
>> > >>>> # no generation models, no weight-generation section
>> > >>>>
>> > >>>> # word penalty
>> > >>>> [weight-w]
>> > >>>> -1
>> > >>>>
>> > >>>> [cube-pruning-pop-limit]
>> > >>>> 1000
>> > >>>>
>> > >>>> [non-terminals]
>> > >>>> X
>> > >>>>
>> > >>>> [search-algorithm]
>> > >>>> 3
>> > >>>>
>> > >>>> [inputtype]
>> > >>>> 3
>> > >>>>
>> > >>>> [max-chart-span]
>> > >>>> 20
>> > >>>> 1000
>> > >>>>
>> > >>>> The content of mert.out file is:
>> > >>>>
>> > >>>> nohup: ignoring input
>> > >>>> main::create_extractor_script() called too early to check prototype
>> > >>>> at
>> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > >>>> line 681.
>> > >>>> Using SCRIPTS_ROOTDIR: /usr/local/share/moses/mosesdecoder/scripts
>> > >>>> Use of uninitialized value $extractorargs in concatenation (.) or
>> > >>>> string at
>> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/mert-moses.pl
>> > >>>> line 370.
>> > >>>> filtering the phrase tables... Tue May 8 18:02:26 EEST 2012
>> > >>>> exec:
>> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-gi
>> > >>>>ve n- input.pl --Hierarchical ./filtered
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /m oses.ini
>> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
>> > >>>> Executing:
>> > >>>> /usr/local/share/moses/mosesdecoder/scripts/training/filter-model-gi
>> > >>>>ve n- input.pl --Hierarchical ./filtered
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /m oses.ini
>> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et>
>> > >>>> filterphrases.out 2>
>> > >>>> filterphrases.err
>> > >>>> Asking moses for feature names and values from filtered/moses.ini
>> > >>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses
>> > >>>> -config filtered/moses.ini -inputtype 0 -show-weights>
>> > >>>> ./features.list Defined parameters (per moses.ini or switch):
>> > >>>> config: filtered/moses.ini
>> > >>>> cube-pruning-pop-limit: 1000
>> > >>>> input-factors: 0
>> > >>>> inputtype: 0
>> > >>>> lmodel-file: 8 0 3
>> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
>> > >>>> mapping: 0 T 0 1 T 1
>> > >>>> max-chart-span: 20 1000
>> > >>>> non-terminals: X
>> > >>>> search-algorithm: 3
>> > >>>> show-weights:
>> > >>>> ttable-file: 6 0 0 5
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz 6 0 0 1
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar ttable-limit: 20
>> > >>>> weight-l: 0.5000
>> > >>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
>> > >>>> weight-w: -1
>> > >>>> Loading lexical distortion models...have 0 models
>> > >>>> Start loading LanguageModel
>> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en :
>> > >>>> [0.000] seconds
>> > >>>> Finished loading LanguageModels : [0.000] seconds
>> > >>>> Using uniform ttable-limit of 20 for all translation tables.
>> > >>>> Start loading PhraseTable
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz : [0.000]
>> > >>>> seconds
>> > >>>> filePath:
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar : [0.000]
>> > >>>> seconds
>> > >>>> filePath:
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar Finished loading phrase tables : [0.000] seconds
>> > >>>> Start loading phrase table from
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz : [0.000]
>> > >>>> seconds
>> > >>>> Start loading new format pt model : [0.000] seconds
>> > >>>> Finished loading phrase tables : [72.000] seconds
>> > >>>> Start loading phrase table from
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar : [72.000]
>> > >>>> seconds
>> > >>>> Start loading new format pt model : [72.000] seconds
>> > >>>> Finished loading phrase tables : [72.000] seconds
>> > >>>> MERT starting values and ranges for random generation:
>> > >>>> lm = 0.500 ( 0.00 .. 1.00)
>> > >>>> w = -1.000 ( 0.00 .. 1.00)
>> > >>>> tm = 0.200 ( 0.00 .. 1.00)
>> > >>>> tm = 0.200 ( 0.00 .. 1.00)
>> > >>>> tm = 0.200 ( 0.00 .. 1.00)
>> > >>>> tm = 0.200 ( 0.00 .. 1.00)
>> > >>>> tm = 0.200 ( 0.00 .. 1.00)
>> > >>>> tm = 1.000 ( 0.00 .. 1.00)
>> > >>>> run 1 start at Tue May 8 18:05:59 EEST 2012
>> > >>>> Parsing --decoder-flags: ||
>> > >>>> Saving new config to: ./run1.moses.ini
>> > >>>> Saved: ./run1.moses.ini
>> > >>>> Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000
>> > >>>> 0.200000 0.200000 1.000000
>> > >>>> DECODER_CFG = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
>> > >>>> 0.057143 0.057143 0.057143 0.285714
>> > >>>> Executing: /usr/local/share/moses/mosesdecoder/dist/bin/moses
>> > >>>> -config filtered/moses.ini -inputtype 0 -w -0.285714 -lm 0.142857
>> > >>>> -tm 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714
>> > >>>> -n-best-list run1.best100.out 100 -input-file
>> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et>
>> > >>>> run1.out
>> > >>>> (1) run decoder to produce n-best lists
>> > >>>> params =
>> > >>>> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
>> > >>>> 0.057143 0.057143 0.057143 0.285714
>> > >>>> Defined parameters (per moses.ini or switch):
>> > >>>> config: filtered/moses.ini
>> > >>>> cube-pruning-pop-limit: 1000
>> > >>>> input-factors: 0
>> > >>>> input-file:
>> > >>>> /home/kairit/Documents/korpused/parallel/europarl-v6.et-en.dev.et
>> > >>>> inputtype: 0
>> > >>>> lmodel-file: 8 0 3
>> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en
>> > >>>> mapping: 0 T 0 1 T 1
>> > >>>> max-chart-span: 20 1000
>> > >>>> n-best-list: run1.best100.out 100
>> > >>>> non-terminals: X
>> > >>>> search-algorithm: 3
>> > >>>> ttable-file: 6 0 0 5
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz 6 0 0 1
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar ttable-limit: 20
>> > >>>> weight-l: 0.142857
>> > >>>> weight-t: 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714
>> > >>>> weight-w: -0.285714
>> > >>>> Loading lexical distortion models...have 0 models
>> > >>>> Start loading LanguageModel
>> > >>>> /home/kairit/Documents/korpused/lm/europarl-v6.et-en.blm.en :
>> > >>>> [0.000] seconds
>> > >>>> Finished loading LanguageModels : [0.000] seconds
>> > >>>> Using uniform ttable-limit of 20 for all translation tables.
>> > >>>> Start loading PhraseTable
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz : [0.000]
>> > >>>> seconds
>> > >>>> filePath:
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz Start loading PhraseTable
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar : [0.000]
>> > >>>> seconds
>> > >>>> filePath:
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar Finished loading phrase tables : [0.000] seconds
>> > >>>> Start loading phrase table from
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/mert-work
>> > >>>>/f il tered/phrase-table.0-0.1.1.gz : [0.000]
>> > >>>> seconds
>> > >>>> Start loading new format pt model : [0.000] seconds
>> > >>>> Finished loading phrase tables : [73.000] seconds
>> > >>>> Start loading phrase table from
>> > >>>> /home/kairit/Documents/korpused/working/hierarchical_et_en/train/mod
>> > >>>>el /g lue-grammar : [73.000]
>> > >>>> seconds
>> > >>>> Start loading new format pt model : [73.000] seconds
>> > >>>> Finished loading phrase tables : [73.000] seconds
>> > >>>> Created input-output object : [73.000] seconds
>> > >>>> Translating line 0 in thread id 0
>> > >>>> ERROR:ERROR: search. Aborting
>> > >>>>
>> > >>>> Aborted
>> > >>>> Exit code: 134
>> > >>>> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
>> > >>>> 0.057143 0.057143 0.057143 0.057143 0.285714
>> > >>>>
>> > >>>>
>> > >>>> Thanks if you can help me to find the problem.
>> > >>>>
>> > >>>> Kairit
>> > >>>>
>> > >>>>
>> > >>>> _______________________________________________
>> > >>>> Moses-support mailing list
>> > >>>> [email protected]
>> > >>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> > >>
>> > >> _______________________________________________
>> > >> Moses-support mailing list
>> > >> [email protected]
>> > >> http://mailman.mit.edu/mailman/listinfo/moses-support
>> > >
>> > > --
>> > > Barry Haddow
>> > > University of Edinburgh
>> > > +44 (0) 131 651 3173
>> > >
>> > > --
>> > > The University of Edinburgh is a charitable body, registered in
>> > > Scotland, with registration number SC005336.
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > [email protected]
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>> --
>> Barry Haddow
>> University of Edinburgh
>> +44 (0) 131 651 3173
>>
>
> --
> Barry Haddow
> University of Edinburgh
> +44 (0) 131 651 3173
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
