hi stefan 1. make sure you haven't run out of disk space 2. make sure every sentence in your test file ends with a new line
On Fri, Jan 27, 2012 at 4:11 PM, Stefan Dumitrescu < [email protected]> wrote: > Hi Ken, > > The phrase-table.0-0.1.1 is 0 bytes long.. > > I have commented out the abort() command and now it just skips over with > the same error: > > sdumitrescu /usr/local/trans/work/roen2 > rm -r -f 1/mert > sdumitrescu /usr/local/trans/work/roen2 > ./mert_1t.sh > > main::create_extractor_script() called too early to check prototype at > /usr/local/trans/tools/moses/scripts/training/mert-moses.pl line 674. > Using SCRIPTS_ROOTDIR: /usr/local/trans/tools/moses/scripts > filtering the phrase tables... Fri Jan 27 11:03:40 EET 2012 > exec: /usr/local/trans/tools/moses/scripts/training/ > filter-model-given-input.pl ./filtered > /usr/local/trans/work/roen2/1/model/moses.ini /usr/local/trans/work/roen2/ > test.ro > Executing: /usr/local/trans/tools/moses/scripts/training/ > filter-model-given-input.pl ./filtered > /usr/local/trans/work/roen2/1/model/moses.ini /usr/local/trans/work/roen2/ > test.ro > filterphrases.out 2> filterphrases.err > > Asking moses for feature names and values from filtered/moses.ini > Executing: /usr/local/trans/tools/moses/moses-cmd/src/moses -threads 16 > -config filtered/moses.ini -inputtype 0 -show-weights > ./features.list > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > distortion-limit: 6 > input-factors: 0 > inputtype: 0 > lmodel-file: 8 0 5 > /usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin > mapping: 0 T 0 > show-weights: > threads: 16 > ttable-file: 0 0 0 5 > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > ttable-limit: 20 > weight-d: 0.6 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 > weight-w: -1 > Loading lexical distortion models...have 0 models > Start loading LanguageModel > /usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin : [0.000] seconds > Finished loading LanguageModels : [0.000] seconds > Start loading PhraseTable > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 : [0.000] > seconds > filePath: /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > Finished loading phrase tables : [0.000] seconds > Start loading phrase table from > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 : [0.000] > seconds > Reading /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > could not close file > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 using zlib > Finished loading phrase tables : [0.000] seconds > MERT starting values and ranges for random generation: > d = 0.600 ( 0.00 .. 1.00) > lm = 0.500 ( 0.00 .. 1.00) > w = -1.000 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > tm = 0.200 ( 0.00 .. 1.00) > run 1 start at Fri Jan 27 11:03:49 EET 2012 > Parsing --decoder-flags: |-threads 16| > Saving new config to: ./run1.moses.ini > Saved: ./run1.moses.ini > (1) run decoder to produce n-best lists > > .. and continues to the second run when it stops normally. > > I have created the test.ro/en files which contain just one sentence, for > testing purposes (instead of dev.ro/en). > > Maybe the error is not gzip related, I'm guessing the phrase-table.0-0.1.1 > does not get written in the first place? > > Stefan > > p.s. the script file again: > sdumitrescu /usr/local/trans/work/roen2 > cat mert_1t.sh > #!/bin/sh > > /usr/local/trans/tools/moses/scripts/training/mert-moses.pl \ > /usr/local/trans/work/roen2/test.ro \ > /usr/local/trans/work/roen2/test.en \ > > /usr/local/trans/tools/moses/moses-cmd/src/moses \ > /usr/local/trans/work/roen2/1/model/moses.ini \ > --mertdir /usr/local/trans/tools/moses/mert \ > --working-dir /usr/local/trans/work/roen2/1/mert \ > --rootdir /usr/local/trans/tools/moses/scripts \ > --decoder-flags "-threads 16" > > > > > > > On 1/26/2012 4:55 PM, Kenneth Heafield wrote: > > Hi, > > That's my error message. Unfortunately, zlib doesn't provide a nice > way to extract a more verbose error from gzclose: > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=354310 > > It's possible that the MERT filtering didn't work/complete properly, > leaving a corrupt gzipped file. Can you verify its integrity with > > zcat /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > >/dev/null > > for me? Also, based on the number of asterisks in your output, it looks > like an exception was thrown somewhere early in the file, processing > this exception led this destructor being called, and the true exception > was masked because the destructor won the race to kill the process. Try > commenting out abort() in util/file_piece.cc line 61. That will let the > process continue, possibly giving you the exception being thrown as a > more detailed error message. > > Kenneth > > On 01/26/2012 07:27 AM, Stefan Dumitrescu wrote: > > Hello again, > > I have a strange problem with MERT. I have trained several models and I > am attempting to tune them. While for the factored models the MERT > scripts run ok, for the unfactored model it fails with this error: > > sdumitrescu /usr/local/trans/work/roen2 > ./mert_1.sh > main::create_extractor_script() called too early to check prototype at > /usr/local/trans/tools/moses/scripts/training/mert-moses.pl line 674. > Using SCRIPTS_ROOTDIR: /usr/local/trans/tools/moses/scripts > filtering the phrase tables... Thu Jan 26 14:14:07 EET 2012 > exec: > /usr/local/trans/tools/moses/scripts/training/filter-model-given-input.pl > ./filtered > /usr/local/trans/work/roen2/1/model/moses.ini > /usr/local/trans/corpus/new/dev.0.ro > Executing: > /usr/local/trans/tools/moses/scripts/training/filter-model-given-input.pl > ./filtered > /usr/local/trans/work/roen2/1/model/moses.ini > /usr/local/trans/corpus/new/dev.0.ro > filterphrases.out 2> > filterphrases.err > Asking moses for feature names and values from filtered/moses.ini > Executing: /usr/local/trans/tools/moses/moses-cmd/src/moses -threads 16 > -config filtered/moses.ini -inputtype 0 -show-weights > ./features.list > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > distortion-limit: 6 > input-factors: 0 > inputtype: 0 > lmodel-file: 8 0 5 /usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin > mapping: 0 T 0 > show-weights: > threads: 16 > ttable-file: 0 0 0 5 > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > ttable-limit: 20 > weight-d: 0.6 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 > weight-w: -1 > Loading lexical distortion models...have 0 models > Start loading LanguageModel > /usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin : [0.000] seconds > Finished loading LanguageModels : [0.000] seconds > Start loading PhraseTable > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 : > [0.000] seconds > filePath: /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > Finished loading phrase tables : [0.000] seconds > Start loading phrase table from > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 : > [0.000] seconds > Reading /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > *could not close file > /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 using zlib* > Aborted > Exit code: 134 > Failed to run moses with the config filtered/moses.ini at > /usr/local/trans/tools/moses/scripts/training/mert-moses.pl line 1072. > > The script for model 1 (mert_1.sh) is: > /usr/local/trans/tools/moses/scripts/training/mert-moses.pl \ > /usr/local/trans/corpus/new/dev.ro \ > /usr/local/trans/corpus/new/dev.en \ > /usr/local/trans/tools/moses/moses-cmd/src/moses \ > /usr/local/trans/work/roen2/1/model/moses.ini \ > --mertdir /usr/local/trans/tools/moses/mert \ > --working-dir /usr/local/trans/work/roen2/1/mert \ > --rootdir /usr/local/trans/tools/moses/scripts \ > --decoder-flags "-threads 16" > > and it is similar to the other scripts (instead of folder 1 i have > folder 2,3, etc) for which i get no error and tuning is working ok. > Also, all the models are ok, i have used them to translate without > problems. > > Any idea why i get this error? > > Thank you, > Stefan > > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
