Hi Ken,
The phrase-table.0-0.1.1 is 0 bytes long..
I have commented out the abort() command and now it just skips over with
the same error:
sdumitrescu /usr/local/trans/work/roen2 > rm -r -f 1/mert
sdumitrescu /usr/local/trans/work/roen2 > ./mert_1t.sh
main::create_extractor_script() called too early to check prototype at
/usr/local/trans/tools/moses/scripts/training/mert-moses.pl line 674.
Using SCRIPTS_ROOTDIR: /usr/local/trans/tools/moses/scripts
filtering the phrase tables... Fri Jan 27 11:03:40 EET 2012
exec:
/usr/local/trans/tools/moses/scripts/training/filter-model-given-input.pl ./filtered
/usr/local/trans/work/roen2/1/model/moses.ini
/usr/local/trans/work/roen2/test.ro
Executing:
/usr/local/trans/tools/moses/scripts/training/filter-model-given-input.pl ./filtered
/usr/local/trans/work/roen2/1/model/moses.ini
/usr/local/trans/work/roen2/test.ro > filterphrases.out 2> filterphrases.err
Asking moses for feature names and values from filtered/moses.ini
Executing: /usr/local/trans/tools/moses/moses-cmd/src/moses -threads 16
-config filtered/moses.ini -inputtype 0 -show-weights > ./features.list
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
distortion-limit: 6
input-factors: 0
inputtype: 0
lmodel-file: 8 0 5
/usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin
mapping: 0 T 0
show-weights:
threads: 16
ttable-file: 0 0 0 5
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
ttable-limit: 20
weight-d: 0.6
weight-l: 0.5000
weight-t: 0.20 0.20 0.20 0.20 0.20
weight-w: -1
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin : [0.000] seconds
Finished loading LanguageModels : [0.000] seconds
Start loading PhraseTable
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 :
[0.000] seconds
filePath: /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
Finished loading phrase tables : [0.000] seconds
Start loading phrase table from
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 :
[0.000] seconds
Reading /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
could not close file
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 using zlib
Finished loading phrase tables : [0.000] seconds
MERT starting values and ranges for random generation:
d = 0.600 ( 0.00 .. 1.00)
lm = 0.500 ( 0.00 .. 1.00)
w = -1.000 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
tm = 0.200 ( 0.00 .. 1.00)
run 1 start at Fri Jan 27 11:03:49 EET 2012
Parsing --decoder-flags: |-threads 16|
Saving new config to: ./run1.moses.ini
Saved: ./run1.moses.ini
(1) run decoder to produce n-best lists
.. and continues to the second run when it stops normally.
I have created the test.ro/en files which contain just one sentence, for
testing purposes (instead of dev.ro/en).
Maybe the error is not gzip related, I'm guessing the
phrase-table.0-0.1.1 does not get written in the first place?
Stefan
p.s. the script file again:
sdumitrescu /usr/local/trans/work/roen2 > cat mert_1t.sh
#!/bin/sh
/usr/local/trans/tools/moses/scripts/training/mert-moses.pl \
/usr/local/trans/work/roen2/test.ro \
/usr/local/trans/work/roen2/test.en \
/usr/local/trans/tools/moses/moses-cmd/src/moses \
/usr/local/trans/work/roen2/1/model/moses.ini \
--mertdir /usr/local/trans/tools/moses/mert \
--working-dir /usr/local/trans/work/roen2/1/mert \
--rootdir /usr/local/trans/tools/moses/scripts \
--decoder-flags "-threads 16"
On 1/26/2012 4:55 PM, Kenneth Heafield wrote:
Hi,
That's my error message. Unfortunately, zlib doesn't provide a nice
way to extract a more verbose error from gzclose:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=354310
It's possible that the MERT filtering didn't work/complete properly,
leaving a corrupt gzipped file. Can you verify its integrity with
zcat /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
>/dev/null
for me? Also, based on the number of asterisks in your output, it looks
like an exception was thrown somewhere early in the file, processing
this exception led this destructor being called, and the true exception
was masked because the destructor won the race to kill the process. Try
commenting out abort() in util/file_piece.cc line 61. That will let the
process continue, possibly giving you the exception being thrown as a
more detailed error message.
Kenneth
On 01/26/2012 07:27 AM, Stefan Dumitrescu wrote:
Hello again,
I have a strange problem with MERT. I have trained several models and I
am attempting to tune them. While for the factored models the MERT
scripts run ok, for the unfactored model it fails with this error:
sdumitrescu /usr/local/trans/work/roen2> ./mert_1.sh
main::create_extractor_script() called too early to check prototype at
/usr/local/trans/tools/moses/scripts/training/mert-moses.pl line 674.
Using SCRIPTS_ROOTDIR: /usr/local/trans/tools/moses/scripts
filtering the phrase tables... Thu Jan 26 14:14:07 EET 2012
exec:
/usr/local/trans/tools/moses/scripts/training/filter-model-given-input.pl
./filtered
/usr/local/trans/work/roen2/1/model/moses.ini
/usr/local/trans/corpus/new/dev.0.ro
Executing:
/usr/local/trans/tools/moses/scripts/training/filter-model-given-input.pl
./filtered
/usr/local/trans/work/roen2/1/model/moses.ini
/usr/local/trans/corpus/new/dev.0.ro> filterphrases.out 2>
filterphrases.err
Asking moses for feature names and values from filtered/moses.ini
Executing: /usr/local/trans/tools/moses/moses-cmd/src/moses -threads 16
-config filtered/moses.ini -inputtype 0 -show-weights> ./features.list
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
distortion-limit: 6
input-factors: 0
inputtype: 0
lmodel-file: 8 0 5 /usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin
mapping: 0 T 0
show-weights:
threads: 16
ttable-file: 0 0 0 5
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
ttable-limit: 20
weight-d: 0.6
weight-l: 0.5000
weight-t: 0.20 0.20 0.20 0.20 0.20
weight-w: -1
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/usr/local/trans/corpus/lm/big/big.5g.surface.en.lm.bin : [0.000] seconds
Finished loading LanguageModels : [0.000] seconds
Start loading PhraseTable
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 :
[0.000] seconds
filePath: /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
Finished loading phrase tables : [0.000] seconds
Start loading phrase table from
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 :
[0.000] seconds
Reading /usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
*could not close file
/usr/local/trans/work/roen2/1/mert/filtered/phrase-table.0-0.1.1 using zlib*
Aborted
Exit code: 134
Failed to run moses with the config filtered/moses.ini at
/usr/local/trans/tools/moses/scripts/training/mert-moses.pl line 1072.
The script for model 1 (mert_1.sh) is:
/usr/local/trans/tools/moses/scripts/training/mert-moses.pl \
/usr/local/trans/corpus/new/dev.ro \
/usr/local/trans/corpus/new/dev.en \
/usr/local/trans/tools/moses/moses-cmd/src/moses \
/usr/local/trans/work/roen2/1/model/moses.ini \
--mertdir /usr/local/trans/tools/moses/mert \
--working-dir /usr/local/trans/work/roen2/1/mert \
--rootdir /usr/local/trans/tools/moses/scripts \
--decoder-flags "-threads 16"
and it is similar to the other scripts (instead of folder 1 i have
folder 2,3, etc) for which i get no error and tuning is working ok.
Also, all the models are ok, i have used them to translate without
problems.
Any idea why i get this error?
Thank you,
Stefan
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
