I am using an EMS that runs fine on (Linux) machine A but when I moved to Linux
machine B, I keep getting an EOF error at the extract- sort step, see the
STDERR below.
Machine A and machine B run the same identical versions of Linux Redhat and
Moses 1.0.
I notice in the /model directory, that aligned-grow-diag-final-and, lex.e2f
and lex.f2e are generated ok but the extract files(sorted,.o.sorted and
.inv.sorted) are all empty(zero bytes). Not sure what the prior step to
extract-sort is.
I am using a small test set (2,000 sentences) for de-en to get this working.
I have run the step manually on this machine and got the same error.
I suspect it may have a problem with sort or maybe extract, but am at a loss as
to what to do to fix the prob.
I am using this for training:
training-options = "-mgiza -mgiza-cpus 6 -sort-buffer-size 10G"
Any ideas what to do will be greatly appreciated.
Zai
----------STDERR below
[root@cf90server 1]# vi TRAINING_extract-phrases.1.STDERR
Executing: ln -s /apps/moses/de-en/training/corpus.1.en
/apps/moses/de-en/model/tmp.31546/target.00000
Executing: ln -s /apps/moses/de-en/training/corpus.1.de
/apps/moses/de-en/model/tmp.31546/source.00000
Executing: ln -s /apps/moses/de-en/model/aligned.1.grow-diag-final-and
/apps/moses/de-en/model/tmp.31546/align.00000
/apps/moses/mosesInstalls/mosesdecoder/scripts/../bin/extract
/apps/moses/de-en/model/tmp.31546/target.00000
/apps/moses/de-en/model/tmp.31546/source.00000
/apps/moses/de-en/model/tmp.31546/align.00000
/apps/moses/de-en/model/tmp.31546/extract.00000 7 orientation --model wbe-msd
--GZOutput --SentenceOffset 0 2>> /dev/stderr
zcat /apps/moses/de-en/model/tmp.31546/extract.00000.gz | LC_ALL=C sort -S 12G
-T /apps/moses/de-en/model/tmp.31546 | gzip -c >
/apps/moses/de-en/model/extract.1.sorted.gz
zcat /apps/moses/de-en/model/tmp.31546/extract.00000.inv.gz | LC_ALL=C sort -S
12G -T /apps/moses/de-en/model/tmp.31546 | gzip -c >
/apps/moses/de-en/model/extract.1.inv.sorted.gz
zcat /apps/moses/de-en/model/tmp.31546/extract.00000.o.gz | LC_ALL=C sort -S
12G -T /apps/moses/de-en/model/tmp.31546 | gzip -c >
/apps/moses/de-en/model/extract.1.o.sorted.gz
zcat: /apps/moses/de-en/model/tmp.31546/extract.00000.gz: unexpected end of file
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
