Check your step 2 log for these errors and warnings:
WARNING: The following sentence pair has source/target sentence
length ration more than
ERROR: Forbidden zero sentence length 0
These should be fatal errors that terminate processing, but they are
not. Both of these log entries mean GIZA++ alignment files will be
empty, but they are non-zero length because of the tarball overhead. In
turn, these errors are not caught until step 6.
Best regards,
Tom Hoar
Chief Executive Officer
*/Precision Translation Tools Pte Ltd/*
Singapore/Thailand
Web: www.precisiontranslationtools.com
<http://www.precisiontranslationtools.com>
Thailand Mobile: +66 87 345-1875
Skype call: tahoar <skype:tahoar?call>
Skype chat: tahoar <skype:tahoar>
On 6/30/2016 8:09 AM, moses-support-requ...@mit.edu wrote:
Date: Thu, 30 Jun 2016 01:09:53 +0000
From: Denny Liu<junxian...@hotmail.com>
Subject: Re: [Moses-support] Step (6) score phrases failed in training
translation model -- error "not in gzip format"
To: Hieu Hoang<hieuho...@gmail.com>
Hi Hieu,
Thanks for your reply.
Actually I am following the Baseline System guideline to train a translation
system, here is the command was used to train the translation model.
nohup nice /scratch/MT/mosesdecoder/scripts/training/train-model.perl -root-dir train
-corpus /scratch/MT/corpus/fa_en_zh.clean -f en -e zh -alignment grow-diag-final-and
-reordering msd-bidirectional-fe -lm 0:3:/scratch/MT/lm/fa_en_zh.blm.zh:8
-external-bin-dir /scratch/MT/mosesdecoder/tools >& training.out &
However, after the above step is done, suppose a big size phrase-table.gz will be created
in the directory ~/working/train/model. But somehow it's failed in the middle. From the
training.out, seems it's stopping in the step6 "score phrases". That?s why I
tried the steps6 again to find out.
With your suggestion, I executed this command.
/scratch/MT/mosesdecoder/scripts/training/train-model.perl -mgiza -mgiza-cpus 8
-sort-buffer-size 8G -sort-compress gzip -cores 8 -parallel -dont-zip -first-step 6
-last-step 6 -external-bin-dir /scratch/MT/mosesdecoder/tools -f en -e zh -alignment
grow-diag-final-and -max-phrase-length 5 -reordering msd-bidirectional-fe -score-options
'--GoodTuring --MinScore 2:0.0001' -parallel -extract-file
/scratch/MT/working/train/model/extract -lexical-file /scratch/MT/working/train/model/lex
-phrase-translation-table /scratch/MT/working/train/model/phrase-table >&
training.out &
The following message is shown at the end in training.out log.
...Score v2.1 -- scoring methods for extracted rules
adjusting phrase translation probabilities with Good Turing discounting
Loading lexical translation table from /scratch/MT/working/train/model/lex.f2e.
.........................................................................
I think the next step should be running inverse mode, but this message is not
shown in the training.out log:
Score v2.1 -- scoring methods for extracted rules
using inverse mode
Loading lexical translation table from /scratch/MT/working/train/model/lex.e2f
................................................
Also the file phrase-table.gz is empty now in the directory
~/working/train/model.
What is the next step should be performed to create phrase-table? What is the command we
can use to complete the rest of (6) to (9) steps to complete "Training the
Translation System"?
(6) score phrases
(7) learn reordering model
(8) learn generation model
(9) create decoder config file
Thanks,
Denny
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
