Adding
-first-step 5 -last-step 5
will just run step 5 (phrase extraction)
On 02/11/16 12:01, Hasan Sait ARSLAN wrote:
For instance, could you show me an example?
Thanks,
2016-11-02 13:57 GMT+02:00 Barry Haddow <[email protected]
<mailto:[email protected]>>:
Hi Hasan
You can use the first-step and last-step arguments to run steps
manually.
cheers - Barry
On 02/11/16 11:46, Hasan Sait ARSLAN wrote:
Sorry,
as I see my model folder, I can say that Lexical translation part
also went well.
so, the only steps I have to run manually are'
* 5 Extract phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ExtractPhrases>
* 6 Score phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases>
* 7 Reordering model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildReorderingModel>
* 8 Generation model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildGenerationModel>
* 9 Configuration file
<http://www.statmt.org/moses/?n=FactoredTraining.CreateConfigurationFile>
2016-11-02 13:43 GMT+02:00 Hasan Sait ARSLAN
<[email protected] <mailto:[email protected]>>:
Hi Barry,
Actually I know where I it stuck.
As I check my train folder, I can see the all the files which
show that
* 1 Prepare data
<http://www.statmt.org/moses/?n=FactoredTraining.PrepareData>
* 2 Run GIZA
<http://www.statmt.org/moses/?n=FactoredTraining.RunGIZA>
* 3 Align words
<http://www.statmt.org/moses/?n=FactoredTraining.AlignWords>
went well.
So, is there any way to run the remaining
* 4 Lexical translation
<http://www.statmt.org/moses/?n=FactoredTraining.GetLexicalTranslationTable>
* 5 Extract phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ExtractPhrases>
* 6 Score phrases
<http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases>
* 7 Reordering model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildReorderingModel>
* 8 Generation model
<http://www.statmt.org/moses/?n=FactoredTraining.BuildGenerationModel>
* 9 Configuration file
<http://www.statmt.org/moses/?n=FactoredTraining.CreateConfigurationFile>
steps manually?
2016-11-02 13:27 GMT+02:00 Barry Haddow
<[email protected] <mailto:[email protected]>>:
Hi Hasan
It's not really possible to debug without a log file, so
I think that unfortunately you should start again.
I suggest you start with a much smaller corpus to try to
debug your setup. If you use 100,000 sentences it should
train within hours.
cheers - Barry
On 02/11/16 11:13, Hasan Sait ARSLAN wrote:
Hi Barry,
Unfortunately I didn't keep the log file. Is it really a
hopeless situation?
2016-11-02 13:10 GMT+02:00 Barry Haddow
<[email protected]
<mailto:[email protected]>>:
Hi Hasan
You should have run train_model.perl something like
this:
nohup nice /path/to/train_model.perl
ARGS_FOR_TRAINING &> log &
You just need the log file
cheers - Barry
On 02/11/16 11:00, Hasan Sait ARSLAN wrote:
Hi Barry,
I don't have any idea about where it is saved.
Cheers,
2016-11-02 12:55 GMT+02:00 Barry Haddow
<[email protected]
<mailto:[email protected]>>:
Hi Hasan
If your phrase table is empty, that would
explain why tuning didn't work. Something went
wrong earlier in the process. Could you post
your log file from train_model.perl,
cheers - Barry
On 02/11/16 10:45, Hasan Sait ARSLAN wrote:
Now currently I have a bigger problem. I have
trained my data for 5 days, and the folder
"train" is 39 G, but there is no any phrases
saved on phrase table. It is annoying. What
should I do now? I hope I won't need to rerun
everything from the scratch
2016-11-02 12:26 GMT+02:00 Barry Haddow
<[email protected]
<mailto:[email protected]>>:
Hi Hasan
The error message should be written into
filterphrases.err, inside your working
directory,
cheers - Barry
On 02/11/16 10:02, Hasan Sait ARSLAN wrote:
Hi Hieu,
I did in the way, you want. Plus, I am
sure the path and file names are
correctly spelt.
But still, I get the same error.
2016-11-01 21:41 GMT+02:00 Hasan Sait
ARSLAN <[email protected]
<mailto:[email protected]>>:
Thank you Hieu, I will do it, and
give you feedback about it.
Kind Regards,
2016-11-01 21:37 GMT+02:00 Hieu Hoang
<[email protected]
<mailto:[email protected]>>:
the command you execute looks ok.
the problem is likely to be with
your using tuning data.
You should look at your tuning
data, make sure you've spelt the
path and filename correctly, use
a very small subset (eg. start
with 1 parallel sentence) and
increase the number of sentences
until you find the problem
On 01/11/2016 12:30, Hasan Sait
ARSLAN wrote:
Hello,
I have trained dataset for 5
days, and for 2-3 days I am
dealing with the bug in tuning,
but still couldn't solve the
problem.
I use the following command to run:
*/home/sait/mosesdecoder/scripts/training/mert-moses.pl
<http://mert-moses.pl>
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr>
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.en
/home/sait/mosesdecoder/bin/moses
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
--working-dir
/home/sait/Kairit/Task3/working_tr-en/mert
--decoder-flags "--threads 32"
*
Then I get the following error:
*Using SCRIPTS_ROOTDIR:
/home/sait/mosesdecoder/scripts
Assuming
--mertdir=/home/sait/mosesdecoder/bin
filtering the phrase tables... T
nov 1 19:17:39 EET 2016
exec:
/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
./filtered
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr>
Executing:
/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
./filtered
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr> >
filterphrases.out 2>
filterphrases.err
Exit code: 255
ERROR: Failed to run
'/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
./filtered
/home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
/home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
<http://dev.clean.tr>'. at
/home/sait/mosesdecoder/scripts/training/mert-moses.pl
<http://mert-moses.pl> line 1748.
*
I have searched over the
internet about the error, but
unfortunately I couldn't find
any beneficial solution.
It is so annoying, and eats my
time, please help me to solve my
problem.
Thanks,
_______________________________________________
Moses-support mailing list
[email protected]
<mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
<http://mailman.mit.edu/mailman/listinfo/moses-support>
The University of Edinburgh is a
charitable body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable
body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body,
registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body,
registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
