Hi Barry, I have rerun training starting from step 5, and I kept log file. I have checked log file, and it doesn't show any bug about creating phrase-table, but phrase-table is still not being created. I am attaching log file, could you please check it, and show me where is the problem?
Thanks, log.gz <https://drive.google.com/file/d/0BxvJK3H5ZKsnYzJiZmhjUWI0Qlk/view?usp=drive_web> 2016-11-02 14:28 GMT+02:00 Barry Haddow <[email protected]>: > Adding > -first-step 5 -last-step 5 > will just run step 5 (phrase extraction) > > > On 02/11/16 12:01, Hasan Sait ARSLAN wrote: > > For instance, could you show me an example? > > Thanks, > > 2016-11-02 13:57 GMT+02:00 Barry Haddow <[email protected]>: > >> Hi Hasan >> >> You can use the first-step and last-step arguments to run steps manually. >> >> cheers - Barry >> >> >> On 02/11/16 11:46, Hasan Sait ARSLAN wrote: >> >> Sorry, >> >> as I see my model folder, I can say that Lexical translation part also >> went well. >> >> so, the only steps I have to run manually are' >> >> >> - 5 Extract phrases >> <http://www.statmt.org/moses/?n=FactoredTraining.ExtractPhrases> >> - 6 Score phrases >> <http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases> >> - 7 Reordering model >> <http://www.statmt.org/moses/?n=FactoredTraining.BuildReorderingModel> >> - 8 Generation model >> <http://www.statmt.org/moses/?n=FactoredTraining.BuildGenerationModel> >> - 9 Configuration file >> <http://www.statmt.org/moses/?n=FactoredTraining.CreateConfigurationFile> >> >> >> >> >> >> 2016-11-02 13:43 GMT+02:00 Hasan Sait ARSLAN <[email protected] >> >: >> >>> Hi Barry, >>> >>> Actually I know where I it stuck. >>> >>> As I check my train folder, I can see the all the files which show that >>> >>> >>> - 1 Prepare data >>> <http://www.statmt.org/moses/?n=FactoredTraining.PrepareData> >>> - 2 Run GIZA >>> <http://www.statmt.org/moses/?n=FactoredTraining.RunGIZA> >>> - 3 Align words >>> <http://www.statmt.org/moses/?n=FactoredTraining.AlignWords> >>> >>> went well. >>> >>> So, is there any way to run the remaining >>> >>> >>> >>> - 4 Lexical translation >>> >>> <http://www.statmt.org/moses/?n=FactoredTraining.GetLexicalTranslationTable> >>> - 5 Extract phrases >>> <http://www.statmt.org/moses/?n=FactoredTraining.ExtractPhrases> >>> - 6 Score phrases >>> <http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases> >>> - 7 Reordering model >>> <http://www.statmt.org/moses/?n=FactoredTraining.BuildReorderingModel> >>> - 8 Generation model >>> <http://www.statmt.org/moses/?n=FactoredTraining.BuildGenerationModel> >>> - 9 Configuration file >>> <http://www.statmt.org/moses/?n=FactoredTraining.CreateConfigurationFile> >>> >>> steps manually? >>> >>> 2016-11-02 13:27 GMT+02:00 Barry Haddow <[email protected]>: >>> >>>> Hi Hasan >>>> >>>> It's not really possible to debug without a log file, so I think that >>>> unfortunately you should start again. >>>> >>>> I suggest you start with a much smaller corpus to try to debug your >>>> setup. If you use 100,000 sentences it should train within hours. >>>> >>>> cheers - Barry >>>> >>>> >>>> On 02/11/16 11:13, Hasan Sait ARSLAN wrote: >>>> >>>> Hi Barry, >>>> >>>> Unfortunately I didn't keep the log file. Is it really a hopeless >>>> situation? >>>> >>>> 2016-11-02 13:10 GMT+02:00 Barry Haddow <[email protected]>: >>>> >>>>> Hi Hasan >>>>> >>>>> You should have run train_model.perl something like this: >>>>> >>>>> nohup nice /path/to/train_model.perl ARGS_FOR_TRAINING &> log & >>>>> >>>>> You just need the log file >>>>> >>>>> cheers - Barry >>>>> >>>>> >>>>> On 02/11/16 11:00, Hasan Sait ARSLAN wrote: >>>>> >>>>> Hi Barry, >>>>> >>>>> I don't have any idea about where it is saved. >>>>> >>>>> Cheers, >>>>> >>>>> 2016-11-02 12:55 GMT+02:00 Barry Haddow <[email protected]>: >>>>> >>>>>> Hi Hasan >>>>>> >>>>>> If your phrase table is empty, that would explain why tuning didn't >>>>>> work. Something went wrong earlier in the process. Could you post your >>>>>> log >>>>>> file from train_model.perl, >>>>>> >>>>>> cheers - Barry >>>>>> >>>>>> >>>>>> On 02/11/16 10:45, Hasan Sait ARSLAN wrote: >>>>>> >>>>>> Now currently I have a bigger problem. I have trained my data for 5 >>>>>> days, and the folder "train" is 39 G, but there is no any phrases saved >>>>>> on >>>>>> phrase table. It is annoying. What should I do now? I hope I won't need >>>>>> to >>>>>> rerun everything from the scratch >>>>>> >>>>>> 2016-11-02 12:26 GMT+02:00 Barry Haddow <[email protected]>: >>>>>> >>>>>>> Hi Hasan >>>>>>> >>>>>>> The error message should be written into filterphrases.err, inside >>>>>>> your working directory, >>>>>>> >>>>>>> cheers - Barry >>>>>>> >>>>>>> >>>>>>> On 02/11/16 10:02, Hasan Sait ARSLAN wrote: >>>>>>> >>>>>>> Hi Hieu, >>>>>>> >>>>>>> I did in the way, you want. Plus, I am sure the path and file names >>>>>>> are correctly spelt. >>>>>>> But still, I get the same error. >>>>>>> >>>>>>> 2016-11-01 21:41 GMT+02:00 Hasan Sait ARSLAN < >>>>>>> [email protected]>: >>>>>>> >>>>>>>> Thank you Hieu, I will do it, and give you feedback about it. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> >>>>>>>> 2016-11-01 21:37 GMT+02:00 Hieu Hoang <[email protected]>: >>>>>>>> >>>>>>>>> the command you execute looks ok. the problem is likely to be with >>>>>>>>> your using tuning data. >>>>>>>>> >>>>>>>>> You should look at your tuning data, make sure you've spelt the >>>>>>>>> path and filename correctly, use a very small subset (eg. start with 1 >>>>>>>>> parallel sentence) and increase the number of sentences until you >>>>>>>>> find the >>>>>>>>> problem >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 01/11/2016 12:30, Hasan Sait ARSLAN wrote: >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I have trained dataset for 5 days, and for 2-3 days I am dealing >>>>>>>>> with the bug in tuning, but still couldn't solve the problem. >>>>>>>>> >>>>>>>>> I use the following command to run: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> */home/sait/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>>> <http://mert-moses.pl> >>>>>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr >>>>>>>>> <http://dev.clean.tr> >>>>>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.en >>>>>>>>> /home/sait/mosesdecoder/bin/moses >>>>>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini >>>>>>>>> --working-dir >>>>>>>>> /home/sait/Kairit/Task3/working_tr-en/mert --decoder-flags "--threads >>>>>>>>> 32" * >>>>>>>>> Then I get the following error: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Using SCRIPTS_ROOTDIR: /home/sait/mosesdecoder/scripts Assuming >>>>>>>>> --mertdir=/home/sait/mosesdecoder/bin filtering the phrase tables... T >>>>>>>>> nov 1 19:17:39 EET 2016 exec: >>>>>>>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl >>>>>>>>> <http://filter-model-given-input.pl> ./filtered >>>>>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini >>>>>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr >>>>>>>>> <http://dev.clean.tr> >>>>>>>>> Executing: >>>>>>>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl >>>>>>>>> <http://filter-model-given-input.pl> ./filtered >>>>>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini >>>>>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr >>>>>>>>> <http://dev.clean.tr> > >>>>>>>>> filterphrases.out 2> filterphrases.err Exit code: 255 ERROR: Failed >>>>>>>>> to run >>>>>>>>> '/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl >>>>>>>>> <http://filter-model-given-input.pl> ./filtered >>>>>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini >>>>>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr >>>>>>>>> <http://dev.clean.tr>'. >>>>>>>>> at /home/sait/mosesdecoder/scripts/training/mert-moses.pl >>>>>>>>> <http://mert-moses.pl> line 1748. * >>>>>>>>> I have searched over the internet about the error, but >>>>>>>>> unfortunately I couldn't find any beneficial solution. >>>>>>>>> >>>>>>>>> It is so annoying, and eats my time, please help me to solve my >>>>>>>>> problem. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Moses-support mailing >>>>>>>>> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Moses-support mailing >>>>>>> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>> >>>>>>> >>>>>>> >>>>>>> The University of Edinburgh is a charitable body, registered in >>>>>>> Scotland, with registration number SC005336. >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> The University of Edinburgh is a charitable body, registered in >>>>>> Scotland, with registration number SC005336. >>>>>> >>>>>> >>>>> >>>>> >>>>> The University of Edinburgh is a charitable body, registered in >>>>> Scotland, with registration number SC005336. >>>>> >>>>> >>>> >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> >>>> >>> >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
