How much memory and how many cores does your computer have? How many sentences are in your parallel corpus?
The training shouldn't be overwhelming your computer, it should take the same amount of memory & cpu whatever the data size or model. However, the hierarchical model takes about 10 time longer to train than the phrase-based, and the models are 10 times bigger. when tuning, imo, you need at least 4gb to run the hierarchical decoder, but much more memory is better. With phrase-based, you can easily do it with 2GB. if you're trying to create an online model, to run fast, you'll need a computer that has enough memory as the size of the model. This could be 100GB or 200GB or more for hiero models. Another way is to use suffix arrays, at the cost of slower overall decoding time. In summary, hiero models are a challenge to run. Phrase-based is easy. ps. if you're starting out with Moses, I recommend using the EMS. http://www.statmt.org/moses/?n=FactoredTraining.EMS you should take an example file and change it to your needs. The files are in the moses repository under scripts/ems/example The EMS saves you from have to learn all the arguments in every script. However, it calls those scripts so you can see exactly how to call them if you want. On 9 March 2013 13:30, Per Tunedal <[email protected]> wrote: > > Hi, > the training seems to be an overwhelming task for my computer. If it > ever succeeds, I will have to undertake the even more demanding task of > tuning. Can anything be done to accelerate it? > > Specifically, I wonder if it's feasible to prune the translation table > before doing the tuning. > > Yours, > Per Tunedal > > PS I've abandoned the idea of building a Hierarchical phrase model, I'm > now trying to make a phrase-based system. I suppose that would use less > resources. > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
