Hi Per EMS now supports IRSTLM. In any case, you can always train the LM outside EMS.
Cheers - Barry Per Tunedal <[email protected]> wrote: >Hi, >I'm using IRSTML, so the EMS isn't an option, is it? >http://www.statmt.org/moses/?n=Moses.Baseline : >"The only wrinkle is that EMS currently (April 2012) does not support >training with IRSTLM, so either you'll have to install and use SRILM >(and refer to the main [EMS -> FactoredTraining.EMS]] documentation to >see how to set up training) or see above for instructions on how to >train the LM with IRSTLM (if you haven't already done so). " > >I've counted to 2 million sentences in the corpus. My computer has 8 GB >RAM and 4 cores. I'm happy I've reverted to building a phrase-based >model, instead of a hierarchical one. > >Yours, >Per Tunedal > >On Sat, Mar 9, 2013, at 15:53, Hieu Hoang wrote: >> How much memory and how many cores does your computer have? How many >> sentences are in your parallel corpus? >> >> The training shouldn't be overwhelming your computer, it should take the >> same amount of memory & cpu whatever the data size or model. However, the >> hierarchical model takes about 10 time longer to train than the >> phrase-based, and the models are 10 times bigger. >> >> when tuning, imo, you need at least 4gb to run the hierarchical decoder, >> but much more memory is better. With phrase-based, you can easily do it >> with 2GB. >> >> if you're trying to create an online model, to run fast, you'll need a >> computer that has enough memory as the size of the model. This could be >> 100GB or 200GB or more for hiero models. >> >> Another way is to use suffix arrays, at the cost of slower overall >> decoding >> time. >> >> In summary, hiero models are a challenge to run. Phrase-based is easy. >> >> >> >> ps. if you're starting out with Moses, I recommend using the EMS. >> http://www.statmt.org/moses/?n=FactoredTraining.EMS >> you should take an example file and change it to your needs. The files >> are >> in the moses repository under >> scripts/ems/example >> The EMS saves you from have to learn all the arguments in every script. >> However, it calls those scripts so you can see exactly how to call them >> if >> you want. >> >> >> >> On 9 March 2013 13:30, Per Tunedal <[email protected]> wrote: >> >> > >> > Hi, >> > the training seems to be an overwhelming task for my computer. If it >> > ever succeeds, I will have to undertake the even more demanding task of >> > tuning. Can anything be done to accelerate it? >> > >> > Specifically, I wonder if it's feasible to prune the translation table >> > before doing the tuning. >> > >> > Yours, >> > Per Tunedal >> > >> > PS I've abandoned the idea of building a Hierarchical phrase model, I'm >> > now trying to make a phrase-based system. I suppose that would use less >> > resources. >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> >> >> >> -- >> Hieu Hoang >> Research Associate >> University of Edinburgh >> http://www.hoang.co.uk >_______________________________________________ >Moses-support mailing list >[email protected] >http://mailman.mit.edu/mailman/listinfo/moses-support > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
