I also got the same error and resolved this out of memory type problem by binarising the phrase-table and lexicalised reordering models.
On Thu, Oct 4, 2012 at 2:34 PM, Hieu Hoang <[email protected]> wrote: > yep, you've run out of memory. > > you should binarize your phrase table, reordering model, and language model > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc3 > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc4 > > http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel#ntoc5 > > http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel#ntoc19 > > > On 04/10/2012 01:30, Ehsan wrote: > > Hi, > > When I try to run moses using: > mosesdecoder/bin/moses -f > /media/5C66CD4866CD239C/working/train/model/moses.ini > > after a few minutes I get this error: > .... > filePath: /media/5C66CD4866CD239C/working/train/model/phrase-table.gz > Finished loading phrase tables : [218.299] seconds > Start loading phrase table from > /media/5C66CD4866CD239C/working/train/model/phrase-table.gz : [218.299] > seconds > Reading /media/5C66CD4866CD239C/working/train/model/phrase-table.gz > > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > > **************************************************************************************************** > Exception: std::bad_alloc > > do you have any idea why this might occur? is there a possibility that > my laptop ran out of RAM memory > during "Training the translation System" step and didn't finish the > training? I didn't get any error messages, and > you can see the last few lines in my training.out file in the email right > below this one. > > I haven't done the mert (Tunining), so I am just using the moses.ini > befor tuning. > > A general question would be, is there a setup or version of Moses that > can be run and trained on > laptop resources (Core i7, 8GB RAM)? > > thanks, > Alef > ----- Forwarded Message ----- > *From:* Ehsan <[email protected]> <[email protected]> > *To:* > *Cc:* > *Sent:* Wednesday, October 3, 2012 11:16:02 AM > *Subject:* Re: [Moses-support] testing moses (weird behaviour) > > Hi, > > thanks for the reply. you were correct and my phrase-table.gz was empty! > So, I've redone all the steps again. This time I have the > phrase-table.gz of size 205.3 MB in my ".../working/train/model" > directory along side with moses.ini and other files. > > However, I still suspect that having my hardware resources > (comparing with your set up: 2 hours on Intel Xeon X5550, with approx > 120G Ram) > it should have taken a lot more than just 5 hours. > > So, is it possible for you to check the last few lines on my > "training.out" dump file ? > These are the last lines on training.out file: > > ... > (8) learn generation model @ Wed Oct 3 06:01:50 EDT 2012 > no generation model requested, skipping step > (9) create moses.ini @ Wed Oct 3 06:01:50 EDT 2012 > > I just want to make sure that "Training the translation System" was done > properly. > > it resulted in having 4 directories in ".../working" directory: > model size 640.8 MB > giza.fr-en size 193.7 MB > giza.en-fr size 193.4 MB > corpus size 54.1 MB > > thanks, > Alef > ------------------------------ > *From:* Philipp Koehn <[email protected]> <[email protected]> > *To:* Ehsan <[email protected]> <[email protected]> > *Cc:* ""[email protected]" <[email protected]>" > <[email protected]> <[email protected]> > *Sent:* Tuesday, October 2, 2012 7:22:40 PM > *Subject:* Re: [Moses-support] testing moses (weird behaviour) > > Hi, > > according to this output, none of the input words are recognized. > Make sure that your phrase translation table actually contains > French words at the source. > > -phi > > On Tue, Oct 2, 2012 at 5:28 PM, Ehsan <[email protected]> wrote: > > Hi, > > I am almost done with all the steps ( > http://www.statmt.org/moses/?n=Moses.Baseline#ems) > and doing the test using moses.ini (not the mert version): > > ....working]$ echo "faire revenir les militants sur le terrain et > convaincre que le vote est utile ." | > ~/MosesSMT/mosesdecoder/bin/moses -f model/moses.ini > > But the output is strength. As the best translation instead of getting > an English sentence I get the same French sentence! any idea why this > happens? I guess I am doing a silly mistake somewhere. > > here is the output dump for the above sentence: > > Defined parameters (per moses.ini or switch): > config: model/moses.ini > distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6 > .../MosesSMT/working/model/reordering-table.wbe-msd-bidirectional-fe.gz > distortion-limit: 6 > input-factors: 0 > lmodel-file: 8 0 3 .../MosesSMT/lm/news-commentary-v7.fr-en.blm.en > mapping: 0 T 0 > ttable-file: 0 0 0 5 .../MosesSMT/working/model/phrase-table.gz > ttable-limit: 20 > weight-d: 0.3 0.3 0.3 0.3 0.3 0.3 0.3 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 > weight-w: -1 > .../MosesSMT/mosesdecoder/bin > Loading lexical distortion models...have 1 models > Creating lexical reordering... > weights: 0.300 0.300 0.300 0.300 0.300 0.300 > Loading table into memory...done. > Start loading LanguageModel > .../MosesSMT/lm/news-commentary-v7.fr-en.blm.en : [0.001] seconds > Finished loading LanguageModels : [0.076] seconds > Start loading PhraseTable ..../MosesSMT/working/model/phrase-table.gz : > [0.076] seconds > filePath: ..../MosesSMT/working/model/phrase-table.gz > Finished loading phrase tables : [0.076] seconds > Start loading phrase table from > ..../MosesSMT/working/model/phrase-table.gz : [0.076] seconds > Reading ..../MosesSMT/working/model/phrase-table.gz > > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 > > **************************************************************************************************** > Finished loading phrase tables : [0.093] seconds > IO from STDOUT/STDIN > Created input-output object : [0.093] seconds > Translating line 0 in thread id 3048442736 > Translating: faire revenir les militants sur le terrain et convaincre que > le vote est utile . > > Line 0: Collecting options took 0.001 seconds > Line 0: Search took 0.013 seconds > faire revenir les militants sur le terrain et convaincre que le vote est > utile . > BEST TRANSLATION: faire|UNK|UNK|UNK revenir|UNK|UNK|UNK les|UNK|UNK|UNK > militants|UNK|UNK|UNK sur|UNK|UNK|UNK le|UNK|UNK|UNK terrain|UNK|UNK|UNK > et|UNK|UNK|UNK convaincre|UNK|UNK|UNK que|UNK|UNK|UNK le|UNK|UNK|UNK > vote|UNK|UNK|UNK est|UNK|UNK|UNK utile|UNK|UNK|UNK .|UNK|UNK|UNK > [111111111111111] [total=-1552.939] <<0.000, -15.000, -1500.000, 0.000, > 0.000, 0.000, 0.000, 0.000, 0.000, -135.878, 0.000, 0.000, 0.000, 0.000, > 0.000>> 0-0 > Line 0: Translation took 0.014 seconds total > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- Shahid Siddiq
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
