Hi, on 32-bit machines the actual maximum memory size for processes is 3 GB. To overcome this, make sure that you are using a binary translation table, binary reordering model, and a language model that takes up less than 3GB of RAM.
-phi On Sat, Nov 1, 2008 at 7:58 AM, jun li <[EMAIL PROTECTED]> wrote: > Hi, anybody. > > I think it maybe caused by not enought memory. > After I changed the mert-moses.perl to mert-moses-new.pl, and observed > the memory using by "top", > the message is displayed as follow: > moses: TargetPhrase.cpp:195: void > Moses::AddAlignmentElement(Moses::AlignmentPhraseInserter&, const > std::string&, size_t, size_t, std::list<unsigned int, > std::allocator<unsigned int> >&): Assertion alignPhraseVector.size() > == phraseSize' failed. > Aborted > Exit code: 134 > After I delete the line of Assertion. Then I got a bad allocation message: > terminate called after throwing an instance of 'St9bad_alloc' > what(): St9bad_alloc > Aborted > Exit code: 134 > > Although the memory used 75%, but the total used memory is about 3.9G. > ( other programm running at the same time). > So I believe the europarl corpus will need more than 4G mem to train and tune. > I will try irstlm later. > > Does anybody have any comment or suggestion? > Thanks very much. > > > On Sat, Nov 1, 2008 at 7:56 AM, jun li <[EMAIL PROTECTED]> wrote: >> Hi, >> >> I re-runed the scripts from the train model step. And it is Ok now. Thanks. >> But at tunning step.( run mert-moses.pl ) >> I still got a error message: >> ........ >> run 1 start at Sat Nov 1 07:28:54 CST 2008 >> Parsing --decoder-flags: || >> Saving new config to: ./run1.moses.ini >> Saved: ./run1.moses.ini >> ........ >> >> Creating lexical reordering... >> weights: 0.111 0.111 0.111 0.111 0.111 0.111 >> Loading table into memory...done. >> Created lexical orientation reordering >> Start loading LanguageModel >> /global/home/lj/moses/working-dir/lm/europarl.lm : [100.000] seconds >> Finished loading LanguageModels : [241.000] seconds >> Start loading PhraseTable >> /global/home/lj/moses/working-dir/tuning/filtered/phrase-table.0-0.1 : >> [241.000] seconds >> filePath: >> /global/home/lj/moses/working-dir/tuning/filtered/phrase-table.0-0.1 >> Unexpected error. >> Aborted >> Exit code: 134 >> The decoder died. CONFIG WAS -w 0.000000 -lm 0.111111 -d 0.111111 >> 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 -tm 0.033333 >> 0.022222 0.033333 0.022222 0.000000 >> >> My OS is ubuntu, and memory is 4G. the error just happened at memory >> used only 75% by moses. >> >> Thanks. >> >> >> On Fri, Oct 31, 2008 at 3:22 PM, Philipp Koehn <[EMAIL PROTECTED]> wrote: >>> Hi, >>> >>> the misplacement of the reordering model file is fixed >>> in the latest version of the scripts. Please update. >>> To fix the model you have just trained, move the reordering >>> model file into the model directory and fix the reference >>> in model/moses.ini so that it points to the file. >>> >>> -phi >>> >>> On Wed, Oct 29, 2008 at 2:43 AM, jun li <[EMAIL PROTECTED]> wrote: >>>> Hi, >>>> I followed the instructions on wmt07 ( >>>> http://www.statmt.org/wmt07/baseline.html ) and build a system. >>>> At the Train Model step, the script is as follow: >>>> • Run training script: >>>> bin/moses-scripts/scripts-YYYYMMDD-HHMM/training/train-factored-phrase-model.perl >>>> -scripts-root-dir bin/moses-scripts/scripts-YYYYMMDD-HHMM -root-dir >>>> working-dir -corpus working-dir/corpus/europarl.lowercased -f fr -e en >>>> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm >>>> 0:5:working-dir/lm/europarl.lm:0 >>>> >>>> After that I got a file named: >>>> reordering-table.msd-bidirectional-fe.0.5.gz at current directory ( >>>> where I run the script above) >>>> And the working-dir/model/moses.ini got a key-value as follow in the >>>> distortion (reordering) files section, >>>> [distortion-file] >>>> 0-0 msd-bidirectional-fe 6 >>>> /global/home/lj/moses/working-dir/model/msd-table.0-0.bi.fe.0.5.gz >>>> >>>> So the next step Tuning got a error message: >>>> Using SCRIPTS_ROOTDIR: bin/moses-scripts/scripts-YYYYMMDD-HHMM >>>> moses.ini:31:File does not exist or empty: >>>> /global/home/lj/moses/working-dir/model/msd-table.0-0.bi.fe.0.5.gz >>>> SYNC distortionchecking weight-count for ttable-file >>>> >>>> Do you have any suggestions? >>>> How should I change the parameters of train-factored-phrase-model.perl ? >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Best wishes, >>>> >>>> -- >>>> Li Jun >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>>> >>> >> >> >> >> -- >> Li Jun >> > > > > -- > Li Jun > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
