Hi Andrew In your moses.ini file I see the following line:
> 0.00820395 on dev /Users/ipadmin/Desktop/blcorpus/tuning.true.src This is extremely low, and suggests something went wrong in tuning ... although this should not cause a memory leak. I also noticed that you are decoding files of 700,000 sentences. The problem with this is that Moses will load the whole file into memory before it starts decoding. Again, this should not take up 21G of RAM, but it would be worth trying decoding a small set to see if you still have the memory problems, cheers - Barry On 29/10/13 06:10, Andrew Shin wrote: > I apologize since my previous question on the same topic didn't > specify enough. > > I'm following the steps in > http://www.statmt.org/moses/?n=Moses.Baseline exactly the same way, > and when I try to run Moses with the following command: > ~/mosesdecoder/bin/moses -f ~/working/mert-work/moses.ini (often with > -output-hypo-score option) > it starts eating up RAM until there's no space and starts taking hard > disk space as swap memory, > while it says its reading from phrase table. > It went up to 17GB of swap memory and pretty much stopped. > > I binarised the phrase table and reordering table, and the following > are the sizes of the binarised files > -rw-r--r-- 1 ipadmin staff 856K Oct 28 10:45 phrase-table.binphr.idx > -rw-r--r-- 1 ipadmin staff 140M Oct 28 10:45 > phrase-table.binphr.srctree.wa > -rw-r--r-- 1 ipadmin staff 1.6M Oct 28 10:45 > phrase-table.binphr.srcvoc > -rw-r--r-- 1 ipadmin staff 1.2G Oct 28 10:45 > phrase-table.binphr.tgtdata.wa > -rw-r--r-- 1 ipadmin staff 1.5M Oct 28 10:45 > phrase-table.binphr.tgtvoc > -rw-r--r-- 1 ipadmin staff 856K Oct 28 11:00 > reordering-table.binlexr.idx > -rw-r--r-- 1 ipadmin staff 1.3G Oct 28 11:00 > reordering-table.binlexr.srctree > -rw-r--r-- 1 ipadmin staff 916M Oct 28 11:00 > reordering-table.binlexr.tgtdata > -rw-r--r-- 1 ipadmin staff 1.6M Oct 28 11:00 > reordering-table.binlexr.voc0 > -rw-r--r-- 1 ipadmin staff 1.5M Oct 28 11:00 > reordering-table.binlexr.voc1 > > I also attached moses.ini which I tried to run. > I'm very sure that this is not the way it should be, but can't figure > out what is wrong since I followed the tutorial above..(and it's > worked before) Please let me know if there is any additional > information I should provide. > > I sincerely thank you very much for your time. > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
