Running the Moses binary doesn't use either of the perl scripts. I've had experiences where moses buffered the translated output and accumulated significant RAM, but never hard drive space.
What's the size of your nbest.txt output file? Is it possible that 25 the 25 n-best output sentences for each of 700,000 source sentences are consuming all your hard drive? I'm at a loss. Maybe someone else has experience with this. On 10/28/2013 07:38 PM, Andrew Shin wrote: > thank you for your answer. > > I've trained, tuned, and binarised the data and ran the moses with the > usual command > ~/mosesdecoder/bin/moses -f ~/working/mert-work/moses.ini > with the option -output-hypo-score -n-best-list nbest.txt 25. > Eating up the disk drive happens after that when it says reading from > phrase table. > My input files were about 700,000 lines each. > I followed the steps exactly from > http://www.statmt.org/moses/?n=Moses.Baseline > and it worked without a problem last time.. > > Could it be train-model.perl and mert-moses.pl are being used at that > stage? > If it's still unclear, is there an estimate of how much disk would be > needed for 700,000 lines of input? > > Thank you very much for your time in advance. > > > ------------------------------------------------------------------------ > Date: Mon, 28 Oct 2013 18:43:40 +0700 > From: [email protected] > To: [email protected] > Subject: Re: [Moses-support] Running moses fills up hard disk space.. > > Your description is a bit unclear. Are you talking about running the > Moses binary? Running Moses with binarized phrase table, reordering > table and language model (or hierarchical equivalent) -- and > configuring the moses.ini file to use them -- should not consume any > additional hard disk space. So, what exactly did you binarize and what > are you running? > > If you're talking about running train-model.perl or mert-moses.pl, > these two scripts can consume considerable hard drive space while they > do their work. It's not uncommon to consume 100-200 GB of hard drive > space if you have a large, complex corpus. > > > > On 10/28/2013 10:09 AM, Andrew Shin wrote: > > Hello, > > > I'm running Moses after binarising it, > and while it's reading from phrase table, > it rapidly fills up hard disk space until there is no more space.. > I had 21GB and it wasn't enough.. > Once I quit terminal, the space is freed again.. > I successfully ran Moses before, so I don't understand why this > happens.. > Is this natural? If so, how much reserve space do I need? > and If not, what could have gone wrong? > > Thank you very much for your help in advance. > > > _______________________________________________ > Moses-support mailing list > [email protected] <mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > _______________________________________________ Moses-support mailing > list [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
