Hi, not at the moment. It probably makes sense to clear out the cache every 100 or 1000 sentences, but this is currently not implemented. Would be very east to do, though.
-phi On Wed, Apr 23, 2008 at 1:20 PM, David Talbot <[EMAIL PROTECTED]> wrote: > Hi Hieu, > > Is there any way to limit the size of the cache or to empty it at the > end of each sentence? > > Cheers, > David > > > > > Quoting Hieu Hoang <[EMAIL PROTECTED]>: > > > Hey holger > > > > Yep, the decoder caches all used phrases. Put > > [use-persistent-cache] > > 0 > > in your ini file or on commanad line to switch it off. > > > > The binary phrase table implementation also caches. I'm not sure how big of > > a problem that is, and I'm not sure if the cache can be turned off. > > > > -----Original Message----- > > From: Holger Schwenk [mailto:[EMAIL PROTECTED] > > Sent: 23 April 2008 00:12 > > To: Philipp Koehn; Hieu Hoang > > Cc: Holger Schwenk > > Subject: memory increase when translating very large corpora with binary > > phrase tables > > > > Hello Philipp and Hieu, > > > > I want to translate very large corpora of >10M words. Therefore I use > binary > > phrase tables since filtering would probably result in a very large table. > > > > When Moses starts it needs about 9G of RAM (I've a big LM) which seems > > reasonable to me. However, after several hours of processing (>1M > > words) memory has increased to 20GB. > > > > Is this normal ? Does Moses implement a cache of all previously requested > > phrases that continuously increases ? If yes, should I switch off > > "-use-persistent-cache" or split all the data into smallre pieces ? > > > > Thanks for your help > > > > Holger > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
