Thanks. I found a possible problem with the way I built Moses with IRSTLM. So, I started from scratch and I'm rebuilding phrase tables and language models. Should be ready for further testing tomorrow. I'll pass my results when it's done. Stand by...
Tom On Tue, 1 Jun 2010 13:40:35 +0100, Barry Haddow <[email protected]> wrote: > Hi Tom > > I think 4G ram should be enough for the model you describe, so I don't > know > why moses is getting killled. How much memory does it use? Is moses using > the > binarised models? Note that there needs to be a 1 at the start of the > ttable > specification for this to happen. eg > [ttable-file] > 1 0 0 5 /afs/inf.ed.ac.uk/group/bhaddow/models/fr-en-nc/phrase-table.1 > > If you run with '-v 1' then you should be able to see which table is being > loaded when the memory exhaustion occurred. > > regards > Barry > > On Saturday 29 May 2010 19:23, [email protected] wrote: >> I'm troubleshooting a new moses system with these components: >> 1) GIZA++ (SVN rev 8, v 1.0.3) >> 2) IRSTLM (SVN rev 38, v 5.40.01) >> 3) Moses (SVN rev 3210, dated 4-26-2010) >> 4) Ubuntu-server 10.04 LTS 64-bit. >> 5) 3.4 Ghz Pentium-D with 4gb ram. >> >> Using a 3-gram lm, the system works as expected. Training, tuning and >> evaluation a small (135K pairs) en-nl subset of europarl.v5 work fine. >> BLEU >> score was 23. >> >> I then built a 5-gram model, edited the moses.ini config and started >> mert-moses-new. It creates a filtered model, and then launches moses. The >> memory usage grows and within 10 minutes, the system kills moses. >> >> In both cases, the lm is only the target half of the bitext corpus, about >> 135K lines. >> >> The moses.ini files: >> >> [lmodel-file] >> 1 0 3 /media/models/irstlm/europarl.v5.mini/3-gram.nl.blm >> >> [lmodel-file] >> 1 0 5 /media/models/irstlm/europarl.v5.mini/5-gram.nl.blm >> >> I know of one other who has anyone the same problem with the 4-1-2010 >> moses build and irstlm from March/April last year. >> >> Any suggestions? Could it be the new Ubuntu or the g++-4.4.1 compiler? >> >> Thanks, >> Tom >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
