Hi, are you sorting the phrase table? Check the command as described on the Moses web site.
-phi On Wed, Jul 9, 2008 at 8:21 PM, Megan Elmore ([EMAIL PROTECTED]) <[EMAIL PROTECTED]> wrote: > Hello, > > Thanks very much for your quick reply. I am currently trying to generate a > binary phrase table but am getting an error: > > ERROR: xsource phrase already inserted (B)! > line(17): '000 - ||| 000 รข ||| (0) (1) ||| (0) (1) ||| 0.5 0.540651 0.25 > 0.178456 2.718' > f: 2 0 2 > > Does this indicate a problem with my phrase table or with the > processPhraseTable process? In the event that I need to run the training > process differently - what error or warning messages, if any, that are > generated during the training process would let me know of any errors in my > phrase table? > > Currently, the phrase table generated during the training process was left in > a gzip'ped format as phrase-table.0-0.gz - I am not sure if this is relevant, > but maybe the odd naming (as opposed to just "phrase-table" listed in the > online documentation) sheds light on a step of the training process that did > not complete normally for me? > > -Megan > > ----- Original Message ----- > From: Philipp Koehn <[EMAIL PROTECTED]> > Date: Wednesday, July 9, 2008 2:25 pm > Subject: Re: [Moses-support] phrase table memory issue > To: "Megan Elmore ([EMAIL PROTECTED])" <[EMAIL PROTECTED]> > Cc: [email protected] > >> Hi, >> >> this is a sign that the phrase table is too big to load into memory, >> there are three options: >> (a) use the binary phrase table >> (b) filter the phrase table for the test set you are using >> (c) both >> >> See the Moses web page for details. >> >> -phi >> >> On Wed, Jul 9, 2008 at 7:17 PM, Megan Elmore ([EMAIL PROTECTED]) >> <[EMAIL PROTECTED]> wrote: >> > Hello, >> > >> > I have installed Moses and run the training process using the >> europarl corpus but am now having problems with the decoder loading >> the phrase table. Like a previous message on this list, I am >> getting the error >> > >> > terminate called after throwing an instance of 'std::bad_alloc' >> > what(): St9bad_alloc >> > Aborted >> > >> > while the decoder is trying to load the phrase table, regardless >> of the machine I run the decoder on (I've tried four now). Is there >> a way I can optimize how much space the phrase table uses? Or is >> there something that could be going wrong in the training or >> decoding processes? I am not sure where to look for the error but >> with a little direction I could keep trying to debug it. >> > >> > Thanks, >> > -Megan E. >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > >> > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
