Well-spotted Hieu, that was my mistake! thanks a lot for your quick and very helpful help
Bruno ________________________________________ From: Hieu Hoang [[email protected]] Sent: Friday, February 18, 2011 1:49 PM To: Pouliquen, Bruno Cc: [email protected] Subject: Re: [Moses-support] Bug when using KenLM in decoding and the phrase table line in the ini file is: [ttable-file] 1 0 0 5 path/pt and NOT [ttable-file] 2 0 0 5 path/pt If it is, then I don't know what else is wrong On 18/02/2011 19:39, Pouliquen, Bruno wrote: > Hieu, > > I am using phrase-based decoding, > FYI: the binarizing process is : > zcat MYMODEL/phrase-table.0,1-0,1.gz | sort | processPhraseTable -ttable 0 0 > - -nscores 5 -out MYMODEL/phrase-table.0,1-0,1 > > (note that the decoder works fine with SRI lm) > > Bruno > ________________________________________ > From: [email protected] [[email protected]] On Behalf > Of Hieu Hoang [[email protected]] > Sent: Friday, February 18, 2011 1:26 PM > To: [email protected] > Subject: Re: [Moses-support] Bug when using KenLM in decoding > > are you using phrase-based or chart decoding? There's 2 versions of > binary/on-demand phrase-tables: > 1. The Binary phrase-table for phrase-based. > 2. The OnDisk is for chart decoding. > > it's a bit of a confusing situation, apologies > > On 18/02/2011 19:14, Pouliquen, Bruno wrote: >> I tried to install KenLM using the latest version of Moses (trunk Revision >> 3884), the decoder loads the LM without problems but then fails when >> decoding with following output: >> (the decoding is from French to English, I use 5-grams Language model, 2 >> factors, binarized phrase tables) >> >> ------------------------------------ >> Loading lexical distortion models...have 1 models >> Creating lexical reordering... >> weights: 0.300 0.300 0.300 0.300 0.300 0.300 >> binary file loaded, default OFF_T: -1 >> Start loading LanguageModel MYMODEL-kenlm.bin : [1.000] seconds >> Finished loading LanguageModels : [3.000] seconds >> weight-generation: 0.300 0.000 >> MYMODEL/generation.0,1-0.gz >> Start loading PhraseTable MYMODEL/phrase-table.0,1-0,1 : [21.000] seconds >> filePath: MYMODEL//phrase-table.0,1-0,1 >> Start loading PhraseTable MYMODEL//phrase-table.0-0 : [21.000] seconds >> filePath:MYMODEL/phrase-table.0-0 >> Finished loading phrase tables : [21.000] seconds >> IO from STDOUT/STDIN >> Created input-output object : [21.000] seconds >> Translating: test|U >> >> moses: OnDiskWrapper.cpp:59: bool OnDiskPt::OnDiskWrapper::OpenForLoad(const >> std::string&): Assertion `m_fileSource.is_open()' failed. >> Aborted >> ------------------------------------ >> the LM line in moses.ini is: >> 8 0 5 MYMODEL-kenlm.bin >> >> FYI: I also tried without binarizing the language model, using the standard >> ARPA format, it takes longer to load but the result is the same >> >> I have the impression that the binarized phrasetables may be the source of >> the error... >> >> Does anybody has any idea? Did I missed something? >> >> thanks in advance, >> Bruno Pouliquen >> >> >> >> World Intellectual Property Organization Disclaimer: >> >> This electronic message may contain privileged, confidential and >> copyright protected information. If you have received this e-mail >> by mistake, please immediately notify the sender and delete this >> e-mail and all its attachments. Please ensure all e-mail attachments >> are scanned for viruses prior to opening or using. >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > World Intellectual Property Organization Disclaimer: > > This electronic message may contain privileged, confidential and > copyright protected information. If you have received this e-mail > by mistake, please immediately notify the sender and delete this > e-mail and all its attachments. Please ensure all e-mail attachments > are scanned for viruses prior to opening or using. > World Intellectual Property Organization Disclaimer: This electronic message may contain privileged, confidential and copyright protected information. If you have received this e-mail by mistake, please immediately notify the sender and delete this e-mail and all its attachments. Please ensure all e-mail attachments are scanned for viruses prior to opening or using. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
