Hi, Thanks for the response.
I tried this option too, if I run the command without '--text yes' option then the command runs fine, However I wanted to ask one thing does this give me an arpa file or a binarized one? Because when I run the next command mentioned in the manual: 6. /home/renu/Desktop/mosesdecoder/bin/build_binary news-commentary-v8.fr-en.arpa.en news-commentary-v8.fr-en.blm.en I get the following output: Reading news-commentary-v8.fr-en.arpa.en ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 **************************************************************************************************** lm/read_arpa.cc:63 in void lm::ReadARPACounts(util::FilePiece&, std::vector<long long unsigned int>&) threw FormatLoadException because `line.size() >= 4 && StringPiece(line.data(), 4) == "blmt"'. This looks like an IRSTLM binary file. Did you forget to pass --text yes to compile-lm? Byte: 40 File: news-commentary-v8.fr-en.arpa.en ERROR The last second line put in bold indicates that the one I am using is a binary file. Does that mean I already have a binary file and I do not need to use step 6 mentioned above (which infact is for converting from arpa to binary file) Thanks Renu On December 5, 2013 at 4:19 PM Hieu Hoang <[email protected]> wrote: > I'm not sure what is > --text yes > this is how the EMS runs IRSTLM compile-lm: > .../compile-lm .../europarl_pos.lm.4 .../europarl_pos.binlm.4 > > > > On 4 December 2013 15:58, renubalyan <[email protected] > <mailto:[email protected]> > wrote: > > > Hi, > > > > I am building the baseline system based on Moses manual instructions. > > > > I have installed Moses, GIZA++ and IRSTLM as mentioned in the manual. > > The corpus preparation (tokenization, ...cleaning) steps also goes well. > > > > However when I move to Language Model Training: I have some problems > > > > I am following these steps: > > > > 1. mkdir ~/lm > > > > 2. cd ~/lm > > > > 3. /home/renu/Desktop/irstlm/bin/add-start-end.sh < > > /home/renu/Desktop/corpus/news-commentary-v8.fr-en.true.en> > > news-commentary-v8.fr-en.sb.en > > > > 4. export IRSTLM=/home/renu/Desktop/irstlm; > > /home/renu/Desktop/irstlm/bin/build-lm.sh -i news-commentary-v8.fr-en.sb.en > > -t ./tmp -p -s improved-kneser-ney -o news-commentary-v8.fr-en.lm.en > > > > 5. /home/renu/Desktop/irstlm/bin/compile-lm --text yes > > news-commentary-v8.fr-en.lm.en.gz news-commentary-v8.fr-en.arpa.en > > > > Steps 1-4 work well but step 5 gives me -------(Warning:Too many > > parameters) > > > > I have searched the web for any possible solution but could not find any. > > > > I am not able to move ahead, kindly help. > > > > Thanks > > Renu > > > > > > > > ------------------------------------------------------------------------------------------------------------------------------- > > This e-mail is for the sole use of the intended recipient(s) and may > > contain confidential and privileged information. If you are not the > > intended recipient, please contact the sender by reply e-mail and destroy > > all copies and the original message. Any unauthorized review, use, > > disclosure, dissemination, forwarding, printing or copying of this email > > is strictly prohibited and appropriate legal action will be taken. > > > > > > ------------------------------------------------------------------------------------------------------------------------------- > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] <mailto:[email protected]> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > <http://mailman.mit.edu/mailman/listinfo/moses-support> > > > > > > -- > Hieu Hoang > Research Associate > University of Edinburgh > http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu> > > ------------------------------------------------------------------------------------------------------------------------------- This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. -------------------------------------------------------------------------------------------------------------------------------
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
