what is the exact command you executed? can you make the data files available for download so people can replicate the problem
Hieu Hoang http://www.hoang.co.uk/hieu On 8 August 2016 at 09:41, Jyoti Srivastava <[email protected]> wrote: > Hi > > At the time of training I am getting following error: > > Use of uninitialized value in concatenation (.) or string at > /home/abc/mosesdecoder/scripts/training/train-model.perl line 1089, > <IN_EN> line 1. > Unknown word 'University ' > Use of uninitialized value in concatenation (.) or string at > /home/abc/mosesdecoder/scripts/training/train-model.perl line 1089, > <IN_EN> line 1. > Unknown word 'नदी ' > Use of uninitialized value in concatenation (.) or string at > /home/abc/mosesdecoder/scripts/training/train-model.perl line 1089, > <IN_EN> line 2. > Unknown word 'River ' > Use of uninitialized value in concatenation (.) or string at > /home/abc/mosesdecoder/scripts/training/train-model.perl line 1089, > <IN_EN> line 2. > Unknown word '। > > It is not able to read the last word of each sentence. and giving it as > unknown word. and all the path that I have given in command are absolute > not relative. I have done with other corpus with the same command. it was > running but with this corpus I am getting this problem. > please please please help me. > > On Sun, Aug 7, 2016 at 7:52 PM, <[email protected]> wrote: > >> Send Moses-support mailing list submissions to >> [email protected] >> >> To subscribe or unsubscribe via the World Wide Web, visit >> http://mailman.mit.edu/mailman/listinfo/moses-support >> or, via email, send a message with subject or body 'help' to >> [email protected] >> >> You can reach the person managing the list at >> [email protected] >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of Moses-support digest..." >> >> >> Today's Topics: >> >> 1. Re: Compiling Moses with openmp (g++-6) on Mac OS 10.11? >> (Hieu Hoang) >> 2. Problem in Tuning (Jyoti Srivastava) >> 3. Re: Problem in Tuning (Hieu Hoang) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Sun, 7 Aug 2016 00:07:16 +0200 >> From: Hieu Hoang <[email protected]> >> Subject: Re: [Moses-support] Compiling Moses with openmp (g++-6) on >> Mac OS 10.11? >> To: James <[email protected]>, [email protected] >> Message-ID: <[email protected]> >> Content-Type: text/plain; charset="windows-1252" >> >> You're better of using Linux, in a virtual machine if you have to. >> >> I'm a Mac user myself but the reality is most open source MT software is >> developed on Linux and not really tested on other platforms >> >> >> On 03/08/2016 10:45, James wrote: >> > Hi, >> > >> > Cause we cannot use clang equipped with openmp on Mac OS 10.11, so >> > I've been working around how to compile Moses using G++ (g++-6) >> > instead of clang++, but so far no luck. >> > >> > What I want to do is to use some neural network language models like >> > nplm / OxLM with Moses, and these LMs require openmp which is not >> > officially supported by clang/clang++ on Mac OS 10.11. >> > >> > Here is my command and it didn't successfully compile the Moses: >> > >> > "bjam --with-boost=/Users/james/Desktop/Workspace/moses/boost_1_61_0 >> > --with-cmph=/Users/james/Desktop/Workspace/moses/cmph-2.0 >> > --with-xmlrpc-/Users/james/Desktop/Workspace/moses/xmlrpc-c-1.33.17 >> > -j12 --with-nplm=/Users/james/Desktop/Workspace/moses/nplm" >> > >> > And, my system specs are: >> > >> > Macbook Air 13inch, >> > Processor 1.7GHz Intel Core i7, >> > Memory 8GB DDR3, >> > Storage: 80GB free of 500GB. >> > >> > BTW, if there is alternative neural language model I can use for Moses >> > (without using openmp), please let me know. This would be really >> > appreciated... Thanks. >> > >> > Or, is there any way to use clang with openmp on Mac OS 10.11? >> > >> > -James >> > >> > >> > >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: http://mailman.mit.edu/mailman/private/moses-support/attachm >> ents/20160806/18d8ae57/attachment-0001.html >> >> ------------------------------ >> >> Message: 2 >> Date: Sun, 7 Aug 2016 19:07:24 +0530 >> From: Jyoti Srivastava <[email protected]> >> Subject: [Moses-support] Problem in Tuning >> To: [email protected] >> Message-ID: >> <CAFxbX9vJhji0oBAfEYj+Q+oDi5Pu6tt5J_TXD=gBR7-kWiTnuQ@mail. >> gmail.com> >> Content-Type: text/plain; charset="utf-8" >> >> Hi >> >> I installed Moses on Ubuntu and run the baseline system successfully with >> the English-French corpus given on the website. but now when I am running >> it on English-Hindi corpus of ACL 2014 hen At the time of tuning it is >> giving following error >> >> nohup: ignoring input >> Using SCRIPTS_ROOTDIR: /home/abc/mosesdecoder/scripts >> filtering the phrase tables... Sun Aug 7 18:52:36 IST 2016 >> exec: /home/abc/mosesdecoder/scripts/training/filter-model-given-input.pl >> ./filtered /home/abc/ACL_working/train/model/moses.ini >> /home/abc/ACL_corpus/newsdev2014.true.en >> Executing: /home/abc/mosesdecoder/scripts/training/ >> filter-model-given-input.pl ./filtered >> /home/abc/ACL_working/train/model/moses.ini >> /home/abc/ACL_corpus/newsdev2014.true.en > filterphrases.out 2> >> filterphrases.err >> Asking moses for feature names and values from filtered/moses.ini >> Executing: /home/abc/mosesdecoder/bin/moses -config filtered/moses.ini >> -show-weights >> exec: /home/abc/mosesdecoder/bin/moses -config filtered/moses.ini >> -show-weights >> Executing: /home/abc/mosesdecoder/bin/moses -config filtered/moses.ini >> -show-weights > ./features.list 2> /dev/null >> Exit code: 1 >> ERROR: Failed to run '/home/abc/mosesdecoder/bin/moses -config >> filtered/moses.ini -show-weights'. at >> /home/abc/mosesdecoder/scripts/training/mert-moses.pl line 1748. >> >> Even I ran the same command as before in English-French corpus. I ran it >> successfully on other system too with same English-Hindi corpus. but this >> time I tried it many times but every time it is giving same error. >> Please help me. >> >> On Fri, Aug 5, 2016 at 9:31 PM, <[email protected]> wrote: >> >> > Send Moses-support mailing list submissions to >> > [email protected] >> > >> > To subscribe or unsubscribe via the World Wide Web, visit >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > or, via email, send a message with subject or body 'help' to >> > [email protected] >> > >> > You can reach the person managing the list at >> > [email protected] >> > >> > When replying, please edit your Subject line so it is more specific >> > than "Re: Contents of Moses-support digest..." >> > >> > >> > Today's Topics: >> > >> > 1. Using Moses on Ubuntu (rcm) >> > 2. Crushed tuning neural model (Tomasz Gawryl) >> > >> > >> > ---------------------------------------------------------------------- >> > >> > Message: 1 >> > Date: Fri, 5 Aug 2016 08:59:21 +0200 >> > From: rcm <[email protected]> >> > Subject: [Moses-support] Using Moses on Ubuntu >> > To: [email protected] >> > Message-ID: <[email protected]> >> > Content-Type: text/plain; charset=utf-8; format=flowed >> > >> > >> > Hello! >> > >> > Finally, I will compile Moses on Ubuntu successfully. >> > >> > Then, for the first time, I download the sample model, run the decoder >> > and everything was ok, the German text was translated to English text: >> > "das ist ein kleines haus" --> "this is a small house" >> > >> > I analysed the files in the sample model, and there was: >> > - moses.ini: configuration file >> > - nbest.txt: (setting in moses.ini) file with a lot of variations over >> > the ONLY input phrase and values for: "d", "lm", "w", "tm" and another >> > value >> > - phrase-table: (setting in moses.ini) file with a translation table >> > over the ONLY words in the input phrase to the English language and a >> > statistical value >> > - trans-detail: translation hypothesis details...(I dont understand what >> > is that!) >> > >> > But, what must I do if I need translate another phrases?? >> > >> > By another side, in www.statmt.org/moses/ web site, in the Online Demos >> > part, there are a link for (for example): >> > English -> {German, Spanish, French} >> > I can type any sentence, and its translation (to German, for example) is >> > displayed. >> > Is running in the background Moses as Service ? >> > >> > Finally, my goal is translate any word wrote in a Office document using >> > Moses, in simular way that Microsoft Translator or WordLingo or another >> > systems are using nowadays. >> > Can I communicate in some way with a Moses as Service (like moses web >> > site above) or must I run my owner Moses as Service to translate any >> > word wrote in a Office document?? >> > Is possible add Moses like Online Machine Translation in the Translation >> > Options panel in Word Office, like Microsoft Translator and WordLingo in >> > Office 2010 (for example) ?? >> > >> > Thanks in advance. >> > >> > >> > ------------------------------ >> > >> > Message: 2 >> > Date: Fri, 5 Aug 2016 10:35:06 +0200 >> > From: "Tomasz Gawryl" <[email protected]> >> > Subject: [Moses-support] Crushed tuning neural model >> > To: <[email protected]> >> > Message-ID: <[email protected]> >> > Content-Type: text/plain; charset="iso-8859-2" >> > >> > Hi, >> > >> > I have strange problem while tuning neural model. Tuning crashes with >> > following error: >> > >> > >> > >> > LOG: >> > >> > >> > >> > EXECUTE STEPS >> > >> > number of steps doable or running: 1 at pi?, 5 sie 2016, 09:50:58 CEST >> > >> > doable: TUNING:tune >> > >> > executing >> > /home/moses/working/experiments/ENPL33M6NRN/steps/7/TUNING_tune.7 via >> sh >> > (1 >> > active) >> > >> > step TUNING:tune crashed >> > >> > >> > >> > >> > >> > TUNING_tune.7.STDERR: >> > >> > >> > >> > Not a label, not a score 'nan'. Failed to parse the scores string: >> > 'NPLMNEURAL= nan LexicalReordering0= 0 0 0 0 0 0 Distortion0= -8 LM0= >> > -67.8925 WordPenalty0= -5 PhrasePenalty0= 5 TranslationModel0= 0 0 0 0' >> of >> > nbestlist 'run1.best100.out' at >> > /home/moses/src/mosesdecoder/scripts/training/mert-moses.pl line 1503. >> > >> > >> > >> > Run.best100.out: >> > >> > >> > >> > moses@smtserver:~/working/experiments/ENPL33M6NRN/tuning/tmp.7$ head >> > run1.best100.out >> > >> > 0 ||| let . something 's try ||| NPLMNEURAL= nan >> LexicalReordering0= >> > 0 >> > 0 0 0 0 0 Distortion0= -8 LM0= -67.8925 WordPenalty0= -5 >> PhrasePenalty0= 5 >> > TranslationModel0= 0 0 0 0 ||| nan >> > >> > 0 ||| try let . 's something ||| NPLMNEURAL= nan >> LexicalReordering0= >> > 0 >> > 0 0 0 0 0 Distortion0= -13 LM0= -68.7339 WordPenalty0= -5 >> PhrasePenalty0= 5 >> > TranslationModel0= 0 0 0 0 ||| nan >> > >> > 0 ||| something 's . try let ||| NPLMNEURAL= nan >> LexicalReordering0= >> > 0 >> > 0 0 0 0 0 Distortion0= -14 LM0= -70.253 WordPenalty0= -5 >> PhrasePenalty0= 5 >> > TranslationModel0= 0 0 0 0 ||| nan >> > >> > >> > >> > >> > >> > What is the reason of appearing these "nan" values? I've successfully >> > trained smaller (1000 sentences) corpus using the same config file. >> > >> > >> > >> > Please help! :) >> > >> > >> > >> > Regards, >> > >> > Thomas >> > >> > -------------- next part -------------- >> > An HTML attachment was scrubbed... >> > URL: http://mailman.mit.edu/mailman/private/moses-support/ >> > attachments/20160805/933d8db9/attachment-0001.html >> > >> > ------------------------------ >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > >> > End of Moses-support Digest, Vol 118, Issue 13 >> > ********************************************** >> > >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: http://mailman.mit.edu/mailman/private/moses-support/attachm >> ents/20160807/0a1e14c4/attachment-0001.html >> >> ------------------------------ >> >> Message: 3 >> Date: Sun, 7 Aug 2016 16:22:10 +0200 >> From: Hieu Hoang <[email protected]> >> Subject: Re: [Moses-support] Problem in Tuning >> To: Jyoti Srivastava <[email protected]> >> Cc: moses-support <[email protected]> >> Message-ID: >> <CAEKMkbjiMbt___2r1mK5H4MxgTk6L1G19333RCxbj2Yc=1H5Ow@mail. >> gmail.com> >> Content-Type: text/plain; charset="utf-8" >> >> make sure all your paths are absolute, not relative >> >> Hieu Hoang >> http://www.hoang.co.uk/hieu >> >> On 7 August 2016 at 15:37, Jyoti Srivastava <[email protected]> wrote: >> >> > Hi >> > >> > I installed Moses on Ubuntu and run the baseline system successfully >> with >> > the English-French corpus given on the website. but now when I am >> running >> > it on English-Hindi corpus of ACL 2014 hen At the time of tuning it is >> > giving following error >> > >> > nohup: ignoring input >> > Using SCRIPTS_ROOTDIR: /home/abc/mosesdecoder/scripts >> > filtering the phrase tables... Sun Aug 7 18:52:36 IST 2016 >> > exec: /home/abc/mosesdecoder/scripts/training/filter-model-given- >> input.pl >> > ./filtered /home/abc/ACL_working/train/model/moses.ini >> > /home/abc/ACL_corpus/newsdev2014.true.en >> > Executing: /home/abc/mosesdecoder/scripts/training/filter-model- >> > given-input.pl ./filtered /home/abc/ACL_working/train/model/moses.ini >> > /home/abc/ACL_corpus/newsdev2014.true.en > filterphrases.out 2> >> > filterphrases.err >> > Asking moses for feature names and values from filtered/moses.ini >> > Executing: /home/abc/mosesdecoder/bin/moses -config filtered/moses.ini >> > -show-weights >> > exec: /home/abc/mosesdecoder/bin/moses -config filtered/moses.ini >> > -show-weights >> > Executing: /home/abc/mosesdecoder/bin/moses -config filtered/moses.ini >> > -show-weights > ./features.list 2> /dev/null >> > Exit code: 1 >> > ERROR: Failed to run '/home/abc/mosesdecoder/bin/moses -config >> > filtered/moses.ini -show-weights'. at /home/abc/mosesdecoder/ >> > scripts/training/mert-moses.pl line 1748. >> > >> > Even I ran the same command as before in English-French corpus. I ran it >> > successfully on other system too with same English-Hindi corpus. but >> this >> > time I tried it many times but every time it is giving same error. >> > Please help me. >> > >> > On Fri, Aug 5, 2016 at 9:31 PM, <[email protected]> wrote: >> > >> >> Send Moses-support mailing list submissions to >> >> [email protected] >> >> >> >> To subscribe or unsubscribe via the World Wide Web, visit >> >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> or, via email, send a message with subject or body 'help' to >> >> [email protected] >> >> >> >> You can reach the person managing the list at >> >> [email protected] >> >> >> >> When replying, please edit your Subject line so it is more specific >> >> than "Re: Contents of Moses-support digest..." >> >> >> >> >> >> Today's Topics: >> >> >> >> 1. Using Moses on Ubuntu (rcm) >> >> 2. Crushed tuning neural model (Tomasz Gawryl) >> >> >> >> >> >> ---------------------------------------------------------------------- >> >> >> >> Message: 1 >> >> Date: Fri, 5 Aug 2016 08:59:21 +0200 >> >> From: rcm <[email protected]> >> >> Subject: [Moses-support] Using Moses on Ubuntu >> >> To: [email protected] >> >> Message-ID: <[email protected]> >> >> Content-Type: text/plain; charset=utf-8; format=flowed >> >> >> >> >> >> Hello! >> >> >> >> Finally, I will compile Moses on Ubuntu successfully. >> >> >> >> Then, for the first time, I download the sample model, run the decoder >> >> and everything was ok, the German text was translated to English text: >> >> "das ist ein kleines haus" --> "this is a small house" >> >> >> >> I analysed the files in the sample model, and there was: >> >> - moses.ini: configuration file >> >> - nbest.txt: (setting in moses.ini) file with a lot of variations over >> >> the ONLY input phrase and values for: "d", "lm", "w", "tm" and another >> >> value >> >> - phrase-table: (setting in moses.ini) file with a translation table >> >> over the ONLY words in the input phrase to the English language and a >> >> statistical value >> >> - trans-detail: translation hypothesis details...(I dont understand >> what >> >> is that!) >> >> >> >> But, what must I do if I need translate another phrases?? >> >> >> >> By another side, in www.statmt.org/moses/ web site, in the Online >> Demos >> >> part, there are a link for (for example): >> >> English -> {German, Spanish, French} >> >> I can type any sentence, and its translation (to German, for example) >> is >> >> displayed. >> >> Is running in the background Moses as Service ? >> >> >> >> Finally, my goal is translate any word wrote in a Office document using >> >> Moses, in simular way that Microsoft Translator or WordLingo or another >> >> systems are using nowadays. >> >> Can I communicate in some way with a Moses as Service (like moses web >> >> site above) or must I run my owner Moses as Service to translate any >> >> word wrote in a Office document?? >> >> Is possible add Moses like Online Machine Translation in the >> Translation >> >> Options panel in Word Office, like Microsoft Translator and WordLingo >> in >> >> Office 2010 (for example) ?? >> >> >> >> Thanks in advance. >> >> >> >> >> >> ------------------------------ >> >> >> >> Message: 2 >> >> Date: Fri, 5 Aug 2016 10:35:06 +0200 >> >> From: "Tomasz Gawryl" <[email protected]> >> >> Subject: [Moses-support] Crushed tuning neural model >> >> To: <[email protected]> >> >> Message-ID: <[email protected]> >> >> Content-Type: text/plain; charset="iso-8859-2" >> >> >> >> Hi, >> >> >> >> I have strange problem while tuning neural model. Tuning crashes with >> >> following error: >> >> >> >> >> >> >> >> LOG: >> >> >> >> >> >> >> >> EXECUTE STEPS >> >> >> >> number of steps doable or running: 1 at pi?, 5 sie 2016, 09:50:58 CEST >> >> >> >> doable: TUNING:tune >> >> >> >> executing >> >> /home/moses/working/experiments/ENPL33M6NRN/steps/7/TUNING_tune.7 via >> sh >> >> (1 >> >> active) >> >> >> >> step TUNING:tune crashed >> >> >> >> >> >> >> >> >> >> >> >> TUNING_tune.7.STDERR: >> >> >> >> >> >> >> >> Not a label, not a score 'nan'. Failed to parse the scores string: >> >> 'NPLMNEURAL= nan LexicalReordering0= 0 0 0 0 0 0 Distortion0= -8 LM0= >> >> -67.8925 WordPenalty0= -5 PhrasePenalty0= 5 TranslationModel0= 0 0 0 >> 0' of >> >> nbestlist 'run1.best100.out' at >> >> /home/moses/src/mosesdecoder/scripts/training/mert-moses.pl line 1503. >> >> >> >> >> >> >> >> Run.best100.out: >> >> >> >> >> >> >> >> moses@smtserver:~/working/experiments/ENPL33M6NRN/tuning/tmp.7$ head >> >> run1.best100.out >> >> >> >> 0 ||| let . something 's try ||| NPLMNEURAL= nan >> >> LexicalReordering0= 0 >> >> 0 0 0 0 0 Distortion0= -8 LM0= -67.8925 WordPenalty0= -5 >> PhrasePenalty0= 5 >> >> TranslationModel0= 0 0 0 0 ||| nan >> >> >> >> 0 ||| try let . 's something ||| NPLMNEURAL= nan >> >> LexicalReordering0= 0 >> >> 0 0 0 0 0 Distortion0= -13 LM0= -68.7339 WordPenalty0= -5 >> PhrasePenalty0= >> >> 5 >> >> TranslationModel0= 0 0 0 0 ||| nan >> >> >> >> 0 ||| something 's . try let ||| NPLMNEURAL= nan >> >> LexicalReordering0= 0 >> >> 0 0 0 0 0 Distortion0= -14 LM0= -70.253 WordPenalty0= -5 >> PhrasePenalty0= 5 >> >> TranslationModel0= 0 0 0 0 ||| nan >> >> >> >> >> >> >> >> >> >> >> >> What is the reason of appearing these "nan" values? I've successfully >> >> trained smaller (1000 sentences) corpus using the same config file. >> >> >> >> >> >> >> >> Please help! :) >> >> >> >> >> >> >> >> Regards, >> >> >> >> Thomas >> >> >> >> -------------- next part -------------- >> >> An HTML attachment was scrubbed... >> >> URL: http://mailman.mit.edu/mailman/private/moses-support/attachm >> >> ents/20160805/933d8db9/attachment-0001.html >> >> >> >> ------------------------------ >> >> >> >> _______________________________________________ >> >> Moses-support mailing list >> >> [email protected] >> >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> >> >> >> End of Moses-support Digest, Vol 118, Issue 13 >> >> ********************************************** >> >> >> > >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: http://mailman.mit.edu/mailman/private/moses-support/attachm >> ents/20160807/6695a864/attachment.html >> >> ------------------------------ >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> End of Moses-support Digest, Vol 118, Issue 14 >> ********************************************** >> > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
