When I train the hierarchical phrase-based model using the following command: tools/moses-scripts/scripts-20100517-1641/training/train-model.perl -scripts-root-dir tools/moses-scripts/scripts-20100517-1641 -root-dir hier-dir -corpus hier-dir/corpus/raw -f ch -e uy -alignment grow-diag-final-and -parallel -hierarchical -glue-grammar -max-phrase-length 5 -lm 0:3:/home/top/programming/language_model/hier-dir/lm/raw.uy.lm:0 errors occurred,seeming this: ...... (6.2) creating table half hier-dir/model/rule-table.half.f2e @ Thu May 20 17:08:43 CST 2010 tools/moses-scripts/scripts-20100517-1641/training/phrase-extract/score hier-dir/model/extract.sorted hier-dir/model/lex.f2e hier-dir/model/rule-table.half.f2e --Hierarchical Executing: tools/moses-scripts/scripts-20100517-1641/training/phrase-extract/score hier-dir/model/extract.sorted hier-dir/model/lex.f2e hier-dir/model/rule-table.half.f2e --Hierarchical Score v2.0 written by Philipp Koehn scoring methods for extracted rules processing hierarchical rules Loading lexical translation table from hier-dir/model/lex.f2e. .score: score.cpp:454: void outputPhrasePair(std::vector<PhraseAlignment*, std::allocator<PhraseAlignment*> >&, float): Assertion `bestAlignment->alignedToT[ j ].size() == 1' failed. ERROR: Execution of: tools/moses-scripts/scripts-20100517-1641/training/phrase-extract/score hier-dir/model/extract.sorted hier-dir/model/lex.f2e hier-dir/model/rule-table.half.f2e --Hierarchical died with signal 6, without coredump I have no idear about it , Could you help me? thank you !<br><br>在2010-05-20 00:04:29，[email protected] 写道： >Send Moses-support mailing list submissions to > [email protected] > >To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support >or, via email, send a message with subject or body 'help' to > [email protected] > >You can reach the person managing the list at > [email protected] > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of Moses-support digest..." > > >Today's Topics: > > 1. Re: experiment management system and Moses scripts (Philipp Koehn) > 2. Re: Moses seems to hang ([email protected]) > 3. Use of unfactored training data set in moses??? > (sripirakas sakthithasan) > 4. Re: Moses seems to hang (Barry Haddow) > 5. Use of unfactored training data set in moses??? > (sripirakas sakthithasan) > > >---------------------------------------------------------------------- > >Message: 1 >Date: Tue, 18 May 2010 18:40:42 +0100 >From: Philipp Koehn <[email protected]> >Subject: Re: [Moses-support] experiment management system and Moses > scripts >To: Suzy Howlett <[email protected]> >Cc: [email protected] >Message-ID: > <[email protected]> >Content-Type: text/plain; charset=ISO-8859-1 > >Hi, > >thanks for trying out experiment.perl, please let me know of any >other problems you encounter. > >The two solutions you indicate are both correct. I just updated in >the main branch ~/mosesdecoder/scripts/released_files so that >should be up to date now. > >-phi > >On Tue, May 18, 2010 at 3:23 AM, Suzy Howlett <[email protected]> wrote: >> Hi all, >> >> I've just downloaded and installed the latest subversion revision >> (3261) and I'm trying out the new experiment management system, but >> the Moses scripts are causing me problems. When I point the config >> variable 'moses-script-dir' to ~/mosesdecoder/scripts (the scripts >> directory inside the repository), executing the ems crashes because >> BINDIR is not set in train-model.perl. However if I instead run 'make >> release' from ~/mosesdecoder/scripts and set moses-scripts-dir to >> point to the directory thus created, the ems crashes when checking the >> config because that scripts directory doesn't include generic/mteval- >> v12.pl or the tokenizer and ems directories. Is the solution to update >> ~/mosesdecoder/scripts/released_files, or to change BINDIR in train- >> model.perl without running 'make release', or something else? >> >> Thanks, >> Suzy >> >> >> -- >> Suzy Howlett >> http://web.science.mq.edu.au/~showlett/ >> >> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > >------------------------------ > >Message: 2 >Date: Tue, 18 May 2010 20:37:36 -0700 >From: <[email protected]> >Subject: Re: [Moses-support] Moses seems to hang >To: <[email protected]> >Message-ID: > <[email protected]> >Content-Type: text/plain; charset="utf-8" > >Thanks Franz and Raphael, > >Re: echo. I was using echo, but didn't copy to email. Sorry. > >I rebuilt the model with closer attention to steps in the new link. Seems >that I didn't wait long enough before. It finally translated after over an >hour. It took 3776 seconds to load the phrase table. That's after I edited >the moses.ini file to use the binarized table (I thought). The moses >output and the moses.ini config are attached. > >What am I missing? > >Tom > > >On Sat, 15 May 2010 16:13:30 +0100, Raphael Payen ><[email protected]> >wrote: >>> Here's the input: >>> >>> dit is niet gebeurd . | moses -f ./model/moses.ini >> >> You need to write "echo" before your input, otherwise "dit" is not a >> valid command. >> >> Also, when moses does nothing, try to type "dit is niet gebeurd" then >> Ctrl-d instead of Ctrl-c >-------------- next part -------------- >A non-text attachment was scrubbed... >Name: moses.ini >Type: application/x-wine-extension-ini >Size: 1025 bytes >Desc: not available >Url : >http://mailman.mit.edu/mailman/private/moses-support/attachments/20100518/1b5300aa/moses-0001.bin >-------------- next part -------------- >*********** translation ************ >u...@moses01:~/home$ echo dit is niet gebeurd . | moses -f >./model/moses.ini >Defined parameters (per moses.ini or switch): > config: ./model/moses (copy).ini > distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6 >./model/reordering-table > distortion-limit: 6 > input-factors: 0 > lmodel-file: 1 0 3 ./irstlm-3.en.blm.mm > mapping: 0 T 0 > ttable-file: 0 0 0 5 ./model/phrase-table > ttable-limit: 20 > weight-d: 0.3 0.3 0.3 0.3 0.3 0.3 0.3 > weight-l: 0.5000 > weight-t: 0.2 0.2 0.2 0.2 0.2 > weight-w: -1 >Loading lexical distortion models...have 1 models >Creating lexical reordering... >weights: 0.300 0.300 0.300 0.300 0.300 0.300 >binary file loaded, default OFF_T: -1 >Start loading LanguageModel ./irstlm-3.en.blm.mm : [0.000] seconds >In LanguageModelIRST::Load: nGramOrder = 3 >Loading LM file (no MAP) >blmt >loadbin() >mapping 34988 1-grams >mapping 559604 2-grams >mapping 1698783 3-grams >done >OOV code is 34987 >IRST: m_unknownId=34987 >Finished loading LanguageModels : [0.000] seconds >Start loading PhraseTable ./model/phrase-table : [0.000] seconds >filePath: ./model/phrase-table >Finished loading phrase tables : [3776.000] seconds >IO from STDOUT/STDIN >Created input-output object : [3776.000] seconds >Translating: dit is niet gebeurd . > >Collecting options took 0.080 seconds >Search took 1.190 seconds >BEST TRANSLATION: this is not the case . [11111] [total=-6.312] <<0.000, >-6.000, 0.000, -1.802, 0.000, 0.000, -0.151, 0.000, 0.000, -13.278, >-6.422, >-7.611, -4.700, -8.698, 2.000>> >this is not the case . >reset caches >reset mmap >len = 528244 >sync = 0 >running msync... >done. Running munmap... >done >len = 8398016 >sync = 0 >running msync... >done. Running munmap... >done >len = 11892697 >sync = 0 >running msync... >done. Running munmap... >done >Translation took 2.120 seconds >Finished translating >End. : [4077.000] seconds > >------------------------------ > >Message: 3 >Date: Tue, 18 May 2010 22:42:41 -0700 (PDT) >From: sripirakas sakthithasan <[email protected]> >Subject: [Moses-support] Use of unfactored training data set in > moses??? >To: [email protected] >Message-ID: <[email protected]> >Content-Type: text/plain; charset="us-ascii" > >Dear All, > >I am doing a research on the development of statistical translation system for >Sri Lankan local languages (Sinhala and Tamil). The available corpus is >unfactored and it was created by me.. So, I would like to know whether the >script found in >moses-scripts/scripts-timestamp/training/train-factored-phrase-model.perl is >suitable for the training. The manual on "Moses use" specifies a script called >"train-model.perl" for unfactored model training, which I was unable to >locate. Expecting your help as soon as possible. I am really thanking your >help to solve the above issue. > >Thank you. > > > > >-------------- next part -------------- >An HTML attachment was scrubbed... >URL: >http://mailman.mit.edu/mailman/private/moses-support/attachments/20100518/b957dad7/attachment-0001.htm > >------------------------------ > >Message: 4 >Date: Wed, 19 May 2010 09:57:17 +0100 >From: Barry Haddow <[email protected]> >Subject: Re: [Moses-support] Moses seems to hang >To: [email protected] >Message-ID: <[email protected]> >Content-Type: Text/Plain; charset="utf-8" > >Hi > >To use the binary phrase table, change this line > >0 0 0 5 /media/models/europarl.v5.mini-3-nl-en/model/phrase-table > >to > >1 0 0 5 /media/models/europarl.v5.mini-3-nl-en/model/phrase-table > >regards >Barry > > >On Wednesday 19 May 2010 04:37:36 [email protected] wrote: >> Thanks Franz and Raphael, >> >> Re: echo. I was using echo, but didn't copy to email. Sorry. >> >> I rebuilt the model with closer attention to steps in the new link. Seems >> that I didn't wait long enough before. It finally translated after over an >> hour. It took 3776 seconds to load the phrase table. That's after I edited >> the moses.ini file to use the binarized table (I thought). The moses >> output and the moses.ini config are attached. >> >> What am I missing? >> >> Tom >> >> >> On Sat, 15 May 2010 16:13:30 +0100, Raphael Payen >> <[email protected]> >> >> wrote: >> >> Here's the input: >> >> >> >> dit is niet gebeurd . | moses -f ./model/moses.ini >> > >> > You need to write "echo" before your input, otherwise "dit" is not a >> > valid command. >> > >> > Also, when moses does nothing, try to type "dit is niet gebeurd" then >> > Ctrl-d instead of Ctrl-c >> > > > >------------------------------ > >Message: 5 >Date: Wed, 19 May 2010 04:56:16 -0700 (PDT) >From: sripirakas sakthithasan <[email protected]> >Subject: [Moses-support] Use of unfactored training data set in > moses??? >To: [email protected] >Message-ID: <[email protected]> >Content-Type: text/plain; charset="us-ascii" > > > >Dear All, > >I am doing a research on the development of statistical translation system for >Sri Lankan local languages (Sinhala and Tamil). The available corpus is >unfactored and it was created by me.. So, I would like to know whether the >script found in >moses-scripts/scripts-timestamp/training/train-factored-phrase-model.perl is >suitable for the training. The manual on "Moses use" specifies a script called >"train-model.perl" for unfactored model training, which I was unable to >locate. Expecting your help as soon as possible. I am really thanking your >help to solve the above issue. > >Thank you. > > > > > > > >-------------- next part -------------- >An HTML attachment was scrubbed... >URL: >http://mailman.mit.edu/mailman/private/moses-support/attachments/20100519/e9c983f5/attachment-0001.htm > >------------------------------ > >_______________________________________________ >Moses-support mailing list >[email protected] >http://mailman.mit.edu/mailman/listinfo/moses-support > > >End of Moses-support Digest, Vol 43, Issue 20 >*********************************************
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
