Hi hieu， Sure, the ini file is attached and here’re some lines of the phrase table. Thanks very much.
[X][X] 大 [X][X] 启动 [X] ||| [X][X] [X][X] スタ�`ト [X] ||| 0.291666 0.125 0.636364 1 2.718 ||| 0-0 2-1 ||| 0.171429 0.0785714 [X][X] 大 [X][X] 启动 [X] ||| [X][X] [X][X] スタ�`ト と [X] ||| 0.289156 0.125 0.363636 0.0344828 2.718 ||| 0-0 2-1 ||| 0.0988095 0.0785714 [X][X] 大 负载 [X] ||| [X][X] で スタ�`ト [X] ||| 0.54054 1 0.631579 1 2.718 ||| 0-0 ||| 0.264286 0.22619 [X][X] 大 负载 [X] ||| [X][X] で スタ�`ト と [X] ||| 0.542636 1 0.368421 0.0344828 2.718 ||| 0-0 ||| 0.153571 0.22619 [X][X] 大 负载 下 [X] ||| [X][X] で スタ�`ト [X] ||| 0.27027 0.125 0.631579 1 2.718 ||| 0-0 ||| 0.264286 0.113095 [X][X] 大 负载 下 [X] ||| [X][X] で スタ�`ト と [X] ||| 0.271318 0.125 0.368421 0.0344828 2.718 ||| 0-0 ||| 0.153571 0.113095 [X][X] 大 负载 下 启动 [X] ||| [X][X] で スタ�`ト [X] ||| 0.189189 0.015625 0.636364 1 2.718 ||| 0-0 ||| 0.264286 0.0785714 [X][X] 大 负载 下 启动 [X] ||| [X][X] で スタ�`ト と [X] ||| 0.186046 0.015625 0.363636 0.0344828 2.718 ||| 0-0 ||| 0.153571 0.0785714 [X][X] 开关 [X] ||| [X][X] �g�� [X] ||| 1 1 1 1 2.718 ||| 0-0 ||| 0.166667 0.166667 [X][X] 性 [X] ||| [X][X] 配�� [X] ||| 1 1 1 1 2.718 ||| 0-0 ||| 0.333333 0.333333 [X][X] 性 [X][X] [X] ||| [X][X] [X][X] 配�� [X] ||| 1 1 1 1 2.718 ||| 2-0 0-1 ||| 0.166667 0.166667 [X][X] 性 排 [X] ||| 1 [X][X] 配�� [X] ||| 0.6 0.333333 1 1 2.718 ||| 0-1 ||| 0.416667 0.25 [X][X] 性 排 线 [X] ||| 1 [X][X] 配�� [X] ||| 0.4 0.0416667 1 1 2.718 ||| 0-1 ||| 0.416667 0.166667 [X][X] 排 [X] ||| 1 [X][X] [X] ||| 0.6 0.333333 1 1 2.718 ||| 0-1 ||| 0.416667 0.25 [X][X] 排 线 [X] ||| 1 [X][X] [X] ||| 0.4 0.0416667 1 1 2.718 ||| 0-1 ||| 0. 416667 0.166667 [X][X] 方法 [X] ||| 方法 [X][X] [X] ||| 1 0.666667 1 1 2.718 ||| 0-1 ||| 0.5 0.5 [X][X] 板 [X] ||| �Mみ立て [X][X] [X] ||| 0.5 1 1 1 2.718 ||| 0-1 ||| 0.4 0. 2 ---------------------------------------------------- Best wishes! Xianhua Li 发件人: Hieu Hoang [mailto:[email protected]] 发送时间: 2011年6月30日 9:54 收件人: Li Xianhua 抄送: [email protected]; [email protected] 主题: Re: [Moses-support] CreateFromStringNewFormat-Assertion `nextPos != string::npos' failed can you post a copy of the ini file and a few lines of the phrase table. I'll take a look @ it On 30 June 2011 08:25, Li Xianhua <[email protected]> wrote: thanks for your reply. As I run like this: ${mosesScript}/training/mert-moses.pl \ --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/mert \ "${dataDir}/${devSet}/dev.src" \ "${dataDir}/${devSet}/dev.ref" \ "${decoder}" \ "${extractini}" \ --working-dir "${mertDir}" \ --no-filter-phrase-table the Assertion problem still exists: moses_chart: Phrase.cpp:236: void Moses::Phrase::CreateFromStringNewFormat(Moses::FactorDirection, const std::vector<long unsigned int, std::allocator<long unsigned int> >&, const std::string&, const std::string&, Moses::Word&): Assertion `nextPos != string::npos' failed. Aborted Exit code: 134 The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 I looked into MERTDIR, mert and extractor are there. So I think the compile is successful. Meanwhile, if I run like this: ${mosesScript}/training/mert-moses.pl \ --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/mert \ "${dataDir}/${devSet}/dev.src" \ "${dataDir}/${devSet}/dev.ref" \ "${decoder}" \ "${extractini}" \ --working-dir "${mertDir}" The filtering will filter all rules, leaving nothing in phrase-table.0-0.1. 1 Then mert goes well for twice, and when it come to translate the test corpus, that assertion problem comes up again. So I think there's something wrong either with the rule table or my parameter settings. I'm still working on it... ---------------------------------------------------- Best wishes! Xianhua Li -----邮件原件----- 发件人: [email protected] [mailto:[email protected]] 代表 [email protected] 发送时间: 2011年6月29日 17:02 收件人: [email protected] 主题: Moses-support Digest, Vol 56, Issue 33 Send Moses-support mailing list submissions to [email protected] To subscribe or unsubscribe via the World Wide Web, visit http://mailman.mit.edu/mailman/listinfo/moses-support or, via email, send a message with subject or body 'help' to [email protected] You can reach the person managing the list at [email protected] When replying, please edit your Subject line so it is more specific than "Re: Contents of Moses-support digest..." Today's Topics: 1. Re: to (Li Xianhua) about moses-hierarchical- void Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != string::npos' failed (Thu Vuong Hoai) 2. Re: mert-moses.pl terminates with "[ERROR] Malformed input", Exit code: 134 (Barry Haddow) 3. Re: mert-moses.pl terminates with "[ERROR] Malformed input", Exit code: 134 (Tom Hoar) ---------------------------------------------------------------------- Message: 1 Date: Wed, 29 Jun 2011 11:24:11 +0700 From: Thu Vuong Hoai <[email protected]> Subject: Re: [Moses-support] to (Li Xianhua) about moses-hierarchical- void Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != string::npos' failed To: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset="utf-8" 1. could you please try to use make distclean to clean all generated by build tool then run regenerate-makefiles.sh 2. the second error happen because mert-moses.pl can not found extractor in MERT_DIR, i think you should check the param --mert-dirs, or copy or make symlink of extractor in trunk/mert (this file will be generate after you compile successful) On Wed, Jun 29, 2011 at 10:51 AM, <[email protected]> wrote: > Send Moses-support mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. multithreaded moses, memory and on-disk issues (Dennis Mehay) > 2. moses-hierarchical- void > Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != > string::npos' failed. (Li Xianhua) > > > ---------------------------------------------------------------------- > > Message: 2 > Date: Wed, 29 Jun 2011 11:50:54 +0800 > From: "Li Xianhua" <[email protected]> > Subject: [Moses-support] moses-hierarchical- void > Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != > string::npos' failed. > To: <[email protected]> > Message-ID: <[email protected]> > Content-Type: text/plain; charset="us-ascii" > > Hi, all, > > > > I was used to moses20080711, but now I'm trying the latest version. > Here are some problems I come up with. > > > > 1. Installing > > ./regenerate-makefiles.sh > > > > Calling /usr/bin/aclocal... > > Calling /usr/bin/autoconf... > > Calling /usr/bin/automake... > > Calling /usr/bin/libtoolize > > libtoolize: `config.guess' exists: use `--force' to overwrite > > libtoolize: `config.sub' exists: use `--force' to overwrite > > libtoolize: `ltmain.sh' exists: use `--force' to overwrite > > > > You should now be able to configure and build: > > ./configure [--with-srilm=/path/to/srilm] [--with-irstlm=/path/to/irstlm] > [--with-randlm=/path/to/randlm] [--without-kenlm] [--with-synlm] > [--with-xmlrpc-c=/path/to/xmlrpc-c-config] > > make -j 4 > > > > I tried to use -force but it seems useless. The information doesn't change. > > When doing ./regenerate-makefiles.sh in moses20080711, it's OK. > > Does it matter? > > > > 2. mert-moses.pl > > I added --hierarchical --glue-grammar, deleted --reordering > msd-bidirectional-fe \, the extraction seems correct. > > I come up with something like "extractor not found", so I modified > mert-moses.pl: line296 to $mertdir = "$SCRIPTS_ROOTDIR/../../../mert"; > > Then > > ${mosesScript}/training/mert-moses.pl \ > > "${dataDir}/${devSet}/dev.src" \ > > "${dataDir}/${devSet}/dev.ref" \ > > "${decoder}" \ > > "${extractini}" \ > > --working-dir "${mertDir}" \ > > --decoder-flags "-v 0" \ > > --no-filter-phrase-table > > > > --no-filter-phrase-table is used to avoid filtering rule table. Or should I > just switch my $___FILTER_PHRASE_TABLE in mert-moses.pl to 0 ? > > What should I do if I want to filter hierarchical phrase table in > mert-moses.pl? if seems the filtering is only for phrase-based model. > > > > Here are the output information: > > Using SCRIPTS_ROOTDIR: > > /home/lxh/1moses-latest/mosesdecoder/trunk/scripts/target/scripts-20110628-1 > 353 > > Assuming > > --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/scripts/target/scripts- > 20110628-1353/../../../mert > > Executing: mkdir -p /home/lxh/1moses-test-HPB/0615-mert > > run 1 start at Wed Jun 29 11:21:00 CST 2011 > > Parsing --decoder-flags: |-v 0| > > Saving new config to: ./run1.moses.ini > > Saved: ./run1.moses.ini > > (1) run decoder to produce n-best lists > > params = -v 0 > > Normalizing lambdas: -1 0.5000 0.20 0.20 0.20 0.20 0.20 1.0 > > DECODER_CFG = -w %.6f -lm %.6f -tm %.6f %.6f %.6f %.6f %.6f %.6f > > values = -0.285714285714286 0.142857142857143 0.0571428571428571 > 0.0571428571428571 0.0571428571428571 0.0571428571428571 0.0571428571428571 > 0.285714285714286 > > decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > 0.057143 0.057143 0.285714 > > Executing: > /home/lxh/1moses-latest/mosesdecoder/trunk/moses-chart-cmd/src/moses_chart > -v 0 -config /home/lxh/1moses-test-HPB/0615-train/model/moses.ini > -inputtype 0 -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > 0.057143 0.057143 0.285714 -n-best-list run1.best100.out 100 -input-file > /home/lxh/1moses-test-HPB/x/dev/dev.src > run1.out > > /home/lxh/1moses-test-HPB/x/patent.title.10w.o5.gz: line 34: warning: > non-zero probability for <unk> in closed-vocabulary LM > > Start loading new format pt model : [0.000] seconds > > moses_chart: Phrase.cpp:236: void > Moses::Phrase::CreateFromStringNewFormat(Moses::FactorDirection, const > std::vector<long unsigned int, std::allocator<long unsigned int> >&, const > std::string&, const std::string&, Moses::Word&): Assertion `nextPos != > string::npos' failed. > > Aborted > > Exit code: 134 > > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > 0.057143 > 0.057143 0.057143 0.057143 0.285714 > > I believe this must be some simple mistakes, maybe I've used wrong > parameters, but I can't find the answer. > > Anyone can help? > >> -- Thu. [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support End of Moses-support Digest, Vol 56, Issue 33 ********************************************* _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
