can you post a copy of the ini file and a few lines of the phrase table. I'll take a look @ it
On 30 June 2011 08:25, Li Xianhua <[email protected]> wrote: > > thanks for your reply. > As I run like this: > ${mosesScript}/training/mert-moses.pl \ > --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/mert \ > "${dataDir}/${devSet}/dev.src" \ > "${dataDir}/${devSet}/dev.ref" \ > "${decoder}" \ > "${extractini}" \ > --working-dir "${mertDir}" \ > --no-filter-phrase-table > > the Assertion problem still exists: > moses_chart: Phrase.cpp:236: void > Moses::Phrase::CreateFromStringNewFormat(Moses::FactorDirection, const > std::vector<long unsigned int, std::allocator<long unsigned int> >&, const > std::string&, const std::string&, Moses::Word&): Assertion `nextPos != > string::npos' failed. > Aborted > Exit code: 134 > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > 0.057143 0.057143 0.057143 0.057143 0.285714 > > I looked into MERTDIR, mert and extractor are there. So I think the compile > is successful. > > Meanwhile, if I run like this: > ${mosesScript}/training/mert-moses.pl \ > --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/mert \ > "${dataDir}/${devSet}/dev.src" \ > "${dataDir}/${devSet}/dev.ref" \ > "${decoder}" \ > "${extractini}" \ > --working-dir "${mertDir}" > The filtering will filter all rules, leaving nothing in > phrase-table.0-0.1.1 > Then mert goes well for twice, and when it come to translate the test > corpus, that assertion problem comes up again. > So I think there's something wrong either with the rule table or my > parameter settings. > I'm still working on it... > > ---------------------------------------------------- > Best wishes! > Xianhua Li > > > -----邮件原件----- > 发件人: [email protected] [mailto:[email protected]] > 代表 [email protected] > 发送时间: 2011年6月29日 17:02 > 收件人: [email protected] > 主题: Moses-support Digest, Vol 56, Issue 33 > > Send Moses-support mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. Re: to (Li Xianhua) about moses-hierarchical- void > Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != > string::npos' failed (Thu Vuong Hoai) > 2. Re: mert-moses.pl terminates with "[ERROR] Malformed input", > Exit code: 134 (Barry Haddow) > 3. Re: mert-moses.pl terminates with "[ERROR] Malformed input", > Exit code: 134 (Tom Hoar) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 29 Jun 2011 11:24:11 +0700 > From: Thu Vuong Hoai <[email protected]> > Subject: Re: [Moses-support] to (Li Xianhua) about moses-hierarchical- > void Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != > string::npos' failed > To: [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset="utf-8" > > 1. could you please try to use make distclean to clean all generated by > build tool then run regenerate-makefiles.sh > 2. the second error happen because mert-moses.pl can not found extractor > in > MERT_DIR, i think you should check the param --mert-dirs, or copy or make > symlink of extractor in trunk/mert (this file will be generate after you > compile successful) > > On Wed, Jun 29, 2011 at 10:51 AM, <[email protected]> wrote: > > > Send Moses-support mailing list submissions to > > [email protected] > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://mailman.mit.edu/mailman/listinfo/moses-support > > or, via email, send a message with subject or body 'help' to > > [email protected] > > > > You can reach the person managing the list at > > [email protected] > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Moses-support digest..." > > > > > > Today's Topics: > > > > 1. multithreaded moses, memory and on-disk issues (Dennis Mehay) > > 2. moses-hierarchical- void > > Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != > > string::npos' failed. (Li Xianhua) > > > > > > ---------------------------------------------------------------------- > > > > Message: 2 > > Date: Wed, 29 Jun 2011 11:50:54 +0800 > > From: "Li Xianhua" <[email protected]> > > Subject: [Moses-support] moses-hierarchical- void > > Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos != > > string::npos' failed. > > To: <[email protected]> > > Message-ID: <[email protected]> > > Content-Type: text/plain; charset="us-ascii" > > > > Hi, all, > > > > > > > > I was used to moses20080711, but now I'm trying the latest > version. > > Here are some problems I come up with. > > > > > > > > 1. Installing > > > > ./regenerate-makefiles.sh > > > > > > > > Calling /usr/bin/aclocal... > > > > Calling /usr/bin/autoconf... > > > > Calling /usr/bin/automake... > > > > Calling /usr/bin/libtoolize > > > > libtoolize: `config.guess' exists: use `--force' to overwrite > > > > libtoolize: `config.sub' exists: use `--force' to overwrite > > > > libtoolize: `ltmain.sh' exists: use `--force' to overwrite > > > > > > > > You should now be able to configure and build: > > > > ./configure [--with-srilm=/path/to/srilm] > [--with-irstlm=/path/to/irstlm] > > [--with-randlm=/path/to/randlm] [--without-kenlm] [--with-synlm] > > [--with-xmlrpc-c=/path/to/xmlrpc-c-config] > > > > make -j 4 > > > > > > > > I tried to use -force but it seems useless. The information doesn't > change. > > > > When doing ./regenerate-makefiles.sh in moses20080711, it's OK. > > > > Does it matter? > > > > > > > > 2. mert-moses.pl > > > > I added --hierarchical --glue-grammar, deleted --reordering > > msd-bidirectional-fe \, the extraction seems correct. > > > > I come up with something like "extractor not found", so I modified > > mert-moses.pl: line296 to $mertdir = "$SCRIPTS_ROOTDIR/../../../mert"; > > > > Then > > > > ${mosesScript}/training/mert-moses.pl \ > > > > "${dataDir}/${devSet}/dev.src" \ > > > > "${dataDir}/${devSet}/dev.ref" \ > > > > "${decoder}" \ > > > > "${extractini}" \ > > > > --working-dir "${mertDir}" \ > > > > --decoder-flags "-v 0" \ > > > > --no-filter-phrase-table > > > > > > > > --no-filter-phrase-table is used to avoid filtering rule table. Or should > I > > just switch my $___FILTER_PHRASE_TABLE in mert-moses.pl to 0 ? > > > > What should I do if I want to filter hierarchical phrase table in > > mert-moses.pl? if seems the filtering is only for phrase-based model. > > > > > > > > Here are the output information: > > > > Using SCRIPTS_ROOTDIR: > > > > > /home/lxh/1moses-latest/mosesdecoder/trunk/scripts/target/scripts-20110628-1 > > 353 > > > > Assuming > > > > > --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/scripts/target/scripts- > > 20110628-1353/../../../mert > > > > Executing: mkdir -p /home/lxh/1moses-test-HPB/0615-mert > > > > run 1 start at Wed Jun 29 11:21:00 CST 2011 > > > > Parsing --decoder-flags: |-v 0| > > > > Saving new config to: ./run1.moses.ini > > > > Saved: ./run1.moses.ini > > > > (1) run decoder to produce n-best lists > > > > params = -v 0 > > > > Normalizing lambdas: -1 0.5000 0.20 0.20 0.20 0.20 0.20 1.0 > > > > DECODER_CFG = -w %.6f -lm %.6f -tm %.6f %.6f %.6f %.6f %.6f %.6f > > > > values = -0.285714285714286 0.142857142857143 0.0571428571428571 > > 0.0571428571428571 0.0571428571428571 0.0571428571428571 > 0.0571428571428571 > > 0.285714285714286 > > > > decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > > 0.057143 0.057143 0.285714 > > > > Executing: > > > /home/lxh/1moses-latest/mosesdecoder/trunk/moses-chart-cmd/src/moses_chart > > -v 0 -config /home/lxh/1moses-test-HPB/0615-train/model/moses.ini > > -inputtype 0 -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 > > 0.057143 0.057143 0.285714 -n-best-list run1.best100.out 100 -input-file > > /home/lxh/1moses-test-HPB/x/dev/dev.src > run1.out > > > > /home/lxh/1moses-test-HPB/x/patent.title.10w.o5.gz: line 34: warning: > > non-zero probability for <unk> in closed-vocabulary LM > > > > Start loading new format pt model : [0.000] seconds > > > > moses_chart: Phrase.cpp:236: void > > Moses::Phrase::CreateFromStringNewFormat(Moses::FactorDirection, const > > std::vector<long unsigned int, std::allocator<long unsigned int> >&, > const > > std::string&, const std::string&, Moses::Word&): Assertion `nextPos != > > string::npos' failed. > > > > Aborted > > > > Exit code: 134 > > > > The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 > > 0.057143 > > 0.057143 0.057143 0.057143 0.285714 > > > > I believe this must be some simple mistakes, maybe I've used wrong > > parameters, but I can't find the answer. > > > > Anyone can help? > > > >> > > > -- > Thu. > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 56, Issue 33 > ********************************************* > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
