thanks for your reply.
As I run like this:
${mosesScript}/training/mert-moses.pl \
--mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/mert \
"${dataDir}/${devSet}/dev.src" \
"${dataDir}/${devSet}/dev.ref" \
"${decoder}" \
"${extractini}" \
--working-dir "${mertDir}" \
--no-filter-phrase-table
the Assertion problem still exists:
moses_chart: Phrase.cpp:236: void
Moses::Phrase::CreateFromStringNewFormat(Moses::FactorDirection, const
std::vector<long unsigned int, std::allocator<long unsigned int> >&, const
std::string&, const std::string&, Moses::Word&): Assertion `nextPos !=
string::npos' failed.
Aborted
Exit code: 134
The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143
0.057143 0.057143 0.057143 0.285714
I looked into MERTDIR, mert and extractor are there. So I think the compile is
successful.
Meanwhile, if I run like this:
${mosesScript}/training/mert-moses.pl \
--mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/mert \
"${dataDir}/${devSet}/dev.src" \
"${dataDir}/${devSet}/dev.ref" \
"${decoder}" \
"${extractini}" \
--working-dir "${mertDir}"
The filtering will filter all rules, leaving nothing in phrase-table.0-0.1.1
Then mert goes well for twice, and when it come to translate the test corpus,
that assertion problem comes up again.
So I think there's something wrong either with the rule table or my parameter
settings.
I'm still working on it...
----------------------------------------------------
Best wishes!
Xianhua Li
-----邮件原件-----
发件人: [email protected] [mailto:[email protected]] 代表
[email protected]
发送时间: 2011年6月29日 17:02
收件人: [email protected]
主题: Moses-support Digest, Vol 56, Issue 33
Send Moses-support mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: to (Li Xianhua) about moses-hierarchical- void
Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos !=
string::npos' failed (Thu Vuong Hoai)
2. Re: mert-moses.pl terminates with "[ERROR] Malformed input",
Exit code: 134 (Barry Haddow)
3. Re: mert-moses.pl terminates with "[ERROR] Malformed input",
Exit code: 134 (Tom Hoar)
----------------------------------------------------------------------
Message: 1
Date: Wed, 29 Jun 2011 11:24:11 +0700
From: Thu Vuong Hoai <[email protected]>
Subject: Re: [Moses-support] to (Li Xianhua) about moses-hierarchical-
void Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos !=
string::npos' failed
To: [email protected]
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"
1. could you please try to use make distclean to clean all generated by
build tool then run regenerate-makefiles.sh
2. the second error happen because mert-moses.pl can not found extractor in
MERT_DIR, i think you should check the param --mert-dirs, or copy or make
symlink of extractor in trunk/mert (this file will be generate after you
compile successful)
On Wed, Jun 29, 2011 at 10:51 AM, <[email protected]> wrote:
> Send Moses-support mailing list submissions to
> [email protected]
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/moses-support
> or, via email, send a message with subject or body 'help' to
> [email protected]
>
> You can reach the person managing the list at
> [email protected]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Moses-support digest..."
>
>
> Today's Topics:
>
> 1. multithreaded moses, memory and on-disk issues (Dennis Mehay)
> 2. moses-hierarchical- void
> Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos !=
> string::npos' failed. (Li Xianhua)
>
>
> ----------------------------------------------------------------------
>
> Message: 2
> Date: Wed, 29 Jun 2011 11:50:54 +0800
> From: "Li Xianhua" <[email protected]>
> Subject: [Moses-support] moses-hierarchical- void
> Moses::Phrase::CreateFromStringNewFormat- Assertion `nextPos !=
> string::npos' failed.
> To: <[email protected]>
> Message-ID: <[email protected]>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi, all,
>
>
>
> I was used to moses20080711, but now I'm trying the latest version.
> Here are some problems I come up with.
>
>
>
> 1. Installing
>
> ./regenerate-makefiles.sh
>
>
>
> Calling /usr/bin/aclocal...
>
> Calling /usr/bin/autoconf...
>
> Calling /usr/bin/automake...
>
> Calling /usr/bin/libtoolize
>
> libtoolize: `config.guess' exists: use `--force' to overwrite
>
> libtoolize: `config.sub' exists: use `--force' to overwrite
>
> libtoolize: `ltmain.sh' exists: use `--force' to overwrite
>
>
>
> You should now be able to configure and build:
>
> ./configure [--with-srilm=/path/to/srilm] [--with-irstlm=/path/to/irstlm]
> [--with-randlm=/path/to/randlm] [--without-kenlm] [--with-synlm]
> [--with-xmlrpc-c=/path/to/xmlrpc-c-config]
>
> make -j 4
>
>
>
> I tried to use -force but it seems useless. The information doesn't change.
>
> When doing ./regenerate-makefiles.sh in moses20080711, it's OK.
>
> Does it matter?
>
>
>
> 2. mert-moses.pl
>
> I added --hierarchical --glue-grammar, deleted --reordering
> msd-bidirectional-fe \, the extraction seems correct.
>
> I come up with something like "extractor not found", so I modified
> mert-moses.pl: line296 to $mertdir = "$SCRIPTS_ROOTDIR/../../../mert";
>
> Then
>
> ${mosesScript}/training/mert-moses.pl \
>
> "${dataDir}/${devSet}/dev.src" \
>
> "${dataDir}/${devSet}/dev.ref" \
>
> "${decoder}" \
>
> "${extractini}" \
>
> --working-dir "${mertDir}" \
>
> --decoder-flags "-v 0" \
>
> --no-filter-phrase-table
>
>
>
> --no-filter-phrase-table is used to avoid filtering rule table. Or should I
> just switch my $___FILTER_PHRASE_TABLE in mert-moses.pl to 0 ?
>
> What should I do if I want to filter hierarchical phrase table in
> mert-moses.pl? if seems the filtering is only for phrase-based model.
>
>
>
> Here are the output information:
>
> Using SCRIPTS_ROOTDIR:
>
> /home/lxh/1moses-latest/mosesdecoder/trunk/scripts/target/scripts-20110628-1
> 353
>
> Assuming
>
> --mertdir=/home/lxh/1moses-latest/mosesdecoder/trunk/scripts/target/scripts-
> 20110628-1353/../../../mert
>
> Executing: mkdir -p /home/lxh/1moses-test-HPB/0615-mert
>
> run 1 start at Wed Jun 29 11:21:00 CST 2011
>
> Parsing --decoder-flags: |-v 0|
>
> Saving new config to: ./run1.moses.ini
>
> Saved: ./run1.moses.ini
>
> (1) run decoder to produce n-best lists
>
> params = -v 0
>
> Normalizing lambdas: -1 0.5000 0.20 0.20 0.20 0.20 0.20 1.0
>
> DECODER_CFG = -w %.6f -lm %.6f -tm %.6f %.6f %.6f %.6f %.6f %.6f
>
> values = -0.285714285714286 0.142857142857143 0.0571428571428571
> 0.0571428571428571 0.0571428571428571 0.0571428571428571 0.0571428571428571
> 0.285714285714286
>
> decoder_config = -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143
> 0.057143 0.057143 0.285714
>
> Executing:
> /home/lxh/1moses-latest/mosesdecoder/trunk/moses-chart-cmd/src/moses_chart
> -v 0 -config /home/lxh/1moses-test-HPB/0615-train/model/moses.ini
> -inputtype 0 -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143
> 0.057143 0.057143 0.285714 -n-best-list run1.best100.out 100 -input-file
> /home/lxh/1moses-test-HPB/x/dev/dev.src > run1.out
>
> /home/lxh/1moses-test-HPB/x/patent.title.10w.o5.gz: line 34: warning:
> non-zero probability for <unk> in closed-vocabulary LM
>
> Start loading new format pt model : [0.000] seconds
>
> moses_chart: Phrase.cpp:236: void
> Moses::Phrase::CreateFromStringNewFormat(Moses::FactorDirection, const
> std::vector<long unsigned int, std::allocator<long unsigned int> >&, const
> std::string&, const std::string&, Moses::Word&): Assertion `nextPos !=
> string::npos' failed.
>
> Aborted
>
> Exit code: 134
>
> The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
> 0.057143
> 0.057143 0.057143 0.057143 0.285714
>
> I believe this must be some simple mistakes, maybe I've used wrong
> parameters, but I can't find the answer.
>
> Anyone can help?
>
>>
--
Thu.
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 56, Issue 33
*********************************************
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
