it looks like your phrase-table has 2 input factors and 2 output factors.
When you binarize your phrase-table, you must run it like this:
CreateOnDiskPt 2 2 ....
not
CreateOnDiskPt 1 1 ....
On 05/02/15 04:42, Lisa Kim wrote:
Thank you, Hoang.
After updating latest version, I have different error during tuning
phase. :(
For the some issues, I cannot upload the model files and these are
part of my models and configs.
Do I have some mistakes in the ems configuration ?
Error Message :
Start loading text phrase table. Moses format : [0.014] seconds
terminate called after throwing an instance of 'util::Exception'
what():
moses/TranslationModel/RuleTable/PhraseDictionaryOnDisk.cpp:96 in
virtual void Moses::PhraseDictionaryOnDisk::InitializeForInput(const
Moses::InputType&) threw util::Exception because
`obj->GetMisc("NumSourceFactors") != m_input.size()'.
On-disk phrase table has 1 source factors.. The ini file specified 2
source factors
Aborted (core dumped)
Exit code: 134
Moses.ini file :
# input factors
[input-factors]
0
1
# mapping steps
[mapping]
0 T 0
1 T 1
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home3/ekkim/working/E2K_1502/hierarchical_class/model/phrase-table.1.0,1-0,1
input-factor=0,1 output-factor=0,1
PhraseDictionaryMemory name=TranslationModel1 num-features=1
path=/home3/ekkim/working/E2K_1502/hierarchical_class/model/glue-grammar.1
input-factor=0 output-factor=0
KENLM lazyken=0 name=LM0 factor=0
path=/home3/ekkim/working/E2K_1502/hierarchical_class/lm/travel22.binlm.1
order=5# dense weights for feature functions
KENLM lazyken=0 name=LM1 factor=1
path=/home3/ekkim/working/E2K_1502/hierarchical_class/lm/travel22_mkcls.binlm.1
order=5
# dense weights for feature functions
[weight]
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
TranslationModel1= 1.0
LM0= 0.5
LM1= 0.5
Part of Phrase Table :
[X][X] any|296 discount|413 [X][X] [X] ||| [X][X] 할인|349 혜택|176
이|393 [X][X] [X] ||| 0.0306047 0.00709669 0.0481453 0.000125329 |||
0-4 1-1 1-3 2-1 2-2 3-0 ||| 0.460664 0.292832 0.292832 ||| |||
[X][X] any|296 discount|413 [X][X] a|327 [X] ||| [X][X] 할인|349 혜
택|176 이|393 [X][X] [X] ||| 0.00803946 0.000214715 0.0481453
0.000125329 ||| 0-4 1-1 1-3 2-1 2-2 3-0 ||| 0.460664 0.0769231
0.0769231 ||| |||
[X][X] any|296 discount|413 [X][X] use|136 [X] ||| [X][X] 할인|349 혜
택|176 이|393 [X][X] [X] ||| 0.00950118 2.99551e-06 0.0481453
0.000125329 ||| 0-4 1-1 1-3 2-1 2-2 3-0 ||| 0.460664 0.0909091
0.0909091 ||| |||
[X][X] any|296 discount|413 cards|404 [X] ||| 할인|349 되|161 는|302
카드|184 [X][X] [X] ||| 0.0266944 0.00037872 0.0481453 8.14358e-05 |||
0-4 1-0 2-0 2-1 3-2 3-3 ||| 0.561111 0.311111 0.311111 ||| |||
[X][X] any|296 discount|413 card|404 [X] ||| 할인|349 되|161 는|302 카
드|184 [X][X] [X] ||| 0.0214509 0.00256232 0.0481453 0.00010193 |||
0-4 1-0 1-2 2-0 2-1 3-3 ||| 0.561111 0.25 0.25 ||| |||
[X][X] any|296 discount|413 for|487 [X] ||| 할인|349 [X][X] [X] |||
0.000905141 0.000836177 0.0202717 0.102804 ||| 0-1 1-0 2-0 3-0 |||
17.7303 0.791666 0.333333 ||| |||
Addings in ems cofig file for factor training :
input-factors = word mkcls
output-factors = word mkcls
alignment-factors = "word -> word"
translation-factors = "word+mkcls -> word+mkcls"
decoding-steps = "t0"
2015. 2. 4. 오후 9:07에 "Hieu Hoang" <[email protected]
<mailto:[email protected]>> 님이 작성:
try using the latest version of moses from github.
If it still fails, please make your model files available for
download so I can replicate the problem
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
On 4 February 2015 at 06:54, Lisa Kim <[email protected]
<mailto:[email protected]>> wrote:
Thank you.
It is successful to change the LM name without "=" symbol.
However, decoding still fails with following message.
Could you check the error message?
--------------------
Start loading text phrase table. Moses format : [0.038] seconds
terminate called recursively
terminate called after throwing an instance of 'util::Exception'
what(): Aborted (core dumped)
Exit code: 134
The decoder died. CONFIG WAS -weight-overwrite 'LM1= 0.125000
PhrasePenalty0= 0.050000 WordPenalty0= -0.250000
TranslationModel0= 0.050000 0.050000 0.050000 0.050000
TranslationModel1= 0.250000 LM0= 0.125000'
2015. 2. 3. 오후 8:12에 "Hieu Hoang" <[email protected]
<mailto:[email protected]>> 님이 작성:
as ken mentioned, don't use the = sign in the name of your
LM file. I think it's fixed in the latest version of
moses. Which version do you have, from when ago?
moses_chart should handle factors. Again, if you version
is from long ago, I suggest your update
On 02/02/15 10:06, ekkim214 wrote:
I want to use some factor (class) in the hierarchical model.
I am using the ems script and using KenLM for surface and
factor both.
The building of LM is successful with settings
"-discount_fallback" but
I got an error during tuning phase like this.
Exception: moses/LM/Ken.cpp:399 in Moses::LanguageModel*
Moses::ConstructKenLM(const string&) threw
util::Exception because `args.size() != 2'.
Incorrect format of KenLM property:
path=/home3/ekkim/working/E2K_1501/hierarchical_class/lm/travel22=mkcls.binlm.5
Exit code: 1
Does not "moses_chart" decoder which is distributed in
the package support factored training?
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
