the fuzzy match rule table hasn't been used for a while. I'm not sure if
it still works anymore
On 10/09/2015 13:05, Álvaro Peris wrote:
Hi,
I've been trying to test the Fuzzy Match Rule Table for hierarchical
models, but when I run MERT, I only get syntax errors. I'm fairly new
in hierarchical models, I think my moses.ini config is correct, but I
may be doing something wrong. If I don't use the
PhraseDictionaryFuzzyMatch, it works properly.
I saw a similar thread
(http://permalink.gmane.org/gmane.comp.nlp.moses.user/12239), but I
can't even reach the translation process, it crashes before.
I'd appreciate any help.
This is the trace of the crash:
(6.6) consolidating the two halves @ Thu Sep 10 12:59:50 CEST 2015
Executing:
/home/lvapeab/smt/software/mosesdecoder/scripts/../bin/consolidate
/tmp/moses.XQR9TF/fuzzyMatchFile.pt.half.f2e.gz
/tmp/moses.XQR9TF/fuzzyMatchFile.pt.half.e2f.gz /dev/stdout
--Hierarchical | gzip -c > /tmp/moses.XQR9TF/fuzzyMatchFile.pt.gz
Consolidate v2.0 written by Philipp Koehn
consolidating direct and indirect rule tables
processing hierarchical rules
Executing: rm -f /tmp/moses.XQR9TF/fuzzyMatchFile.pt.half.*
Start loading fuzzy-match phrase model : [1.245] seconds
terminate called after throwing an instance of 'util::Exception'
what():
moses/TranslationModel/RuleTable/PhraseDictionaryFuzzyMatch.cpp:235
in virtual void
Moses::PhraseDictionaryFuzzyMatch::InitializeForInput(const
ttasksptr&) threw util::Exception'.
Syntax error at /tmp/moses.XQR9TF/fuzzyMatchFile.pt.gz:0
Aborted (core dumped)
Exit code: 134
The decoder died. CONFIG WAS -weight-overwrite 'TranslationModel0=
0.057143 0.057143 0.057143 0.057143 TranslationModel1= 0.285714
LM0= 0.142857 WordPenalty0= -0.285714 PhrasePenalty0= 0.057143'
The file which gives the syntax error
(/tmp/moses.2pkyhw/fuzzyMatchFile.pt.gz) contains:
nos hemos de marchar el jueves cinco de enero por la tarde . [X]
||| should leave on Thursday January the fifth in the afternoon .
[X] ||| -nan -nan ||| 0-0 1-0 1-1 10-9 11-10 2-1 3-1 3-2 4-3 5-4
5-6 6-7 7-5 7-8 8-5 9-8 ||| 0 0 0 ||| |||
(Maybe it fails because of the -nan?)
Finally, my moses.ini:
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
1 T 1
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home/lvapeab/smt/tasks/turista/esen/fuzzy/model/rule-table.gz
input-factor=0
output-factor=0
PhraseDictionaryMemory name=TranslationModel1 num-features=1
path=/home/lvapeab/smt/tasks/turista/esen/fuzzy/model/glue-grammar
input-factor=0 output-factor=0 tuneable=true
KENLM name=LM0 factor=0
path=/home/lvapeab/smt/tasks/turista/esen/LM/5gram.en order=5
PhraseDictionaryFuzzyMatch
source=/home/lvapeab/smt/tasks/turista/DATA/training.es
<http://training.es>
target=/home/lvapeab/smt/tasks/turista/DATA/training.en
alignment=/home/lvapeab/smt/tasks/turista/esen/fuzzy/model/aligned.grow-diag-final-and
num-features=0
# dense weights for feature functions
[weight]
# The default weights are NOT optimized for translation quality.
You MUST tune the weights.
# Documentation for tuning is here:
http://www.statmt.org/moses/?n=FactoredTraining.Tuning
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
TranslationModel1= 1
LM0= 0.5
Thank you in advance.
Cheers,
--
Álvaro.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
