Hi all,
I run the new version with the EMS. I get an error when I try a
hierarchical model. The configuration for the hierarchical model works
in the previous revision (r3284).
$SCRIPTS_ROOTDIR/ems/experiment.perl -config config.hierarchical
...ellipsis...
executing /home00/leona/IWSLT10.zh-en/steps/3/TUNING_tune.3 via
sh (1)
number of steps doable or running: 1
step TUNING:tune crashed
number of steps doable or running: 0
The running script of TUNING:tune is
/home/leona/moses-2010-08-09/bin/scripts-20100810-1203//training/mert-moses-new.pl
-mertdir=/home/leona/moses-2010-08-09/mert
/home00/leona/IWSLT10.zh-en/tuning/input.txt.1
/home00/leona/IWSLT10.zh-en/tuning/reference.tok.1.ref
/home/leona/moses-2010-08-09/moses-chart-cmd/src/moses_chart
/home00/leona/IWSLT10.zh-en/model/moses.ini.2 --nbest 100 --working-dir
/home00/leona/IWSLT10.zh-en/tuning/tmp.3 --decoder-flags " -v 0 " --rootdir
/home/leona/moses-2010-08-09/bin/scripts-20100810-1203/
--no-filter-phrase-table
In the STDERR of TUNING:tune,
After default: -l mem_free=0.5G -hard
Using
SCRIPTS_ROOTDIR: /home/leona/moses-2010-08-09/bin/scripts-20100810-1203/
Executing: mkdir -p /home00/leona/IWSLT10.zh-en/tuning/tmp.3
Saved: ./run1.moses.ini
Normalizing lambdas: 0 1 1 0.3 0.2 0.3 0.2 0 0.3
DECODER_CFG = -w %.6f -lm %.6f -d %.6f -tm %.6f %.6f %.6f %.6f
%.6f %.6f
values = 0 0.303030303030303 0.303030303030303
0.0909090909090909 0.0606060606060606 0.0909090909090909
0.0606060606060606 0 0.0909090909090909
Executing: /home/leona/moses-2010-08-09/moses-chart-cmd/src/moses_chart
-v 0 -config /home00/leona/IWSLT10.zh-en/model/moses.ini.2 -inputtype 0 -w
0.000000 -lm 0.303030 -d 0.303030 -tm 0.090909 0.060606 0.090909 0.060606
0.000000 0.090909 -n-best-list run1.best100.out 100 -i
/home00/leona/IWSLT10.zh-en/tuning/input.txt.1 > run1.out
/home00/leona/IWSLT10.zh-en/lm/iwslt10.lm.1: line 23: warning:
non-zero probability for <unk> in closed-vocabulary LM
Start loading new format pt model : [0.000] seconds
Start loading new format pt model : [70.000] seconds
[0..0]= [1..1]= [2..2]= [3..3]= [4..4]= [5..5]= [6..6]= [7..7]=
[8..8]= [9..9]= [10..10]= [11..11]= [12..12]= [13..13]=
[14..14]= [15..15]= [16..16]= [17..17]= [0..1]= [1..2]= [2..3]=
[3..4]= [4..5]= [5..6]= [6..7]= [7..8]= [8..9]= [9..10]=
[10..11]=
...ellipsis...
[10..11]= [11..12]= [12..13]= [13..14]= [0..2]= [1..3]= [2..4]=
[3..5]= [4..6]= [5..7]= [6..8]= [7..9]= [8..10]= [9..11]=
[10..12]= [11..13]= [12..14]= [0..3]= [1..4]= [2..5]= [3..6]=
[4..7]=Segmentation fault
Exit code: 139
The decoder died. CONFIG WAS -w 0.000000 -lm 0.303030 -d
0.303030 -tm 0.090909 0.060606 0.090909 0.060606 0.000000
0.090909
By the way, the new version works with EMS when I try a phrase-based
model with a lexicalized reordering model. Here are differences in
configurations.
$diff config.hierarchical config.moseslex
36c36
< decoder = $moses-src-dir/moses-chart-cmd/src/moses_chart
---
> decoder = $moses-src-dir/moses-cmd/src/moses
266c266
< #lexicalized-reordering = msd-bidirectional-fe
---
> lexicalized-reordering = msd-bidirectional-fe
270c270
< hierarchical-rule-set = true
---
> #hierarchical-rule-set = true
What is the problem?
--
Hwidong Na <[email protected]>
KLE lab, POSTECH, KOREA
2010-08-09 (월), 11:47 +0100, Hieu Hoang:
> Dear Mosers and Mosettes
>
> For your delectation and degustation, we are rolling out a new version
> of moses which adds a bunch of new features and fixed a few bugs too.
> Download at the usual location, or SVN update the trunk.
> https://sourceforge.net/projects/mosesdecoder/
>
> This is probably the last version to support only 1 model during
> decoding. Barry Haddow is waiting in the wings to merge changes into the
> trunk which alllows you to choose which phrase-table, LM, weights etc,
> to use at runtime. It's a big change so we will also tag this release in
> SVN as
> LAST-SINGLE-MODEL-ONLY-2010-08-09
>
> Hope you like it, let us know of any problems pdq.
>
> Hieu
>
> Changes since 24th April, 2010:
> 1. Add option of retaining alignment information in the phrase-based
> phrase table. Decoder loads this information if present. (Hieu Hoang &
> Raphael Payen)
> 2. When extracting rules, if the source or target syntax contains an
> unsupported
> escape sequence (anything other than "<", ">", "&", "&apos",
> and """) then write a warning message and skip the sentence pair
> (instead of asserting).
> 3. In bootstrap-hypothesis-difference-significance.pl, calculates the
> p-value and confidence intervals not only using BLEU, but also the NIST
> score. (Mark Fishel)
> 4. Dynamic Suffix Arrays (Abbey levenberg)
> 5. Merge multithreaded moses into moses (Barry Haddow)
> 6. Continue partial translation (Ondrej Bojar and Ondrej Odchazel)
> 7. Bug fixes, minor bits & bobs. (Philipp Koehn, Christian Hardmeier,
> Hieu Hoang, Barry Haddow, Phil Williams, Ondrej Bojar, Abbey, Mark
> Mishel, Lane Schwartz, Nicola Bertoldi, Raphael, ...)
>
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
