Hi Zai If language model building failed in EMS, then there should be some error messages in
steps/$id/LM_$corpus_train.$id.STDERR where $id is the experiment id and $corpus is the name of the corpus. If it's not clear what went wrong, could you post this log file to the list? cheers - Barry On 01/05/13 20:59, Zai Sarkar wrote: > ###These Baseline commands for IRSTLM work fine using ver 1.0 Moses. A > good LM file is generated: > ----------Baseline script is ok > cd ../../apps/moses/mosesInstalls > export IRSTLM=/apps/moses/mosesInstalls/irstlm > #Generate the LM file > /apps/moses/mosesInstalls/irstlm/bin/add-start-end.sh < > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.clean.en > > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.sb.en export > IRSTLM=$HOME/mosesInstalls/irstlm; > /apps/moses/mosesInstalls/irstlm/bin/build-lm.sh -i > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.sb.en -n 5 -k > 10 -t ./moses/tmp -p -s improved-kneser-ney -o > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.lm.en > # CREATE THE ARPA FILE > /apps/moses/mosesInstalls/irstlm/bin/compile-lm --text yes > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.lm.en.gz > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.arpa.en > # BINARIZE THE ARPA FILE > /apps/moses/mosesInstalls/mosesdecoder/bin/build_binary -i -p 1.5 > probing /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.arpa.en > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.blm.en > > -----------EMS script is NOT ok > ###Trying to do same with EMS an LM as below, but it generates a > temporary irstlm-build-tmp directory with stat4/ dict and ngram .gz > files but no lm fie is generated after the tmp fie gets deleted. > # irstlm > lm-training = "$moses-script-dir/generic/trainlm-irst.perl -cores > $cores -irst-dir $irstlm-dir -temp-dir $working-dir/lm" > settings = "" > # order of the language model > order = 5 > type = 1 > > ### Also tried adding below to EMS, but still no LM saved > # irstlm > lm-binarizer = $irstlm-dir/compile-lm > # kenlm, also set type to 8 > lm-binarizer = "$moses-bin-dir/build_binary -i" > type = 8 > --------------- > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
