Try using the script trainlm-irst2.perl
Look at the config file in the example directory in Moses, or in the premade models that you can download with v1, eg http://www.statmt.org/moses/RELEASE-1.0/models/fr-en/config.pb On 1 May 2013 20:59, Zai Sarkar <[email protected]> wrote: > ###These Baseline commands for IRSTLM work fine using ver 1.0 Moses. A > good LM file is generated: > ----------Baseline script is ok > cd ../../apps/moses/mosesInstalls > export IRSTLM=/apps/moses/mosesInstalls/irstlm > #Generate the LM file > /apps/moses/mosesInstalls/irstlm/bin/add-start-end.sh < > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.clean.en > > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.sb.en export > IRSTLM=$HOME/mosesInstalls/irstlm; > /apps/moses/mosesInstalls/irstlm/bin/build-lm.sh -i > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.sb.en -n 5 -k 10 -t > ./moses/tmp -p -s improved-kneser-ney -o > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.lm.en > # CREATE THE ARPA FILE > /apps/moses/mosesInstalls/irstlm/bin/compile-lm --text yes > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.lm.en.gz > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.arpa.en > # BINARIZE THE ARPA FILE > /apps/moses/mosesInstalls/mosesdecoder/bin/build_binary -i -p 1.5 probing > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.arpa.en > /apps/moses/fr-en/parallel-in/training/Europarl3.fr-en.blm.en > > -----------EMS script is NOT ok > ###Trying to do same with EMS an LM as below, but it generates a temporary > irstlm-build-tmp directory with stat4/ dict and ngram .gz files but no lm > fie is generated after the tmp fie gets deleted. > # irstlm > lm-training = "$moses-script-dir/generic/trainlm-irst.perl -cores $cores > -irst-dir $irstlm-dir -temp-dir $working-dir/lm" > settings = "" > # order of the language model > order = 5 > type = 1 > > ### Also tried adding below to EMS, but still no LM saved > # irstlm > lm-binarizer = $irstlm-dir/compile-lm > # kenlm, also set type to 8 > lm-binarizer = "$moses-bin-dir/build_binary -i" > type = 8 > --------------- > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
