Hi Moses-Support
I firstly build the lm as below:
(base) aaronhan@grove2:~/Moses$ head -20
/home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
\data\
ngram 1=553697
ngram 2=8790963
ngram 3=28838697
\1-grams:
-6.993945 <unk> 0
0 <s> -1.6929523
-2.0611665 </s> 0
-2.3942525 the -0.9461817
-3.6470015 show -0.6162386
-4.1808434 stars -0.5792577
-4.1823444 X -0.53191173
-5.0213814 Girls -0.33374956
-2.5064495 - -0.6326866
-2.5754578 a -0.9741243
-5.1265607 troupe -0.3527482
-2.2843323 of -0.91076595
-4.663717 talented -0.3983
-5.2061567 topless -0.27003714
Then, I binary it as bellow:
/home/aaronhan/Moses/mosesdecoder/bin/build_binary
/home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
/home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
(base) aaronhan@grove2:~/Moses$ ls -lh
/home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
-rw-r--r-- 1 aaronhan users 715M Jul 29 18:38
/home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
when I test it, it looks working out:
(tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English
sentence ?" \
> | /home/aaronhan/Moses/mosesdecoder/bin/query
/home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
is=23 2 -2.5766807 this=321 3 -1.0483611 an=108 3 -2.1032705
English=1731 2 -2.8970845 sentence=3266 3 -2.8317716 ?=815 2
-2.0824053 </s>=2 3 -0.035550244 Total: -13.575124 OOV: 0
Perplexity including OOVs: 86.9567697150713
Perplexity excluding OOVs: 86.9567697150713
OOVs: 0
Tokens: 7
Name:query VmPeak:748300 kB VmRSS:4052 kB RSSMax:503644 kB
user:0.008 sys:0.652 CPU:0.66 real:61.9797
However, when I run the overall Moses training, it produced error as below:
*Wrong format of --lm. Expected: --lm factor:order:filename*
(tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice
/home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl
-root-dir train -corpus
/home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en
-alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
0:3$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir
/home/aaronhan/Moses/mosesdecoder/tools/ >& training.out &
[1] 23346
GNU nano 2.7.4
File: training.out
nohup: ignoring input
Using SCRIPTS_ROOTDIR: /home/aaronhan/Moses/mosesdecoder/scripts
Using single-thread GIZA
using gzip
ERROR: Wrong format of --lm. Expected: --lm factor:order:filename at
/home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl line
605.
Can you let me know how to solve this, please?
Kind regards,
Aaron
--
PhD student, Msc. (excellent), Bsc. (Maths)
Google-Scholar <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> ,
Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides,
Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han>
Google-site <https://sites.google.com/view/poetgarden/home> Linkedin
<https://www.linkedin.com/in/aaronhan/>,
ADAPT <https://www.adaptcentre.ie/> ResearchCentre &
School of Computing, DCU,
Dublin, Ireland
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
