On Mon, 29 Jul 2019, 19:58 Aaron Poet, <[email protected]> wrote:
> Hi Moses-Support > > I firstly build the lm as below: > > (base) aaronhan@grove2:~/Moses$ head -20 > /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en > > \data\ > > ngram 1=553697 > > ngram 2=8790963 > > ngram 3=28838697 > > > \1-grams: > > -6.993945 <unk> 0 > > 0 <s> -1.6929523 > > -2.0611665 </s> 0 > > -2.3942525 the -0.9461817 > > -3.6470015 show -0.6162386 > > -4.1808434 stars -0.5792577 > > -4.1823444 X -0.53191173 > > -5.0213814 Girls -0.33374956 > > -2.5064495 - -0.6326866 > > -2.5754578 a -0.9741243 > > -5.1265607 troupe -0.3527482 > > -2.2843323 of -0.91076595 > > -4.663717 talented -0.3983 > > -5.2061567 topless -0.27003714 > > Then, I binary it as bellow: > > /home/aaronhan/Moses/mosesdecoder/bin/build_binary > /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en > /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en > > (base) aaronhan@grove2:~/Moses$ ls -lh > /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en > > -rw-r--r-- 1 aaronhan users 715M Jul 29 18:38 > /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en > > when I test it, it looks working out: > > (tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English > sentence ?" \ > > > | /home/aaronhan/Moses/mosesdecoder/bin/query > /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en > > is=23 2 -2.5766807 this=321 3 -1.0483611 an=108 3 -2.1032705 > English=1731 2 -2.8970845 sentence=3266 3 -2.8317716 ?=815 2 > -2.0824053 </s>=2 3 -0.035550244 Total: -13.575124 OOV: 0 > > Perplexity including OOVs: 86.9567697150713 > > Perplexity excluding OOVs: 86.9567697150713 > > OOVs: 0 > > Tokens: 7 > > Name:query VmPeak:748300 kB VmRSS:4052 kB RSSMax:503644 kB > user:0.008 sys:0.652 CPU:0.66 real:61.9797 > > However, when I run the overall Moses training, it produced error as > below: *Wrong format of --lm. Expected: --lm factor:order:filename* > > (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice > /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl > -root-dir train -corpus > /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en > -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm > 0:3$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir > /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out & > > [1] 23346 > > You forgot a colon between 3 and $HOME > > GNU nano 2.7.4 > File: training.out > > > > nohup: ignoring input > > Using SCRIPTS_ROOTDIR: /home/aaronhan/Moses/mosesdecoder/scripts > > Using single-thread GIZA > > using gzip > > ERROR: Wrong format of --lm. Expected: --lm factor:order:filename at > /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl line > 605. > > > > Can you let me know how to solve this, please? > > Kind regards, > Aaron > > -- > > PhD student, Msc. (excellent), Bsc. (Maths) > > Google-Scholar > <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> , > Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides, > Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han> > > Google-site <https://sites.google.com/view/poetgarden/home> Linkedin > <https://www.linkedin.com/in/aaronhan/>, > > ADAPT <https://www.adaptcentre.ie/> ResearchCentre & > > School of Computing, DCU, > > Dublin, Ireland > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
