Hi Hieu Thanks for the reply. But, after I added the colon there, it still have some problem in running. *When I tried to run directly on the note of server it shows below:*
(tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl -root-dir train -corpus /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out & [1] 26177 (tensorflow) aaronhan@grove2:~/Moses/working$ It produced a folder 'train' and sub-folder 'corpus' and the empty files "en.vcb.classes en.vcb.classes.cats zh.vcb.classes zh.vcb.classes.cats" (tensorflow) aaronhan@grove2:~/Moses/working$ ls slurm-23199.out slurm-23201.out *train* *train_Moses.job* slurm-23200.out slurm-23202.out training.out (tensorflow) aaronhan@grove2:~/Moses/working$ cd train/ (tensorflow) aaronhan@grove2:~/Moses/working/train$ ls *corpus* (tensorflow) aaronhan@grove2:~/Moses/working/train$ ls corpus/ en.vcb.classes en.vcb.classes.cats zh.vcb.classes zh.vcb.classes.cats (tensorflow) aaronhan@grove2:~/Moses/working/train$ *When I submitted this command as job file to server, it does not appear in the job list/ doesn't to look like running.* Do you have any clue for this, please? Kind regards, On Tue, Jul 30, 2019 at 8:45 AM Hieu Hoang <[email protected]> wrote: > > > On Mon, 29 Jul 2019, 19:58 Aaron Poet, <[email protected]> wrote: > >> Hi Moses-Support >> >> I firstly build the lm as below: >> >> (base) aaronhan@grove2:~/Moses$ head -20 >> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en >> >> \data\ >> >> ngram 1=553697 >> >> ngram 2=8790963 >> >> ngram 3=28838697 >> >> >> \1-grams: >> >> -6.993945 <unk> 0 >> >> 0 <s> -1.6929523 >> >> -2.0611665 </s> 0 >> >> -2.3942525 the -0.9461817 >> >> -3.6470015 show -0.6162386 >> >> -4.1808434 stars -0.5792577 >> >> -4.1823444 X -0.53191173 >> >> -5.0213814 Girls -0.33374956 >> >> -2.5064495 - -0.6326866 >> >> -2.5754578 a -0.9741243 >> >> -5.1265607 troupe -0.3527482 >> >> -2.2843323 of -0.91076595 >> >> -4.663717 talented -0.3983 >> >> -5.2061567 topless -0.27003714 >> >> Then, I binary it as bellow: >> >> /home/aaronhan/Moses/mosesdecoder/bin/build_binary >> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en >> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >> >> (base) aaronhan@grove2:~/Moses$ ls -lh >> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >> >> -rw-r--r-- 1 aaronhan users 715M Jul 29 18:38 >> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >> >> when I test it, it looks working out: >> >> (tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English >> sentence ?" \ >> >> > | /home/aaronhan/Moses/mosesdecoder/bin/query >> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >> >> is=23 2 -2.5766807 this=321 3 -1.0483611 an=108 3 -2.1032705 >> English=1731 2 -2.8970845 sentence=3266 3 -2.8317716 ?=815 2 >> -2.0824053 </s>=2 3 -0.035550244 Total: -13.575124 OOV: 0 >> >> Perplexity including OOVs: 86.9567697150713 >> >> Perplexity excluding OOVs: 86.9567697150713 >> >> OOVs: 0 >> >> Tokens: 7 >> >> Name:query VmPeak:748300 kB VmRSS:4052 kB RSSMax:503644 kB >> user:0.008 sys:0.652 CPU:0.66 real:61.9797 >> >> However, when I run the overall Moses training, it produced error as >> below: *Wrong format of --lm. Expected: --lm factor:order:filename* >> >> (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice >> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl >> -root-dir train -corpus >> /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en >> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm >> 0:3$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir >> /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out & >> >> [1] 23346 >> >> You forgot a colon between 3 and $HOME > >> >> GNU nano 2.7.4 >> File: training.out >> >> >> >> nohup: ignoring input >> >> Using SCRIPTS_ROOTDIR: /home/aaronhan/Moses/mosesdecoder/scripts >> >> Using single-thread GIZA >> >> using gzip >> >> ERROR: Wrong format of --lm. Expected: --lm factor:order:filename at >> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl line >> 605. >> >> >> >> Can you let me know how to solve this, please? >> >> Kind regards, >> Aaron >> >> -- >> >> PhD student, Msc. (excellent), Bsc. (Maths) >> >> Google-Scholar >> <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> , >> Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides, >> Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han> >> >> Google-site <https://sites.google.com/view/poetgarden/home> Linkedin >> <https://www.linkedin.com/in/aaronhan/>, >> >> ADAPT <https://www.adaptcentre.ie/> ResearchCentre & >> >> School of Computing, DCU, >> >> Dublin, Ireland >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > -- PhD student, Msc. (excellent), Bsc. (Maths) Google-Scholar <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> , Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides, Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han> Google-site <https://sites.google.com/view/poetgarden/home> Linkedin <https://www.linkedin.com/in/aaronhan/>, ADAPT <https://www.adaptcentre.ie/> ResearchCentre & School of Computing, DCU, Dublin, Ireland
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
