The moses.ini file looks ok. The segfault is difficult to debug by inspection. It's best sprinkle the decoder code with debugging message and run 1 instance of the decoder to see exactly where it goes wrong. And you should binarise the pt and reordering models to save yourself time
* Looking for MT/NLP opportunities * Hieu Hoang http://moses-smt.org/ On 6 April 2017 at 11:30, Alexander Erdmann <[email protected]> wrote: > For reference, here is the output from the segFault: > > Defined parameters (per moses.ini or switch): > config: /scratch/ae1541/EMNLP/seed/150000/iteration1/mosesTryingGZ.ini > distortion-limit: 6 > feature: UnknownWordPenalty WordPenalty PhrasePenalty > PhraseDictionaryMemory name=TranslationModel0 num-features=4 > path=/scratch/ae1541/EMNLP/seed/150000/iteration0/phrase-table.gz > LexicalReordering name=LexicalReordering0 num-features=6 > type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 > path=/scratch/ae1541/EMNLP/seed/150000/iteration0/reordering-table.gz > Distortion KENLM name=LM0 factor=0 > path=/scratch/ae1541/EMNLP/seed/LM/LM.lev.lm > order=5 > input-factors: 0 > input-file: seed/150000/iteration1/del00.egy > mapping: 0 T 0 > threads: 12 > weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 > TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 > 0.3 Distortion0= 0.3 LM0= 0.5 > line=UnknownWordPenalty > FeatureFunction: UnknownWordPenalty0 start: 0 end: 0 > line=WordPenalty > FeatureFunction: WordPenalty0 start: 1 end: 1 > line=PhrasePenalty > FeatureFunction: PhrasePenalty0 start: 2 end: 2 > line=PhraseDictionaryMemory name=TranslationModel0 num-features=4 > path=/scratch/ae1541/EMNLP/seed/150000/iteration0/phrase-table.gz > FeatureFunction: TranslationModel0 start: 3 end: 6 > line=LexicalReordering name=LexicalReordering0 num-features=6 > type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 > path=/scratch/ae1541/EMNLP/seed/150000/iteration0/reordering-table.gz > FeatureFunction: LexicalReordering0 start: 7 end: 12 > Initializing LexicalReordering.. > line=Distortion > FeatureFunction: Distortion0 start: 13 end: 13 > line=KENLM name=LM0 factor=0 path=/scratch/ae1541/EMNLP/seed/LM/LM.lev.lm > order=5 > FeatureFunction: LM0 start: 14 end: 14 > Loading the LM will be faster if you build a binary file. > Reading /scratch/ae1541/EMNLP/seed/LM/LM.lev.lm > ----5---10---15---20---25---30---35---40---45---50---55--- > 60---65---70---75---80---85---90---95--100 > ************************************************************ > **************************************** > Loading UnknownWordPenalty0 > Loading WordPenalty0 > Loading PhrasePenalty0 > Loading LexicalReordering0 > Loading table into memory...done. > Loading Distortion0 > Loading LM0 > Loading TranslationModel0 > Start loading text phrase table. Moses format : [1574.737] seconds > Reading /scratch/ae1541/EMNLP/seed/150000/iteration0/phrase-table.gz > ----5---10---15---20---25---30---35---40---45---50---55--- > 60---65---70---75---80---85---90---95--100 > ************************************************************ > **************************************** > Created input-output object : [1608.439] seconds > Translating: Translating: A y Translating: jAC Translating: Translating: > AbwTranslating: RRR w hA Al mAd mrwAn > sAdRRR jrydp A m trtr . . jrydp ywmy p . . bTyx yp . . mhlb yp bAl > tEycTranslating: w nA Al byAn Al tAly p > Line 1: Initialize search took Line 8: Initialize search took tqly p 0.006 > seconds total > 0.005 seconds total > Translating: t h y Translating: qwqAl A b mSr RRR rytc bAl Al dAxly p y > AxbAr ? > Line 9: Initialize search took 0.007 seconds total > w b h mfyc RRR > AxwAn hm Hd m Al mslmyn AnDrb > Line 11: Initialize search took Line 0: Initialize search took Al0.010 > seconds total > Hrp 0.009 seconds total > Al Translating: AfAd Al mhlb jrydpmtHdv Al rsmy l wzArp Al nSAry bEd > Translating: yp Al m m A y h Avl Al dAxly w > An p AfAdt Line 7: Initialize search took qyAd ( Line xyr p ll TbE > Al mSlHp 8: Collecting options took Line 2: Initialize search took lhm Al > 0.005 seconds at moses/Manager.cpp:AjElh bHv 0.015 seconds total > 0.013 seconds total > wrAC Amn xyr mvyr Al Al y dwlp tlyfzywn Al sAbq Al mSry A ) Almbjl ytHdv b > Al An lsAn jnAQy h jrydp An bEd Al ftn A tHry w m w Al trtr Al bHv w mtTrf > f yTlb yn y mn Al AHdAv Al mslmyn > hm Line 6: Initialize search took 117 > nA 0.020 seconds total > s *** bSegmentation fault > Register dump: > > RAX: 0000000000000000 RBX: 00000000012e28e0 RCX: 0000000000000000 > RDX: 00007f91b7413010 RSI: 0000000000000000 RDI: 00007f917400b9c0 > RBP: 0000000000000000 R8 : 0000000000000002 R9 : 0000000000000038 > R10: 00007f91aa8f3310 R11: 00007f91b64cfbf0 R12: 00007f917400b028 > R13: 00007f9174004f60 R14: 00007f917400b9c0 R15: 0000000000000002 > RSP: 00007f91aa8f3730 > > RIP: 00007f91b66e8833 EFLAGS: 00010246 > > CS: 0033 FS: 0000 GS: 0000 > > Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000010 > > FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000 > RIP: 00000000 RDP: 00000000 > > ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000 > ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000 > ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000 > ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000 > mxcsr: 1fa4 > XMM0: 00000000000000000000000000000000 XMM1: > 00000000000000000000000000000000 > XMM2: 00000000000000000000000000000000 XMM3: > 00000000000000000000000000000000 > XMM4: 00000000000000000000000000000000 XMM5: > 00000000000000000000000000000000 > XMM6: 00000000000000000000000000000000 XMM7: > 00000000000000000000000000000000 > XMM8: 00000000000000000000000000000000 XMM9: > 00000000000000000000000000000000 > XMM10: 00000000000000000000000000000000 XMM11: > 00000000000000000000000000000000 > XMM12: 00000000000000000000000000000000 XMM13: > 00000000000000000000000000000000 > XMM14: 00000000000000000000000000000000 XMM15: > 00000000000000000000000000000000 > > Backtrace: > y Sdd lA tjmyE hyj rAs wA AHd/share/apps/NYUAD/ > mosesdecoder/3.0/lib/libmoses.so(_ZNK5Moses16LanguageModelKenIN2 > lm5ngram12ProbingModelEE19EvaluateWhenAppliedERKNS_10HypothesisEPKNS_ > 7FFStateEPNS_24ScoreComponentCollectionE+0xa3)[0x7f91b66e8833] > Ely/share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ > ZN5Moses10Hypothesis19EvaluateWhenAppliedERKNS_12SquareMatrixE+0xf0)[ > 0x7f91b64cfab0] > Al /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ > ZN5Moses12SearchNormal16ExpandHypothesisERKNS_10HypothesisERKNS_ > 17TranslationOptionEf+0x165)[0x7f91b65484a5] > bED /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ > ZN5Moses12SearchNormal19ExpandAllHypothesesERKNS_10HypothesisEmm+0x79)[ > 0x7f91b6548829] > > mALine /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ > ZN5Moses12SearchNormal20ProcessOneHypothesisERKNS_10HypothesisE+0x34b)[ > 0x7f91b6548beb] > 10: Initialize search took /share/apps/NYUAD/ > mosesdecoder/3.0/lib/libmoses.so(_ZN5Moses12SearchNormal6DecodeE > v+0x11c)[0x7f91b6548ffc] > DHAyA/share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses. > so(_ZN5Moses7Manager6DecodeEv+0xe3)[0x7f91b64f9173] > sbyrw /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ > ZN5Moses15TranslationTask3RunEv+0x1a5)[0x7f91b656f505] > f /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ > ZN5Moses10ThreadPool7ExecuteEv+0x89)[0x7f91b6724b99] > ./share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_thread.so.1.57.0(+0x1211a)[0x7f91b54ee11a] > /usr/lib64/libpthread.so.0(+0x7dc5)[0x7f91b489cdc5] > y /usr/lib64/libc.so.6(clone+0x6d)[0x7f91b45ca28d] > . > Memory map: > > tlk 00400000-0040b000 r-xp 00000000 00:2b 1254532671 > /share/apps/NYUAD3/mosesdecoder/3.0/bin/moses > 0060b000-0060c000 rw-p 0000b000 00:2b 1254532671 > /share/apps/NYUAD3/mosesdecoder/3.0/bin/moses > 012ce000-52974a000qd rw-p 00000000 00:00 0 > [heap] > 7f9174000000-7f9174021000 rw-p 00000000 00:00 0 > 7f9174021000-7f9178000000 ---p 00000000 00:00 0 > 7f9178000000-7f917807b000 rw-p 00000000 00:00 0 > 7f917807b000-7f917c000000 ---p 00000000 00:00 0 > 7f917c000000-7f917c034000 rw-p 00000000 00:00 0 > 7f917c034000-7f9180000000 ---p 00000000 00:00 0 > 7f9184000000-7f918409e000 rw-p 00000000 00:00 0 > 7f918409e000-7f9188000000 ---p 00000000 00:00 0 > 7f9188000000-7f9188047000 rw-p 00000000 00:00 0 > 7f9188047Al000-7f918c000000 ---p 00000000 00:00 0 > 7f918c000000-7f918c23d000 rw-p 00000000 00:00 0 > 7f918c23d000-7f9190000000 ---p 00000000 00:00 0 > 7f9190000000-7f9190106000 rw-p 00000000 00:00 0 > 7f9190106000-7f9194000000 ---p 00000000 00:00 0 > 7f9194000000-7f91940 c7000 rw-p 00000000 00:00 0 > 7f91940c9000-7f9198000000 ---p 00000000 00:00 0 > 7f9198000000-7f9198106000 rw-p 00000000 00:00 0 > 7f9198106000-7f919c000000 ---p 00000000 00:00 0 > 7f919c000000-7f919c165000 rw-p 00000000 00:00 0 > 7f919c165000-7f91a0000000 ---p tbyn00000000 00:00 0 > 7f919c167000-7f91a0000000 ---p 00000000 00:00 0 > 7f91a0000000-7f91a009f000 rw-p 00000000 00:00 0 > 7f91a009f000-7f91a4000000 ---p 00000000 00:00 0 > 7f91a4000000-7f91a4047000 rw-p 00000000 00:00 0 > 7f91a4047000-7f91a8000000 ---p 00000000 00 :00 0 > 7f91a80f0000-7f91a80f1000 ---p 00000000 00:00 0 > 7f91a80f1000-7f91a88f1000 rw-p 00000000 00:00 0 > [stack:10106] > 7f91a88f1000-7f91a88f2000 ---p 00000000 00:00 0 > 7f91a88f2000-7f91a90f2000 rw-p 00000000 00:00 0 AHdAv > [stack:10105] > 7f91a90f2000-7f91a90f3000 ---p 00000000 00:00 0 > 7f91a90f3000-7f91a98f3000 rw-p 00000000 00:00 0 > [stack:10104] > 7f91a98f3000-7f91a98f4000 ---p 00000000 00:00 0 > 7f91a98f4000-7f91aa0f4000 rw-p 00000000 00:00 0 > [stack:10103] > 7f91aa0f4000-7f91aa0f5000 ---p 00000000 00:00 0 > 7f91aa0f5000-7f91aa8f5000 rw-p 00000000 00:00 0 > [stack:10102] > 7f91aa8f5000-7f91aa8f6000 ---p 00000000 00:00 0 > 7f91aa8f6000-7f91ab0f6000 rw-p 0lnA0000000 00:00 0 > [stack:10101] > 7f91ab0f6000-7f91ab0f7000 ---p 00000000 00:00 0 > 7f91ab0f7000-7f91ab8f7000 rw-p 00000000 00:00 0 > [stack:10100] > 7f91ab8f7000-7f91ab8f8000 ---p 00000000 00:00 0 > 7f91ab8f8000-7f9 1ac0f8000 rw-p 00000000 00:00 0 > [stack:10099] > 7f91ac0f8000-7f91ac0f9000 ---p 00000000 00:00 0 > 7f91ac0f9000-7f91ae8fa000 rw-p 00000000 00:00 0 > [stack:10098] > 7f91aef80000-7f91aef81000 ---p 00000000 00:00 0 > w7f91aef81000-7f91af781000 rw-p 00000000 00:00 0 > [stack:10097] > 7f91af781000-7f91af782000 ---p 00000000 00:00 0 > 7f91af782000-7f91aff82000 rw-p 00000000 00:00 0 > [stack:10096] > 7f91aff82000-7f91aff83000 ---p 00 000000 00:00 0 > 7f91aff83000-7f91b1784000 rw-p 00000000 00:00 0 > [stack:10095] > 7f91b1886000-7f91b18c7000 rw-p 00000000 00:00 0 > 7f91b18c7000-7f91b409f000 rw-p 00000000 00:00 0 > 7f91b409f000-7f91b40ae000 r-xp 00000000 00:12 54468 An > /usr/lib64/libbz2.so.1.0.6 > 7f91b40ae000-7f91b42ad000 ---p 0000f000 00:12 54468 > /usr/lib64/libbz2.so.1.0.6 > 7f91b42ad000-7f91b42ae000 r--p 0000e000 00:12 54468 > /usr/lib64/libbz2.so.1.0.6 > 7f91b42ae00 0-7f91b42af000 rw-p 0000f000 00:12 54468 > /usr/lib64/libbz2.so.1.0.6 > 7f91b42af000-7f91b42d3000 r-xp 00000000 00:12 54446 > /usr/lib64/liblzma.so.5.0.99 > 7f91b42d3000-7f91b44d2000 ---p 00024000 00:12 54446 AlVy > /usr/lib64/liblzma.so.5.0.99 > 7f91b44d2000-7f91b44d3000 r--p 00023000 00:12 54446 > /usr/lib64/liblzma.so.5.0.99 > 7f91b44d3000-7f91b44d4000 rw-p 00024000 00:12 54446 > /usr/lib64/liblzma.so.5.0.99 > 7f91b44d4000-7f 91b468a000 r-xp 00000000 00:12 54398 > /usr/lib64/libc-2.17.so > 7f91b468a000-7f91b488a000 ---p 001b6000 00:12 54398 > /usr/lib64/libc-2.17.so > 7f91b488a000-7f91b488e000 r--p 001b6000 00:12 54398 > /usrAl/lib64/libc-2.17.so > 7f91b488e000-7f91b4890000 rw-p 001ba000 00:12 54398 > /usr/lib64/libc-2.17.so > 7f91b4890000-7f91b4895000 rw-p 00000000 00:00 0 > 7f91b4895000-7f91b48ab000 r-xp 00000000 00:12 54451 > /usr/lib64/libpth read-2.17.so > 7f91b48ab000-7f91b4aab000 ---p 00016000 00:12 54451 > /usr/lib64/libpthread-2.17.so > 7f91b4aab000-7f91b4aac000 r--p 00016000 00:12 54451 > /usr/lib64/libpthread-2.17.so > 7f91b4aac000-7f91b4aad000 rw-p 000170t00 00:12 54451 > /usr/lib64/libpthread-2.17.so > 7f91b4aad000-7f91b4ab1000 rw-p 00000000 00:00 0 > 7f91b4ab1000-7f91b4ac7000 r-xp 00000000 00:2b 390181951 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown-linux-gn > u/lib64/libgcc_s.so.1 > 7f91b4ac7000-7f91b4cc6000 ---p 00016000 00:2b 390181951 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknown-linux-gnu/lib64/libgcc_s.so.1 > 7f91b4cc6000-7f91b4cc7000 r--p 00015000 00:2b 390181951 > slfyyn /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknown-linux-gnu/lib64/libgcc_s.so.1 > 7f91b4cc7000-7f91b4cc8000 rw-p 00016000 00:2b 390181951 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknown-linux-gnu/lib64/libgcc_s.so.1 > 7 f91b4cc8000-7f91b4dc9000 r-xp 00000000 00:12 54414 > /usr/lib64/libm-2.17.so > 7f91b4dc9000-7f91b4fc8000 ---p 00101000 00:12 54414 > /usr/lib64/libm-2.17.so > 7f91b4fc8000-7f91b4fc9000 r--p 00100000 00:12 54414 bdw > /usr/lib64/libm-2.17.so > 7f91b4fc9000-7f91b4fca000 rw-p 00101000 00:12 54414 > /usr/lib64/libm-2.17.so > 7f91b4fca000-7f91b50b5000 r-xp 00000000 00:2b 3536542694 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknown-linux-gnu/4.9.3/libstdc++.so.6.0.20 > 7f91b50b5000-7f91b52b5000 ---p 000eb000 00:2b 3536542694 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknown-linux-gnu/4.9.3/libstdc++.so.6.0.20 > 7f91b52b5000-7f91b52bd000 r--p 000eb000 00:hm 2b 3536542694 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknown-linux-gnu/4.9.3/libstdc++.so.6.0.20 > 7f91b52bd000-7f91b52bf000 rw-p 000f3000 00:2b 3536542694 > /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- > unknomlAmHwn-linux-gnu/4.9.3/libstdc++.so.6.0.20 > 7f91b52bf000-7f91b52d4000 rw-p 00000000 00:00 0 > 7f91b52d4000-7f91b52db000 r-xp 00000000 00:12 54459 > /usr/lib64/librt-2.17.so > 7f91b52db000-7f91b54da000 ---p 00007000 00:12 54459 > /usr/lib64/librt-2.17.so > 7f91b54da000-7f91b54db000 r--p 00006000 00:12 54459 > /usr/lib64/librt-2.17.so > 7f91b54db000-7f91b54dc000 rw-p 00007000 00:12 54459 > /usr/lib64/librt-2.17.so > 7f91b54dc000-7f91b54fd000 r-xp 0mn0000000 00:2b 189775209 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_thread.so.1.57.0 > 7f91b54fd000-7f91b56fc000 ---p 00021000 00:2b 189775209 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/ > avx2/1.57.0/lib/libboost_thread.so.1.57.0 > 7f91b56fc000-7f91b56ff000 rw-p 00020000 00:2b 189775209 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_thread.so.1.57.0 > 7f91b56ff000-7f91b5714000 r-xp 00000000 00:12 544h64 > /usr/lib64/libz.so.1.2.7 > 7f91b5714000-7f91b5913000 ---p 00015000 00:12 54464 > /usr/lib64/libz.so.1.2.7 > 7f91b5913000-7f91b5914000 r--p 00014000 00:12 54464 > /usr/lib64/libz.so.1.2.7 > 7f91b591400 0-7f91b5915000 rw-p 00015000 00:12 54464 > /usr/lib64/libz.so.1.2.7 > 7f91b5915000-7f91b5918000 r-xp 00000000 00:2b 189775207 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_system.so.1.57.0 > 7f91AcElb5918000-7f91b5b17000 ---p 00003000 00:2b 189775207 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_system.so.1.57.0 > 7f91b5b17000-7f91b5b18000 rw-p 00002000 00:2b 189775207 > /share/apps/NYUAD3/boo st/gcc_4.9.3/openmpi_1.10.2/ > avx2/1.57.0/lib/libboost_system.so.1.57.0 > 7f91b5b18000-7f91b5b2e000 r-xp 00000000 00:2b 189756212 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_iostreams.so.1.57.0 > 7f91b5b2e000-7f91bwADH5d2e000 ---p 00016000 00:2b 189756212 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_iostreams.so.1.57.0 > 7f91b5d2e000-7f91b5d30000 rw-p 00016000 00:2b 189756212 > /share/apps/NYUAD3/boost/gcc_4.9. 3/openmpi_1.10.2/avx2/1.57.0/ > lib/libboost_iostreams.so.1.57.0 > 7f91b5d30000-7f91b5d34000 r-xp 00000000 00:12 54392 > /usr/lib64/libSegFault.so > 7f91b5d34000-7f91b5f33000 ---p 00004000 00:12 54392 > /usr/lib64/libSegFaultwA.so > 7f91b5f33000-7f91b5f34000 r--p 00003000 00:12 54392 > /usr/lib64/libSegFault.so > 7f91b5f34000-7f91b5f35000 rw-p 00004000 00:12 54392 > /usr/lib64/libSegFault.so > 7f91b5f35000-7f91b5f38000 r-xp 00000000 00:12 54410 > /usr/lib64/libdl-2.17.so > 7f91b5f38000-7f91b6137000 ---p 00003000 00:12 54410 > /usr/lib64/libdl-2.17.so > 7f91b6137000-7f91b6138000 r--p 00002000 00:12 54410 > /usr/lib64/libdl-2.17.so > 7f91b6138000-7f90.0221b6139000 rw-p 00003000 00:12 54410 > /usr/lib64/libdl-2.17.so > 7f91b6139000-7f91b6150000 r-xp 00000000 00:2b 189756209 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_filesystem.so.1.57.0 > 7f91b seconds total6150000-7f91b634f000 ---p 00017000 00:2b 189756209 > /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1. > 57.0/lib/libboost_filesystem.so.1.57.0 > 7f91b634f000-7f91b6350000 rw-p 00016000 00:2b 189756209 > /share/apps/NYUAD3/ > boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57.0/lib/libboost_ > filesystem.so.1.57.0 > 7f91b6350000-7f91b6868000 r-xp 00000000 00:2b 2421011537 > /share/apps/NYUAD3/mosesdecoder/3.0/lib/libmoses.so > 7f91b6868000-7f91b6a67000 ---p 00518000 00:2b 24210115ftyl37 > /share/apps/NYUAD3/mosesdecoder/3.0/lib/libmoses.so > 7f91b6a67000-7f91b6a82000 rw-p 00517000 00:2b 2421011537 > /share/apps/NYUAD3/mosesdecoder/3.0/lib/libmoses.so > 7f91b6a82000-7f91b73b2000 rw-p 00000000 00:00 0 > 7f91b73b200 0-7f91b73d3000 r-xp 00000000 00:12 54384 > /usr/lib64/ld-2.17.so > 7f91b7413000-7f91b743b000 rw-p 00000000 00:00 0 > 7f91b745c000-7f91b74c6000 rw-p 00000000 00:00 0 > 7f91b75c7000-7f91b75d3000 rw-p 00000000 00:00 0 > 7f91b75d3000-7f91b75d4000Line r--p 00021000 00:12 54384 > /usr/lib64/ld-2.17.so > 7f91b75d4000-7f91b75d5000 rw-p 00022000 00:12 54384 > /usr/lib64/ld-2.17.so > 7f91b75d5000-7f91b75d6000 rw-p 00000000 00:00 0 > 7ffcb1849000-7ffcb186c000 rw-p 00000000 00p:00 0 > [stack] > 7ffcb191f000-7ffcb1921000 r-xp 00000000 00:00 0 > [vdso] > ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 > [vsyscall] > 9: Collecting options took > > On Thu, Apr 6, 2017 at 1:58 AM, Alexander Erdmann <[email protected]> wrote: > >> Hi Moses Team, >> >> I'm running an MT system with 2 phrase tables and 2 reordering tables in >> conjunction using the default 4 probabilities in the former and 6 >> probabilities in the latter. >> >> When I call it with this (and I'm running lots of jobs in parallel so I >> may run this command 20 times at once just varying the suffix on the input >> file 'del') >> >> $MOSESDECODER_ROOT/bin/moses -threads 4 -config >> $PWD/seed/$seedSize/iteration$iterations/moses.ini -input-file >> data/del$var 1> seed/$seedSize/iteration$iterations/del$var.lev 2> >> seed/$seedSize/iteration$iterations/del$var.log >> >> it never translates a single line, but it also doesn't die in the same >> place every time. Sometimes the error message is: >> >> Loading UnknownWordPenalty0 >> >> Loading WordPenalty0 >> >> Loading PhrasePenalty0 >> >> Loading LexicalReordering0 >> >> Loading table into memory...found inconsistent number of probabilities... >> found 4 expected 6 >> >> and sometimes it loads the lexical reordering table just fine but >> segFaults while trying to translate the first line. >> >> I am positive that there are in fact 6 probabilities in every line of the >> reordering table so I'm especially confused about that error... any ideas? >> >> Here is my config file: >> >> # input factors >> [input-factors] >> 0 >> >> # mapping steps >> [mapping] >> 0 T 0 >> 1 T 1 >> >> [distortion-limit] >> 6 >> >> # feature functions >> [feature] >> UnknownWordPenalty >> WordPenalty >> PhrasePenalty >> PhraseDictionaryMemory name=TranslationModel0 num-features=4 >> path=/scratch/ae1541/EMNLP/seed/$$$/iteration0/phrase-table >> PhraseDictionaryMemory name=TranslationModel1 num-features=4 >> path=/scratch/ae1541/EMNLP/seed/$$$/iteration***/phrase-table >> input-factor=0 output-factor=0 >> LexicalReordering name=LexicalReordering0 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/scratch/ae1541/EMNLP/seed/$$$/iteration0/reordering-table >> LexicalReordering name=LexicalReordering1 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/scratch/ae1541/EMNLP/seed/$$$/iteration***/reordering-table >> Distortion >> KENLM name=LM0 factor=0 path=/scratch/ae1541/EMNLP/seed/LM/LM.lev.lm >> order=5 >> >> # dense weights for feature functions >> [weight] >> UnknownWordPenalty0= 1 >> WordPenalty0= -1 >> PhrasePenalty0= 0.2 >> TranslationModel0= 0.2 0.2 0.2 0.2 >> TranslationModel1= 0.2 0.2 0.2 0.2 >> LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 >> LexicalReordering1= 0.3 0.3 0.3 0.3 0.3 0.3 >> Distortion0= 0.3 >> LM0= 0.5 >> >> Thanks! >> >> -- >> Alex Erdmann >> PhD Student in Linguistics at The Ohio State University >> Visiting Scholar at NYU Abu Dhabi >> > > > > -- > Alex Erdmann > PhD Student in Linguistics at The Ohio State University > Visiting Scholar at NYU Abu Dhabi > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
