Just for reference, The issue was that I didn't filter the phrase table beforehand. based on how the model was trained, I knew this wouldn't reduce the size of the phrase table enough to make it worthwhile (or so I thought), but as soon as I filtered the phrase table it stopped segFaulting.
On Fri, Apr 7, 2017 at 4:11 PM, Hieu Hoang <[email protected]> wrote: > The moses.ini file looks ok. > > The segfault is difficult to debug by inspection. It's best sprinkle the > decoder code with debugging message and run 1 instance of the decoder to > see exactly where it goes wrong. And you should binarise the pt and > reordering models to save yourself time > > * Looking for MT/NLP opportunities * > Hieu Hoang > http://moses-smt.org/ > > > On 6 April 2017 at 11:30, Alexander Erdmann <[email protected]> wrote: > >> For reference, here is the output from the segFault: >> >> Defined parameters (per moses.ini or switch): >> config: /scratch/ae1541/EMNLP/seed/150000/iteration1/mosesTryingGZ.ini >> distortion-limit: 6 >> feature: UnknownWordPenalty WordPenalty PhrasePenalty >> PhraseDictionaryMemory name=TranslationModel0 num-features=4 >> path=/scratch/ae1541/EMNLP/seed/150000/iteration0/phrase-table.gz >> LexicalReordering name=LexicalReordering0 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/scratch/ae1541/EMNLP/seed/150000/iteration0/reordering-table.gz >> Distortion KENLM name=LM0 factor=0 >> path=/scratch/ae1541/EMNLP/seed/LM/LM.lev.lm >> order=5 >> input-factors: 0 >> input-file: seed/150000/iteration1/del00.egy >> mapping: 0 T 0 >> threads: 12 >> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 >> TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 >> 0.3 Distortion0= 0.3 LM0= 0.5 >> line=UnknownWordPenalty >> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0 >> line=WordPenalty >> FeatureFunction: WordPenalty0 start: 1 end: 1 >> line=PhrasePenalty >> FeatureFunction: PhrasePenalty0 start: 2 end: 2 >> line=PhraseDictionaryMemory name=TranslationModel0 num-features=4 >> path=/scratch/ae1541/EMNLP/seed/150000/iteration0/phrase-table.gz >> FeatureFunction: TranslationModel0 start: 3 end: 6 >> line=LexicalReordering name=LexicalReordering0 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/scratch/ae1541/EMNLP/seed/150000/iteration0/reordering-table.gz >> FeatureFunction: LexicalReordering0 start: 7 end: 12 >> Initializing LexicalReordering.. >> line=Distortion >> FeatureFunction: Distortion0 start: 13 end: 13 >> line=KENLM name=LM0 factor=0 path=/scratch/ae1541/EMNLP/seed/LM/LM.lev.lm >> order=5 >> FeatureFunction: LM0 start: 14 end: 14 >> Loading the LM will be faster if you build a binary file. >> Reading /scratch/ae1541/EMNLP/seed/LM/LM.lev.lm >> ----5---10---15---20---25---30---35---40---45---50---55---60 >> ---65---70---75---80---85---90---95--100 >> ************************************************************ >> **************************************** >> Loading UnknownWordPenalty0 >> Loading WordPenalty0 >> Loading PhrasePenalty0 >> Loading LexicalReordering0 >> Loading table into memory...done. >> Loading Distortion0 >> Loading LM0 >> Loading TranslationModel0 >> Start loading text phrase table. Moses format : [1574.737] seconds >> Reading /scratch/ae1541/EMNLP/seed/150000/iteration0/phrase-table.gz >> ----5---10---15---20---25---30---35---40---45---50---55---60 >> ---65---70---75---80---85---90---95--100 >> ************************************************************ >> **************************************** >> Created input-output object : [1608.439] seconds >> Translating: Translating: A y Translating: jAC Translating: Translating: >> AbwTranslating: RRR w hA Al mAd mrwAn >> sAdRRR jrydp A m trtr . . jrydp ywmy p . . bTyx yp . . mhlb yp bAl >> tEycTranslating: w nA Al byAn Al tAly p >> Line 1: Initialize search took Line 8: Initialize search took tqly p >> 0.006 seconds total >> 0.005 seconds total >> Translating: t h y Translating: qwqAl A b mSr RRR rytc bAl Al dAxly p y >> AxbAr ? >> Line 9: Initialize search took 0.007 seconds total >> w b h mfyc RRR >> AxwAn hm Hd m Al mslmyn AnDrb >> Line 11: Initialize search took Line 0: Initialize search took Al0.010 >> seconds total >> Hrp 0.009 seconds total >> Al Translating: AfAd Al mhlb jrydpmtHdv Al rsmy l wzArp Al nSAry bEd >> Translating: yp Al m m A y h Avl Al dAxly w >> An p AfAdt Line 7: Initialize search took qyAd ( Line xyr p ll TbE >> Al mSlHp 8: Collecting options took Line 2: Initialize search took lhm Al >> 0.005 seconds at moses/Manager.cpp:AjElh bHv 0.015 seconds total >> 0.013 seconds total >> wrAC Amn xyr mvyr Al Al y dwlp tlyfzywn Al sAbq Al mSry A ) Almbjl ytHdv >> b Al An lsAn jnAQy h jrydp An bEd Al ftn A tHry w m w Al trtr Al bHv w >> mtTrf f yTlb yn y mn Al AHdAv Al mslmyn >> hm Line 6: Initialize search took 117 >> nA 0.020 seconds total >> s *** bSegmentation fault >> Register dump: >> >> RAX: 0000000000000000 RBX: 00000000012e28e0 RCX: 0000000000000000 >> RDX: 00007f91b7413010 RSI: 0000000000000000 RDI: 00007f917400b9c0 >> RBP: 0000000000000000 R8 : 0000000000000002 R9 : 0000000000000038 >> R10: 00007f91aa8f3310 R11: 00007f91b64cfbf0 R12: 00007f917400b028 >> R13: 00007f9174004f60 R14: 00007f917400b9c0 R15: 0000000000000002 >> RSP: 00007f91aa8f3730 >> >> RIP: 00007f91b66e8833 EFLAGS: 00010246 >> >> CS: 0033 FS: 0000 GS: 0000 >> >> Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000010 >> >> FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000 >> RIP: 00000000 RDP: 00000000 >> >> ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000 >> ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000 >> ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000 >> ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000 >> mxcsr: 1fa4 >> XMM0: 00000000000000000000000000000000 XMM1: >> 00000000000000000000000000000000 >> XMM2: 00000000000000000000000000000000 XMM3: >> 00000000000000000000000000000000 >> XMM4: 00000000000000000000000000000000 XMM5: >> 00000000000000000000000000000000 >> XMM6: 00000000000000000000000000000000 XMM7: >> 00000000000000000000000000000000 >> XMM8: 00000000000000000000000000000000 XMM9: >> 00000000000000000000000000000000 >> XMM10: 00000000000000000000000000000000 XMM11: >> 00000000000000000000000000000000 >> XMM12: 00000000000000000000000000000000 XMM13: >> 00000000000000000000000000000000 >> XMM14: 00000000000000000000000000000000 XMM15: >> 00000000000000000000000000000000 >> >> Backtrace: >> y Sdd lA tjmyE hyj rAs wA AHd/share/apps/NYUAD/mosesdeco >> der/3.0/lib/libmoses.so(_ZNK5Moses16LanguageModelKenIN2lm5ng >> ram12ProbingModelEE19EvaluateWhenAppliedERKNS_10HypothesisEP >> KNS_7FFStateEPNS_24ScoreComponentCollectionE+0xa3)[0x7f91b66e8833] >> Ely/share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ZN5M >> oses10Hypothesis19EvaluateWhenAppliedERKNS_12SquareMatrixE+ >> 0xf0)[0x7f91b64cfab0] >> Al /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ZN5Moses >> 12SearchNormal16ExpandHypothesisERKNS_10HypothesisERKNS_17Tr >> anslationOptionEf+0x165)[0x7f91b65484a5] >> bED /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ZN5Moses >> 12SearchNormal19ExpandAllHypothesesERKNS_10HypothesisEmm+ >> 0x79)[0x7f91b6548829] >> >> mALine /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ZN5Moses >> 12SearchNormal20ProcessOneHypothesisERKNS_10HypothesisE+ >> 0x34b)[0x7f91b6548beb] >> 10: Initialize search took /share/apps/NYUAD/mosesdecoder >> /3.0/lib/libmoses.so(_ZN5Moses12SearchNormal6DecodeEv+0x11c) >> [0x7f91b6548ffc] >> DHAyA/share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ >> ZN5Moses7Manager6DecodeEv+0xe3)[0x7f91b64f9173] >> sbyrw /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ZN5Moses >> 15TranslationTask3RunEv+0x1a5)[0x7f91b656f505] >> f /share/apps/NYUAD/mosesdecoder/3.0/lib/libmoses.so(_ZN5Moses >> 10ThreadPool7ExecuteEv+0x89)[0x7f91b6724b99] >> ./share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_thread.so.1.57.0(+0x1211a)[0x7f91b54ee11a] >> /usr/lib64/libpthread.so.0(+0x7dc5)[0x7f91b489cdc5] >> y /usr/lib64/libc.so.6(clone+0x6d)[0x7f91b45ca28d] >> . >> Memory map: >> >> tlk 00400000-0040b000 r-xp 00000000 00:2b 1254532671 >> /share/apps/NYUAD3/mosesdecoder/3.0/bin/moses >> 0060b000-0060c000 rw-p 0000b000 00:2b 1254532671 >> /share/apps/NYUAD3/mosesdecoder/3.0/bin/moses >> 012ce000-52974a000qd rw-p 00000000 00:00 0 >> [heap] >> 7f9174000000-7f9174021000 rw-p 00000000 00:00 0 >> 7f9174021000-7f9178000000 ---p 00000000 00:00 0 >> 7f9178000000-7f917807b000 rw-p 00000000 00:00 0 >> 7f917807b000-7f917c000000 ---p 00000000 00:00 0 >> 7f917c000000-7f917c034000 rw-p 00000000 00:00 0 >> 7f917c034000-7f9180000000 ---p 00000000 00:00 0 >> 7f9184000000-7f918409e000 rw-p 00000000 00:00 0 >> 7f918409e000-7f9188000000 ---p 00000000 00:00 0 >> 7f9188000000-7f9188047000 rw-p 00000000 00:00 0 >> 7f9188047Al000-7f918c000000 ---p 00000000 00:00 0 >> 7f918c000000-7f918c23d000 rw-p 00000000 00:00 0 >> 7f918c23d000-7f9190000000 ---p 00000000 00:00 0 >> 7f9190000000-7f9190106000 rw-p 00000000 00:00 0 >> 7f9190106000-7f9194000000 ---p 00000000 00:00 0 >> 7f9194000000-7f91940 c7000 rw-p 00000000 00:00 0 >> 7f91940c9000-7f9198000000 ---p 00000000 00:00 0 >> 7f9198000000-7f9198106000 rw-p 00000000 00:00 0 >> 7f9198106000-7f919c000000 ---p 00000000 00:00 0 >> 7f919c000000-7f919c165000 rw-p 00000000 00:00 0 >> 7f919c165000-7f91a0000000 ---p tbyn00000000 00:00 0 >> 7f919c167000-7f91a0000000 ---p 00000000 00:00 0 >> 7f91a0000000-7f91a009f000 rw-p 00000000 00:00 0 >> 7f91a009f000-7f91a4000000 ---p 00000000 00:00 0 >> 7f91a4000000-7f91a4047000 rw-p 00000000 00:00 0 >> 7f91a4047000-7f91a8000000 ---p 00000000 00 :00 0 >> 7f91a80f0000-7f91a80f1000 ---p 00000000 00:00 0 >> 7f91a80f1000-7f91a88f1000 rw-p 00000000 00:00 0 >> [stack:10106] >> 7f91a88f1000-7f91a88f2000 ---p 00000000 00:00 0 >> 7f91a88f2000-7f91a90f2000 rw-p 00000000 00:00 0 AHdAv >> [stack:10105] >> 7f91a90f2000-7f91a90f3000 ---p 00000000 00:00 0 >> 7f91a90f3000-7f91a98f3000 rw-p 00000000 00:00 0 >> [stack:10104] >> 7f91a98f3000-7f91a98f4000 ---p 00000000 00:00 0 >> 7f91a98f4000-7f91aa0f4000 rw-p 00000000 00:00 0 >> [stack:10103] >> 7f91aa0f4000-7f91aa0f5000 ---p 00000000 00:00 0 >> 7f91aa0f5000-7f91aa8f5000 rw-p 00000000 00:00 0 >> [stack:10102] >> 7f91aa8f5000-7f91aa8f6000 ---p 00000000 00:00 0 >> 7f91aa8f6000-7f91ab0f6000 rw-p 0lnA0000000 00:00 0 >> [stack:10101] >> 7f91ab0f6000-7f91ab0f7000 ---p 00000000 00:00 0 >> 7f91ab0f7000-7f91ab8f7000 rw-p 00000000 00:00 0 >> [stack:10100] >> 7f91ab8f7000-7f91ab8f8000 ---p 00000000 00:00 0 >> 7f91ab8f8000-7f9 1ac0f8000 rw-p 00000000 00:00 0 >> [stack:10099] >> 7f91ac0f8000-7f91ac0f9000 ---p 00000000 00:00 0 >> 7f91ac0f9000-7f91ae8fa000 rw-p 00000000 00:00 0 >> [stack:10098] >> 7f91aef80000-7f91aef81000 ---p 00000000 00:00 0 >> w7f91aef81000-7f91af781000 rw-p 00000000 00:00 0 >> [stack:10097] >> 7f91af781000-7f91af782000 ---p 00000000 00:00 0 >> 7f91af782000-7f91aff82000 rw-p 00000000 00:00 0 >> [stack:10096] >> 7f91aff82000-7f91aff83000 ---p 00 000000 00:00 0 >> 7f91aff83000-7f91b1784000 rw-p 00000000 00:00 0 >> [stack:10095] >> 7f91b1886000-7f91b18c7000 rw-p 00000000 00:00 0 >> 7f91b18c7000-7f91b409f000 rw-p 00000000 00:00 0 >> 7f91b409f000-7f91b40ae000 r-xp 00000000 00:12 54468 An >> /usr/lib64/libbz2.so.1.0.6 >> 7f91b40ae000-7f91b42ad000 ---p 0000f000 00:12 54468 >> /usr/lib64/libbz2.so.1.0.6 >> 7f91b42ad000-7f91b42ae000 r--p 0000e000 00:12 54468 >> /usr/lib64/libbz2.so.1.0.6 >> 7f91b42ae00 0-7f91b42af000 rw-p 0000f000 00:12 54468 >> /usr/lib64/libbz2.so.1.0.6 >> 7f91b42af000-7f91b42d3000 r-xp 00000000 00:12 54446 >> /usr/lib64/liblzma.so.5.0.99 >> 7f91b42d3000-7f91b44d2000 ---p 00024000 00:12 54446 AlVy >> /usr/lib64/liblzma.so.5.0.99 >> 7f91b44d2000-7f91b44d3000 r--p 00023000 00:12 54446 >> /usr/lib64/liblzma.so.5.0.99 >> 7f91b44d3000-7f91b44d4000 rw-p 00024000 00:12 54446 >> /usr/lib64/liblzma.so.5.0.99 >> 7f91b44d4000-7f 91b468a000 r-xp 00000000 00:12 54398 >> /usr/lib64/libc-2.17.so >> 7f91b468a000-7f91b488a000 ---p 001b6000 00:12 54398 >> /usr/lib64/libc-2.17.so >> 7f91b488a000-7f91b488e000 r--p 001b6000 00:12 54398 >> /usrAl/lib64/libc-2.17.so >> 7f91b488e000-7f91b4890000 rw-p 001ba000 00:12 54398 >> /usr/lib64/libc-2.17.so >> 7f91b4890000-7f91b4895000 rw-p 00000000 00:00 0 >> 7f91b4895000-7f91b48ab000 r-xp 00000000 00:12 54451 >> /usr/lib64/libpth read-2.17.so >> 7f91b48ab000-7f91b4aab000 ---p 00016000 00:12 54451 >> /usr/lib64/libpthread-2.17.so >> 7f91b4aab000-7f91b4aac000 r--p 00016000 00:12 54451 >> /usr/lib64/libpthread-2.17.so >> 7f91b4aac000-7f91b4aad000 rw-p 000170t00 00:12 54451 >> /usr/lib64/libpthread-2.17.so >> 7f91b4aad000-7f91b4ab1000 rw-p 00000000 00:00 0 >> 7f91b4ab1000-7f91b4ac7000 r-xp 00000000 00:2b 390181951 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown-linux-gn >> u/lib64/libgcc_s.so.1 >> 7f91b4ac7000-7f91b4cc6000 ---p 00016000 00:2b 390181951 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown >> -linux-gnu/lib64/libgcc_s.so.1 >> 7f91b4cc6000-7f91b4cc7000 r--p 00015000 00:2b 390181951 >> slfyyn /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown >> -linux-gnu/lib64/libgcc_s.so.1 >> 7f91b4cc7000-7f91b4cc8000 rw-p 00016000 00:2b 390181951 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown >> -linux-gnu/lib64/libgcc_s.so.1 >> 7 f91b4cc8000-7f91b4dc9000 r-xp 00000000 00:12 54414 >> /usr/lib64/libm-2.17.so >> 7f91b4dc9000-7f91b4fc8000 ---p 00101000 00:12 54414 >> /usr/lib64/libm-2.17.so >> 7f91b4fc8000-7f91b4fc9000 r--p 00100000 00:12 54414 bdw >> /usr/lib64/libm-2.17.so >> 7f91b4fc9000-7f91b4fca000 rw-p 00101000 00:12 54414 >> /usr/lib64/libm-2.17.so >> 7f91b4fca000-7f91b50b5000 r-xp 00000000 00:2b 3536542694 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64- >> unknown-linux-gnu/4.9.3/libstdc++.so.6.0.20 >> 7f91b50b5000-7f91b52b5000 ---p 000eb000 00:2b 3536542694 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown- >> linux-gnu/4.9.3/libstdc++.so.6.0.20 >> 7f91b52b5000-7f91b52bd000 r--p 000eb000 00:hm 2b 3536542694 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknown- >> linux-gnu/4.9.3/libstdc++.so.6.0.20 >> 7f91b52bd000-7f91b52bf000 rw-p 000f3000 00:2b 3536542694 >> /share/apps/NYUAD3/gcc/gcc/4.9.3/el7/lib/gcc/x86_64-unknomlA >> mHwn-linux-gnu/4.9.3/libstdc++.so.6.0.20 >> 7f91b52bf000-7f91b52d4000 rw-p 00000000 00:00 0 >> 7f91b52d4000-7f91b52db000 r-xp 00000000 00:12 54459 >> /usr/lib64/librt-2.17.so >> 7f91b52db000-7f91b54da000 ---p 00007000 00:12 54459 >> /usr/lib64/librt-2.17.so >> 7f91b54da000-7f91b54db000 r--p 00006000 00:12 54459 >> /usr/lib64/librt-2.17.so >> 7f91b54db000-7f91b54dc000 rw-p 00007000 00:12 54459 >> /usr/lib64/librt-2.17.so >> 7f91b54dc000-7f91b54fd000 r-xp 0mn0000000 00:2b 189775209 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_thread.so.1.57.0 >> 7f91b54fd000-7f91b56fc000 ---p 00021000 00:2b 189775209 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/ >> avx2/1.57.0/lib/libboost_thread.so.1.57.0 >> 7f91b56fc000-7f91b56ff000 rw-p 00020000 00:2b 189775209 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_thread.so.1.57.0 >> 7f91b56ff000-7f91b5714000 r-xp 00000000 00:12 544h64 >> /usr/lib64/libz.so.1.2.7 >> 7f91b5714000-7f91b5913000 ---p 00015000 00:12 54464 >> /usr/lib64/libz.so.1.2.7 >> 7f91b5913000-7f91b5914000 r--p 00014000 00:12 54464 >> /usr/lib64/libz.so.1.2.7 >> 7f91b591400 0-7f91b5915000 rw-p 00015000 00:12 54464 >> /usr/lib64/libz.so.1.2.7 >> 7f91b5915000-7f91b5918000 r-xp 00000000 00:2b 189775207 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_system.so.1.57.0 >> 7f91AcElb5918000-7f91b5b17000 ---p 00003000 00:2b 189775207 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_system.so.1.57.0 >> 7f91b5b17000-7f91b5b18000 rw-p 00002000 00:2b 189775207 >> /share/apps/NYUAD3/boo st/gcc_4.9.3/openmpi_1.10.2/av >> x2/1.57.0/lib/libboost_system.so.1.57.0 >> 7f91b5b18000-7f91b5b2e000 r-xp 00000000 00:2b 189756212 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_iostreams.so.1.57.0 >> 7f91b5b2e000-7f91bwADH5d2e000 ---p 00016000 00:2b 189756212 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_iostreams.so.1.57.0 >> 7f91b5d2e000-7f91b5d30000 rw-p 00016000 00:2b 189756212 >> /share/apps/NYUAD3/boost/gcc_4.9. 3/openmpi_1.10.2/avx2/1.57.0/l >> ib/libboost_iostreams.so.1.57.0 >> 7f91b5d30000-7f91b5d34000 r-xp 00000000 00:12 54392 >> /usr/lib64/libSegFault.so >> 7f91b5d34000-7f91b5f33000 ---p 00004000 00:12 54392 >> /usr/lib64/libSegFaultwA.so >> 7f91b5f33000-7f91b5f34000 r--p 00003000 00:12 54392 >> /usr/lib64/libSegFault.so >> 7f91b5f34000-7f91b5f35000 rw-p 00004000 00:12 54392 >> /usr/lib64/libSegFault.so >> 7f91b5f35000-7f91b5f38000 r-xp 00000000 00:12 54410 >> /usr/lib64/libdl-2.17.so >> 7f91b5f38000-7f91b6137000 ---p 00003000 00:12 54410 >> /usr/lib64/libdl-2.17.so >> 7f91b6137000-7f91b6138000 r--p 00002000 00:12 54410 >> /usr/lib64/libdl-2.17.so >> 7f91b6138000-7f90.0221b6139000 rw-p 00003000 00:12 54410 >> /usr/lib64/libdl-2.17.so >> 7f91b6139000-7f91b6150000 r-xp 00000000 00:2b 189756209 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_filesystem.so.1.57.0 >> 7f91b seconds total6150000-7f91b634f000 ---p 00017000 00:2b 189756209 >> /share/apps/NYUAD3/boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57 >> .0/lib/libboost_filesystem.so.1.57.0 >> 7f91b634f000-7f91b6350000 rw-p 00016000 00:2b 189756209 >> /share/apps/NYUAD3/ >> boost/gcc_4.9.3/openmpi_1.10.2/avx2/1.57.0/lib/libboost_file >> system.so.1.57.0 >> 7f91b6350000-7f91b6868000 r-xp 00000000 00:2b 2421011537 >> /share/apps/NYUAD3/mosesdecoder/3.0/lib/libmoses.so >> 7f91b6868000-7f91b6a67000 ---p 00518000 00:2b 24210115ftyl37 >> /share/apps/NYUAD3/mosesdecoder/3.0/lib/libmoses.so >> 7f91b6a67000-7f91b6a82000 rw-p 00517000 00:2b 2421011537 >> /share/apps/NYUAD3/mosesdecoder/3.0/lib/libmoses.so >> 7f91b6a82000-7f91b73b2000 rw-p 00000000 00:00 0 >> 7f91b73b200 0-7f91b73d3000 r-xp 00000000 00:12 54384 >> /usr/lib64/ld-2.17.so >> 7f91b7413000-7f91b743b000 rw-p 00000000 00:00 0 >> 7f91b745c000-7f91b74c6000 rw-p 00000000 00:00 0 >> 7f91b75c7000-7f91b75d3000 rw-p 00000000 00:00 0 >> 7f91b75d3000-7f91b75d4000Line r--p 00021000 00:12 54384 >> /usr/lib64/ld-2.17.so >> 7f91b75d4000-7f91b75d5000 rw-p 00022000 00:12 54384 >> /usr/lib64/ld-2.17.so >> 7f91b75d5000-7f91b75d6000 rw-p 00000000 00:00 0 >> 7ffcb1849000-7ffcb186c000 rw-p 00000000 00p:00 0 >> [stack] >> 7ffcb191f000-7ffcb1921000 r-xp 00000000 00:00 0 >> [vdso] >> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 >> [vsyscall] >> 9: Collecting options took >> >> On Thu, Apr 6, 2017 at 1:58 AM, Alexander Erdmann <[email protected]> wrote: >> >>> Hi Moses Team, >>> >>> I'm running an MT system with 2 phrase tables and 2 reordering tables in >>> conjunction using the default 4 probabilities in the former and 6 >>> probabilities in the latter. >>> >>> When I call it with this (and I'm running lots of jobs in parallel so I >>> may run this command 20 times at once just varying the suffix on the input >>> file 'del') >>> >>> $MOSESDECODER_ROOT/bin/moses -threads 4 -config >>> $PWD/seed/$seedSize/iteration$iterations/moses.ini -input-file >>> data/del$var 1> seed/$seedSize/iteration$iterations/del$var.lev 2> >>> seed/$seedSize/iteration$iterations/del$var.log >>> >>> it never translates a single line, but it also doesn't die in the same >>> place every time. Sometimes the error message is: >>> >>> Loading UnknownWordPenalty0 >>> >>> Loading WordPenalty0 >>> >>> Loading PhrasePenalty0 >>> >>> Loading LexicalReordering0 >>> >>> Loading table into memory...found inconsistent number of >>> probabilities... found 4 expected 6 >>> >>> and sometimes it loads the lexical reordering table just fine but >>> segFaults while trying to translate the first line. >>> >>> I am positive that there are in fact 6 probabilities in every line of >>> the reordering table so I'm especially confused about that error... any >>> ideas? >>> >>> Here is my config file: >>> >>> # input factors >>> [input-factors] >>> 0 >>> >>> # mapping steps >>> [mapping] >>> 0 T 0 >>> 1 T 1 >>> >>> [distortion-limit] >>> 6 >>> >>> # feature functions >>> [feature] >>> UnknownWordPenalty >>> WordPenalty >>> PhrasePenalty >>> PhraseDictionaryMemory name=TranslationModel0 num-features=4 >>> path=/scratch/ae1541/EMNLP/seed/$$$/iteration0/phrase-table >>> PhraseDictionaryMemory name=TranslationModel1 num-features=4 >>> path=/scratch/ae1541/EMNLP/seed/$$$/iteration***/phrase-table >>> input-factor=0 output-factor=0 >>> LexicalReordering name=LexicalReordering0 num-features=6 >>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >>> path=/scratch/ae1541/EMNLP/seed/$$$/iteration0/reordering-table >>> LexicalReordering name=LexicalReordering1 num-features=6 >>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >>> path=/scratch/ae1541/EMNLP/seed/$$$/iteration***/reordering-table >>> Distortion >>> KENLM name=LM0 factor=0 path=/scratch/ae1541/EMNLP/seed/LM/LM.lev.lm >>> order=5 >>> >>> # dense weights for feature functions >>> [weight] >>> UnknownWordPenalty0= 1 >>> WordPenalty0= -1 >>> PhrasePenalty0= 0.2 >>> TranslationModel0= 0.2 0.2 0.2 0.2 >>> TranslationModel1= 0.2 0.2 0.2 0.2 >>> LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 >>> LexicalReordering1= 0.3 0.3 0.3 0.3 0.3 0.3 >>> Distortion0= 0.3 >>> LM0= 0.5 >>> >>> Thanks! >>> >>> -- >>> Alex Erdmann >>> PhD Student in Linguistics at The Ohio State University >>> Visiting Scholar at NYU Abu Dhabi >>> >> >> >> >> -- >> Alex Erdmann >> PhD Student in Linguistics at The Ohio State University >> Visiting Scholar at NYU Abu Dhabi >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > -- Alex Erdmann PhD Student in Linguistics at The Ohio State University Visiting Scholar at NYU Abu Dhabi
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
