Hi,
Could it be, you do not have write-access in /tmp ? If not, try "-T ." to create temporary files in the local dir. Best, Marcin W dniu 2015-05-06 18:05, Valmi Dufour-Lussier napisał(a): > Hello everyone, > > I have tried using processPhraseTableMin on my model trained on > Europarl v7 that's just too large to load into memory uncompressed, > but I get a segfault at each attempt. > > Any idea why, or any workaround to try? > > Here is the command used and the output: > > $ processPhraseTableMin -threads 3 -in train/model/phrase-table.gz > -nscores 4 -out binarised-model/phrase-table > Used options: > Text phrase table will be read from: train/model/phrase-table.gz > Output phrase table will be written to: binarised-model/phrase-table.minphr > Step size for source landmark phrases: 2^10=1024 > Source phrase fingerprint size: 16 bits / P(fp)=1.52588e-05 > Selected target phrase encoding: Huffman + PREnc > Maxiumum allowed rank for PREnc: 100 > Number of score components in phrase table: 4 > Single Huffman code set for score components: no > Using score quantization: no > Explicitly included alignment information: yes > Running with 3 threads > > Pass 1/3: Creating hash function for rank assignment > ..................................................[5000000] > ..................................................[10000000] > ..................................................[15000000] > ..................................................[20000000] > ..................................................[25000000] > ..................................................[30000000] > ..................................................[35000000] > ..................................................[40000000] > ..................................................[45000000] > ..................................................[50000000] > ..................................................[55000000] > ..................................................[60000000] > ........................... > > Pass 2/3: Creating source phrase index + Encoding target phrases > *** Segmentation fault > Register dump: > > RAX: 0000000000683fc0 RBX: 0000000000000001 RCX: 0000000000000000 > RDX: 0000000000000001 RSI: 0000000000000008 RDI: 00007ffc83939418 > RBP: 0000000000000008 R8 : 00007fc2575af960 R9 : 00007fc258a35740 > R10: 0000000000000008 R11: 00007fc257279260 R12: 00000000014b8560 > R13: 0000000000000003 R14: 00007ffc83939350 R15: 00007ffc839393f8 > RSP: 00007ffc83938f30 > > RIP: 00007fc25727927b EFLAGS: 00010202 > > CS: 0033 FS: 0000 GS: 0000 > > Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000000 > > FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000 > RIP: 00000000 RDP: 00000000 > > ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000 > ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000 > ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000 > ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000 > mxcsr: 1f80 > XMM0: 0000000000000000000000003cdd67ba XMM1: 0000000000000000000000003cdd67ba > XMM2: 0000000000000000000000003cdd67ba XMM3: 0000000000000000000000003cdd67ba > XMM4: 0000000000000000000000003cdd67ba XMM5: 0000000000000000000000003cdd67ba > XMM6: 0000000000000000000000003cdd67ba XMM7: 0000000000000000000000003cdd67ba > XMM8: 0000000000000000000000003cdd67ba XMM9: 0000000000000000000000003cdd67ba > XMM10: 0000000000000000000000003cdd67ba XMM11: > 0000000000000000000000003cdd67ba > XMM12: 0000000000000000000000003cdd67ba XMM13: > 0000000000000000000000003cdd67ba > XMM14: 0000000000000000000000003cdd67ba XMM15: > 0000000000000000000000003cdd67ba > > Backtrace: > /usr/lib/libc.so.6(fwrite+0x1b)[0x7fc25727927b] > processPhraseTableMin[0x42c4a4] > processPhraseTableMin[0x42c662] > processPhraseTableMin[0x417dc3] > processPhraseTableMin[0x405f18] > /usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7fc257230800] > processPhraseTableMin[0x406d29] > > Memory map: > > 00400000-00483000 r-xp 00000000 08:01 8130298 > /opt/mosesdecoder/bin/processPhraseTableMin > 00682000-00684000 rw-p 00082000 08:01 8130298 > /opt/mosesdecoder/bin/processPhraseTableMin > 00684000-00685000 rw-p 00000000 00:00 0 > 014b7000-014d8000 rw-p 00000000 00:00 0 [heap] > 7fc1f2f90000-7fc215a76000 rw-p 00000000 00:00 0 > 7fc21c000000-7fc21f32f000 rw-p 00000000 00:00 0 > 7fc21f32f000-7fc220000000 ---p 00000000 00:00 0 > 7fc224000000-7fc2276e6000 rw-p 00000000 00:00 0 > 7fc2276e6000-7fc228000000 ---p 00000000 00:00 0 > 7fc228000000-7fc22b34a000 rw-p 00000000 00:00 0 > 7fc22b34a000-7fc22c000000 ---p 00000000 00:00 0 > 7fc22c000000-7fc230000000 rw-p 00000000 00:00 0 > 7fc230000000-7fc234000000 rw-p 00000000 00:00 0 > 7fc234000000-7fc237fff000 rw-p 00000000 00:00 0 > 7fc237fff000-7fc238000000 ---p 00000000 00:00 0 > 7fc238000000-7fc23c000000 rw-p 00000000 00:00 0 > 7fc23c000000-7fc240000000 rw-p 00000000 00:00 0 > 7fc240000000-7fc244000000 rw-p 00000000 00:00 0 > 7fc244000000-7fc2442f6000 rw-p 00000000 00:00 0 > 7fc2442f6000-7fc248000000 ---p 00000000 00:00 0 > 7fc248000000-7fc2486b5000 rw-p 00000000 00:00 0 > 7fc2486b5000-7fc24c000000 ---p 00000000 00:00 0 > 7fc24c000000-7fc24cb09000 rw-p 00000000 00:00 0 > 7fc24cb09000-7fc250000000 ---p 00000000 00:00 0 > 7fc253a09000-7fc253a0a000 ---p 00000000 00:00 0 > 7fc253a0a000-7fc25420a000 rw-p 00000000 00:00 0 > 7fc25520c000-7fc25520d000 ---p 00000000 00:00 0 > 7fc25520d000-7fc255a0d000 rw-p 00000000 00:00 0 > 7fc255a0d000-7fc255a0e000 ---p 00000000 00:00 0 > 7fc255a0e000-7fc25620e000 rw-p 00000000 00:00 0 > 7fc25620e000-7fc25620f000 ---p 00000000 00:00 0 > 7fc25620f000-7fc256a0f000 rw-p 00000000 00:00 0 > 7fc256a0f000-7fc256a10000 ---p 00000000 00:00 0 > 7fc256a10000-7fc257210000 rw-p 00000000 00:00 0 > [stack:22662] > 7fc257210000-7fc2573a9000 r-xp 00000000 08:01 9178200 > /usr/lib/libc-2.21.so > 7fc2573a9000-7fc2575a9000 ---p 00199000 08:01 9178200 > /usr/lib/libc-2.21.so > 7fc2575a9000-7fc2575ad000 r--p 00199000 08:01 9178200 > /usr/lib/libc-2.21.so > 7fc2575ad000-7fc2575af000 rw-p 0019d000 08:01 9178200 > /usr/lib/libc-2.21.so > 7fc2575af000-7fc2575b3000 rw-p 00000000 00:00 0 > 7fc2575b3000-7fc2575cb000 r-xp 00000000 08:01 9178220 > /usr/lib/libpthread-2.21.so > 7fc2575cb000-7fc2577ca000 ---p 00018000 08:01 9178220 > /usr/lib/libpthread-2.21.so > 7fc2577ca000-7fc2577cb000 r--p 00017000 08:01 9178220 > /usr/lib/libpthread-2.21.so > 7fc2577cb000-7fc2577cc000 rw-p 00018000 08:01 9178220 > /usr/lib/libpthread-2.21.so > 7fc2577cc000-7fc2577d0000 rw-p 00000000 00:00 0 > 7fc2577d0000-7fc2577e6000 r-xp 00000000 08:01 9178553 > /usr/lib/libgcc_s.so.1 > 7fc2577e6000-7fc2579e5000 ---p 00016000 08:01 9178553 > /usr/lib/libgcc_s.so.1 > 7fc2579e5000-7fc2579e6000 rw-p 00015000 08:01 9178553 > /usr/lib/libgcc_s.so.1 > 7fc2579e6000-7fc257ae9000 r-xp 00000000 08:01 9178248 > /usr/lib/libm-2.21.so > 7fc257ae9000-7fc257ce9000 ---p 00103000 08:01 9178248 > /usr/lib/libm-2.21.so > 7fc257ce9000-7fc257cea000 r--p 00103000 08:01 9178248 > /usr/lib/libm-2.21.so > 7fc257cea000-7fc257ceb000 rw-p 00104000 08:01 9178248 > /usr/lib/libm-2.21.so > 7fc257ceb000-7fc257ddb000 r-xp 00000000 08:01 9178557 > /usr/lib/libstdc++.so.6.0.20 > 7fc257ddb000-7fc257fdb000 ---p 000f0000 08:01 9178557 > /usr/lib/libstdc++.so.6.0.20 > 7fc257fdb000-7fc257fe3000 r--p 000f0000 08:01 9178557 > /usr/lib/libstdc++.so.6.0.20 > 7fc257fe3000-7fc257fe5000 rw-p 000f8000 08:01 9178557 > /usr/lib/libstdc++.so.6.0.20 > 7fc257fe5000-7fc257ffa000 rw-p 00000000 00:00 0 > 7fc257ffa000-7fc258001000 r-xp 00000000 08:01 9178221 > /usr/lib/librt-2.21.so > 7fc258001000-7fc258200000 ---p 00007000 08:01 9178221 > /usr/lib/librt-2.21.so > 7fc258200000-7fc258201000 r--p 00006000 08:01 9178221 > /usr/lib/librt-2.21.so > 7fc258201000-7fc258202000 rw-p 00007000 08:01 9178221 > /usr/lib/librt-2.21.so > 7fc258202000-7fc258206000 r-xp 00000000 08:01 9178197 > /usr/lib/libSegFault.so > 7fc258206000-7fc258405000 ---p 00004000 08:01 9178197 > /usr/lib/libSegFault.so > 7fc258405000-7fc258406000 r--p 00003000 08:01 9178197 > /usr/lib/libSegFault.so > 7fc258406000-7fc258407000 rw-p 00004000 08:01 9178197 > /usr/lib/libSegFault.so > 7fc258407000-7fc258416000 r-xp 00000000 08:01 9181283 > /usr/lib/libbz2.so.1.0.6 > 7fc258416000-7fc258615000 ---p 0000f000 08:01 9181283 > /usr/lib/libbz2.so.1.0.6 > 7fc258615000-7fc258617000 rw-p 0000e000 08:01 9181283 > /usr/lib/libbz2.so.1.0.6 > 7fc258617000-7fc25863c000 r-xp 00000000 08:01 9186133 > /usr/lib/liblzma.so.5.2.1 > 7fc25863c000-7fc25883b000 ---p 00025000 08:01 9186133 > /usr/lib/liblzma.so.5.2.1 > 7fc25883b000-7fc25883c000 r--p 00024000 08:01 9186133 > /usr/lib/liblzma.so.5.2.1 > 7fc25883c000-7fc25883d000 rw-p 00025000 08:01 9186133 > /usr/lib/liblzma.so.5.2.1 > 7fc25883d000-7fc25885f000 r-xp 00000000 08:01 9178244 > /usr/lib/ld-2.21.so > 7fc258a34000-7fc258a3c000 rw-p 00000000 00:00 0 > 7fc258a5e000-7fc258a5f000 r--p 00021000 08:01 9178244 > /usr/lib/ld-2.21.so > 7fc258a5f000-7fc258a60000 rw-p 00022000 08:01 9178244 > /usr/lib/ld-2.21.so > 7fc258a60000-7fc258a61000 rw-p 00000000 00:00 0 > 7ffc8391a000-7ffc8393b000 rw-p 00000000 00:00 0 [stack] > 7ffc8394d000-7ffc8394f000 r--p 00000000 00:00 0 [vvar] > 7ffc8394f000-7ffc83951000 r-xp 00000000 00:00 0 [vdso] > ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 > [vsyscall] > Segmentation fault (core dumped) > > Best, > Valmi > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support [1] Links: ------ [1] http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
