Hi all,
When working with LMs of order > 6, build_binary informs the user that
kMaxOrder in lm/max_order.hh needs to be set to a higher value and a rebuild is
required.
I'm working with 7grams, set kMaxOrder to 7, and re-compiled (everything).
Binarization went through after this, however there is now an error during
decoding:
*** glibc detected ***
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses: free(): invalid
next size (fast): 0x000000001b4d8930 ***
Using gdb and looking at the backtrace, it seems to me that this happens when
std::vector<lm::WordIndex> indices(m_ngram->Order() - 1) goes out of scope at
the end of the if statement in Evaluate at moses/src/LM/Ken.cpp:243. But beyond
this I'm lost, and it would be great if one of the experts could help me out...
The problem occurred with revision 9ab49bc.
One further observation: When I re-run with a moses-binary compiled with
kMaxOrder=6, but still using the same binarized 7gram-LM, moses doesn't
complain about the incompatibility, and the same error occurs.
Thanks
Stephan
This is the full information I got in gdb:
*** glibc detected ***
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses: free(): invalid
next size (fast): 0x000000001b4d8930 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3a1287230f]
/lib64/libc.so.6(cfree+0x4b)[0x3a1287276b]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x41e0f1]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x41e123]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x41e164]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x41e237]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x587bd4]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x444fb9]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x483efc]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x4842f9]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x48494f]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x484d03]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x451cc7]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x40d3b5]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses[0x409010]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3a1281d994]
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses(__gxx_personality_v0+0x379)[0x408059]
======= Memory map: ========
00400000-00759000 r-xp 00000000 00:19 8080425
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses
00958000-0095a000 rw-p 00358000 00:19 8080425
/ec/dgt/shared/exodus/MosesSuite_9ab49bc_2011-12-01/bin/moses
0095a000-0165c000 rw-p 0095a000 00:00 0
09d64000-1df68000 rw-p 09d64000 00:00 0 [heap]
3a12400000-3a1241c000 r-xp 00000000 68:01 426282
/lib64/ld-2.5.so
3a1261b000-3a1261c000 r--p 0001b000 68:01 426282
/lib64/ld-2.5.so
3a1261c000-3a1261d000 rw-p 0001c000 68:01 426282
/lib64/ld-2.5.so
3a12800000-3a1294e000 r-xp 00000000 68:01 426285
/lib64/libc-2.5.so
3a1294e000-3a12b4d000 ---p 0014e000 68:01 426285
/lib64/libc-2.5.so
3a12b4d000-3a12b51000 r--p 0014d000 68:01 426285
/lib64/libc-2.5.so
3a12b51000-3a12b52000 rw-p 00151000 68:01 426285
/lib64/libc-2.5.so
3a12b52000-3a12b57000 rw-p 3a12b52000 00:00 0
3a13000000-3a130e6000 r-xp 00000000 68:01 599062
/usr/lib64/libstdc++.so.6.0.8
3a130e6000-3a132e5000 ---p 000e6000 68:01 599062
/usr/lib64/libstdc++.so.6.0.8
3a132e5000-3a132eb000 r--p 000e5000 68:01 599062
/usr/lib64/libstdc++.so.6.0.8
3a132eb000-3a132ee000 rw-p 000eb000 68:01 599062
/usr/lib64/libstdc++.so.6.0.8
3a132ee000-3a13300000 rw-p 3a132ee000 00:00 0
3a13400000-3a13482000 r-xp 00000000 68:01 426035
/lib64/libm-2.5.so
3a13482000-3a13681000 ---p 00082000 68:01 426035
/lib64/libm-2.5.so
3a13681000-3a13682000 r--p 00081000 68:01 426035
/lib64/libm-2.5.so
3a13682000-3a13683000 rw-p 00082000 68:01 426035
/lib64/libm-2.5.so
3a15000000-3a1500d000 r-xp 00000000 68:01 426295
/lib64/libgcc_s-4.1.2-20080825.so.1
3a1500d000-3a1520d000 ---p 0000d000 68:01 426295
/lib64/libgcc_s-4.1.2-20080825.so.1
3a1520d000-3a1520e000 rw-p 0000d000 68:01 426295
/lib64/libgcc_s-4.1.2-20080825.so.1
2b9264412000-2b9264413000 rw-p 2b9264412000 00:00 0
2b926441f000-2b92645a9000 rw-p 2b926441f000 00:00 0
2b9264989000-2b9265cca000 rw-p 2b9264989000 00:00 0
2b9265cca000-2b9371db6000 r--s 00000000 fd:34 104415385
/ec/dgt/local/exodus-data/build_exodustest/work/waltest/ENFR_ALL_OPTIMIZATION/lm/training-corpus.lowercase.fr.lm.7gram.arpa.binarized
2b9371db6000-2b9373358000 rw-p 2b9371db6000 00:00 0
7fff05424000-7fff05439000 rw-p 7ffffffea000 00:00 0 [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
Program received signal SIGABRT, Aborted.
0x0000003a12830265 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x0000003a12830265 in raise () from /lib64/libc.so.6
#1 0x0000003a12831d10 in abort () from /lib64/libc.so.6
#2 0x0000003a1286a84b in __libc_message () from /lib64/libc.so.6
#3 0x0000003a1287230f in _int_free () from /lib64/libc.so.6
#4 0x0000003a1287276b in free () from /lib64/libc.so.6
#5 0x000000000041e0f1 in __gnu_cxx::new_allocator<unsigned int>::deallocate
(this=0x7fff05435be0, __p=0x1b4d8930)
at
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/ext/new_allocator.h:94
#6 0x000000000041e123 in std::_Vector_base<unsigned int,
std::allocator<unsigned int> >::_M_deallocate (
this=0x7fff05435be0, __p=0x1b4d8930, __n=6)
at
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:133
#7 0x000000000041e164 in ~_Vector_base (this=0x7fff05435be0)
at
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:119
#8 0x000000000041e237 in ~vector (this=0x7fff05435be0)
at
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/bits/stl_vector.h:272
#9 0x0000000000587bd4 in Evaluate (this=0xd6b3fe0, hypo=@0x1c0aa250,
ps=0x1d1f91d0, out=0x1c0aa2b0)
at moses/src/LM/Ken.cpp:243
#10 0x0000000000444fb9 in Moses::Hypothesis::CalcScore (this=0x1c0aa250,
futureScore=@0x109b5c80)
at moses/src/Hypothesis.cpp:289
#11 0x0000000000483efc in Moses::SearchNormal::ExpandHypothesis
(this=0x109b5ef0, hypothesis=@0x1d1f8f60,
transOpt=@0x1b6bc190, expectedScore=0) at moses/src/SearchNormal.cpp:305
#12 0x00000000004842f9 in Moses::SearchNormal::ExpandAllHypotheses
(this=0x109b5ef0, hypothesis=@0x1d1f8f60,
startPos=0, endPos=3) at moses/src/SearchNormal.cpp:275
#13 0x000000000048494f in Moses::SearchNormal::ProcessOneHypothesis
(this=0x109b5ef0, hypothesis=@0x1d1f8f60)
at moses/src/SearchNormal.cpp:223
#14 0x0000000000484d03 in Moses::SearchNormal::ProcessSentence
(this=0x109b5ef0) at moses/src/SearchNormal.cpp:96
#15 0x0000000000451cc7 in Moses::Manager::ProcessSentence (this=0x7fff05436530)
at moses/src/Manager.cpp:103
#16 0x000000000040d3b5 in TranslationTask::Run (this=0x109b5bf0) at
moses-cmd/src/Main.cpp:106
#17 0x0000000000409010 in main (argc=3, argv=0x7fff05436b98) at
moses-cmd/src/Main.cpp:487
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
