I was able to resolve the other (fork bomb) issue, which is orthogonal to the --with-boost-thread issue, by
rm ltmain.sh touch ltmain.sh ./regenerate-makefiles.sh ./configure --favorite-options make -jN It appears that the ltmain.sh distributed with Moses should never be used. It's only there because automake checks for it (but does nothing with it, so an empty file sufficies) and is then overwritten with a symlink by the later libtoolize. So if nobody objects (read: I'm afraid this will break the build for somebody and want to spread the blame), I'll svn rm ltmain.sh and edit regnerate-makefiles.sh to add if [ ! -f ltmain.sh ]; then echo -e "#!/bin/bash\necho Run libtoolize 1>&2"> ltmain.sh chmod +x ltmain.sh fi before automake. That will prevent people from fork bombing their machine. This will also get rid of that annoying commit message because ltmain.sh won't be in the repository. On 05/19/11 23:47, Tom Hoar wrote: > I'm glad you can replicate the problem. Easier to fix that way. > > On Thu, 19 May 2011 23:42:55 -0400, Kenneth Heafield > <[email protected]> wrote: >> Apparently this is a libtool issue, not one with my code. Probably >> what >> happened is a new libtool went out and broke everything. Somebody >> with >> more experience with GNU awfultools should fix this. >> >> I'm able to reproduce this on one machine. If you watch ps, you'll >> notice that libtool is fork bombing you: >> >> /bin/sh ../libtool --no-reexec >> /bin/sed s%^.*/%% >> >> It appears to be the same as these bugs: >> >> http://mail-index.netbsd.org/netbsd-bugs/2001/11/24/0006.html >> >> >> http://forums.opensuse.org/english/other-forums/development/programming-scripting/402683-kdevelop-problem.html >> >> http://www.handle.net/mail-archive/handle-info/msg00442.html >> >> My code compiles correctly and you can verify this by running >> >> cd kenlm >> ./compile.sh >> >> On 05/19/11 23:17, Tom Hoar wrote: >>> Thanks Ken. It's possible I have other problems on this host. So, if >>> you >>> can't reproduce I'll have to rebuild. >>> >>> @Others: regarding configure's WARNING: unrecognized options: >>> --with-boost-thread, is this option still required? >>> >>> Tom >>> >>> >>> >>> On Thu, 19 May 2011 23:05:53 -0400, Kenneth Heafield >>> <[email protected]> wrote: >>>> Hmmm. . . looks like it's crashing on lm/lm_exception.cc and >>>> lm/config.cc which are mine. But the compiler should throw you an >>>> error >>>> instead of taking infinite memory. See if I can reproduce. >>>> >>>> On 05/19/11 22:58, [email protected] wrote: >>>>> I'm updating to the newest moses trunk thread, 3981. This command >>>>> line >>>>> worked with rev 3675 but now fails: >>>>> >>>>> ./configure --with-srilm=/opt/lib/srilm \ >>>>> --with-irstlm=/opt/lib/irstlm \ >>>>> --with-randlm=/opt/lib/randlm \ >>>>> --enable-threads \ >>>>> --with-xmlrpc-c \ >>>>> --with-boost-thread >>>>> >>>>> Configure reports the boost libraries are installed but then >>>>> fails (see >>>>> configure output below). Has there been a change that obsoletes >>>>> the >>>>> --with-boost-thread option? >>>>> >>>>> Without the --with-boost-thread option, configure still finds the >>>>> boost >>>>> libraries and completes without error. However, "make -j 2" >>>>> starts with >>>>> the following output and hangs. Both CPUs peg 100%, RAM reaches >>>>> +90% >>>>> quickly and swap file usage grows. It's still running, but I >>>>> expect it >>>>> will crash. >>>>> >>>>> Other details: >>>>> OS: Ubuntu 10.04 server >>>>> Build environment: apt-get install linux-headers-`uname -r` >>>>> build-essential zlib1g-dev automake libtool libboost-all-dev >>>>> libxmlrpc-c3-dev >>>>> SRILM release 1.5.12 >>>>> IRSTLM release 5.60.03 >>>>> RANDLM release 0.20 >>>>> >>>>> Thanks, >>>>> Tom >>>>> >>>>> user@host:~$ make -j 2 >>>>> (CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash >>>>> /opt/src/mosesdecoder/missing --run autoheader) >>>>> rm -f stamp-h1 >>>>> touch config.h.in >>>>> cd . && /bin/bash ./config.status config.h >>>>> config.status: creating config.h >>>>> config.status: config.h is unchanged >>>>> make all-recursive >>>>> make[1]: Entering directory `/opt/src/mosesdecoder' >>>>> Making all in kenlm >>>>> make[2]: Entering directory `/opt/src/mosesdecoder/kenlm' >>>>> /bin/bash ../libtool --tag=CXX --mode=compile g++ >>>>> -DHAVE_CONFIG_H >>>>> -I. -I.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64 >>>>> -D_LARGE_FILES >>>>> -pthread -DTRACE_ENABLE=1 -DWITH_THREADS -I/opt/lib/srilm/include >>>>> -I/opt/lib/irstlm/include -I/opt/src/mosesdecoder/kenlm >>>>> -I/opt/lib/randlm/include -g -O2 -MT lm_exception.lo -MD -MP -MF >>>>> .deps/lm_exception.Tpo -c -o lm_exception.lo `test -f >>>>> 'lm/lm_exception.cc' || echo './'`lm/lm_exception.cc >>>>> /bin/bash ../libtool --tag=CXX --mode=compile g++ >>>>> -DHAVE_CONFIG_H >>>>> -I. -I.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64 >>>>> -D_LARGE_FILES >>>>> -pthread -DTRACE_ENABLE=1 -DWITH_THREADS -I/opt/lib/srilm/include >>>>> -I/opt/lib/irstlm/include -I/opt/src/mosesdecoder/kenlm >>>>> -I/opt/lib/randlm/include -g -O2 -MT config.lo -MD -MP -MF >>>>> .deps/config.Tpo -c -o config.lo `test -f 'lm/config.cc' || echo >>>>> './'`lm/config.cc >>>>> >>>>> >>>>> config.log: >>>>> . >>>>> . >>>>> . >>>>> checking for XMLRPC-C... ok >>>>> configure: trace enabled (default) >>>>> configure: Using Boost library >>>>> checking for Boost headers version >= 103600... yes >>>>> checking for Boost's header version... 1_40 >>>>> configure: Building threaded moses >>>>> checking for the flags needed to use pthreads... -pthread >>>>> checking for the toolset name used by Boost for g++... gcc44 -gcc >>>>> checking boost/thread.hpp usability... yes >>>>> checking boost/thread.hpp presence... yes >>>>> checking for boost/thread.hpp... yes >>>>> checking for the Boost thread library... yes >>>>> . >>>>> . >>>>> . >>>>> configure: WARNING: unrecognized options: --with-boost-thread >>>>> >>>>> >>>>> >>>>> On Sat, 12 Mar 2011 15:01:14 +0100, Alexander Fraser >>>>> <[email protected]> wrote: >>>>>> Hi Barry, >>>>>> >>>>>> I got everything to work, thanks again. >>>>>> >>>>>> In case this is useful to someone else... I made two false >>>>>> assumptions: >>>>>> >>>>>> 1) RandLM 0.20 supports multi-threading by default. This is not >>>>>> true, >>>>>> you first need to uncomment two lines in the Makefile (this is >>>>>> documented in the README in src/ ). If you don't do this, and you >>>>>> build a multi-threaded version of Moses against the >>>>>> single-threaded >>>>>> randlm, then the build succeeds (and you can get a usage by >>>>>> running >>>>>> the binary with no arguments), but when you try to run a >>>>>> regression >>>>>> test, it dumps core. >>>>>> >>>>>> 2) The regression tests fail with no external LMs because of some >>>>>> problem. This is also not true, the regression tests require you >>>>>> to >>>>>> configure with SRILM. (For some reason I thought they now worked >>>>>> with >>>>>> KenLM) >>>>>> >>>>>> Cheers, Alex >>>>>> >>>>>> >>>>>> On Sat, Mar 12, 2011 at 12:49 PM, Barry Haddow >>>>>> <[email protected]> >>>>>> wrote: >>>>>>> Hi Alex >>>>>>> >>>>>>> Good to hear you got boost working. However moses shouldn't dump >>>>>>> core, and the >>>>>>> current svn version passes all regression tests for me. >>>>>>> >>>>>>> Can you get a stack trace from the core file? You may have to do >>>>>>> a >>>>>>> non- >>>>>>> optimised build to get something sensible (run export >>>>>>> CXXFLAGS=-g >>>>>>> before >>>>>>> ./configure, I think), >>>>>>> >>>>>>> cheers - Barry >>>>>>> >>>>>>> On Saturday 12 Mar 2011 11:39:56 Alexander Fraser wrote: >>>>>>>> Hi Barry, >>>>>>>> >>>>>>>> Thanks for the reminder to check config.log, it was picking up >>>>>>>> /home/users6/fraser/include/boost-1_34_1_AMF_DISABLED_THIS >>>>>>>> rather >>>>>>>> than >>>>>>>> /home/users6/fraser/include/boost. >>>>>>>> >>>>>>>> It now compiles, and you can get a usage. (I had to upgrade >>>>>>>> IRSTLM >>>>>>>> from 5.50.02 (which there is currently a warning message about) >>>>>>>> to >>>>>>>> 5.60.1 to compile (I got 5.60.01 from the IRSTLM home page, >>>>>>>> sourceforge offers 5.50.01); I also no longer need the >>>>>>>> LD_LIBRARY_PATH). >>>>>>>> >>>>>>>> However, it dumps core as soon as you try to do anything (e.g., >>>>>>>> the >>>>>>>> regression tests on both chart and phrase decoders). >>>>>>>> >>>>>>>> I tried a fresh checkout, with *no* configure flags (i.e., no >>>>>>>> external >>>>>>>> language models, no boost). This also dumps core. Let me know >>>>>>>> if >>>>>>>> you >>>>>>>> can not repeat this, and thanks for your help! >>>>>>>> >>>>>>>> Cheers, Alex >>>>>>>> >>>>>>>> On Fri, Mar 11, 2011 at 7:09 PM, Barry Haddow >>>>>>>> <[email protected]> wrote: >>>>>>>>> Hi Alex >>>>>>>>> >>>>>>>>> The boost macro was recently changed to this one >>>>>>>>> https://github.com/tsuna/boost.m4 >>>>>>>>> which seems to work better in most circumstances. >>>>>>>>> >>>>>>>>> Do you have boost installed in /home/users6/fraser (with lib >>>>>>>>> and >>>>>>>>> include), or just compiled there? Is its version > 1.36? Can >>>>>>>>> you >>>>>>>> send me >>>>>>>>> your config.log? >>>>>>>>> >>>>>>>>> best regards - Barry >>>>>>>>> >>>>>>>>> On Friday 11 March 2011 17:46, Alexander Fraser wrote: >>>>>>>>>> Hi Folks, >>>>>>>>>> >>>>>>>>>> There seems to be some brokenness in configure for version >>>>>>>>>> 3926. >>>>>>>> I >>>>>>>>>> have boost 1.45 installed in my home directory, and the check >>>>>>>> fails: >>>>>>>>>> >>>>>>>>>> % ./regenerate-makefiles.sh >>>>>>>>>> % ./configure >>>>>>>> --with-srilm=/home/users6/fraser/statmt/srilm-1.5.7 >>>>>>>>>> --with-irstlm=/home/users6/fraser/statmt/irstlm-5.50.02 >>>>>>>>>> --with-randlm=/home/users6/fraser/statmt/randlm-v0.20 >>>>>>>> --enable-threads >>>>>>>>>> --with-boost=/home/users6/fraser >>>>>>>> --with-boost-thread=boost_thread >>>>>>>>>> ... >>>>>>>>>> configure: WARNING: unrecognized options: --with-boost-thread >>>>>>>>>> ... >>>>>>>>>> configure: Using Boost library >>>>>>>>>> checking for Boost headers version >= 103600... no >>>>>>>>>> configure: error: cannot find Boost headers version >= 103600 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> It works fine in version 3725: >>>>>>>>>> >>>>>>>>>> % ./regenerate-makefiles.sh >>>>>>>>>> % ./configure >>>>>>>> --with-srilm=/home/users6/fraser/statmt/srilm-1.5.7 >>>>>>>>>> --with-irstlm=/home/users6/fraser/statmt/irstlm-5.50.02 >>>>>>>>>> --with-randlm=/home/users6/fraser/statmt/randlm-v0.20 >>>>>>>> --enable-threads >>>>>>>>>> --with-boost=/home/users6/fraser >>>>>>>> --with-boost-thread=boost_thread >>>>>>>>>> ... >>>>>>>>>> % make -j 4 >>>>>>>>>> ... >>>>>>>>>> % export LD_LIBRARY_PATH=/home/users6/fraser/lib >>>>>>>>>> % moses-cmd/src/moses >>>>>>>>>> (prints usage) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks!! >>>>>>>>>> >>>>>>>>>> Cheers, Alex >>>>>>>>>> _______________________________________________ >>>>>>>>>> Moses-support mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>>>> >>>>>>>>> -- >>>>>>>>> The University of Edinburgh is a charitable body, registered >>>>>>>>> in >>>>>>>>> Scotland, with registration number SC005336. >>>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
