Barry, Your latest configure updates in svn rev 3990 resolved the problem. configure/make now work without the need to manipulate ltmain.sh (below).
Thanks, Tom On Fri, 20 May 2011 01:16:10 -0400, Kenneth Heafield <[email protected]> wrote: > I was able to resolve the other (fork bomb) issue, which is > orthogonal > to the --with-boost-thread issue, by > > rm ltmain.sh > touch ltmain.sh > ./regenerate-makefiles.sh > ./configure --favorite-options > make -jN > > It appears that the ltmain.sh distributed with Moses should never be > used. It's only there because automake checks for it (but does > nothing > with it, so an empty file sufficies) and is then overwritten with a > symlink by the later libtoolize. > > So if nobody objects (read: I'm afraid this will break the build for > somebody and want to spread the blame), I'll svn rm ltmain.sh and > edit > regnerate-makefiles.sh to add > > if [ ! -f ltmain.sh ]; then > echo -e "#!/bin/bash\necho Run libtoolize 1>&2"> ltmain.sh > chmod +x ltmain.sh > fi > > before automake. > > That will prevent people from fork bombing their machine. This will > also get rid of that annoying commit message because ltmain.sh won't > be > in the repository. > > On 05/19/11 23:47, Tom Hoar wrote: >> I'm glad you can replicate the problem. Easier to fix that way. >> >> On Thu, 19 May 2011 23:42:55 -0400, Kenneth Heafield >> <[email protected]> wrote: >>> Apparently this is a libtool issue, not one with my code. Probably >>> what >>> happened is a new libtool went out and broke everything. Somebody >>> with >>> more experience with GNU awfultools should fix this. >>> >>> I'm able to reproduce this on one machine. If you watch ps, you'll >>> notice that libtool is fork bombing you: >>> >>> /bin/sh ../libtool --no-reexec >>> /bin/sed s%^.*/%% >>> >>> It appears to be the same as these bugs: >>> >>> http://mail-index.netbsd.org/netbsd-bugs/2001/11/24/0006.html >>> >>> >>> >>> http://forums.opensuse.org/english/other-forums/development/programming-scripting/402683-kdevelop-problem.html >>> >>> http://www.handle.net/mail-archive/handle-info/msg00442.html >>> >>> My code compiles correctly and you can verify this by running >>> >>> cd kenlm >>> ./compile.sh >>> >>> On 05/19/11 23:17, Tom Hoar wrote: >>>> Thanks Ken. It's possible I have other problems on this host. So, >>>> if >>>> you >>>> can't reproduce I'll have to rebuild. >>>> >>>> @Others: regarding configure's WARNING: unrecognized options: >>>> --with-boost-thread, is this option still required? >>>> >>>> Tom >>>> >>>> >>>> >>>> On Thu, 19 May 2011 23:05:53 -0400, Kenneth Heafield >>>> <[email protected]> wrote: >>>>> Hmmm. . . looks like it's crashing on lm/lm_exception.cc and >>>>> lm/config.cc which are mine. But the compiler should throw you >>>>> an >>>>> error >>>>> instead of taking infinite memory. See if I can reproduce. >>>>> >>>>> On 05/19/11 22:58, [email protected] wrote: >>>>>> I'm updating to the newest moses trunk thread, 3981. This >>>>>> command >>>>>> line >>>>>> worked with rev 3675 but now fails: >>>>>> >>>>>> ./configure --with-srilm=/opt/lib/srilm \ >>>>>> --with-irstlm=/opt/lib/irstlm \ >>>>>> --with-randlm=/opt/lib/randlm \ >>>>>> --enable-threads \ >>>>>> --with-xmlrpc-c \ >>>>>> --with-boost-thread >>>>>> >>>>>> Configure reports the boost libraries are installed but then >>>>>> fails (see >>>>>> configure output below). Has there been a change that obsoletes >>>>>> the >>>>>> --with-boost-thread option? >>>>>> >>>>>> Without the --with-boost-thread option, configure still finds >>>>>> the >>>>>> boost >>>>>> libraries and completes without error. However, "make -j 2" >>>>>> starts with >>>>>> the following output and hangs. Both CPUs peg 100%, RAM reaches >>>>>> +90% >>>>>> quickly and swap file usage grows. It's still running, but I >>>>>> expect it >>>>>> will crash. >>>>>> >>>>>> Other details: >>>>>> OS: Ubuntu 10.04 server >>>>>> Build environment: apt-get install linux-headers-`uname -r` >>>>>> build-essential zlib1g-dev automake libtool libboost-all-dev >>>>>> libxmlrpc-c3-dev >>>>>> SRILM release 1.5.12 >>>>>> IRSTLM release 5.60.03 >>>>>> RANDLM release 0.20 >>>>>> >>>>>> Thanks, >>>>>> Tom >>>>>> >>>>>> user@host:~$ make -j 2 >>>>>> (CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash >>>>>> /opt/src/mosesdecoder/missing --run autoheader) >>>>>> rm -f stamp-h1 >>>>>> touch config.h.in >>>>>> cd . && /bin/bash ./config.status config.h >>>>>> config.status: creating config.h >>>>>> config.status: config.h is unchanged >>>>>> make all-recursive >>>>>> make[1]: Entering directory `/opt/src/mosesdecoder' >>>>>> Making all in kenlm >>>>>> make[2]: Entering directory `/opt/src/mosesdecoder/kenlm' >>>>>> /bin/bash ../libtool --tag=CXX --mode=compile g++ >>>>>> -DHAVE_CONFIG_H >>>>>> -I. -I.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64 >>>>>> -D_LARGE_FILES >>>>>> -pthread -DTRACE_ENABLE=1 -DWITH_THREADS >>>>>> -I/opt/lib/srilm/include >>>>>> -I/opt/lib/irstlm/include -I/opt/src/mosesdecoder/kenlm >>>>>> -I/opt/lib/randlm/include -g -O2 -MT lm_exception.lo -MD -MP >>>>>> -MF >>>>>> .deps/lm_exception.Tpo -c -o lm_exception.lo `test -f >>>>>> 'lm/lm_exception.cc' || echo './'`lm/lm_exception.cc >>>>>> /bin/bash ../libtool --tag=CXX --mode=compile g++ >>>>>> -DHAVE_CONFIG_H >>>>>> -I. -I.. -W -Wall -ffor-scope -D_FILE_OFFSET_BITS=64 >>>>>> -D_LARGE_FILES >>>>>> -pthread -DTRACE_ENABLE=1 -DWITH_THREADS >>>>>> -I/opt/lib/srilm/include >>>>>> -I/opt/lib/irstlm/include -I/opt/src/mosesdecoder/kenlm >>>>>> -I/opt/lib/randlm/include -g -O2 -MT config.lo -MD -MP -MF >>>>>> .deps/config.Tpo -c -o config.lo `test -f 'lm/config.cc' || >>>>>> echo >>>>>> './'`lm/config.cc >>>>>> >>>>>> >>>>>> config.log: >>>>>> . >>>>>> . >>>>>> . >>>>>> checking for XMLRPC-C... ok >>>>>> configure: trace enabled (default) >>>>>> configure: Using Boost library >>>>>> checking for Boost headers version >= 103600... yes >>>>>> checking for Boost's header version... 1_40 >>>>>> configure: Building threaded moses >>>>>> checking for the flags needed to use pthreads... -pthread >>>>>> checking for the toolset name used by Boost for g++... gcc44 >>>>>> -gcc >>>>>> checking boost/thread.hpp usability... yes >>>>>> checking boost/thread.hpp presence... yes >>>>>> checking for boost/thread.hpp... yes >>>>>> checking for the Boost thread library... yes >>>>>> . >>>>>> . >>>>>> . >>>>>> configure: WARNING: unrecognized options: --with-boost-thread >>>>>> >>>>>> >>>>>> >>>>>> On Sat, 12 Mar 2011 15:01:14 +0100, Alexander Fraser >>>>>> <[email protected]> wrote: >>>>>>> Hi Barry, >>>>>>> >>>>>>> I got everything to work, thanks again. >>>>>>> >>>>>>> In case this is useful to someone else... I made two false >>>>>>> assumptions: >>>>>>> >>>>>>> 1) RandLM 0.20 supports multi-threading by default. This is not >>>>>>> true, >>>>>>> you first need to uncomment two lines in the Makefile (this is >>>>>>> documented in the README in src/ ). If you don't do this, and >>>>>>> you >>>>>>> build a multi-threaded version of Moses against the >>>>>>> single-threaded >>>>>>> randlm, then the build succeeds (and you can get a usage by >>>>>>> running >>>>>>> the binary with no arguments), but when you try to run a >>>>>>> regression >>>>>>> test, it dumps core. >>>>>>> >>>>>>> 2) The regression tests fail with no external LMs because of >>>>>>> some >>>>>>> problem. This is also not true, the regression tests require >>>>>>> you >>>>>>> to >>>>>>> configure with SRILM. (For some reason I thought they now >>>>>>> worked >>>>>>> with >>>>>>> KenLM) >>>>>>> >>>>>>> Cheers, Alex >>>>>>> >>>>>>> >>>>>>> On Sat, Mar 12, 2011 at 12:49 PM, Barry Haddow >>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>> Hi Alex >>>>>>>> >>>>>>>> Good to hear you got boost working. However moses shouldn't >>>>>>>> dump >>>>>>>> core, and the >>>>>>>> current svn version passes all regression tests for me. >>>>>>>> >>>>>>>> Can you get a stack trace from the core file? You may have to >>>>>>>> do >>>>>>>> a >>>>>>>> non- >>>>>>>> optimised build to get something sensible (run export >>>>>>>> CXXFLAGS=-g >>>>>>>> before >>>>>>>> ./configure, I think), >>>>>>>> >>>>>>>> cheers - Barry >>>>>>>> >>>>>>>> On Saturday 12 Mar 2011 11:39:56 Alexander Fraser wrote: >>>>>>>>> Hi Barry, >>>>>>>>> >>>>>>>>> Thanks for the reminder to check config.log, it was picking >>>>>>>>> up >>>>>>>>> /home/users6/fraser/include/boost-1_34_1_AMF_DISABLED_THIS >>>>>>>>> rather >>>>>>>>> than >>>>>>>>> /home/users6/fraser/include/boost. >>>>>>>>> >>>>>>>>> It now compiles, and you can get a usage. (I had to upgrade >>>>>>>>> IRSTLM >>>>>>>>> from 5.50.02 (which there is currently a warning message >>>>>>>>> about) >>>>>>>>> to >>>>>>>>> 5.60.1 to compile (I got 5.60.01 from the IRSTLM home page, >>>>>>>>> sourceforge offers 5.50.01); I also no longer need the >>>>>>>>> LD_LIBRARY_PATH). >>>>>>>>> >>>>>>>>> However, it dumps core as soon as you try to do anything >>>>>>>>> (e.g., >>>>>>>>> the >>>>>>>>> regression tests on both chart and phrase decoders). >>>>>>>>> >>>>>>>>> I tried a fresh checkout, with *no* configure flags (i.e., no >>>>>>>>> external >>>>>>>>> language models, no boost). This also dumps core. Let me know >>>>>>>>> if >>>>>>>>> you >>>>>>>>> can not repeat this, and thanks for your help! >>>>>>>>> >>>>>>>>> Cheers, Alex >>>>>>>>> >>>>>>>>> On Fri, Mar 11, 2011 at 7:09 PM, Barry Haddow >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> Hi Alex >>>>>>>>>> >>>>>>>>>> The boost macro was recently changed to this one >>>>>>>>>> https://github.com/tsuna/boost.m4 >>>>>>>>>> which seems to work better in most circumstances. >>>>>>>>>> >>>>>>>>>> Do you have boost installed in /home/users6/fraser (with lib >>>>>>>>>> and >>>>>>>>>> include), or just compiled there? Is its version > 1.36? Can >>>>>>>>>> you >>>>>>>>> send me >>>>>>>>>> your config.log? >>>>>>>>>> >>>>>>>>>> best regards - Barry >>>>>>>>>> >>>>>>>>>> On Friday 11 March 2011 17:46, Alexander Fraser wrote: >>>>>>>>>>> Hi Folks, >>>>>>>>>>> >>>>>>>>>>> There seems to be some brokenness in configure for version >>>>>>>>>>> 3926. >>>>>>>>> I >>>>>>>>>>> have boost 1.45 installed in my home directory, and the >>>>>>>>>>> check >>>>>>>>> fails: >>>>>>>>>>> >>>>>>>>>>> % ./regenerate-makefiles.sh >>>>>>>>>>> % ./configure >>>>>>>>> --with-srilm=/home/users6/fraser/statmt/srilm-1.5.7 >>>>>>>>>>> --with-irstlm=/home/users6/fraser/statmt/irstlm-5.50.02 >>>>>>>>>>> --with-randlm=/home/users6/fraser/statmt/randlm-v0.20 >>>>>>>>> --enable-threads >>>>>>>>>>> --with-boost=/home/users6/fraser >>>>>>>>> --with-boost-thread=boost_thread >>>>>>>>>>> ... >>>>>>>>>>> configure: WARNING: unrecognized options: >>>>>>>>>>> --with-boost-thread >>>>>>>>>>> ... >>>>>>>>>>> configure: Using Boost library >>>>>>>>>>> checking for Boost headers version >= 103600... no >>>>>>>>>>> configure: error: cannot find Boost headers version >= >>>>>>>>>>> 103600 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> It works fine in version 3725: >>>>>>>>>>> >>>>>>>>>>> % ./regenerate-makefiles.sh >>>>>>>>>>> % ./configure >>>>>>>>> --with-srilm=/home/users6/fraser/statmt/srilm-1.5.7 >>>>>>>>>>> --with-irstlm=/home/users6/fraser/statmt/irstlm-5.50.02 >>>>>>>>>>> --with-randlm=/home/users6/fraser/statmt/randlm-v0.20 >>>>>>>>> --enable-threads >>>>>>>>>>> --with-boost=/home/users6/fraser >>>>>>>>> --with-boost-thread=boost_thread >>>>>>>>>>> ... >>>>>>>>>>> % make -j 4 >>>>>>>>>>> ... >>>>>>>>>>> % export LD_LIBRARY_PATH=/home/users6/fraser/lib >>>>>>>>>>> % moses-cmd/src/moses >>>>>>>>>>> (prints usage) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks!! >>>>>>>>>>> >>>>>>>>>>> Cheers, Alex >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Moses-support mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> The University of Edinburgh is a charitable body, registered >>>>>>>>>> in >>>>>>>>>> Scotland, with registration number SC005336. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Moses-support mailing list >>>>>>> [email protected] >>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
