Yeah it was a serparate issue, glad it was found though. Wrt moses, can you confirm which revision of Moses you have?
Regards, Oliver On Tue, Mar 15, 2011 at 05:57:33PM +0100, Alexander Fraser wrote: > Hi Oliver -- > > No worries, I don't actually use querylm, I just wanted a minimally > reproducible problem; sounds like I instead found a second problem :-) > > When I build Moses single threaded, with single threaded RandLM, > everything works fine. > > When I build Moses multi threaded, with multi threaded RandLM, then I > get a core dump if I try to use a RandLM. This occurs after it prints: > > Start loading phrase table from > /mount/corpora12/d5/statmt/fraser/experiments/0022/1000_INFLECT3_WMT2009_en_de/gdfa_noclosure/tuning_lm_parallel_RANDLM/filtered/phrase-table.0-0.1.1 > : [0.000] seconds > > The same multi-threaded binary built with multi-threaded RandLM works > fine with KenLM with 6 threads (only change in moses.ini is the Ken LM > line). > > Let me know if you need some kind of debug output. > > Cheers, Alex > > > On Tue, Mar 15, 2011 at 4:41 PM, Oliver Wilson <[email protected]> wrote: > > Hi Alex, > > > > The problem you are experiencing with the querylm utility is down to an > > oversight on my part. Compiling with threading support requires you to > > init caches etc on RandLM instances which I didn't do for querylm. I'm > > working on a fix now and will mail you off list when it's done. > > > > Are you still experiencing problems when using multithreaded RandLM from > > within moses? > > > > Regards, > > Oliver > > > > On Tue, Mar 15, 2011 at 03:58:46PM +0100, Alexander Fraser wrote: > >> Hi Miles, > >> > >> Thanks a lot for releasing a version of RandLM which supports > >> multi-threading. I'd very much like to use it together with Moses, but > >> am having significant problems. > >> > >> > >> Although I initially noticed the problem with Moses, I was able to > >> reproduce it using basic functionality: > >> > >> R=randlm_v0.20_no_threads > >> > >> echo "This is a test of the emergency broadcast system ." | $R/buildlm > >> -order 5 -struct BloomMap -falsepos 8 -values 8 -output-prefix model > >> -tmp-dir . > >> > >> echo "This is a test of" > tmp_kill2 > >> > >> $R/querylm -randlm model.BloomMap -test-path tmp_kill2 -test-type > >> ngrams -order 5 > >> > >> [works fine] > >> > >> > >> > >> R=randlm_v0.20_threads > >> > >> echo "This is a test of the emergency broadcast system ." | $R/buildlm > >> -order 5 -struct BloomMap -falsepos 8 -values 8 -output-prefix model > >> -tmp-dir . > >> > >> echo "This is a test of" > tmp_kill2 > >> > >> $R/querylm -randlm model.BloomMap -test-path tmp_kill2 -test-type > >> ngrams -order 5 > >> > >> [dumps core - if you do this in gdb, and do a backtrace > >> > >> #0 ?0x000000000043633b in randlm::RandLMHashCache::check(unsigned int > >> const*, int, float*, void const**) () > >> #1 ?0x000000000042440e in > >> randlm::StupidBackoffRandLM::getProb(unsigned int const*, int, int*, > >> void const**) () > >> #2 ?0x0000000000404bb6 in randlm::RandLMQuery::query() () > >> #3 ?0x0000000000409fd6 in main () > >> > >> ] > >> > >> > >> I built randlm on a linux machine that has boost 1.44 installed > >> (presumably with the linux distribution; note that this is not in my > >> home directory as I was doing over the weekend if you happened to see > >> those emails with Barry). For the no threads version, I made no > >> changes to the Makefile. For the threads version I used these two > >> lines (note that the first one is slightly changed from your commented > >> out lines, with "-mt" added). > >> > >> LDFLAGS+=-lboost_thread-mt > >> DEFS+=-DMULTITHREADED > >> > >> > >> Thanks for your help. > >> > >> Cheers, Alex > >> _______________________________________________ > >> Moses-support mailing list > >> [email protected] > >> http://mailman.mit.edu/mailman/listinfo/moses-support > >> > > > > -- > > The University of Edinburgh is a charitable body, registered in > > Scotland, with registration number SC005336. > > > > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
