Hi Oliver -- No worries, I don't actually use querylm, I just wanted a minimally reproducible problem; sounds like I instead found a second problem :-)
When I build Moses single threaded, with single threaded RandLM, everything works fine. When I build Moses multi threaded, with multi threaded RandLM, then I get a core dump if I try to use a RandLM. This occurs after it prints: Start loading phrase table from /mount/corpora12/d5/statmt/fraser/experiments/0022/1000_INFLECT3_WMT2009_en_de/gdfa_noclosure/tuning_lm_parallel_RANDLM/filtered/phrase-table.0-0.1.1 : [0.000] seconds The same multi-threaded binary built with multi-threaded RandLM works fine with KenLM with 6 threads (only change in moses.ini is the Ken LM line). Let me know if you need some kind of debug output. Cheers, Alex On Tue, Mar 15, 2011 at 4:41 PM, Oliver Wilson <[email protected]> wrote: > Hi Alex, > > The problem you are experiencing with the querylm utility is down to an > oversight on my part. Compiling with threading support requires you to > init caches etc on RandLM instances which I didn't do for querylm. I'm > working on a fix now and will mail you off list when it's done. > > Are you still experiencing problems when using multithreaded RandLM from > within moses? > > Regards, > Oliver > > On Tue, Mar 15, 2011 at 03:58:46PM +0100, Alexander Fraser wrote: >> Hi Miles, >> >> Thanks a lot for releasing a version of RandLM which supports >> multi-threading. I'd very much like to use it together with Moses, but >> am having significant problems. >> >> >> Although I initially noticed the problem with Moses, I was able to >> reproduce it using basic functionality: >> >> R=randlm_v0.20_no_threads >> >> echo "This is a test of the emergency broadcast system ." | $R/buildlm >> -order 5 -struct BloomMap -falsepos 8 -values 8 -output-prefix model >> -tmp-dir . >> >> echo "This is a test of" > tmp_kill2 >> >> $R/querylm -randlm model.BloomMap -test-path tmp_kill2 -test-type >> ngrams -order 5 >> >> [works fine] >> >> >> >> R=randlm_v0.20_threads >> >> echo "This is a test of the emergency broadcast system ." | $R/buildlm >> -order 5 -struct BloomMap -falsepos 8 -values 8 -output-prefix model >> -tmp-dir . >> >> echo "This is a test of" > tmp_kill2 >> >> $R/querylm -randlm model.BloomMap -test-path tmp_kill2 -test-type >> ngrams -order 5 >> >> [dumps core - if you do this in gdb, and do a backtrace >> >> #0 0x000000000043633b in randlm::RandLMHashCache::check(unsigned int >> const*, int, float*, void const**) () >> #1 0x000000000042440e in >> randlm::StupidBackoffRandLM::getProb(unsigned int const*, int, int*, >> void const**) () >> #2 0x0000000000404bb6 in randlm::RandLMQuery::query() () >> #3 0x0000000000409fd6 in main () >> >> ] >> >> >> I built randlm on a linux machine that has boost 1.44 installed >> (presumably with the linux distribution; note that this is not in my >> home directory as I was doing over the weekend if you happened to see >> those emails with Barry). For the no threads version, I made no >> changes to the Makefile. For the threads version I used these two >> lines (note that the first one is slightly changed from your commented >> out lines, with "-mt" added). >> >> LDFLAGS+=-lboost_thread-mt >> DEFS+=-DMULTITHREADED >> >> >> Thanks for your help. >> >> Cheers, Alex >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
