Thanks Matthias. This problem doesn't always happen. Even on the same data, sometimes it can run without crash, sometimes it doesn't. But, by now, the error only happens when the decoder outputs features. (-n-best-list or -output-search-graph-hypergraph)
Cheers Liangyou On Wed, Mar 11, 2015 at 5:38 PM, Matthias Huck <[email protected]> wrote: > Hi, > > I've recently been using these sparse feature functions without any > issues in multi-threaded chart-based decoding. There might be a problem > with thread safety, but I currently can't tell why you got the > segmentation fault. You should investigate this in more detail. > > Cheers, > Matthias > > > > On Wed, 2015-03-11 at 12:12 +0000, Liangyou Li wrote: > > When I run moses with sparse features in multi-threads ( parameter > -threads all), I got segmentation fault. > > > > > > The three sparse features are: > > SourceWordDeletionFeature factor=0 > > TargetWordInsertionFeature factor=0 > > WordTranslationFeature input-factor=0 output-factor=0 > > > > > > The exact command I used is: > > moses_chart -threads all -f moses.ini -i input -n-best-list nbest > 100 distinct > > > > > > This error happens kind of randomly. But it only happens when sparse > features and multi-threads are used. > > > > > > I've tried several times to use gdb to trace the error. Fortunately, I > just get the back-trace info, as listed: > > > > > > Program received signal SIGSEGV, Segmentation fault. > > [Switching to Thread 0x7fffdae77700 (LWP 32390)] > > 0x00007ffff711f5e3 in std::basic_ostream<char, std::char_traits<char> >& > std::operator<< <char, std::char_traits<char>, std::allocator<char> > >(std::basic_ostream<char, std::char_traits<char> >&, > std::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&) () from /usr/lib64/libstdc++.so.6 > > Missing separate debuginfos, use: zypper install > glibc-debuginfo-2.15-22.17.1.x86_64 > libgcc47-debuginfo-4.7.1_20120723-1.1.1.x86_64 > liblzma5-debuginfo-5.0.3-12.2.2.x86_64 > libstdc++47-debuginfo-4.7.1_20120723-1.1.1.x86_64 > zlib-debuginfo-1.2.7-2.1.2.x86_64 > > (gdb) backtrace > > #0 0x00007ffff711f5e3 in std::basic_ostream<char, > std::char_traits<char> >& std::operator<< <char, std::char_traits<char>, > std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&, > std::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&) () from /usr/lib64/libstdc++.so.6 > > #1 0x0000000000584711 in Moses::operator<< (out=..., name=...) at > moses/FeatureVector.cpp:141 > > #2 0x0000000000440232 in > Moses::ScoreComponentCollection::GetVectorForProducer (this=0x7fffad0e9380, > sp=0xe2aab0) at moses/ScoreComponentCollection.cpp:293 > > #3 0x00000000004407da in > Moses::ScoreComponentCollection::OutputFeatureScores (this=0x7fffad0e9380, > out=..., ff=0xe2aab0, lastName="LM0") at > moses/ScoreComponentCollection.cpp:351 > > #4 0x000000000044055a in > Moses::ScoreComponentCollection::OutputAllFeatureScores > (this=0x7fffad0e9380, out=...) at moses/ScoreComponentCollection.cpp:319 > > #5 0x0000000000561ce4 in Moses::ChartManager::OutputNBestList > (this=0x7fffacbe0010, collector=0x9320d210, nBestList=std::vector of length > 100, capacity 128 = {...}, translationId=569) > > at moses/ChartManager.cpp:381 > > #6 0x00000000005619d6 in Moses::ChartManager::OutputNBest > (this=0x7fffacbe0010, collector=0x9320d210) at moses/ChartManager.cpp:335 > > #7 0x0000000000467cd6 in Moses::TranslationTask::Run (this=0x214d90b0) > at moses/TranslationTask.cpp:111 > > #8 0x0000000000530b69 in Moses::ThreadPool::Execute > (this=0x7fffffffd710) at moses/ThreadPool.cpp:61 > > #9 0x0000000000534ecd in boost::_mfi::mf0<void, > Moses::ThreadPool>::operator() (this=0xf263088, p=0x7fffffffd710) at > /home/lly/plateform/boost_1_55_0/include/boost/bind/mem_fn_template.hpp:49 > > #10 0x0000000000534e30 in > boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > >::operator()<boost::_mfi::mf0<void, Moses::ThreadPool>, boost::_bi::list0> > (this=0xf263098, f=..., a=...) > > at /home/lly/plateform/boost_1_55_0/include/boost/bind/bind.hpp:253 > > #11 0x0000000000534dd5 in boost::_bi::bind_t<void, > boost::_mfi::mf0<void, Moses::ThreadPool>, > boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > >::operator() > (this=0xf263088) > > at > /home/lly/plateform/boost_1_55_0/include/boost/bind/bind_template.hpp:20 > > #12 0x0000000000534d9a in > boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, > Moses::ThreadPool>, boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > > > >::run (this=0xf262ed0) > > at > /home/lly/plateform/boost_1_55_0/include/boost/thread/detail/thread.hpp:117 > > #13 0x00000000008b4752 in thread_proxy () > > #14 0x00007ffff6965e0e in start_thread () from /lib64/libpthread.so.0 > > #15 0x00007ffff669d2cd in clone () from /lib64/libc.so.6 > > > > > > Has anyone had the problem before ? Any ideas on solving this ? > > Many Thanks ? > > > > > > > > > > PS: In my experiment, I found the function " void > CompletedRuleCollection::Add(const TargetPhraseCollection &tpc, const > StackVec &stackVec, const std::vector<float> &stackScores, const > ChartParserCallback &outColl) " does not consider " m_ruleLimit " . So > after adding parameter " -rule-limit 0 ", the decoder can only collect one > translation option. > > > > > > > > > > Cheers > > Liangyou > > > > > > > > > > > > > > -- > > > > > > Liangyou Li > > CNGL > > School of Computing > > Dublin City University > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > -- Liangyou Li CNGL School of Computing Dublin City University
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
