Hi Liangyou
This is a bit strange. The stacktrace suggests corruption in the feature
vector, but it would be surprising that we haven't seen it before. Can
you reproduce the problem when running under valgrind? The memory could
be getting corrupted by a completely different source, which we'll
really only find with a memory checker.
cheers - Barry
On 11/03/15 17:50, Liangyou Li wrote:
Thanks Matthias.
This problem doesn't always happen.
Even on the same data, sometimes it can run without crash, sometimes
it doesn't.
But, by now, the error only happens when the decoder outputs features.
(-n-best-list or -output-search-graph-hypergraph)
Cheers
Liangyou
On Wed, Mar 11, 2015 at 5:38 PM, Matthias Huck <[email protected]
<mailto:[email protected]>> wrote:
Hi,
I've recently been using these sparse feature functions without any
issues in multi-threaded chart-based decoding. There might be a
problem
with thread safety, but I currently can't tell why you got the
segmentation fault. You should investigate this in more detail.
Cheers,
Matthias
On Wed, 2015-03-11 at 12:12 +0000, Liangyou Li wrote:
> When I run moses with sparse features in multi-threads (
parameter -threads all), I got segmentation fault.
>
>
> The three sparse features are:
> SourceWordDeletionFeature factor=0
> TargetWordInsertionFeature factor=0
> WordTranslationFeature input-factor=0 output-factor=0
>
>
> The exact command I used is:
> moses_chart -threads all -f moses.ini -i input -n-best-list
nbest 100 distinct
>
>
> This error happens kind of randomly. But it only happens when
sparse features and multi-threads are used.
>
>
> I've tried several times to use gdb to trace the error.
Fortunately, I just get the back-trace info, as listed:
>
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffdae77700 (LWP 32390)]
> 0x00007ffff711f5e3 in std::basic_ostream<char,
std::char_traits<char> >& std::operator<< <char,
std::char_traits<char>, std::allocator<char>
>(std::basic_ostream<char, std::char_traits<char> >&,
std::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&) () from /usr/lib64/libstdc++.so.6
> Missing separate debuginfos, use: zypper install
glibc-debuginfo-2.15-22.17.1.x86_64
libgcc47-debuginfo-4.7.1_20120723-1.1.1.x86_64
liblzma5-debuginfo-5.0.3-12.2.2.x86_64
libstdc++47-debuginfo-4.7.1_20120723-1.1.1.x86_64
zlib-debuginfo-1.2.7-2.1.2.x86_64
> (gdb) backtrace
> #0 0x00007ffff711f5e3 in std::basic_ostream<char,
std::char_traits<char> >& std::operator<< <char,
std::char_traits<char>, std::allocator<char>
>(std::basic_ostream<char, std::char_traits<char> >&,
std::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&) () from /usr/lib64/libstdc++.so.6
> #1 0x0000000000584711 in Moses::operator<< (out=..., name=...)
at moses/FeatureVector.cpp:141
> #2 0x0000000000440232 in
Moses::ScoreComponentCollection::GetVectorForProducer
(this=0x7fffad0e9380, sp=0xe2aab0) at
moses/ScoreComponentCollection.cpp:293
> #3 0x00000000004407da in
Moses::ScoreComponentCollection::OutputFeatureScores
(this=0x7fffad0e9380, out=..., ff=0xe2aab0, lastName="LM0") at
moses/ScoreComponentCollection.cpp:351
> #4 0x000000000044055a in
Moses::ScoreComponentCollection::OutputAllFeatureScores
(this=0x7fffad0e9380, out=...) at
moses/ScoreComponentCollection.cpp:319
> #5 0x0000000000561ce4 in Moses::ChartManager::OutputNBestList
(this=0x7fffacbe0010, collector=0x9320d210, nBestList=std::vector
of length 100, capacity 128 = {...}, translationId=569)
> at moses/ChartManager.cpp:381
> #6 0x00000000005619d6 in Moses::ChartManager::OutputNBest
(this=0x7fffacbe0010, collector=0x9320d210) at
moses/ChartManager.cpp:335
> #7 0x0000000000467cd6 in Moses::TranslationTask::Run
(this=0x214d90b0) at moses/TranslationTask.cpp:111
> #8 0x0000000000530b69 in Moses::ThreadPool::Execute
(this=0x7fffffffd710) at moses/ThreadPool.cpp:61
> #9 0x0000000000534ecd in boost::_mfi::mf0<void,
Moses::ThreadPool>::operator() (this=0xf263088, p=0x7fffffffd710)
at
/home/lly/plateform/boost_1_55_0/include/boost/bind/mem_fn_template.hpp:49
> #10 0x0000000000534e30 in
boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*>
>::operator()<boost::_mfi::mf0<void, Moses::ThreadPool>,
boost::_bi::list0> (this=0xf263098, f=..., a=...)
> at
/home/lly/plateform/boost_1_55_0/include/boost/bind/bind.hpp:253
> #11 0x0000000000534dd5 in boost::_bi::bind_t<void,
boost::_mfi::mf0<void, Moses::ThreadPool>,
boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> >
>::operator() (this=0xf263088)
> at
/home/lly/plateform/boost_1_55_0/include/boost/bind/bind_template.hpp:20
> #12 0x0000000000534d9a in
boost::detail::thread_data<boost::_bi::bind_t<void,
boost::_mfi::mf0<void, Moses::ThreadPool>,
boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > > >::run
(this=0xf262ed0)
> at
/home/lly/plateform/boost_1_55_0/include/boost/thread/detail/thread.hpp:117
> #13 0x00000000008b4752 in thread_proxy ()
> #14 0x00007ffff6965e0e in start_thread () from
/lib64/libpthread.so.0
> #15 0x00007ffff669d2cd in clone () from /lib64/libc.so.6
>
>
> Has anyone had the problem before ? Any ideas on solving this ?
> Many Thanks ?
>
>
>
>
> PS: In my experiment, I found the function " void
CompletedRuleCollection::Add(const TargetPhraseCollection &tpc,
const StackVec &stackVec, const std::vector<float> &stackScores,
const ChartParserCallback &outColl) " does not consider "
m_ruleLimit " . So after adding parameter " -rule-limit 0 ", the
decoder can only collect one translation option.
>
>
>
>
> Cheers
> Liangyou
>
>
>
>
>
>
> --
>
>
> Liangyou Li
> CNGL
> School of Computing
> Dublin City University
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected] <mailto:[email protected]>
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
Liangyou Li
CNGL
School of Computing
Dublin City University
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
