[
https://issues.apache.org/jira/browse/IMPALA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358986#comment-17358986
]
ASF subversion and git services commented on IMPALA-10413:
----------------------------------------------------------
Commit f112f92e217802e952e213a0e26b7c74db5c7379 in impala's branch
refs/heads/branch-4.0.0 from xqhe
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f112f92 ]
IMPALA-10413: fix impalad crashes when canceling the retrying query
The crash happens when canceling the retrying query.
If the original query was unregistered while the new query was
being created, it will call HandleRetryFailure to abort the new
query. But the status is ok, so when calling Status::AddDetail
impalad will crash.
After the WaitAsync interface called and before the
retry_request_state moved to retried_client_request_state_ , if
abort the new retry query, retry_request_state need to call
Finalize, otherwise the wait-thread will leak.
In some cases like canceled the original query or closed the session
we may not create the new query, so we also check whether the query
is retried.
Tests:
Add test in tests/custom_cluster/test_query_retries.py and manually
tested 100 times to make sure that there was no Impalad crash
Change-Id: I4fd7228acd0a70d33859029052239f9b9f795e5d
Reviewed-on: http://gerrit.cloudera.org:8080/16911
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Impala crashing when retrying failed query
> ------------------------------------------
>
> Key: IMPALA-10413
> URL: https://issues.apache.org/jira/browse/IMPALA-10413
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.0
> Reporter: Xianqing He
> Assignee: Xianqing He
> Priority: Minor
> Fix For: Impala 4.0
>
>
> When retrying failed query, it may crash if cancel the query
> The core stack below
> {code:java}
> #0 0x00007f1b20e87387 in raise () from /lib64/libc.so.6
> #1 0x00007f1b20e88a78 in abort () from /lib64/libc.so.6
> #2 0x00007f1b23b754b9 in os::abort(bool) () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #3 0x00007f1b23d93db6 in VMError::report_and_die() () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #4 0x00007f1b23b7f505 in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #5 0x00007f1b23b72678 in signalHandler(int, siginfo_t*, void*) () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #6 <signal handler called>
> #7 push_back (__x=..., this=0x28) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/include/c++/7.5.0/bits/stl_vector.h:941
> #8 AddDetail (d=..., this=0x0) at
> /data/Impala-ASF/be/src/util/error-util.h:114
> #9 impala::Status::AddDetail (this=this@entry=0x7f1a4b971740, msg=...) at
> /data/Impala-ASF/be/src/common/status.cc:236
> #10 0x000000000190a4fc in impala::QueryDriver::HandleRetryFailure
> (this=this@entry=0xb1b2880, status=status@entry=0x7f1a4b971740,
> error_msg=error_msg@entry=0x7f1a4b971860,
> request_state=request_state@entry=0x9ca7e00, retry_query_id=...)
> at /data/Impala-ASF/be/src/runtime/query-driver.cc:351
> #11 0x000000000190c605 in impala::QueryDriver::RetryQueryFromThread
> (this=0xb1b2880, error=..., query_driver=...) at
> /data/Impala-ASF/be/src/runtime/query-driver.cc:293
> #12 0x0000000001912459 in operator() (a2=..., a1=..., p=<optimized out>,
> this=<optimized out>) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:280
> #13 operator()<boost::_mfi::mf2<void, impala::QueryDriver, const
> impala::Status&, std::shared_ptr<impala::QueryDriver> >, boost::_bi::list0>
> (a=<synthetic pointer>, f=..., this=<optimized out>)
> at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:398
> #14 operator() (this=<optimized out>) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222
> #15
> boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
> boost::_mfi::mf2<void, impala::QueryDriver, impala::Status const&,
> std::shared_ptr<impala::QueryDriver> >,
> boost::_bi::list3<boost::_bi::value<impala::QueryDriver*>,
> boost::_bi::value<impala::Status>,
> boost::_bi::value<std::shared_ptr<impala::QueryDriver> > > >, void>::invoke
> (function_obj_ptr=...)
> at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:159
> #16 0x00000000015386f2 in operator() (this=0x7f1a4b971c00) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:770
> #17 impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*) (name=..., category=...,
> functor=..., parent_thread_info=<optimized out>,
> thread_started=0x7f1ad5750ec0) at /data/Impala-ASF/be/src/util/thread.cc:360
> #18 0x0000000001539b6b in operator()<void (*)(const
> std::__cxx11::basic_string<char>&, const std::__cxx11::basic_string<char>&,
> boost::function<void()>, const impala::ThreadDebugInfo*, impala::Promise<long
> int>*), boost::_bi::list0> (
> a=<synthetic pointer>,
> f=@0xa20de78: 0x15383f0
> <impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)>, this=0xa20de80) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:531
> #19 operator() (this=0xa20de78) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222
> #20 boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run() (this=0xa20dcc0) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:116
> #21 0x0000000001d97f82 in thread_proxy ()
> #22 0x00007f1b244dcea5 in start_thread () from /lib64/libpthread.so.0
> #23 0x00007f1b20f4f8dd in clone () from /lib64/libc.so.6
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
