[
https://issues.apache.org/jira/browse/IMPALA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358990#comment-17358990
]
ASF subversion and git services commented on IMPALA-10413:
----------------------------------------------------------
Commit b18e0705e636e7a2c413b6171bb2571fc55a5331 in impala's branch
refs/heads/branch-4.0.0 from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b18e070 ]
IMPALA-10704: Fix retried query id not being unregistered when retry fails
When query retry fails in RetryQueryFromThread(), the retried query id
may not be unregistered if the failure happens before we store the
retry_request_state. In this case, QueryDriver::Unregister() has no way
to get the retried query id so it's not deleted. Note that the retried
query id is registered in RetryQueryFromThread() so should be deleted
later. This finally results in a leak in the query driver map, where
queries in it are shown as in-flight queries.
test_retry_query_result_cacheing_failed and
test_retry_query_set_query_in_flight_failed (added in IMPALA-10413)
asserts one in-flight query at the end. This is satisfied by the leak.
Instead, we should verify no running queries at the end.
This patch adds a new field in QueryDriver to remember the registered
retry query id as a backup way for getting it when query retry fails
before we store the ClientRequestState of the retried query (so
retried_client_request_state_ is null).
Tests:
- Run test_retry_query_result_cacheing_failed and
test_retry_query_set_query_in_flight_failed 100 times.
Change-Id: I074526799d68041a425b2379e74f8d8b45ce892a
Reviewed-on: http://gerrit.cloudera.org:8080/17465
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Impala crashing when retrying failed query
> ------------------------------------------
>
> Key: IMPALA-10413
> URL: https://issues.apache.org/jira/browse/IMPALA-10413
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.0
> Reporter: Xianqing He
> Assignee: Xianqing He
> Priority: Minor
> Fix For: Impala 4.0
>
>
> When retrying failed query, it may crash if cancel the query
> The core stack below
> {code:java}
> #0 0x00007f1b20e87387 in raise () from /lib64/libc.so.6
> #1 0x00007f1b20e88a78 in abort () from /lib64/libc.so.6
> #2 0x00007f1b23b754b9 in os::abort(bool) () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #3 0x00007f1b23d93db6 in VMError::report_and_die() () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #4 0x00007f1b23b7f505 in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #5 0x00007f1b23b72678 in signalHandler(int, siginfo_t*, void*) () from
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre/lib/amd64/server/libjvm.so
> #6 <signal handler called>
> #7 push_back (__x=..., this=0x28) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/include/c++/7.5.0/bits/stl_vector.h:941
> #8 AddDetail (d=..., this=0x0) at
> /data/Impala-ASF/be/src/util/error-util.h:114
> #9 impala::Status::AddDetail (this=this@entry=0x7f1a4b971740, msg=...) at
> /data/Impala-ASF/be/src/common/status.cc:236
> #10 0x000000000190a4fc in impala::QueryDriver::HandleRetryFailure
> (this=this@entry=0xb1b2880, status=status@entry=0x7f1a4b971740,
> error_msg=error_msg@entry=0x7f1a4b971860,
> request_state=request_state@entry=0x9ca7e00, retry_query_id=...)
> at /data/Impala-ASF/be/src/runtime/query-driver.cc:351
> #11 0x000000000190c605 in impala::QueryDriver::RetryQueryFromThread
> (this=0xb1b2880, error=..., query_driver=...) at
> /data/Impala-ASF/be/src/runtime/query-driver.cc:293
> #12 0x0000000001912459 in operator() (a2=..., a1=..., p=<optimized out>,
> this=<optimized out>) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:280
> #13 operator()<boost::_mfi::mf2<void, impala::QueryDriver, const
> impala::Status&, std::shared_ptr<impala::QueryDriver> >, boost::_bi::list0>
> (a=<synthetic pointer>, f=..., this=<optimized out>)
> at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:398
> #14 operator() (this=<optimized out>) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222
> #15
> boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
> boost::_mfi::mf2<void, impala::QueryDriver, impala::Status const&,
> std::shared_ptr<impala::QueryDriver> >,
> boost::_bi::list3<boost::_bi::value<impala::QueryDriver*>,
> boost::_bi::value<impala::Status>,
> boost::_bi::value<std::shared_ptr<impala::QueryDriver> > > >, void>::invoke
> (function_obj_ptr=...)
> at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:159
> #16 0x00000000015386f2 in operator() (this=0x7f1a4b971c00) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:770
> #17 impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*) (name=..., category=...,
> functor=..., parent_thread_info=<optimized out>,
> thread_started=0x7f1ad5750ec0) at /data/Impala-ASF/be/src/util/thread.cc:360
> #18 0x0000000001539b6b in operator()<void (*)(const
> std::__cxx11::basic_string<char>&, const std::__cxx11::basic_string<char>&,
> boost::function<void()>, const impala::ThreadDebugInfo*, impala::Promise<long
> int>*), boost::_bi::list0> (
> a=<synthetic pointer>,
> f=@0xa20de78: 0x15383f0
> <impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)>, this=0xa20de80) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:531
> #19 operator() (this=0xa20de78) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222
> #20 boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run() (this=0xa20dcc0) at
> /data/Impala-ASF/toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:116
> #21 0x0000000001d97f82 in thread_proxy ()
> #22 0x00007f1b244dcea5 in start_thread () from /lib64/libpthread.so.0
> #23 0x00007f1b20f4f8dd in clone () from /lib64/libc.so.6
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
