[
https://issues.apache.org/jira/browse/IMPALA-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570983#comment-16570983
]
ASF subversion and git services commented on IMPALA-7397:
---------------------------------------------------------
Commit 70bf9ea29636a6fbecfd7a003cc746d3a0046edb in impala's branch
refs/heads/master from [~twmarshall]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=70bf9ea ]
IMPALA-7397: fix DCHECK in impala::AggregationNode::Close
As part of IMPALA-110, all of the logic of performing aggregations was
refactored out of the aggregation ExecNode and into Aggregators. Each
Aggregator manages its own memory, so a DCHECK was added in
AggregationNode::Close to ensure that no allocations were
made from the regular ExecNode mem pools.
This DCHECK is violated if the node was assigned conjuncts that
allocate mem in Prepare - even though the conjuncts are evaluated in
the Aggregator, we still initialize them in ExecNode::Prepare.
This patch solves the problem by creating a copy of the TPlanNode
without the conjuncts to pass to ExecNode. In the future, when
TAggregator is introduced, we can get rid of this by directly
assigning conjuncts to Aggregators.
Note that this doesn't affect StreamingAggregationNode, which never
has conjuncts assigned to it, but this patch fixes an incorrect DCHECK
that enforces this.
Testing:
- Added a regression test for this case.
Change-Id: I65ae00ac23a62ab9f4a7ff06ac9ad5cece80c39d
Reviewed-on: http://gerrit.cloudera.org:8080/11132
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> dcheck in impala::AggregationNode::Close
> ----------------------------------------
>
> Key: IMPALA-7397
> URL: https://issues.apache.org/jira/browse/IMPALA-7397
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Michael Brown
> Assignee: Thomas Tauber-Marshall
> Priority: Blocker
> Labels: crash, query_generator
>
> The dcheck is {{Check failed: expr_results_pool() == nullptr ||
> expr_results_pool()->total_allocated_bytes() == 0}}. The random query
> generator found this numerous times. I was able to reproduce it, and the
> dcheck fails on all 3 impalads. Stack is the same on all 3.
>
> {noformat}
> #6 0x000000000437933e in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x0000000002e8637b in
> impala::AggregationNode::Close(impala::RuntimeState*) (this=0x12223c00,
> state=0xdb275a0)
> at /home/mikeb/Impala/be/src/exec/aggregation-node.cc:120
> #8 0x0000000001df2f20 in impala::FragmentInstanceState::Close()
> (this=0x12222c00) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:320
> #9 0x0000000001defd9f in impala::FragmentInstanceState::Exec()
> (this=0x12222c00) at
> /home/mikeb/Impala/be/src/runtime/fragment-instance-state.cc:97
> #10 0x0000000001dff35d in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> (this=0xdd31800, fis=0x12222c00)
> at /home/mikeb/Impala/be/src/runtime/query-state.cc:401
> #11 0x0000000001dfdafc in impala::QueryState::<lambda()>::operator()(void)
> const (__closure=0x7f0aae0feca8) at
> /home/mikeb/Impala/be/src/runtime/query-state.cc:341
> #12 0x0000000001e0007b in
> boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::<lambda()>,
> void>::invoke(boost::detail::function::function_buffer &)
> (function_obj_ptr=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #13 0x0000000001c22a6a in boost::function0<void>::operator()() const
> (this=0x7f0aae0feca0)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #14 0x00000000020303bb in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*) (name="exec-finstance
> (finst:c44493cc1e941104:fb3d183b00000004)", category="fragment-execution",
> functor=..., parent_thread_info=0x7f0aaf100950,
> thread_started=0x7f0aaf0ff8e0) at /home/mikeb/Impala/be/src/util/thread.cc:356
> #15 0x0000000002038493 in boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> (this=0x12222fc0, f=@0x12222fb8: 0x2030054
> <impala::Thread::SuperviseThread(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)>, a=...) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #16 0x00000000020383b7 in boost::_bi::bind_t<void, void (*)(std::string
> const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo
> const*, impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()() (this=0x12222fb8)
> at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #17 0x000000000203837a in boost::detail::thread_data<boost::_bi::bind_t<void,
> void (*)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run() (this=0x12222e00) at
> /home/mikeb/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #18 0x00000000032a19ea in thread_proxy ()
> #19 0x00007f0b45aa76ba in start_thread (arg=0x7f0aae0ff700) at
> pthread_create.c:333
> #20 0x00007f0b457dd41d in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> {noformat}
> An example pared down query:
> {noformat}
> SELECT
> c_custkey,
> MIN(c_nationkey)
> FROM tpch_kudu.customer
> GROUP BY
> c_custkey,
> c_nationkey
> HAVING
> (EXTRACT(HOUR FROM CAST('0' AS TIMESTAMP) + INTERVAL c_nationkey YEAR)) !=
> c_custkey;
> {noformat}
> I could not figure out a more simplified HAVING expression that triggers the
> bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
