[
https://issues.apache.org/jira/browse/IMPALA-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169808#comment-17169808
]
Quanlong Huang commented on IMPALA-10040:
-----------------------------------------
The cause is that we are not checking RowBatch::AtCapacity() in the loop of
FOREACH_ROW_LIMIT macro. RowBatch::AtCapacity() could be true when its
mem_usage is high enough.
Most of the current codes that generating a new row batch are in the following
pattern:
* Calculate num_rows_to_process based on num of rows avaliable in the dst batch
* Iterate dst batch using FOREACH_ROW_LIMIT macro.
* Assuming num_rows_to_process of rows are added to dst batch.
I think we should check RowBatch::AtCapacity() in FOREACH_ROW_LIMIT and
FOREACH_ROW macros. And deal with the case when the loop stop due to mem_usage
of a row batch is high enough outside the loop.
> Crash on UnionNode when codegen is disabled
> -------------------------------------------
>
> Key: IMPALA-10040
> URL: https://issues.apache.org/jira/browse/IMPALA-10040
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Priority: Critical
>
> Saw a crash when ran a UNION query with codegen disabled:
> {code}
> F0803 15:37:44.551749 24805 union-node-ir.cc:26]
> fd41196430b5c449:0a195a2500000006] Check failed: !dst_batch->AtCapacity()
> *** Check failure stack trace: ***
> @ 0x514aa8c google::LogMessage::Fail()
> @ 0x514c37c google::LogMessage::SendToLog()
> @ 0x514a3ea google::LogMessage::Flush()
> @ 0x514dfe8 google::LogMessageFatal::~LogMessageFatal()
> @ 0x286c323 impala::UnionNode::MaterializeExprs()
> @ 0x286c983 impala::UnionNode::MaterializeBatch()
> @ 0x286798a impala::UnionNode::GetNextMaterialized()
> @ 0x2868ac4 impala::UnionNode::GetNext()
> @ 0x225f77c impala::FragmentInstanceState::ExecInternal()
> @ 0x225be20 impala::FragmentInstanceState::Exec()
> @ 0x2285c35 impala::QueryState::ExecFInstance()
> @ 0x2284037
> _ZZN6impala10QueryState15StartFInstancesEvENKUlvE_clEv
> @ 0x22877d6
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala10QueryState15StartFInstancesEvEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x2053061 boost::function0<>::operator()()
> @ 0x2676bff impala::Thread::SuperviseThread()
> @ 0x267eb9c boost::_bi::list5<>::operator()<>()
> @ 0x267eac0 boost::_bi::bind_t<>::operator()()
> @ 0x267ea81 boost::detail::thread_data<>::run()
> @ 0x3e514e1 thread_proxy
> @ 0x7f6575c326b9 start_thread
> @ 0x7f65727fe4dc clone
> {code}
> The query is
> {code}
> I0803 15:37:44.273838 24616 Frontend.java:1508]
> fd41196430b5c449:0a195a2500000000] Analyzing query: create table my_bigstrs
> stored as parquet as
> select *, repeat(string_col, 100000) as bigstr
> from functional.alltypes
> order by id
> limit 10
> union all
> select *, repeat(string_col, 10000000) as bigstr
> from functional.alltypes
> order by id
> limit 10 db: default
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
