[
https://issues.apache.org/jira/browse/IMPALA-11029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17446224#comment-17446224
]
Wenzhe Zhou commented on IMPALA-11029:
--------------------------------------
In AnalyticEvalNode.constructEqExprs(), bufferedTupleDesc is created by calling
DescriptorTable.copyTupleDescriptor().
In DescriptorTable.copyTupleDescriptor(), a TupleDescriptor object is created
with slots copied from source TupleDescriptor, but its "path" member variable
is set as null. This make the table associated with the bufferedTupleDesc as
non Kudu table, and SlotDescriptor.isKuduStringSlot() always returns false for
its copied slots. TupleDescriptor.getSlotSize() add 4 padding bytes for Kudu
string slot. When the function is called to calculate slot size for string type
slots for bufferedTupleDesc, the slot size will be 4 bytes less than the slot
size calculated for the source TupleDescriptor if the table associated with
source TupleDescriptor is Kudu table. This cause
"Preconditions.checkState(d.getByteSize() == src.getByteSize())" in
DescriptorTable.copyTupleDescriptor() fail to throw exception.
We should copy "path" from source TupleDescriptor for bufferedTupleDesc before
coping slots from source TupleDescriptor.
> DescriptorTable.copyTupleDescriptor throw exception for Kudu table
> ------------------------------------------------------------------
>
> Key: IMPALA-11029
> URL: https://issues.apache.org/jira/browse/IMPALA-11029
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Fix For: Impala 4.1.0
>
>
> This is a customer reported bug. In the case, planner throw exception with
> following stack trace:
> I1115 07:52:39.779211 14750 jni-util.cc:286]
> 4a467bd0ffba8c4b:0ce1e98a00000000] java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at
> org.apache.impala.analysis.DescriptorTable.copyTupleDescriptor(DescriptorTable.java:85)
> at
> org.apache.impala.planner.AnalyticEvalNode.constructEqExprs(AnalyticEvalNode.java:184)
> at
> org.apache.impala.planner.AnalyticEvalNode.init(AnalyticEvalNode.java:158)
> at
> org.apache.impala.planner.AnalyticPlanner.createSortGroupPlan(AnalyticPlanner.java:495)
> at
> org.apache.impala.planner.AnalyticPlanner.createSingleNodePlan(AnalyticPlanner.java:139)
> at
> org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:295)
> at
> org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:170)
> at
> org.apache.impala.planner.Planner.createPlanFragments(Planner.java:120)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:249)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1562)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1939)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1779)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1644)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1614)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> I1115 07:52:39.779265 14750 status.cc:129] 4a467bd0ffba8c4b:0ce1e98a00000000]
> IllegalStateException: null
> @ 0x1f94ad9 impala::Status::Status()
> @ 0x297e8ed impala::JniUtil::GetJniExceptionMsg()
> @ 0x26b4180 impala::JniCall::Call<>()
> @ 0x26b0c07 impala::JniUtil::CallJniMethod<>()
> @ 0x26aec22 impala::Frontend::GetExecRequest()
> @ 0x30dda65 impala::QueryDriver::RunFrontendPlanner()
> @ 0x2713244 impala::ImpalaServer::ExecuteInternal()
> @ 0x2712791 impala::ImpalaServer::Execute()
> @ 0x285f9f4 impala::ImpalaServer::ExecuteStatementCommon()
> @ 0x2860895 impala::ImpalaServer::ExecuteStatement()
> @ 0x27727c7
> apache::hive::service::cli::thrift::TCLIServiceProcessorT<>::process_ExecuteStatement()
> @ 0x27f4161
> apache::hive::service::cli::thrift::TCLIServiceProcessorT<>::dispatchCall()
> @ 0x27f3d58
> impala::ImpalaHiveServer2ServiceProcessorT<>::dispatchCall()
> @ 0x1e6ee6f apache::thrift::TDispatchProcessor::process()
> @ 0x2348784
> apache::thrift::server::TAcceptQueueServer::Task::run()
> @ 0x233bfea impala::ThriftThread::RunRunnable()
> @ 0x233d606 boost::_mfi::mf2<>::operator()()
> @ 0x233d49a boost::_bi::list3<>::operator()<>()
> @ 0x233d1f0 boost::_bi::bind_t<>::operator()()
> @ 0x233d0b9
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x22ab5cd boost::function0<>::operator()()
> @ 0x2a76d5e impala::Thread::SuperviseThread()
> @ 0x2a7f6ae boost::_bi::list5<>::operator()<>()
> @ 0x2a7f5d2 boost::_bi::bind_t<>::operator()()
> @ 0x2a7f593 boost::detail::thread_data<>::run()
> @ 0x43763b0 thread_proxy
> @ 0x7fd8ae06bea4 start_thread
> @ 0x7fd8aa9df9fc __clone
>
> The bug could be reproduced in Impala upstream with following queries:
> create database mydb;
> CREATE TABLE mydb.my_tab1 (
> id1 INT NOT NULL,
> agrmt INT NOT NULL,
> big_id BIGINT NOT NULL,
> outdated_flag STRING NOT NULL,
> mod_ts TIMESTAMP NOT NULL,
> PRIMARY KEY (id1, agrmt)
> )
> PARTITION BY HASH (id1) PARTITIONS 2
> STORED AS KUDU;
> CREATE TABLE mydb.my_tab2 (
> cl_id INT NOT NULL,
> cl_agrmt INT NOT NULL,
> outdat STRING NULL,
> mod_dat TIMESTAMP NULL,
> PRIMARY KEY (cl_id, cl_agrmt)
> )
> PARTITION BY HASH (cl_id) PARTITIONS 2
> STORED AS KUDU;
> insert into mydb.my_tab1 (
> id1,
> agrmt,
> big_id,
> outdated_flag,
> mod_ts)
> select
> cl_id,
> cast(row_number() over(order by null) as int),
> cl_agrmt,
> 'Y',
> case when outdat='Y' and mod_dat is not null then mod_dat else now() end
> from mydb.my_tab2 i
> left join mydb.my_tab1 u
> on u.big_id=i.cl_agrmt
> left join (
> select id1, big_id
> from mydb.my_tab1
> group by id1, big_id) uu
> on uu.big_id=i.cl_agrmt
> where u.big_id is null;
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
