[
https://issues.apache.org/jira/browse/IMPALA-11029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448397#comment-17448397
]
ASF subversion and git services commented on IMPALA-11029:
----------------------------------------------------------
Commit b13a17b9cff1f93c44a3a9143e1ba0769ee125f7 in impala's branch
refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b13a17b ]
IMPALA-11029: DescriptorTable.copyTupleDescriptor throw exception for Kudu table
In DescriptorTable.copyTupleDescriptor(), a TupleDescriptor object is
created with slots copied from source TupleDescriptor, but its "path"
member variable is set as null. This make the table associated with the
copied TupleDescriptor object detected as non Kudu table, and
SlotDescriptor.isKuduStringSlot() always returns false for its copied
slots.
TupleDescriptor.getSlotSize() add 4 padding bytes for Kudu string slot.
When this function is called to calculate slot size for string type
slots for copied TupleDescriptor object, the slot size will be 4 bytes
less than the slot size calculated for the source TupleDescriptor if the
table associated with source TupleDescriptor is Kudu table. This cause
Preconditions.checkState(d.getByteSize() == src.getByteSize()) failed to
throw exception.
To fix it, we should copy "path" from source TupleDescriptor for the
copied TupleDescriptor object before copying slots.
Testing:
- Added a new test case to verify the bug fixing.
- Passed exhaustive tests.
Change-Id: Ib88e005910134f2b7575c06fa02ce11890418a41
Reviewed-on: http://gerrit.cloudera.org:8080/18037
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> DescriptorTable.copyTupleDescriptor throw exception for Kudu table
> ------------------------------------------------------------------
>
> Key: IMPALA-11029
> URL: https://issues.apache.org/jira/browse/IMPALA-11029
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Fix For: Impala 4.1.0
>
>
> This is a customer reported bug. In the case, planner throw exception with
> following stack trace:
> I1115 07:52:39.779211 14750 jni-util.cc:286]
> 4a467bd0ffba8c4b:0ce1e98a00000000] java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at
> org.apache.impala.analysis.DescriptorTable.copyTupleDescriptor(DescriptorTable.java:85)
> at
> org.apache.impala.planner.AnalyticEvalNode.constructEqExprs(AnalyticEvalNode.java:184)
> at
> org.apache.impala.planner.AnalyticEvalNode.init(AnalyticEvalNode.java:158)
> at
> org.apache.impala.planner.AnalyticPlanner.createSortGroupPlan(AnalyticPlanner.java:495)
> at
> org.apache.impala.planner.AnalyticPlanner.createSingleNodePlan(AnalyticPlanner.java:139)
> at
> org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:295)
> at
> org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:170)
> at
> org.apache.impala.planner.Planner.createPlanFragments(Planner.java:120)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:249)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1562)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1939)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1779)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1644)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1614)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> I1115 07:52:39.779265 14750 status.cc:129] 4a467bd0ffba8c4b:0ce1e98a00000000]
> IllegalStateException: null
> @ 0x1f94ad9 impala::Status::Status()
> @ 0x297e8ed impala::JniUtil::GetJniExceptionMsg()
> @ 0x26b4180 impala::JniCall::Call<>()
> @ 0x26b0c07 impala::JniUtil::CallJniMethod<>()
> @ 0x26aec22 impala::Frontend::GetExecRequest()
> @ 0x30dda65 impala::QueryDriver::RunFrontendPlanner()
> @ 0x2713244 impala::ImpalaServer::ExecuteInternal()
> @ 0x2712791 impala::ImpalaServer::Execute()
> @ 0x285f9f4 impala::ImpalaServer::ExecuteStatementCommon()
> @ 0x2860895 impala::ImpalaServer::ExecuteStatement()
> @ 0x27727c7
> apache::hive::service::cli::thrift::TCLIServiceProcessorT<>::process_ExecuteStatement()
> @ 0x27f4161
> apache::hive::service::cli::thrift::TCLIServiceProcessorT<>::dispatchCall()
> @ 0x27f3d58
> impala::ImpalaHiveServer2ServiceProcessorT<>::dispatchCall()
> @ 0x1e6ee6f apache::thrift::TDispatchProcessor::process()
> @ 0x2348784
> apache::thrift::server::TAcceptQueueServer::Task::run()
> @ 0x233bfea impala::ThriftThread::RunRunnable()
> @ 0x233d606 boost::_mfi::mf2<>::operator()()
> @ 0x233d49a boost::_bi::list3<>::operator()<>()
> @ 0x233d1f0 boost::_bi::bind_t<>::operator()()
> @ 0x233d0b9
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x22ab5cd boost::function0<>::operator()()
> @ 0x2a76d5e impala::Thread::SuperviseThread()
> @ 0x2a7f6ae boost::_bi::list5<>::operator()<>()
> @ 0x2a7f5d2 boost::_bi::bind_t<>::operator()()
> @ 0x2a7f593 boost::detail::thread_data<>::run()
> @ 0x43763b0 thread_proxy
> @ 0x7fd8ae06bea4 start_thread
> @ 0x7fd8aa9df9fc __clone
>
> The bug could be reproduced in Impala upstream with following queries:
> create database mydb;
> CREATE TABLE mydb.my_tab1 (
> id1 INT NOT NULL,
> agrmt INT NOT NULL,
> big_id BIGINT NOT NULL,
> outdated_flag STRING NOT NULL,
> mod_ts TIMESTAMP NOT NULL,
> PRIMARY KEY (id1, agrmt)
> )
> PARTITION BY HASH (id1) PARTITIONS 2
> STORED AS KUDU;
> CREATE TABLE mydb.my_tab2 (
> cl_id INT NOT NULL,
> cl_agrmt INT NOT NULL,
> outdat STRING NULL,
> mod_dat TIMESTAMP NULL,
> PRIMARY KEY (cl_id, cl_agrmt)
> )
> PARTITION BY HASH (cl_id) PARTITIONS 2
> STORED AS KUDU;
> insert into mydb.my_tab1 (
> id1,
> agrmt,
> big_id,
> outdated_flag,
> mod_ts)
> select
> cl_id,
> cast(row_number() over(order by null) as int),
> cl_agrmt,
> 'Y',
> case when outdat='Y' and mod_dat is not null then mod_dat else now() end
> from mydb.my_tab2 i
> left join mydb.my_tab1 u
> on u.big_id=i.cl_agrmt
> left join (
> select id1, big_id
> from mydb.my_tab1
> group by id1, big_id) uu
> on uu.big_id=i.cl_agrmt
> where u.big_id is null;
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
