[
https://issues.apache.org/jira/browse/IMPALA-13868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936927#comment-17936927
]
ASF subversion and git services commented on IMPALA-13868:
----------------------------------------------------------
Commit 627bbdd2a436ecb1a62825976f6afc2903395af1 in impala's branch
refs/heads/master from Yida Wu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=627bbdd2a ]
IMPALA-13868: Fix ASAN build failed in test_ai_generate_text_exprs
In ParseImpalaOptions(), when the input options cannot be parsed,
the error message is constructed incorrectly by directly using the
StringVal pointer without considering its length. This may result
in reading beyond the allocated memory, leading to unexpected
behavior.
This issue was introduced by IMPALA-13565 and triggered ASAN
errors due to new test cases added in IMPALA-13812
(query_test/test_exprs.py). While IMPALA-13565 included unit
tests (AiFunctionsTest in expr-test.cc) for this code, the
newly added ee tests test_ai_generate_text_exprs() in
IMPALA-13812 run in parallel, making it much easier to trigger
this issue.
The patch fixes the issue by ensuring the error message is
constructed using both the pointer and its length.
Tests:
Passed the ASAN build tests.
Change-Id: I9f4656e256bb9b31acc2653c3b910788ddf03f2b
Reviewed-on: http://gerrit.cloudera.org:8080/22632
Reviewed-by: Riza Suminto <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Asan build failed in test_ai_generate_text_exprs
> ------------------------------------------------
>
> Key: IMPALA-13868
> URL: https://issues.apache.org/jira/browse/IMPALA-13868
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.5.0
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
> Labels: broken-build
> Fix For: Impala 5.0.0
>
>
> Running query_test/test_exprs.py in the ASAN build consistently fails at one
> AiGenerateText test case, which testing the handling of 'Error parsing Impala
> options'. The issue is likely caused by improper usage of stringstream in
> https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/be/src/exprs/ai-functions.cc#L129.
> {code:java}
> =================================================================
> ==26567==ERROR: AddressSanitizer: use-after-poison on address 0x6210008ca21c
> at pc 0x00000217c628 bp 0x7f38044928f0 sp 0x7f38044920a0
> READ of size 3841 at 0x6210008ca21c thread T508
> ==26567==AddressSanitizer: while reporting a bug found another one. Ignoring.
> E0316 19:52:50.080662 29280 LiteralExpr.java:216]
> 5c49cc395b09b4b5:98a5006b00000000] Failed to evaluate expr
> 'aes_encrypt('ABC', '12345678901234567890123456789012', '',
> '1234567890123456')': Invalid AES 'mode':
> #0 0x217c627 (/impala/Impala/be/build/debug/service/impalad+0x217c627)
> #1 0x7f3d04199a69 in std::char_traits<char>::length(char const*)
> /mnt/source/gcc/build-10.4.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/char_traits.h:399:25
> #2 0x7f3d04199a69 in std::basic_ostream<char, std::char_traits<char> >&
> std::operator<< <std::char_traits<char> >(std::basic_ostream<char,
> std::char_traits<char> >&, char const*)
> /mnt/source/gcc/build-10.4.0/x86_64-pc-linux-gnu/libstdc++-v3/include/ostream:612:44
> #3 0x4944cc5 in impala::ParseImpalaOptions(impala_udf::StringVal const&,
> rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>&, impala::AiFunctions::AiFunctionsOptions&)
> /impala/Impala/be/src/exprs/ai-functions.cc:129:44
> #4 0x494d551 in impala_udf::StringVal
> impala::AiFunctions::AiGenerateTextInternal<false,
> (impala::AiFunctions::AI_PLATFORM)1>(impala_udf::FunctionContext*,
> std::basic_string_view<char, std::char_traits<char> > const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, bool)
> /impala/Impala/be/src/exprs/ai-functions.cc:190:7
> #5 0x4b015bf in impala_udf::StringVal
> impala::AiFunctions::AiGenerateTextHelper<false>(impala_udf::FunctionContext*,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&)
> /impala/Impala/be/src/exprs/ai-functions-ir.cc:226:14
> #6 0x49e109a in impala_udf::StringVal
> impala::ScalarFnCall::InterpretEval<impala_udf::StringVal>(impala::ScalarExprEvaluator*,
> impala::TupleRow const*) const
> /impala/Impala/be/src/exprs/scalar-fn-call.cc:490:7
> #7 0x49b1d14 in
> impala::ScalarFnCall::GetStringValInterpreted(impala::ScalarExprEvaluator*,
> impala::TupleRow const*) const
> /impala/Impala/be/src/exprs/scalar-fn-call.cc:556:1
> #8 0x4979c7f in
> impala::ScalarExpr::GetStringVal(impala::ScalarExprEvaluator*,
> impala::TupleRow const*) const
> /impala/Impala/be/src/exprs/scalar-expr.inline.h:63:1
> #9 0x49769cb in impala::ScalarExprEvaluator::GetValue(impala::ScalarExpr
> const&, impala::TupleRow const*)
> /impala/Impala/be/src/exprs/scalar-expr-evaluator.cc:318:38
> #10 0x2e4a731 in void impala::Tuple::MaterializeExprs<false,
> false>(impala::TupleRow*, impala::TupleDescriptor const&,
> impala::ScalarExprEvaluator* const*, impala::MemPool*,
> std::vector<impala::StringValue*, std::allocator<impala::StringValue*> >*,
> std::vector<std::pair<impala::CollectionValue*, long>,
> std::allocator<std::pair<impala::CollectionValue*, long> > >*, int*, int*,
> int*) /impala/Impala/be/src/runtime/tuple.cc:279:27
> #11 0x3c0227f in void impala::Tuple::MaterializeExprs<false,
> false>(impala::TupleRow*, impala::TupleDescriptor const&,
> std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::MemPool*,
> std::vector<impala::StringValue*, std::allocator<impala::StringValue*> >*,
> std::vector<std::pair<impala::CollectionValue*, long>,
> std::allocator<std::pair<impala::CollectionValue*, long> > >*, int*)
> /impala/Impala/be/src/runtime/tuple.h:229:5
> #12 0x3def605 in
> impala::UnionNode::MaterializeExprs(std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::TupleRow*,
> unsigned char*, impala::RowBatch*)
> /impala/Impala/be/src/exec/union-node-ir.cc:29:14
> #13 0x3cf74f1 in impala::UnionNode::GetNextConst(impala::RuntimeState*,
> impala::RowBatch*) /impala/Impala/be/src/exec/union-node.cc:295:5
> #14 0x3cf7e05 in impala::UnionNode::GetNext(impala::RuntimeState*,
> impala::RowBatch*, bool*) /impala/Impala/be/src/exec/union-node.cc:329:5
> #15 0x2ee04e0 in impala::FragmentInstanceState::ExecInternal()
> /impala/Impala/be/src/runtime/fragment-instance-state.cc:446:7
> #16 0x2edc6a6 in impala::FragmentInstanceState::Exec()
> /impala/Impala/be/src/runtime/fragment-instance-state.cc:104:14
> #17 0x2dace0e in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /impala/Impala/be/src/runtime/query-state.cc:1013:24
> #18 0x2cf07c6 in boost::function0<void>::operator()() const
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
> #19 0x39bffef in
> impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /impala/Impala/be/src/util/thread.cc:360:3
> #20 0x39ccadb in boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()> const&, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> #21 0x51dd596 in thread_proxy
> (/impala/Impala/be/build/debug/service/impalad+0x51dd596)
> #22 0x7f3d03c176da in start_thread
> (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
> #23 0x7f3d0097b61e in clone
> /build/glibc-CVJwZb/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> 0x6210008cb100 is located 0 bytes to the right of 4096-byte region
> [0x6210008ca100,0x6210008cb100)
> allocated by thread T508 here:
> #0 0x2217daf in __interceptor_malloc
> (/impala/Impala/be/build/debug/service/impalad+0x2217daf)
> #1 0x2d81723 in impala::MemPool::FindChunk(long, bool)
> /impala/Impala/be/src/runtime/mem-pool.cc:132:45
> #2 0x41458e6 in unsigned char* impala::MemPool::Allocate<true>(long, int)
> /impala/Impala/be/src/runtime/mem-pool.h:295:9
> #3 0x41458e6 in impala::MemPool::TryAllocateAligned(long, int)
> /impala/Impala/be/src/runtime/mem-pool.h:130
> #4 0x495edb7 in impala::AllocateAnyVal(impala::RuntimeState*,
> impala::MemPool*, impala::ColumnType const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, impala_udf::AnyVal**)
> /impala/Impala/be/src/exprs/anyval-util.cc:34:39
> #5 0x49adb80 in
> impala::ScalarFnCall::OpenEvaluator(impala_udf::FunctionContext::FunctionStateScope,
> impala::RuntimeState*, impala::ScalarExprEvaluator*) const
> /impala/Impala/be/src/exprs/scalar-fn-call.cc:152:5
> #6 0x4974706 in impala::ScalarExprEvaluator::Open(impala::RuntimeState*)
> /impala/Impala/be/src/exprs/scalar-expr-evaluator.cc:155:16
> #7 0x4974926 in
> impala::ScalarExprEvaluator::Open(std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::RuntimeState*)
> /impala/Impala/be/src/exprs/scalar-expr-evaluator.cc:160:42
> #8 0x3cf40bb in impala::UnionNode::Open(impala::RuntimeState*)
> /impala/Impala/be/src/exec/union-node.cc:176:5
> #9 0x2edfac8 in impala::FragmentInstanceState::Open()
> /impala/Impala/be/src/runtime/fragment-instance-state.cc:426:5
> #10 0x2edc4f5 in impala::FragmentInstanceState::Exec()
> /impala/Impala/be/src/runtime/fragment-instance-state.cc:95:12
> #11 0x2dace0e in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /impala/Impala/be/src/runtime/query-state.cc:1013:24
> #12 0x2cf07c6 in boost::function0<void>::operator()() const
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
> #13 0x39bffef in
> impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /impala/Impala/be/src/util/thread.cc:360:3
> #14 0x39ccadb in boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()> const&, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> #15 0x51dd596 in thread_proxy
> (/impala/Impala/be/build/debug/service/impalad+0x51dd596)
> Thread T508 created by T507 here:
> #0 0x215f7e2 in __interceptor_pthread_create
> (/impala/Impala/be/build/debug/service/impalad+0x215f7e2)
> #1 0x51dc89d in boost::thread::start_thread_noexcept()
> (/impala/Impala/be/build/debug/service/impalad+0x51dc89d)
> #2 0x39c5086 in boost::thread::thread<void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()> const&, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> >, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long,
> (impala::PromiseMode)0>*>(void (*)(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> >, std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> >, boost::function<void ()>, impala::ThreadDebugInfo*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:424:13
> #3 0x39bf44d in
> impala::Thread::StartThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread,
> std::default_delete<impala::Thread> >*, bool)
> /impala/Impala/be/src/util/thread.cc:317:13
> #4 0x2dac1cb in impala::Status
> impala::Thread::Create<impala::QueryState::StartFInstances()::$_15>(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, impala::QueryState::StartFInstances()::$_15 const&,
> std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool)
> /impala/Impala/be/src/util/thread.h:74:12
> #5 0x2dab307 in impala::QueryState::StartFInstances()
> /impala/Impala/be/src/runtime/query-state.cc:917:11
> #6 0x2d8c980 in
> impala::QueryExecMgr::ExecuteQueryHelper(impala::QueryState*)
> /impala/Impala/be/src/runtime/query-exec-mgr.cc:170:7
> #7 0x2d987eb in boost::_bi::bind_t<void, boost::_mfi::mf1<void,
> impala::QueryExecMgr, impala::QueryState*>,
> boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>,
> boost::_bi::value<impala::QueryState*> > >::operator()()
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> #8 0x2cf07c6 in boost::function0<void>::operator()() const
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
> #9 0x39bffef in
> impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /impala/Impala/be/src/util/thread.cc:360:3
> #10 0x39ccadb in boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()> const&, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /impala/Impala/toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> #11 0x51dd596 in thread_proxy
> (/impala/Impala/be/build/debug/service/impalad+0x51dd596)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
