[
https://issues.apache.org/jira/browse/IMPALA-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158585#comment-17158585
]
ASF subversion and git services commented on IMPALA-9941:
---------------------------------------------------------
Commit 6d1a707cf50c2297972a69a7c0469afe4a78d739 in impala's branch
refs/heads/master from Adam Tamas
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6d1a707 ]
IMPALA-9941: ExprTest.CastExprs fails when running with ASAN fix
Fixed the issue with the ASAN build where too short tokens in
timestamps fails because of indexing out of bounds because of a
missing check.
Updated some missed files connected to the drop of dateless timestamps
(IMPALA-9531) regarding tests, comments.
Testing:
- Ran CORE tests
- Ran ASAN with EE_TEST_SHARDS=6 (with the help of IMPALA-9887)
Change-Id: I75bd47b6627a2e338c46dc354f7e67d34c1abbcf
Reviewed-on: http://gerrit.cloudera.org:8080/16179
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> ExprTest.CastExprs fails when running with ASAN
> -----------------------------------------------
>
> Key: IMPALA-9941
> URL: https://issues.apache.org/jira/browse/IMPALA-9941
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Assignee: Adam Tamas
> Priority: Blocker
> Labels: broken-build
>
> I was running ASAN jobs to test the sharding fix. When running the backend
> tests under ASAN, ExprTest.CastExprs fails with the following message:
>
> {noformat}
> ==4725==ERROR: AddressSanitizer: use-after-poison on address 0x6210004b3d27
> at pc 0x000003accb3e bp 0x7fabbdb14310 sp 0x7fabbdb14308
> READ of size 1 at 0x6210004b3d27 thread T29762
> #0 0x3accb3d in
> impala::datetime_parse_util::SimpleDateFormatTokenizer::GetDefaultFormatContext(char
> const*, int, bool, bool)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/datetime-simple-date-format-parser.cc:345:24
> #1 0x3c7053d in impala::TimestampParser::ParseSimpleDateFormat(char
> const*, int, boost::gregorian::date*, boost::posix_time::time_duration*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/timestamp-parse-util.cc:109:7
> #2 0x3c73f9b in impala::TimestampValue::ParseSimpleDateFormat(char
> const*, int)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/timestamp-value.cc:53:18
> #3 0x307d292 in
> impala::CastFunctions::CastToTimestampVal(impala_udf::FunctionContext*,
> impala_udf::StringVal const&)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/cast-functions-ir.cc:321:10
> #4 0x3045142 in impala_udf::TimestampVal
> impala::ScalarFnCall::InterpretEval<impala_udf::TimestampVal>(impala::ScalarExprEvaluator*,
> impala::TupleRow const*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-fn-call.cc:457:7
> #5 0x3010e33 in
> impala::ScalarFnCall::GetTimestampValInterpreted(impala::ScalarExprEvaluator*,
> impala::TupleRow const*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-fn-call.cc:525:1
> #6 0x2fad98e in
> impala::ScalarExpr::GetTimestampVal(impala::ScalarExprEvaluator*,
> impala::TupleRow const*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-expr.inline.h:60:1
> #7 0x2faa522 in impala::ScalarExprEvaluator::GetValue(impala::ScalarExpr
> const&, impala::TupleRow const*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-expr-evaluator.cc:318:41
> #8 0x3c7c6bf in void impala::Tuple::MaterializeExprs<false,
> false>(impala::TupleRow*, impala::TupleDescriptor const&,
> impala::ScalarExprEvaluator* const*, impala::MemPool*, impala::StringValue**,
> int*, int*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/tuple.cc:234:27
> #9 0x2e26840 in void impala::Tuple::MaterializeExprs<false,
> false>(impala::TupleRow*, impala::TupleDescriptor const&,
> std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::MemPool*,
> std::vector<impala::StringValue*, std::allocator<impala::StringValue*> >*,
> int*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/tuple.h:191:5
> #10 0x2e339d6 in
> impala::UnionNode::MaterializeExprs(std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::TupleRow*,
> unsigned char*, impala::RowBatch*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/union-node-ir.cc:29:14
> #11 0x2e2d5c1 in impala::UnionNode::GetNextConst(impala::RuntimeState*,
> impala::RowBatch*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/union-node.cc:289:5
> #12 0x2e2ded5 in impala::UnionNode::GetNext(impala::RuntimeState*,
> impala::RowBatch*, bool*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/union-node.cc:323:5
> #13 0x3b50780 in impala::FragmentInstanceState::ExecInternal()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:416:7
> #14 0x3b4c8e2 in impala::FragmentInstanceState::Exec()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:106:14
> #15 0x3be69ce in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:815:24
> #16 0x3922446 in boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:770:14
> #17 0x4ab5a89 in
> impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
> #18 0x4ac10d8 in void
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void
> (*&)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:531:9
> #19 0x4ac0f2b in boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222:16
> #20 0x4e929d1 in thread_proxy
> (/data0/jenkins/workspace/impala-private-parameterized/repos/Impala/be/build/debug/service/unifiedbetests+0x4e929d1)
> #21 0x7fb4b8ca9e24 in start_thread (/lib64/libpthread.so.0+0x7e24)
> #22 0x7fb4b581e34c in __clone (/lib64/libc.so.6+0xf834c)
> 0x6210004b3d27 is located 39 bytes inside of 4096-byte region
> [0x6210004b3d00,0x6210004b4d00)
> allocated by thread T29762 here:
> #0 0x1df7dc0 in malloc
> /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:67
> #1 0x3bd1353 in impala::MemPool::FindChunk(long, bool)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/mem-pool.cc:132:45
> #2 0x22f37b6 in unsigned char* impala::MemPool::Allocate<true>(long, int)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/mem-pool.h:295:9
> #3 0x22f37b6 in impala::MemPool::TryAllocateAligned(long, int)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/mem-pool.h:130
> #4 0x2f8a087 in impala::AllocateAnyVal(impala::RuntimeState*,
> impala::MemPool*, impala::ColumnType const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, impala_udf::AnyVal**)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/anyval-util.cc:34:39
> #5 0x300c720 in
> impala::ScalarFnCall::OpenEvaluator(impala_udf::FunctionContext::FunctionStateScope,
> impala::RuntimeState*, impala::ScalarExprEvaluator*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-fn-call.cc:152:5
> #6 0x2fa8266 in impala::ScalarExprEvaluator::Open(impala::RuntimeState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-expr-evaluator.cc:141:16
> #7 0x2fa8486 in
> impala::ScalarExprEvaluator::Open(std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::RuntimeState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exprs/scalar-expr-evaluator.cc:146:42
> #8 0x2e2a18b in impala::UnionNode::Open(impala::RuntimeState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/union-node.cc:169:5
> #9 0x3b4fd68 in impala::FragmentInstanceState::Open()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:396:5
> #10 0x3b4c735 in impala::FragmentInstanceState::Exec()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:97:12
> #11 0x3be69ce in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:815:24
> #12 0x3922446 in boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:770:14
> #13 0x4ab5a89 in
> impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
> #14 0x4ac10d8 in void
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void
> (*&)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:531:9
> #15 0x4ac0f2b in boost::_bi::bind_t<void, void
> (*)(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, boost::function<void
> ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222:16
> #16 0x4e929d1 in thread_proxy
> (/data0/jenkins/workspace/impala-private-parameterized/repos/Impala/be/build/debug/service/unifiedbetests+0x4e929d1){noformat}
> It reproduces on an ASAN build (buildall.sh -asan -skiptests) with the
> following command:
>
>
> {noformat}
> be/build/latest/exprs/expr-test
> --gtest_filter=Instantiations/ExprTest.CastExprs*{noformat}
> This is related to the fix for "IMPALA-9531: Dropped support for dateless
> timestamps". When I revert that change, the test passes.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
