[
https://issues.apache.org/jira/browse/IMPALA-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riza Suminto resolved IMPALA-14604.
-----------------------------------
Fix Version/s: Impala 5.0.0
Assignee: Riza Suminto
Resolution: Fixed
> Potential heap-use-after-free in HdfsFsCache::GetConnection
> -----------------------------------------------------------
>
> Key: IMPALA-14604
> URL: https://issues.apache.org/jira/browse/IMPALA-14604
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> Downstream ASAN build caught heap-use-after-free in
> HdfsFsCache::GetConnection (hdfs-fs-cache.cc).
> {code:java}
> Error Message
> Address Sanitizer message detected in
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/logs/be_tests/LastTest.log
> Standard Error
> ==822570==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x6030004ab1e0 at pc 0x0000028ebd4e bp 0x7fffeda23290 sp 0x7fffeda22a40
> READ of size 2 at 0x6030004ab1e0 thread T0
> #0 0x28ebd4d in memmove
> (/data0/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x28ebd4d)
> #1 0x7f7803fbfec4 in java_lang_String::create_from_str(char const*,
> JavaThread*) [clone .part.258]
> (/usr/lib/jvm/java-17/lib/server/libjvm.so+0x84fec4)
> #2 0x7f7803fc0139 in java_lang_String::create_oop_from_str(char const*,
> JavaThread*) (/usr/lib/jvm/java-17/lib/server/libjvm.so+0x850139)
> #3 0x7f7804041581 in jni_NewStringUTF
> (/usr/lib/jvm/java-17/lib/server/libjvm.so+0x8d1581)
> #4 0x9549516 in newJavaStr
> /grid/0/jenkins/workspace/workspace/CDH-parallel-redhat8/SOURCES/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c:76:13
> #5 0x9549516 in hadoopConfSetStr
> /grid/0/jenkins/workspace/workspace/CDH-parallel-redhat8/SOURCES/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c:925:12
> #6 0x9542828 in hdfsBuilderConnect
> /grid/0/jenkins/workspace/workspace/CDH-parallel-redhat8/SOURCES/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c:720:16
> #7 0x6ee497a in
> impala::HdfsFsCache::GetConnection(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, hdfs_internal**,
> boost::unordered::unordered_map<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >, hdfs_internal*,
> boost::hash<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> std::allocator<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const, hdfs_internal*> > >*,
> std::vector<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > >, std::allocator<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > > > > const*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/hdfs-fs-cache.cc:113:13
> #8 0x7095a22 in impala::TmpDirS3::GetConnection(impala::TmpFileMgr*,
> hdfs_internal**)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:855:35
> #9 0x7095d33 in impala::TmpDirS3::VerifyAndCreate(impala::MetricGroup*,
> std::vector<bool, std::allocator<bool> >*, bool, impala::TmpFileMgr*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:866:3
> #10 0x7089631 in
> impala::TmpFileMgr::InitCustom(std::vector<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > > > const&, bool, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, bool,
> impala::MetricGroup*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:339:34
> #11 0x31cf5f7 in
> impala::TmpFileMgrTest_TestRemoteUploadToNonExistentPath_Test::TestBody()
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:2067:3
> #12 0x96a13ec in void
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #13 0x96a13ec in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> #14 0x968619d in testing::Test::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2687:50
> #15 0x968619d in testing::Test::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2677:6
> #16 0x9686354 in testing::TestInfo::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2836:14
> #17 0x9686510 in testing::TestSuite::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:3015:33
> #18 0x9686510 in testing::TestSuite::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2968:6
> #19 0x96992ae in testing::internal::UnitTestImpl::RunAllTests()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:5920:47
> #20 0x9686724 in bool
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
> bool>(testing::internal::UnitTestImpl*, bool
> (testing::internal::UnitTestImpl::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #21 0x9686724 in bool
> testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
> bool>(testing::internal::UnitTestImpl*, bool
> (testing::internal::UnitTestImpl::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> #22 0x9686724 in testing::UnitTest::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:5484:55
> #23 0x297d8a1 in main
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
> #24 0x7f780176d7e4 in __libc_start_main (/lib64/libc.so.6+0x3a7e4)
> #25 0x287977d in _start
> (/data0/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x287977d)
> 0x6030004ab1e0 is located 0 bytes inside of 26-byte region
> [0x6030004ab1e0,0x6030004ab1fa)
> freed by thread T0 here:
> #0 0x297b22f in operator delete(void*)
> (/data0/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x297b22f)
> #1 0x6ee4936 in
> impala::HdfsFsCache::GetConnection(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, hdfs_internal**,
> boost::unordered::unordered_map<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >, hdfs_internal*,
> boost::hash<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> std::allocator<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const, hdfs_internal*> > >*,
> std::vector<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > >, std::allocator<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > > > > const*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/hdfs-fs-cache.cc:110:11
> #2 0x7095a22 in impala::TmpDirS3::GetConnection(impala::TmpFileMgr*,
> hdfs_internal**)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:855:35
> #3 0x7095d33 in impala::TmpDirS3::VerifyAndCreate(impala::MetricGroup*,
> std::vector<bool, std::allocator<bool> >*, bool, impala::TmpFileMgr*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:866:3
> #4 0x7089631 in
> impala::TmpFileMgr::InitCustom(std::vector<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > > > const&, bool, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, bool,
> impala::MetricGroup*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:339:34
> #5 0x31cf5f7 in
> impala::TmpFileMgrTest_TestRemoteUploadToNonExistentPath_Test::TestBody()
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:2067:3
> #6 0x96a13ec in void
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #7 0x96a13ec in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> previously allocated by thread T0 here:
> #0 0x297a4bf in operator new(unsigned long)
> (/data0/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x297a4bf)
> #1 0x29884ce in void std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*,
> char*, std::forward_iterator_tag)
> /data/jenkins/workspace/impala-cdh_main-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/basic_string.tcc:219:14
> #2 0x2a7f4c1 in std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > >::pair(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> >, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > > const&)
> /data/jenkins/workspace/impala-cdh_main-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_pair.h:314:17
> #3 0x6ee490b in
> impala::HdfsFsCache::GetConnection(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, hdfs_internal**,
> boost::unordered::unordered_map<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >, hdfs_internal*,
> boost::hash<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > >,
> std::allocator<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const, hdfs_internal*> > >*,
> std::vector<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > >, std::allocator<std::pair<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > > > > const*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/hdfs-fs-cache.cc:107:28
> #4 0x7095a22 in impala::TmpDirS3::GetConnection(impala::TmpFileMgr*,
> hdfs_internal**)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:855:35
> #5 0x7095d33 in impala::TmpDirS3::VerifyAndCreate(impala::MetricGroup*,
> std::vector<bool, std::allocator<bool> >*, bool, impala::TmpFileMgr*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:866:3
> #6 0x7089631 in
> impala::TmpFileMgr::InitCustom(std::vector<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > > > const&, bool, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, bool,
> impala::MetricGroup*)
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:339:34
> #7 0x31cf5f7 in
> impala::TmpFileMgrTest_TestRemoteUploadToNonExistentPath_Test::TestBody()
> /data/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:2067:3
> #8 0x96a13ec in void
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #9 0x96a13ec in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> SUMMARY: AddressSanitizer: heap-use-after-free
> (/data0/jenkins/workspace/impala-cdh_main-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x28ebd4d)
> in memmove
> Shadow bytes around the buggy address:
> 0x0c068008d5e0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
> 0x0c068008d5f0: fd fd fa fa fd fd fd fa fa fa fd fd fd fd fa fa
> 0x0c068008d600: fd fd fd fa fa fa fd fd fd fd fa fa fd fd fd fa
> 0x0c068008d610: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
> 0x0c068008d620: fd fd fa fa 00 00 06 fa fa fa 00 00 00 00 fa fa
> =>0x0c068008d630: 00 00 00 fa fa fa fa fa fa fa fa fa[fd]fd fd fd
> 0x0c068008d640: fa fa 00 00 00 fa fa fa fd fd fd fa fa fa fa fa
> 0x0c068008d650: fa fa fa fa fa fa fa fa fa fa fd fd fd fa fa fa
> 0x0c068008d660: fd fd fd fa fa fa fd fd fd fd fa fa fd fd fd fd
> 0x0c068008d670: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
> 0x0c068008d680: fd fd fa fa fa fa fa fa fa fa fd fd fd fd fa fa
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone: cb
> ==822570==ABORTING
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
