[
https://issues.apache.org/jira/browse/IMPALA-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839692#comment-17839692
]
ASF subversion and git services commented on IMPALA-13004:
----------------------------------------------------------
Commit a4a755d173822d3e123a871ffad6203ca98ff9f5 in impala's branch
refs/heads/branch-4.4.0 from Yida Wu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a4a755d17 ]
IMPALA-13004: Fix heap-use-after-free error in ExprTest AiFunctionsTest
The issue is that the code previously used a std::string_view to
hold the data which is actually returned by rapidjson::Document.
However, the rapidjson::Document object gets destroyed after
creating the std::string_view. This meant the std::string_view
referenced memory that was no longer valid, leading to a
heap-use-after-free error.
This patch fixes this issue by modifying the function to
return a std::string instead of a std::string_view. When the
function returns a string, it creates a copy of the
data from rapidjson::Document. This ensures the returned
string has its own memory allocation and doesn't rely on
the destroyed rapidjson::Document.
Tests:
Reran the asan build and passed.
Change-Id: I3bb9dcf9d72cce7ad37d5bc25821cf6ee55a8ab5
Reviewed-on: http://gerrit.cloudera.org:8080/21315
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> heap-use-after-free error in ExprTest AiFunctionsTest
> -----------------------------------------------------
>
> Key: IMPALA-13004
> URL: https://issues.apache.org/jira/browse/IMPALA-13004
> Project: IMPALA
> Issue Type: Bug
> Components: be
> Affects Versions: Impala 4.4.0
> Reporter: Andrew Sherman
> Assignee: Yida Wu
> Priority: Critical
> Fix For: Impala 4.4.0
>
>
> In an ASAN test, expr-test fails:
> {code}
> ==1601==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x63100152c826 at pc 0x00000298f841 bp 0x7ffc91fff460 sp 0x7ffc91fff458
> READ of size 2 at 0x63100152c826 thread T0
> #0 0x298f840 in rapidjson::GenericValue<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::GetType() const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:936:62
> #1 0x298d852 in bool rapidjson::GenericValue<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>
> >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator, 0u>
> >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator, 0u>&) const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1769:16
> #2 0x298d8d0 in bool rapidjson::GenericValue<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>
> >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator, 0u>
> >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator, 0u>&) const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1790:21
> #3 0x298d9e8 in bool rapidjson::GenericValue<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>
> >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator, 0u>
> >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator, 0u>&) const
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1781:21
> #4 0x28a0707 in impala_udf::StringVal
> impala::AiFunctions::AiGenerateTextInternal<false>(impala_udf::FunctionContext*,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, bool)
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:140:11
> #5 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody()
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12
> #6 0x8aeaa4c in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c)
> #7 0x8ae3ec4 in testing::Test::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae3ec4)
> #8 0x8ae4007 in testing::TestInfo::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4007)
> #9 0x8ae40e4 in testing::TestCase::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae40e4)
> #10 0x8ae45db in testing::internal::UnitTestImpl::RunAllTests()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae45db)
> #11 0x8ae4682 in testing::UnitTest::Run()
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4682)
> #12 0x249ac19 in main
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
> #13 0x7f4b0b911554 in __libc_start_main (/lib64/libc.so.6+0x22554)
> #14 0x2396af6 in _start
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2396af6)
> 0x63100152c826 is located 38 bytes inside of 65560-byte region
> [0x63100152c800,0x63100153c818)
> freed by thread T0 here:
> #0 0x2466ea7 in __interceptor_free
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2466ea7)
> #1 0x299656b in
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::Clear()
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:148:13
> #2 0x29964cd in
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::~MemoryPoolAllocator()
>
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:140:9
> #3 0x2996499 in rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>::Destroy()
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2391:9
> #4 0x298e47d in rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>::~GenericDocument()
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2073:9
> #5 0x28a0682 in impala_udf::StringVal
> impala::AiFunctions::AiGenerateTextInternal<false>(impala_udf::FunctionContext*,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, bool)
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:136:3
> #6 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody()
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12
> #7 0x8aeaa4c in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c)
> previously allocated by thread T0 here:
> #0 0x246706f in __interceptor_malloc
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x246706f)
> #1 0x298f3bf in
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::AddChunk(unsigned
> long)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:240:81
> #2 0x298f2bc in
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::Malloc(unsigned
> long)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:182:18
> #3 0x299a81c in rapidjson::GenericValue<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>
> >::SetArrayRaw(rapidjson::GenericValue<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >*, unsigned int,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>&)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1937:68
> #4 0x299a7b6 in rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>::EndArray(unsigned int)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2371:43
> #5 0x29989a0 in void rapidjson::GenericReader<rapidjson::UTF8<char>,
> rapidjson::UTF8<char>, rapidjson::CrtAllocator>::ParseArray<0u,
> rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>, rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>
> >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>&, rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>&)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/reader.h:686:21
> #6 0x299858e in void rapidjson::GenericReader<rapidjson::UTF8<char>,
> rapidjson::UTF8<char>, rapidjson::CrtAllocator>::ParseObject<0u,
> rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>, rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>
> >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>&, rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>&)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/reader.h:621:13
> #7 0x29976c8 in rapidjson::ParseResult
> rapidjson::GenericReader<rapidjson::UTF8<char>, rapidjson::UTF8<char>,
> rapidjson::CrtAllocator>::Parse<0u,
> rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>, rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>
> >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>&, rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>&)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/reader.h:501:13
> #8 0x299726a in rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>& rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>::ParseStream<0u, rapidjson::UTF8<char>,
> rapidjson::EncodedInputStream<rapidjson::UTF8<char>, rapidjson::MemoryStream>
> >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>,
> rapidjson::MemoryStream>&)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2159:40
> #9 0x2996f91 in rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>& rapidjson::GenericDocument<rapidjson::UTF8<char>,
> rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>,
> rapidjson::CrtAllocator>::Parse<0u, rapidjson::UTF8<char>
> >(rapidjson::UTF8<char>::Ch const*, unsigned long)
> /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2248:9
> #10 0x28a01ae in impala_udf::StringVal
> impala::AiFunctions::AiGenerateTextInternal<false>(impala_udf::FunctionContext*,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, impala_udf::StringVal const&,
> impala_udf::StringVal const&, bool)
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:109:15
> #11 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody()
> /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12
> #12 0x8aeaa4c in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
