Attila Bukor has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/23785 )
Change subject: KUDU-3545 fix race condition when (un)registring EH frames
......................................................................
KUDU-3545 fix race condition when (un)registring EH frames
This patch addresses race condition in libgcc which is triggered by
concurrent registering/unregistering of EH frames by LLVM's
RTDyldMemoryManager. Also, since libgcc expects a null-terminated
list of FDEs, this patch adds extra 4 zero bytes in the end of each
section allocated for EH frames.
I added a new test scenario to reproduce the race, and it was triggering
TSAN warnings every time if rolling back the fix (see below). For more
robust and more reliable race reproduction, I added a new flag
(tagged as 'hidden', default value is 1) to control the maximum number
of threads in the codegen compiler thread pool:
--codegen_compiler_manager_pool_max_threads_num
We might start using it later to allow for parallel compilation of row
projection functions in the presence of concurrent scan requests.
WARNING: ThreadSanitizer: data race (pid=3406438)
Write of size 8 at 0x7b4000008228 by thread T6 (mutexes: write M1879):
#0 memmove sanitizer_common/sanitizer_common_interceptors.inc:791:3
(codegen-test+0x2edb86)
#1 <null> <null> (libgcc_s.so.1+0x1345d)
#2 llvm::RuntimeDyldELF::registerEHFrames()
#3 llvm::RuntimeDyld::registerEHFrames()
#4 llvm::MCJIT::finalizeLoadedModules()
#5 llvm::MCJIT::finalizeObject()
#6 kudu::codegen::ModuleBuilder::Compile()
...
Previous write of size 8 at 0x7b4000008228 by thread T5 (mutexes: write
M1869):
#0 malloc
/root/Projects/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:652:5
(codegen-test+0x2b2c34)
#1 <null> <null> (libgcc_s.so.1+0x1318a)
#2 llvm::RuntimeDyldELF::registerEHFrames()
#3 llvm::RuntimeDyld::registerEHFrames()
#4 llvm::MCJIT::finalizeLoadedModules()
#5 llvm::MCJIT::finalizeObject()
#6 kudu::codegen::ModuleBuilder::Compile()
#7 kudu::codegen::RowProjectorFunctions::Create()
#8 kudu::codegen::CodeGenerator::CompileRowProjector()
...
SUMMARY: ThreadSanitizer: data race (/lib64/libgcc_s.so.1+0x1345d)
Change-Id: Ia91ada71f663451e11149a190def6a8ccd0b4ef5
Reviewed-on: http://gerrit.cloudera.org:8080/23785
Reviewed-by: Zoltan Martonka <[email protected]>
Reviewed-by: Attila Bukor <[email protected]>
Tested-by: Attila Bukor <[email protected]>
---
M src/kudu/codegen/CMakeLists.txt
M src/kudu/codegen/codegen-test.cc
M src/kudu/codegen/compilation_manager.cc
A src/kudu/codegen/jit_frame_manager.cc
A src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
6 files changed, 213 insertions(+), 3 deletions(-)
Approvals:
Zoltan Martonka: Looks good to me, but someone else must approve
Attila Bukor: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/23785
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia91ada71f663451e11149a190def6a8ccd0b4ef5
Gerrit-Change-Number: 23785
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Abhishek Chennaka <[email protected]>
Gerrit-Reviewer: Ashwani Raina <[email protected]>
Gerrit-Reviewer: Attila Bukor <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Chovan <[email protected]>
Gerrit-Reviewer: Zoltan Martonka <[email protected]>
