[
https://issues.apache.org/jira/browse/IMPALA-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033935#comment-17033935
]
ASF subversion and git services commented on IMPALA-5904:
---------------------------------------------------------
Commit ca6c8d43d78de3c4fa5e4606882b0865e2bce5b4 in impala's branch
refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ca6c8d4 ]
IMPALA-5904: Add full_tsan option and fix several TSAN bugs
This patch adds an additional build flag -full_tsan in addition to the
existing -tsan build flag. -full_tsan is equivalent to the current -tsan
behavior, and -tsan is changed to set the ignore_noninstrumented_modules
flag to true. ignore_noninstrumented_modules causes TSAN to ignore any
modules that are not TSAN-instrumented. This is necessary to get TSAN to
play nicely with Java, since Java is not TSAN-instrumented (see
https://wiki.openjdk.java.net/display/tsan/Main and JDK-8208520). While
this might decrease the number of issues surfaced by TSAN, it drastically
decreases the amount of noise produced by TSAN because the JVM is not
running TSAN-instrumented code. Without this flag set to true, almost
every single backend test fails with the error:
WARNING: ThreadSanitizer: data race (pid=12939)
Write of size 1 at 0x7fcbe379c4c6 by thread T31:
#0 strncpy
/mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:650
(unifiedbetests+0x1b2a4ad)
#1 <null> <null> (libjvm.so+0x90e706)
This patch fixes various TSAN bugs (e.g. data races) reported while
running backend tests and E2E against a TSAN build (it does not make
Impala completely TSAN-clean). This patch makes the following changes:
* Fixes several bugs involving issues with updating shared variables
between threads
* Fixes a few race conditions in test classes
* Where possible, existing locks are used to fix any data races; in cases
where the locking logic is non-trivial, atomics are used
* There are a few places where variables are marked as 'volatile'
presumably for synchronization purposes; TSAN flags these 'volatile'
variables as unsafe, and according to
https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#Rconc-volatile
using 'volatile' for synchronization is dangerous; in these cases, the
'volatile' variables are changed to 'atomic' variables
* This patch adds a suppression file (bin/tsan-suppresions.txt) similar to
the UBSAN suppresion file (bin/ubsan-suppresions.txt)
Testing:
* Ran exhaustive tests
* Ran core tests w/ ASAN build
* Manually re-ran backend tests against a TSAN build and made sure the
reported errors are gone
Change-Id: I3d7ef5c228afd5882e145e6f53885b355d6c25a0
Reviewed-on: http://gerrit.cloudera.org:8080/15116
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Enable ThreadSanitizer for Impala
> ---------------------------------
>
> Key: IMPALA-5904
> URL: https://issues.apache.org/jira/browse/IMPALA-5904
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Reporter: Tim Armstrong
> Assignee: Sahil Takiar
> Priority: Major
> Labels: ramp-up
> Attachments: impalad.ERROR
>
>
> It would be great to be able to automatically detect data races in Impala
> using ThreadSanitizer to avoid tricky-to-reproduce bugs. This issue tracks
> enabling ThreadSanitizer, fixing bugs and adding suppressions to get to the
> point where Impala runs cleanly with the sanitizer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
