Hello Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11886
to look at the new patch set (#3).
Change subject: run-test: don't report leaks that don't fail test
......................................................................
run-test: don't report leaks that don't fail test
LeakSanitizer will report a leak when allocating a string in
SuperviseThread. It's unclear why this is the case, but upon inspecting
the code, it seems like a false positive. The stack trace is as follows:
=================================================================
==93677==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 58 byte(s) in 1 object(s) allocated from:
#0 0x5318c8 in operator new(unsigned long)
/data/8/awong/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cc:92
#1 0x3ae3a9c3c8 in std::string::_Rep::_S_create(unsigned long, unsigned
long, std::allocator<char> const&) (/usr/lib64/libstdc++.so.6+0x3ae3a9c3c8)
#2 0x3ae3a9d19a in std::string::_Rep::_M_clone(std::allocator<char> const&,
unsigned long) (/usr/lib64/libstdc++.so.6+0x3ae3a9d19a)
#3 0x3ae3a9d5eb in std::string::reserve(unsigned long)
(/usr/lib64/libstdc++.so.6+0x3ae3a9d5eb)
#4 0x3ae3a9d770 in std::string::append(unsigned long, char)
(/usr/lib64/libstdc++.so.6+0x3ae3a9d770)
#5 0x7f518f799c12 in strings::SubstituteAndAppend(std::string*,
StringPiece, strings::internal::SubstituteArg const&,
strings::internal::SubstituteArg const&, strings::internal::SubstituteArg
const&, strings::internal::SubstituteArg const&,
strings::internal::SubstituteArg const&, strings::internal::SubstituteArg
const&, strings::internal::SubstituteArg const&,
strings::internal::SubstituteArg const&, strings::internal::SubstituteArg
const&, strings::internal::SubstituteArg const&)
../src/kudu/gutil/strings/substitute.cc:110:3
#6 0x536e76 in strings::Substitute(StringPiece,
strings::internal::SubstituteArg const&, strings::internal::SubstituteArg
const&, strings::internal::SubstituteArg const&,
strings::internal::SubstituteArg const&, strings::internal::SubstituteArg
const&, strings::internal::SubstituteArg const&,
strings::internal::SubstituteArg const&, strings::internal::SubstituteArg
const&, strings::internal::SubstituteArg const&,
strings::internal::SubstituteArg const&)
../src/kudu/gutil/strings/substitute.h:188:3
#7 0x7f5190590860 in kudu::Thread::SuperviseThread(void*)
../src/kudu/util/thread.cc:607:17
#8 0x3ae0e079d0 in start_thread (/lib64/libpthread.so.0+0x3ae0e079d0)
#9 0x3ae0ae88fc in clone (/lib64/libc.so.6+0x3ae0ae88fc)
This appears to be affecting a number tests, but generally only lines #0
and #1 are present in the logs, making them difficult to debug (a
developer would have to rerun the test with specific ASAN_OPTIONS to
unwind the stacktrace more). Namely, exactly_once_writes-itest
(KUDU-2517), kudu-admin-test (KUDU-2583), and rebalancer-tool-test
(untracked via Jira) all show the top of the above stack trace, and
based on the full stack trace, it seems these are all false positives.
The presence of issues like
https://github.com/google/sanitizers/issues/757 confirms that
LeakSanitizer can report false positives in workloads with high
concurrency. Generally, the test binary will return an error in the face
of real leaks, but in tests like the ones mentioned, the test may log
messages reporting leaks, but not actually return an error because the
"leak" was transient (e.g. see GenericServiceImpl::CheckLeaks).
We currently inject errors into JUnit XML report if any leaks are
reported, even for false positives, since the leak messages still find
their way into the logs. This patch updates this to only inject these
errors if the test also returned an error.
For clarity, I also threw in a log statement to
GenericServiceImpl::CheckLeaks denoting false positives.
Change-Id: I1f199795c48bd9b6106110aae132ec165eb0f647
---
M build-support/run-test.sh
M src/kudu/server/generic_service.cc
2 files changed, 10 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/11886/3
--
To view, visit http://gerrit.cloudera.org:8080/11886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f199795c48bd9b6106110aae132ec165eb0f647
Gerrit-Change-Number: 11886
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <[email protected]>
