Hello Tim Armstrong, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16426
to look at the new patch set (#2).
Change subject: IMPALA-9740, IMPALA-9403: Fix remaining custom cluster TSAN
errors
......................................................................
IMPALA-9740, IMPALA-9403: Fix remaining custom cluster TSAN errors
This patch fixes the remaining TSAN errors reported while running custom
cluster tests. After this patch, TSAN can be enabled for custom cluster
tests (currently it is only run for be tests).
Adds a data race suppression for
HdfsColumnarScanner::ProcessScratchBatchCodegenOrInterpret, which
usually calls a codegen function. TSAN currently does not support
codegen functions, so this warning needs to be suppressed. The call
stack of this warning is:
#0 kudu::BlockBloomFilter::Find(unsigned int) const
kudu/util/block_bloom_filter.cc:257:7
#1 <null> <null> (0x7f19af1c74cd)
#2
impala::HdfsColumnarScanner::ProcessScratchBatchCodegenOrInterpret(impala::RowBatch*)
exec/hdfs-columnar-scanner.cc:106:10
#3 impala::HdfsColumnarScanner::TransferScratchTuples(impala::RowBatch*)
exec/hdfs-columnar-scanner.cc:66:34
Fixes a data race in DmlExecState::FinalizeHdfsInsert where a local
HdfsFsCache::HdfsFsMap is unsafely passed between threads of a
HdfsOperationSet. HdfsOperationSet instances are run in a
HdfsOpThreadPool and each operation is run in one of the threads from
the pool. Each operation uses HdfsFsCache::GetConnection to get a hdfsFs
instance. GetConnection can take in a 'local_cache' of hdfsFs instances
before using the global map. The race condition is that the same local
cache is used for all operations in HdfsOperationSet.
Testing:
* Re-ran TSAN tests and confirmed the data races have disappeared
Change-Id: If1658a9b56d220e2cfd1f8b958604edcdf7757f4
---
M be/src/util/hdfs-bulk-ops.cc
M be/src/util/hdfs-bulk-ops.h
M bin/tsan-suppressions.txt
3 files changed, 18 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/16426/2
--
To view, visit http://gerrit.cloudera.org:8080/16426
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If1658a9b56d220e2cfd1f8b958604edcdf7757f4
Gerrit-Change-Number: 16426
Gerrit-PatchSet: 2
Gerrit-Owner: Sahil Takiar <stak...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
