Sahil Takiar created IMPALA-9388:
------------------------------------
Summary: Possible data race in BloomFilter::BucketFindAVX2
Key: IMPALA-9388
URL: https://issues.apache.org/jira/browse/IMPALA-9388
Project: IMPALA
Issue Type: Sub-task
Components: Backend
Reporter: Sahil Takiar
TSAN reports the following data race:
{code:java}
WARNING: ThreadSanitizer: data race (pid=20418)
Read of size 8 at 0x7b100015f150 by thread T1101:
#0 impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53
(impalad+0x21b1009)
#1 <null> <null> (0x7fe7b58dd882)
#2
impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*,
std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*,
bool*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:1140:29
(impalad+0x285aa2e)
#3 impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:456:19
(impalad+0x2858410)
#4 impala::HdfsParquetScanner::ProcessSplit()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:353:21
(impalad+0x285740b)
#5 impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext,
std::allocator<impala::FilterContext> > const&, impala::MemPool*,
impala::io::ScanRange*, long*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:514:21
(impalad+0x249e0bb)
#6 impala::HdfsScanNode::ScannerThread(bool, long)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:415:7
(impalad+0x249d6b7)
#7
impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0::operator()()
const
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:337:13
(impalad+0x249ec06)
#8
boost::detail::function::void_function_obj_invoker0<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0,
void>::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
(impalad+0x249ea19)
#9 boost::function0<void>::operator()() const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
(impalad+0x1cf2e21)
#10 impala::Thread::SuperviseThread(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
(impalad+0x22a3c06)
#11 void boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
>::operator()<void (*)(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void
(*&)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
(impalad+0x22abcbc)
#12 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
impala::Promise<long, (impala::PromiseMode)0>*),
boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
(impalad+0x22abbd3)
#13 boost::detail::thread_data<boost::_bi::bind_t<void, void
(*)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
(impalad+0x22ab8c0)
#14 thread_proxy <null> (impalad+0x2f7ccc9)
Previous write of size 8 at 0x7b100015f150 by thread T126 (mutexes: write
M940824749541653120):
#0 impala::BloomFilter::Init(int)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:56:14
(impalad+0x21b0647)
#1 impala::BloomFilter::Init(impala::BloomFilterPB const&, unsigned char
const*, unsigned long)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:63:3
(impalad+0x21b080f)
#2
impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
const&, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:339:39
(impalad+0x1f3f88f)
#3 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&,
kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17
(impalad+0x1f2cd2b)
#4 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB
const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9
(impalad+0x205a237)
#5
impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
const&, scoped_refptr<kudu::rpc::ResultTracker>
const&)::$_7::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13
(impalad+0x2102d44)
#6 std::_Function_handler<void (google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*),
impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
const&, scoped_refptr<kudu::rpc::ResultTracker>
const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message
const*, google::protobuf::Message*, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2
(impalad+0x2102b2e)
#7 std::function<void (google::protobuf::Message const*,
google::protobuf::Message*,
kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14
(impalad+0x278aa38)
#8 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3
(impalad+0x278a39e)
#9 impala::ImpalaServicePool::RunThread()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15
(impalad+0x1e199df)
#10 boost::_mfi::mf0<void,
impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29
(impalad+0x1e1fe36)
#11 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*>
>::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>,
boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void,
impala::ImpalaServicePool>&, boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9
(impalad+0x1e1fd8a)
#12 boost::_bi::bind_t<void, boost::_mfi::mf0<void,
impala::ImpalaServicePool>,
boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
(impalad+0x1e1fd13)
#13
boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
boost::_mfi::mf0<void, impala::ImpalaServicePool>,
boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
void>::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
(impalad+0x1e1fb09)
#14 boost::function0<void>::operator()() const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
(impalad+0x1cf2e21)
#15 impala::Thread::SuperviseThread(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
(impalad+0x22a3c06)
#16 void boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
>::operator()<void (*)(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void
(*&)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
(impalad+0x22abcbc)
#17 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
impala::Promise<long, (impala::PromiseMode)0>*),
boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
(impalad+0x22abbd3)
#18 boost::detail::thread_data<boost::_bi::bind_t<void, void
(*)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
(impalad+0x22ab8c0)
#19 thread_proxy <null> (impalad+0x2f7ccc9)
Location is heap block of size 64 at 0x7b100015f140 allocated by thread T126:
#0 operator new(unsigned long)
/mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:44
(impalad+0x191d233)
#1
impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
const&, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:323:36
(impalad+0x1f3f6ec)
#2 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&,
kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17
(impalad+0x1f2cd2b)
#3 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB
const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9
(impalad+0x205a237)
#4
impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
const&, scoped_refptr<kudu::rpc::ResultTracker>
const&)::$_7::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13
(impalad+0x2102d44)
#5 std::_Function_handler<void (google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*),
impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
const&, scoped_refptr<kudu::rpc::ResultTracker>
const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message
const*, google::protobuf::Message*, kudu::rpc::RpcContext*)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2
(impalad+0x2102b2e)
#6 std::function<void (google::protobuf::Message const*,
google::protobuf::Message*,
kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14
(impalad+0x278aa38)
#7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3
(impalad+0x278a39e)
#8 impala::ImpalaServicePool::RunThread()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15
(impalad+0x1e199df)
#9 boost::_mfi::mf0<void,
impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29
(impalad+0x1e1fe36)
#10 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*>
>::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>,
boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void,
impala::ImpalaServicePool>&, boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9
(impalad+0x1e1fd8a)
#11 boost::_bi::bind_t<void, boost::_mfi::mf0<void,
impala::ImpalaServicePool>,
boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
(impalad+0x1e1fd13)
#12
boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
boost::_mfi::mf0<void, impala::ImpalaServicePool>,
boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
void>::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
(impalad+0x1e1fb09)
#13 boost::function0<void>::operator()() const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
(impalad+0x1cf2e21)
#14 impala::Thread::SuperviseThread(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
(impalad+0x22a3c06)
#15 void boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
>::operator()<void (*)(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void
(*&)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
(impalad+0x22abcbc)
#16 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
impala::Promise<long, (impala::PromiseMode)0>*),
boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
(impalad+0x22abbd3)
#17 boost::detail::thread_data<boost::_bi::bind_t<void, void
(*)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
(impalad+0x22ab8c0)
#18 thread_proxy <null> (impalad+0x2f7ccc9)
Mutex M940824749541653120 is already destroyed.
Thread T1101 (tid=31826, running) created by thread T443 at:
#0 pthread_create
/mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889
(impalad+0x18b4c1b)
#1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9)
#2 boost::thread::thread<void (*)(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>,
impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void
(*)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>,
impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13
(impalad+0x22a62bb)
#3 impala::Thread::StartThread(std::string const&, std::string const&,
boost::function<void ()> const&, std::unique_ptr<impala::Thread,
std::default_delete<impala::Thread> >*, bool)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13
(impalad+0x22a35a5)
#4 impala::Status
impala::Thread::Create<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0>(std::string
const&, std::string const&,
impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0
const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*,
bool)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:74:12
(impalad+0x249d21d)
#5
impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:340:14
(impalad+0x249c86f)
#6
impala::HdfsScanNode::AddDiskIoRanges(std::vector<impala::io::ScanRange*,
std::allocator<impala::io::ScanRange*> > const&, impala::EnqueueLocation)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:212:24
(impalad+0x249ce8e)
#7 impala::HdfsScanner::IssueFooterRanges(impala::HdfsScanNodeBase*,
impala::THdfsFileFormat::type const&, std::vector<impala::HdfsFileDesc*,
std::allocator<impala::HdfsFileDesc*> > const&)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scanner.cc:836:5
(impalad+0x2389be8)
#8
impala::HdfsParquetScanner::IssueInitialRanges(impala::HdfsScanNodeBase*,
std::vector<impala::HdfsFileDesc*, std::allocator<impala::HdfsFileDesc*> >
const&)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:84:10
(impalad+0x2852ddb)
#9 impala::HdfsScanNodeBase::IssueInitialScanRanges(impala::RuntimeState*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node-base.cc:600:9
(impalad+0x235ce29)
#10 impala::HdfsScanNode::GetNext(impala::RuntimeState*, impala::RowBatch*,
bool*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:90:21
(impalad+0x249ba2c)
#11 impala::FragmentInstanceState::ExecInternal()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:382:7
(impalad+0x1f1c514)
#12 impala::FragmentInstanceState::Exec()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14
(impalad+0x1f1a3a6)
#13 impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:694:24
(impalad+0x1f2ca53)
#14 impala::QueryState::StartFInstances()::$_7::operator()() const
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:603:35
(impalad+0x1f2eba2)
#15
boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::$_7,
void>::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
(impalad+0x1f2e9c9)
#16 boost::function0<void>::operator()() const
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
(impalad+0x1cf2e21)
#17 impala::Thread::SuperviseThread(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
(impalad+0x22a3c06)
#18 void boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
>::operator()<void (*)(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void
(*&)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list0&, int)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
(impalad+0x22abcbc)
#19 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
impala::Promise<long, (impala::PromiseMode)0>*),
boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
>::operator()()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
(impalad+0x22abbd3)
#20 boost::detail::thread_data<boost::_bi::bind_t<void, void
(*)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
boost::_bi::value<impala::ThreadDebugInfo*>,
boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run()
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
(impalad+0x22ab8c0)
#21 thread_proxy <null> (impalad+0x2f7ccc9)
Thread T126 (tid=20900, running) created by main thread at:
#0 pthread_create
/mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889
(impalad+0x18b4c1b)
#1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9)
#2 boost::thread::thread<void (*)(std::string const&, std::string const&,
boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>,
impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void
(*)(std::string const&, std::string const&, boost::function<void ()>,
impala::ThreadDebugInfo const*, impala::Promise<long,
(impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>,
impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*)
/data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13
(impalad+0x22a62bb)
#3 impala::Thread::StartThread(std::string const&, std::string const&,
boost::function<void ()> const&, std::unique_ptr<impala::Thread,
std::default_delete<impala::Thread> >*, bool)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13
(impalad+0x22a35a5)
#4 impala::Status impala::Thread::Create<void
(impala::ImpalaServicePool::*)(), impala::ImpalaServicePool*>(std::string
const&, std::string const&, void (impala::ImpalaServicePool::* const&)(),
impala::ImpalaServicePool* const&, std::unique_ptr<impala::Thread,
std::default_delete<impala::Thread> >*, bool)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:81:12
(impalad+0x1e1ba47)
#5 impala::ImpalaServicePool::Init(int)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:91:5
(impalad+0x1e19725)
#6 impala::RpcMgr::RegisterService(int, int,
kudu::rpc::GeneratedServiceIf*, impala::MemTracker*)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/rpc-mgr.cc:163:3
(impalad+0x1e08ba4)
#7 impala::DataStreamService::Init()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:78:3
(impalad+0x2059b2f)
#8 impala::ExecEnv::Init()
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/exec-env.cc:391:3
(impalad+0x1e50481)
#9 ImpaladMain(int, char**)
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/impalad-main.cc:72:3
(impalad+0x2075af7)
#10 main
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/daemon-main.cc:37:12
(impalad+0x191f96a)SUMMARY: ThreadSanitizer: data race
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53
in impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const{code}
Notice that part of the stack right before {{BloomFilter::BucketFindAVX2}} is
missing. I think because the Bloom Filter functions calls are codegened.
AFAICT, the stack should be something like:
{{HdfsParquetScanner::AssembleRows}} ->
{{HdfsParquetScanner::TransferScratchTuples}} ->
{{codegend_process_scratch_batch_fn_}} -> {{FilterContext::Eval}} ->
{{RuntimeFilter::Eval}} -> {{RuntimeFilter::Eval}} -> {{BloomFilter::Find}}
I haven't been able to successfully get codegen to run with
{{-fsanitize=thread}} yet, which is why part of the stack is missing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
