[
https://issues.apache.org/jira/browse/IMPALA-9388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Tauber-Marshall reassigned IMPALA-9388:
----------------------------------------------
Assignee: Thomas Tauber-Marshall
> Possible data race in BloomFilter::BucketFindAVX2
> -------------------------------------------------
>
> Key: IMPALA-9388
> URL: https://issues.apache.org/jira/browse/IMPALA-9388
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Thomas Tauber-Marshall
> Priority: Major
>
> TSAN reports the following data race:
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=20418)
> Read of size 8 at 0x7b100015f150 by thread T1101:
> #0 impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53
> (impalad+0x21b1009)
> #1 <null> <null> (0x7fe7b58dd882)
> #2
> impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*,
> std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*,
> bool*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:1140:29
> (impalad+0x285aa2e)
> #3 impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:456:19
> (impalad+0x2858410)
> #4 impala::HdfsParquetScanner::ProcessSplit()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:353:21
> (impalad+0x285740b)
> #5 impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext,
> std::allocator<impala::FilterContext> > const&, impala::MemPool*,
> impala::io::ScanRange*, long*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:514:21
> (impalad+0x249e0bb)
> #6 impala::HdfsScanNode::ScannerThread(bool, long)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:415:7
> (impalad+0x249d6b7)
> #7
> impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0::operator()()
> const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:337:13
> (impalad+0x249ec06)
> #8
> boost::detail::function::void_function_obj_invoker0<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0,
> void>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
> (impalad+0x249ea19)
> #9 boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
> (impalad+0x1cf2e21)
> #10 impala::Thread::SuperviseThread(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
> (impalad+0x22a3c06)
> #11 void boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
> (impalad+0x22abcbc)
> #12 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> (impalad+0x22abbd3)
> #13 boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
> (impalad+0x22ab8c0)
> #14 thread_proxy <null> (impalad+0x2f7ccc9)
> Previous write of size 8 at 0x7b100015f150 by thread T126 (mutexes: write
> M940824749541653120):
> #0 impala::BloomFilter::Init(int)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:56:14
> (impalad+0x21b0647)
> #1 impala::BloomFilter::Init(impala::BloomFilterPB const&, unsigned char
> const*, unsigned long)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:63:3
> (impalad+0x21b080f)
> #2
> impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
> const&, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:339:39
> (impalad+0x1f3f88f)
> #3 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB
> const&, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17
> (impalad+0x1f2cd2b)
> #4 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB
> const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9
> (impalad+0x205a237)
> #5
> impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
> const&, scoped_refptr<kudu::rpc::ResultTracker>
> const&)::$_7::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13
> (impalad+0x2102d44)
> #6 std::_Function_handler<void (google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*),
> impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
> const&, scoped_refptr<kudu::rpc::ResultTracker>
> const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message
> const*, google::protobuf::Message*, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2
> (impalad+0x2102b2e)
> #7 std::function<void (google::protobuf::Message const*,
> google::protobuf::Message*,
> kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14
> (impalad+0x278aa38)
> #8 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3
> (impalad+0x278a39e)
> #9 impala::ImpalaServicePool::RunThread()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15
> (impalad+0x1e199df)
> #10 boost::_mfi::mf0<void,
> impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x1e1fe36)
> #11 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*>
> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>,
> boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void,
> impala::ImpalaServicePool>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9
> (impalad+0x1e1fd8a)
> #12 boost::_bi::bind_t<void, boost::_mfi::mf0<void,
> impala::ImpalaServicePool>,
> boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> (impalad+0x1e1fd13)
> #13
> boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
> boost::_mfi::mf0<void, impala::ImpalaServicePool>,
> boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
> void>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
> (impalad+0x1e1fb09)
> #14 boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
> (impalad+0x1cf2e21)
> #15 impala::Thread::SuperviseThread(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
> (impalad+0x22a3c06)
> #16 void boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
> (impalad+0x22abcbc)
> #17 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> (impalad+0x22abbd3)
> #18 boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
> (impalad+0x22ab8c0)
> #19 thread_proxy <null> (impalad+0x2f7ccc9)
> Location is heap block of size 64 at 0x7b100015f140 allocated by thread
> T126:
> #0 operator new(unsigned long)
> /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:44
> (impalad+0x191d233)
> #1
> impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
> const&, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:323:36
> (impalad+0x1f3f6ec)
> #2 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB
> const&, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17
> (impalad+0x1f2cd2b)
> #3 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB
> const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9
> (impalad+0x205a237)
> #4
> impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
> const&, scoped_refptr<kudu::rpc::ResultTracker>
> const&)::$_7::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13
> (impalad+0x2102d44)
> #5 std::_Function_handler<void (google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*),
> impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity>
> const&, scoped_refptr<kudu::rpc::ResultTracker>
> const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message
> const*, google::protobuf::Message*, kudu::rpc::RpcContext*)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2
> (impalad+0x2102b2e)
> #6 std::function<void (google::protobuf::Message const*,
> google::protobuf::Message*,
> kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14
> (impalad+0x278aa38)
> #7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3
> (impalad+0x278a39e)
> #8 impala::ImpalaServicePool::RunThread()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15
> (impalad+0x1e199df)
> #9 boost::_mfi::mf0<void,
> impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x1e1fe36)
> #10 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*>
> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>,
> boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void,
> impala::ImpalaServicePool>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9
> (impalad+0x1e1fd8a)
> #11 boost::_bi::bind_t<void, boost::_mfi::mf0<void,
> impala::ImpalaServicePool>,
> boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> (impalad+0x1e1fd13)
> #12
> boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
> boost::_mfi::mf0<void, impala::ImpalaServicePool>,
> boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
> void>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
> (impalad+0x1e1fb09)
> #13 boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
> (impalad+0x1cf2e21)
> #14 impala::Thread::SuperviseThread(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
> (impalad+0x22a3c06)
> #15 void boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
> (impalad+0x22abcbc)
> #16 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> (impalad+0x22abbd3)
> #17 boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
> (impalad+0x22ab8c0)
> #18 thread_proxy <null> (impalad+0x2f7ccc9)
> Mutex M940824749541653120 is already destroyed.
> Thread T1101 (tid=31826, running) created by thread T443 at:
> #0 pthread_create
> /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889
> (impalad+0x18b4c1b)
> #1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9)
> #2 boost::thread::thread<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string,
> boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long,
> (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string,
> boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long,
> (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13
> (impalad+0x22a62bb)
> #3 impala::Thread::StartThread(std::string const&, std::string const&,
> boost::function<void ()> const&, std::unique_ptr<impala::Thread,
> std::default_delete<impala::Thread> >*, bool)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13
> (impalad+0x22a35a5)
> #4 impala::Status
> impala::Thread::Create<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0>(std::string
> const&, std::string const&,
> impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0
> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread>
> >*, bool)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:74:12
> (impalad+0x249d21d)
> #5
> impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:340:14
> (impalad+0x249c86f)
> #6
> impala::HdfsScanNode::AddDiskIoRanges(std::vector<impala::io::ScanRange*,
> std::allocator<impala::io::ScanRange*> > const&, impala::EnqueueLocation)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:212:24
> (impalad+0x249ce8e)
> #7 impala::HdfsScanner::IssueFooterRanges(impala::HdfsScanNodeBase*,
> impala::THdfsFileFormat::type const&, std::vector<impala::HdfsFileDesc*,
> std::allocator<impala::HdfsFileDesc*> > const&)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scanner.cc:836:5
> (impalad+0x2389be8)
> #8
> impala::HdfsParquetScanner::IssueInitialRanges(impala::HdfsScanNodeBase*,
> std::vector<impala::HdfsFileDesc*, std::allocator<impala::HdfsFileDesc*> >
> const&)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:84:10
> (impalad+0x2852ddb)
> #9
> impala::HdfsScanNodeBase::IssueInitialScanRanges(impala::RuntimeState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node-base.cc:600:9
> (impalad+0x235ce29)
> #10 impala::HdfsScanNode::GetNext(impala::RuntimeState*,
> impala::RowBatch*, bool*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:90:21
> (impalad+0x249ba2c)
> #11 impala::FragmentInstanceState::ExecInternal()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:382:7
> (impalad+0x1f1c514)
> #12 impala::FragmentInstanceState::Exec()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14
> (impalad+0x1f1a3a6)
> #13 impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:694:24
> (impalad+0x1f2ca53)
> #14 impala::QueryState::StartFInstances()::$_7::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:603:35
> (impalad+0x1f2eba2)
> #15
> boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::$_7,
> void>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11
> (impalad+0x1f2e9c9)
> #16 boost::function0<void>::operator()() const
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
> (impalad+0x1cf2e21)
> #17 impala::Thread::SuperviseThread(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3
> (impalad+0x22a3c06)
> #18 void boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*>
> >::operator()<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
> (impalad+0x22abcbc)
> #19 boost::_bi::bind_t<void, void (*)(std::string const&, std::string
> const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >
> >::operator()()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
> (impalad+0x22abbd3)
> #20 boost::detail::thread_data<boost::_bi::bind_t<void, void
> (*)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long,
> (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >
> >::run()
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17
> (impalad+0x22ab8c0)
> #21 thread_proxy <null> (impalad+0x2f7ccc9)
> Thread T126 (tid=20900, running) created by main thread at:
> #0 pthread_create
> /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889
> (impalad+0x18b4c1b)
> #1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9)
> #2 boost::thread::thread<void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string,
> boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long,
> (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string,
> boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long,
> (impala::PromiseMode)0>*)
> /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13
> (impalad+0x22a62bb)
> #3 impala::Thread::StartThread(std::string const&, std::string const&,
> boost::function<void ()> const&, std::unique_ptr<impala::Thread,
> std::default_delete<impala::Thread> >*, bool)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13
> (impalad+0x22a35a5)
> #4 impala::Status impala::Thread::Create<void
> (impala::ImpalaServicePool::*)(), impala::ImpalaServicePool*>(std::string
> const&, std::string const&, void (impala::ImpalaServicePool::* const&)(),
> impala::ImpalaServicePool* const&, std::unique_ptr<impala::Thread,
> std::default_delete<impala::Thread> >*, bool)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:81:12
> (impalad+0x1e1ba47)
> #5 impala::ImpalaServicePool::Init(int)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:91:5
> (impalad+0x1e19725)
> #6 impala::RpcMgr::RegisterService(int, int,
> kudu::rpc::GeneratedServiceIf*, impala::MemTracker*)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/rpc-mgr.cc:163:3
> (impalad+0x1e08ba4)
> #7 impala::DataStreamService::Init()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:78:3
> (impalad+0x2059b2f)
> #8 impala::ExecEnv::Init()
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/exec-env.cc:391:3
> (impalad+0x1e50481)
> #9 ImpaladMain(int, char**)
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/impalad-main.cc:72:3
> (impalad+0x2075af7)
> #10 main
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/daemon-main.cc:37:12
> (impalad+0x191f96a)SUMMARY: ThreadSanitizer: data race
> /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53
> in impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int)
> const{code}
> Notice that part of the stack right before {{BloomFilter::BucketFindAVX2}} is
> missing. I think because the Bloom Filter functions calls are codegened.
> AFAICT, the stack should be something like:
> {{HdfsParquetScanner::AssembleRows}} ->
> {{HdfsParquetScanner::TransferScratchTuples}} ->
> {{codegend_process_scratch_batch_fn_}} -> {{FilterContext::Eval}} ->
> {{RuntimeFilter::Eval}} -> {{RuntimeFilter::Eval}} -> {{BloomFilter::Find}}
> I haven't been able to successfully get codegen to run with
> {{-fsanitize=thread}} yet, which is why part of the stack is missing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
