Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14177 )
Change subject: KUDU-2780: create thread for auto-rebalancing ...................................................................... Patch Set 22: (1 comment) http://gerrit.cloudera.org:8080/#/c/14177/22/src/kudu/master/auto_rebalancer-test.cc File src/kudu/master/auto_rebalancer-test.cc: http://gerrit.cloudera.org:8080/#/c/14177/22/src/kudu/master/auto_rebalancer-test.cc@513 PS22, Line 513: ScopedRegisterSink reg(&capture_logs); : : // Bring up a new tserver. : ASSERT_OK(cluster_->AddTabletServer()); : : // The TSManager should still believe the original tservers are available, : // so the auto-rebalancer should attempt to schedule replica moves from those : // tservers to the new one. : NO_FATALS(CheckSomeMovesScheduled()); Based on the TSAN error, seems like we might want to have 'reg' go out of scope before reading 'capture_logs'. Read of size 8 at 0x7ffcce5306c8 by main thread: #0 std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::end() const /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/c++/v1/vector:1539:30 (auto_rebalancer-test+0x4fb40a) #1 kudu::master::AutoRebalancerTest_TestHandlingFailedTservers_Test::TestBody() /home/jenkins-slave/workspace/kudu-master/2/src/kudu/master/auto_rebalancer-test.cc:522:3 (auto_rebalancer-test+0x4f3826) #2 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x54f9f) #3 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438:14 (libgmock.so+0x54f9f) #4 testing::Test::Run() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474:5 (libgmock.so+0x33f38) #5 testing::TestInfo::Run() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656:11 (libgmock.so+0x351dc) #6 testing::TestCase::Run() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774:28 (libgmock.so+0x35ef6) #7 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649:43 (libgmock.so+0x423a6) #8 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402:10 (libgmock.so+0x55e4f) #9 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438:14 (libgmock.so+0x55e4f) #10 testing::UnitTest::Run() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257:10 (libgmock.so+0x41c61) #11 RUN_ALL_TESTS() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/gtest/gtest.h:2233:46 (libkudu_test_main.so+0x47bb) #12 main /home/jenkins-slave/workspace/kudu-master/2/src/kudu/util/test_main.cc:106:13 (libkudu_test_main.so+0x3636) Previous write of size 8 at 0x7ffcce5306c8 by thread T127 (mutexes: write M114, read M118): #0 std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >::push_back(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/c++/v1/vector:1660:9 (auto_rebalancer-test+0x50bc55) #1 kudu::StringVectorSink::send(int, char const*, char const*, int, tm const*, char const*, unsigned long) /home/jenkins-slave/workspace/kudu-master/2/src/kudu/util/logging_test_util.h:34:18 (auto_rebalancer-test+0x50ba29) #2 google::LogDestination::LogToSinks(int, char const*, char const*, int, tm const*, char const*, unsigned long) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/glog-0.3.5/src/logging.cc:790:21 (libglog.so.0+0x1675d) #3 google::LogMessage::SendToLog() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/glog-0.3.5/src/logging.cc:1379:5 (libglog.so.0+0xe1c2) #4 google::LogMessage::Flush() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/glog-0.3.5/src/logging.cc:1311:5 (libglog.so.0+0xf211) #5 google::LogMessage::~LogMessage() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/src/glog-0.3.5/src/logging.cc:1270:3 (libglog.so.0+0xefae) #6 kudu::tserver::Heartbeater::Thread::DoHeartbeat(kudu::master::MasterErrorPB*, kudu::rpc::ErrorStatusPB*) /home/jenkins-slave/workspace/kudu-master/2/src/kudu/tserver/heartbeater.cc:449:5 (libtserver.so+0xef4c0) #7 kudu::tserver::Heartbeater::Thread::RunThread() /home/jenkins-slave/workspace/kudu-master/2/src/kudu/tserver/heartbeater.cc:570:21 (libtserver.so+0xf06f1) #8 boost::_mfi::mf0<void, kudu::tserver::Heartbeater::Thread>::operator()(kudu::tserver::Heartbeater::Thread*) const /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/boost/bind/mem_fn_template.hpp:49:29 (libtserver.so+0x105db9) #9 void boost::_bi::list1<boost::_bi::value<kudu::tserver::Heartbeater::Thread*> >::operator()<boost::_mfi::mf0<void, kudu::tserver::Heartbeater::Thread>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, kudu::tserver::Heartbeater::Thread>&, boost::_bi::list0&, int) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/boost/bind/bind.hpp:259:9 (libtserver.so+0x105d0a) #10 boost::_bi::bind_t<void, boost::_mfi::mf0<void, kudu::tserver::Heartbeater::Thread>, boost::_bi::list1<boost::_bi::value<kudu::tserver::Heartbeater::Thread*> > >::operator()() /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/boost/bind/bind.hpp:1222:16 (libtserver.so+0x105c73) #11 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, kudu::tserver::Heartbeater::Thread>, boost::_bi::list1<boost::_bi::value<kudu::tserver::Heartbeater::Thread*> > >, void>::invoke(boost::detail::function::function_buffer&) /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/boost/function/function_template.hpp:159:11 (libtserver.so+0x105a29) #12 boost::function0<void>::operator()() const /home/jenkins-slave/workspace/kudu-master/2/thirdparty/installed/tsan/include/boost/function/function_template.hpp:770:14 (libkrpc.so+0xfce41) #13 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/kudu-master/2/src/kudu/util/thread.cc:675:3 (libkudu_util.so+0x2d263a) -- To view, visit http://gerrit.cloudera.org:8080/14177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifca25d1063c07047cf2123e6792b3c7395be20e4 Gerrit-Change-Number: 14177 Gerrit-PatchSet: 22 Gerrit-Owner: Hannah Nguyen <hannah.ngu...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Hannah Nguyen <hannah.ngu...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Sun, 15 Mar 2020 01:17:48 +0000 Gerrit-HasComments: Yes
