Alexey Serbin has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9574 )

Change subject: KUDU-2295 fix nullptr dereference in Tablet
......................................................................

KUDU-2295 fix nullptr dereference in Tablet

Prior to this patch, in the case of concurrent events of shutting
down a tablet and running a snapshot scan on it, sometimes the code
ended up trying to access already freed members of a Tablet object,
resulting in stack traces like below.

The traces were captured from modified raft_consensus_stress-itest
(added a single read thread for TestWorkload) runs using dist_test.

(after RYW changes):
PC: @     0x7f563b5e31cb std::atomic_bool::load()
*** SIGSEGV (@0x1f8) received by PID 19893 (TID 0x7f561ce82700) from PID 504; 
stack trace: ***
    @     0x7f5638713330 (unknown) at ??:0
    @     0x7f563b5e31cb std::atomic_bool::load() at ??:0
    @     0x7f563b609c31 kudu::tablet::MvccManager::is_open() at ??:0
    @     0x7f563b6085f3 kudu::tablet::MvccManager::CheckOpen() at ??:0
    @     0x7f563b607fc5 kudu::tablet::MvccManager::WaitUntil() at ??:0
    @     0x7f563b608938 
kudu::tablet::MvccManager::WaitForSnapshotWithAllCommitted() at ??:0
    @     0x7f563ca61b55 
kudu::tserver::TabletServiceImpl::HandleScanAtSnapshot() at ??:0
    @     0x7f563ca5c0e2 
kudu::tserver::TabletServiceImpl::HandleNewScanRequest() at ??:0
    @     0x7f563ca59793 kudu::tserver::TabletServiceImpl::Scan() at ??:0
    @     0x7f5637324e4d 
kudu::tserver::TabletServerServiceIf::TabletServerServiceIf()::$_5::operator()()
 at ??:0
    @     0x7f5637324c92 std::_Function_handler<>::_M_invoke() at ??:0
    @     0x7f563648992b std::function<>::operator()() at ??:0
    @     0x7f56364891ed kudu::rpc::GeneratedServiceIf::Handle() at ??:0
    @     0x7f563648b5e6 kudu::rpc::ServicePool::RunThread() at ??:0
    @     0x7f563648dc29 boost::_mfi::mf0<>::operator()() at ??:0
    @     0x7f563648db90 boost::_bi::list1<>::operator()<>() at ??:0
    @     0x7f563648db3a boost::_bi::bind_t<>::operator()() at ??:0
    @     0x7f563648d91d 
boost::detail::function::void_function_obj_invoker0<>::invoke() at ??:0
    @     0x7f5636430078 boost::function0<>::operator()() at ??:0
    @     0x7f563472c08d kudu::Thread::SuperviseThread() at ??:0
    @     0x7f563870b184 start_thread at ??:0
    @     0x7f5630a2affd clone at ??:0

(before RYW changes):
PC: @     0x7f1e02025790 scoped_refptr<>::operator->()
*** SIGSEGV (@0x160) received by PID 8782 (TID 0x7f1de3c7e700) from PID 352; 
stack trace: ***
    @     0x7f1dfdcfc330 (unknown) at ??:0
    @     0x7f1e02025790 scoped_refptr<>::operator->() at ??:0
    @     0x7f1e00ae62e7 kudu::tablet::Tablet::GetTabletAncientHistoryMark() at 
??:0
    @     0x7f1e00ae627d kudu::tablet::Tablet::GetHistoryGcOpts() at ??:0
    @     0x7f1e02012c53 kudu::tserver::(anonymous 
namespace)::VerifyNotAncientHistory() at ??:0
    @     0x7f1e0201223b 
kudu::tserver::TabletServiceImpl::HandleScanAtSnapshot() at ??:0
    @     0x7f1e0200c6dd 
kudu::tserver::TabletServiceImpl::HandleNewScanRequest() at ??:0
    @     0x7f1e02009d33 kudu::tserver::TabletServiceImpl::Scan() at ??:0
    @     0x7f1dfc90de4d 
kudu::tserver::TabletServerServiceIf::TabletServerServiceIf()::$_5::operator()()
 at ??:0
    @     0x7f1dfc90dc92 std::_Function_handler<>::_M_invoke() at ??:0
    @     0x7f1dfba728ab std::function<>::operator()() at ??:0
    @     0x7f1dfba7216d kudu::rpc::GeneratedServiceIf::Handle() at ??:0
    @     0x7f1dfba74526 kudu::rpc::ServicePool::RunThread() at ??:0
    @     0x7f1dfba76ad9 boost::_mfi::mf0<>::operator()() at ??:0
    @     0x7f1dfba76a40 boost::_bi::list1<>::operator()<>() at ??:0
    @     0x7f1dfba769ea boost::_bi::bind_t<>::operator()() at ??:0
    @     0x7f1dfba767cd 
boost::detail::function::void_function_obj_invoker0<>::invoke() at ??:0
    @     0x7f1dfba190f8 boost::function0<>::operator()() at ??:0
    @     0x7f1df9d1788d kudu::Thread::SuperviseThread() at ??:0
    @     0x7f1dfdcf4184 start_thread at ??:0
    @     0x7f1df6023ffd clone at ??:0
    @                0x0 (unknown)

Change-Id: Ib4b8cc2a307e5a0bd6019bd155f33c1565fca513
Reviewed-on: http://gerrit.cloudera.org:8080/9574
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <a...@cloudera.com>
Reviewed-by: Mike Percy <mpe...@apache.org>
---
M src/kudu/tserver/tablet_service.cc
M src/kudu/tserver/tablet_service.h
2 files changed, 15 insertions(+), 14 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, but someone else must approve
  Mike Percy: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/9574
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib4b8cc2a307e5a0bd6019bd155f33c1565fca513
Gerrit-Change-Number: 9574
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>

Reply via email to