Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16839 )
Change subject: [client] make metacache reset safe ...................................................................... Patch Set 1: (1 comment) > Thanks for finding and fixing this! Thank you for fast review! http://gerrit.cloudera.org:8080/#/c/16839/1//COMMIT_MSG Commit Message: PS1: I also verified that the TxnManagerTest.BeginManyTransactions scenario no longer fails with similar warning once this fix applied. Before: http://dist-test.cloudera.org//job?job_id=aserbin.1607497251.54221 After: http://dist-test.cloudera.org//job?job_id=aserbin.1607497958.62002 The relevant part of the ASAN reports for TxnManagerTest.BeginManyTransactions before the fix was: AddressSanitizer: heap-use-after-free on address 0x606000569fd8 at pc 0x0000004d6d8a bp 0x7f83c62f27d0 sp 0x7f83c62f1f80 READ of size 1342177400 at 0x606000569fd8 thread T168 (rpc reactor-136) #0 0x4d6d89 in __asan_memcpy thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cc:22:3273 #1 0x52f96d in std::char_traits<char>::copy(char*, char const*, unsigned long) ../../../include/c++/8/bits/char_traits.h:352:33 #2 0x52f6eb in std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) ../../../include/c++/8/bits/basic_string.tcc:1081:2 #3 0x52f50d in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&) ../../../include/c++/8/bits/basic_string.tcc:611:35 #4 0x7f8429d95347 in kudu::client::internal::RemoteTabletServer::ToString() const src/kudu/client/meta_cache.cc:225:16 #5 0x7f842ace83db in kudu::rpc::RetriableRpc<kudu::client::internal::RemoteTabletServer, kudu::tserver::CoordinateTransactionRequestPB, kudu::tserver::CoordinateTransactionResponsePB>::SendRpcCb(kudu::Status const&) src/kudu/rpc/retriable_rpc.h:323:85 #6 0x7f842acf2400 in kudu::rpc::RetriableRpc<kudu::client::internal::RemoteTabletServer, kudu::tserver::CoordinateTransactionRequestPB, kudu::tserver::CoordinateTransactionResponsePB>::ReplicaFoundCb(kudu::Status const&, kudu::client::internal::RemoteTabletServer*)::'lambda'()::operator()() const src/kudu/rpc/retriable_rpc.h:307:32 #7 0x7f8421245a30 in kudu::rpc::OutboundCall::CallCallback() src/kudu/rpc/outbound_call.cc:274:5 #8 0x7f8421245f00 in kudu::rpc::OutboundCall::SetResponse(std::unique_ptr<kudu::rpc::CallResponse, std::default_delete<kudu::rpc::CallResponse> >) src/kudu/rpc/outbound_call.cc:306:5 #9 0x7f84211f336e in kudu::rpc::Connection::HandleCallResponse(std::unique_ptr<kudu::rpc::InboundTransfer, std::default_delete<kudu::rpc::InboundTransfer> >) src/kudu/rpc/connection.cc:735:14 0x606000569ff9 is located 0 bytes to the right of 57-byte region [0x606000569fc0,0x606000569ff9) freed by thread T70 (rpc worker-1265) here: #0 0x517650 in operator delete(void*) thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cc:160:399 #1 0x531aee in std::string::_Rep::_M_destroy(std::allocator<char> const&) ../../../include/c++/8/bits/basic_string.tcc:899:29 #2 0x529dfe in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() ../../../include/c++/8/bits/basic_string.h:3640:19 #3 0x7f8429db5bd3 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::string const, kudu::client::internal::RemoteTabletServer*>, true> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<std::string const, kudu::client::internal::RemoteTabletServer*>, true>*) ../../../include/c++/8/bits/hashtable_policy.h:2100:7 #4 0x7f8429db5ae5 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::string const, kudu::client::internal::RemoteTabletServer*>, true> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<std::string const, kudu::client::internal::RemoteTabletServer*>, true>*) ../../../include/c++/8/bits/hashtable_policy.h:2113:4 #5 0x7f8429db5961 in std::_Hashtable<std::string, std::pair<std::string const, kudu::client::internal::RemoteTabletServer*>, std::allocator<std::pair<std::string const, kudu::client::internal::RemoteTabletServer*> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::clear() ../../../include/c++/8/bits/hashtable.h:2050:13 #6 0x7f8429da8188 in kudu::client::internal::MetaCache::ClearCache() src/kudu/client/meta_cache.cc:1257:3 #7 0x7f8429cb408d in kudu::client::KuduTableAlterer::Alter() src/kudu/client/client.cc:1487:41 #8 0x7f842ad3679e in kudu::transactions::TxnSystemClient::AddTxnStatusTableRangeWithClient(long, long, kudu::client::KuduClient*) src/kudu/transactions/txn_system_client.cc:119:9 #9 0x7f842f0b87e0 in kudu::transactions::TxnSystemClient::AddTxnStatusTableRange(long, long) src/kudu/transactions/txn_system_client.h:75:12 #10 0x7f842f0b628a in kudu::transactions::TxnManager::BeginTransaction(std::string const&, kudu::MonoTime const&, long*, unsigned int*) src/kudu/master/txn_manager.cc:187:48 #11 0x7f842f0b999d in kudu::transactions::TxnManagerServiceImpl::BeginTransaction(kudu::transactions::BeginTransactionRequestPB const*, kudu::transactions::BeginTransactionResponsePB*, kudu::rpc::RpcContext*) src/kudu/master/txn_manager_service.cc:75:42 previously allocated by thread T166 (rpc reactor-136) here: #0 0x516c98 in operator new(unsigned long) thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cc:99:386 #1 0x52f7a8 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) ../../../include/c++/8/bits/basic_string.tcc:1057:49 #2 0x5328d2 in std::string::_M_mutate(unsigned long, unsigned long, unsigned long) ../../../include/c++/8/bits/basic_string.tcc:929:16 #3 0x53281e in std::string::_M_replace_safe(unsigned long, unsigned long, char const*, unsigned long) ../../../include/c++/8/bits/basic_string.tcc:1134:7 #4 0x7f841c621af8 in google::protobuf::internal::EpsCopyInputStream::ReadString(char const*, int, std::string*) thirdparty/src/protobuf-3.14.0/src/google/protobuf/parse_context.h:156:26 #5 0x7f841c621af8 in google::protobuf::internal::InlineGreedyStringParser(std::string*, char const*, google::protobuf::internal::ParseContext*) thirdparty/src/protobuf-3.14.0/src/google/protobuf/parse_context.cc:441:25 #6 0x7f84292a9a7a in kudu::master::TSInfoPB::_InternalParse(char const*, google::protobuf::internal::ParseContext*) build/coordinator-retry.asan/src/kudu/master/master.pb.cc:10257:17 #7 0x7f842937253e in char const* google::protobuf::internal::ParseContext::ParseMessage<kudu::master::TSInfoPB>(kudu::master::TSInfoPB*, char const*) thirdparty/installed/uninstrumented/include/google/protobuf/parse_context.h:647:14 #8 0x7f84292e0693 in kudu::master::GetTableLocationsResponsePB::_InternalParse(char const*, google::protobuf::internal::ParseContext*) build/coordinator-retry.asan/src/kudu/master/master.pb.cc:15172:24 SUMMARY: AddressSanitizer: heap-use-after-free thirdparty/src/llvm-9.0.0.src/projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cc:22:3273 in __asan_memcpy Shadow bytes around the buggy address: 0x0c0c800a53a0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c800a53b0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd 0x0c0c800a53c0: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c800a53d0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c800a53e0: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd =>0x0c0c800a53f0: fd fd fd fd fa fa fa fa fd fd fd[fd]fd fd fd fd 0x0c0c800a5400: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c800a5410: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd 0x0c0c800a5420: fd fd fd fd fa fa fa fa fa fa fa fa fa fa fa fa 0x0c0c800a5430: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa 0x0c0c800a5440: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa -- To view, visit http://gerrit.cloudera.org:8080/16839 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03ec9318526fbfc2da9b068eb3bbd9cd996efbca Gerrit-Change-Number: 16839 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 09 Dec 2020 07:23:09 +0000 Gerrit-HasComments: Yes
