[
https://issues.apache.org/jira/browse/HBASE-25581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell resolved HBASE-25581.
-----------------------------------------
Resolution: Incomplete
> BUG - HBase native client memory leak - threads keep running after connection
> is closed
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-25581
> URL: https://issues.apache.org/jira/browse/HBASE-25581
> Project: HBase
> Issue Type: Sub-task
> Reporter: Julia Emelianova
> Priority: Major
>
> Hello everyone,
> First of all, thank you for creating such an amazing product
> I'm trying to implement an open-source PHP-extension based on HBase native
> client. It seems more promising performance-wise than Thrifts we're currently
> using
> In terms of PHP-fpm, we need to be able to open and close connection multiple
> times in one process, and here's when the library leaks memory
> Calling 'Close' methods on Table and Clients is not enough for closing all of
> the threads.
> A simple program
>
> {code:java}
> int main() {
> zoo_set_debug_level( (ZooLogLevel)4);
> google::SetCommandLineOption("GLOG_minloglevel", "0");
> std::shared_ptr<hbase::Configuration> conf =
> std::make_shared<hbase::Configuration>();
> conf->Set(hbase::ZKUtil::kHBaseZookeeperQuorum_, "quorum");
> conf->Set(hbase::ZKUtil::kHBaseZookeeperClientPort_, "2181");
> conf->Set(hbase::ZKUtil::kHBaseZnodeParent_, "/hbase-unsecure");
> createClient(conf);
> int i = 0;
> while (true) {
> i++;
> std::cout << "Slept " << i << std::endl;
> sleep(5);
> }
> return 0;
> }
> void createClient(const std::shared_ptr<hbase::Configuration>& conf)
> {
> std::cout << "opening " << std::endl << std::endl << std::endl;
> auto tn =
> std::make_shared<hbase::pb::TableName>(folly::to<hbase::pb::TableName>("table"));
> auto client = std::make_unique<hbase::Client>(*conf);
> auto table = client->Table(*tn);
> std::cout << "closing " << std::endl << std::endl << std::endl;
> table->Close();
> client->Close();
> }
> {code}
>
> Logs :
> {code:java}
> [julia@fdf346c7aa2b myhbase]$ ./myhbase
> opening
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0216 08:54:44.391023 5259 location-cache.cc:77] Connecting to ZooKeeper.
> Quorum:quorum
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@726: Client
> environment:zookeeper.version=zookeeper C client 3.4.8
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@730: Client
> environment:host.name=fdf346c7aa2b
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@737: Client
> environment:os.name=Linux
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@738: Client
> environment:os.arch=5.8.1-1.el7.elrepo.x86_64
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@739: Client
> environment:os.version=#1 SMP Tue Aug 11 12:01:11 EDT 2020
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@747: Client
> environment:user.name=(null)
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@755: Client
> environment:user.home=/home/julia
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@log_env@767: Client
> environment:user.dir=/usr/src/hbase/myhbase
> 2021-02-16 08:54:44,391:5259(0x7fa88b38d3c0):ZOO_INFO@zookeeper_init@800:
> Initiating client connection, host=quorum sessionTimeout=90000 watcher=(nil)
> sessionId=0 sessionPasswd=<null> context=(nil) flags=0
> 2021-02-16 08:54:44,400:5259(0x7fa88b38d3c0):ZOO_DEBUG@start_threads@221:
> starting threads...
> 2021-02-16 08:54:44,400:5259(0x7fa7a77fe700):ZOO_DEBUG@do_io@367: started IO
> thread
> 2021-02-16 08:54:44,400:5259(0x7fa7a6ffd700):ZOO_DEBUG@do_completion@459:
> started completion thread
> closing
> I0216 08:54:44.400823 5259 client.cc:56] called destructor on Client
> 2021-02-16 08:54:44,400:5259(0x7fa7a77fe700):ZOO_INFO@check_events@1728:
> initiated connection to server [192.168.197.12:2181]
> 2021-02-16 08:54:44,403:5259(0x7fa7a77fe700):ZOO_INFO@check_events@1775:
> session establishment complete on server [192.168.197.12:2181],
> sessionId=0x27640b8a9bd0932, negotiated timeout=60000
> 2021-02-16 08:54:44,403:5259(0x7fa7a77fe700):ZOO_DEBUG@check_events@1781:
> Calling a watcher for a ZOO_SESSION_EVENT and the state=ZOO_CONNECTED_STATE
> 2021-02-16
> 08:54:44,403:5259(0x7fa7a6ffd700):ZOO_DEBUG@process_completions@2132: Calling
> a watcher for node [], type = -1 event=ZOO_SESSION_EVENT
> I0216 08:54:44.403533 5259 table.cc:50] called destructor on Table
> Slept 1
> Slept 2
> Slept 3
> Slept 4
> Slept 5
> 2021-02-16
> 08:55:04,421:5259(0x7fa7a77fe700):ZOO_WARN@zookeeper_interest@1570: Exceeded
> deadline by 19ms
> 2021-02-16
> 08:55:04,421:5259(0x7fa7a77fe700):ZOO_DEBUG@zookeeper_process@2218: Got ping
> response in 0 ms
> Slept 6
> Slept 7
> Slept 8
> Slept 9
> 2021-02-16
> 08:55:24,436:5259(0x7fa7a77fe700):ZOO_WARN@zookeeper_interest@1570: Exceeded
> deadline by 14ms
> 2021-02-16
> 08:55:24,436:5259(0x7fa7a77fe700):ZOO_DEBUG@zookeeper_process@2218: Got ping
> response in 0 ms
> Slept 10
> Slept 11
> Slept 12
> {code}
> As you can see, after calling destructors on Table and Client, a connection
> to Zookeeper is still open, no other destructors are called.
> If I'm calling createClient() function multiple times, I can see consumed
> memory is growing and being released only after the program exits
>
> Best regards,
> Julia
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
