Qinghui Xu created HIVE-22306:
---------------------------------
Summary: Use nonblocking thrift server for metastore
Key: HIVE-22306
URL: https://issues.apache.org/jira/browse/HIVE-22306
Project: Hive
Issue Type: Improvement
Components: Metastore
Reporter: Qinghui Xu
Currently hive metastore's threads are blocking for network io (it's using
`TThreadPoolServer` behind the scene), which means with increasing use cases
(in our tech stack there are different services relying on it, hiveserver2,
spark, presto, and more, all with a significant number of users), to handle all
connections it needs either a big thread pool or many instances with smaller
thread pools. And often, those metastores will see their thread pool saturated,
while the cpu usage is still quite low, just because most connections stay idle
and only run a query from time to time. This is thus a great misuse of the
computation resources.
Thus I propose to use a non blocking threading model, and run computation
asynchronously.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
