[
https://issues.apache.org/jira/browse/HIVE-28545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-28545:
-------------------------------
Description:
We add the global lock on HMSHandler.getMSForConf by HIVE-27117, which is aim
to address the flaky test:
TestCompactionMetrics.testInitiatorFailuresCountedCorrectly, however this
global lock could contribute to another deadlock problem. Imaging this, if
there is no idle connection in the connection pool, and HMSHandler needs getMS
to commit or. rollback the transaction, a new client chimes in and takes over
this global lock to initialize the ObjectStore, the initialization will be
pending as there is no available connection, which prevents others to get the
global lock to free the occupied connection, I wrote a test to demonstrate this
case.
This global lock could result to performance penalty which can block other
clients in concurrent workloads. In worst cases, it can make other clients read
timeout.
was:
We add the global lock on HMSHandler.getMSForConf by HIVE-27117, which is aim
to address the flaky test:
TestCompactionMetrics.testInitiatorFailuresCountedCorrectly, however this
global lock could contribute to another deadlock problem. Imaging this, if
there is no idle connection in the connection pool, HMSHandler needs getMS to
commit or. rollback the transaction, a new client chimes in and takes over this
global lock to initialize the ObjectStore, the initialization will be pending
as there is no available connection, which prevents others to get the global
lock to free the occupied connection, I wrote a test to demonstrate this case.
This global lock could result to performance penalty which can block other
clients in concurrent workloads. In worst cases, it can make other clients read
timeout.
> Remove global lock on HMSHandler.getMSForConf which is prone to deadlock
> ------------------------------------------------------------------------
>
> Key: HIVE-28545
> URL: https://issues.apache.org/jira/browse/HIVE-28545
> Project: Hive
> Issue Type: Bug
> Security Level: Public(Viewable by anyone)
> Components: Standalone Metastore
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Major
> Fix For: 4.1.0
>
>
> We add the global lock on HMSHandler.getMSForConf by HIVE-27117, which is aim
> to address the flaky test:
> TestCompactionMetrics.testInitiatorFailuresCountedCorrectly, however this
> global lock could contribute to another deadlock problem. Imaging this, if
> there is no idle connection in the connection pool, and HMSHandler needs
> getMS to commit or. rollback the transaction, a new client chimes in and
> takes over this global lock to initialize the ObjectStore, the initialization
> will be pending as there is no available connection, which prevents others to
> get the global lock to free the occupied connection, I wrote a test to
> demonstrate this case.
> This global lock could result to performance penalty which can block other
> clients in concurrent workloads. In worst cases, it can make other clients
> read timeout.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
