Miklos Szurap created HIVE-27414:
------------------------------------
Summary: HiveServer2 is not shut down properly in OOM situations
when HoS is used
Key: HIVE-27414
URL: https://issues.apache.org/jira/browse/HIVE-27414
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 3.0.0, 2.4.0, 3.1.0, 2.3.0, 2.2.0, 2.1.1, 2.0.0, 1.2.0
Reporter: Miklos Szurap
When an OOM happens, the HiveServer2.stop() is called from the oomHook. That
shuts down the HS2, however if the default execution engine is not "spark" but
the HiveOnSpark execution engine has been used by any session before, then the
check in the stop() is not properly evaluated and hence it does not shut down
the SparkSessionManagerImpl properly:
{code}
if (hiveConf != null &&
hiveConf.getVar(ConfVars.HIVE_EXECUTION_ENGINE).equals("spark")) {
try {
SparkSessionManagerImpl.getInstance().shutdown();
} catch(Exception ex) {
LOG.error("Spark session pool manager failed to stop during HiveServer2
shutdown.", ex);
}
}
{code}
This leaves back a "user" (non-daemon) thread related to the
SparkSessionManagerImpl, so the JVM does not exit, the HS2 is unresponsive to
user requests as the CLI service and thrift service has been stopped before.
Due to HIVE-24411 the OOM is not handled well, so even the
{{-XX:OnOutOfMemoryError}} flag is useless and the JVM is not killed.
The
[HiveServer2.stop()|https://github.com/apache/hive/blob/37a4fd3515fd4c54025b2f1e8c443914f33b5652/service/src/java/org/apache/hive/service/server/HiveServer2.java#L939-L945]
method should use a check like SparkSessionManagerImpl.isInited().
(this method does not exist - just gave as an example)
The master branch does not have SparkSessionManagerImpl anymore, this is
applicable only to the earlier versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
