Dilip Biswal created SPARK-23275:
------------------------------------
Summary: hive/tests have been failing when run locally on the
laptop (Mac) with OOM
Key: SPARK-23275
URL: https://issues.apache.org/jira/browse/SPARK-23275
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.3.1
Reporter: Dilip Biswal
hive tests have been failing when they are run locally (Mac Os) after a recent
change in the trunk. After running the tests for some time, the test fails with
OOM with Error: unable to create new native thread.
I noticed the thread count goes all the way up to 2000+ after which we start
getting these OOM errors. Most of the threads seem to be related to the
connection pool in hive metastore (BoneCP-xxxxx-xxxx ). This behaviour change
is happening after we made the following change to HiveClientImpl.reset()
{code}
def reset(): Unit = withHiveState {
try {
// code
} finally {
runSqlHive("USE default") ===> this is causing the issue
}
{code}
I am proposing to temporarily back-out part of a fix made to address
SPARK-23000 to resolve this issue while we work-out the exact reason for this
sudden increase in thread counts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
