[
https://issues.apache.org/jira/browse/SPARK-22793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zuotingbing updated SPARK-22793:
--------------------------------
Affects Version/s: 2.2.1
> Memory leak in Spark Thrift Server
> ----------------------------------
>
> Key: SPARK-22793
> URL: https://issues.apache.org/jira/browse/SPARK-22793
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.2, 2.2.1
> Reporter: zuotingbing
> Priority: Critical
>
> 1. Start HiveThriftServer2.
> 2. Connect to thriftserver through beeline.
> 3. Close the beeline.
> 4. repeat step2 and step 3 for several times, which caused the leak of Memory.
> we found there are many directories never be dropped under the path
> {code:java}
> hive.exec.local.scratchdir
> {code} and
> {code:java}
> hive.exec.scratchdir
> {code} , as we know the scratchdir has been added to deleteOnExit when it be
> created. So it means that the cache size of FileSystem deleteOnExit will keep
> increasing until JVM terminated.
> In addition, we use
> {code:java}
> jmap -histo:live [PID]
> {code} to printout the size of objects in HiveThriftServer2 Process, we can
> find the object "org.apache.spark.sql.hive.client.HiveClientImpl" and
> "org.apache.hadoop.hive.ql.session.SessionState" keep increasing even though
> we closed all the beeline connections, which caused the leak of Memory.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
