GitHub user SaintBacchus opened a pull request:
https://github.com/apache/spark/pull/7239
[SPARK-8839][SQL]ThriftServer2 will remove session and execution no matter
it's finished or not.
[Code](https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala#L220)
in HiveThriftServer2 use `take` to get the element which will be removed.
In scala
[doc](http://www.scala-lang.org/api/2.10.4/#scala.collection.IterableLike)
`take` had a note:
> Note: might return different results for different runs, unless the
underlying collection type is ordered.
`take` does not take the first elements in the list, and it may remove some
session and execution which still will be used.
So add a check before removing it, but this solution will cause all the
unfinished execution keep in the memory.
[Jira Address](https://issues.apache.org/jira/browse/SPARK-8839)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SaintBacchus/spark SPARK-8839
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7239.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7239
----
commit 9d5ceb8f980830a4a6be6c09bacbae9f005f734d
Author: huangzhaowei <[email protected]>
Date: 2015-07-06T11:49:39Z
[SPARK-8839][SQL]ThriftServer2 will remove session and execution no matter
it's finished or not.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
