[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-21 Thread zuotingbing 
Github user zuotingbing commented on the issue:

https://github.com/apache/spark/pull/19989
  
OK , please move to https://github.com/apache/spark/pull/20029. Thanks all.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-17 Thread zuotingbing 
Github user zuotingbing commented on the issue:

https://github.com/apache/spark/pull/19989
  
we can find the cache size of FileSystem `deleteOnExit` will keep 
increasing.
[

![mshot](https://user-images.githubusercontent.com/24823338/34095036-d1fcc408-e40a-11e7-9599-2acdd96da2d9.png)
](url)


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-17 Thread zuotingbing 
Github user zuotingbing commented on the issue:

https://github.com/apache/spark/pull/19989
  
as i debug, every time when i connect to thrift server through beeline, the 
`SessionState.start(state)` will be called two times. one is in 
`HiveSessionImpl:open` , **another is in `HiveClientImpl` for sql `use 
default`** .
SessionManager.java#L151 or `HiveSessionImpl:close` only to clean the first.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-16 Thread liufengdb 
Github user liufengdb commented on the issue:

https://github.com/apache/spark/pull/19989
  
I think this method can take care of resource clean up automatically: 
https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/SessionManager.java#L151

Can you really make a heap dump and find out why the sessions are not 
cleaned up? 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-16 Thread gatorsmile 
Github user gatorsmile commented on the issue:

https://github.com/apache/spark/pull/19989
  
cc @liufengdb 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-15 Thread mgaido91 
Github user mgaido91 commented on the issue:

https://github.com/apache/spark/pull/19989
  
I am not sure about this change actually. In this way all the users would 
use the same `metadataHive`. This might have also concurrency issue. Did you 
experienced a OOM error due to the memory leak?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-15 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19989
  
Can one of the admins verify this patch?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-15 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19989
  
Can one of the admins verify this patch?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org