Review Request 32288: HIVE-10006 RSC has memory leak while execute multi queries

2015-03-20 Thread chengxiang li

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32288/
---

Review request for hive.


Bugs: HIVE-10006
https://issues.apache.org/jira/browse/HIVE-10006


Repository: hive


Description
---

In RSC, while spark call CombineHiveInputFormat::getSplits to split the job 
into tasks in a thread called dag-scheduler-event-loop, MapWork would be 
added to a ThreadLocal map of dag-scheduler-event-loop, and never get 
removed. As the dag-scheduler-event-loop thread is a long live and daemon 
thread, so all the MapWorks would be hold in the ThreadLocal map until RSC jvm 
crash or exit.


Diffs
-

  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
 1667894 
  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java
 1667894 
  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryObjectInspectorFactory.java
 1667894 
  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/MetadataListStructObjectInspector.java
 1667894 
  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java
 1667894 
  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java
 1667894 
  
branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java
 1667894 

Diff: https://reviews.apache.org/r/32288/diff/


Testing
---


Thanks,

chengxiang li



Re: Review Request 32288: HIVE-10006 RSC has memory leak while execute multi queries

2015-03-20 Thread Xuefu Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32288/#review77193
---


Not sure if the diff is for a different bug.

- Xuefu Zhang


On March 20, 2015, 6:22 a.m., chengxiang li wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/32288/
 ---
 
 (Updated March 20, 2015, 6:22 a.m.)
 
 
 Review request for hive.
 
 
 Bugs: HIVE-10006
 https://issues.apache.org/jira/browse/HIVE-10006
 
 
 Repository: hive
 
 
 Description
 ---
 
 In RSC, while spark call CombineHiveInputFormat::getSplits to split the job 
 into tasks in a thread called dag-scheduler-event-loop, MapWork would be 
 added to a ThreadLocal map of dag-scheduler-event-loop, and never get 
 removed. As the dag-scheduler-event-loop thread is a long live and daemon 
 thread, so all the MapWorks would be hold in the ThreadLocal map until RSC 
 jvm crash or exit.
 
 
 Diffs
 -
 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
  1667894 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java
  1667894 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryObjectInspectorFactory.java
  1667894 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/MetadataListStructObjectInspector.java
  1667894 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java
  1667894 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java
  1667894 
   
 branches/spark/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.java
  1667894 
 
 Diff: https://reviews.apache.org/r/32288/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 chengxiang li