[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-30 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Resolution: Done
Status: Resolved  (was: Patch Available)

Investigation is done. Problem is fixed via HIVE-8920.

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch, 
 HIVE-9084.2-spark.patch, HIVE-9084.3-spark.patch, HIVE-9084.4-spark.patch, 
 HIVE-9084.4-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-15 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.3-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch, 
 HIVE-9084.2-spark.patch, HIVE-9084.3-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-15 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.4-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch, 
 HIVE-9084.2-spark.patch, HIVE-9084.3-spark.patch, HIVE-9084.4-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-15 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.4-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch, 
 HIVE-9084.2-spark.patch, HIVE-9084.3-spark.patch, HIVE-9084.4-spark.patch, 
 HIVE-9084.4-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-13 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.2-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch, 
 HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-12 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.2-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.1-spark.patch, 
 HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-12 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.2-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.1-spark.patch, 
 HIVE-9084.2-spark.patch, HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-12 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.2-spark.patch

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.1-spark.patch, 
 HIVE-9084.2-spark.patch, HIVE-9084.2-spark.patch, HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-12 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: (was: HIVE-9084.2-spark.patch)

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-12 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: (was: HIVE-9084.1-spark.patch)

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-12 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: (was: HIVE-9084.2-spark.patch)

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.2-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-11 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Issue Type: Sub-task  (was: Bug)
Parent: HIVE-7292

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-11 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Description: 
In recent ptest run (Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
 test groupby_multi_single_reducer.q failed w/ the following stacktrace:

{code}
java.lang.RuntimeException: Map operator initialization failed
at 
org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
at 
org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
at 
org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
at 
org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
at 
org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
at 
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at 
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
at 
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:56)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
at 
org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
... 16 more
{code}

This failure is again about IOContext object, which needs further investigation.





 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang

 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-11 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.1-spark.patch

Patch #1 is for experimental only. It changed the the HashMap to 
ConcurrentHashMap plus a few messages for debugging.

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-11 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Status: Patch Available  (was: Open)

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9084) Investigate IOContext object initialization problem [Spark Branch]

2014-12-11 Thread Xuefu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang updated HIVE-9084:
--
Attachment: HIVE-9084.1-spark.patch

Not sure why ptest didn't kick in. Reattach the same patch to retry.

 Investigate IOContext object initialization problem [Spark Branch]
 --

 Key: HIVE-9084
 URL: https://issues.apache.org/jira/browse/HIVE-9084
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
 Attachments: HIVE-9084.1-spark.patch, HIVE-9084.1-spark.patch


 In recent ptest run (Test results: 
 http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/511/testReport),
  test groupby_multi_single_reducer.q failed w/ the following stacktrace:
 {code}
 java.lang.RuntimeException: Map operator initialization failed
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:136)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:54)
   at 
 org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:29)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at 
 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:167)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:601)
   at 
 org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
   at 
 org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
   at org.apache.spark.scheduler.Task.run(Task.scala:56)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.io.IOContext.copy(IOContext.java:119)
   at 
 org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:97)
   ... 16 more
 {code}
 This failure is again about IOContext object, which needs further 
 investigation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)