[
https://issues.apache.org/jira/browse/HIVE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chengxiang Li updated HIVE-8780:
--------------------------------
Attachment: 8780.1-spark.patch
Spark job with empty source data would not be submmited to spark cluster
actually, so no JobEnd/JobStart event are posted to Spark listener bus. As Hive
monitor spark job state by spark listener, it would never get any job state, we
use JavaFutureAction to fetch job state in this patch.
> insert1.q and ppd_join4.q hangs with hadoop-1 [Spark Branch]
> ------------------------------------------------------------
>
> Key: HIVE-8780
> URL: https://issues.apache.org/jira/browse/HIVE-8780
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Jimmy Xiang
> Attachments: HIVE-8780.1-spark.patch, insert1.q-spark.png,
> insert1.q.jstack, itests.patch
>
>
> In working on HIVE-8758, found these tests hang at
> {noformat}
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor.startMoni
> tor(SparkJobMonitor.java:129)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java
> :111)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:161)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.ja
> va:85)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1644)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1216)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:2
> 47)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:345)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:832)
> at
> org.apache.hadoop.hive.cli.TestSparkCliDriver.runTest(TestSparkCliDri
> ver.java:3706)
> at
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ppd_join4
> (TestSparkCliDriver.java:2790)
> {noformat}
> Both tests hang at the same place. There could be other hanging tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
