[
https://issues.apache.org/jira/browse/HIVE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201649#comment-14201649
]
Chengxiang Li commented on HIVE-8780:
-------------------------------------
Hi, [~jxiang], SparkJobMonitor check job state in an loop until job
finished/failed, and sleep check interval time after checking, so it should be
normal to get this stacktrace, SparkJobMonitor should not block spark job
execution as the job is submitted asynchronous before we call
SparkJobMonitor::startMonitor. could you test with a single qtest file and
upload a full JVM stacktrace when it's long enough to be sure job is hanging?
> insert1.q and ppd_join4.q hangs with hadoop-1 [Spark Branch]
> ------------------------------------------------------------
>
> Key: HIVE-8780
> URL: https://issues.apache.org/jira/browse/HIVE-8780
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Jimmy Xiang
>
> In working on HIVE-8758, found these tests hang at
> {noformat}
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor.startMoni
> tor(SparkJobMonitor.java:129)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java
> :111)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:161)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.ja
> va:85)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1644)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1216)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:2
> 47)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:345)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:832)
> at
> org.apache.hadoop.hive.cli.TestSparkCliDriver.runTest(TestSparkCliDri
> ver.java:3706)
> at
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ppd_join4
> (TestSparkCliDriver.java:2790)
> {noformat}
> Both tests hang at the same place. There could be other hanging tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
