This is an automated email from the ASF dual-hosted git repository.

jshao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-livy.git


The following commit(s) were added to refs/heads/master by this push:
     new a90f4fa  [LIVY-642] Fix a rare status happened in yarn cause SparkApp 
change into error state
a90f4fa is described below

commit a90f4fac8be27a38cc961c24043a494a739ff188
Author: wangzhefeng <wangzhef...@didichuxing.com>
AuthorDate: Fri Aug 30 11:53:03 2019 +0800

    [LIVY-642] Fix a rare status happened in yarn cause SparkApp change into 
error state
    
    some of batch session returned failed state while spark yarn application is 
succeeded. and in livy log,i find the reason:
    
    ```
    2019-08-23 05:32:48,096 ERROR [Logging.scala:56] - Unknown YARN state 
RUNNING for app application_1559632632227_47808044 with final status SUCCEEDED.
    ```
    
    it means yarn application state is RUNNING while final app status is 
SUCCEEDED, it's a correct status in yarn, means AM has not sync app state yet. 
and livy mapped this situatition as SparkApp.state.FAILED.
    So, i think we should map this state into SparkApp.state.RUNNING,waiting 
for AM sync.
    
    Author: wangzhefeng <wangzhef...@didichuxing.com>
    
    Closes #204 from Jassy1994/LIVY-642.
---
 server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala 
b/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala
index 06d00a0..9762032 100644
--- a/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala
+++ b/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala
@@ -217,7 +217,8 @@ class SparkYarnApp private[utils] (
            (YarnApplicationState.SUBMITTED, FinalApplicationStatus.UNDEFINED) |
            (YarnApplicationState.ACCEPTED, FinalApplicationStatus.UNDEFINED) =>
         SparkApp.State.STARTING
-      case (YarnApplicationState.RUNNING, FinalApplicationStatus.UNDEFINED) =>
+      case (YarnApplicationState.RUNNING, FinalApplicationStatus.UNDEFINED) |
+           (YarnApplicationState.RUNNING, FinalApplicationStatus.SUCCEEDED) =>
         SparkApp.State.RUNNING
       case (YarnApplicationState.FINISHED, FinalApplicationStatus.SUCCEEDED) =>
         SparkApp.State.FINISHED

Reply via email to