Kevin Kim created SPARK-23492:
---------------------------------

             Summary: Application shows up as running in history server even 
when latest attempt has completed
                 Key: SPARK-23492
                 URL: https://issues.apache.org/jira/browse/SPARK-23492
             Project: Spark
          Issue Type: Bug
          Components: Spark Core, YARN
    Affects Versions: 2.2.0
            Reporter: Kevin Kim


Hi, the Spark history server API "/application?status=running" returns 
applications whose latest attempts have completed (see example output below). 
Spark code says that an application is considered "running" when any one of the 
attempts are not "completed". 
[https://github.com/apache/spark/blob/1cc34f3e58c92dd06545727e9d931008a1082bbf/core/src/main/scala/org/apache/spark/status/api/v1/ApplicationListResource.scala#L44]

In my case, attempt 1 is shown as incomplete, but the attempt has finished and 
is stamped with an endTime. Is this a bug in Spark history server?
{code:java}
"attempts" : [ {
"attemptId" : "2",
"startTime" : "2018-02-14T23:59:17.785GMT",
"endTime" : "2018-02-15T08:08:28.927GMT",
"lastUpdated" : "2018-02-15T08:08:28.949GMT",
"duration" : 29351142,
"sparkUser" : {omitted},
"completed" : true,
"startTimeEpoch" : 1518652757785,
"endTimeEpoch" : 1518682108927,
"lastUpdatedEpoch" : 1518682108949
}, {
"attemptId" : "1",
"startTime" : "2018-02-14T23:53:02.629GMT",
"endTime" : "2018-02-14T23:59:13.426GMT",
"lastUpdated" : "2018-02-23T05:03:45.434GMT",
"duration" : 370797,
"sparkUser" : {omitted},
"completed" : false,
"startTimeEpoch" : 1518652382629,
"endTimeEpoch" : 1518652753426,
"lastUpdatedEpoch" : 1519362225434
} ]
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to