[
https://issues.apache.org/jira/browse/EAGLE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huizhi Lu updated EAGLE-492:
----------------------------
Description:
1)
It is because the app was killed or terminated unexpectedly without completion.
In the log file, it has only SparkListenerBlockManagerAdded timestamp.
"Timestamp":1470768362299
And the code initiated executor starttime with this timestamp.
No executor or app endtime available, so executor endtime = 0L.
Finally, totalExecutionTime = -1470768362299
The code logic does not consider the situation when executor or app Endtime =
0L.
Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed
and not completed.
was:
It is because the app was killed or terminated unexpectedly without completion.
In the log file, it has only SparkListenerBlockManagerAdded timestamp.
"Timestamp":1470768362299
And the code initiated executor starttime with this timestamp.
No executor or app endtime available, so executor endtime = 0L.
Finally, totalExecutionTime = -1470768362299
The code logic does not consider the situation when executor or app Endtime =
0L.
Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed
and not completed.
> Fix negative total execution time and make some code optimization for spark
> history job.
> ----------------------------------------------------------------------------------------
>
> Key: EAGLE-492
> URL: https://issues.apache.org/jira/browse/EAGLE-492
> Project: Eagle
> Issue Type: Bug
> Reporter: Huizhi Lu
> Assignee: Huizhi Lu
>
> 1)
> It is because the app was killed or terminated unexpectedly without
> completion. In the log file, it has only SparkListenerBlockManagerAdded
> timestamp. "Timestamp":1470768362299
> And the code initiated executor starttime with this timestamp.
> No executor or app endtime available, so executor endtime = 0L.
> Finally, totalExecutionTime = -1470768362299
> The code logic does not consider the situation when executor or app Endtime =
> 0L.
> Solution: If totalExecutionTime < 0L, set it = 0L. Stating the app is killed
> and not completed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
