[
https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or updated SPARK-6197:
-----------------------------
Labels: backport-needed (was: )
> handle json parse exception for eventlog file not finished writing
> -------------------------------------------------------------------
>
> Key: SPARK-6197
> URL: https://issues.apache.org/jira/browse/SPARK-6197
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.3.0
> Reporter: Zhang, Liye
> Assignee: Zhang, Liye
> Priority: Minor
> Labels: backport-needed
> Fix For: 1.4.0
>
>
> This is a following JIRA for
> [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107]. In
> [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107], webUI can
> display event log files that with suffix *.inprogress*. However, the eventlog
> file may be not finished writing for some abnormal cases (e.g. Ctrl+C), In
> which case, the file maybe truncated in the last line, leading to the line
> being not in valid Json format. Which will cause Json parse exception when
> reading the file.
> For this case, we can just ignore the last line content, since the history
> for abnormal cases showed on web is only a reference for user, it can
> demonstrate the past status of the app before terminated abnormally (we can
> not guarantee the history can show exactly the last moment when app encounter
> the abnormal situation).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
