[
https://issues.apache.org/jira/browse/SPARK-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011171#comment-15011171
]
Jean-Baptiste Onofré commented on SPARK-11189:
----------------------------------------------
I confirm, it has been fixed by:
{code}
[SPARK-6197][CORE] handle json exception when hisotry file not finished writing
For details, please refer to
[SPARK-6197](https://issues.apache.org/jira/browse/SPARK-6197)
Author: Zhang, Liye <liye.zh...@intel.com>
Closes #4927 from liyezhang556520/jsonParseError and squashes the following
commits: 5cbdc82 [Zhang, Liye] without unnecessary wrap 2b48831 [Zhang, Liye]
small changes with sean owen's comments 2973024 [Zhang, Liye] handle json
exception when file not finished writing
{code}
> History server is not able to parse some application report
> -----------------------------------------------------------
>
> Key: SPARK-11189
> URL: https://issues.apache.org/jira/browse/SPARK-11189
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.6.0
> Reporter: Jean-Baptiste Onofré
>
> In some case, history server is not able to parse an application report.
> For instance, with JavaTC example:
> {code}
> com.fasterxml.jackson.core.JsonParseException: Unexpected end-of-input: was
> expecting closing '"' for name
> at [Source: {"Event":"SparkListenerTaskEnd","Stage ID":245,"Stage Attempt
> ID":0,"Task Type":"ShuffleMapTask","Task End Reason":{"Rea; line: 1, column:
> 241]
> at
> com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419)
> at
> com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
> at
> com.fasterxml.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:445)
> at
> com.fasterxml.jackson.core.json.ReaderBasedJsonParser._parseName2(ReaderBasedJsonParser.java:1284)
> at
> com.fasterxml.jackson.core.json.ReaderBasedJsonParser._parseName(ReaderBasedJsonParser.java:1268)
> at
> com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:618)
> at
> org.json4s.jackson.JValueDeserializer.deserialize(JValueDeserializer.scala:34)
> at
> org.json4s.jackson.JValueDeserializer.deserialize(JValueDeserializer.scala:42)
> at
> org.json4s.jackson.JValueDeserializer.deserialize(JValueDeserializer.scala:35)
> at
> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3066)
> at
> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2161)
> at org.json4s.jackson.JsonMethods$class.parse(JsonMethods.scala:19)
> at org.json4s.jackson.JsonMethods$.parse(JsonMethods.scala:44)
> at
> org.apache.spark.scheduler.ReplayListenerBus.replay(ReplayListenerBus.scala:58)
> at
> org.apache.spark.deploy.master.Master.rebuildSparkUI(Master.scala:950)
> at
> org.apache.spark.deploy.master.Master.removeApplication(Master.scala:812)
> at
> org.apache.spark.deploy.master.Master.org$apache$spark$deploy$master$Master$$finishApplication(Master.scala:790)
> at
> org.apache.spark.deploy.master.Master$$anonfun$receive$1$$anonfun$applyOrElse$21.apply(Master.scala:382)
> at
> org.apache.spark.deploy.master.Master$$anonfun$receive$1$$anonfun$applyOrElse$21.apply(Master.scala:382)
> at scala.Option.foreach(Option.scala:236)
> at
> org.apache.spark.deploy.master.Master$$anonfun$receive$1.applyOrElse(Master.scala:382)
> at
> org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:105)
> at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:206)
> at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:99)
> at
> org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:224)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
