[
https://issues.apache.org/jira/browse/SPARK-4471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or updated SPARK-4471:
-----------------------------
Assignee: SuYan
> blockManagerIdFromJson function throws exception while BlockManagerId be
> null in MetadataFetchFailedException
> --------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4471
> URL: https://issues.apache.org/jira/browse/SPARK-4471
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.1.0, 1.1.1
> Reporter: SuYan
> Assignee: SuYan
> Priority: Minor
>
> 1. when throws MetadataFetchFailedException, it will create a
> FetchFailedException, which BlockManagerId is null.
> FetchFailedException(null, shuffleId, -1, reduceId)
> 2. JsonProtocol.TaskEndReasonToJson, mark BlockManagerId as JNothing
> 3. while use JsonProtocol.TaskEndReasonFromJson, it didn't take JNothing
> into account, so BlockManagerIdFromJson will throw exception
> error log belows:
> org.json4s.package$MappingException: Did not find value which can be
> converted into java.lang.String
> at org.json4s.reflect.package$.fail(package.scala:96)
> at org.json4s.Extraction$.convert(Extraction.scala:554)
> at org.json4s.Extraction$.extract(Extraction.scala:331)
> at org.json4s.Extraction$.extract(Extraction.scala:42)
> at
> org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
> at
> org.apache.spark.util.JsonProtocol$.blockManagerIdFromJson(JsonProtocol.scala:662)
> at
> org.apache.spark.util.JsonProtocol$.taskEndReasonFromJson(JsonProtocol.scala:643)
> at
> org.apache.spark.util.JsonProtocol$.taskEndFromJson(JsonProtocol.scala:450)
> at
> org.apache.spark.util.JsonProtocol$.sparkEventFromJson(JsonProtocol.scala:410)
> at
> org.apache.spark.scheduler.ReplayListenerBus$$anonfun$replay$2$$anonfun$apply$1.apply(ReplayListenerBus.scala:71)
> at
> org.apache.spark.scheduler.ReplayListenerBus$$anonfun$replay$2$$anonfun$apply$1.apply(ReplayListenerBus.scala:69)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
