mridulm commented on a change in pull request #31876:
URL: https://github.com/apache/spark/pull/31876#discussion_r617229751
##########
File path: core/src/main/scala/org/apache/spark/TaskEndReason.scala
##########
@@ -81,7 +81,7 @@ case object Resubmitted extends TaskFailedReason {
*/
@DeveloperApi
case class FetchFailed(
- bmAddress: BlockManagerId, // Note that bmAddress can be null
+ bmAddress: Location, // Note that bmAddress can be null
Review comment:
There are two changes here: a) change to public api (programmatic, REST
api, etc), b) change to generated/persisted data.
Both need to be addressed.
Specifically about (b), ​with `bmAddress` changing to `Location` - since it
need not be a `BlockManagerId` anymore - json serde needs to account for it.
Currently, it is a TODO
[here](https://github.com/apache/spark/pull/31876/files#diff-d87296ea32a40cc886384a0afaa3396ab8e68a5bbf9c05e399dc91fc86a7d7c4).
Spark history server, REST consumers, other apps depending on event files -
will need a way to identify Location type and serde all valid Location types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
