Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22112#discussion_r215070653
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1513,37 +1513,34 @@ private[spark] class DAGScheduler(
}
}
+ def generateErrorMessage(stage: Stage): String = {
+ "A shuffle map stage with indeterminate output was
failed and retried. " +
+ s"However, Spark cannot rollback the $stage to
re-process the input data, " +
+ "and has to fail this job. Please eliminate the
determinism by checkpointing " +
--- End diff --
`determinism ` -> `indeterminism `
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
