Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21577#discussion_r196163389
--- Diff:
core/src/main/scala/org/apache/spark/mapred/SparkHadoopMapRedUtil.scala ---
@@ -81,7 +81,7 @@ object SparkHadoopMapRedUtil extends Logging {
logInfo(message)
// We need to abort the task so that the driver can reschedule
new attempts, if necessary
committer.abortTask(mrTaskContext)
- throw new CommitDeniedException(message, stageId, splitId,
taskAttemptNumber)
+ throw new CommitDeniedException(message, ctx.stageId(), splitId,
ctx.attemptNumber())
--- End diff --
shall we also include stage attempt number in the exception?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
