Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19848#discussion_r154137939
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ---
@@ -106,6 +106,12 @@ abstract class FileCommitProtocol {
*/
def commitTask(taskContext: TaskAttemptContext): TaskCommitMessage
+ /**
+ * Commits a task which blongs to a specific stage after the writes
succeed.
+ * Must be called on the executors when running tasks.
+ */
+ private[spark] def commitTask(taskContext: TaskAttemptContext, stageId:
Int): TaskCommitMessage
--- End diff --
These classes are in an `.internal` package so there's no need for
`private[spark]`. Or at least that's what I see in the sbt build scripts.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
