[ https://issues.apache.org/jira/browse/SPARK-29499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021560#comment-17021560 ]
Xingbo Jiang commented on SPARK-29499: -------------------------------------- [~dongjoon] Yes, this is master only, since it's a new API we don't need to backport. > Add mapPartitionsWithIndex for RDDBarrier > ----------------------------------------- > > Key: SPARK-29499 > URL: https://issues.apache.org/jira/browse/SPARK-29499 > Project: Spark > Issue Type: Bug > Components: PySpark, Spark Core > Affects Versions: 2.4.4 > Reporter: Xianyang Liu > Assignee: Xianyang Liu > Priority: Major > Fix For: 3.0.0 > > > There is only one method in `RDDBarrier`. We often use the partition index as > a label for the current partition. We need to get the index from > `TaskContext` index in the method of `mapPartitions` which is not convenient. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org