[ https://issues.apache.org/jira/browse/SPARK-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385221#comment-14385221 ]
Apache Spark commented on SPARK-6584: ------------------------------------- User 'SaintBacchus' has created a pull request for this issue: https://github.com/apache/spark/pull/5240 > Provide ExecutorPrefixTaskLocation to support the rdd which can be aware of > partition's executor location. > ----------------------------------------------------------------------------------------------------------- > > Key: SPARK-6584 > URL: https://issues.apache.org/jira/browse/SPARK-6584 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 1.4.0 > Reporter: SaintBacchus > > The function *RDD.getPreferredLocations* can only be set the host awareness > prefer locations. > If some *RDD* wants to be scheduled by executor(such as BlockRDD), spark can > do nothing for this. > So I want to provide *ExecutorPrefixTaskLocation* to support the rdd which > can be aware of partition's executor location. This mechanism can avoid data > transfor in the case of many executor in the same host. > I think it's very useful especially for *SparkStreaming* since the > *Receriver* save data into the *BlockManger* and then become a BlockRDD -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org