[
https://issues.apache.org/jira/browse/SPARK-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-1524.
------------------------------
Resolution: Won't Fix
> TaskSetManager'd better not schedule tasks which has no preferred executorId
> using PROCESS_LOCAL in the first search process
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-1524
> URL: https://issues.apache.org/jira/browse/SPARK-1524
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: YanTang Zhai
> Priority: Minor
>
> ShuffleMapTask is constructed with TaskLocation which has only host not
> (host, executorID) pair in DAGScheduler.
> When TaskSetManager schedules ShuffleMapTask which has no preferred
> executorId using specific execId host and PROCESS_LOCAL locality level, no
> tasks match the given locality constraint in the first search process.
> We also find that the host used by Scheduler is hostname while the host used
> by TaskLocation is IP in our cluster. The tow hosts do not match, that makes
> pendingTasksForHost HashMap empty and the finding task process against our
> expectation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
