[
https://issues.apache.org/jira/browse/YARN-18?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618446#comment-13618446
]
Arun C Murthy commented on YARN-18:
-----------------------------------
[~vicaya] / [~djp] - Can you please describe what is happening here? I see a
lot of factories and then no changes to actually support node-group...
What do I need to read-up to understand what is going on here?
One thought - seems to me, (maybe longer term), we need to abstract the notion
of topology in the Scheduler without having to make changes every time we need
to add another layer in the topology... think of complex star-topologies etc.
Thoughts?
Another thing to consider is that, for now, it might be better to implement a
new NodeGroup Scheduler rather than change all existing Schedulers?
> Make locatlity in YARN's container assignment and task scheduling pluggable
> for other deployment topology
> ---------------------------------------------------------------------------------------------------------
>
> Key: YARN-18
> URL: https://issues.apache.org/jira/browse/YARN-18
> Project: Hadoop YARN
> Issue Type: New Feature
> Affects Versions: 2.0.3-alpha
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments:
> HADOOP-8474-ContainerAssignmentTaskScheduling-pluggable.patch,
> MAPREDUCE-4309.patch, MAPREDUCE-4309-v2.patch, MAPREDUCE-4309-v3.patch,
> MAPREDUCE-4309-v4.patch, MAPREDUCE-4309-v5.patch, MAPREDUCE-4309-v6.patch,
> MAPREDUCE-4309-v7.patch, YARN-18.patch, YARN-18-v2.patch, YARN-18-v3.1.patch,
> YARN-18-v3.2.patch, YARN-18-v3.patch, YARN-18-v4.1.patch, YARN-18-v4.2.patch,
> YARN-18-v4.3.patch, YARN-18-v4.patch, YARN-18-v5.1.patch, YARN-18-v5.patch
>
>
> There are several classes in YARN’s container assignment and task scheduling
> algorithms that relate to data locality which were updated to give preference
> to running a container on other locality besides node-local and rack-local
> (like nodegroup-local). This propose to make these data structure/algorithms
> pluggable, like: SchedulerNode, RMNodeImpl, etc. The inner class
> ScheduledRequests was made a package level class to it would be easier to
> create a subclass, ScheduledRequestsWithNodeGroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
