[
https://issues.apache.org/jira/browse/YARN-4962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243136#comment-15243136
]
Daniel Templeton commented on YARN-4962:
----------------------------------------
This is a common issue in mixed node clusters. The typical solution is to load
the cluster "from different ends." If the cluster has nodes of type A and type
B (say, regular and big memory) and jobs of type a and b (where type b jobs
need type B machines), the scheduler should schedule type a jobs to type A
nodes, spilling into type B nodes only when there are no more type A nodes
available. Type b jobs only run on type B nodes.
In Grid Engine, the way that's implemented is with node labels. All type B
nodes are labeled as such. All type b jobs are submitted with a hard request
for type B nodes (hard = requires). All other jobs are submitted with a soft
request for !type B nodes (soft = preference).
> support filling up containers on node one by one
> -------------------------------------------------
>
> Key: YARN-4962
> URL: https://issues.apache.org/jira/browse/YARN-4962
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: sandflee
>
> we had a gpu cluster, jobs with bigger resource request couldn't be satisfied
> for node is running the jobs with smaller resource request. we didn't open
> reserve system because gpu jobs may run days or weeks. we expect scheduler
> allocate containers to fill the node , then there will be resource to run
> jobs with big resource request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
