[
https://issues.apache.org/jira/browse/YARN-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365914#comment-15365914
]
Yu commented on YARN-5261:
--------------------------
Yarn-5261 main purpose is: lease the unused resource on some nodes which
already having some important services (outside of Yarn), and make the
lease/reclaim controllable by the nodes' owners (landlords).
It also uses zookeeper to make the lease/reclaim operation recoverable even
there's RM restarting or failover, so it's very stable.
I checked the issues you mentioned, and I got:
* Yarn-5215:
It has similar purpose as my issue: leverage the unused
resources on some nodes.
But, it's another approach, and can get different result.
It assumes all nodes are schedulable by Yarn, and changes the
unallocated resource by reducing the external utilization on each node
dynamically.
My issue (Yarn-5261) is different:
1. don't assume all nodes are schedulable (though Yarn NM is
running on all nodes). The nodes are schedulable only when the landlord leases
them. So, it's more controllable.
2. the total schedulable resource is estimated by the landlord
and set when node is leased, which is steadier than changing after calculating
the utilization. And, landlord can plan how to use the resources better, not
rely on automatic calculating.
3. when the schedulable resource is over used, the containers
will be killed to make sure obeying the landlord.
* Yarn-1011:
It's more like a Yarn scheduler enhancement for containers, but
don't consider the external utilization at all.
It's not proper for the scenario that running yarn on nodes
with other services.
* Yarn-5202:
It's about resource overcommit, not the same purpose.
So, they look similar, but are different. Users can use them based on their
requirements.
> Lease/Reclaim Extension to Yarn
> -------------------------------
>
> Key: YARN-5261
> URL: https://issues.apache.org/jira/browse/YARN-5261
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
> Reporter: Yu
> Attachments: YARN-5261-1.patch, Yarn-5261.pdf
>
>
> In some clusters outside of Yarn, the machines resources are not fully
> utilized, e.g., resource usage is quite low at night.
> To better utilize the resources while keep the existing SLA of the cluster,
> Lease/Reclaim Extension to Yarn is introduced.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
