[
https://issues.apache.org/jira/browse/YARN-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540135#comment-13540135
]
Karthik Kambatla commented on YARN-2:
-------------------------------------
Thanks Arun, that should be okay. Created YARN-294 for the same, and added you
as a watcher there.
Just to continue discussion on cpu shares (I understand we are not addressing
this here in this JIRA):
IIUC, picking a high number of virtual cores (2^x - e.g. x=10 ) helps with:
# Request for less than a full core. The higher the number, the smaller the
share one can get. We can achieve this through a float as well, but we know the
issues with float precision and floating point operations.
# Ask for more resources than available - for x=10, all 1024 shares can be
requested - leading to sharing the physical cores.
# Weighted sharing of resources asked. Say A asks for 256 and B asks for 768,
the cores can be shared in 1:3 ratio as they should be.
I agree this is not the only way to achieve these. Also, agree that the number
of virtual cores need not be 1024 as in cgroups; if we go this path, it doesn't
hurt to pick that number though.
> Enhance CS to schedule accounting for both memory and cpu cores
> ---------------------------------------------------------------
>
> Key: YARN-2
> URL: https://issues.apache.org/jira/browse/YARN-2
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacityscheduler, scheduler
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 2.0.3-alpha
>
> Attachments: MAPREDUCE-4327.patch, MAPREDUCE-4327.patch,
> MAPREDUCE-4327.patch, MAPREDUCE-4327-v2.patch, MAPREDUCE-4327-v3.patch,
> MAPREDUCE-4327-v4.patch, MAPREDUCE-4327-v5.patch, YARN-2-help.patch,
> YARN-2.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch,
> YARN-2.patch
>
>
> With YARN being a general purpose system, it would be useful for several
> applications (MPI et al) to specify not just memory but also CPU (cores) for
> their resource requirements. Thus, it would be useful to the
> CapacityScheduler to account for both.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
