[
https://issues.apache.org/jira/browse/YARN-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13664616#comment-13664616
]
Timothy St. Clair commented on YARN-689:
----------------------------------------
Hi folks,
+1 in agreement with [~tucu00], around resource requests. I'm not intimately
familiar with the inner workings of YARN, but I have fair amount of experience
with other schedulers. They typically get around this is through expression
syntax/lang where the admin can define policies in order to tune to their
environment workloads, where %quantization boundaries are ideal (e.g. best fit
in [X] chunks, where [X] could be whole(MB) or fractional(CPU) units), with
fragmentation being the biggest problem with this flexibility.
Use Case:
A request comes in for 20MB, .5CPUs(cpu_shares in cgroups), 1 booster_rock, and
3 GPUs. That request is then evaluated against an
expression(min,max,whatever) during the activation time, which then splices the
resource appropriately.
Either way, this treads into a known space that exists around resource
splicing, utilization, and such and such.
Ref1:
http://spinningmatt.wordpress.com/2012/11/13/no-longer-thinking-in-slots-thinking-in-aggregate-resources-and-consumption-policies/
Ref2: Every paper & talk that Wilkes gives.
Cheers,
Tim
> Add multiplier unit to resourcecapabilities
> -------------------------------------------
>
> Key: YARN-689
> URL: https://issues.apache.org/jira/browse/YARN-689
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api, scheduler
> Affects Versions: 2.0.4-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: YARN-689.patch, YARN-689.patch, YARN-689.patch
>
>
> Currently we overloading the minimum resource value as the actual multiplier
> used by the scheduler.
> Today with a minimum memory set to 1GB, requests for 1.5GB are always
> translated to allocation of 2GB.
> We should decouple the minimum allocation from the multiplier.
> The multiplier should also be exposed to the client via the
> RegisterApplicationMasterResponse
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
