[
https://issues.apache.org/jira/browse/YARN-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13729833#comment-13729833
]
Steve Loughran commented on YARN-1024:
--------------------------------------
I was the one trying to convince Sandy that a uniform core metric is dangerous,
it's like when a MIP was a VAX-equivalent Million Instructions.
# different parts have different performance in terms of FPU and memory IO
bandwidth, even if the integer perf is the same. (hence people like to get
Intel parts over AMD parts on EC2 allocations).
# there's also the hyperthreading issue; is an HT core the equivalent of a real
core (no, but Linux treats them the same, AFAIK).
# over time, as 2007 gets further away, the metric becomes less relevant.
# EC2 also includes RAM (e.g m1.small has same CPU as m1.medium, only less RAM;
AWS considers medium as having 2x ECUs.
One thing I was arguing against in YARN-972 is allocating fractions of a real
core: if I say "1 core", I get a single core, irrespective of performance. If
EC2s are used, and I ask for 1 ECU, does that mean that I get 0.50 of a bigger
core, or a free upgrade.
I'm happy if I ask for 8 ECUs and get a guarantee of not being on a CPU with <8
ECUs, making it a minimum requirement of the CPU perf.
> Define a virtual core unambigiously
> -----------------------------------
>
> Key: YARN-1024
> URL: https://issues.apache.org/jira/browse/YARN-1024
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
>
> We need to clearly define the meaning of a virtual core unambiguously so that
> it's easy to migrate applications between clusters.
> For e.g. here is Amazon EC2 definition of ECU:
> http://aws.amazon.com/ec2/faqs/#What_is_an_EC2_Compute_Unit_and_why_did_you_introduce_it
> Essentially we need to clearly define a YARN Virtual Core (YVC).
> Equivalently, we can use ECU itself: *One EC2 Compute Unit provides the
> equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
