[
https://issues.apache.org/jira/browse/YARN-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13729672#comment-13729672
]
Philip Zeyliger commented on YARN-1024:
---------------------------------------
A virtual core might be doing two things, and they're in conflict with each
other. On one hand, we're saying it's a schedulable resource: a node manager
might be configured to allow up to, say, 20 virtual cores. On the other hand,
it's a proxy for the horsepower of the machine, which is what an ECU is.
Let's take a concrete example: a single-threaded, CPU-bound simulation library.
(The Pi mapreduce example is such an example.) As a writer of said library, I
might want to tell the scheduler that, when run, the library will spin one CPU
(because it's single-threaded, and because it doesn't block on other resources
ever). It will spin that CPU at 2007 levels or at 2012 levels: it'll spin
whatever is available to it. If I were to package my simulator and give it to
other people on other clusters, it would still be true that it spins one CPU.
Its runtime, however, would vary depending on the horsepower.
I think it's more important for this simulation example to be able to declare
"I'm going to spin 1 of your 16 CPUs, buddy" rather than "I'd like 1 unit of
2007 technology." The latter seems less useful for scheduling. (Meanwhile,
the latter is very useful for billing/charge-back.)
Both notions are useful, but I'm trying to argue that they're quite different,
and I think the "number of cores you'd eat typically" one is the more useful
one to me.
> Define a virtual core unambigiously
> -----------------------------------
>
> Key: YARN-1024
> URL: https://issues.apache.org/jira/browse/YARN-1024
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
>
> We need to clearly define the meaning of a virtual core unambiguously so that
> it's easy to migrate applications between clusters.
> For e.g. here is Amazon EC2 definition of ECU:
> http://aws.amazon.com/ec2/faqs/#What_is_an_EC2_Compute_Unit_and_why_did_you_introduce_it
> Essentially we need to clearly define a YARN Virtual Core (YVC).
> Equivalently, we can use ECU itself: *One EC2 Compute Unit provides the
> equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
