[
https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178832#comment-16178832
]
Daniel Templeton commented on YARN-6504:
----------------------------------------
The vast majority of users still only ask for 1 core and 1GB RAM, so the scope
of the problem is limited. I don't see where a partial solution actually
solves anything. Let's take it case by case:
* Users who only use CPU and memory: there's no practical difference between
specifying CPU and memory, and specifying a profile and then overriding one (or
both!) of the values.
* Users who use other resource types: if a user's job depends on something
other than CPU and memory, letting them only override CPU and memory doesn't
save them from having to retest with profiles.
The main driving reason for having the override capability is that job's which
don't ask for a profile get the default profile and need to be able to ignore
it. What if we get rid of the default profile? I don't think it's actually
very useful. It let's the admin set the default resource request for the
cluster, but you can do that for each of the application frameworks already,
and I don't think it makes much sense to have a default that is ignorant of the
application to which it is applied. I also really don't like that we're
creating two sets of configurations to control the same thing. If we take away
the default, the override capability isn't useful anymore. If you ask for a
profile, you get the profile. If not, you get the resources you ask for. If
you didn't ask for any, you get the usual defaults for your application
framework.
> Add support for resource profiles in MapReduce
> ----------------------------------------------
>
> Key: YARN-6504
> URL: https://issues.apache.org/jira/browse/YARN-6504
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Attachments: YARN-6504-YARN-3926.001.patch,
> YARN-6504.YARN-3926.002.patch, YARN-6504.YARN-3926.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
