[
https://issues.apache.org/jira/browse/YARN-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176401#comment-16176401
]
Sunil G commented on YARN-7237:
-------------------------------
Thanks [~leftnoteasy] for the idea. I also think this is better
Some high level thoughts on this context
# In some cases, minimum and maximum could be set to some infinity values. In
that case, we might need to bound this to a some reasonable upper values
instead of INT_MAX or LONG_MAX (in case of new resources). Because below config
is valid and it does not have any min or max
{noformat}
<property>
<name>yarn.resource-types</name>
<value>resource1</value>
</property>
<property>
<name>yarn.resource-types.resource1.units</name>
<value>G</value>
</property>
{noformat}
# Override behavior makes sense to me as well. Could we document it also well
enough that we will have clear clarity on same for users/admins as well.
I am still reviewing code and will share comments if any soon.
> Cleanup usages of ResourceProfiles
> ----------------------------------
>
> Key: YARN-7237
> URL: https://issues.apache.org/jira/browse/YARN-7237
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Priority: Critical
> Attachments: YARN-7237.001.patch, YARN-7237.002.patch,
> YARN-7237.003.patch
>
>
> While doing tests, there're a couple of issues:
> 1) When use {{ProfileCapability#getProfileCapabilityOverride}}, it does
> overwrite of whatever specified in resource-profiles.json when value >= 0.
> Which is different from javadocs of {{ProfileCapability}}
> bq. For example, if you have a resource profile "small" that maps to <4096M,
> 2 cores, 1 gpu> and you set the capability override to <8192M, 0 cores, 0
> gpu>, then the actual resource allocation on the ResourceManager will be
> <8192M, 2 cores, 1 gpu>
> To me, the correct behavior should do overwrite when value > 0. The reason
> is, by default resource value will be set to 0, For example, assume we have a
> profile {{"a" = (mem=3, vcore=5, res_1=7)}}, and create a
> capability-overwrite (capability = new resource(8). The final result should
> be (mem=8, vcore=5, res_1=7), instead of (mem=8, vcore=0, res_1=0).
> 2) ResourceProfileManager now loads minimum/maximum profile from config file
> (resource-profiles.json), to me this is not correct because minimum/maximum
> allocation for each resource types are already specified inside
> {{resource-types.xml}}. We should always use
> {{ResourceUtils#getResourceTypesMinimum/MaximumAllocation}} to get from
> resource-types.xml and yarn-site.xml. This value will be added to profiles so
> client can get these configs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
