[
https://issues.apache.org/jira/browse/YARN-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193214#comment-16193214
]
Wangda Tan commented on YARN-7237:
----------------------------------
[~templedf],
Thanks for the additional comments,
bq. If you insist on keeping the minimum and maximum profiles, then you need
tests to validate that the profile names are actually disallowed.
I think you points are valid, let me think more and we can decide in the next
patch. As you said, minimum/maximum doesn't make a lot of sense because we
already receive min/max when submit new app or register AM. It is not possible
for app to ask a resource which all values are maximum.
bq. In the patch you removed testResourceProfiles(), but it looks like the test
was testing more than just the minimum and maximum profiles. My question is
whether it might not be better to just remove the minimum and maximum profile
portions of the test rather than removing the whole test.
I just removed from TestApplicationMasterService to TestResourceProfiles.
If you're fine with the latest patch, could you commit it and let's move more
discussions to next JIRA?
> Cleanup usages of ResourceProfiles
> ----------------------------------
>
> Key: YARN-7237
> URL: https://issues.apache.org/jira/browse/YARN-7237
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Priority: Critical
> Attachments: YARN-7237.001.patch, YARN-7237.002.patch,
> YARN-7237.003.patch, YARN-7237.004.patch, YARN-7237.005.patch
>
>
> While doing tests, there're a couple of issues:
> 1) When use {{ProfileCapability#getProfileCapabilityOverride}}, it does
> overwrite of whatever specified in resource-profiles.json when value >= 0.
> Which is different from javadocs of {{ProfileCapability}}
> bq. For example, if you have a resource profile "small" that maps to <4096M,
> 2 cores, 1 gpu> and you set the capability override to <8192M, 0 cores, 0
> gpu>, then the actual resource allocation on the ResourceManager will be
> <8192M, 2 cores, 1 gpu>
> To me, the correct behavior should do overwrite when value > 0. The reason
> is, by default resource value will be set to 0, For example, assume we have a
> profile {{"a" = (mem=3, vcore=5, res_1=7)}}, and create a
> capability-overwrite (capability = new resource(8). The final result should
> be (mem=8, vcore=5, res_1=7), instead of (mem=8, vcore=0, res_1=0).
> 2) ResourceProfileManager now loads minimum/maximum profile from config file
> (resource-profiles.json), to me this is not correct because minimum/maximum
> allocation for each resource types are already specified inside
> {{resource-types.xml}}. We should always use
> {{ResourceUtils#getResourceTypesMinimum/MaximumAllocation}} to get from
> resource-types.xml and yarn-site.xml. This value will be added to profiles so
> client can get these configs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
