[jira] [Updated] (YARN-4418) AM Resource Limit per partition can be updated to ResourceUsage as well
[
https://issues.apache.org/jira/browse/YARN-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4418:
--
Attachment: 0005-YARN-4418.patch
Thank you [~leftnoteasy] for the comments.
Updated patch by adding parameterized test cases for missing entries in
{{ResourceUsage}} and {{QueueCapacities}} in to their respective test classes.
Kindly help to review.
> AM Resource Limit per partition can be updated to ResourceUsage as well
> ---
>
> Key: YARN-4418
> URL: https://issues.apache.org/jira/browse/YARN-4418
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: 0001-YARN-4418.patch, 0002-YARN-4418.patch,
> 0003-YARN-4418.patch, 0004-YARN-4418.patch, 0005-YARN-4418.patch
>
>
> AMResourceLimit is now extended to all partitions after YARN-3216. Its also
> better to track this ResourceLimit in existing {{ResourceUsage}} so that REST
> framework can be benefited to avail this information easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4418) AM Resource Limit per partition can be updated to ResourceUsage as well
[
https://issues.apache.org/jira/browse/YARN-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4418:
--
Attachment: 0004-YARN-4418.patch
Thank you [~leftnoteasy] for the explanation. That sounds good.
So as you mentioned earlier, we will have this metric updated only when
{{activateApplications}} is called. and this covers all cases. Uploading a
patch addressing same. Kindly help to check.
> AM Resource Limit per partition can be updated to ResourceUsage as well
> ---
>
> Key: YARN-4418
> URL: https://issues.apache.org/jira/browse/YARN-4418
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: 0001-YARN-4418.patch, 0002-YARN-4418.patch,
> 0003-YARN-4418.patch, 0004-YARN-4418.patch
>
>
> AMResourceLimit is now extended to all partitions after YARN-3216. Its also
> better to track this ResourceLimit in existing {{ResourceUsage}} so that REST
> framework can be benefited to avail this information easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4418) AM Resource Limit per partition can be updated to ResourceUsage as well
[
https://issues.apache.org/jira/browse/YARN-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4418:
--
Attachment: 0003-YARN-4418.patch
Uploading a patch by handling only NO_LABEL scenario. [~leftnoteasy] pls help
to check this corner case, and if its not valid I remove the check.
> AM Resource Limit per partition can be updated to ResourceUsage as well
> ---
>
> Key: YARN-4418
> URL: https://issues.apache.org/jira/browse/YARN-4418
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: 0001-YARN-4418.patch, 0002-YARN-4418.patch,
> 0003-YARN-4418.patch
>
>
> AMResourceLimit is now extended to all partitions after YARN-3216. Its also
> better to track this ResourceLimit in existing {{ResourceUsage}} so that REST
> framework can be benefited to avail this information easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4418) AM Resource Limit per partition can be updated to ResourceUsage as well
[
https://issues.apache.org/jira/browse/YARN-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4418:
--
Attachment: 0002-YARN-4418.patch
Uploading a newer version of patching after looking into test failures.
One point to note:
AMLimit is now a part of ResourceUsage. ResourceUsage was updated when
containers were allocated to it, else there wont be any entries added to the
ResourceUsage. However, AMLimit may slightly vary here as it need not have to
wait for any real resourceUsage to get it calculated. It can take
queue/partitionCapacity to get the capacity.
In first patch, I tried to calculate/update AMLimit for all accessible labels.
I was slightly in half mind here, due to some unwanted resource usage updates
here came also with this (0 for all other enums in ResourceUsage) . So I
changed to update AMLimit for those labels which are now Used/Added in
ResourceUsage.
[~leftnoteasy] pls help to share your point of view here so that we can
finalize on this.
> AM Resource Limit per partition can be updated to ResourceUsage as well
> ---
>
> Key: YARN-4418
> URL: https://issues.apache.org/jira/browse/YARN-4418
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: 0001-YARN-4418.patch, 0002-YARN-4418.patch
>
>
> AMResourceLimit is now extended to all partitions after YARN-3216. Its also
> better to track this ResourceLimit in existing {{ResourceUsage}} so that REST
> framework can be benefited to avail this information easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4418) AM Resource Limit per partition can be updated to ResourceUsage as well
[
https://issues.apache.org/jira/browse/YARN-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4418:
--
Attachment: 0001-YARN-4418.patch
Attaching an initial patch for updating AM Limit per partition limit to
ResourceUsage for queue.
I was trying to update latest {{AM Limit per partition }} just before creating
DAO object. This can help in getting latest and updated limit info (for
NO_LABEL, we may consider queue used capacity which can go upto max-capacity
and limit will vary in such cases).
[~leftnoteasy]pls help to review the same and share your thoughts.
> AM Resource Limit per partition can be updated to ResourceUsage as well
> ---
>
> Key: YARN-4418
> URL: https://issues.apache.org/jira/browse/YARN-4418
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: 0001-YARN-4418.patch
>
>
> AMResourceLimit is now extended to all partitions after YARN-3216. Its also
> better to track this ResourceLimit in existing {{ResourceUsage}} so that REST
> framework can be benefited to avail this information easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
