[jira] [Updated] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
[ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-7410: Fix Version/s: 2.10.0 > Cleanup FixedValueResource to avoid dependency to ResourceUtils > --- > > Key: YARN-7410 > URL: https://issues.apache.org/jira/browse/YARN-7410 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.1.0 >Reporter: Sunil Govindan >Assignee: Wangda Tan >Priority: Major > Fix For: 3.0.0, 2.10.0 > > Attachments: YARN-7410.001.patch, YARN-7410.002.patch, > YARN-7410.003.patch, YARN-7410.branch-3.0.003.patch > > > After YARN-7307, Client/AM don't need to keep a up-to-dated resource-type.xml > in the classpath. Instead, they can use YarnClient/ApplicationMasterProtocol > APIs to get the resource types from RM and refresh local types. > One biggest issue of this approach is FixedValueResource: Since we initialize > FixedValueResource in static block, and they won't be updated if resource > types refreshed. > So we need to properly update FixedValueResource to make it can get > up-to-date results -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
[ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7410: - Attachment: YARN-7410.branch-3.0.003.patch > Cleanup FixedValueResource to avoid dependency to ResourceUtils > --- > > Key: YARN-7410 > URL: https://issues.apache.org/jira/browse/YARN-7410 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.1.0 >Reporter: Sunil G >Assignee: Wangda Tan > Fix For: 3.1.0 > > Attachments: YARN-7410.001.patch, YARN-7410.002.patch, > YARN-7410.003.patch, YARN-7410.branch-3.0.003.patch > > > After YARN-7307, Client/AM don't need to keep a up-to-dated resource-type.xml > in the classpath. Instead, they can use YarnClient/ApplicationMasterProtocol > APIs to get the resource types from RM and refresh local types. > One biggest issue of this approach is FixedValueResource: Since we initialize > FixedValueResource in static block, and they won't be updated if resource > types refreshed. > So we need to properly update FixedValueResource to make it can get > up-to-date results -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
[ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7410: - Attachment: YARN-7410.003.patch Attached ver.3 patch addressed comments from [~templedf]. > Cleanup FixedValueResource to avoid dependency to ResourceUtils > --- > > Key: YARN-7410 > URL: https://issues.apache.org/jira/browse/YARN-7410 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.1.0 >Reporter: Sunil G >Assignee: Wangda Tan > Attachments: YARN-7410.001.patch, YARN-7410.002.patch, > YARN-7410.003.patch > > > After YARN-7307, Client/AM don't need to keep a up-to-dated resource-type.xml > in the classpath. Instead, they can use YarnClient/ApplicationMasterProtocol > APIs to get the resource types from RM and refresh local types. > One biggest issue of this approach is FixedValueResource: Since we initialize > FixedValueResource in static block, and they won't be updated if resource > types refreshed. > So we need to properly update FixedValueResource to make it can get > up-to-date results -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
[ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7410: - Description: After YARN-7307, Client/AM don't need to keep a up-to-dated resource-type.xml in the classpath. Instead, they can use YarnClient/ApplicationMasterProtocol APIs to get the resource types from RM and refresh local types. One biggest issue of this approach is FixedValueResource: Since we initialize FixedValueResource in static block, and they won't be updated if resource types refreshed. So we need to properly update FixedValueResource to make it can get up-to-date results was: Currently FixedValue Resource constants has some dependencies to ResourceUtils. This jira will help to cleanup this dependencies. Thanks [~leftnoteasy] for finding the same. > Cleanup FixedValueResource to avoid dependency to ResourceUtils > --- > > Key: YARN-7410 > URL: https://issues.apache.org/jira/browse/YARN-7410 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.1.0 >Reporter: Sunil G >Assignee: Wangda Tan > Attachments: YARN-7410.001.patch, YARN-7410.002.patch > > > After YARN-7307, Client/AM don't need to keep a up-to-dated resource-type.xml > in the classpath. Instead, they can use YarnClient/ApplicationMasterProtocol > APIs to get the resource types from RM and refresh local types. > One biggest issue of this approach is FixedValueResource: Since we initialize > FixedValueResource in static block, and they won't be updated if resource > types refreshed. > So we need to properly update FixedValueResource to make it can get > up-to-date results -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
[ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7410: - Attachment: YARN-7410.002.patch Saw about 6% performance regression for previous attached patch. Updated patch to show approach of lazy-reinitialization for FixedValueResource. Now didn't see noticeable performance regression for 2 resource types. and 10% performance regression for 3 resource types or more. Need additional optimizations. > Cleanup FixedValueResource to avoid dependency to ResourceUtils > --- > > Key: YARN-7410 > URL: https://issues.apache.org/jira/browse/YARN-7410 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.1.0 >Reporter: Sunil G >Assignee: Wangda Tan > Attachments: YARN-7410.001.patch, YARN-7410.002.patch > > > Currently FixedValue Resource constants has some dependencies to > ResourceUtils. This jira will help to cleanup this dependencies. > Thanks [~leftnoteasy] for finding the same. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
[ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-7410: -- Attachment: YARN-7410.001.patch Attaching an initial patch on behalf of [~leftnoteasy] and submitting for jenkins > Cleanup FixedValueResource to avoid dependency to ResourceUtils > --- > > Key: YARN-7410 > URL: https://issues.apache.org/jira/browse/YARN-7410 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.1.0 >Reporter: Sunil G >Assignee: Wangda Tan > Attachments: YARN-7410.001.patch > > > Currently FixedValue Resource constants has some dependencies to > ResourceUtils. This jira will help to cleanup this dependencies. > Thanks [~leftnoteasy] for finding the same. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
