[jira] [Resolved] (YARN-11667) Federation: ResourceRequestComparator occurs NPE when using low version of hadoop submit application
[ https://issues.apache.org/jira/browse/YARN-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qiuliang resolved YARN-11667. - Resolution: Won't Do > Federation: ResourceRequestComparator occurs NPE when using low version of > hadoop submit application > > > Key: YARN-11667 > URL: https://issues.apache.org/jira/browse/YARN-11667 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy >Affects Versions: 3.4.0 >Reporter: qiuliang >Priority: Major > Labels: pull-request-available > > When a application is submitted using a lower version of hadoop and the > Resource Request built by AM has no ExecutionTypeRequest. After the Resource > Request is submitted to AMRMProxy, the NPE occurs when AMRMProxy reconstructs > the Allocate Request to add Resource Request to its ask -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-11667) Federation: ResourceRequestComparator occurs NPE when using low version of hadoop submit application
qiuliang created YARN-11667: --- Summary: Federation: ResourceRequestComparator occurs NPE when using low version of hadoop submit application Key: YARN-11667 URL: https://issues.apache.org/jira/browse/YARN-11667 Project: Hadoop YARN Issue Type: Bug Components: amrmproxy Affects Versions: 3.4.0 Reporter: qiuliang When a application is submitted using a lower version of hadoop and the Resource Request built by AM has no ExecutionTypeRequest. After the Resource Request is submitted to AMRMProxy, the NPE occurs when AMRMProxy reconstructs the Allocate Request to add Resource Request to its ask -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-9437) RMNodeImpls occupy too much memory and causes RM GC to take a long time
qiuliang created YARN-9437: -- Summary: RMNodeImpls occupy too much memory and causes RM GC to take a long time Key: YARN-9437 URL: https://issues.apache.org/jira/browse/YARN-9437 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.9.1 Reporter: qiuliang Attachments: 1.png, 2.png, 3.png We use hadoop-2.9.1 in our production environment with 1600+ nodes. 95.63% of RM memory is occupied by RMNodeImpl. Analysis of RM memory found that each RMNodeImpl has approximately 14M. The reason is that there is a 13W+ completedcontainers in each RMNodeImpl that has not been released. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8978) For fair scheduler, application with higher priority should also get priority resources for running AM
qiuliang created YARN-8978: -- Summary: For fair scheduler, application with higher priority should also get priority resources for running AM Key: YARN-8978 URL: https://issues.apache.org/jira/browse/YARN-8978 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Reporter: qiuliang In order to allow important applications to run earlier, we used priority scheduling in the fair scheduler, and FairSharePolicy uses YARN-6307. Considering this situation, there are two applications (with different priorities) in the same queue and both are accepted. Both applications are demanding and hungry when dispatched to the queue. Next, calculate the weight ratio. Since the used resources of both applications are 0, the weight ratio is also 0. The priority is invalid in this case. Low-priority applications may get resources to run AM earlier than high-priority applications. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
