[jira] [Updated] (YARN-4162) Scheduler info in REST, is currently not displaying partition specific queue information similar to UI
[ https://issues.apache.org/jira/browse/YARN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-4162: Attachment: YARN-4162.v1.001.patch Hi [~wangda], Please find the patch for the REST interface for partition specific CS information > Scheduler info in REST, is currently not displaying partition specific queue > information similar to UI > -- > > Key: YARN-4162 > URL: https://issues.apache.org/jira/browse/YARN-4162 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-4162.v1.001.patch > > > When Node Labels are enabled then REST Scheduler Information should also > provide partition specific queue information similar to the existing Web UI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4162) Scheduler info in REST, is currently not displaying partition specific queue information similar to UI
[ https://issues.apache.org/jira/browse/YARN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-4162: Description: When Node Labels are enabled then REST Scheduler Information should also provide partition specific queue information similar to the existing Web UI (was: When Node Labels are enabled then CLI & REST Scheduler Information should also provide partition specific queue information similar to the existing Web UI) > Scheduler info in REST, is currently not displaying partition specific queue > information similar to UI > -- > > Key: YARN-4162 > URL: https://issues.apache.org/jira/browse/YARN-4162 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > > When Node Labels are enabled then REST Scheduler Information should also > provide partition specific queue information similar to the existing Web UI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4162) Scheduler info in REST, is currently not displaying partition specific queue information similar to UI
[ https://issues.apache.org/jira/browse/YARN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-4162: Summary: Scheduler info in REST, is currently not displaying partition specific queue information similar to UI (was: Scheduler info in REST & CLI, is currently not displaying partition specific queue information similar to UI) > Scheduler info in REST, is currently not displaying partition specific queue > information similar to UI > -- > > Key: YARN-4162 > URL: https://issues.apache.org/jira/browse/YARN-4162 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > > When Node Labels are enabled then CLI & REST Scheduler Information should > also provide partition specific queue information similar to the existing Web > UI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4169) jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
[
https://issues.apache.org/jira/browse/YARN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-4169:
Attachment: YARN-4169.v1.002.patch
Attaching a patching fixing the both of [~steve_l] comments. Missed to
understand the first comment earlier!
[~rohithsharma], can you please take a look at the patch
> jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
> -
>
> Key: YARN-4169
> URL: https://issues.apache.org/jira/browse/YARN-4169
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
> Environment: Jenkins
>Reporter: Steve Loughran
>Assignee: Naganarasimha G R
>Priority: Critical
> Attachments: YARN-4169.v1.001.patch, YARN-4169.v1.002.patch
>
>
> Test failing in [[Jenkins build
> 402|https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/402/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestNodeStatusUpdaterForLabels/testNodeStatusUpdaterForNodeLabels/]
> {code}
> java.lang.NullPointerException: null
> at java.util.HashSet.(HashSet.java:118)
> at
> org.apache.hadoop.yarn.nodelabels.NodeLabelTestBase.assertNLCollectionEquals(NodeLabelTestBase.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels.testNodeStatusUpdaterForNodeLabels(TestNodeStatusUpdaterForLabels.java:268)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2062) Too many InvalidStateTransitionExceptions from NodeState.NEW on RM failover
[
https://issues.apache.org/jira/browse/YARN-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909918#comment-14909918
]
Neelesh Srinivas Salian commented on YARN-2062:
---
[~ka...@cloudera.com] is this still valid?
> Too many InvalidStateTransitionExceptions from NodeState.NEW on RM failover
> ---
>
> Key: YARN-2062
> URL: https://issues.apache.org/jira/browse/YARN-2062
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
> Attachments: yarn-2062-1.patch
>
>
> On busy clusters, we see several
> {{org.apache.hadoop.yarn.state.InvalidStateTransitonException}} for events
> invoked against NEW nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4169) jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
[
https://issues.apache.org/jira/browse/YARN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909853#comment-14909853
]
Hadoop QA commented on YARN-4169:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 9m 20s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | javac | 7m 46s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 24s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 3 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 28s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 13s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 1m 59s | Tests failed in
hadoop-yarn-common. |
| {color:red}-1{color} | yarn tests | 8m 4s | Tests failed in
hadoop-yarn-server-nodemanager. |
| {color:red}-1{color} | yarn tests | 54m 48s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 91m 1s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.yarn.logaggregation.TestAggregatedLogsBlock |
| | hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesContainers |
| | hadoop.yarn.server.nodemanager.webapp.TestNMWebServices |
| | hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestChildQueueOrder
|
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
|
| | hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12762611/YARN-4169.v1.002.patch
|
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / f0f984e |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/9278/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9278/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9278/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9278/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9278/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9278/console |
This message was automatically generated.
> jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
> -
>
> Key: YARN-4169
> URL: https://issues.apache.org/jira/browse/YARN-4169
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
> Environment: Jenkins
>Reporter: Steve Loughran
>Assignee: Naganarasimha G R
>Priority: Critical
> Attachments: YARN-4169.v1.001.patch, YARN-4169.v1.002.patch
>
>
> Test failing in [[Jenkins build
> 402|https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/402/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestNodeStatusUpdaterForLabels/testNodeStatusUpdaterForNodeLabels/]
> {code}
> java.lang.NullPointerException: null
> at java.util.HashSet.(HashSet.java:118)
> at
> org.apache.hadoop.yarn.nodelabels.NodeLabelTestBase.assertNLCollectionEquals(NodeLabelTestBase.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels.testNodeStatusUpdaterForNodeLabels(TestNodeStatusUpdaterForLabels.java:268)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2353) FairScheduler: Update demand asynchronously instead of in the Update Thread
[ https://issues.apache.org/jira/browse/YARN-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909919#comment-14909919 ] Neelesh Srinivas Salian commented on YARN-2353: --- [~ka...@cloudera.com],if you are not looking into it, I would like to help out. Either way, could you please add some description to help elaborate the improvement? Thank you. > FairScheduler: Update demand asynchronously instead of in the Update Thread > --- > > Key: YARN-2353 > URL: https://issues.apache.org/jira/browse/YARN-2353 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Affects Versions: 2.4.1 >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4162) Scheduler info in REST, is currently not displaying partition specific queue information similar to UI
[
https://issues.apache.org/jira/browse/YARN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909844#comment-14909844
]
Hadoop QA commented on YARN-4162:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 10s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 51s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 1s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 48s | The applied patch generated 7
new checkstyle issues (total was 43, now 50). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 30s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 27s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 60m 5s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 99m 55s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.rmapp.attempt.TestRMAppAttemptTransitions |
| | hadoop.yarn.server.resourcemanager.rmapp.attempt.TestAMLivelinessMonitor |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12762606/YARN-4162.v1.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f0f984e |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9277/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9277/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9277/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9277/console |
This message was automatically generated.
> Scheduler info in REST, is currently not displaying partition specific queue
> information similar to UI
> --
>
> Key: YARN-4162
> URL: https://issues.apache.org/jira/browse/YARN-4162
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, client, resourcemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-4162.v1.001.patch
>
>
> When Node Labels are enabled then REST Scheduler Information should also
> provide partition specific queue information similar to the existing Web UI
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (YARN-2062) Too many InvalidStateTransitionExceptions from NodeState.NEW on RM failover
[
https://issues.apache.org/jira/browse/YARN-2062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla resolved YARN-2062.
Resolution: Invalid
> Too many InvalidStateTransitionExceptions from NodeState.NEW on RM failover
> ---
>
> Key: YARN-2062
> URL: https://issues.apache.org/jira/browse/YARN-2062
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
> Attachments: yarn-2062-1.patch
>
>
> On busy clusters, we see several
> {{org.apache.hadoop.yarn.state.InvalidStateTransitonException}} for events
> invoked against NEW nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4066) Large number of queues choke fair scheduler
[ https://issues.apache.org/jira/browse/YARN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johan Gustavsson updated YARN-4066: --- Attachment: YARN-4066-3.patch The formatting issues have been taken care of, sorry about that. > Large number of queues choke fair scheduler > --- > > Key: YARN-4066 > URL: https://issues.apache.org/jira/browse/YARN-4066 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 2.7.1 >Reporter: Johan Gustavsson >Assignee: Johan Gustavsson > Attachments: YARN-4066-2.patch, YARN-4066-3.patch, yarn-4066-1.patch > > > Due to synchronization and all the loops performed during queue creation, > setting a large amount of queues (12000+) will completely choke the > scheduler. To deal with this some optimization to > "QueueManager.updateAllocationConfiguration(AllocationConfiguration > queueConf)" should be done to reduce the amount of unnesecary loops. The > attached patch have been tested to work with atleast 96000 queues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4066) Large number of queues choke fair scheduler
[
https://issues.apache.org/jira/browse/YARN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14910065#comment-14910065
]
Hadoop QA commented on YARN-4066:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 34s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 47s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 59s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 49s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 29s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 29s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 60m 4s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 99m 11s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12762639/YARN-4066-3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1c030c6 |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9280/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9280/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9280/console |
This message was automatically generated.
> Large number of queues choke fair scheduler
> ---
>
> Key: YARN-4066
> URL: https://issues.apache.org/jira/browse/YARN-4066
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Johan Gustavsson
>Assignee: Johan Gustavsson
> Attachments: YARN-4066-2.patch, YARN-4066-3.patch, yarn-4066-1.patch
>
>
> Due to synchronization and all the loops performed during queue creation,
> setting a large amount of queues (12000+) will completely choke the
> scheduler. To deal with this some optimization to
> "QueueManager.updateAllocationConfiguration(AllocationConfiguration
> queueConf)" should be done to reduce the amount of unnesecary loops. The
> attached patch have been tested to work with atleast 96000 queues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4066) Large number of queues choke fair scheduler
[ https://issues.apache.org/jira/browse/YARN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johan Gustavsson updated YARN-4066: --- Attachment: YARN-4066-2.patch > Large number of queues choke fair scheduler > --- > > Key: YARN-4066 > URL: https://issues.apache.org/jira/browse/YARN-4066 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 2.7.1 >Reporter: Johan Gustavsson >Assignee: Johan Gustavsson > Attachments: YARN-4066-2.patch, yarn-4066-1.patch > > > Due to synchronization and all the loops performed during queue creation, > setting a large amount of queues (12000+) will completely choke the > scheduler. To deal with this some optimization to > "QueueManager.updateAllocationConfiguration(AllocationConfiguration > queueConf)" should be done to reduce the amount of unnesecary loops. The > attached patch have been tested to work with atleast 96000 queues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4169) jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
[
https://issues.apache.org/jira/browse/YARN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14910034#comment-14910034
]
Rohith Sharma K S commented on YARN-4169:
-
Hi [~Naganarasimha],
Quickly gone through the patch.. some comments.
# I see many sleep int the tests. I have doubt that just adding sleep would
solve random failure permanently. Again it depends on the machine it is running
and other several factors. If wants to check NodeStausUpdator is in waiting
state, probably can get thread state OR wait for thread state.
# Not related to this JIRA, can you add timeout for the tests.
> jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
> -
>
> Key: YARN-4169
> URL: https://issues.apache.org/jira/browse/YARN-4169
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
> Environment: Jenkins
>Reporter: Steve Loughran
>Assignee: Naganarasimha G R
>Priority: Critical
> Attachments: YARN-4169.v1.001.patch, YARN-4169.v1.002.patch
>
>
> Test failing in [[Jenkins build
> 402|https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/402/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestNodeStatusUpdaterForLabels/testNodeStatusUpdaterForNodeLabels/]
> {code}
> java.lang.NullPointerException: null
> at java.util.HashSet.(HashSet.java:118)
> at
> org.apache.hadoop.yarn.nodelabels.NodeLabelTestBase.assertNLCollectionEquals(NodeLabelTestBase.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels.testNodeStatusUpdaterForNodeLabels(TestNodeStatusUpdaterForLabels.java:268)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4205) Add a service for monitoring application life time out
[
https://issues.apache.org/jira/browse/YARN-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14910062#comment-14910062
]
Rohith Sharma K S commented on YARN-4205:
-
It seems build machine has issue. I will kick off the jenkins again. I applied
the patch locally and run the tests. All tests cases are running.
Some comments
# In {{RMAppLifeTimeMonitorService}} : Along with {{if
(!application.isAppFinalStateStored()) {}}, also should handle states such as
FINAL_SAVING, KILLING. Probably these states can be predefined in static
variables.
# In test, some of the commented lines can be removed if not used.
> Add a service for monitoring application life time out
> --
>
> Key: YARN-4205
> URL: https://issues.apache.org/jira/browse/YARN-4205
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
>Reporter: nijel
>Assignee: nijel
> Attachments: YARN-4205_01.patch, YARN-4205_02.patch
>
>
> This JIRA intend to provide a lifetime monitor service.
> The service will monitor the applications where the life time is configured.
> If the application is running beyond the lifetime, it will be killed.
> The lifetime will be considered from the submit time.
> The thread monitoring interval is configurable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4066) Large number of queues choke fair scheduler
[
https://issues.apache.org/jira/browse/YARN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14910003#comment-14910003
]
Hadoop QA commented on YARN-4066:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 42s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 50s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 2s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 50s | The applied patch generated 4
new checkstyle issues (total was 15, now 18). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 30s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 29s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 60m 3s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 99m 25s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12762628/YARN-4066-2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1c030c6 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9279/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9279/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9279/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9279/console |
This message was automatically generated.
> Large number of queues choke fair scheduler
> ---
>
> Key: YARN-4066
> URL: https://issues.apache.org/jira/browse/YARN-4066
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Affects Versions: 2.7.1
>Reporter: Johan Gustavsson
>Assignee: Johan Gustavsson
> Attachments: YARN-4066-2.patch, yarn-4066-1.patch
>
>
> Due to synchronization and all the loops performed during queue creation,
> setting a large amount of queues (12000+) will completely choke the
> scheduler. To deal with this some optimization to
> "QueueManager.updateAllocationConfiguration(AllocationConfiguration
> queueConf)" should be done to reduce the amount of unnesecary loops. The
> attached patch have been tested to work with atleast 96000 queues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
