[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973100#comment-14973100 ] Hudson commented on YARN-3738: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #593 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/593/]) YARN-3738. Add support for recovery of reserved apps running under (arun suresh: rev ab8eb8770c8b8bff41dacb1a399d75906abb1ac4) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/ReservationSystemTestUtil.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestWorkPreservingRMRestart.java > Add support for recovery of reserved apps running under dynamic queues > -- > > Key: YARN-3738 > URL: https://issues.apache.org/jira/browse/YARN-3738 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Fix For: 2.8.0 > > Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch, > YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Capacity Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973068#comment-14973068 ] Hudson commented on YARN-3738: -- FAILURE: Integrated in Hadoop-trunk-Commit #8703 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8703/]) YARN-3738. Add support for recovery of reserved apps running under (arun suresh: rev ab8eb8770c8b8bff41dacb1a399d75906abb1ac4) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestWorkPreservingRMRestart.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/ReservationSystemTestUtil.java > Add support for recovery of reserved apps running under dynamic queues > -- > > Key: YARN-3738 > URL: https://issues.apache.org/jira/browse/YARN-3738 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Fix For: 2.8.0 > > Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch, > YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Capacity Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973062#comment-14973062 ] Subru Krishnan commented on YARN-3738: -- Thanks [~adhoot] for reviewing and [~asuresh] for committing the patch. > Add support for recovery of reserved apps running under dynamic queues > -- > > Key: YARN-3738 > URL: https://issues.apache.org/jira/browse/YARN-3738 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Fix For: 2.8.0 > > Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch, > YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Capacity Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (YARN-3737) Add support for recovery of reserved apps (running under dynamic queues) to Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh resolved YARN-3737. --- Resolution: Done Fix Version/s: 2.8.0 > Add support for recovery of reserved apps (running under dynamic queues) to > Fair Scheduler > -- > > Key: YARN-3737 > URL: https://issues.apache.org/jira/browse/YARN-3737 > Project: Hadoop YARN > Issue Type: Sub-task > Components: fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Fix For: 2.8.0 > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Fair Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973052#comment-14973052 ] Arun Suresh commented on YARN-3738: --- Committed this to trunk and branch-2, Thanks for the patch [~subru] and for the review [~adhoot] !! > Add support for recovery of reserved apps running under dynamic queues > -- > > Key: YARN-3738 > URL: https://issues.apache.org/jira/browse/YARN-3738 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Fix For: 2.8.0 > > Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch, > YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Capacity Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3738) Add support for recovery of reserved apps running under dynamic queues
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-3738: -- Summary: Add support for recovery of reserved apps running under dynamic queues (was: Add support for recovery of reserved apps (running under dynamic queues) to Capacity Scheduler) > Add support for recovery of reserved apps running under dynamic queues > -- > > Key: YARN-3738 > URL: https://issues.apache.org/jira/browse/YARN-3738 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch, > YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Capacity Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps (running under dynamic queues) to Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973046#comment-14973046 ] Arun Suresh commented on YARN-3738: --- I agree that not much can be done about the checkstyle warning and the tests seem to run fine locally for me as well. Committing this shortly.. > Add support for recovery of reserved apps (running under dynamic queues) to > Capacity Scheduler > -- > > Key: YARN-3738 > URL: https://issues.apache.org/jira/browse/YARN-3738 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch, > YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch > > > YARN-3736 persists the current state of the Plan to the RMStateStore. This > JIRA covers recovery of the Plan, i.e. dynamic reservation queues with > associated apps as part Capacity Scheduler failover mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4292) ResourceUtilization should be a part of NodeInfo REST API
[
https://issues.apache.org/jira/browse/YARN-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4292:
--
Attachment: 0002-YARN-4292.patch
Adding a new test case in new patch.
Also attaching a REST api o/p from real cluster.
{code}
{
nodes: {
node: [1]
0: {
rack: "/default-rack"
state: "RUNNING"
id: "localhost:25006"
nodeHostName: "localhost"
nodeHTTPAddress: "localhost:25008"
lastHealthUpdate: 1445746317950
version: "3.0.0-SNAPSHOT"
healthReport: ""
numContainers: 0
usedMemoryMB: 0
availMemoryMB: 8192
usedVirtualCores: 0
availableVirtualCores: 8
nodePhysicalMemoryMB: 4641
nodeVirtualMemoryMB: 0
nodeCPUUsage: 10.576282501220703
containersPhysicalMemoryMB: 0
containersVirtualMemoryMB: 0
containersCPUUsage: 0
}-
-
}-
}
{code}
> ResourceUtilization should be a part of NodeInfo REST API
> -
>
> Key: YARN-4292
> URL: https://issues.apache.org/jira/browse/YARN-4292
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Sunil G
> Attachments: 0001-YARN-4292.patch, 0002-YARN-4292.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4224) Change the ATSv2 reader side REST interface to conform to current REST APIs' in YARN
[
https://issues.apache.org/jira/browse/YARN-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972877#comment-14972877
]
Hadoop QA commented on YARN-4224:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 24s | Findbugs (version ) appears to
be broken on YARN-2928. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 8m 2s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 31s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 19s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 14s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 13s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 43s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 52s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 3m 32s | Tests passed in
hadoop-yarn-server-timelineservice. |
| | | 42m 30s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12768555/YARN-4224-YARN-2928.01.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / 3c4e424 |
| Release Audit |
https://builds.apache.org/job/PreCommit-YARN-Build/9562/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| hadoop-yarn-server-timelineservice test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9562/artifact/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9562/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9562/console |
This message was automatically generated.
> Change the ATSv2 reader side REST interface to conform to current REST APIs'
> in YARN
>
>
> Key: YARN-4224
> URL: https://issues.apache.org/jira/browse/YARN-4224
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Varun Saxena
>Assignee: Varun Saxena
> Attachments: YARN-4224-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4224) Change the ATSv2 reader side REST interface to conform to current REST APIs' in YARN
[ https://issues.apache.org/jira/browse/YARN-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-4224: --- Attachment: YARN-4224-YARN-2928.01.patch > Change the ATSv2 reader side REST interface to conform to current REST APIs' > in YARN > > > Key: YARN-4224 > URL: https://issues.apache.org/jira/browse/YARN-4224 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Attachments: YARN-4224-YARN-2928.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4283) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[
https://issues.apache.org/jira/browse/YARN-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972815#comment-14972815
]
Hadoop QA commented on YARN-4283:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 24m 21s | Pre-patch HADOOP-11890 has 4
extant Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 49s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 22s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 3m 3s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 2s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 6m 3s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 2m 1s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 0m 26s | Tests passed in
hadoop-yarn-server-common. |
| {color:green}+1{color} | yarn tests | 9m 0s | Tests passed in
hadoop-yarn-server-nodemanager. |
| {color:green}+1{color} | yarn tests | 62m 6s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| {color:green}+1{color} | yarn tests | 0m 24s | Tests passed in
hadoop-yarn-server-web-proxy. |
| | | 128m 11s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12768539/YARN-4283-HADOOP-11890.1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HADOOP-11890 / fa64e68 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/HADOOP-11890FindbugsWarningshadoop-yarn-common.html
|
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/HADOOP-11890FindbugsWarningshadoop-yarn-server-nodemanager.html
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-web-proxy test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9561/console |
This message was automatically generated.
> hadoop-yarn Avoid unsafe split and append on fields that might be IPv6
> literals
> ---
>
> Key: YARN-4283
> URL: https://issues.apache.org/jira/browse/YARN-4283
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Nemanja Matkovic
>Assignee: Nemanja Matkovic
> Labels: ipv6
> Attachments: YARN-4283-HADOOP-11890.1.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> hadoop-yarn part of HADOOP-12122 task
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972779#comment-14972779
]
Steve Loughran commented on YARN-4251:
--
Edward; not related to this JIRA. You can open anotherone saying "cannot bind
[0.0.0.0:0] java.net.BindException: Address already in use;"
but if you do that, I'm going to say "have you followed the wiki link? Did you
follow the steps to diagnose?". So do that first, OK?
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
> at org.apache.hadoop.ipc.Server.bind(Server.java:486)
> at org.apache.hadoop.ipc.Server$Listener.(Server.java:646)
> at org.apache.hadoop.ipc.Server.(Server.java:2399)
> at org.apache.hadoop.ipc.RPC$Server.(RPC.java:946)
> at
[jira] [Commented] (YARN-4221) Store user in app to flow table
[ https://issues.apache.org/jira/browse/YARN-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972747#comment-14972747 ] Varun Saxena commented on YARN-4221: Thanks [~sjlee0] for the review and commit. > Store user in app to flow table > --- > > Key: YARN-4221 > URL: https://issues.apache.org/jira/browse/YARN-4221 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Varun Saxena >Assignee: Varun Saxena > Fix For: YARN-2928 > > Attachments: YARN-4221-YARN-2928.01.patch, > YARN-4221-YARN-2928.02.patch, YARN-4221-YARN-2928.03.patch > > > We should store user as well in in app to flow table. > For queries where user is not supplied and flow context can be retrieved from > app to flow table, we should take the user from app to flow table instead of > considering UGI as default user. > This is as per discussion on YARN-3864 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972737#comment-14972737
]
Edward Capriolo commented on YARN-4251:
---
I did post this to the ML, and CDH user lists. I'm digging deeper because no
one there answers. I don't know how any version of YARN could ship where
launching 2-3 jobs at once causes prod issues without someone noticing it.
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
> at org.apache.hadoop.ipc.Server.bind(Server.java:486)
> at org.apache.hadoop.ipc.Server$Listener.(Server.java:646)
> at org.apache.hadoop.ipc.Server.(Server.java:2399)
> at org.apache.hadoop.ipc.RPC$Server.(RPC.java:946)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Se
[jira] [Resolved] (YARN-4295) Fix Dodgy code Warnings in ContainersLauncher
[
https://issues.apache.org/jira/browse/YARN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula resolved YARN-4295.
Resolution: Duplicate
> Fix Dodgy code Warnings in ContainersLauncher
> -
>
> Key: YARN-4295
> URL: https://issues.apache.org/jira/browse/YARN-4295
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>
> {noformat}
> BCUnchecked/unconfirmed cast from
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent
> to
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.SignalContainersLauncherEvent
> in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncherEvent)
> Bug type BC_UNCONFIRMED_CAST (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncherEvent)
> Actual type
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent
> Expected
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.SignalContainersLauncherEvent
> Value loaded from event
> At ContainersLauncher.java:[line 146]
> {noformat}
> https://builds.apache.org/job/PreCommit-YARN-Build/9550/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-nodemanager.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4283) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/YARN-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated YARN-4283: --- Attachment: YARN-4283-HADOOP-11890.1.patch Same patch as in MAPREDUCE-6523 just in YARN > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: YARN-4283 > URL: https://issues.apache.org/jira/browse/YARN-4283 > Project: Hadoop YARN > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: YARN-4283-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > hadoop-yarn part of HADOOP-12122 task -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4283) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/YARN-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972706#comment-14972706 ] Nemanja Matkovic commented on YARN-4283: First patch posted here, moving back to YARN-4283 > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: YARN-4283 > URL: https://issues.apache.org/jira/browse/YARN-4283 > Project: Hadoop YARN > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Original Estimate: 48h > Remaining Estimate: 48h > > hadoop-yarn part of HADOOP-12122 task -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (YARN-4283) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/YARN-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic reopened YARN-4283: Assignee: Nemanja Matkovic Reopening as now I was added as contributor and can make progress on YARN jira. > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: YARN-4283 > URL: https://issues.apache.org/jira/browse/YARN-4283 > Project: Hadoop YARN > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Original Estimate: 48h > Remaining Estimate: 48h > > hadoop-yarn part of HADOOP-12122 task -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972697#comment-14972697
]
Hudson commented on YARN-4289:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #534 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/534/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
* hadoop-yarn-project/CHANGES.txt
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972696#comment-14972696
]
Hudson commented on YARN-4296:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #534 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/534/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972678#comment-14972678
]
Brahma Reddy Battula commented on YARN-4251:
[~appodictic] *As it's not related to this jira* , Please post this query in
[user mailing list| http://hadoop.apache.org/mailing_lists.html] with logs,it
can be analysed further.
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
> at org.apache.hadoop.ipc.Server.bind(Server.java:486)
> at org.apache.hadoop.ipc.Server$Listener.(Server.java:646)
> at org.apache.hadoop.ipc.Server.(Server.java:2399)
> at org.apache.hadoop.ipc.RPC$Server.(RPC.java:946)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server.(ProtobufRpcEngine.java:537)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972673#comment-14972673
]
Hudson commented on YARN-4296:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2471 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2471/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972674#comment-14972674
]
Hudson commented on YARN-4289:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2471 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2471/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972646#comment-14972646
]
Edward Capriolo commented on YARN-4251:
---
I can see now that this patch is ONLY for a test case. So I am having the
problem in production. I get this problem binding to 0.0.0,0:0.
Looking around I have found, which I think to be the related property:
yarn.app.mapreduce.am.job.client.port-range
Range of ports that the MapReduce AM can use when binding.
Leave blank if you want all possible ports.
For example 5-50050,50100-50200
As far as I can tell it is set blank. so I do not understand why if has port
conflicts. That should mean it can use ANY port.
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtil
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972639#comment-14972639
]
Brahma Reddy Battula commented on YARN-4251:
[~appodictic] do you mean, you applied this patch and still you are getting
bind exception..? can you provide the logs such that I can look into your logs
like why same port is got allocated..?
{quote}Does ServerSocketUtil.getPort(45020, 10)); mean that we are using a
range of 10 ports? That does not seem like many concurrent jobs?{quote}
no, it will check whether 45020 port is free or not, If not free,it will retry
10 times(while retrying, try to get random free port)...
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
> at org.apache.hadoop.ipc.Se
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972634#comment-14972634
]
Edward Capriolo commented on YARN-4251:
---
To be clear I sometimes get that see the above error when running multiple
concurrent map/reduce processes.
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
> at org.apache.hadoop.ipc.Server.bind(Server.java:486)
> at org.apache.hadoop.ipc.Server$Listener.(Server.java:646)
> at org.apache.hadoop.ipc.Server.(Server.java:2399)
> at org.apache.hadoop.ipc.RPC$Server.(RPC.java:946)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server.(ProtobufRpcEngine.java:537)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.j
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972631#comment-14972631
]
Edward Capriolo commented on YARN-4251:
---
Does ServerSocketUtil.getPort(45020, 10)); mean that we are using a range of
10 ports? That does not seem like many concurrent jobs?
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1052)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart$MyResourceManager.serviceStart(TestAMRMClientOnRMRestart.java:560)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart.testAMRMClientOnAMRMTokenRollOverOnRMRestart(TestAMRMClientOnRMRestart.java:463)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
> at org.apache.hadoop.ipc.Server.bind(Server.java:486)
> at org.apache.hadoop.ipc.Server$Listener.(Server.java:646)
> at org.apache.hadoop.ipc.Server.(Server.java:2399)
> at org.apache.hadoop.ipc.RPC$Server.(RPC.java:946)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server.(ProtobufRpcEngine.java:537)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine.getS
[jira] [Commented] (YARN-4251) TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing
[
https://issues.apache.org/jira/browse/YARN-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972629#comment-14972629
]
Edward Capriolo commented on YARN-4251:
---
This fix does not help. I still get the same message binding to 0.0.0.0:0. With
whatever gets bunded with cloudera 5.4.2
at
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
{noformat}
at sun.nio.ch.Net.bind(Net.java:436)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.apache.hadoop.ipc.Server.bind(Server.java:407)
... 19 more
2015-10-04 19:31:10,567 INFO [main] org.apache.hadoop.service.AbstractService:
Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.net.BindException: Problem binding to [0.0.0.0:0] java.net.BindException:
Address already in use; For more details see:
http://wiki.apache.org/hadoop/BindException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException:
Problem binding to [0.0.0.0:0] java.net.BindException: Address already in use;
For more details see: http://wiki.apache.org/hadoop/BindException
at
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
at
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:119)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1084)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1500)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1496)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429)
Caused by: java.net.BindException: Problem binding to [0.0.0.0:0]
java.net.BindException: Address already in use; For more details see:
http://wiki.apache.org/hadoop/BindException
{noformat}
Can we re-open?
> TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is
> failing
> -
>
> Key: YARN-4251
> URL: https://issues.apache.org/jira/browse/YARN-4251
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: YARN-4251.patch
>
>
> *Trace*
> {noformat}
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:9030]
> java.net.BindException: Address already in use: bind; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:143)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:592)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:975)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1016)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1667)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1012)
> at
> org.apach
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972621#comment-14972621
]
Hudson commented on YARN-4289:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #592 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/592/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
* hadoop-yarn-project/CHANGES.txt
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972620#comment-14972620
]
Hudson commented on YARN-4296:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #592 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/592/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
* hadoop-yarn-project/CHANGES.txt
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972618#comment-14972618 ] Pradeep Subrahmanion commented on YARN-1565: Findbugs doesn't list any new warnings. No new issues in latest result. [~ste...@apache.org] - Could you please take a look at latest patch ? > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch, YARN-1565-004.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core &c for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3848) TestNodeLabelContainerAllocation is timing out
[ https://issues.apache.org/jira/browse/YARN-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972606#comment-14972606 ] Varun Saxena commented on YARN-3848: Thanks [~ste...@apache.org] for your suggestion. > TestNodeLabelContainerAllocation is timing out > -- > > Key: YARN-3848 > URL: https://issues.apache.org/jira/browse/YARN-3848 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3848.01.patch, test_output.txt > > > A number of builds, pre-commit and otherwise, have been failing recently > because TestNodeLabelContainerAllocation has timed out. See > https://builds.apache.org/job/Hadoop-Yarn-trunk/969/, YARN-3830, YARN-3802, > or YARN-3826 for examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4295) Fix Dodgy code Warnings in ContainersLauncher
[
https://issues.apache.org/jira/browse/YARN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972600#comment-14972600
]
Varun Saxena commented on YARN-4295:
Dupe of YARN-4223
> Fix Dodgy code Warnings in ContainersLauncher
> -
>
> Key: YARN-4295
> URL: https://issues.apache.org/jira/browse/YARN-4295
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>
> {noformat}
> BCUnchecked/unconfirmed cast from
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent
> to
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.SignalContainersLauncherEvent
> in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncherEvent)
> Bug type BC_UNCONFIRMED_CAST (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncherEvent)
> Actual type
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent
> Expected
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.SignalContainersLauncherEvent
> Value loaded from event
> At ContainersLauncher.java:[line 146]
> {noformat}
> https://builds.apache.org/job/PreCommit-YARN-Build/9550/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-nodemanager.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972590#comment-14972590
]
Hudson commented on YARN-4296:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2524 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2524/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972591#comment-14972591
]
Hudson commented on YARN-4289:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2524 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2524/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
* hadoop-yarn-project/CHANGES.txt
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972577#comment-14972577
]
Hudson commented on YARN-4289:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #580 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/580/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
* hadoop-yarn-project/CHANGES.txt
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972576#comment-14972576
]
Hudson commented on YARN-4296:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #580 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/580/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972567#comment-14972567
]
Hudson commented on YARN-4289:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #1315 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1315/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
* hadoop-yarn-project/CHANGES.txt
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972566#comment-14972566
]
Hudson commented on YARN-4296:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #1315 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1315/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
* hadoop-yarn-project/CHANGES.txt
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4296) DistributedShell Log.info is not friendly
[
https://issues.apache.org/jira/browse/YARN-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972560#comment-14972560
]
Hudson commented on YARN-4296:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8702 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8702/])
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via
(stevel: rev 446212a39ecf295cdd0e0b06ab870b48c5e6b500)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
> DistributedShell Log.info is not friendly
> -
>
> Key: YARN-4296
> URL: https://issues.apache.org/jira/browse/YARN-4296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
>Affects Versions: 2.5.2, 2.7.1
>Reporter: Xiaowei Wang
> Fix For: 2.8.0
>
> Attachments: YARN-4296.0.patch
>
>
> In java/org/apache/hadoop/yarn/applications/distributedshell/Client.java,
> line 453，Log.info is not friendly ,I think,nodeAddress and its content should
> be separated with "="
> The origin Log.info is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddresscloud101414328.wd.nm.ss.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddressrsync.cloud10141032027.wd.nm.nop:23999, nodeRackName/default-rack,
> nodeNumContainers19
> {noformat}
> A better way is
> {noformat}
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=cloud101414328.wd.nm.ss.nop:45454,
> nodeAddress=cloud101414328.wd.nm.ss.nop:23999, nodeRackName=/default-rack,
> nodeNumContainers=19
> 15/10/24 10:20:26 INFO distributedshell.Client: Got node report from ASM for,
> nodeId=rsync.cloud10141032027.wd.nm.nop:45454,
> nodeAddress=rsync.cloud10141032027.wd.nm.nop:23999,
> nodeRackName=/default-rack, nodeNumContainers=19
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4289) TestDistributedShell failing with bind exception
[
https://issues.apache.org/jira/browse/YARN-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972557#comment-14972557
]
Hudson commented on YARN-4289:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8701 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8701/])
YARN-4289. TestDistributedShell failing with bind exception. (Brahma (stevel:
rev 14f8dd04230ff42c5235eb240f84b8112c5302b5)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
> TestDistributedShell failing with bind exception
>
>
> Key: YARN-4289
> URL: https://issues.apache.org/jira/browse/YARN-4289
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0
>
> Attachments: YARN-4289.patch
>
>
> *Stacktrace*
> {noformat}
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> ApplicationHistoryServer failed to start. Final state is STOPPED
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper.serviceStart(MiniYARNCluster.java:726)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setupInternal(TestDistributedShell.java:91)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.setup(TestDistributedShell.java:71)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [0.0.0.0:10200]
> java.net.BindException: Address already in use; For more details see:
> http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryClientService.serviceStart(ApplicationHistoryClientService.java:93)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:118)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ApplicationHistoryServerWrapper$1.run(MiniYARNCluster.java:715)
> {noformat}
> https://builds.apache.org/job/Hadoop-Yarn-trunk/1290/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3528) Tests with 12345 as hard-coded port break jenkins
[
https://issues.apache.org/jira/browse/YARN-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-3528:
-
Fix Version/s: 2.8.0
> Tests with 12345 as hard-coded port break jenkins
> -
>
> Key: YARN-3528
> URL: https://issues.apache.org/jira/browse/YARN-3528
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.8.0
> Environment: ASF Jenkins
>Reporter: Steve Loughran
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Labels: test
> Fix For: 2.8.0
>
> Attachments: YARN-3528-002.patch, YARN-3528-003.patch,
> YARN-3528-004.patch, YARN-3528-005.patch, YARN-3528-006.patch,
> YARN-3528-007.patch, YARN-3528-008.patch, YARN-3528-009.patch,
> YARN-3528-branch2.patch, YARN-3528.patch
>
>
> A lot of the YARN tests have hard-coded the port 12345 for their services to
> come up on.
> This makes it impossible to have scheduled or precommit tests to run
> consistently on the ASF jenkins hosts. Instead the tests fail regularly and
> appear to get ignored completely.
> A quick grep of "12345" shows up many places in the test suite where this
> practise has developed.
> * All {{BaseContainerManagerTest}} subclasses
> * {{TestNodeManagerShutdown}}
> * {{TestContainerManager}}
> + others
> This needs to be addressed through portscanning and dynamic port allocation.
> Please can someone do this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4169) jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
[
https://issues.apache.org/jira/browse/YARN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972552#comment-14972552
]
Steve Loughran commented on YARN-4169:
--
TestNodeStatusUpdaterForLabels failing
{code}
Error Message: Heartbeat dint receive even after waiting
java.lang.AssertionError: Heartbeat dint receive even after waiting
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels$ResourceTrackerForLabels.waitTillHeartbeat(TestNodeStatusUpdaterForLabels.java:126)
at
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels.testNodeStatusUpdaterForNodeLabels(TestNodeStatusUpdaterForLabels.java:285)
{code}
> jenkins trunk+java build failed in TestNodeStatusUpdaterForLabels
> -
>
> Key: YARN-4169
> URL: https://issues.apache.org/jira/browse/YARN-4169
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
> Environment: Jenkins
>Reporter: Steve Loughran
>Assignee: Naganarasimha G R
>Priority: Critical
> Attachments: YARN-4162.v2.005.patch, YARN-4169.v1.001.patch,
> YARN-4169.v1.002.patch, YARN-4169.v1.003.patch, YARN-4169.v1.004.patch
>
>
> Test failing in [[Jenkins build
> 402|https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/402/testReport/junit/org.apache.hadoop.yarn.server.nodemanager/TestNodeStatusUpdaterForLabels/testNodeStatusUpdaterForNodeLabels/]
> {code}
> java.lang.NullPointerException: null
> at java.util.HashSet.(HashSet.java:118)
> at
> org.apache.hadoop.yarn.nodelabels.NodeLabelTestBase.assertNLCollectionEquals(NodeLabelTestBase.java:103)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels.testNodeStatusUpdaterForNodeLabels(TestNodeStatusUpdaterForLabels.java:268)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4294) [JDK8] Fix javadoc errors caused by wrong reference and illegal tag
[
https://issues.apache.org/jira/browse/YARN-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972517#comment-14972517
]
Hudson commented on YARN-4294:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #2469 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2469/])
YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and (aajisaka:
rev 7781fe1b9e89488b37217f81c246c1398407e47a)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptRequest.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsMappingProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptResponse.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMDelegatedNodeLabelsUpdater.java
> [JDK8] Fix javadoc errors caused by wrong reference and illegal tag
> ---
>
> Key: YARN-4294
> URL: https://issues.apache.org/jira/browse/YARN-4294
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, documentation
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Blocker
> Fix For: 2.8.0
>
> Attachments: YARN-4294.00.patch
>
>
> {{mvn package -Pdist -Dtar -DskipTests}} fails on JDK8 by illegal javadoc.
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptRequest.java:33:
> error: reference not found
> [ERROR] * @see
> ApplicationClientProtocol#failApplicationAttempt(FailApplicationAttemptRequest)
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptResponse.java:33:
> error: reference not found
> [ERROR] * @see
> ApplicationClientProtocol#failApplicationAttempt(FailApplicationAttemptRequest)
> [ERROR] ^
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3738) Add support for recovery of reserved apps (running under dynamic queues) to Capacity Scheduler
[
https://issues.apache.org/jira/browse/YARN-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972488#comment-14972488
]
Hadoop QA commented on YARN-3738:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 44s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 8m 7s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 51s | The applied patch generated 1
new checkstyle issues (total was 189, now 188). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 30s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 29s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 62m 10s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 103m 29s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12768455/YARN-3738-v4.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 7781fe1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9560/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9560/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9560/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9560/console |
This message was automatically generated.
> Add support for recovery of reserved apps (running under dynamic queues) to
> Capacity Scheduler
> --
>
> Key: YARN-3738
> URL: https://issues.apache.org/jira/browse/YARN-3738
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, resourcemanager
>Reporter: Subru Krishnan
>Assignee: Subru Krishnan
> Attachments: YARN-3738-v2.patch, YARN-3738-v3.patch,
> YARN-3738-v3.patch, YARN-3738-v4.patch, YARN-3738-v4.patch, YARN-3738.patch
>
>
> YARN-3736 persists the current state of the Plan to the RMStateStore. This
> JIRA covers recovery of the Plan, i.e. dynamic reservation queues with
> associated apps as part Capacity Scheduler failover mechanism.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4294) [JDK8] Fix javadoc errors caused by wrong reference and illegal tag
[
https://issues.apache.org/jira/browse/YARN-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972472#comment-14972472
]
Hudson commented on YARN-4294:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #533 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/533/])
YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and (aajisaka:
rev 7781fe1b9e89488b37217f81c246c1398407e47a)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptResponse.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMDelegatedNodeLabelsUpdater.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/RMNodeLabelsMappingProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptRequest.java
> [JDK8] Fix javadoc errors caused by wrong reference and illegal tag
> ---
>
> Key: YARN-4294
> URL: https://issues.apache.org/jira/browse/YARN-4294
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, documentation
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Blocker
> Fix For: 2.8.0
>
> Attachments: YARN-4294.00.patch
>
>
> {{mvn package -Pdist -Dtar -DskipTests}} fails on JDK8 by illegal javadoc.
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptRequest.java:33:
> error: reference not found
> [ERROR] * @see
> ApplicationClientProtocol#failApplicationAttempt(FailApplicationAttemptRequest)
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FailApplicationAttemptResponse.java:33:
> error: reference not found
> [ERROR] * @see
> ApplicationClientProtocol#failApplicationAttempt(FailApplicationAttemptRequest)
> [ERROR] ^
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
