[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370058#comment-15370058
]
Junping Du commented on YARN-5297:
--
Thanks [~varun_saxena] for review and commit. I think it is OK to go to
branch-2.9 given this is not a critical issue.
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Fix For: 2.9.0
>
> Attachments: YARN-5297-v2.patch, YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368205#comment-15368205
]
Hudson commented on YARN-5297:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #10068 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10068/])
YARN-5297. Avoid printing a stack trace when recovering an app after the
(varunsaxena: rev 5252562edf1f65a9c5d502016df8bb92fbe78095)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297-v2.patch, YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368203#comment-15368203
]
Varun Saxena commented on YARN-5297:
Committed to trunk, branch-2.
[~djp], is it required to go in branch-2.8 ?
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297-v2.patch, YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368145#comment-15368145
]
Varun Saxena commented on YARN-5297:
Thanks [~djp] for the replies.
bq. I think we should remove RM audit log here to be consistent with other
exception handling cases nearby.
Yeah there are no audit logs in surrounding cases. I am ok with removing it but
wasnt sure if it wont be used by anyone.
bq. finishApplicationMaster is a different case and the log message level is
error instead of info.
Missed the error log. We would not have a way out then. As you said, it would
be printed in rare cases anyways.
Checkstyle is unrelated.
Will commit the patch.
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297-v2.patch, YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368105#comment-15368105
]
Junping Du commented on YARN-5297:
--
bq. should we be removing the audit log ? Typically in production, audit logs
will go into a separate log file. And some people may use it as an input in
some of their tools as well. We in our company do not use it but are we sure
this audit log wont be useful to any of the users?
I think we should remove RM audit log here to be consistent with other
exception handling cases nearby. I didn't see how useful for audit log here in
logging a commonly happen event.
bq. Moreover, ApplicationMasterNotRegisteredException is thrown from
ApplicationMasterService#finishApplicationMaster as well.
finishApplicationMaster is a different case and the log message level is error
instead of info. There are very rarely case when AM just about to finish while
RM get restarted, so logs (include audit log) is not very annoying but could be
helpful to remind something wrong here.
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297-v2.patch, YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368048#comment-15368048
]
Varun Saxena commented on YARN-5297:
[~djp], should we be removing the audit log ? Typically in production, audit
logs will go into a separate log file. And some people may use it as an input
in some of their tools as well. We in our company do not use it but are we sure
this audit log wont be useful to any of the users ?
Moreover, ApplicationMasterNotRegisteredException is thrown from
ApplicationMasterService#finishApplicationMaster as well. We will have 3 logs
in this flow as well.
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297-v2.patch, YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367987#comment-15367987
]
Hadoop QA commented on YARN-5297:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 36s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 35s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 17s
{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 31 unchanged - 3 fixed = 32 total (was 34) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 36m 22s
{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 44s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12816846/YARN-5297-v2.patch |
| JIRA Issue | YARN-5297 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0cd0a5783ac1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 30ee57c |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/12230/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/12230/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/12230/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Avoid printing a stack trace when recovering an app
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362766#comment-15362766
]
Junping Du commented on YARN-5297:
--
That's a nice catch, [~varun_saxena]. I think we can remove 2 duplicated one in
ApplicationMasterService. Will update a patch.
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15356646#comment-15356646
]
Varun Saxena commented on YARN-5297:
Thanks [~djp] for the patch.
The change looks fine. We however will now have 3 log statements one after
another talking about the same thing, one of them being an audit log though.
Should we remove the corresponding log in ApplicationMasterService (on allocate
and unregistration of AM) ? We can instead choose to suppress the exception too.
{noformat}
2016-06-30 12:18:37,207 [IPC Server handler 11 on 35017] INFO
org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: AM is
not registered for known application attempt:
appattempt_1467269207554_0001_01 or RM had restarted after AM registered .
AM should re-register.
2016-06-30 12:18:37,207 [IPC Server handler 11 on 35017] WARN
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=root
IP=192.168.90.188 OPERATION=App Master Heartbeats
TARGET=ApplicationMasterService RESULT=FAILURE DESCRIPTION=AM is not
registered for known application attempt: appattempt_1467269207554_0001_01
or RM had restarted after AM registered . AM should re-register. PERMISSIONS=
APPID=application_1467269207554_0001
APPATTEMPTID=appattempt_1467269207554_0001_01
2016-06-30 12:18:37,209 [IPC Server handler 11 on 35017] INFO
org.apache.hadoop.ipc.Server: IPC Server handler 11 on 35017, call
org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
192.168.90.188:53563 Call#34 Retry#0:
org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
is not registered for known application attempt:
appattempt_1467269207554_0001_01 or RM had restarted after AM registered .
AM should re-register.
{noformat}
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
> Attachments: YARN-5297.patch
>
>
> The exception trace is unnecessary, and can cause confusion.
> {code}
> 2016-06-16 22:02:54,262 INFO ipc.Server (Server.java:logException(2401)) -
> IPC Server handler 0 on 8030, call
> org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB.allocate from
> 172.22.79.149:42698 Call#2241 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationMasterNotRegisteredException: AM
> is not registered for known application attempt:
> appattempt_1466112179488_0001_01 or RM had restarted after AM registered
> . AM should re-register.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:454)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
> at
> org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> {code}
> cc [~djp]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15356522#comment-15356522
]
Hadoop QA commented on YARN-5297:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 34m 55s
{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:85209cc |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12813825/YARN-5297.patch |
| JIRA Issue | YARN-5297 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 6b1dc5de4c8c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 846ada2 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/12164/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/12164/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Avoid printing a stack trace when recovering an app after the RM restarts
> -
>
> Key: YARN-5297
> URL: https://issues.apache.org/jira/browse/YARN-5297
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Junping Du
>
[jira] [Commented] (YARN-5297) Avoid printing a stack trace when recovering an app after the RM restarts
[
https://issues.apache.org/jira/browse/YARN-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351925#comment-15351925
]
Hadoop QA commented on YARN-5297:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 35s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 40s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 2s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 37m 14s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
| | hadoop.yarn.server.resourcemanager.TestRMAdminService |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:85209cc |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12813825/YARN-5297.patch |
| JIRA Issue | YARN-5297 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux dc603cb40fd9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 9683eab |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/12136/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-YARN-Build/12136/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/12136/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
