[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073267#comment-14073267 ] Hudson commented on YARN-2147: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1841 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1841/]) YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1612950) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.6.0 > > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073209#comment-14073209 ] Hudson commented on YARN-2147: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1814 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1814/]) YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1612950) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.6.0 > > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073098#comment-14073098 ] Hudson commented on YARN-2147: -- FAILURE: Integrated in Hadoop-Yarn-trunk #622 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/622/]) YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1612950) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.6.0 > > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072426#comment-14072426 ] Hudson commented on YARN-2147: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5956 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5956/]) YARN-2147. client lacks delegation token exception details when application submit fails. Contributed by Chen He (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1612950) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Fix For: 3.0.0, 2.6.0 > > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072390#comment-14072390 ] Jason Lowe commented on YARN-2147: -- The test timeouts were an artifact of a period where we were requiring each test to have a timeout to work around a surefire timeout bug, but we no longer need each test to have one. It's not going to hurt if present even for tests that shouldn't need them as long as the timeout is reasonable for the test. +1 lgtm. Committing this. > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055269#comment-14055269 ] Chen He commented on YARN-2147: --- all tests in this class has timeout, just to make it consistent. Thank you for your comment, [~vvasudev]. > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048180#comment-14048180
]
Hadoop QA commented on YARN-2147:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12653227/YARN-2147-v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4150//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4150//console
This message is automatically generated.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch,
> YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048118#comment-14048118 ] Varun Vasudev commented on YARN-2147: - The patch looks mostly good. Agree with [~daryn] on testing the message. Question about the timeout - it doesn't look like the test needs a timeout, any particular reason why you added it? > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147-v5.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047946#comment-14047946 ] Daryn Sharp commented on YARN-2147: --- Code looks fine. Currently the test verifies the stringified token is in the exception's message. However since the mock is throwing an exception explicitly with the stringified token, we don't know if the code change is actually catching and adding the token. The mock should throw a generic string of say, "boom". Then check the caught exception against something like "Failed to renew token: : boom". > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, > YARN-2147-v4.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039375#comment-14039375
]
Hadoop QA commented on YARN-2147:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651718/YARN-2147-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4034//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4034//console
This message is automatically generated.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch,
> YARN-2147-v4.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039271#comment-14039271
]
Hadoop QA commented on YARN-2147:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651709/YARN-2147-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4033//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4033//console
This message is automatically generated.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039220#comment-14039220 ] Chen He commented on YARN-2147: --- Thank you for the comments, [~daryn]. I updated the v3 patch with your suggestion. > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Attachments: YARN-2147-v2.patch, YARN-2147-v3.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034612#comment-14034612
]
Daryn Sharp commented on YARN-2147:
---
I don't think the patch handles the use case it's designed for. If job
submission failed with a bland "Read timed out", then logging all the tokens in
the RM log doesn't help the end user, nor does the RM log even answer the
question "which token timed out"?
What you really want to do is change
{{DelegationTokenRenewer#handleAppSubmitEvent}} to trap exceptions from
{{renewToken}}. Wrap the exception with a more descriptive exception that
stringifies to the user as "Can't renew token : Read timed out".
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032502#comment-14032502
]
Hadoop QA commented on YARN-2147:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650567/YARN-2147-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3997//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3997//console
This message is automatically generated.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147-v2.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[ https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032441#comment-14032441 ] Chen He commented on YARN-2147: --- Thank you for the comment, [~ozawa]. Patch updated. > client lacks delegation token exception details when application submit fails > - > > Key: YARN-2147 > URL: https://issues.apache.org/jira/browse/YARN-2147 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Chen He >Priority: Minor > Attachments: YARN-2147-v2.patch, YARN-2147-v2.patch, YARN-2147.patch > > > When an client submits an application and the delegation token process fails > the client can lack critical details needed to understand the nature of the > error. Only the message of the error exception is conveyed to the client, > which sometimes isn't enough to debug. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032071#comment-14032071
]
Tsuyoshi OZAWA commented on YARN-2147:
--
[~airbots], thank you for taking this JIRA.
One minor nits from me: Should we add spaces before and after the {{!=}} and
before {{(}}?
{code}
+if(credentials!=null) {
{code}
Otherwise changes looks good to me.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029968#comment-14029968
]
Hadoop QA commented on YARN-2147:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650154/YARN-2147-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3975//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3975//console
This message is automatically generated.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147-v2.patch, YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029871#comment-14029871
]
Hadoop QA commented on YARN-2147:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12650136/YARN-2147.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3974//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3974//console
This message is automatically generated.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Chen He
>Priority: Minor
> Attachments: YARN-2147.patch
>
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2147) client lacks delegation token exception details when application submit fails
[
https://issues.apache.org/jira/browse/YARN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027979#comment-14027979
]
Jason Lowe commented on YARN-2147:
--
For example, here's a sample log from a client submitting a job that failed:
{noformat}
2014-05-14 10:36:16,111 [JobControl] INFO
org.apache.hadoop.mapred.ResourceMgrDelegate - Submitted application
application_1394826486018_9924515 to ResourceManager at xx/xx:xx
2014-05-14 10:36:16,116 [JobControl] INFO
org.apache.hadoop.mapreduce.JobSubmitter - Cleaning up the staging area
/user/xx/.staging/job_1394826486018_9924515
2014-05-14 10:36:16,117 [JobControl] ERROR
org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException
as:xx (auth:SIMPLE) cause:java.io.IOException: Failed to run job : Read timed
out
2014-05-14 10:36:16,118 [JobControl] INFO
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob - xx got an error
while submitting
java.io.IOException: Failed to run job : Read timed out
at
org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:410)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1284)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
at
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at
org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
{noformat}
All the user sees is a read timeout but no details as to where it was
connecting or what service was involved. Was this a timeout connecting to the
RM? A timeout on the RM side? Something else entirely? Hard to tell from
just "Read timed out". Looking at the exception logged at the RM side the full
stacktrace shows that it was timing out trying to grab a delegation token from
a remote server for webhdfs. Those kinds of details need to be conveyed back
to the client, either via the full stacktrace from the RM exception or via a
more informative exception message when delegation token renewal fails during
app submission.
> client lacks delegation token exception details when application submit fails
> -
>
> Key: YARN-2147
> URL: https://issues.apache.org/jira/browse/YARN-2147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Priority: Minor
>
> When an client submits an application and the delegation token process fails
> the client can lack critical details needed to understand the nature of the
> error. Only the message of the error exception is conveyed to the client,
> which sometimes isn't enough to debug.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
