[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170816#comment-14170816
]
Hudson commented on YARN-2377:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #711 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/711/])
YARN-2377. Localization exception stack traces are not passed as diagnostic
info. Contributed by Gera Shegalov (jlowe: rev
a56ea0100215ecf2e1471a18812b668658197239)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/SerializedException.java
* hadoop-yarn-project/CHANGES.txt
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Fix For: 2.6.0
Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170962#comment-14170962
]
Hudson commented on YARN-2377:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1901 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1901/])
YARN-2377. Localization exception stack traces are not passed as diagnostic
info. Contributed by Gera Shegalov (jlowe: rev
a56ea0100215ecf2e1471a18812b668658197239)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/SerializedException.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Fix For: 2.6.0
Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171042#comment-14171042
]
Hudson commented on YARN-2377:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1926 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1926/])
YARN-2377. Localization exception stack traces are not passed as diagnostic
info. Contributed by Gera Shegalov (jlowe: rev
a56ea0100215ecf2e1471a18812b668658197239)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/SerializedException.java
* hadoop-yarn-project/CHANGES.txt
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Fix For: 2.6.0
Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169700#comment-14169700
]
Jason Lowe commented on YARN-2377:
--
+1 latest patch lgtm. The audit failure is unrelated, see YARN-2667.
Committing this.
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169732#comment-14169732
]
Hudson commented on YARN-2377:
--
FAILURE: Integrated in Hadoop-trunk-Commit #6252 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/6252/])
YARN-2377. Localization exception stack traces are not passed as diagnostic
info. Contributed by Gera Shegalov (jlowe: rev
a56ea0100215ecf2e1471a18812b668658197239)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/SerializedException.java
* hadoop-yarn-project/CHANGES.txt
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Fix For: 2.6.0
Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164576#comment-14164576
]
Hadoop QA commented on YARN-2377:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12673802/YARN-2377.v02.patch
against trunk revision 2a51494.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5340//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5340//artifact/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5340//console
This message is automatically generated.
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch, YARN-2377.v02.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132856#comment-14132856
]
Naganarasimha G R commented on YARN-2377:
-
Hi [~jira.shegalov]
In your stringify method you are using
SerializedExceptionPBImpl#getMessage, #getCause #getRemoteTrace. Will that not
deserialize the exception ?
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132994#comment-14132994
]
Gera Shegalov commented on YARN-2377:
-
Hi [~Naganarasimha], there is no deserialization in a sense of converting bytes
to the original exception class. This fields are already strings in
yarn_protos.proto:
{code}
33 message SerializedExceptionProto {
34 optional string message = 1;
35 optional string trace = 2;
36 optional string class_name = 3;
37 optional SerializedExceptionProto cause = 4;
38 }
{code}
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14125846#comment-14125846
]
Gera Shegalov commented on YARN-2377:
-
[~kasha], do you agree with the points above?
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109479#comment-14109479
]
Karthik Kambatla commented on YARN-2377:
+1 to improving the debuggability here.
Can we re-use StringUtils.stringifyException, preferably in
ResourceLocalizationService?
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110221#comment-14110221
]
Gera Shegalov commented on YARN-2377:
-
Hi [~kasha], I considered {{StringUtils#stringifyException}} but discarded it
due to the following disadvantages:
# redundant deserialization of the exception object just for the sake of
serializing it right away
# as a consequence, hypothetically, when localization service runs as a
separate process with a dedicated classpath, we can encounter a
{{ClassNotFoundException}} during deserialization
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2377) Localization exception stack traces are not passed as diagnostic info
[
https://issues.apache.org/jira/browse/YARN-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14082064#comment-14082064
]
Hadoop QA commented on YARN-2377:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659061/YARN-2377.v01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4504//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4504//console
This message is automatically generated.
Localization exception stack traces are not passed as diagnostic info
-
Key: YARN-2377
URL: https://issues.apache.org/jira/browse/YARN-2377
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: YARN-2377.v01.patch
In the Localizer log one can only see this kind of message
{code}
14/07/31 10:29:00 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
hdfs://ha-nn-uri-0:8020/tmp/hadoop-yarn/staging/gshegalov/.staging/job_1406825443306_0004/job.jar,
1406827248944, PATTERN, (?:classes/|lib/).* }, java.net.UnknownHos
tException: ha-nn-uri-0
{code}
And then only {{ java.net.UnknownHostException: ha-nn-uri-0}} message is
propagated as diagnostics.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
