[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914392#comment-13914392 ] Hudson commented on MAPREDUCE-5754: --- FAILURE: Integrated in Hadoop-Yarn-trunk #494 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/494/]) MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1572269) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestEvents.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1393307629410_0001-1393307687476-user-Sleep+job-1393307723835-0-0-FAILED-default-1393307693920.jhist Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Fix For: 2.4.0 Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch, MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch, MAPREDUCE-5754.v05.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914526#comment-13914526 ] Hudson commented on MAPREDUCE-5754: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1686 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1686/]) MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1572269) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestEvents.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1393307629410_0001-1393307687476-user-Sleep+job-1393307723835-0-0-FAILED-default-1393307693920.jhist Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Fix For: 2.4.0 Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch, MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch, MAPREDUCE-5754.v05.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13914605#comment-13914605 ] Hudson commented on MAPREDUCE-5754: --- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1711 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1711/]) MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1572269) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestEvents.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1393307629410_0001-1393307687476-user-Sleep+job-1393307723835-0-0-FAILED-default-1393307693920.jhist Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Fix For: 2.4.0 Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch, MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch, MAPREDUCE-5754.v05.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13913192#comment-13913192 ] Jason Lowe commented on MAPREDUCE-5754: --- +1, lgtm. I'll commit this later today. Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch, MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch, MAPREDUCE-5754.v05.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13913551#comment-13913551 ] Hudson commented on MAPREDUCE-5754: --- SUCCESS: Integrated in Hadoop-trunk-Commit #5234 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5234/]) MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1572269) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestEvents.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/avro/Events.avpr * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobUnsuccessfulCompletionEvent.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1393307629410_0001-1393307687476-user-Sleep+job-1393307723835-0-0-FAILED-default-1393307693920.jhist Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Fix For: 2.4.0 Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch, MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch, MAPREDUCE-5754.v05.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[
https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911473#comment-13911473
]
Hadoop QA commented on MAPREDUCE-5754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12630923/MAPREDUCE-5754.v03.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4372//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4372//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4372//console
This message is automatically generated.
Preserve Job diagnostics in history
---
Key: MAPREDUCE-5754
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver, mr-am
Affects Versions: 2.2.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch,
MAPREDUCE-5754.v03.patch
History does not store the runtime diagnostics information. JobHistoryParser
tries to blame a task. We propose to preserve the original runtime
diagnostics that covers all the cases including the job being killed. This is
particularly important in the context of user-supplied diagnostic message as
in MAPREDUCE-5648.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[
https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13912281#comment-13912281
]
Hadoop QA commented on MAPREDUCE-5754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12631039/MAPREDUCE-5754.v04.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4374//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4374//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4374//console
This message is automatically generated.
Preserve Job diagnostics in history
---
Key: MAPREDUCE-5754
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver, mr-am
Affects Versions: 2.2.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch,
MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch
History does not store the runtime diagnostics information. JobHistoryParser
tries to blame a task. We propose to preserve the original runtime
diagnostics that covers all the cases including the job being killed. This is
particularly important in the context of user-supplied diagnostic message as
in MAPREDUCE-5648.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[
https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13912532#comment-13912532
]
Hadoop QA commented on MAPREDUCE-5754:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12631152/MAPREDUCE-5754.v05.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4377//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4377//console
This message is automatically generated.
Preserve Job diagnostics in history
---
Key: MAPREDUCE-5754
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver, mr-am
Affects Versions: 2.2.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch,
MAPREDUCE-5754.v03.patch, MAPREDUCE-5754.v04.patch, MAPREDUCE-5754.v05.patch
History does not store the runtime diagnostics information. JobHistoryParser
tries to blame a task. We propose to preserve the original runtime
diagnostics that covers all the cases including the job being killed. This is
particularly important in the context of user-supplied diagnostic message as
in MAPREDUCE-5648.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13910961#comment-13910961 ] Jason Lowe commented on MAPREDUCE-5754: --- The patch has gone stale and no longer applies cleanly. My apologies for not getting to this sooner. Some other comments on the patch: - Nit: Is it appropriate to put a particular diagnostic string in the Job interface? Wondering if this is more appropriately placed in JobImpl, as the wording of diagnostic messages seems implementation-specific and not something that necessarily belongs in the interface. - Timeout for TestEvents.testEvent was commented out. - Why are we joining diagnostics with ','? JobImpl.getReport() joins with '\n' so there would be some inconsistency. - Why does JobUnsuccessfulCompletionEvent.getDiagnostics() explicitly check for N/A and translate it to an empty string? Can we just have the Avro spec default to and remove this check or is there another use-case for this transform? - Did you run a test where the history server tried to parse an old history file generated before this change? It *should* work given the Avro default, but it would be nice to confirm since there were issues in the past with compatibility. Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[ https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1392#comment-1392 ] Gera Shegalov commented on MAPREDUCE-5754: -- Thanks [~jlowe] for a review. I tested the change on an old history file as this is a major concern for upgrades. I'll follow up on your other points as well. Preserve Job diagnostics in history --- Key: MAPREDUCE-5754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 2.2.0 Reporter: Gera Shegalov Assignee: Gera Shegalov Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch History does not store the runtime diagnostics information. JobHistoryParser tries to blame a task. We propose to preserve the original runtime diagnostics that covers all the cases including the job being killed. This is particularly important in the context of user-supplied diagnostic message as in MAPREDUCE-5648. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[
https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13898971#comment-13898971
]
Hadoop QA commented on MAPREDUCE-5754:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12628458/MAPREDUCE-5754.v02.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4353//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4353//console
This message is automatically generated.
Preserve Job diagnostics in history
---
Key: MAPREDUCE-5754
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver, mr-am
Affects Versions: 2.2.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: MAPREDUCE-5754.v01.patch, MAPREDUCE-5754.v02.patch
History does not store the runtime diagnostics information. JobHistoryParser
tries to blame a task. We propose to preserve the original runtime
diagnostics that covers all the cases including the job being killed. This is
particularly important in the context of user-supplied diagnostic message as
in MAPREDUCE-5648.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-5754) Preserve Job diagnostics in history
[
https://issues.apache.org/jira/browse/MAPREDUCE-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13898817#comment-13898817
]
Hadoop QA commented on MAPREDUCE-5754:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12628431/MAPREDUCE-5754.v01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1549 javac
compiler warnings (more than the trunk's current 1546 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs:
org.apache.hadoop.mapreduce.jobhistory.TestEvents
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4352//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4352//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4352//console
This message is automatically generated.
Preserve Job diagnostics in history
---
Key: MAPREDUCE-5754
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5754
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver, mr-am
Affects Versions: 2.2.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Attachments: MAPREDUCE-5754.v01.patch
History does not store the runtime diagnostics information. JobHistoryParser
tries to blame a task. We propose to preserve the original runtime
diagnostics that covers all the cases including the job being killed. This is
particularly important in the context of user-supplied diagnostic message as
in MAPREDUCE-5648.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
