[jira] [Commented] (MAPREDUCE-6787) Allow job_conf.xml to be downloadable on the job overview page in JHS
[
https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534930#comment-15534930
]
Hadoop QA commented on MAPREDUCE-6787:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 2s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green}
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app
generated 0 new + 214 unchanged - 1 fixed = 214 total (was 215) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} hadoop-mapreduce-client-hs in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 50s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 7s
{color} | {color:green} hadoop-mapreduce-client-hs in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 0s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831039/mapreduce6787.002.patch
|
| JIRA Issue | MAPREDUCE-6787 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a86d70c8598a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 10be459 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6748/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
[jira] [Resolved] (MAPREDUCE-6783) Default setting 'inheritParentEnv=false' caused mr executor failed to initialize
[ https://issues.apache.org/jira/browse/MAPREDUCE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DENG FEI resolved MAPREDUCE-6783. - Resolution: Duplicate > Default setting 'inheritParentEnv=false' caused mr executor failed to > initialize > > > Key: MAPREDUCE-6783 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6783 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0-alpha1 >Reporter: DENG FEI > > MR default classpath depend on the "HADOOP_MAPRED_HOME" , defined at > MRJobConfig#DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH. > On 3.0.0.alpha1 , contain "Remove parent's env vars from child processes " > which commit by [#Robert Kanter] ,but it's seem not found in jira or release > note. DefaultContainerExecutor & DefaultLinuxContinerExecutor default the > 'inheritParentEnv' as false,there is no any approach to set the > "HADOOP_MAPRED_HOME" ,the AM failed by "Could not find or load main class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6787) Allow job_conf.xml to be downloadable on the job overview page in JHS
[ https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6787: -- Attachment: mapreduce6787.002.patch New patch uploaded to address the reported warnings > Allow job_conf.xml to be downloadable on the job overview page in JHS > - > > Key: MAPREDUCE-6787 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6787 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: mapreduce6787.001.patch, mapreduce6787.002.patch > > > The job overview page in JHS provides the path to the job.xml file, but it is > not a link that users can click on to download the job xml file directly from > JHS. We could provide a download link in JHS for better usability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6787) Allow job_conf.xml to be downloadable on the job overview page in JHS
[
https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534485#comment-15534485
]
Hadoop QA commented on MAPREDUCE-6787:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 50s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 49s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 27s
{color} | {color:red} hadoop-mapreduce-project/hadoop-mapreduce-client: The
patch generated 1 new + 64 unchanged - 0 fixed = 65 total (was 64) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 50s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green}
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app
generated 0 new + 214 unchanged - 1 fixed = 214 total (was 215) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} hadoop-mapreduce-client-hs in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 12s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 29s
{color} | {color:green} hadoop-mapreduce-client-hs in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 35s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
|
| | Found reliance on default encoding in
org.apache.hadoop.mapreduce.v2.app.webapp.dao.ConfInfo.deserialize(String):in
org.apache.hadoop.mapreduce.v2.app.webapp.dao.ConfInfo.deserialize(String):
String.getBytes() At ConfInfo.java:[line 80] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12830998/mapreduce6787.001.patch
|
| JIRA Issue | MAPREDUCE-6787 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs
[jira] [Updated] (MAPREDUCE-6787) Allow job_conf.xml to be downloadable on the job overview page in JHS
[ https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6787: -- Attachment: mapreduce6787.001.patch > Allow job_conf.xml to be downloadable on the job overview page in JHS > - > > Key: MAPREDUCE-6787 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6787 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: mapreduce6787.001.patch > > > The job overview page in JHS provides the path to the job.xml file, but it is > not a link that users can click on to download the job xml file directly from > JHS. We could provide a download link in JHS for better usability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6787) Allow job_conf.xml to be downloadable on the job overview page in JHS
[ https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6787: -- Status: Patch Available (was: Open) > Allow job_conf.xml to be downloadable on the job overview page in JHS > - > > Key: MAPREDUCE-6787 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6787 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: mapreduce6787.001.patch > > > The job overview page in JHS provides the path to the job.xml file, but it is > not a link that users can click on to download the job xml file directly from > JHS. We could provide a download link in JHS for better usability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6788) core-site settings do not propagate correctly when running an MR job.
David Tucker created MAPREDUCE-6788:
---
Summary: core-site settings do not propagate correctly when
running an MR job.
Key: MAPREDUCE-6788
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6788
Project: Hadoop Map/Reduce
Issue Type: Bug
Environment: HDP 2.3 + kerberos + Ambari 2.1.0
Reporter: David Tucker
Trying to run a sleep job with DEBUG logs on produces the following error:
15/12/02 17:27:35 DEBUG ssl.FileBasedKeyStoresFactory: CLIENT TrustStore:
/etc/security/clientKeys/all.jks
15/12/02 17:27:35 DEBUG impl.TimelineClientImpl: Cannot load customized ssl
related configuration. Fallback to system-generic settings.
java.io.FileNotFoundException: /etc/security/clientKeys/all.jks (No such file
or directory)
The workaround that fixed this for me was prepending "`hadoop classpath`:" to
mapreduce.application.classpath.
Here is a comparison of the 2:
mapred.application.classpath (before)
$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:
$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/common/*:
$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/yarn/*:
$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:
$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:
/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:
/etc/hadoop/conf/secure
`hadoop classpath` (prepended)
/usr/hdp/2.3.2.0-2950/hadoop/conf:
/usr/hdp/2.3.2.0-2950/hadoop/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop/.//*:
/usr/hdp/2.3.2.0-2950/hadoop-hdfs/./:
/usr/hdp/2.3.2.0-2950/hadoop-hdfs/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop-hdfs/.//*:
/usr/hdp/2.3.2.0-2950/hadoop-yarn/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop-yarn/.//*:
/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/.//*:
/usr/share/java/mysql-connector-java-5.1.17.jar:
/usr/share/java/mysql-connector-java.jar:
/usr/hdp/2.3.2.0-2950/tez/*:
/usr/hdp/2.3.2.0-2950/tez/lib/*:
/usr/hdp/2.3.2.0-2950/tez/conf
(Migrated from AMBARI-14205)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6771) RMContainerAllocator sends container diagnostics event after corresponding completion event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533237#comment-15533237 ] Haibo Chen commented on MAPREDUCE-6771: --- Thanks for your reviews Jason! > RMContainerAllocator sends container diagnostics event after corresponding > completion event > --- > > Key: MAPREDUCE-6771 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6771 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.7.3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: TaUnsuccessfullyEventEmission.jpg, > mapreduce6771.001.patch, mapreduce6771.002.patch, mapreduce6771.003.patch, > mapreduce6771.004.patch, mapreduce6771.branch-2.8.patch > > > Task containers can go over their resource limit, and killed by Node Manager. > Then MR AM gets notified of the container status and diagnostics information > through its heartbeat with RM. However, it is possible that the diagnostics > information never gets into .jhist file, so when the job completes, the > diagnostics information associated with the failed task attempts is empty. > This makes it hard for users to root cause job failures that are often caused > by memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6713) Distcp doesn't provide any option to override the default staging directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533190#comment-15533190 ] Sahil Takiar commented on MAPREDUCE-6713: - Hey [~kamrul], are you still working on this issue? Interested in finding a way to avoid using the .distcp.tmp file because if the target filesystem is S3, then using a .distcp.tmp file involves doing a rename from the tmp file to the final output name. Since renames on S3 require re-writing the file, this can cause a big performance hit for Distcp on S3. > Distcp doesn't provide any option to override the default staging directory > --- > > Key: MAPREDUCE-6713 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6713 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: distcp >Affects Versions: 2.5.1 >Reporter: Mohammad Kamrul Islam >Assignee: Mohammad Kamrul Islam > > *Current state and shortcoming* > === > By default, distcp writes temporary files into > $TARGET_PATH/.distcp.tmp/$taskatttempttid. (See > RetriableFileCopyCommand#getTmpFile). There is no way a user can override > this staging/tmp directory. The problem is obvious in S3 with versioning. For > example, user wants to turn on S3 versioning only for his target directory > but not the staging/tmp directory. Current distcp also creates versioning for > staging directory which can contain a lot of temporary files. If user can > override this path by a non-versioned S3 path for staging, it will make > things cleaner. > > *Proposed solution* > == > Provide a new option(-stage) where user can optionally provide a path from > target FS. Distcp mapper tasks will write distcp temporary files into that > directory. > *Possible Confusions* > = > There is another distcp option (-tmp) which can be assumed to serve the same > purpose. But this option works only with "-atomic" option which has a > different meaning of temporary files. > Another confusion could be the staging directory used by mapreduce framework. > The proposed temp directory is for distcp specific. > Working on a patch to upload. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6771) RMContainerAllocator sends container diagnostics event after corresponding completion event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6771: -- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.7.4 2.8.0 Status: Resolved (was: Patch Available) Thanks, [~haibochen]! I committed this to trunk, branch-2, branch-2.8, and branch-2.7. > RMContainerAllocator sends container diagnostics event after corresponding > completion event > --- > > Key: MAPREDUCE-6771 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6771 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.7.3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: TaUnsuccessfullyEventEmission.jpg, > mapreduce6771.001.patch, mapreduce6771.002.patch, mapreduce6771.003.patch, > mapreduce6771.004.patch, mapreduce6771.branch-2.8.patch > > > Task containers can go over their resource limit, and killed by Node Manager. > Then MR AM gets notified of the container status and diagnostics information > through its heartbeat with RM. However, it is possible that the diagnostics > information never gets into .jhist file, so when the job completes, the > diagnostics information associated with the failed task attempts is empty. > This makes it hard for users to root cause job failures that are often caused > by memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6771) RMContainerAllocator sends container diagnostics event after corresponding completion event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533118#comment-15533118 ] Hudson commented on MAPREDUCE-6771: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10513 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10513/]) MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event (jlowe: rev a1b8251bf7a7e9b776c4483fa01f7d453420eba4) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java > RMContainerAllocator sends container diagnostics event after corresponding > completion event > --- > > Key: MAPREDUCE-6771 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6771 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.7.3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: TaUnsuccessfullyEventEmission.jpg, > mapreduce6771.001.patch, mapreduce6771.002.patch, mapreduce6771.003.patch, > mapreduce6771.004.patch, mapreduce6771.branch-2.8.patch > > > Task containers can go over their resource limit, and killed by Node Manager. > Then MR AM gets notified of the container status and diagnostics information > through its heartbeat with RM. However, it is possible that the diagnostics > information never gets into .jhist file, so when the job completes, the > diagnostics information associated with the failed task attempts is empty. > This makes it hard for users to root cause job failures that are often caused > by memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6771) RMContainerAllocator sends container diagnostics event after corresponding completion event
[ https://issues.apache.org/jira/browse/MAPREDUCE-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533073#comment-15533073 ] Jason Lowe commented on MAPREDUCE-6771: --- +1 for the 2.8 patch as well. Committing this. > RMContainerAllocator sends container diagnostics event after corresponding > completion event > --- > > Key: MAPREDUCE-6771 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6771 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.7.3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: TaUnsuccessfullyEventEmission.jpg, > mapreduce6771.001.patch, mapreduce6771.002.patch, mapreduce6771.003.patch, > mapreduce6771.004.patch, mapreduce6771.branch-2.8.patch > > > Task containers can go over their resource limit, and killed by Node Manager. > Then MR AM gets notified of the container status and diagnostics information > through its heartbeat with RM. However, it is possible that the diagnostics > information never gets into .jhist file, so when the job completes, the > diagnostics information associated with the failed task attempts is empty. > This makes it hard for users to root cause job failures that are often caused > by memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6787) Allow job_conf.xml to be downloadable on the job overview page in JHS
[ https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6787: -- Summary: Allow job_conf.xml to be downloadable on the job overview page in JHS (was: Allow job_conf.xml file to be downloadable on the job overview page in JHS) > Allow job_conf.xml to be downloadable on the job overview page in JHS > - > > Key: MAPREDUCE-6787 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6787 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Reporter: Haibo Chen >Assignee: Haibo Chen > > The job overview page in JHS provides the path to the job.xml file, but it is > not a link that users can click on to download the job xml file directly from > JHS. We could provide a download link in JHS for better usability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6787) Allow job_conf.xml file to be downloadable on the job overview page in JHS
[ https://issues.apache.org/jira/browse/MAPREDUCE-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6787: -- Summary: Allow job_conf.xml file to be downloadable on the job overview page in JHS (was: Allow job_conf.xml file to be downloaded on the job overview page in JHS) > Allow job_conf.xml file to be downloadable on the job overview page in JHS > -- > > Key: MAPREDUCE-6787 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6787 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Reporter: Haibo Chen >Assignee: Haibo Chen > > The job overview page in JHS provides the path to the job.xml file, but it is > not a link that users can click on to download the job xml file directly from > JHS. We could provide a download link in JHS for better usability. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
