[jira] [Created] (MAPREDUCE-4280) LocalJobRunner doesn't honor custom OutputCommiter cleanupJob
Subroto Sanyal created MAPREDUCE-4280: - Summary: LocalJobRunner doesn't honor custom OutputCommiter cleanupJob Key: MAPREDUCE-4280 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4280 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.1 Reporter: Subroto Sanyal I have an implementation SampleFileOutputCommiter which extends org.apache.hadoop.mapred.FileOutputCommitter . The implementation has specific code to be executed during cleanupJob() execution. When the framework(LocalJobRunner) makes a call to commitJob(), the framework never takes care of calling the cleanupJob() instead it calls org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitJob() which internally calls its own cleanupJob(). Though the method cleanupJob is deprecated but, still I feel the framework should take care of executing it as it is being executed from org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter. Currently the framework is not letting the Jobs written with MRV1 to run properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4281) LocalJobRunner doesn't handle Jobs using o.a.h.mapreduce.OutputCommitter
Subroto Sanyal created MAPREDUCE-4281: - Summary: LocalJobRunner doesn't handle Jobs using o.a.h.mapreduce.OutputCommitter Key: MAPREDUCE-4281 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4281 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 1.0.0 Reporter: Subroto Sanyal Backport MAPREDUCE-3563 for 1.0 version -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4282) Convert Forrest docs to APT
Eli Collins created MAPREDUCE-4282: -- Summary: Convert Forrest docs to APT Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0 Reporter: Eli Collins MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4283) Display tail of aggregated logs by default
Jason Lowe created MAPREDUCE-4283: - Summary: Display tail of aggregated logs by default Key: MAPREDUCE-4283 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mrv2 Affects Versions: 0.23.3, 2.0.1-alpha Reporter: Jason Lowe Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log. When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4284) Allow setting yarn.nodemanager.delete.debug-delay-sec on a per-job basis
Ahmed Radwan created MAPREDUCE-4284: --- Summary: Allow setting yarn.nodemanager.delete.debug-delay-sec on a per-job basis Key: MAPREDUCE-4284 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4284 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan The yarn.nodemanager.delete.debug-delay-sec property is helpful in debugging jobs (inspecting container logs/local dirs after the job finishes). Currently it is a nodemanager property and changing it requires restarting the nodemanager. In a production cluster this can be a real problem. It is better to have this property set on a per-job basis and not requiring the restart of nodemanagers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4284) Allow setting yarn.nodemanager.delete.debug-delay-sec on a per-job basis
[ https://issues.apache.org/jira/browse/MAPREDUCE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy resolved MAPREDUCE-4284. -- Resolution: Invalid Allow setting yarn.nodemanager.delete.debug-delay-sec on a per-job basis Key: MAPREDUCE-4284 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4284 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan The yarn.nodemanager.delete.debug-delay-sec property is helpful in debugging jobs (inspecting container logs/local dirs after the job finishes). Currently it is a nodemanager property and changing it requires restarting the nodemanager. In a production cluster this can be a real problem. It is better to have this property set on a per-job basis and not requiring the restart of nodemanagers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
